What to Watch for in a Solution Test Interview
Many interviewers try to keep the user on the happy path rather than watch where the user wants to go.
Instead of talking when a screen first comes up…start watching and waiting.
Silence and frustration are necessary for teams to see what they need to fix.
Your job as the interviewer is to let it happen and find out why.
Here are more tips on what to watch for and how to react in a variety of common situations:
If a user finds the right path and accomplishes their goal without help, this is usually great usability.
Too many interviewers waste time by asking the user to explain successful clicks. This interrupts the flow.
Just take a note that the user could easily navigate the concept and focus your follow-up on whether the user would actually use the concept.
…and isn’t thinking out loud.
Say…"Tell me why you selected that."
Sometimes interrupting flow is necessary to know what exactly is going on in the user's mind.
As you interview more users, you will develop a sense of when to interrupt and when to keep silent.
…let them go on and see where it leads them…don’t interrupt unless they get stuck.
Maybe they are showing you a new idea to solve their pain point.
Keep an open mind.
When they have stopped their progress through the solution, you can ask about why they chose that unusual path.
If they get off track, then just remind them to focus on the goal mentioned at the start of the interview.
WAIT…Let it be awkward for 10 seconds.
Then say…”Go ahead and think out loud.”
Still stuck…”What would you do next?”
Still quiet…Remind the user to accomplish the goal described at the beginning of the test.
Still nothing...(time to move on)…Tell them where to click so you can get to next screen.
(I hate doing this but when the user doesn't get it, move on...explaining yourself is not worth the time...fix the problem for the next test...remember to choose velocity over perfection during Product Discovery)
Say…”It’s okay. We haven’t built that part yet” (don't say anything more)
If you want more info say…”What did you expect to happen?”
If you're still interested…”What would you want to do on that next screen if it existed?”
Avoid pushing the user through your solution when they might not actually want to use your solution.
One of the questions I ask in interviews to keep users moving is: “What would you do next?”
If you sense that the user isn’t interested in the solution then you might add: “Perhaps you wouldn’t do anything in this situation?”
Scientists start their research with the Null Hypothesis. This is where they assume their solution will NOT work and gather evidence to prove that it does work.
Many Product teams do the opposite and assume that their solution works from the moment they thought it up.
Make sure to collect and analyze the evidence before making a decision.
Jim is a coach for Product Management leaders and teams in early stage startups, tech companies and Fortune 100 corporations.
Previously, Jim was an engineer, product manager and leader at startups where he developed raw ideas into successful products several times. He co-founded PowerReviews which grew to 1,200+ clients and sold for $168 million. He product-managed and architected one of the first ecommerce systems at Fogdog.com which had a $450 million IPO.
Jim is based in San Francisco and helps clients engage their customers to test and validate ideas in ecommerce, machine learning, reporting/analysis, API development, computer vision, online payments, digital health, marketplaces, and more.
He graduated from Stanford University with a BS in Computer Science.
