An interview with Jim Morris. Jim's a product discovery & experimentation coach who wants teams to stop wasting their time with discovery if they're not going to do anything with it. He's currently running Product Discovery Group out in Silicon Valley.
Listen here: https://www.oneknightinproduct.com/jim-morris/
We talk about a lot, including:
The goals of Product Discovery Group, the problems he helps to solve, how he got started as a product discovery coach and that time he hung out with Jeff Bezos
How many companies see funding as the ultimate validation of their idea but forget to talk to their customers and check if the idea is actually viable for the business
Why we need to remember that product discovery is not just there as an artificial stage gate to delay decision making and should always serve the overall business goals
How there are bad product companies with good product managers and good product companies with bad product managers, and how Silicon Valley startups are in the same boat as the rest of us when it comes to good product discovery practices
How we can bed product discovery in with leadership, how to persuade them that there's a different way to lead, and how to skill up product teams that have never done product discovery before
The concept of a Solution Test, the importance of presenting multiple solutions, why you have to get interactive rather than just show stuff, and why you should never concentrate on usability first
How to apply structure to your discovery data collection to make it easier to extract insights from the data and turn them into action
Episode Transcript
Jason Knight
Hello and welcome to the show. I'm your host, Jason Knight, and on each episode of this podcast, I'll be having inspiring conversations with passionate product people. Now, when it comes to inspiration, sometimes.
Jason Knight
We all need a little bit of help.
Jason Knight
If you follow me on Twitter, you'll know I'm a passionate advocate for mentorship. The first quarter of this year, I've mentored 76 different people, but I've realized.
Jason Knight
I just don't scale.
Jason Knight
Because of this, I have teamed up with a buddy to help more mentors and mentees find each other. So if you want to find out more about that, check out onenightinproduct.com mentor where you can sign up to be.
Jason Knight
A mentor, mentee, or both.
Jason Knight
That's onenightinproduct.com mentor.
Jason Knight
Or check the show notes for details.
Jason Knight
On tonight's episode, we talk about discovery and experimentation. No, not creating our own product management Frankenstein's monster, but making the right moves and doing a solution test to understand.
Jason Knight
Whether people actually want to use our products.
Jason Knight
We talk about why some companies struggle with the very concept of discovery, how we might win, skeptics around how to run good experiments, and how to analyze the data you get back and turn it into action. We also ponder whether all these cool Silicon Valley startups we hear about are any good at this sort of stuff, or if they're just stuck in the same boat as the rest of us. For this and much more, please join.
Jason Knight
Us on One Night in Product.
Jason Knight
So my guest tonight is Jim Morris. Jim's a data driven, sports obsessed product.
Jason Knight
Discovery coach who says he's been in over 600 user interviews in his time.
Jason Knight
And I'm hoping he didn't come away from those with 600 different feature requests on his backlog.
Jason Knight
Jim says he's a convicted monopolist, which.
Jason Knight
I definitely have questions about, as well.
Jason Knight
As a former restaurant host and King.
Jason Knight
Gardner, who says he hates his cat.
Jason Knight
Staying out past dinner time when he's.
Jason Knight
Not anxiously looking at the cat flap.
Jason Knight
He's busy getting teams to work together and helping product managers be successful, which.
Jason Knight
He'S doing with his own consultancy, product Discovery Group. Hi, Jim. How are you tonight?
Jim Morris
Doing great, Jason. Thanks for having me.
Jason Knight
No problem.
Jason Knight
It's good to have you here. I'm looking forward to what we're going to discover tonight.
Jason Knight
But before any of that, I do.
Jason Knight
Have to ask, you described yourself as a convicted monopolist. So does that literally mean you were.
Jason Knight
Playing Monopoly, got the go to jail card?
Jason Knight
Or do you mean something a little.
Jason Knight
Bit more interesting than that?
Jim Morris
No jail, but my company was number two in the market.
Jim Morris
The number one company in the market bought us.
Jim Morris
The US federal government decided that the.
Jim Morris
Combined company was a monopoly monopolistic player in the market, and they took the combined company trial and they convicted the combined company mostly on the emails of the acquiring company.
Jim Morris
Really don't want to write down if.
Jim Morris
We do this acquisition, it will reduce.
Jim Morris
Barriers to entry. It'll increase barriers to entry and reduce.
Jim Morris
Pricing pressure on our sales deals.
Jason Knight
This is the sort of thing that gets brought up on that tech email site that you see all the old Apple exposes and stuff. So you're up there with all these.
Jason Knight
Other malfeasance that's being caused by these other companies.
Jason Knight
But did you get in a lot.
Jason Knight
Of trouble for that or slap on the wrist or how did they treat.
Jim Morris
You for us as employees?
Jim Morris
I mean, I was a founder of the acquired company. It doesn't necessarily have any individual impact. We did convert into shares and money into the acquiring company.
Jim Morris
It just meant that we had to split out.
Jason Knight
Oh, there you go.
Jim Morris
Yeah, as a product leader was one.
Jim Morris
Day I was the leader of a large group, and the next day I.
Jim Morris
Was in a very small spin out company, rebuilding everything from scratch because the.
Jim Morris
Acquiring company had let go most of.
Jim Morris
The employees of my company into intervening two years.
Jason Knight
Wow. There you go.
Jason Knight
What a story.
Jason Knight
That's a limited Netflix series right there.
Jason Knight
So let's get back down to business.
Jason Knight
End and pass Go and collect our $200 or whatever it is these days.
Jason Knight
So you're the founder of Product Discovery.
Jason Knight
Group out there in Silicon Valley, and.
Jason Knight
I can probably take a guess at.
Jason Knight
What you do with Product Discovery Group.
Jason Knight
But just in your own words, specifically.
Jason Knight
What problem are you solving with Product Discovery Group?
Jim Morris
Sure.
Jim Morris
So I coach product leaders to create and grow successful product organizations, mostly focused.
Jim Morris
On connecting with their customers and connecting with their data.
Jim Morris
And then as part of that, I.
Jim Morris
Work also with cross functional teams as.
Jim Morris
A group to teach them product discovery.
Jim Morris
Techniques that are setting outcome goals as.
Jim Morris
Well as recruiting and interviewing users on.
Jim Morris
A regular basis and then kind of.
Jim Morris
Putting that together to create solutions and.
Jim Morris
Test them so that they don't just make stuff up and give it to engineers.
Jason Knight
Never happens.
Jason Knight
Right, but who's hiring you to come.
Jason Knight
Into these companies then? Are you being hired by the product teams?
Jason Knight
Are you being hired by the leaders.
Jason Knight
The exec suite of the business? Are you kind of being hired by individuals to kind of come in via the backdoor? Like, how do you actually get in.
Jason Knight
There in the first place?
Jim Morris
Yeah, it's usually heads of product or CEOs. So if there's no head of product.
Jim Morris
It'S often a CEO.
Jim Morris
That's usually in the smaller companies, sometimes they're passing off product to their employees for the first time.
Jason Knight
Right.
Jim Morris
And so helping manage that transition.
Jim Morris
But yeah, it's usually those groups now.
Jason Knight
That'S been going for a while. I think that's been going for six.
Jason Knight
Or so years now.
Jason Knight
So you've been at it for a.
Jason Knight
Bit, but before that, you've had an illustrious career across tech product, even spent.
Jason Knight
A bit of time as a CTO from time to time as well.
Jason Knight
So you've obviously got that engineering and tech background as well.
Jason Knight
But you've now decided to double down on product, product discovery, product teams, as we just said.
Jason Knight
So I guess the question there is what was it that made you, of.
Jason Knight
All of those things that you could have chosen, decide that it was product discovery and coaching of product discovery that.
Jason Knight
You really wanted to focus on?
Jim Morris
I was lucky enough to be an.
Jim Morris
Engineer in the beginning of what we.
Jim Morris
Know is the internet and actually met Jeff Bezos.
Jim Morris
He sat on our couch in our engineering area one time.
Jason Knight
Did he bring two pieces with him?
Jim Morris
Yeah, I'm not sure.
Jim Morris
He wasn't really that famous back then. Amazon was only selling books.
Jim Morris
My company was selling sporting goods. So we were considering a peanut butter meets chocolate arrangement, I assume. I was not an executive in this.
Jim Morris
Company but what I learned as an.
Jim Morris
Engineer and leading engineering groups and product.
Jim Morris
Because before there were product managers, it was people like us who were leading.
Jim Morris
Product is that we now can build anything. The tools to build them are so much easier than before. I can rent servers, I can often.
Jim Morris
Find resources all around the world to help me become a team and build something.
Jim Morris
But the ideas that we use in this process, the ideas that we build.
Jim Morris
Are just as naive and raw as they were 25 years ago. And this is the sad state of.
Jim Morris
The industry, is that we are continuing to build often what comes to mind.
Jim Morris
Without really checking with, reflecting these ideas.
Jim Morris
Off of the world or checking if they're successful even after we launch them.
Jason Knight
Yeah, again, sure that never happens but.
Jason Knight
Sounds like the sort of thing that.
Jason Knight
Would be terrifying if it did.
Jason Knight
But what sort of companies are you working with mainly then? Like you're out in Silicon Valley, so.
Jason Knight
Are you mainly doing a Silicon Valley.
Jason Knight
Tour and working with Silicon Valley startups or scale ups or are you kind of working internationally and working with a bunch of different companies around the world?
Jason Knight
Like who's your customer base or your ideal customer base?
Jim Morris
Yeah, on the product leader coaching, it.
Jim Morris
Varies and it's quite geographically diverse.
Jim Morris
I have folks in England or Scotland.
Jim Morris
All the way east coast and west coast.
Jim Morris
Those hours tend to work nicely all.
Jim Morris
Around the clock in terms of the.
Jim Morris
Teams they tend to be in. I mean they vary from startups to corporations. Startups don't have execution problems. They typically have this two piece of team but they often have an idea problem that they take funding of the startup as validation when really it should be validated by the market.
Jim Morris
Corporations often have both problems where it.
Jim Morris
Can be hard to execute and can.
Jim Morris
It be hard for product teams to.
Jim Morris
Actually have a voice in.
Jim Morris
Corporations typically have very strong sales, marketing and executives.
Jim Morris
And so product is growing and is.
Jim Morris
New and they don't quite know where.
Jim Morris
To put it and how it forms a part of the successful part of the company.
Jim Morris
And so often what I'm doing in these companies is helping them, helping product.
Jim Morris
Make its place and then of course earn its place. These product people should always be earning our keep with checking with customers, following.
Jim Morris
Our data, creating great deliverables for engineering that are based on reality, not just our opinions.
Jason Knight
But do you find and based on.
Jason Knight
What you just described, obviously you're working.
Jason Knight
With small and large companies. Do you find I mean I've worked for large companies before and I can.
Jason Knight
Definitely speak to some of those execution.
Jason Knight
Problems and the kind of top down.
Jason Knight
Management and of course all the different.
Jason Knight
Ways that big companies can suck at doing that kind of stuff.
Jason Knight
Do you find that's something that you can make a really big impact on like when you go in there or.
Jason Knight
Is it just really kind of a.
Jason Knight
Case of almost incremental change and just trying to make them a little bit better to try and push them a little bit down a better path? But the there's only certain amounts that you can do in the kind of.
Jason Knight
Time that you get with these people.
Jim Morris
It depends on the commitment of the.
Jim Morris
Leaders of the company.
Jim Morris
So when I work with cross functional.
Jim Morris
Teams, they soak up the techniques and the experience about working with customers and pulling data and using data in conversations to convince people whether it's qualitative or quantitative. Leaders often have a harder time giving.
Jim Morris
Up their micro control of decision making.
Jim Morris
And so as we talk about actually using data to make decisions, most people are trying to put in objective and key results frameworks. We have data as goals and so that's really the if the company is.
Jim Morris
Committed from the top to change not.
Jim Morris
Just discovery is not something that is an isolated technique.
Jim Morris
I could teach you to interview users.
Jim Morris
But if you're not learning from interviewing users and you're not changing your mind from interviewing users, you should not be doing it. But when you change your mind that.
Jim Morris
Means that a roadmap item might change.
Jim Morris
But that roadmap item might feel sacred.
Jim Morris
In your company and so it starts.
Jim Morris
At the top by not making sacred roadmaps.
Jason Knight
Yeah, and almost combating the cognitive biases as well. Like the sunk cost fallacies and the fact that people have gone out there.
Jason Knight
And maybe either fallen in love with an idea or they've maybe even worse.
Jason Knight
Still made a commitment either to the board that they're going to do this new thing or they're going to go.
Jason Knight
Out and make a commitment to a.
Jason Knight
Client that they're going to sell this new thing or that they're going to do this new thing so that they can get the contract signed and stuff like that.
Jason Knight
I mean all things that again I'm sure never happen but if they did would be absolutely terrifying. But yeah, so you're a big advocate.
Jason Knight
For experimentation and product discovery. You've talked about it just now and.
Jason Knight
Obviously that's the whole point of the product discovery group as well. But I guess these days it's not exactly controversial.
Jason Knight
Certainly in product circles, it's very much.
Jason Knight
In fashion, both through the efforts of coaches like yourself, other coaches that we're.
Jason Knight
All aware of, as well as some.
Jason Knight
Of the popular books that are out there. It's out there and people are talking.
Jason Knight
About this a lot and obviously in many cases they've been talking about it.
Jason Knight
For some time as well, but it definitely seems to be cresting a wave at the moment.
Jason Knight
But you've touched on it yourself as well.
Jason Knight
Not everyone's doing it enough, some people aren't doing it at all.
Jason Knight
Across all of the types of companies that you work for, do you find that even if they're not doing it.
Jason Knight
And maybe don't really have a great.
Jason Knight
Idea about how they're going to do it, that there's at least an appetite to go out and do it? Or do you think that there are.
Jason Knight
Some companies out there that almost on.
Jason Knight
A point of principle, don't do it because they think that they know everything.
Jason Knight
And they don't need that sort of thing at all?
Jim Morris
There are definitely companies who believe they don't need that sort of thing at all.
Jim Morris
And these might be the companies that.
Jim Morris
Adopt a lot of these fake Agile frameworks.
Jason Knight
Hey, here we go, let's do Safe.
Jim Morris
I know it's a rabbit hole of.
Jim Morris
Discussion, but what it does is it takes something like Agile, which is an.
Jim Morris
Inherently messy, incremental process. And with this inherently messy and incremental process, it doesn't lend itself well to.
Jim Morris
Leaders who want to predict the future.
Jim Morris
And who want some control over the future. When I say control over the future, I think this is impossible. But people don't believe that and so.
Jim Morris
They want this fake control over the.
Jim Morris
Future and so they take Agile and.
Jim Morris
They put it in a framework and.
Jim Morris
It no longer becomes Agile, but you.
Jim Morris
Still call it Agile.
Jim Morris
So it's a little bit of gaslighting, especially for the employees who thought they.
Jim Morris
Were going to be able to do.
Jim Morris
Incremental improvement and come up with innovations at their level. But in reality, all the resources are there's.
Jim Morris
Stage gate decision making by heads of.
Jim Morris
Engineering who won't let you control the backlog of the engineers because they won't.
Jim Morris
Spend the time, or don't spend the.
Jim Morris
Time with their colleagues to agree on.
Jim Morris
Important metrics for the company, which is the way you delegate without freaking out.
Jim Morris
The way you macromanage rather than micromanage.
Jason Knight
Yeah, for sure.
Jason Knight
And obviously that touches on another potential rabbit hole there of the whole horror of shadow roadmaps and engineering roadmaps and.
Jason Knight
Everything being kind of hidden.
Jason Knight
I was once called a transparency Nazi for suggesting that might be a bit of a bad idea to do, but.
Jim Morris
You know, some people don't like work in progress.
Jim Morris
Some people expect you to be polished.
Jim Morris
When you walk into any meeting and.
Jim Morris
They feel like it's a waste of.
Jim Morris
Their time to discuss the intricacies of.
Jim Morris
Whether you should or shouldn't do something.
Jim Morris
And I'm not a big fan of.
Jim Morris
Wasting time, but I am a fan.
Jim Morris
Of if we're going to talk about.
Jim Morris
Whether we should do something, ideally it's based in some form of reality.
Jim Morris
And as we're doing it, why don't.
Jim Morris
I show you so that you have.
Jim Morris
Some influence in the beginning as opposed.
Jim Morris
To surprising me at the end.
Jim Morris
This is like why we talk to users. We can do discovery with our executives and with our users.
Jim Morris
And that's one of Teresa Torres's big.
Jim Morris
Things, is managing stakeholders.
Jim Morris
Just talk to them more often.
Jason Knight
Oh, yeah, for sure. I mean, I kind of started to.
Jason Knight
Label that and I don't know if it was me who labeled it that or someone else who labeled it that and I just copied them, which is.
Jason Knight
Almost certainly what really happened.
Jason Knight
The kind of idea of full stack.
Jason Knight
Discovery and just not like you get.
Jason Knight
Some people to sit there and they're so precious about the fact that they have to talk to users.
Jason Knight
And that's really the only type of discovery that's valid. And if it's not coming through their product lens, then it's completely invalid and.
Jason Knight
Just point in time, just one deal or whatever like that. And it's like, yeah, sure, it kind of is that. But of course, all conversations basically contribute to a bigger picture. And I think it's really incumbent on.
Jason Knight
All product teams to listen to everyone, right?
Jason Knight
They can wait the evidence accordingly. They can wait the discussions accordingly, but.
Jason Knight
They can't just not listen to them.
Jim Morris
Well, product folks, often discovery gets a.
Jim Morris
Bad name because it takes on a.
Jim Morris
Life of its own.
Jim Morris
Product discovery is only relevant if it.
Jim Morris
Solves a business problem. We mean to solve customer problems to the benefit of the business. And so what's the benefit of the business?
Jim Morris
Executives will say it's a stage gate decision making. And we'll say the executives will say.
Jim Morris
I'm going to make money and more revenue or profit by doing the math.
Jim Morris
In my head and assigning features to.
Jim Morris
A product team, when in reality, they.
Jim Morris
Should take their top level metrics, break them down one or two levels and start to dole those out to teams instead of these features. And then if teams adopt this business attitude, their stakeholders will respect them more.
Jim Morris
Because they're speaking two languages.
Jim Morris
Executives are terrified of not getting your paycheck in the bank by raising investment.
Jim Morris
Or hitting revenue numbers.
Jim Morris
And you get the luxury of getting.
Jim Morris
A paycheck all the time, but it does owe it to the business to be related to your work has to.
Jim Morris
Be tied to the business.
Jim Morris
And I think that's where not only.
Jim Morris
Are we're not selling stakeholders, we're doing discovery, but we're also trying to relate.
Jim Morris
And make relevant our work.
Jason Knight
Yeah, absolutely. I think it's important we want those.
Jason Knight
People to come towards us, of course, and start to think a little bit more like us and accept the ways that we think. But I think to do that, the.
Jason Knight
Bargain has to be that we try.
Jason Knight
And think a little bit how they.
Jason Knight
Think as well and not necessarily in the middle, but not at one end or the other.
Jason Knight
But I'd like to think that Silicon Valley tech startups have all drunk the koolaid in this regard and all out there doing product stuff properly. Like that's the basis and you're shaking your head already. So I'm sure this isn't the case.
Jason Knight
But a lot of the classic books are based on the practices of people.
Jason Knight
That have worked for some of the.
Jason Knight
Biggest Silicon Valley tech companies in the world.
Jim Morris
Right?
Jason Knight
So is it the case that those really good buy the book product discovery or product practice companies are even in.
Jason Knight
A minority in Silicon Valley or is.
Jason Knight
There like just a strong minority of not that type of company in Silicon Valley as well?
Jim Morris
Well, I'll first say that the sad.
Jim Morris
Truth in product is that there are successful teams that have bad product managers.
Jim Morris
And no product discipline at all and.
Jim Morris
There are unsuccessful teams that have highly.
Jim Morris
Disciplined, very good product people on them.
Jim Morris
It sometimes relates to whether you're in.
Jim Morris
A rocket ship business or not, were you there at the right place at the right time?
Jim Morris
And what we do is to increase the probability of success and to turn.
Jim Morris
Ordinary employees into extraordinary employees through doing these techniques.
Jim Morris
So the issue with Silicon Valley is.
Jim Morris
That you get a lot of external validation when you receive funding, when you have free lunch, when you have a great environment.
Jim Morris
And really you can get isolated from the fact that there are customers out there who need to pay for your services so that your company is viable.
Jim Morris
In the long term.
Jim Morris
So I don't see it as a widespread concept in Silicon Valley that we're operating with a data first, qualitative or quantitative data first approach.
Jim Morris
Sometimes we are just operating fast, hoping.
Jim Morris
To see what sticks.
Jim Morris
And so there's a lot of great execution in Silicon Valley that's what I.
Jim Morris
Would say is probably the number one.
Jim Morris
Asset is well, execution and wealthy people.
Jim Morris
Who give money to people they don't.
Jim Morris
Know in risky areas in the rest.
Jim Morris
Of the world, wealthy people give money.
Jim Morris
To people they know in non risky areas.
Jason Knight
Yeah, that makes a lot of sense.
Jason Knight
But it's also slightly disheartening that even in the heartland of tech, people are still struggling to do this right. So what hope do the rest of us have?
Jason Knight
But speaking about that hope, I mean, we've talked about there are some companies.
Jason Knight
That don't like to do this stuff.
Jason Knight
Either on purpose or they just don't know how to do it. Yet, or they think that it's a bit of a bind or it takes.
Jason Knight
Too long, or all of those things that we just mentioned. But you're obviously going into these companies.
Jason Knight
And I guess at least in some.
Jason Knight
Situations, persuading them that that is the way to go.
Jason Knight
Now, you could argue, and maybe should argue, that of course, if people are.
Jason Knight
Coming to you, that there's almost a.
Jason Knight
Natural appetite for that anyway, because someone.
Jason Knight
That didn't want to do this probably.
Jason Knight
Isn'T going to get you.
Jason Knight
In in the first place, but presuming.
Jason Knight
That you've gone into at least some.
Jason Knight
Skeptical companies or ones where the product team were maybe up for this, but.
Jason Knight
The leadership team weren't, and you needed to help them bed that in and.
Jason Knight
Actually get the buy in to do that. What are some of the approaches that.
Jason Knight
Have worked best for you to kind.
Jason Knight
Of crest that hill and go down the other side into the wonderful meadows of discovery?
Jim Morris
I get them to testing with users.
Jim Morris
Within 30 days, and sometimes that requires.
Jim Morris
Me to write the recruiting screener for the users. I don't like to do work for.
Jim Morris
My clients, but I have longer engagements.
Jim Morris
And so what that allows me to.
Jim Morris
Do is to have them do some.
Jim Morris
Things, have me do some things and get to this point of talking to users. And once they talk to five or.
Jim Morris
Six users, they start to understand the approach, which is, I'm not going to.
Jim Morris
Show one solution, I'm showing multiple solutions.
Jim Morris
Which is a required part of my approach.
Jim Morris
And then I'm going to talk to.
Jim Morris
Users in a way that allows them.
Jim Morris
To react to things so that I.
Jim Morris
Get more honest feedback. And when they see the honest feedback.
Jim Morris
Engineers who say, what can we learn from five users?
Jim Morris
Start to understand what they can learn.
Jim Morris
From five users, can't learn everything.
Jim Morris
But Jacob Nielsen's quote is zero interviews gives zero insight.
Jim Morris
Yeah.
Jim Morris
So I do that and then, of course, in the rest of my engagement.
Jim Morris
It becomes everything they do, they start.
Jim Morris
To do on their own, so that.
Jim Morris
By the time I leave, they're not.
Jim Morris
Only hooked on it because they've talked.
Jim Morris
To users, because that's one of the hooking points. They're actually able to do all the various steps.
Jim Morris
The second thing is I convince management.
Jim Morris
That as you grow, it's going to.
Jim Morris
Be hard to tell everybody what to.
Jim Morris
Do and that you're actually better off spending the ten to 20 hours in your roadmap discussions.
Jim Morris
What if we took half of that.
Jim Morris
And talked about metrics?
Jim Morris
Right. So the idea that there's a different way to manage that's, the hardest part.
Jim Morris
Is we teach discovery, but we don't teach the different way to manage.
Jim Morris
So the managers, even though we criticize.
Jim Morris
Them for not allowing discovery, we don't actually work with them to understand that, to teach them there's a different way.
Jim Morris
To manage that can work to their advantage.
Jim Morris
If you as a manager can convey.
Jim Morris
Business reality to your employees, you will.
Jim Morris
Be able to scale and become a great executive.
Jim Morris
But if you as a manager still.
Jim Morris
Want to be a PM working through.
Jim Morris
Other people, it won't scale.
Jim Morris
We we are not Steve Jobs. We are not Elon Musk.
Jim Morris
These are not the examples to follow.
Jason Knight
Yeah, well, there's a number of reasons why we shouldn't follow Elon Musk, for example, although obviously we'd own Twitter by.
Jason Knight
Now if we were, I guess, and did it properly.
Jason Knight
Okay, so that sounds fair enough.
Jason Knight
And obviously trying to make that more.
Jason Knight
Cultural change, I guess, sometimes works and sometimes it doesn't. It really depends on the company and the buy in you can get, as.
Jason Knight
You say, but assuming that you do.
Jason Knight
Get that buy in and that there is scope then to go out and.
Jason Knight
Do some proper discovery or get your.
Jason Knight
Five users or whatever it is that.
Jason Knight
You want to achieve.
Jason Knight
If you're working for a company that's.
Jason Knight
Never done it before or working with.
Jason Knight
A team that's never done it before, it's quite possible that these like, if.
Jason Knight
We'Re talking to people or talking about.
Jason Knight
People that have maybe been in a.
Jason Knight
More of a feature, factory top down, delivery type product management role, sort of project manager, really. Not really a product manager. If they don't have the skills, they're.
Jason Knight
Not just going to be able to go and ask good questions of the.
Jason Knight
People that they want to do discovery.
Jason Knight
Interviews with, because they'll probably go in.
Jason Knight
There and reaffirm all their biases and lead the respondents and do all of.
Jason Knight
That stuff that we're told not to.
Jason Knight
Do when we talk about discovery. So I guess the question that comes out of that is what techniques or.
Jason Knight
Methods do you go into these people with then?
Jason Knight
To try and get them up to.
Jason Knight
Speed as quickly as possible, to make sure that when they do go out.
Jason Knight
And talk to these customers, especially if.
Jason Knight
It'S the first time and you're trying to validate the approach that they're actually getting good results out of those interviews.
Jason Knight
And that they're asking the right questions and getting something useful out the back of it.
Jim Morris
Yeah, I've concentrated on one product discovery.
Jim Morris
Technique, which I call the solution test.
Jim Morris
And that's before you do usability, because.
Jim Morris
You don't quite know if it's the solution you want.
Jim Morris
Usability is just as it confuse people.
Jim Morris
Not do they want to use it. My teams don't ask the hard question.
Jim Morris
Often, does anybody want this? And so I go straight for that question. You can read the Sprint book.
Jim Morris
It's not in there.
Jim Morris
So do they want it?
Jim Morris
And as we approach that existential question.
Jim Morris
We'Re also looking at the interview as.
Jim Morris
A way to well, I'll say that for the solution test, I have people.
Jim Morris
Create variety of solutions, usually fake prototypes, as a way to get reactions from users. When I ask you, hey, would you.
Jim Morris
Use a concept and I describe it to you verbally, maybe I show you.
Jim Morris
A screenshot of it.
Jim Morris
You're not really participating in that concept. You're like in a movie theater watching it.
Jim Morris
The distinction is quite real. When I send you the link and have you interact with it and share.
Jim Morris
Me your screen, as I get users to interact with things, the reactions become more honest.
Jim Morris
And so the solution test for me.
Jim Morris
Hits a very sweet spot in terms.
Jim Morris
Of authenticity that is in some ways.
Jim Morris
Hard to mess up.
Jim Morris
Now, you can sort of lead people.
Jim Morris
Through, but I do the first couple of interviews, right? So I give them theory, I model.
Jim Morris
It, I watch them interview, and then.
Jim Morris
I give them advice.
Jim Morris
It's typical didactic technique. And I get QA people, engineers, I.
Jim Morris
Get designers used to tell me they.
Jim Morris
Were terrified of talking to users. And it's sort of a script.
Jim Morris
Although once we get to the prototype.
Jim Morris
Itself, there's not much to say except set up the mindset, give them the reason they're there.
Jim Morris
You need to buy an iPhone accessory. You need to pick a primary care.
Jim Morris
Physician and then just stop talking.
Jim Morris
And so if we get to awkward silence, then I know we're getting somewhere. And when they get users struggling and.
Jim Morris
They learn not to talk during that.
Jim Morris
Struggle, they realize that making good software is hard and that they should really step back and just have a humility type moment. So that's the in terms of avoiding leading questions, there's a structure I give them.
Jim Morris
The nice thing about a solution test.
Jim Morris
Interview is it also helps validate is it the problem? Is the problem correct? Is the opportunity interesting? Do you have the right users that you've recruited?
Jim Morris
Are your personas good or bad?
Jim Morris
Is your jobs to be done actually good or bad?
Jim Morris
And we put all this stuff together in the Sprint book. They do it in a week. I usually do it in a couple.
Jim Morris
Of weeks because I'm not worth them 24/7.
Jim Morris
But once they create this prototype, and.
Jim Morris
I call them experiments, the experiment is meant to learn something, not meant to get them to say yes. And so if they can get that.
Jim Morris
Approach, I turn everybody into interviewers. I don't bother trying to make non designers make prototypes.
Jim Morris
We do.
Jim Morris
If it's a Sprint week, then we.
Jim Morris
Have to do that. But otherwise, most teams it's just the designer. But we will get into that whole.
Jim Morris
Flow and then pass it to the designer.
Jim Morris
Whereas most people will skip making that.
Jim Morris
Experience, and they don't actually learn how to make a good experience.
Jim Morris
They'll just sort of make a bunch.
Jim Morris
Of bullet points and hand it over.
Jason Knight
Yeah, that makes a lot of sense. I was going to talk a bit about experimentation as well, because I know that you've got a fiery passion for it. And we've obviously already touched on some.
Jason Knight
Of the elements of it. So I guess if we then assume.
Jason Knight
That some of these prototypes or mockups.
Jason Knight
Or whatever we want to call them.
Jason Knight
Are basically mini experiments that you're going.
Jason Knight
To run with users to test the.
Jason Knight
Desirability, usability, feasibility, all of the stuff.
Jason Knight
That we might want to test with these people.
Jason Knight
What are some of the key things.
Jason Knight
To look out for?
Jason Knight
I was going to ask about a good experiment, but let's flip that and say, what are some of the key.
Jason Knight
Things not to do when you're putting.
Jason Knight
One of these experiments in front of people? If you want to get good results.
Jim Morris
Of it, one, don't build it in software, right?
Jim Morris
So people, when they hear about experimentation and discovery, they think immediately to a.
Jim Morris
B testing or multivariate testing and say, no, I've done a lot of that. That requires you to build two solutions or five or six in order to get one winner. And if you release that to the.
Jim Morris
Public, those two to five solutions have.
Jim Morris
To be privacy compliant, secure, disaster recovery compliant. It's just a lot of engineering time that doesn't go into the value to the user. Once you know the value, sure, invest in that stuff.
Jim Morris
So don't build it as much as possible.
Jim Morris
Most of my teams are making fake prototypes.
Jim Morris
Whether it's a report, API, documentation, clickable prototypes.
Jim Morris
That's one thing is don't build it.
Jim Morris
The other is don't just show it.
Jim Morris
There has to be an experiential element we're not holding in our hands, but.
Jim Morris
We'Re looking at it at our own screen.
Jim Morris
We can interact with it.
Jim Morris
The fake report might be a bunch.
Jim Morris
Of Excel data that works in an Excel graph.
Jim Morris
It doesn't have to be a web.
Jim Morris
UI that's fully functional. Don't make one long solution. This just happened with the team. I said, we're going to make these experiments.
Jim Morris
We're going to do multiple solutions.
Jim Morris
Everyone did sketching.
Jim Morris
We said to the designer, like, here's the sketches.
Jim Morris
Let's make some prototypes. And the designer sort of fused them into one long experience, which is what people do because they're always asked not to make works in progress, but to.
Jim Morris
Make a final experience. If they were to go to a.
Jim Morris
Design review and say, here's my three ideas, that would be frustrating for leaders.
Jim Morris
Who want to give a yes no decision.
Jim Morris
So the ideas from the team had been incorporated, so the designer was listening, but as an artifact to show you. You would click through it, you would experience them, and you'd be like, yeah, okay. That's the response that users have.
Jim Morris
That's interesting. I like it. I'd use that.
Jim Morris
And those are all lies. So what we do is, I want you to slice it up, take one.
Jim Morris
Piece of the experience.
Jim Morris
If it's ecommerce, don't do search and.
Jim Morris
Navigate, filter, pick something, read the reviews.
Jim Morris
Add to cart billing, shipping address, credit card checkout.
Jim Morris
Give me add to cart. Give me three variations of add to cart.
Jim Morris
What that means is I'm getting the.
Jim Morris
Variation from my designer. If I've got a subject matter expert.
Jim Morris
Who doesn't build experiences but has a.
Jim Morris
Great idea for add to Cart.
Jim Morris
They're talking to me about how they.
Jim Morris
Can cross sell on the add to cart page.
Jim Morris
Or, I mean, add to cart pages.
Jim Morris
Have been bloated and overdone at this.
Jim Morris
Point, but you get the idea that.
Jim Morris
At some point they thought, oh, let's innovate this page.
Jim Morris
For better, for worse.
Jason Knight
Gamifying yeah.
Jim Morris
So don't make big, long prototypes.
Jim Morris
One, because they make for long, boring user tests that don't give you results.
Jim Morris
And don't do it.
Jim Morris
So again, if you're testing with users.
Jim Morris
And you're not changing your mind or.
Jim Morris
Pivoting, just stop doing it, you're wasting time. Now, usability testing has its place.
Jim Morris
Like, you don't want to confuse people.
Jim Morris
So I might take a final prototype.
Jim Morris
That'S almost ready for engineering that's pixel.
Jim Morris
Perfect at the very end of discovery as we've been refining it and run it through one more time.
Jim Morris
Sure, usability does matter, but not unless people want to use it.
Jim Morris
Okay?
Jim Morris
And then don't make a prototype that.
Jim Morris
Is like you would use it in real life. I coach designers to make prototypes that are ugly because it may not follow.
Jim Morris
The company style guide if I have.
Jim Morris
A call to action.
Jim Morris
And the company style guide is really subtle. When a screen shows up, I don't.
Jim Morris
Want to tell a user they don't do anything.
Jim Morris
Would you click this button?
Jim Morris
I want the button or the link to be so obvious that they have.
Jim Morris
To tell me they're not going to.
Jim Morris
Click it or they're going to click it. So as we lead people through an.
Jim Morris
Experiment, I often do things to the thing that you're interacting with so that.
Jim Morris
I don't have to talk to you.
Jim Morris
So I have to talk to you, Jason. Again, it's not a leading question, but.
Jim Morris
I might want to explain myself. It's a natural thing to want to explain myself.
Jim Morris
And so I need to take myself out of it. And part of it is making the.
Jim Morris
Solution test that does this.
Jim Morris
Those are a couple of the things.
Jim Morris
Don't overdo it.
Jim Morris
Don't overbuild.
Jim Morris
People will put a lot of buttons.
Jim Morris
Like they said, make it obvious, and.
Jim Morris
They'Ll build the resulting pages for all those buttons. Well, if no one clicks on buttons.
Jim Morris
One, two, three and four, and they only click on buttons five and six.
Jim Morris
Well, next user test build the buttons.
Jim Morris
For six and seven, because they often think, they don't believe or they don't.
Jim Morris
Understand that user testing is continuous, that.
Jim Morris
I'm going to come back to this.
Jim Morris
Topic so I can actually tackle a.
Jim Morris
Subset of what's in my brain.
Jim Morris
They make long prototypes, but that's because.
Jim Morris
They'Re always trying to build long projects. So if we take Agile, the one thing that was successful in software is.
Jim Morris
We just build things and release them.
Jim Morris
Faster, like engineers can't tell you why.
Jim Morris
We can't tell you we're bad at software, but if we do it faster.
Jim Morris
We'Re better at software.
Jim Morris
Actually, in product, it works like that too.
Jim Morris
Our ideas, our dreams are big, but.
Jim Morris
If we start to write them down, we realize how actually bad they are. But if we break them down and do discovery in this, like, let's do.
Jim Morris
Three versions of Add to Cart. Add to Cart becomes awesome.
Jim Morris
Then we do three versions of billing and shipping. That looks great.
Jim Morris
And the users, actually, they can handle this.
Jim Morris
They can handle experience A on this page and like, slightly different experience B.
Jim Morris
And then Experience C. Users are not timid or afraid. I mean, they put up with a lot.
Jim Morris
I mean, just go to Craigslist, Ebay, Amazon.
Jim Morris
These are frustrating sites to use, but they love these sites.
Jim Morris
Yeah.
Jason Knight
So when you get out the back of the testing that you're talking about.
Jason Knight
The experimentation, obviously the interviews that you.
Jason Knight
Get to do as well, you're coming.
Jason Knight
Out of that, as we discussed earlier.
Jason Knight
With some qualitative and quantitative data that you want to do something with.
Jason Knight
Now, obviously, we could probably talk all day about what to do with that data and some of the cool techniques.
Jason Knight
You could use, but I guess if you were working with a less than data savvy team and you wanted to teach them some of the basics about.
Jason Knight
What to do with that data, so they can actually turn what they've done into insight.
Jason Knight
Like, do you have any frameworks or.
Jason Knight
Techniques that you kind of start them.
Jason Knight
Off with to help them to do that?
Jim Morris
Yeah.
Jim Morris
One reason teams do not do discovery.
Jim Morris
Again, or why teams don't replicate a design Sprint, is because analyzing qualitative data sucks. It takes time, especially if you're new to it.
Jim Morris
If I have an unstructured set of interviews, I talked about solution tests.
Jim Morris
Let's talk about generative interviews where they're unstructured. If I have five or six of those, I have to go through half of those interviews. And if it's 5 hours, call it.
Jim Morris
Two and a half hours of interviews.
Jim Morris
To understand what I'm looking for.
Jason Knight
Yeah.
Jim Morris
And in that last part of that.
Jim Morris
I might find something that I want to go back to the first interview.
Jim Morris
And find if there's a quote or a feeling about.
Jim Morris
And I've checked with some qualitative researchers that it's about one and a half.
Jim Morris
Passes through the data, which in 5.
Jim Morris
Hours, it's seven and a half hours. Although I do listen to people at.
Jason Knight
A faster clip just like me with.
Jim Morris
Podcasts, as long as they don't talk.
Jim Morris
Like President Obama, which is slow and then fast and then slow and then fast.
Jim Morris
So it helps to be organized upfront.
Jim Morris
Before you start the first interview, especially with solution tests because they're so structured, which is, again, the reason I start.
Jim Morris
There, because if you start with unstructured interviews, there's so much everyone can take their own understanding from it.
Jim Morris
But if we actually think about a.
Jim Morris
Couple of solutions and exercise our brain to what the solution might be, we're.
Jim Morris
Actually in a better mental state to evaluate someone's reaction to it rather than these kind of very raw interviews that.
Jim Morris
Are just most common in discovery.
Jim Morris
Hey, what do you think? What do you like?
Jim Morris
What's your pain point? Let me follow you around for the day. These are hard to analyze, so let's.
Jim Morris
Just say that a structured interview is.
Jim Morris
Easier with a prototype. They're going to make one primary hypothesis about this study.
Jim Morris
Users, they want to do this specific thing.
Jim Morris
Then on each of the screens we're doing, we're also making choices.
Jim Morris
Like in the text message, I'm going.
Jim Morris
To say, Hi, Jason.
Jim Morris
In the text message, I'm going to put the last four digits of your.
Jim Morris
Credit card or not. In the text message, I'm going to put a link that says, blah, blah, blah.org, this.
Jim Morris
And so even a text message might.
Jim Morris
Have five hypotheses in it.
Jim Morris
We might make three or four different.
Jim Morris
Text messages to see which one users.
Jim Morris
Feel most comfortable clicking or some other action. So then as, as we get through the interviews for the known hypotheses, we can then just kind of check off.
Jim Morris
Yes, no, liked, personalization, didn't like it. Not by asking them, do you want your name in there or not? Just by saying which of these text.
Jim Morris
Messages or push notifications makes you feel most confident about proceeding on this task.
Jim Morris
Right.
Jim Morris
And we might have to read between the lines, but we're going to find.
Jim Morris
One that works or not.
Jim Morris
Actually, a lot of people don't want to click on text messages. They will just go around to the.
Jim Morris
Website that you've mentioned that your car repair is ready.
Jim Morris
They're not going to go to this. They're not going to click on that.
Jim Morris
Link from what is maybe the car.
Jim Morris
Repair company, maybe not. But that's okay.
Jim Morris
The text message had its purpose, so that hypothesis failed.
Jim Morris
But the primary hypothesis of actually going to pick up your car when you.
Jim Morris
Get the notification worked. And so I give them this structure that allows them, ideally within half an hour of the last interview, if they're filling it out as they go to make a distinction.
Jim Morris
Four out of five users agree with the primary hypothesis.
Jim Morris
Or at the end of the interview.
Jim Morris
If I'm like interviewing, it's a live event.
Jim Morris
Did I cover that?
Jim Morris
And my colleagues on the interview can say, look, they can chat me.
Jim Morris
You didn't ask the big question.
Jim Morris
Would you be disappointed if you didn't.
Jim Morris
Get a notification that your car was ready?
Jim Morris
I asked that user says, very disappointed, somewhat disappointed.
Jim Morris
There's some reaction. And then we note that so that.
Jim Morris
Sometimes we get through interviews and we.
Jim Morris
Forget to ask the big question.
Jim Morris
And the big question is, would you.
Jim Morris
Use this or not? It's not that simple. Because users will often say yes when.
Jim Morris
They don't mean it.
Jason Knight
The mom test, right?
Jim Morris
Yeah, that's my great structure, is the.
Jim Morris
Hypotheses to hang it all off of.
Jim Morris
And I also have a technique I.
Jim Morris
Call the user analysis grid.
Jim Morris
It happens when I'm in when I.
Jim Morris
Used to do in person testing. And you can't take a lot of.
Jim Morris
Notes, but I want to make sure.
Jim Morris
That I covered each of the main points.
Jim Morris
And so I'd have users as rows.
Jim Morris
And my main points as columns. And I could usually put five or.
Jim Morris
Six users data on one clipboard's worth of stuff.
Jim Morris
And that grid is now in Excel.
Jim Morris
Or Google Sheets when I'm online, which is most of the testing now.
Jason Knight
Some excellent techniques there.
Jason Knight
Where can people find you after this.
Jason Knight
If they want to speak to you.
Jason Knight
About anything that they've heard on the podcast tonight?
Jason Knight
Or find out a bit more about Discovery or tell you if they've seen.
Jim Morris
Your cat staying out late?
Jim Morris
Yeah, if you see my cat around.
Jim Morris
The corner, please call the number on her collar.
Jason Knight
Have you put a GPS track on her yet?
Jim Morris
I put an air tag on her, yes.
Jim Morris
We love our cat, and she's a little bit wayward.
Jim Morris
We'll see if that works. Anyways, as passive tracking, it's a pretty.
Jim Morris
Amazing device, and it hasn't seemed to.
Jim Morris
Annoy her in terms of finding me.
Jim Morris
I do not wear a tracker.
Jim Morris
I guess my phone is a tracker.
Jim Morris
You can go to Productdiscoverygroup.com or you can just Google Product Discovery Group and.
Jim Morris
You can find me from there. Twitter at SFJ morris like San Francisco.
Jim Morris
Jim Morris, SFJ morris or LinkedIn.
Jim Morris
There's a lot of Jim Morris's in the world. Pretty generic name. But you put Product or Product Discovery.
Jim Morris
After my name, that'll usually work.
Jason Knight
I'll go and do that and save everyone the effort by putting it into.
Jason Knight
The show notes so that Gumber finds you friction free.
Jason Knight
Well, that's been a really fantastic chat. Obviously really grateful you spent some of your time talking about some really interesting topics. And hopefully we can inspire some people.
Jason Knight
To think a little bit differently about.
Jason Knight
Discovery or at least start trying to do it in the first place. Hopefully we can stay in touch. But yeah, as for now, thanks for taking the time.
Jim Morris
Yeah. Thanks, Jason. I really have enjoyed your podcast episodes, and I'm super happy to be on it.
Jason Knight
As always, thanks for listening. I hope you found the episode ins and insightful. If you did, again, I can only.
Jason Knight
Encourage you to pop over to one.
Jason Knight
Night inputup.com, check out some of my other fantastic guests, sign up for the main list, or subscribe on your favorite podcast app, and make sure you share with your friends so you and Nay can never miss another episode again. I'll be back soon with another inspiring.
Jason Knight
Guest, but as for now, thanks and good night.