Speed vs Quality when using AI in Discovery
For many years, I revered the Sprint book for finding ideas in a fast, five day, easy to follow recipe.
The Sprint book inspired teams to get out of the office and test their ideas quickly.
After running design sprints, I saw a pattern…teams got speed, but not quality…usually because we weren’t anchored to a real customer problem and a business outcome.
It ended up feeling like we were going through the motions of discovering new ideas but not innovating anything useful.
AI also accelerates a design cycle but this even greater increase in speed can shortcut deep thinking.
The lack of deep thinking allows bias to creep back in. (I believe bias is our natural state that requires intention to remove).
Combining bias and speed can lead to low quality findings and even more wasted engineering time on products that we want to build but may not get customer adoption.
To reduce the friction of discovering new ideas while maintaining quality, teams need to find the middle ground between AI acceleration and AI dependence. AI not only doesn’t magically “fix” discovery, it can amplify bad habits.
My tried and true process to avoid AI-enabled discovery mistakes is to collaborate with cross-functional peers where everyone has knowledge of the customer problem, accountability to a business outcome, ideates individually, and takes multiple options for solutions to customers for feedback.
AI unlocks the creative process
The impact of new AI tools is real. These tools allow us to build prototypes much faster, create draft interview scripts, design Ideal Customer Profiles to recruit users and more. This can be done in hours instead of days or weeks.
And it can be done by a variety of people on the team.
Previously, a designer or a user researcher did most of the work of Discovery.
Now the product manager, the engineer, the data scientist can manifest an idea into an experience to test with users.
Leaders should expect AI-enhanced teams to more quickly go through their idea backlog, find the winners, discard the losers and focus on the right product.
For designers, AI-based productivity enhancements should enable them to do less drudgery work and focus on creating new ideas, new intellectual property (which the AIs don't do very well, yet).
Bundled vs state of the art AI tools
Companies that adopt the right AI tools will move faster than their competition, and have a chance to compete with new startup entrants (think about the early adopters of cloud and how they scaled faster than the competition).
Companies new to the market are hungrier and tend to use the high quality AI tools.
Larger companies tend to use lesser quality, bundled AI tools. This checks the box that their teams have access to AI tools but it’s not the same. Bundled tools are convenient but not state of the art especially in new products like these AI tools.
In my observations, access to bundled AI tools may not produce meaningful productivity gains because the user experience and output quality are behind the state of the art.
Engineers get better AI tools
The exception to bundling is in engineering where teams at companies of all sizes appear to have access to the best AI tools. It’s the non-engineers that get stuck with the bundled tools.
The engineering counterparts at my clients tell me they see their engineers achieving 30% to 2,500% productivity gains but report that AI usage in product and design is falling behind. Okay, “2,500%” is obnoxiously high but an engineering leader did claim that to me. The engineering leader’s worry is that non-engineers will fall behind as engineers get faster and faster.
Yes, productivity enhancements are easier for engineers since their input and output is mainly working with code and the computer.
The productivity gains for Product and Design tend to be harder since they work on varied types of deliverables (documents, slides, spreadsheets, custom tools, designs, prototypes, etc) and use humans as inputs and outputs more often.
What AI does NOT change about Product
If your product is garbage, no one will use it. No matter how much AI you shove into it.
Making products faster doesn't necessarily mean you're going to win, because you could be flooding the market with sub-standard products, even ones that hurt the quality of your brand.
The rules of testing ideas with customers, understanding what the market wants, watching the metrics…all of these concepts still apply.
Product adoption is still a ruthless, relentless process of selecting and discarding products based on their utility.
Think about how the survival of the fittest happens on everyone’s iPhone homepage. If we don’t use an app we move it off the home screen.
For business products, this eventual switch to the best products happens more slowly due to contracts, procurement rules, integrations, training, rollout and so on. But the process of evolution will happen there too.
Product Discovery Fundamentals
Jim coaches Product Management organizations in startups, growth stage companies and Fortune 100s.
He's a Silicon Valley founder with over two decades of experience including an IPO ($450 million) and a buyout ($168 million). These days, he coaches Product leaders and teams to find product-market fit and accelerate growth across a variety of industries and business models.
Jim graduated from Stanford University with a BS in Computer Science and currently lectures at University of California, Berkeley in Product Management.
