Why Success is a Bad Thing for Product Leaders
“Success bias” led me to waste hundreds of hours of engineering time on poorly thought out ideas.
Having a product that wins in the market is extremely satisfying.
Trying to repeat that success is equally frustrating for many Product Leaders.
These failures can result from a Product Leader's “success bias.”
Did we attribute success to ourselves and our team? Or was it more important to be in the right place at the right time with just enough of a feature set?
Early in my career, I had a success that led to future failures. This “success bias” led me to waste hundreds of hours of engineering time on poorly thought out ideas.
So how did this happen?
Early Success
Four of us started PowerReviews in 2005 to bring product reviews capabilities to every online retailer. We knew product reviews were a winner. Amazon had pioneered them to great effect with books. The PowerReviews team had been at the online retailer Fogdog.com (IPO in 1999) where we built the product reviews capability for our loyal customers. They loved it and reviews increased our sales and loyalty. We knew other retailers would be interested due to research indicating that Amazon was taking business from others in part because they offered reviews.
We built a solid product reviews platform that 1,000+ online retailers purchased.
But each time we wanted to create a follow-on product...a new standalone product...we failed.
Future Failures
One of the turning points of my career came when we hired our first Product Marketer. He asked me, the Head of Product, how many people used our social network integrations. I didn’t know and it took us a week to gather the info.
When we looked at the numbers, we found that less than 1% of users ever used this product.
We had built these integrations based on market hype. We learned later that purchasers may buy on hype but won’t renew without value.
Another influential mistake was building an underdeveloped idea. We wanted to enable ecommerce shoppers to ask a question about a product for sale online. Sure enough, consumers had lots of questions but our clients, the online retailers, didn’t have enough personnel to answer the questions. So unanswered questions accumulated on their product pages. We didn’t work closely enough with our clients to plan for this outcome. One by one they removed this new offering from their site. Having unanswered questions was worse than having no questions at all.
In each instance of creating a new product, we were overconfident from our previous success with product reviews. Our “success bias” prevented us from delivering real value. We did as little as possible to close the sale. In the long run, lack of value resulted in failed products.
The Steps to Avoid Failure
Stop assuming you know the answer. Future failures are often caused when we assume our new ideas will have the same success as our previous ideas. Avoid “success bias” and be open-minded enough to know that educated guesses and a successful track record are not enough to make a great product.
Validate ideas with real users. Don’t waste time writing code for unvalidated ideas. Embrace the unknown and do your homework about which product to spend resources and time on. In Product, that homework is called Product Discovery. Product Discovery combines a growth mindset with a collection of techniques to validate ideas with real users before writing code. It's efficient and effective at finding products with high potential. At PowerReviews, we had done a form of Discovery by seeing product reviews in action at our previous company, Fogdog.com. We witnessed the consumer adoption, excitement and value. But we had "success bias" and imagined this success happening with everything we built. In retrospect, we needed to let go of good ideas so we could find the great ideas.
Root for the underdog. Don’t sell out future products by always allocating resources and energy to current offerings. Avoiding “success bias” depends on true commitment from more than just Product and Engineering. New products are underdogs. They have no revenue and don’t naturally command the attention of employees who have more incentives to focus on the core product (irate customers, commissions, bugs, etc). Thus, the success of follow-on products requires leadership from the top of the organization to commit serious efforts into future success, not just optimizing the core product.
Always Be Renewing. Be disciplined and watch the metrics. We assumed that our follow-on products would perform as well as product reviews. We often sold new products with suggestive and not quite metric backed marketing. In the end, we could close the initial deal but struggled to renew. Instead of Always Be Closing, the subscription economy dictates that Product teams should Always Be Renewing. If you pay attention to real value delivered, you will find customers who love your product. These customers will renew. You can write a case study about their success, host a webinar and have your customer tell the world about it. No prospects believe you during the sales process, but they will believe a similar company who tells a data-driven success story on your behalf.
Leverage an expert. You can self-teach these concepts including Product Discovery via books and the Internet. To get up to speed faster, hire an expert coach to teach the Product team how to validate ideas with real users effectively and efficiently. When Product teams (including the engineers writing the code) work on unvetted ideas they can waste up to $100,000 per month (depending on the team size) for months and months with very little returned revenue. Hiring a coach can cut out these wasted hours and help teams learn to validate ideas quicker with more confidence.
The Joy
Letting go of assumptions and letting go of “success bias” means facing the potential of failure.
By using the steps above, my teams experienced the joy of avoiding future failures. We gathered evidence using Product Discovery. We validated and then built just the concepts that created value.
Where previous conversations devolved into “me vs you” dynamics, new discussions focused on data which improved decision making and morale. We all focused on customer value over opinions and hunches. Executives let go of poorly received ideas and rallied the company to market and sell ideas that showed actual potential.
Our most successful product launch included metrics from day one. We recruited existing clients into an early access program. After two weeks of collecting data, we had enough information to prove our value. This armed our sales teams with real world analysis they could present to prospects. We even published a case study from the #1 non-Amazon online retailer in America.
We counteracted our “success bias” and avoided failure. This product launch ended up being central for PowerReviews’ acquisition months later for $168 million at a 13x multiple.
