An Unsettling Trend: The Analytics-Free Environment
Too many Product teams operate in an “analytics-free” environment.
They don’t track product usage, business results or customer satisfaction.
Without analytics, we lack the "eyes and ears" to see and hear what’s happening with our users.
[ This is Part 1 of the Analytics Transformation series]
After I launched my first product in 1996, I wondered if anyone had used it.
We didn’t have an “analytics” system then but I was curious.
The product was a digital postcard creator for the travel site Backroads.com.
Would-be buyers would visit the Backroads website, find a trip they wanted to take and send their friends a digital postcard to convince them to join them on the trip.
As the engineer on the product, I knew exactly where the data was.
Even though it was an “analytics-free” environment, I sought out the data so that we could improve the next product.
Almost everyday, I would logon to the server and count the files on the server to know how many postcards had been generated. Then, I would check the log files to see if anyone had actually looked at the digital postcard that had been sent to them.
All of this data helped us improve the product and made us smarter technologists.
Some of my clients store private data such as electronic medical records, surveillance video, and credit cards. In these cases, companies need to be vigilant in de-identifying, anonymizing and encrypting certain information as well as limiting access to a need to know basis.
Working in a regulated industry is not an excuse for having an “analytics-free” environment.
Fast forward 20 years and my team created a website that grew to 60 million annual visitors. This website was the sole source of revenue for our company. It started out as an “analytics-free” environment.
When revenue went up or down, my CEO would ask me why.
Without data, we had no “eyes and ears” to explain these trends.
Slowly and steadily, we instrumented the entire site to create different ways to analyze the 60 million user experiences happening every year: Google Analytics, Adobe Analytics, CrazyEgg, etc.
As the leader of this team, I made sure we had a read-only, non-production, regularly updated copy of our database where we could run queries (and not take down the website). Everyone on the team had access to this data and the other data sources. Many of the team members knew SQL.
We were laser focused on every move we made.
One Holiday season, we conducted a multivariate test. The winning design produced a 60% increase in revenue. Another time, we tried to suggest alternate products to users who were reading about a negatively rated product. No matter what we did, we were unable to retain users after visiting certain negatively rated products so we rolled back our attempts and moved on.
Analytics drove these decisions.
Even though most people associate Product Discovery with qualitative data such as user interviews, every team should be discovering with quantitative data as well.
Everyone starts in an “analytics-free” environment.
Can you be the leader who breaks through blockers and provides self-serve access to data?
Can you be the product manager, designer, engineer who learns to find and analyze data?
When will you act to add analytics tools to the “analytics-free” parts of your applications?
The answer isn’t necessarily to hire an analytics specialist. I’ve never started my analytics journey with an analytics specialist.
It starts with curiosity, free time and access to data.
This is such a problem that I’ve created a new Product Discovery Group engagement focused on analytics at the Product group level (leaders, their teams and their specialists).
You can transform your team’s approach, access to and usage of analytics.
Reach out to me to learn more.
Jim coaches Product Management organizations in startups, scale ups and Fortune 100s.
He's a Silicon Valley entrepreneur with over two decades of experience including an IPO ($450 million) and a buyout ($168 million). These days, he coaches Product leaders and teams to find product-market fit and accelerate growth across a variety of industries and business models.
Jim graduated from Stanford University with a BS in Computer Science and currently lectures at UC Berkeley in Product Management.
