Overcome Common Barriers to User Testing
Agile Engineering wastes time when you send ideas to the engineers without having user tested them
User testing on site at Legoland in Carlsbad, California
You have the best engineers that recruiter’s fees can find and you’re wasting their time. Untested ideas are the business equivalent of guessing. You probably don’t let your finance team guess at the numbers when closing the books. Why would you let your Product Managers guess at ideas and occupy precious engineering time. I know why you are sending untested ideas to engineers.
These are the common barriers to user testing:
You're too busy
You’re a Product Manager and your boss told you that you’d be the CEO of your product line. Sounds great until you realize that you’re actually responsible for everything in the software development lifecycle. You run the agile team. You write the documentation and release notes. You make your own wireframes. You try to do your own user testing. You do the marketing and social media outreach. All while keeping up with current trends in an amazingly fast moving environment.
Soon enough, you’ll be sucked into every part of the business as the subject matter expert, sales engineer, product marketer, customer support agent (gasp!) and slowly you’ll stop creating new and interesting ideas. You’ll be overworked so your creativity will plummet. You get too close to deadlines and do the bare minimum to get through meetings with the executives by making elaborate mockups and presenting confidently. Bad idea. You should push back. Eventually the metrics will bear out that your ideas were just guesses. Mediocrity is inevitable.
Don’t have right kind of Designer on staff
Many people educated and employed in the field of Design are Visual Designers and Graphic Artists. These professionals were more common in the early days of Internet development because of the transition from traditional media where these designers were in high demand. Marketers, MBAs and executives tend to be more comfortable with these types of designers since they are still in the push mentality of business where branding, beauty and boldness are more important than usability, value and simplicity. Don Draper wins every time when we mistake Advertising and Branding for User Experience. Television and Radio are the land of Advertising and Branding. Websites and Apps are the land of User Experience. Don’t confuse them.
Every tech business needs a User Experience Designer (or UX Designer for short). Other folks call it a User Interaction Designer (UI Designer) or Director of Usability. If you can have only one designer on staff, choose the UX/UI designer. You can always outsource the Visual Design at the end of the project when you know your User Experience is awesome. One way to justify this choice is that modern designs for websites and apps are likely simpler designs in the age of Mobile First.
Wrong team in place
Simply put. You likely don’t have a qualified UX Designer on staff. Stop here. Find one. That’s your top priority.
If you have a qualified UX Designer, then read on.
Do you have a core Product team to cover the necessary areas of Product Management?
UX designer owns the user
Product Manager owns the holistic product especially the product-market fit where the product works for users and hits the desired metrics of the company (such as revenue or contracts signed)
Lead Engineer owns the feasibility of the product (Can it be built?)
Data Analyst owns the creation and reporting of the Key Results that serve the Objectives (think OKRs here) that hold the team accountable for results and hold the executives accountable for not micro-managing
You’re lucky and smart like Steve Jobs and Elon Musk
In Silicon Valley, the stories of Steve Jobs (Pixar, Apple) and Elon Musk (PayPal, Tesla, SpaceX, Solar City) are legendary. You are not them so how can you succeed. It’s all about raising the probabilities of success and having less reliance on luck and guessing. You can raise the probabilities of success by meeting users and investing in user testing. Get that probability higher by eliminating bad ideas and nurturing ideas with potential. Expand and tweak the ideas that your target users love not necessarily the ones that your CEO/Founder/stakeholder loves. Without testing, you might eventually deliver the right product but that’s right up there with the concept that thousands of monkeys banging on typewriters will eventually deliver a Shakespeare quality play.
Unfortunately, many product teams are dominated or even run by the CEO or Founder or stakeholder. In practice, this is how most companies are starter. Too often, ideas go straight from the CEO to the engineer without any user validation. Venture Capital funding should not be seen as the validation of an idea. That money should be used to evolve that initial idea so that it works with the market and users to make a great product. This is the wise premise of the The Lean Startup.
You have as many engineers as Google does
Most companies don’t have hundreds of engineers. Google can launch 50 products and be successful if just a handful win. Your hit rate must be higher. You probably need to have 1 winner for every 2 or 3 products that you launch. You can raise that probability of success through user testing. Google does user test their products but they have a much wider margin of error than all other companies.
Never tested ideas before and don’t know how (no culture of testing)
Maybe you have an engineering focused culture. Maybe you’ve never tested before. There’s no time like the present to start. Pick up a copy of Inspired by Marty Cagan or go to one of his public workshops. In the meantime, sign up for an account on usertesting.com and start running people through your existing website or app. Ask them to compare and contrast your site with your top competitors. If you’re a self-improver, you’ll find plenty of things to improve via these simple tests.
Before you send those new mockups to your engineers, run some comparison tests with your current website or app using the site usabilityhub.com. You’ll appear more confident in front of others when you tell them that you worked with target users to choose the company’s next user interface through repeated testing rather than the usual technique of “Just trust me”.
Users don’t know what they want
You believe that feedback from any given user is just anecdotal. Once you witness the confusion experienced by a couple users at your seemingly “ready for engineering” mockup, then you’ll start to believe that a few users can teach you quite a bit. Start with a few user tests and you’ll quickly notice and then fix confusing user interface elements before you send your design to the engineering team saving a few weeks of wasted development.
Don’t boil the ocean with each round of user testing. Boil a thimble.
You might work for a Deadline-Oriented Perfectionist Executive
As a Tech Executive, you have taught the team to present concepts with finished pictures and polished mockups. Why not. This is what you’ve been used to for years. When presented with a concept, you tend to ask questions about the visual design or marketing…Is that the right color? Is that link or button in the right location? Is the name of the feature correct?
Instead, ask what users thought of the concept. How did the concept evolve from the beginning to what you are seeing now. Understanding this evolution is key to understanding how your product team is engaging with your target market and will make the company smarter and more nimble.
Fair enough, but maybe the team is too slow. So you set a deadline for a presentation on a new concept. Combine this deadline atmosphere with a line of questioning that always focused on the look and feel of a concept (rather than its usability and value) and you might be working for a Deadline-Oriented Perfectionist Executive. This is typical since many executives don’t know how to manage Product teams. Unfortunately, this environment relies too much on luck. You’re not encouraging your team to evolve their ideas through working with users.
You’re not giving them the time to test and the intellectual space to fail.
Time and space will raise the probability of success and reduce the reliance on luck. Remove educated guesses…insert evidence. The transparency of explaining and publishing the actual tests within the company will increase buy-in. You can still create deadlines but those should be focused on milestones such as completing the first prototype, completing the first round of testing, turning around the learnings from user testing in a fixed amount of time. If you move in this direction, the team will likely let you know what’s getting in the way of testing and then the real conversations in a successful product-oriented business begin.
Jim Morris, Product Discovery Group
Jim coaches Product teams to collaborate with each other and seek customer input early and often during the design and ideation phase.
