View Recording Here
UXDX: Design Process Series
Join seasoned UX professionals and leaders, in kicking off our design process series next week. Presentations by:
Head of UX for Pie Insurance, Kevin Philpott on Blitz Research as well as Product Discovery Group founder Jim Morris teaching us about how to innovate with quick experiments.
Open for all to tune into.
More info here: https://uxdx.com/community/community-design-process-usa-2022-11-09/
Remember to sign up to the UXDX newsletter on the site or follow us on social media for updates on these series and more.
Transcript
Rory Madden
Like to welcome Jim. Hi, Jim.
Jim Morris
Awesome. Thanks, Rory. How are you doing? Can you hear me all right? Yes.
Rory Madden
And everything should be up and running and ready for you to go, so I'll hand over to you. Thank you very much.
Jim Morris
Perfect. So today we're going to focus on three simple words innovate, quick, and experiments, and should be a lot of fun. So just background on me. I'm a Stanford computer science major, graduate a long time ago, started my career selling sporting goods online. Back when Amazon only sold books, that company went IPO, then started a company that was a product reviews platform, and we sold to our competitor. And then the federal government considered us a monopoly since we were one and two in the market, and so we were spun out. And after that ten year experience in product reviews, I went out on my own as a product management coach. And so though I started in engineering, I'm here in the product world because this is where we're defining what's going to be the next great product. So I also teach at Berkeley. Right now, I'm coaching consumer organizations, a lot of digital health organizations, and then a grouping of tech development tools, financial, tech, then other companies at the organization, the squad, and the individual level. So what do I see out there in the world? I see a lot of building, build, build.
Jim Morris
Okay. And in that process, people are really falling into what I call the product discovery Valley of Death. And this is where you have great ideas, but you don't test them. And when you don't test them, there's the risk of people not adopting them, not seeing value. You may have great ideas, you may have interviewed folks, but you haven't taken the solutions back to those folks to really see if they would use them. And so today, it's all about these quick experiments. This episode is brought so that you do this solution testing step, and you avoid the Valley of Death. Okay, so let's start with a story of an entrepreneur who spent $200,000 building a website that nobody went to, and he had found an outsourced company to build his dream idea. But the issue with the dream idea is that he wasn't able to market it and didn't quite understand what people would see value in it. It was valuable to him, but he hadn't done the research with others. And so when he came to me, we started from scratch. We built some prototypes to test this website. And I'll tell you what this website does.
Jim Morris
Imagine that you have a home project, and you need a plumber or an electrician. Well, you have to go find somebody. You might be doing it online. A lot of people ask their friends and their family. The premise of this site was that you would ask your neighbors, and so how can you facilitate the asking neighbors? We have to find them what neighborhood are you in? And we tested the interest in this idea, and we tested how people could find out about it. And what we learned led us to create a very bare bones, no code website. So we started with fake prototypes. Quick Experiments moved to the next set of Quick Experiments, which was to use a Squarespace site with Google spreadsheets and to drive traffic to that site from Nextdoor and Facebook Groups. We'd learned about Nextdoor and Facebook as a large source of these plumber and electrician referrals from our customer interviews. When this site got too overloaded and we started to actually drive traffic, and you can imagine Squarespace plus Google Sheets is much cheaper than a $200,000 site custom. He then went on to build a site using Webflow, another sort of more advanced nocol.
Jim Morris
Then he also decided that this market needed he needed to pivot his idea into a home services subscription service. So for plumber and electrician, whatever you needed. What about a monthly fee to have that taken care of? So through these Quick Experiments, this entrepreneur was able to pivot and then grow his understanding of his market. And this was in a fraction of the time it took to build this other site that no traffic was going to. So again, Quick Experiments can help you innovate to get to the right idea. And so this is really what we're looking for. Instead of building all the time, it's learning building and then measuring, of course. Did you get that traffic? Did the outcome? Did you make money? So really redefining product development here again, people are familiar with this. Instead of ideas going to delivery, we're going to add discovery in. So today we'll talk about discovery as innovating. With Quick Experiments, the first step, the first word is innovate. You really need your cross functional team. This is important. What Rory said matters. As you get more folks on the team earlier in the process and you solicit their ideas, you're going to get a much richer set of ideas to start from that might be in the problem space, that might be in the solution space.
Jim Morris
But you need this cross functional team. I have clients that have a data scientist on every cross functional team. I have clients that have doctors on every cross functional team. So sometimes you need subject matter experts, sometimes data experts, somebody from customer support, maybe. Again, Rory mentioned that team size of up to eight. I think that's great. You got to keep it low. There are T shaped personalities that can help you if you have multiple specialties that need to be represented. Okay, so you need engineers. Most common problem I see is that folks create cross functional teams but don't drive engineer participation. And so what they're losing is what's just now possible. Really? Engineers are one of our greatest sources of innovation. They can hear the problem, and they can apply perhaps a new solution they've been thinking about or reading about, or one that might just come out of what their current work stream is. So the concept of just now possible is really important for teams. You might be missing this if you don't have engineers. So that innovate where it's really important. So what are popular engineer started companies? Well, Google, of course.
Jim Morris
Facebook, Microsoft, Apple. So a lot of the most commonly used companies today had either all or more than half engineer founders. Okay. It's really important to make sure that you don't lose touch with your engineers, keep them involved. Amazon took some just now possible technologies. Secure websites, widespread Internet access to launch. In the 90s, my company, Power Reviews, we were a product reviews widget inside of the site. We grew to 1200 clients in 20 countries, all powered by Iframes and Ajax. Tesla really started with this concept of long lasting battery. So there are enabling technologies that allow certain companies to get over the hump. So what are we doing with engineers? We're doing collaborative solutioning with those data scientists, with those doctors, creating that team, really working up front together. So what does each individual do? Well, certainly you're contributing ideas. One of the best ways is to do sketching. So on that innovation side, I want you to innovate by drawing. It not just a designer. I want all of you drawing. These are examples of sketches. These are things that drive innovation. When you have people write down their ideas and then you take these ideas to your users in lightweight forms.
Jim Morris
The Sprint book is a great resource for the type of activity. So I recommend checking this out. You don't need to do these things in five days. You can just take these exercises and apply them every sprint, every discovery cycle. Okay, so let's move on to quick. Quick is a lot of fun. We talked about innovation. Well, quick. What does quick mean? Well, here's a thought experiment. It's very quick. Hello? Wan web. In 1921, a Swiss psychologist invented the Rorschach test. This test got very popular in the 1960s as a way to understand aspects of patients personalities. So you show somebody something and then they give you a reaction. This was sort of the simplest form of a quick experiment. Well, what's a great quick usability experiment? Again, we're trying to figure out, are we confusing the user? So I'm going to slightly move you from psychology into the physical world and into technology. Can they accomplish their task? It's usually testing one flow. So what's quick here in usability? Well, one flow is this dining room table that I was creating, that I was planning in my house. How wide would it be? What was the length of it?
Jim Morris
How far from the ground? Everything was built with cardboard. I had family members sit around the table to test this out. So again, this quick experiment of building it in cardboard rather than in wood, where it was really expensive, led us to understand all the parameters, the usability of this item, and then we're able to build this table. So usability experiments are really valuable. If you can do them quickly, you can learn before you spend all the time in engineering and building. So what's another form of experiment that's quick? Well, I'm going to show you a backlog prioritization experiment. So when we talk about the theory and the practicality, this is a lot of fun here. Let's say you've got a lot of items in your backlog, and I can tell you that the various frameworks for prioritization aren't that useful because you're often making up numbers for what's the impact, what's the confidence, what's the effort? If you do some discovery about the ideas, do some user testing, you can learn about the priority with respect to your customers value. Do they want this thing? That's really what we're interested in. So let's go talk to users.
Jim Morris
Backlog prioritization. So you got multiple items. How can we do rapid discovery to find out which one will produce the most value? Okay, so we have to put ourselves this is a thought experiment in itself. I want you to pretend that you work for Zoom and that you have thought of a bunch of ideas for improving breakout rooms. I teach these classes at Berkeley. They have 30 students in them. Each team is three people. So often in my classes, I have ten breakout rooms for five to 10 minutes, and I need to decide I'd love to have more tools to help me manage and improve. So one experiment is let's say I'm a Zoom user, and I've got Zoom on my phone, and Zoom starts sending me push messages to gauge my interest. So I call these micro prototypes. Let's read this text here. Zoom now offers advanced features to manage the success and health of breakout rooms. I can gauge whether somebody's interested in breakout room improvements right away. Would they swipe to learn? Would they dismiss? Have them talk about it in front of you. So this is a moderated interview. I show this.
Jim Morris
Either I'm with them and this is a prototype, or that's on a computer screen. And over Zoom actually talking about Zoom, I show a different idea. Zoom now offers the ability to listen into a breakout room without having to enter it. A new feature. See who has video turned on or off in breakout rooms without having to enter them. Okay, new feature for breakout rooms. Zoom shows you which rooms are the quietest and which rooms have the most people talking. Again, you may not be the product manager for Zoom breakout rooms or Zoom itself. What I'm encouraging you to do is to think of the ideas in your backlog and quick ways to explain them to users and to get their interest. Let the users help you with quick experiments, decide what to work on backlog prioritization. I did this with a client. They worked with educational technology. We went into a classroom, and we showed a variety of ideas, and the team was able to hear the excitement and understand the response as to which of these ideas was most valuable. So you, too, can go in to your target customers with something simple, almost like a marketing prompt, just to gauge value.
Jim Morris
Then you can start to build bigger prototypes on it. So try the microprototype idea. Okay, so again, we were really small with these micro prototypes. What do I see in the market? Lots of stuff. People testing too much, creating these end to end experiences. So what I want you to do, I want you to not test an entire experience. I want you to pick a piece of that experience to innovate. This is an ecommerce flow. How about just picking add to cart and really testing something like so, again, we can be quicker if we can focus. You can certainly test each of these other aspects, but to be Quick, you want to test them one at a time, not in these mega testing things, testing sessions. I also want you to start with the vertical slice. So what is really valuable about that add to cart area? Can you launch something? Can you, from end to end, deliver a piece of value quickly? So when I say Quick, you may have a vision of that entire product that is reflected in that left pyramid that's full. What I really want you to do is iterate with these vertical slices.
Jim Morris
Okay? When I say Quick, what does that mean? What time frame am I talking about? Here's a one quarter timeline where you learn over a four to six week period by testing these four prototypes. You build it, and you launch it, and then you do some measurement. This is not an science. Everyone's going to have to tailor the surrender. This is not one size fits all, like Rory said. But what this is is a way to think about what Quick means. Some people in the game, gaming companies or in Google search might be testing multiple things a day. But here in that learn phase, you're running through Quick experiments to figure out what is working and what's valuable. So let's move on to experiments. You're doing great. We got to innovate. We've got quick. Now let's do experiments. So again, solution first versus experiment first. In the solution first world, you've got that one long flow, and you just build it. No, testing experiment first. Is this respect for the market? That may be able to help you figure out what's actually working. Well, multiple flows, evaluating, maybe building. Okay, so let's take something where we've got multiple flows.
Jim Morris
We make a compare and contrast moment that reveals the user's choice. And then if I were to pull this all together, when I have multiple flows with experiments, I'm actually bringing more of those sketched ideas, more of those ideas from my team to the user. So there's more room for innovation if I'm putting more ideas into the market more often. So innovation can often be a quantity game. So here we go. I had you focus down to the add to cart. Well, what does experiment mean? Experiment means give me three add to cart experiences. Could be four, it could be five. I've tested up to seven experiences in a half an hour. It depends on what you're testing. What is valuable to the user? Is it a crosssell? Is it an upsell? Is it a discount coupon? Is it a service offering? But you're going to add these multiple options in. And what experiments do is they help you avoid confirmation bias. When you offer no choices to users effectively, you're saying, hey, choose this thing. When you offer choices, you're being open minded. And as you're open minded, the idea from that data scientist might really come through as the winner or the doctor that you put on the team.
Jim Morris
Doesn't have to be the product manager, doesn't have to be the designer. In fact, the engineer might come up with the idea that the user selects. So as we go from big to small, you have this big flow you want to test. This is an analytics experience. Let's just try that dashboard. You still have a lot going on in that dashboard. Let's pick one of these graphics. Is this graphic? Is this visualization driving value? Some of you might be analytics PMS out there. Go to the next level is now I want to take that and I want you to create multiple options. And that's what brings the experiment aspect in. So to recap this, you start with that one little module there and you do some experimentation. This is where you're going to build an incredible product one piece at a time, rather than this big bang testing where you are doing long experiments that really never get off the ground. So let's pull it all together. So let's do innovate quick and experiments. I'm going to show you something and then we'll be out of here. This is great zoom breakout rooms. This is a link.
Jim Morris
You can go look at this prototype and play with it yourself. This is just an example of giving three options and a compare contrast that I built myself in Figma. This is how easy it is with Figma. It's a great tool. There are lots of other tools you can prototype with. So here's a scenario of a bunch of folks. I'm going to put them into breakout rooms. Then I have to figure out which one to visit. I'm just giving you screenshots because we're in a presentation situation. But this would be an option that this would be a flow that someone would play with. Option one shows these items. Option two shows some videos that are on and off. Option three has no videos, but it does offer me the ability to listen in and listening into Zoom breakout rooms without people knowing it sounds creepy. It's also one of the most highly selected options when I give this prototype to folks, and then at the end, there's a compare and contrast moment. So, again, this is the way you pull these quick. This is how you innovate with quick experiments, is to make these things that aren't pixel perfect, but get your point across.
Jim Morris
Okay, so closing out here, don't settle for blah feedback. I like this. That's interesting. This is all kind of math feedback. This is not what you want. People are being nice to you. I want to hear, oh, wow, is this available? I want that. Emotional reaction, users mindset, content, prototypes. These make great experiments. I really encourage you to get out there and innovate with quick experiments. I've done a lot of writing on solution test interviews, so really check this out. And again, stay in touch. Thanks for listening. And here's my LinkedIn, my Twitter, and you can reach out to me through Rory. Thanks for having me.
Rory Madden
Great. Thank you very much, Jim. And just as you were going through your kind of experiments with Zoom, I've realized we were getting spams.