Page 206
TOOL VALIDATION CANVAS
Original concept by Ash Maurya
With your experiments in place, it’s time to start testing them and tracking the progress over time. Sometimes your tests will return positive, sometimes negative. Along the way you’ll iterate – adding and changing as you go. This tool will help track your progress over time.
|
|
|
FOCUS check progress |
± 15 MIN session |
TEAM all together |
TRACK YOUR PIVOTS
Running one experiment is almost never enough to know you’re right. Some startups make many pivots before they find the right product-market fit. In every case, it’s absolutely essential that you know where you’ve been before moving on. It would be a waste of time and resources to continue to run the same exact experiment over and over again, waiting for the results to magically change. Looking back will help you understand the choices you have already made and avoid the resurfacing of invalidated assumptions later in the process.
VALIDATION PROCESS
The goal of the validation process is to learn as much as possible, as fast as possible. You’ll want to spend as little time and effort as you can in this process, while you maximize the outcome. With that in mind, you’ll need to run experiments iteratively. The Validation Canvas is the central nervous system of this process.
The Validation Canvas was created by Ash Maurya’s Lean Startup movement and adapted for this publication.
YOUR BEST GUESS
It starts with the value proposition you have at this moment. This is your current “best guess” with respect to who your customer is, what problem you solve for them, and what your solution is for that problem. No need to make this overly complicated. Start small, with the simplest solution you can test. Over time, pivots will change that best guess.
EXPERIMENT
Your current best guess is based on assumptions. Find the riskiest one: the one that, if it’s wrong, completely disproves your best guess. Choose a way to test that assumption and define what the minimum criteria for success are. Plug this stuff into the experiment canvas and run the experiment.
When it comes to experimentation methods, you can choose from things like exploration, pitching, or even a concierge model. Through exploration, you’ll learn more about the problem you’re trying to solve for.
Pitching will help you understand how important your customer thinks the problem is. Is it a must have or a nice to have? A concierge model will help you understand whether you can deliver on the customer’s expectations at first, doing it by hand. 
Page 207
 |
DOWNLOAD Download the Validation Canvas from www.designabetterbusiness.com |
CHECKLIST
You tracked your experiment.
NEXT STEP
Pivot, persevere, or redo.
Page 208
Page 209
Page 210
VALIDATION HACKS
TIP! The next time you plan an experiment, go through it first with a few of your colleagues and fix any problems. After all, it would be terrible if you went out and asked thousands of people the wrong questions.
QUALITATIVE VS. QUANTITATIVE
Although the results from a quantitative test are often easier to interpret, your first step in experimentation is to find out what to test. Do a qualitative experiment for that. What are the typical things people do? Why do they think they do those things? Remember, running a qualitative test does not mean you can’t gather numerical data.
Qualitative experiments are great when you want to capture richer data about what customers experience. Make sure you’re testing what people actually do, rather than what they say they do. It’s also important not to ask about future behavior, as your customer probably won’t be able to answer without guessing (the future is uncertain). Instead, ask about current behavior.
After doing that, it’s often great to back it up with a quantitative test to see how many people actually display the same behaviors in reality. A qualitative test will provide insight as to how well your quantitative test is geared up to measure the things you want to measure.
Keep in mind that qualitative tests are difficult to use in situations where you are testing the response to a very small change. If you want to test two different colors of button online, the data you’ll get from a qualitative test would be quite useless.
Another thing that people will happily tell you when you ask them is that they will (or won’t) buy your product. However, that information is useless. Only a test where they actually buy it has any real value.
RUN A SMALL-SCALE TEST
Running an experiment takes time and effort. Before going out and doing a large-scale experiment, try it out first on a small scale to iron out any issues with the test itself.
On the Discovery Channel show Mythbusters, the hosts regularly try out their experiments on a small scale to see what the possible outcomes could be and to make sure their large-scale tests will yield reliable results.
DON’T INFLUENCE THE RESULT
When running your test, make absolutely sure nothing you do or say is secretly influencing your result. Don’t “sell” your prototype to the test subjects. Let them experience it as they would without you present.
Online, this is quite easy to do using analytics, but offline it can be more difficult. Go out of your way to present the prototype in as natural a way as possible.
One way of doing this is to have a test subject record their experience using a camera or notepad that you give them beforehand.
Page 211
TESTING THE COMPETITION
When you don’t yet have a prototype to test, or want to get a head start, try this: get people to test a competing product or service. Find out what they say about it.
Even if you don’t have any direct competitors yet, this can give you valuable insight. Some of the assumptions you have about your own idea hold true for other things as well.
There are some cool examples of appliance companies which have employed this simple experiment using a similar commercially available kitchen product, for instance, and have invalidated some of their most important assumptions. For the price of a food processor and an afternoon of their time, they figured out they had to really make a big pivot.
OFFLINE A-B TESTING
Online, A-B testing is all the rage. Show users different versions of the same advertisement or web page, and find out which one they click on the most. The data will tell you beyond doubt if the change makes sense to implement.
Offline, you can use exactly the same tactic. You don’t have to show every test subject the same prototype. Mock up a brochure of your product (using Keynote or PowerPoint, for instance) and change the price, show different colors, or play with another variable, and see how this influences the result.
Make sure you only change one variable at the same time, though, or your result will be confused!
If you run a few experiments at the same time with a different value for the variable you are testing, you can save a lot of time.
TIP! Include a few versions that are out of your comfort zone. If you are testing a price range, include a price that is bordering on the ridiculous, too. Maybe it turns out that customers find that price less ridiculous than you thought.
