What does it mean to practice rigorous testing in a business environment? If you're a resource-strapped entrepreneur, you might think that running experiments with your products and services is the last thing you should be concerned about.

Think again. If you continue down the path of following your own bias-fraught intuition, the unreliable words of consumers, or the whim of influencers around you, then be prepared for many of your solutions to fail.

Experimentation points you more assuredly in the direction of success by systematically weeding out what works and what doesn't. Building the capability to rigorously test new products and services, and iterations thereafter, undoubtedly saves time, energy, and resources in the long run by accelerating the learning process and getting closer to effective and responsible solutions.

A Rigorous Approach to Solution Development

If there's one thing behavioral science tells us, it's that there are all sorts of reasons that people don't do what's in their best interests. This has big implications for you as a founder: Your users won't act in rational ways. Hence, the reason to adopt savvy experimenting practices.

But this also means that you yourself are liable to succumb to barriers to behavior that's in your best interest. Experimentation seems daunting, with big barriers to learning and proper execution. How can you begin to use rigorous experiments to find your way? Whether you're course-correcting a wayward service, adapting a product to increase its effectiveness, or developing a new solution, let's demystify the rigorous approach.

If you're looking for a more reliable path to success, start with these simplified steps, adapted from the scientific method and a behavioral science approach to problem solving.

7-Step Experimentation

  1. Identify the key behavior your product or service aims to change

Your key behavior should be actionable, measurable, and uncomfortably specific. Your team should agree that this behavior is the single most important metric to track the success of your product or service. If your users do this one thing, you will have accomplished what you set out to do. This quantifiable, observable behavior will be what you measure in your test.

  1. Choose one critical barrier to your key behavior

What is a particularly tricky barrier getting in the way of your users completing the key behavior? Think of all the things that could possibly get a user hung up on their path to the key behavior. Chose a barrier that's large enough that if eliminated, could have significant impact on rate of key behavior completion. 

  1. Choose a behavioral principle that you think might get users past this barrier 

The change you'll want to test is something you think will get users over the barrier and closer to completing the key behavior. Leveraging research-backed behavioral principles by adding them into your existing product can help users to overcome barriers typically caused by cognitive biases or insufficient motivation. Consider which principle-- like social proof, loss aversion, or incentives-- you think my be effective in increasing the number of people who complete the key behavior.

  1. Write down your hypothesis

To inform your hypothesis, look back at what you have guessed so far: that people are 1. not completing the key behavior, 2. because of a critical barrier, and 3. if we use a behavioral principle, then we can increase the number of people who complete the key behavior.

You can think about your hypothesis in this simple formula:

If we do this...then this will happen...because...

  1. Create test conditions, finalizing the details of your test design  

To set your experimental design, you need to determine your test conditions. Your control will be the step or section in your current product or service with no changes. Your test condition will be the same step in your current product or service plus the addition of the behavioral principle aimed at getting more people closer to completing the key behavior.

  1. Choose a population you will test on, and randomize to your test conditions

Determine who your test participants will be and how many participants will make up your sample size. Don't forget to randomly assign to test conditions. This is vital for scientifically reliable results-- you want to control for any bias that may lead a certain type of participant getting a particular test condition.  

  1. Follow your plan, run your test, and reflect on the results

When you run your test, carefully follow your predetermined plan. To ensure you get the most out of your time and effort, consider before you run the experiment what decisions you will make based on what you learn from the data. After you execute your test, analyze the results, and make appropriate implementations, reflect on your experimentation process and what you can do better next time.

Practice the Process

The steps outlined here are a digestible approach to experimentation, albeit an over-simplification. In addition, the behavioral science bent frames tests as a way to better learn what might get users to complete a key behavior that your product or service sets out to encourage.

Testing is not a one-time endeavor-- practice is important to develop your skill. But even with one under your belt, you're well on your way to building a testing capability that help you make more reliable business decisions that lead to more effective products. 


Published on: Jun 14, 2018