Innovation Management: How to Learn the Right Thing at the Right Time

By Elijah Eilert

“We failed but we have learned so much,” is often heard from innovators. In principle, we should welcome this, as innovation teams sharing their failures is one of the signs of a healthy and effective innovation ecosystem. But, we need to ensure that ‘learning’ does not turn into an excuse for poor innovation discipline.

We don’t need to gather additional data around questions like: are parents of young children tired? Or if a certain blockchain technology works for a solution without a problem.

Researcher learning the wrong thing at the wrong time

As humans, we often focus on what is fun, or what can garner instant attention, but that’s the wrong way to prioritise.

It is correct, that in their quest for Product/Market-Fit, above all else, innovators need to learn. To be more precise, gather insights. We won’t delve too far into the difference here but think about the activity as market research to generate information. This is what an organisation or VC pays for – more certainty through additional data about the business model that belongs to the organisation, not an individual that is learning something. But this is problematic because for the information to be of any value, it needs to be the correct information.

Going forward, I will be using learning, gathering insights and additional information interchangeably. Learning sometimes sounds a bit nicer and less nerdy but you get it, it’s often kinda wrong.

So how do entrepreneurs, teams and managers need to prioritise learning? What are the most valuable insights? In theory, it is relatively simple. New insights are valuable if they close the knowledge gap. Again, there is no reason to learn when most of it is known anyway. The biggest knowledge gap exists around the riskiest assumptions of how to produce, deliver and capture value (i.e. a business model).

This is not a new concept but doing it well is by no means a given. The following two methods are your best options for identifying and prioritising what needs to be learned next:

Assumption Mapping
Innovation Accounting

If you are not convinced that collecting data/learning/gathering additional insights is the most important thing for innovation teams please read the next part of the article. Otherwise, please feel free to skip it.

Why Learning (and Failing) is so Important

Let’s see if I can convince you.

A fundamental truth of innovation is that each new idea brings uncertainty. 50x returns entirely devoid of risk rarely exist. (Being a dictator helps, but since I hate abuse, I don’t want you to go down this path and if you are already in a dictator-like position I am asking you to step down.)

The more innovative an idea is, the more assumptions are being made. Assumptions about what customers want, how much they are willing to pay, how to reach them and so on. This is what makes the investment so risky. If you need most things to go right in your innovation efforts, change your current products a bit but don’t expect high returns. Even this might only work for a while but eventually, someone else will come along and render you irrelevant.

In their pursuit of transformational innovation, organisations have two choices. Pretend to be able to pick the best idea, draw up a plan and execute it. Thus effectively placing one big bet or running one large experiment if you will. This will test all their assumptions, but the results will only roll in many years and many millions into the future. The chances of success doing it this way are low.

The alternative? Place lots of small bets. Learn quickly and efficiently to weed out the losers as early as possible and only double down on an idea when additional data support this. Some projects will eventually have sufficient certainty to commit to scaling. This point is commonly referred to as Product-Market-Fit (PMF).

As Tristan Kromer puts it in this podcast: “Innovation is a game of information. The person with more information is the person that is eventually going to win, or, at least wins more consistently with a greater number of projects over time.”

Failure is a natural side effect of trying lots of things that have never been done before.

As teams move on in the product life cycle, they need to gather fewer insights and set more executable tasks. On the way to a mature business model, these two things are inversely proportional.

Convinced?

How to Learn the Right Thing at the Right Time

Valid methods to guide innovators in finding out what insights to gather next simply need to identify and prioritise assumptions.

Assumption Mapping is a great and effective way to get started. Innovation Accounting takes it to the next level, as it can quantify the amount of uncertainty within each assumption in the business model.

1. Assumption Mapping

With your weapon of choice like sticky notes, some tape and wall, Mural or other online tools, do the following:

Take some sticky notes and write down all assumptions that need to be true for your business model to become a reality. Think about what makes it desirable, viable and feasible. You may want to colour code them accordingly. One assumption per sticky note.
Create the following 2*2

Place your assumptions along X-axis and make sure they don’t overlap – the goal is to prioritise!
Move them up and down the Y-axis, make sure nothing overlaps – the goal is to prioritise!
The assumptions in the top right corner are the ones you need more insights into most urgently – those should be your priority!.

There have been great articles written about how to do it well, so I won’t be going into too much detail about it.

This method fosters team congruence. Everybody has to agree on what is more and less certain as well as more or less important. This offers a vast improvement to common prioritisation approaches, making it more likely to gather high-value data and learn the right thing at the right time.

But beware, this method does come with a few challenges. The scale is unclear – what is the right distance between sticky notes? This may seem trivial if you run the exercise once, but if you use it to continuously de-risk your business model, sticky notes need to be moved after an experiment is run as new insights have been gathered.

But how far does the sticky note need to be moved? When is it moved far enough to claim Product/Market-Fit? You can see where this is heading. Peter LePiane has described the issues with this approach very well already here.

This method should be in the toolbox of all innovation teams. It provides the first step to a more objective and transparent prioritisation of risky assumptions. It is an incredibly useful tool and as a big fan of the saying “don’t let perfect be the enemy of good” it is still a great tool to figure out what insight to gather next and of course where next to fail!

2. Innovation Accounting

Innovation Accounting is the best tool to prioritise assumptions and define the correct learning goals.

Being an accountant is like being a historian in a well-organised archive, you get to delve into lots of historical data. Being an innovation accountant means you have very little historical data but you still need to predict the future.

Innovators are by definition data-poor. Traditional financial modelling asks questions of innovators they are unable to answer truthfully. Business cases for innovation just don’t work. Instead of making things up and writing great numerical fiction, innovation accounting provides the ability to make more accurate and honest predictions about the most likely outcome of even an early-stage idea.

Innovation accounting also provides the ability to quantify the amount of uncertainty of each assumption in the model. Therefore, it offers a superior prioritisation method allowing us to understand where additional data are needed most urgently. Usefully, it can also calculate how much the additional information is worth, allowing us to put a price on how much an experiment is likely to cost.

Check out this 4 step innovation accounting process any team can follow to find out exactly how it’s done.

The above gives us a simplified overview of how to build a hypothesis-driven financial model based upon key variables that are expressed as a range, rather than a single number.

As hinted, this method makes clear what assumption is most uncertain or what variable is most impactful, especially if assisted by a Sensitivity Analysis. Excitingly it also solves the challenge of placing a sticky note on the 2*2 after an experiment is run. Updating the range of a variable based on new insights from an experiment quantifies this. It translates learning into dollars if you will and demonstrates progress innovation teams are making on their way to Product/Market-Fit.

On the other hand, this method on its own may miss some critical assumptions, for example, legal questions. Using both methods can therefore be beneficial. As in all things, qualitative and quantitative should live side by side, not in opposition.

“Because startups often accidentally build something nobody wants, it doesn’t matter much if they do it on time and on budget. The goal of a startup is to figure out the right thing to build – the thing customers want and will pay for – as quickly as possible.” Eric Ries

Eris Ries reminds us why for innovation teams early in the Product Life Cycle, the rule is that learning equals progress. The ability to ensure that teams learn the right thing at the right time makes it easier to put this into practice. With this being accepted, it becomes easier to manage and create better accountability for teams – organisations deserve better and so do innovation teams.

Lessons Learned

Failure is part of the innovation process & failure is a lousy excuse when the wrong thing was investigated
Building the wrong thing is wasteful and learning the wrong thing is also wasteful

Happy Learning (or gathering insights:)!