I was fortunate enough to take some time out to attend the Melbourne Cup Carnival this year on what turned out to be an absolutely gorgeous spring day, perfect for a day at the races. As one tends to do when they attend the races, I decided to, as Australians tend to say, “have a punt”.

I set myself a budget of $100 and placed some bets based on variables that had absolutely nothing to do with knowledge of horseracing.

I’m talking about odds, the horse’s name and the jockey’s colors. Now, while this strategy may be flawed, I acknowledged that this was a high risk strategy and hedged my bets across several runners, hoping to at least cover my initial outlay. I managed to recoup my costs and came out on top by a reasonable margin. However, if I had taken this $100 and deposited it into say, a term deposit with a bank, my motivations for doing so and expectations of returns, would have differed significantly.

One strategy is obviously high risk and high reward, whilst the other is predictable, low risk and low reward. However, if we take this simple concept and apply it to how large companies tend to allocate resources for innovation projects, a disconnect becomes apparent. Large organisations apply the same metrics and evaluation criteria on potentially disruptive, risky, ‘out of the box’ innovation as they do for incremental improvements and business as usual investment decisions. Clearly this makes no sense.

Traditional Accounting Metrics Don’t Work

Large organisations exist to execute on a repeatable business model, and as such processes, policies and frameworks have been implemented to ensure this execution goes off without a hitch. Decision makers, as such, tend to place only safe bets that promise low to moderate rewards, based on an criteria of evaluating investments on some or all of the following variables:

• market size
• gross margins
• revenue potential
• time to breakeven
• performance demands
• existing customer requirements
• return on investment
• risk profile
• short term KPIs

But the nature of disruptive innovation is such that:

• the market is small or insignificant, initially
• they promise low margins, initially
• they deliver small revenues, initially
• they can take years to deliver a sufficient return on investment
• they are often not good enough for existing customers, initially

Disruptive innovations however get better over time and as such, the market grows, the margins get larger and the revenue potential becomes significant.

The consequence of this is that large organisations miss out on opportunities to invest in or support potentially disruptive innovations and find themselves investing only on stretching their existing S-curve and sustaining their existing business model until they are disrupted by newcomers who embraced the disruptive innovation in a timely manner.

Just ask Blockbuster, Kodak, Compaq, Borders, Foxtel, taxi networks, mainframe vendors and on.

Airbnb is a great example of a company  that made only US$200 per week in its first year of operating, but today is worth US$25B, a market capitalisation greater than that of the Starwood, Marriott and Hilton Hotel groups respectively. Clearly, not investing in or pulling the plug early on disruptive innovation based purely on traditional metrics such as return on investment can not only restrict a company from exploring new growth opportunities, but render them unable to compete in a disrupted landscape.

Focus On Innovation Metrics Instead

So, what can large companies with a repeatable business model to execute upon and external constraints and considerations such as regulators and shareholders do?

Two questions we need to ask:

• What evaluation criteria should we apply when assessing disruptive innovations?
• How do we determine and measure success to justify ongoing support of a project?

First, we must recognise that our objective at this stage is not so much on delivering a fully fleshed out product to market. Rather, our objective is to findproduct market fit. Unlike the mothership, we are searching for a repeatable business model, not executing upon an existing one. The risk associated with doing something new is at its highest at inception, and we need to focus our efforts on lowering that risk through immediate customer interactions towards finding product market fit.

Second, it’s imperative that we step away from traditional metrics and look at disruptive innovation through the lens of innovation metrics.

What evaluation criteria should we apply when assessing disruptive innovations?

• apply the disruptive innovation litmus test (does this idea have the potential to disrupt or is it a low risk, low reward incremental improvement?)
• what customer job are we addressing and are customers over-served or under-served by existing solutions?
• value proposition (are we actually solving a problem?)
• ability to deliver technology (can we build it?)
• existing competition (are we entering a treacherous red sea or a clear blue sea?)
• analogs and antilogs (has it been done before? are there stories of success and failure?)
• testability (can we test it relatively quickly, economically and effectively using our existing networks and ability to prototype?)

So, say these metrics have been satisfied and some funding has been allocated to an innovation project to explore a potentially disruptive concept.

How do we determine and measure success to justify ongoing support of a project?

First, we need to determine our baseline - where we are today? Second, where do we want to be tomorrow? Ignore dollars for the moment and focus instead on customer engagement.

Measuring Progress and Success

We should be focusing on what Lean Startup author Eric Ries refers to as actionable metrics, you know, metrics that actually help us make decisions and take action. There’s no point knowing that visitors to our prototype website doubled, if we don’t know why.

Actionable metrics include:

• per customer metrics (for example, the number of pageviews per new and returning visitor)
• split tests (A/B experimentation where we can simultaneously test different variations of offers to learn what works and what doesn’t)
• funnel metrics (measuring acquisition, activation, retention, referral and revenue with each customer interaction - the focus being mostly on acquisition and activation initially)

For example, say it’s 2006 and we’re building a cloud storage solution. We have a number of assumptions underlying our business model.

Two that stand out include:

• people will store their files online instead of on local storage devices
• people will pay a monthly subscription fee of $X fee to do so

Rather than look at this through a traditional lens and say something like “we expect this new innovation to generate 5% of our revenue growth target for the year and if it doesn’t within 6 months we’ll can it”, we should be bringing it back to the aforementioned innovation metrics.

Where are we today and where do we want to be tomorrow?

Say, we build a simple prototype - a landing page with an overview of the proposed product and a form asking visitors to leave their email address to express interest. This is before we’ve built anything at all and are simply trying to determine market appetite for the concept. Perhaps we can take it a step further and include a “subscribe now” button with a “$10 per month” price next to it.

Using the aforementioned metrics, we might measure the following:

• for every 100 visitors, how many express interest?
• for every 100 visitors, how many hit the mock subscribe now button?

Where are we today?

We might find that, initially, for every 100 visitors, perhaps for the first few weeks we get 0 expressions of interest. What then? Do we can the project? Or do we start making changes?

Initially, the market size is small and new concepts take time to ferment. We might find that our initial market is a group of techies who are keen on trying new things as opposed to the broader mainstream who are yet to come to terms with such a new technology. There are so many business model unknowns when exploring disruptive innovation such as the target market and customer persona, how best to reach them, what the pricing model should be, what the marketing message, branding and ad copy needs to look like, what core features people actually care about and so on. Get one thing wrong and it could be the difference between success and failure.

So we start making changes of course. Changes to ad copy, to price points, to customer acquisition strategies, to target markets and so on. We can then apply innovation metrics to see whether or not any of these changes had a positive effect on visitor engagement.Assume we brought back the price point and shifted our target market from a B2B model targeting enterprise to B2C, targeting tech-heads. Suddenly, at the end of our second month we’re finding that 10 out of 100 visitors are expressing interest and 3 are hitting the mock subscribe button. We are moving in the right direction and as such, the numbers should be used to report learning, an improvement in visitor engagement and ongoing support for the project.However, if we run hundreds of smart tests over several months and see a negligible improvement in metrics, then perhaps it might be worthwhile exploring a different idea or concept.

Using this approach, we can set monthly or preferably quarterly milestones, where reporting is based on innovation metrics and learnings as opposed to ROI. Once we’ve reached where we want to be tomorrow (say, 5 out of 100 visitors ‘sign up’), we can then begin to apply a slightly more traditional lens having more confidence that we’ve found product market fit and are putting to market a product that solves problems people are willing to pay for.

Finally, it’s incredibly beneficial for the ongoing support of innovation projects that we adopt a mindset of being patient for revenue, but not profit. Any profitable pursuit, even if it’s barely in the black, is unlikely to get canned in times of corporate restructure and cost centre culling.

In summary:

• traditional accounting metrics don’t support the allocation of resources to potentially disruptive innovations
• if we don’t explore disruptive innovations we are likely to find ourselves disrupted by companies that do
• we should explore using innovation metrics when evaluating potentially disruptive ideas and measuring success
• metrics should be actionable
• be patient for revenue, but impatient for profit