Innovation metrics: why NPV and Product Vitality aren’t enough

March 10, 2021

When I speak with R&D and innovation leaders about their key metrics, two measures rise to the top: NPV (Net Present Value) and PVI (Product Vitality Index). These are the de facto standards for assessing innovation impact. Nothing else comes close.

When pressed, the very same leaders express a great deal of frustration. In almost every KPIs conversation I’ve had, the same grumbles keep coming up, often unprompted: NPV and Product Vitality aren’t getting the job done. We need something better.

What’s going on?

Don’t treat NPV and Product Vitality like silver bullets

No one is suggesting NPV and Product Vitality are useless. Far from it: They’re good at what they’re designed to accomplish. But they’re not a fix for everything.

A global technology leader at a major chemicals company summed it up for me this way: “We use the Vitality Index to track overall results. But it’s an imperfect tool, because not all innovations are created equal. I’d love to find a better approach.” And I’ve heard similar remarks about NPV more times than I can count.

Firms calculate Product Vitality Index as the percentage of revenue from new product introductions over a certain period of time – typically the past three or five years. It is a lagging indicator that measures realized value from innovation. By contrast, Net Present Value will be familiar to many as a classic tool of the Finance department, used as a predictive measure of future value. In an innovation context, it tends to be employed on a project-by-project basis. When a project reaches a certain stage gate – especially if that gate constitutes a funding threshold – NPV can help determine whether to approve the next level of investment.

There are pitfalls in applying these metrics to an innovation portfolio. With Product Vitality, it can be tricky to settle on the best definition of “new product.” Opinions will vary on the correct length of time to continue counting products as “new.” And it’s not always clear-cut whether a new technology represents a wholly new product, or just a feature upgrade. Furthermore, Product Vitality can omit real value from inventions that aren’t strictly product-related but still very much a part of innovation’s remit.

With NPV, the artificial distinction of project-by-project “funding silos” can obscure the implicit value of expertise gained by the organization along the way. It also tends to hide the indirect contribution of stalled or “backburnered” projects in the success of other, more promising initiatives. And because NPV is a capital budgeting method, it requires substantial rejiggering (and/or guesswork) to become an accurate reflection of innovation’s potential impact.

Despite all this, I’m convinced that NPV and Product Vitality aren’t the real culprits. Beyond the surface issues, there’s a deeper set of problems. Today’s R&D and innovation leaders face wicked-hard measurement challenges, for which good off-the-shelf options don’t exist. Left without suitable levers for the job, companies resort to the best tools they can find, thus stretching the limits of what NPV and Product Vitality are designed to do. The result is widespread frustration, simmering beneath the surface.

The time value of innovation: why Product Vitality falls short

Any business function benefits from having a clear yardstick for its own success. For R&D and innovation, the current gold standard is Product Vitality and the resulting ROI calculations.

There’s a problem, though: Product Vitality measurements take years to accrue. Let’s suppose your organization calculates Product Vitality from new products introduced within the past three years. Depending on your industry, the upstream technology development cycles can last anywhere from a year to over a decade. That means that Product Vitality as a success indicator is lagging the original R&D bets by at least four to five years, and likely much more – perhaps a decade or two.

This time-delay effect can yield hard-to-fix problems in the reporting fidelity. As the VP of Innovation at a large manufacturing company explained, “I’ve down-prioritized Product Vitality because it creates perverse incentives. People avoid taking risks because they don’t see how it will show up in the numbers.” And the innovation leader of a global chemicals company lamented: “Our researchers keep gaming the system, trying to count every project as a quote-unquote new product.”

The core issue is that Product Vitality is a lagging indicator, whereas R&D and innovation teams must place their bets in real-time, trying to hit the sweet spots of technology readiness levels. Inventing a better steam engine was hugely valuable in the 1800s, but would generate scant returns today. Same with transistors and vacuum tubes. And so on.

Scoring well on Product Vitality keeps the CFO happy, but it does not provide the insights to run a better-performing R&D or innovation function. By the time Product Vitality can “prove” it was worth investing in quantum computing, you’ve already placed a decade’s worth of follow-on bets. Behind the legit-sounding headline numbers, it’s still intuition and guesswork all the way down.

Layers of uncertainty: the limits of NPV for innovation

Similarly, NPV is a great tool for corporate financial investments, but a poor proxy for potential innovation value. That’s because an innovation’s likely impact is difficult to isolate clearly, especially when it’s early in the pipeline – or when it’s on the speculative end of the risk/uncertainty spectrum. When you’re trying to invent the future, it’s hard to predict exactly what will happen.

Consider fusion power, a technology that might possibly go mainstream. Perhaps it will become a key component of a renewables-led energy future – maybe even a dominant power source. Or it might never pan out. At the moment, no one really knows. A quick search in Wellspring Scout returned 47 startups in the space, whereas corporate activity has been halting, at best. Despite the possibility of once-in-a-generation spoils, not a single major energy corporation is represented on the Fusion Industry Association member list.

When there are compounding layers of uncertainty, the logic of corporate accounting tends to malfunction. In the case of nuclear fusion, the technology might not pan out – along any of multiple dimensions. On top of that, speculative innovations often come with uncertainty in the supply chain; a wide array of legal and regulatory risks; looming questions around consumer adoption; the possibility for brand conflict; cannibalization of core revenue streams; and more. In fact, there may never be a time when fusion power looks like a surefire bet. But by the time the uncertainty fades, much of the upside will have evaporated as well.

Similar dynamics factor into less speculative innovations too, just with fewer uncertainty multipliers and (somewhat) smaller error bars. Essentially, the variance in possible outcomes will still be too wide for conventional valuation methods to be effective.

As an R&D leader at a F500 company said to me recently, “We use NPV all the time, even though it’s virtually useless for our early-stage pipeline.”

Yikes.

Is there a better way to measure innovation?

If you’re going to tackle these issues, it’s important to think and act systematically. Measuring innovation is not a tactical or unidimensional problem; neither are the corresponding solutions.

Consider starting with your pipeline management processes and structures. Too many companies use the same metrics across all aspects of the innovation portfolio. This makes no sense, but companies do it anyway because all they have is a one-size-fits-all development process.

To do better, it’s important to create fundamentally different “swim lanes” for different categories of innovation risk and reward. By separating incremental innovations from breakthrough projects, you can apply different metrics to each swim lane, ensuring that KPIs are fit-to-purpose. This also makes it much easier to review the innovation portfolio’s degree of alignment with overall company strategy. Too many organizations drift away from their agreed-upon ambitions by accident – because they can’t readily stress-test whether they are still allocating innovation resources appropriately across horizon levels.

For Horizon-1 (incremental) efforts, it may be acceptable to continue using NPV as the go-to litmus test. For more speculative bets (Horizon-3), consider throwing out the typical corporate-finance logic altogether. For the most adventurous projects, it can be useful to adapt valuation methods from VCs and seed-stage investors – even if the “entrepreneurs” in your case happen to work on an internal team. For example, in a wide variety of instances, you can gain substantial predictive power by combining NPV with real options value analysis.

We’ve seen this work well in practice. The CTO of a F500 materials company, for one example, architected his global innovation portfolio with these principles in mind. At the front end of the pipeline, he employed a small pool of discretionary resources to incubate speculative bets. Once projects could be de-risked enough to make a more straightforward investment case, they would exit stealth mode to appear in pipeline reports shared with senior leadership. By combining real options analysis with more traditional financial metrics, the CTO was able to portray a much more accurate picture of the potential for value creation. This has led to more informed decision making and better productivity across the innovation portfolio.

This still leaves open a raft of related questions. If you’re using different metrics in different parts of the portfolio, how do you aggregate them into an overall assessment of success? How do you create performance incentives for staff whose projects span the portfolio from incremental to breakthrough? And how do you make sure that technologists and innovators can still collaborate effectively across swim lanes?

To be blunt, these questions should be eminently tractable. Unfortunately, for many organizations, they become roadblocks to progress.

It is becoming fashionable to flog Product Vitality and NPV as “the problem.” Yet in many cases, such complaints serve as an excuse to perpetuate the status quo. Effectively, leaders sweep under the rug the underlying structural issues that are slowly killing their innovation capacity.

Many of today’s leading companies – even large global corporations – still attempt to manage innovation pipelines through heavily customized versions of project-management or general-purpose collaboration systems. Some even resort to spreadsheets, email, and word-of-mouth (yes, really). The consequence is that core innovation performance data is either not captured at all, or it is spread among disparate information silos – with neither a centralized reporting engine, nor modern analytics capabilities.

On a portfolio-wide level, the lack of integrated data renders Product Vitality as the only realistic KPI, because more granular measures simply aren’t feasible. Furthermore, because they lack comprehensive data from their own systems, innovation leaders must rely on other sources, such as PPM or ERP systems. Often they must cobble together patchwork information from a range of sources, with varying degrees of trustworthiness.

On a project-by-project level, managers may have scant visibility into exactly what’s happening, an endemic weakness that gets rationalized as “letting the players play.” When it’s time for a funding decision, suddenly project visibility improves drastically to ensure a green light. After that, the information flow retreats into the shadows until the next milestone. This creates reporting holes, lack of accountability, and potential for gaming the system. It also prevents colleagues from pitching in or sharing expertise, because they don't really know what's going on.

Before you blame the metrics, take a hard look at a) how you organize the innovation pipeline, and b) how you track on-the-ground project activities. If you’re managing an undifferentiated mass of poorly documented and/or potentially redundant development efforts, then I’d say metrics are the least of your concerns.

If you’d like even more depth on next-gen innovation metrics, be sure to check out our recent webinar on Innovation KPIs: moving from reactive to predictive.

Photo by Lukas from Pexels

Share

LEARN WHAT WELLSPRING CAN DO FOR YOU

Let's Talk