Common Trap: Trailing Measures

Companies tend to use metrics that are inappropriate for measuring their product development pipeline. The appeal of measuring something tangible, convenient, and safe (even if inappropriate) usually overrides (for performance review reasons) the need to implement the right measures. Unfortunately, in many cases, being unaware of existing proper measures can drive decision making.

One of the most typical mistakes is measuring the number of patents applied for each year. The underlying assumption is that the higher the number, the more innovative the company is. With this as the individual performance measure, employees start applying for as many patents as they can, at a typical cost of $7,000-$10,000 per patent.

The issue with this approach is that only a few of those patents may prove crucial, as patents have difficult-to-predict hit rates. In the worst case, perhaps none of patents can be monetized in the form of an innovative product. This information will not be available until years into the future.

A better way is to measure the percentage of revenues that are coming from products introduced in the last five years. While this method is a much better predictor of performance, it suffers from a time lag: five years of waiting.

The common flaw in these two measures is that they are both trailing measures. That means that you will not know until it is too late whether you are doing better or worse.

Implementing a Leading Measure

Implementing a 100% pure leading measure is simply not possible, as it is not possible to predict the future. However, it is possible to do significantly better than the simple trailing measures mentioned above.

At steady-state, a company can get a pretty good idea of the hit rate of its breakthrough projects by looking at historical hit rates. Similarly, the historical incremental innovation hit rate can be established. In our experience, these numbers are as follows:

• Front End hit rate: 30%
• Near Term hit rate: 50%
• Incremental Innovation hit rate: 80%

Similarly, the size of the hit, in terms of the present value of the aggregate of future revenues, can also be predicted using historical data. Then, in order to measure the size of the innovation pipeline, one only has to aggregate the potential revenues from all of the projects, and, using the proper multiplier, figure out the potential return.

An Example

Number of Front End projects: 30
Add all of the potential revenues: $15M+$250M+ .. +$75M = $800M

Number of Near Term projects: 10
Add all of the potential revenues: $35M+$50M+ .. +$130M = $300M

Number of incremental innovation projects: 6
Add all of the potential revenues: $50M+$80M+ .. +$700M = $150M

($800M × 30%) + ($300M ×50%) + ($150M × 80%) = $510M

That is the current size of the innovation pipeline. Once this is established, then annual growth goals are implemented for individuals, departments, and the company leadership. However, in order to prevent potential gaming of the system, historical percentages need to be monitored and updated often.

One of the major pitfalls of this approach is the urge to improve the hit rate. Counterintuitively, this often results in less innovation as people start making safer and safer choices.

Conclusion

Venture Capital (VC) organizations typically have a 30-40% hit rate. However, when they hit the jackpot, the hits not only cover the misses, but also pay back exponentially. A lot of innovative ideas came from VC innovation pipelines that are managed as documented here. AOL, Netscape, Google, and Amazon are just a few examples of their hits.