Do you know your product team's win rate?

Ask a product team how the last quarter went and you will hear about velocity and features shipped. Ask which features moved the metric they were built to move, and the room usually goes quiet. If you don’t measure your product team’s win rate, you are shipping on faith. A large share of the work may be moving nothing.

The cost is hidden. A feature that does nothing still has to be built, tested, supported and maintained for years.

The uncomfortable base rate

Ronny Kohavi ran experimentation at Microsoft and Bing. Across thousands of tests he has reported a rough split: about one-third of shipped ideas help, one-third do nothing, one-third actively hurt.

I won’t pretend the exact figure is the same in every product; it isn’t. The order of magnitude holds.

The third that hurts is worse. You spent the time and went backwards.

You can’t manage what you never counted

Without a number, all of this stays invisible. A team that ships ten features a quarter feels productive. It has no way of knowing whether it shipped three wins or zero, because nobody checks each idea against the metric it was meant to move.

Output gets measured (features out the door) and impact gets assumed. The wasted thirds live in that gap.

Win rate closes the gap. It is the share of shipped ideas that measurably moved their target metric.

Say you shipped twelve changes this quarter. Four of them moved the metric you predicted, by an amount you can see in the data. Your win rate is four in twelve, a third.

A number you can act on

A win rate gives you a baseline to improve against and tells you where to spend the next unit of effort.

Say you measure honestly for two quarters and land at one win in five. Read that as a signal rather than a verdict.

Maybe the ideas are weak. Maybe they are fine but the targeting is off. Maybe you are shipping in batches too big to read. Each has a different fix, and you can only tell which one applies because you have a rate to move.

Push it from one in five to one in three and you have lifted the same team’s impact by more than half.

An A/B test before a full rollout catches the hurting third before it reaches everyone.

What about the stuff you can’t A/B-test?

Not everything is A/B-testable, and heavy measurement does kill team tempo. A B2B tool with two hundred accounts cannot run a clean experiment on a button colour; you will never reach significance.

A platform rebuild, a brand change, a bet that only pays off over a year: none of those fit neatly into a controlled test. And yes, a team that gates every change behind a measurement ritual will grind to a stop.

I’m arguing for keeping score, not for a randomised trial on every change. When a clean experiment is impossible, write down the metric an idea was meant to move. Ship it. Check the metric afterwards against a sensible baseline. That is a weaker read than a controlled test, and it still beats assuming.

Match the rigour to the stakes. Big, reversible, frequent changes get the full experiment. Rare, expensive bets get a clear prediction and an honest look afterwards.

Where to start

Start this week. Take the last ten things your team shipped. For each one, write down the metric it was meant to move, then look at whether it moved. Count the wins. That fraction is your starting win rate.

If you want to know whether the rate you land on is any good, that is what a benchmark is for: https://benchmark.scilla.studio.