Do you know your product team's win rate?

Ask a product team how the last quarter went and you will hear about velocity, about features shipped, about the roadmap landing on time. Ask which of those features actually moved the metric they were built to move, and the room usually goes quiet. Here’s what I think: if you don’t measure your product team’s win rate, you are shipping on faith, and a large share of the work may be moving nothing at all.

That matters because the cost is hidden. A feature that does nothing still has to be built, tested, supported, and maintained for years, and you keep paying for it long after the launch slide is forgotten.

The uncomfortable base rate

Start with what experimentation programs have repeatedly found. Across teams that test their ideas rigorously, the rough split lands near one-third helps, one-third does nothing, one-third actively hurts. That pattern shows up again and again in mature A/B testing programs (Ronny Kohavi, who ran experimentation at Microsoft and Bing, has reported numbers in this neighbourhood across thousands of tests). I won’t pretend the exact figure is the same in every product; it isn’t. The point is the order of magnitude. Most of what a team ships is not a clean win, and a real slice of it makes the product worse.

Sit with the second third for a moment. Roughly one in three ideas not only fails to help, it costs to build and costs again every month it stays in the codebase. The third that hurts is worse still: you spent the time, and you went backwards.

You can’t manage what you never counted

The interesting thing is how invisible all of this stays without a number. A team that ships ten features a quarter and feels productive has no way of knowing whether it shipped three wins or zero, because nobody is checking each idea against the metric it was supposed to move. Output gets measured (features out the door) and impact gets assumed. The gap between those two is exactly where the wasted thirds live.

Win rate is the number that closes the gap. Define it plainly: the share of shipped ideas that measurably moved their target metric. So if you shipped twelve changes this quarter, and four of them moved the metric you predicted by an amount you can actually see in the data, your win rate is four in twelve, a third. That single fraction tells you more about your product process than a full sprint board, because it is the one number that connects effort to outcome.

A number you can act on

Once you have it, the decisions change. A win rate gives you a baseline to improve against, and it tells you where to spend the next unit of effort. Say you measure honestly for two quarters and land at one win in five. Read that as a signal rather than a verdict: maybe the ideas are weak, maybe they are fine but the targeting is off, maybe you are shipping in batches too big to read. Each of those has a different fix, and you can only tell which one applies because you have a rate to move. Push it from one in five to one in three and you have lifted the impact of the same team by more than half, without hiring anyone or shipping faster.

This is also where experimentation earns its keep. Running an A/B test before a full rollout is how you catch the hurting third before it reaches everyone. The test is not bureaucracy; it is the measuring instrument that lets you keep score at all.

The honest objection

Now you might be thinking: not everything is A/B-testable, and heavy measurement kills team tempo. That is a fair objection, and it is partly right. A B2B tool with two hundred accounts cannot run a clean experiment on a button colour; you will never reach significance. A platform rebuild, a brand change, a bet that only pays off over a year: none of those fit neatly into a controlled test. And yes, a team that gates every change behind a measurement ritual will grind to a stop, and morale goes with it.

So I am not arguing that every change needs a randomised trial. I am arguing for keeping score. When a clean experiment is impossible, you can still write down the metric an idea was meant to move, ship it, and check the metric afterwards against a sensible baseline. That is a weaker read than a controlled test, and it still beats assuming. The teams that lose tempo are usually the ones measuring everything to the same heavy standard; the trick is to match the rigour to the stakes. Big, reversible, frequent changes get the full experiment. Rare, expensive bets get a clear prediction and an honest look afterwards. Either way, the question stays the same: did it move the thing it was supposed to move?

And the tempo worry cuts the other way too. Nothing kills momentum like a year of work that quietly moved nothing, discovered only when someone finally looks.

Where to start

So here is the first move, and you can do it this week. Take the last ten things your team shipped. For each one, write down the metric it was meant to move, then go and look at whether it moved. Count the wins. That fraction is your starting win rate, and the act of counting will tell you, fast, how much of your roadmap has been running on faith.

If you want to know whether the rate you land on is any good, that is what a benchmark is for: https://benchmark.scilla.studio.