The Missing Data
The story of Abraham Wald and the WW2 bombers is the perfect metaphor for modern Product Management. The Generals made a logical error: They assumed the data they had was the entire data. They forgot about the data they didn’t have: The planes lying at the bottom of the ocean.
In SaaS, your “Ocean” is your Churn Rate.
The “Power User” Trap
We love our Power Users. They answer our surveys. They join our beta programs. They rave about us on Twitter. So, we build features for them.
- “Add more advanced keyboard shortcuts!”
- “Give us a dark mode!”
- “We need a deeper API integration!”
You build these things. The Power Users are happy. But your growth stalls. Why?
Because the users who quit on Day 1 didn’t care about keyboard shortcuts. They quit because the font was too small to read, or the sign-up flow was broken, or the value proposition was unclear. The Power Users survived those issues (maybe they had big monitors or more patience). The Churned Users did not.
By listening only to the survivors, you are Armoring the Wings. You are making the product better for people who already love it, while ignoring the fatal flaws that drive everyone else away.
How to Find the “Missing Planes”
To fix this, you have to invert your research strategy.
1. The “Exit Interview” is more valuable than the “NPS Survey” NPS surveys are usually filled out by active users. Exit surveys are filled out by the dead planes. When a user clicks “Cancel Subscription,” that is the single most important moment to capture data. Don’t make it a generic dropdown. Ask: “What is the one thing that, if fixed, would have made you stay?”
2. Watch the “Bounce” Session Replays Use tools like Hotjar or Microsoft Clarity. Don’t watch the sessions of users who successfully checked out. Watch the sessions of users who stayed for 10 seconds and closed the tab. Where did they hover? What did they click that didn’t work? That is your engine failure.
3. Beware of “Feature Requests” Feature requests come from people who are still using the product. They want moreutility. People who left usually didn’t need more features; they needed the existing features to be easier. Survivorship bias pushes products toward Complexity. Growth requires Simplicity.
Conclusion
It feels good to talk to happy customers. It validates our ego. But growth comes from studying the unhappy ones. Don’t look at the bullet holes on the wings. Look for the empty spaces. That’s where the lethal blows are landing.