js-d.xyz

Unbounded Retention

I am a stickler for metrics.

The devil is in the detail so much. If the definitions are wrong, or misleading, not only are they not helpful, they are actively unhelpful. Whenever I get presented wrong or misleading metrics in a start-up pitch, it frustrates me. Either it's an honest mistake, or a deliberate attempt to pull the wool over investors' eyes. Either way, it's really not good enough if you're asking people for money.

Unbounded retention is a particular bugbear.

Put simply, unbounded retention shows the percentage of a user cohort that engages with a product at time X or any time after time X. For that reason, it's sometimes called "on or after" retention. It means I could open an app on day 1, and day 185, and for all the days 2-184 I would be showing as a retained user. By contrast, bounded retention shows the percentage of a cohort that engages with a product on exactly time X. I would only show up on a bounded basis on day 1 retention and day 185 retention in the above example.

For the vast majority of consumer apps, unbounded retention is a pretty meaningless metric for reasons that should be obvious. What we're looking for with consumer apps is sticky habit formation, but unbounded retention doesn't show that at all. On an unbounded basis, the user that engages every day between day 1 and day 185 is indistinct from the user that engages only on day 1 and day 185. On a bounded basis, the difference would be unmissable.

There are some circumstances where bounded retention doesn't make sense: for example where the product is a sporadic rather than habitual use case, like travel. But in these scenarios, rather than defaulting to unbounded retention I think there are more meaningful metrics to look at to try to understand stickiness and customer loyalty (e.g. 6/12/24 month repeat rates).

In fact, I'd go as far as to say I'm unsure if there is any scenario where unbounded retention is a high-information metric. It might feel easier, because it's much more forgiving, but don't kid yourself!

#analytics #metrics #product #startups #venture capital