I'm ramping up on a project to understand how Firefox retains users. Right now I'm trying to build some context quickly. For example, what's our monthly retention? How about our annual retention? There's a bunch of interesting and nuanced measurement questions that we'll eventually have to answer, but for now I'm just interested in getting some quick back-of-the-envelope numbers.
There's a conflict here though. I get a lot of value from having loose and squishy estimates. But, if I share these numbers the results often get passed around via word-of-mouth and the numbers start to look more solid than they are (a second cousin to citogenesis).
It's easier to express uncertainty in person than it is in writing. In person, I can shrug my shoulders and rock my hand back and forth signaling, "Kinda, sorta, maybe...". In writing, caveats take up a lot of space and often confuse muddy my point.
Instead, I've started describing these soft-and-squishy numbers as "pub true" or "true enough". Basically, the idea is that they'd hold up in a bar conversation but they're not meant to be used for much else. It gets you into the ballpark but you probably shouldn't bet the business on them.
I'm happy with the results so far. Folks seem to register that we're having a casual discussion of the numbers. It feels like "pub true" status gets passed along with the results. It's a nice and succinct description.
An Order of Magnitude
In particular, I use I use "pub true" numbers to unblock some early stage conversations. When I'm starting a new project conversations are often stilted by trying to be too accurate. Folks are reticent to share numbers if they don't have exact numbers.
For example, I might ask someone "How many releases do we do a year?". In a formal environment, they might think for a bit and come up with "I don't know" even though I know they have more context than I do.
In their head, my peer's thinking, "We usually release monthly but we had some bug-fix releases last year, so it could be as many as 15." The problem is, I know next to nothing so any new context is useful.
I find that re-framing the conversation to something casual gets better results. If I throw out some ridiculous numbers I can usually get a reasonable estimate back. For example, I might say "So like 100 releases or 50?". Pretty often folks come back with a quick and informal, "Oh, no - nothing like that. Maybe, like 12. Definitely no more than 20."
Great. Pub true. Hand wavy, quick and effective.