Every NPS programme produces two artefacts: a number, and a column of text nobody has time to read. The number goes in the board deck. The text — the part that actually says why — sits in a spreadsheet until the next survey overwrites it.
This is backwards, and most teams know it. A score of 34 tells you nothing you can act on: not what to fix, not for whom, not whether it’s the same problem as last quarter. The verbatims contain all of that. Here’s how to get it out of them.
Why the score alone can’t guide a decision
NPS compresses a rich opinion into one integer and then subtracts integers from each other. Three specific things get destroyed in that compression:
- The reason. Two 6s can mean “your product is fine but expensive” and “your support ghosted me for a week.” Identical scores, opposite roadmaps.
- The direction. A score that moved from 31 to 34 might be the same customers slightly happier, or a wave of new promoters masking a churning cohort. The number can’t tell you which.
- The size of the fix. “Detractors dislike onboarding” could be four people or forty percent. Without theme counts the score gives you no way to weigh anything.
The mistake almost everyone makes: analysing all comments together
The instinct is to throw every verbatim into one pile and find the top themes. Don’t. Segment by score band first, then theme within each band. The reason is a genuinely counter-intuitive property of feedback data:
Detractor themes are almost never the inverse of promoter themes. Promoters rave about what your product uniquely does — the thing that made them choose you. Detractors complain about hygiene: reliability, support latency, price, a broken import. Nobody writes “I would recommend you if only your killer feature were better.” They write “your killer feature is great, but it went down twice last month.”
This has a direct operational consequence: improving what promoters love rarely converts detractors. The two groups are telling you about different layers of the product, and analysing them as one pile blends the strategy signal into the maintenance signal until neither is legible.
| Band | What they mostly tell you | What to do with it |
|---|---|---|
| Promoters (9–10) | Your actual differentiator, in customer language | Marketing copy, positioning, what never to break |
| Passives (7–8) | The gap between “fine” and “worth recommending” | The highest-leverage roadmap input, and the most ignored |
| Detractors (0–6) | Hygiene failures: reliability, support, price, a specific broken flow | Fix list, and churn-risk outreach |
If you only have time for one band, read the passives. Promoters will stay a while and detractors are often already leaving; passives are the population whose mind can still be changed by something you ship this quarter.
A method that fits in an afternoon
For a few hundred responses, this works without any tooling beyond a spreadsheet:
- 1. Split by band into three tabs. Do not merge them again until the very end.
- 2. Read 30 from each band and write the theme list — not before. A taxonomy invented in advance will bend the data to fit it; one derived from a sample fits the data you actually have.
- 3. Code every response against that fixed list, allowing up to two themes each. Add a theme only when three responses genuinely don’t fit anything.
- 4. Count distinct respondents per theme per band, and — if you can join it — the revenue behind each theme.
- 5. Compare to last wave using the same theme list. This is the whole reason step 3 insisted on a fixed list.
That last step is where manual NPS analysis usually dies. Each survey wave gets coded by whoever has time, with a fresh taxonomy, so wave-over-wave comparison is impossible and the programme quietly degrades into score-watching. If you do nothing else differently: keep the theme list, in a file, and reuse it.
Three biases to state out loud in the report
- Response bias. Typical NPS response rates run in the low double digits at best. The people who answer are disproportionately the delighted and the furious — the same extremes problem app store reviews have.
- Comment bias within responders. Only a fraction of scorers write anything, and they skew toward the strongest opinions. Your verbatim themes describe the vocal subset of a self-selected subset.
- Timing bias. A survey sent after a successful action measures that moment, not the relationship. Two identical products can post very different scores based purely on when the prompt fires.
None of these invalidate the exercise — they just mean the honest framing is “among users who responded, the top themes were X,” not “our users think X.” Stating it also protects the analysis from the executive who wants one number to be the truth.
When to automate this
The economics change when NPS stops being quarterly. If you run continuous or transactional surveys, manual coding cannot keep up and the wave-over-wave comparison — the entire value — is the first thing to break. That’s the point to move to a pipeline that labels each response for sentiment, type and aspect, clusters verbatims by meaning so the taxonomy stays stable by construction, and keeps the score band attached so you can still slice promoter versus detractor themes.
In Sentriment that shape maps directly: survey responses arrive by CSV or API, the score band rides along as a property, themes form automatically and stay comparable across waves, and each theme carries the revenue of the accounts behind it. Asking “what do detractors say about pricing?” returns a verdict with counts and the actual quotes — which is, in the end, all an NPS programme was ever supposed to produce.
Related reading
- How to analyze customer feedback: a method that survives contact with reality — the general version of this, across every channel.