Summaries keep dropping the one number that matters

Started 6d ago 4 replies 744 views

6d ago

I summarise weekly operations reports for a hospital network. No patient data, just things like bed occupancy, discharge times and staffing gaps. The reports run 6 to 10 pages.

The summaries read beautifully. The problem: about one time in five, they drop the single most important figure. Last week the summary said "occupancy remained high" but left out that one ward hit 104%, which was the whole story.

Current prompt, roughly:

Summarise this report for senior managers in 200 words.
Focus on the most important changes.

What am I missing? I don't want to make it longer; I just want the right numbers to survive.

4 replies

Accepted answer
Mei-Lin Zhao
6d ago
"Most important" is doing too much work. The model's idea of important is "mentioned often", not "operationally alarming". Two fixes that work well together:
1. A must-carry list. Tell it which metrics must always appear with their exact values, e.g. any occupancy over 95%, any discharge delay over 4 hours.
2. A numbers check at the end. Ask for a small table, metric | value in report | value in summary, after the summary. Missing or altered numbers become visible in seconds.
```
After the summary, add a table listing every metric from the
must-carry list, its value in the report, and its value in your
summary. Write MISSING if it isn't in the summary.
```
Lena Hartmann
6d ago
One more reason it happens: if the 104% figure sits in the middle of a long report, it's in the zone models attend to least reliably. Put the must-carry list both before and after the report in your prompt. Repeating instructions at the end is cheap and helps a lot. More in The Context Window Is Not a Memory.
Fatima Benali
5d ago
The Source-Checked Summary prompt in the library pairs nicely with this: every claim gets a short quote from the source in brackets. It makes summaries a bit longer, but you can strip the quotes once you trust it.
Arjun PillaiOP
4d ago
Results: must-carry list plus the numbers table, with the list repeated after the report. Twenty test reports, zero dropped figures, and the table caught one rounding change (97.6% became 98%) that I'd never have noticed. Thank you.

Join the conversation