Describe, never compute
A language model writes fluent sentences, and fluent sentences are easy to trust. So when an agent's summary carries numbers and names, let the application compute them and let the model only describe what it was given. A checker then drops any sentence that goes further.
Watch the checker read the agent's summary
The agent has just worked incident INC-4873 (Data exfiltration through the chat tool) in the simulator. The application gave it the facts; the agent wrote 6 sentences.
The agent's summary
What the application counted
Every sentence and count as a list
- Kept: INC-4873 fired on profile-api in prod.
- Kept: 5 of the 10 steps ran on their own.
- Kept: The other outcomes together: 2 step(s).
- Dropped: In total, 8 steps ran or were drafted. · Own arithmetic: 8 = 5 + 3. The application computes; the model only describes.
- Dropped: About 250 customer records were in the dump. · Invented figure: 250 is not among the facts the agent was given.
- Dropped: The heap dump was stored in
dumps-2026-storage. · Unseen identifier: dumps-2026-storage was never shown to the agent.
The application's counts: ran on their own 5; ran after a human approved them 3; all other outcomes 2; total 10.
What you just watched
The agent finished an incident and wrote a summary. Before anyone reads it, the checker compares every sentence with the facts the application gave the model, and drops a sentence for one of three reasons:
- An invented figure: a number that is not among the facts, such as "about 1,200 customers" when nobody counted customers.
- Its own arithmetic: a number the model worked out by adding, subtracting, multiplying or dividing the facts, or words such as "in total", "combined" or "twice". The application does the sums; the model only reports them.
- An unseen identifier: a host, service, ticket or resource name the model was never shown. A wrong name in a summary is how the next fix goes to the wrong system.
Why drop the sentence instead of fixing it
The checker never corrects a sentence, because it cannot know what the model meant. A dropped sentence leaves a gap you can see; a quietly corrected one hides the mistake. The rest of the summary still goes out, and every dropped sentence is listed with its reason, so you can fix the prompt or give the model the fact it was missing.
App-computed counts with an "all other" bucket
Models are poor at counting long lists. So the application counts, and hands the model a short table: the largest categories, then one "all other" bucket for the rest. The parts always add up to the total, so the model has no reason to add anything up, and a reader can check the sum at a glance.
- The application groups and counts; the model never sees the raw list.
- Keep the top few categories and put the rest in "all other", so the table stays short and complete.
- Give the total as well. Then every number the model may need is already written down.
Where it fails
The checker is a safety net, not a proof. It reads figures written with digits, so a number written in words ("three") slips through: ask the model to write figures as digits. And it cannot tell whether a sentence that uses the right numbers draws the right conclusion. A person still reads the summary before it reaches a customer.
Try it in the simulator
Switch on "Describe, never compute" when you set up an incident, run it, and read the agent's summary: the sentences that went out, and every dropped sentence with its reason. Switch it off to see what the agent would have told you.
Let the application compute; let the model describe.
Check your understanding
4 questions. Pick an answer for each, then check. Nothing leaves your browser session.
1. The facts say 7.4% errors against an SLO of 0.5%. The model writes "The error rate was 14.8 times the SLO." What does the checker do?
2. Why does the table of counts end with an "all other" bucket?
3. The summary names payments-db-02, which appears nowhere in the facts. What happens?
4. Which job belongs to the model?