AI's glossary explained 27 terms. My notes only explained 5.
The prompt most of us start with
Tested in GPT-5.5 (OpenAI API), Oct 2026
You join a project, paste in the kickoff notes and ask for a glossary. Ours were fictional Project Harbor notes, full of terms like CAB, ASN, RAG and SKU.
It sounds like the project wrote it
Real output · GPT-5.5 (OpenAI API) · Oct 2026
The notes never explain ASN or RAG, yet both got confident definitions. It also merged "steering group" and "SteerCo" into one row, which the notes never say.
Most of those rows weren't in the notes
Only 5 of the 27 rows are explained in the notes. The glossary gave every row a confident meaning and marked none of them as a guess.
Three rules change the job
Tested in GPT-5.5 (OpenAI API), Oct 2026
The fix is to use only the notes, copy the line each term comes from, and write "Not defined in the notes" when a term isn't explained.
Now the gaps are visible
Real output · GPT-5.5 (OpenAI API) · Oct 2026
12 of its 19 rows came back "Not defined in the notes," and the guesses sit in a separate list marked for the team to confirm.
The quotes held. One row still slipped.
All 11 quoted lines matched the notes word for word. But the Cutover row slipped in an inference, since the notes only define "Cutover weekend." Anthropic says these techniques reduce made-up answers but don't eliminate them.
Try it: Read the middle column, not just the quotes.
Source: Reduce hallucinations (Claude API docs, Anthropic)
Before you paste real notes
Remove names, account numbers and other personal details first, and check the tool's data policy or your employer's approved tools before uploading real documents.
Try it: Paste your own notes in and confirm each "general guess" with a teammate.
Sources and assumptions
- Reduce hallucinations (Claude API docs, Anthropic): 'External knowledge restriction: Explicitly instruct Claude to only use information from provided documents and not its general knowledge.' 'Verify with citations: Make Claude's response auditable by having it cite quotes and sources for each of its claims... If it can't find a quote, it must retract the claim.' 'Allow Claude to say "I don't know"'. The direct-quotes tip is framed 'For tasks involving long documents (>20k tokens)'. Note: 'although these techniques significantly reduce hallucinations, they don't eliminate them entirely.' (checked 2026-10-27)
Assumptions:
- The kickoff notes, Project Harbor, Northwind Supply, Priya and Ben are all fictional, written for this test.
- Both runs used GPT-5.5 through the OpenAI API (engine/ai-test.js, 2026-10-28 UTC, i.e. Oct 2026), not the ChatGPT app. The Claude CLI test runner failed in this research session. Two failed 'claude' entries with output null sit at the top of tests.json; ignore them, and never label any card as Claude.
- Anthropic's guidance is written for Claude, and its direct-quote tip targets long documents (>20k tokens). Our notes are short, and the test ran on GPT-5.5. The post cites the docs only for the general technique (use only the provided documents, back claims with quotes, allow 'not defined'), not as proof about GPT-5.5.
- 'Notes explained 5' counts WMS (warehouse management system), Cutover weekend, UAT, Hypercare and Pick face, the only terms the notes describe. 'Harbor' is the project name. '27 terms' is the number of rows in the before table.
- '12 of 19 rows' counts the after table's rows marked 'Not defined in the notes' (Leeds DC, IT, CAB, ASN feed, RAG status, steering group, amber, go-live, SKU rationalisation, phase 1, Dock-to-stock, SteerCo).
- The quote check was done by hand against the notes: all 11 distinct quoted lines in the after table match word for word, apart from the leading '- ' bullet.
- The general-knowledge meanings (e.g. CAB = Change Advisory Board) are the model's guesses. The post doesn't claim they're right or wrong for this fictional project; they're the things to confirm with the team. The two runs even spelled ASN differently: 'Advanced Shipping Notice' (before) and 'advance shipping notice' (after).
- Privacy line for the slides: remove names, client and account details before pasting real project notes, and check the tool's data policy or your employer's approved tools first.
The short version
- A plain "glossary" prompt blends guesses with the notes
- Ask for a quote beside every term
- Have it write "Not defined in the notes"
- Keep its guesses in a separate list
- Read the middle column too


