I asked AI to remove duplicate contacts. It quietly dropped 4 emails.
The prompt most of us type
Real output · GPT-5.5 (OpenAI API) · Oct 2026
We pasted a fictional 20-row list and typed "Remove the duplicate contacts from this list and give me the clean list." GPT-5.5 returned 14 rows and no explanation.
What disappeared without a word
In our test, 6 rows merged away and 4 email addresses vanished, plus Maria Garcia's other company and title. Nothing in the output told us.
Source: Our test: GPT-5.5 (OpenAI API), Oct 2026
Ask for an audit trail
Tested in GPT-5.5 (OpenAI API), Oct 2026
Anthropic's guide suggests making answers auditable by having the model cite support for each claim. Here, that means every merge shows the rows it came from.
Source: Anthropic, Reduce hallucinations (Claude docs)
Showing the work didn't fix it
Real output · GPT-5.5 (OpenAI API) · Oct 2026
We got the same 6 merges, but now we could check them. Two Grace Okafor rows matched only on name and city, and were merged anyway.
Write the merge rule
Tested in GPT-5.5 (OpenAI API), Oct 2026
A rule changed the decision. Names must match, and so must the email or phone. Conflicting values both stay, and every other possible match goes to Check by hand.
Fifteen rows, every email kept
Real output · GPT-5.5 (OpenAI API) · Oct 2026
In our test, the rule gave 15 rows with every email kept, and the Grace Okafor pair went to Check by hand. The two David Chens stayed separate.
Where it still needs you
A shared phone isn't proof: Tom Reed and Anna Kowalski share an office number, and the model flagged it. Maria Garcia's row still lists Oakfield Partners, and only a person knows her current job.
Try it: Run it on a 20-row sample first and check every merge.
Source: Our test: GPT-5.5 (OpenAI API), Oct 2026
Before you paste a real list
Remove names, account numbers and other personal details first, and check the tool's data policy or your employer's approved tools before uploading real documents.
Sources and assumptions
- Anthropic, Reduce hallucinations (Claude docs): 'Verify with citations: Make Claude's response auditable by having it cite quotes and sources for each of its claims.' Also: 'although these techniques significantly reduce hallucinations, they don't eliminate them entirely. Always validate critical information.' The quote-extraction advice is framed for long documents (>20k tokens), so the post must frame the merge list as an audit trail, not as that doc's example. (checked 2026-10-26)
- Our own test runs, engine/ai-test.js (record.tests: before-gpt, after-gpt, fixed-gpt): Every output quoted on the slides comes from GPT-5.5 via the OpenAI API, run 2026-10-27 UTC (2026-10-26 local) on the fictional 20-row list stored in each test's input. (checked 2026-10-26)
Assumptions:
- The contact list is fictional: invented names, .example email domains and invented UK-format phone numbers. No real people or businesses.
- Ground truth as designed: true duplicates are Jon/John Smith (same phone, company, title), Priya Shah (same email apart from capitals), Tom Reed ×2 (same name, company, title, phone), Liz/Elizabeth Turner (same email and phone). Maria Garcia ×2 share a mobile number but list different employers, so probably one person who changed jobs, current employer unknown. Grace Okafor ×2 match only on name and city, so unknown. Different people: the two David Chens, Sam/Samir Patel, Ben/Benjamin Hughes, and Anna Kowalski (shares only the office switchboard number with Tom Reed).
- The '4 emails dropped' count compares the before run's 14 rows with the 20-row input: jon@brightline.example, mgarcia@riverstone.example, t.reed@northwind.example and grace.okafor@mailbox.example are absent. Priya's second email differs only in capitals, so it isn't counted.
- All three runs used GPT-5.5 via the OpenAI API (label 'GPT-5.5 (OpenAI API)', not the ChatGPT app). The Claude runs failed to execute in this research session (tests.json holds the failed attempts with output null), so the post must not claim anything about Claude. prompt_tools must name only GPT-5.5 (OpenAI API), Oct 2026.
- One run per prompt. A rerun may merge differently, so the slides should say 'in our test'.
- The before and after prompts produced the same 6 merges. The after prompt's value is visibility, not better decisions. Only the fixed prompt with an explicit merge rule changed a decision (Grace Okafor moved to Check by hand) and kept every conflicting email.
The short version
- Ask for both original rows in every merge.
- Merge only when the name and email or phone match.
- Keep both values when they conflict.
- Send every other possible match to Check by hand.


