I asked two AIs to answer an angry customer, and both offered a full refund I never approved.
What the Reel says
I asked two AIs to answer an angry customer, and both offered a full refund I never approved.
My whole prompt was "Write a reply to this customer complaint."
Both added expedited reshipping too.
But neither knew my business, so each offered what good service usually offers.
Once sent, that reads as a promise. A tribunal once made Air Canada pay after its chatbot described a refund that didn't exist.
To fix this, I listed what went wrong, what I'd offer and what I couldn't promise, like a delivery date.
15%
refund I approved
But GPT-5.5 still hinted at speed: "the fastest way we can." Read every draft first.
Tell the AI what you can't give, and it stops giving it away.
Sources and assumptions
- Our test runs: Claude Sonnet 5.5 and GPT-5.5 (OpenAI API), fictional complaint from 'Priya Shah', order #4417, shop 'Fern & Thread': The before/after outputs quoted in record.tests: the vague prompt led both models to offer a full refund and free expedited reshipping or a replacement, and the template prompt kept both to free express shipping, a 15% refund and no Saturday promise (checked 2026-11-11)
- Anthropic, Prompting best practices (Claude docs; the old prompt-templates-and-variables URL redirects here): "Think of Claude as a brilliant but new employee who lacks context on your norms and workflows. The more precisely you explain what you want, the better the result." Also: "Providing context or motivation behind your instructions... can give Claude more information about your goals and help it deliver more targeted responses." The page also mentions prompts that mix instructions, context, examples and variable inputs. It does not present a separate templates-and-variables guide, so the post does not attribute the template to Anthropic. (checked 2026-11-11)
- CBS News: Air Canada chatbot costs airline discount it wrongly offered customer: The chatbot told Jake Moffatt that Air Canada would partially refund his ticket under its bereavement policy if he applied within 90 days. The tribunal: "It should be obvious to Air Canada that it is responsible for all the information on its website. It makes no difference whether the information comes from a static page or a chatbot." Air Canada was ordered to pay $812 CAD in damages and tribunal fees. (checked 2026-11-11)
- Air Canada chatbot ruling: search-result summaries (Daily Hive, Times Colonist, American Bar Association Business Law Today): The British Columbia Civil Resolution Tribunal decided in February 2024 that Air Canada "did not take reasonable care to ensure its chatbot was accurate". Sources give the date as Feb 14 (decision) or Feb 18, 2024, so the post says only "February 2024". This page was seen in search results but not opened; the claim on screen rests on the CBS article. (checked 2026-11-11)
Assumptions:
- The complaint, customer (Priya Shah), order #4417, products and shop (Fern & Thread, signer Sam) are fictional.
- Each prompt was run once per model on 2026-11-11 (API timestamps 2026-11-12 UTC). Outputs can vary from run to run, so the post shows what happened in this test and doesn't promise it every time.
- GPT-5.5 was run through the OpenAI API, not the ChatGPT app. Claude was Claude Sonnet 5.5.
- The Air Canada case was a customer-facing chatbot, not an AI-drafted email that a person sent. It is used only to show that businesses can be held to what their AI tells customers, not as a ruling about emails.


