.jpeg)
The test was deliberately unglamorous. Over five weeks, I put ChatGPT through a series of ordinary travel queries that any business traveller might ask: late check-out rules at a Marriott in Lagos, cancellation fees at a boutique hotel in Lisbon, whether a specific Hilton allowed pets, early check-in policies at a Tokyo business hotel, and how a loyalty points transfer worked between two chains.
 |
| ai-hallucination-hotel-booking-audit-summary.jpg |
The questions were chosen because their answers live in one place only — the hotel's own terms and conditions. This is precisely the category of query where a language model has no reliable training signal, yet must still produce an answer.
In four of the five cases, it did produce an answer. It was wrong every time.
The falsehoods were not vague. In the Lagos scenario, ChatGPT stated a specific late check-out window, attached a fee structure in naira, and cited a "Marriott Bonvoy Elite member exemption" that does not exist. In the Tokyo case, it invented an "international early check-in programme" with a plausible-sounding registration link. Each fabrication came wrapped in the same calm, authoritative tone the model uses when it is correct.
 |
| marriott-tokyo-hotel-check-in-rules-test.jpg |
The detection method was disarmingly simple: every claim was checked against the hotel's official policy page. This is worth stating plainly, because it is the entire lesson.
The first red flag was specificity without sourcing. Genuine hotel policies are oddly worded, hedged, and full of exceptions ("subject to availability," "at participating properties"). The fabricated policies were too clean no caveats, no variance. Real corporate language is messy; invented language is polished.
The second flag was internal inconsistency across sessions. When the same question was asked three days apart, ChatGPT gave three different cancellation fee schedules for the same Lisbon hotel. A true policy does not drift. A hallucinated one does, because each generation samples from probability, not from a document.
 |
| verifying-hotel-terms-and-conditions-policy.jpg |
The third flag and the most commercially dangerous was invented citation. Rather than say "I don't know," the model frequently attached a reference to a section of the hotel's website that did not exist. This echoes the wider industry pattern: on the Columbia Journalism Review's citation benchmark, ChatGPT hallucinated source attributions in 67% of responses, while even Perplexity the best performer — fabricated over one in three citations.
 |
| air-canada-legal-precedent-ai-liability.jpg |
A fabricated hotel policy is not an amusing glitch. It is a financial liability with a paper trail. The Moffatt v. Air Canada ruling established that companies cannot disown the false promises their AI tools make: the British Columbia Civil Resolution Tribunal ordered the airline to pay $812.02 CAD in damages and fees after its chatbot invented a retroactive bereavement refund.
The tribunal member's words that it makes "no difference whether the information comes from a static page or a chatbot" are now the operating legal reality for any business deploying conversational AI.
The economics compound the risk. The Vectara HHEM leaderboard reports hallucination rates of 1.4% for GPT-5 and 1.5% for GPT-4o in summarisation tasks — figures that sound reassuring until they are applied to a chatbot handling a million customer queries, which translates to roughly 14,000 to 15,000 fabricated responses before a single human intervenes.
 |
| vectara-gpt4o-hallucination-rate-table-data.jpg |
| Metric | Figure | Context |
|---|---|---|
| Fabricated hotel policies in test | 4 of 5 scenarios (80%) | Author's structured test, Sept 2026 |
| ChatGPT citation hallucination rate | 67% | Columbia Journalism Review benchmark |
| GPT-4o summarisation hallucination rate | 1.5% | Vectara HHEM leaderboard (vs. 0.7% for Gemini 2.0 Flash) |
| Average verification time per claim | ~90 seconds | Manual check against official hotel policy pages |
| Legal precedent cost | $812.02 CAD | Moffatt v. Air Canada, damages awarded to a customer misled by an AI-invented policy |
The travel and hospitality sector is deploying AI concierges faster than it is building controls for them. The pattern identified in this test fluent fabrication, invented citations, session-to-session drift is a solvable engineering problem, but only for organisations willing to treat verification as a feature, not an afterthought.
Until retrieval-grounded systems become the norm rather than the exception, the cheapest quality-assurance tool in business remains a human being with a browser and 90 seconds. The question for every hospitality executive deploying an AI concierge is not whether their model can answer every question it is whether they have built the discipline to verify the ones it can't.
 |
|
No comments:
Post a Comment