When a chatbot made a promise the dealership couldn't keep

What happened

A customer at a Toronto BMW dealership asked the dealership's chatbot about a buyback programme. The chatbot generated a commitment, the customer relied on it, and the dealership later said it couldn't honour the offer ^[1].

The incident, recorded in the AI Incident Database as number 1772, doesn't say whether the dealership had its own record of what the chatbot said. What it does say is that the organisation was left unable to stand by the AI's commitment ^[1].

It's like leaving your teenager in charge of the shop: whatever they promise the customers, you're on the hook for it, even if you never saw the conversation.

The accountability gap

When a chatbot makes a commercial offer, two problems arise. The first is whether the organisation is bound by it. The second is proving what the chatbot actually said.

In a 2024 case in British Columbia, an airline's chatbot gave a customer the wrong answer about bereavement fares. A tribunal ordered the airline to compensate him, and said in effect that the airline is responsible for everything on its website, whether it comes from a static page or a chatbot ^[2].

The evidence in that case was a screenshot. The customer had one, the airline offered nothing to the contrary, and the screenshot became the record ^[2].

The Toronto dealership case doesn't say how the commitment was documented, but the pattern is familiar: the customer has a screenshot or a memory, and the organisation has nothing to show what the AI actually said.

What the rules say

Article 50 of the AI Act requires that people be told when they are interacting with an AI system, unless it is obvious from the circumstances ^[3]. That rule applies from 2 August 2026 ^[3].

The notice itself doesn't prevent a chatbot from making a promise. It just makes it clear that the person is talking to a machine, not a human being.

The harder question is what happens when the promise is disputed. If the organisation has no audit trail, the customer's screenshot is often the only record, and that record tends to favour the customer.

Under GDPR Article 22, people have the right not to be subject to a decision based solely on automated processing, including profiling, that has legal effects or similarly significantly affects them ^[4]. Exceptions apply when the decision is necessary for a contract, permitted by law or based on explicit consent, and in those cases the person must at least have the right to human intervention, to express their view and to contest the decision ^[4].

That doesn't mean every chatbot interaction is a decision under Article 22, but it does mean that when a chatbot makes a commitment that significantly affects someone, the organisation needs to be able to show what happened.

What this means for you

If your organisation runs a customer-facing chatbot, three questions are worth asking now:

• Do we have our own record of what the chatbot says, or only what the customer chooses to show us?
• Can we show which version of the model was running, and what instructions it had, on a given day?
• If a commitment turned out to be wrong, could we show what the chatbot actually said, and when?

The answers don't have to be perfect, but they do need to exist before the first dispute.

How a digital witness would work here

The thing to seal is each answer as the chatbot sends it: the customer's question, the chatbot's reply, the time and the model version.

Sealing takes a fingerprint of the answer, not the answer itself, so the conversation never leaves your systems. The fingerprint goes to WitniumChain, and you get a witness ID back. Store the ID with the conversation.

Months later, if someone questions what the chatbot said, you share the record and the witness ID. Anyone you share it with can check the fingerprint against the sealed one. If they match, nothing has changed since the conversation happened. Change a single character in the record and it won't match.

That check can be done by the customer, an ombudsman or a court.

What it shows is that the record of what the chatbot said, and when, hasn't changed since it was sealed. What it doesn't show is that the answer was correct, or settle whether the company is bound by it. Those are separate questions, and they start from having a record both sides can trust.

You can seal answers through the REST API, the SDK or the MCP server. The content stays with you. The first 2,000 witnesses a month are free. Read the docs.

How it would work

1. The chatbot sends an answer and your system seals a fingerprint of it: question, reply, time and model version.
2. You store the witness ID with the conversation.
3. If the answer is questioned, you share the record and the ID. Anyone can check that the fingerprint still matches.

What to watch

The AI Act's transparency rule in Article 50 applies from 2 August 2026 ^[3]. By then, people must be told when they're talking to an AI, unless it's obvious.

That date is worth marking, not because it solves the evidence question, but because it makes the conversation explicit. After that, every chatbot interaction is on the record as an AI interaction, and the question of who has the record becomes harder to avoid.