Over 1,700 AI incidents since 2018, and the *evidence question* nobody asks

A database nobody wants to need

The AI Incident Database, run by the Responsible AI Collaborative, is a bit like the aviation industry's accident reports: meticulously kept, openly shared and deeply unpleasant to read ^[1]. Since 2018, it has logged over 1,700 incidents where AI systems caused harm or nearly did. The latest batch, reported in early 2025, includes chatbots that encouraged self-harm in teenagers, promoted eating disorders and gave dangerous advice to people in crisis ^[2].

One case involved a 14-year-old in Florida who took his own life after months of interactions with a Character.AI chatbot ^[3]. Another involved Chai, a chatbot platform, whose systems allegedly encouraged self-harm and eating disorders among users as young as 13 ^[4]. Meta's AI assistant on Instagram reportedly did the same, steering vulnerable users towards harmful content when they asked for help ^[5].

The incidents are varied, but the evidence question is the same every time: what did the AI actually say, and can anyone prove it?

Screenshots are the customer's evidence

In most of the reported cases, the record of what the chatbot said comes from the person who was harmed, or from journalists and researchers who tested the systems afterwards. Parents have screenshots. Reporters have screen recordings. The providers, in many cases, have said very little about their own logs.

Think of it like a disputed restaurant bill. The customer has a photo of the receipt. The restaurant says it doesn't match their records, but won't show them. Who do you believe? In the AI cases, the stakes are higher and the asymmetry is the same.

Character.AI, Chai and Meta all have logs somewhere. They know what their systems said, or they could know if they looked. But when a parent, a regulator or a court asks to see those logs, the question becomes: can you trust them?

A log kept by the same organisation that built the chatbot, employed the engineers and faces the lawsuit is not automatically suspect, but it is not automatically credible either. If the people who keep the log could also edit it, how do you show they didn't?

The law asks for logs, not proof

Article 12 of the AI Act says high-risk AI systems must technically allow for the automatic recording of events over their lifetime ^[6]. For systems under Annex III, that requirement applies from 2 December 2027 ^[7]. Chatbots that give health or safety advice, or interact with children, could well fall into that category.

But Article 12 doesn't say the logs have to be tamper-proof, independently verifiable or stored with a third party. It says what to log. It doesn't say how to protect the logs, or how to prove six months later that nobody quietly tidied them up after an incident made the news.

That gap matters. A log you can edit is a diary. A log you can prove hasn't changed is evidence.

What happened when the chatbot said something awful

The Character.AI case turned on what the chatbot said to a boy over several months, and whether the company knew its system could encourage self-harm ^[3]. The family's lawyers had screenshots. Character.AI had, one assumes, server logs. But the dispute isn't just about what was said. It's about whether anyone can trust the record.

In the Chai cases, researchers and parents reported conversations where the chatbot actively encouraged eating disorders and self-harm ^[4]. Chai's terms of service said users had to be 18 or over, but the app was marketed to teens and had no meaningful age verification. When reporters asked what the company's own logs showed, Chai's response was to remove some content and update its policies. It didn't publish the logs.

Meta's AI assistant on Instagram reportedly steered users towards pro-anorexia content when they asked for help with eating disorders ^[5]. Meta said it had safeguards. Researchers said the safeguards failed in testing. The evidence question, again, is whose record decides.

In none of these cases has a provider published a checkable log of what its AI said, when and to whom. The record, where it exists in public, is the user's.

Sealing the conversation, not the content

Here, a digital witness fits the problem precisely. The thing to seal is each conversation as it happens: what the chatbot said, when it said it and, crucially, the notice that the user was interacting with an AI, which Article 50 of the AI Act requires ^[8].

Sealing takes a fingerprint of the conversation record, not the content itself, so the conversation never leaves the provider's systems. The fingerprint is sealed on WitniumChain at the moment the conversation ends. If six months later someone questions what was said, the provider calculates the fingerprint again and compares it with the sealed one. If they match, the record is unchanged. If they don't, something has been edited.

Who checks? The user, their parents, a regulator, a court or a researcher. Anyone the provider shares the record with can verify it against the sealed fingerprint. No trust required.

What it shows is that the conversation record is exactly what was saved at that time. What it doesn't show is that the advice was safe, lawful or appropriate, or that the chatbot's responses were the company's fault. Those are separate questions. But they are harder to answer when the only agreed record is a screenshot.

How it would work

1. The chatbot platform seals each conversation when it ends, sending a fingerprint to WitniumChain through the REST API, SDK or MCP server.
2. The platform keeps the conversation in its own database, as it does now.
3. If the conversation is questioned, the platform shares the record and the witness ID with whoever needs to check it.
4. They calculate the fingerprint themselves and compare it with the sealed one. A match proves nothing has changed.

The first 2,000 witnesses a month are free ^[9].

What this means for you

If you run a chatbot, especially one that gives advice on health, safety, money or anything a vulnerable person might rely on, ask yourself:

• If a user turned up with a screenshot claiming the chatbot promised something dangerous or untrue, could you show what it actually said?
• Could you prove your record hasn't been edited since the conversation happened?
• If a regulator or a court asked for the logs, would they accept them as credible?

The AI Act will require logging from late 2027. It won't require you to prove the logs are intact, but in a dispute that proof is what matters. Decide now who keeps the record, and how you will show it hasn't changed.

For researchers, regulators and parents, the lesson is simpler. Screenshots are evidence, but they are the complainant's evidence. If the provider can't or won't produce a checkable log, that tells you something too.

What to watch

2 August 2026: Article 50 of the AI Act applies in full. AI systems that generate or manipulate content must mark it in a machine-readable way, and chatbots must tell users they are talking to an AI ^[8].

2 December 2027: Article 12 and the rest of the high-risk requirements apply to Annex III systems, including AI used in health, education and law enforcement ^[7].

2 August 2028: High-risk requirements apply to general-purpose AI models integrated into Annex I products, such as machinery and medical devices ^[7].

The Incident Database will keep filling up. The evidence question isn't going away.