AI agent observability is struggling to keep up with hallucinations

The diagnosis gap

When a server goes down at 3 a.m., IT teams increasingly ask an AI agent what went wrong. A recent survey by Splunk of 2,050 IT professionals found that 62% now use AI agents to diagnose system failures ^[1]. That sounds efficient, until you notice the next number: 56% of those teams have no way to measure whether the agent got it right ^[1].

It's a bit like asking a colleague to check the boiler and taking their word for it without looking at the readings yourself. Most of the time it works, but when the heating fails in January, you want more than someone's say-so.

The survey calls this the observability gap. Organisations are letting AI agents make the call on what broke and why, but they cannot later show what the agent concluded, or check whether it hallucinated part of the diagnosis. That matters most when the diagnosis leads to an incident report, a customer explanation or a decision about liability.

What Article 12 asks for

From 2 December 2027, high-risk AI systems under Annex III of the AI Act must technically allow for the automatic recording of events over their lifetime ^[2]. Article 12 says what to log. It does not say how to prove the log is unchanged.

Diagnostic AI agents in critical infrastructure or financial services are likely to fall under those high-risk rules. The law will ask for the logs; whether they hold up as evidence is another matter.

A log kept by the same organisation that runs the agent can be edited by someone with administrator rights. That does not mean it will be edited, but it means the log itself does not prove it was not. When an incident leads to a claim or a regulatory question, that gap becomes a problem.

The hallucination risk

AI agents do not always stick to the facts. They sometimes fill gaps with plausible-sounding nonsense, a behaviour known as hallucination. In a diagnostic context, that might mean an agent confidently points to the wrong root cause, or invents a log entry that supports its conclusion.

The Splunk survey does not break down how often diagnostic agents get it wrong, but the fact that over half the teams using them have no way to check suggests the risk is not being managed ^[1]. When the agent's conclusion goes into an incident report or a post-mortem, and someone later questions it, the organisation needs to show what the agent actually said and did.

Without that, the agent's word is all there is.

Where a digital witness fits

This is a case where a digital witness fits well. The thing to seal is each diagnostic conclusion as the agent writes it: the symptoms it was given, the tools it called, the reasoning it logged and the conclusion it reached. Sealing takes a fingerprint of that record, not the record itself, so the diagnosis and any sensitive details stay on your systems.

When someone questions the diagnosis weeks or months later, anyone you share the record with can check that it is exactly what the agent wrote at the time. If a single word has changed, the fingerprint will not match.

What it does not do is prove the diagnosis was correct. It shows the record is unchanged, which is a different question. An agent can hallucinate confidently and have that hallucination sealed just as reliably as a correct answer. The value is that you can later show what the agent actually concluded, rather than what someone remembers it concluding, or what the log says after three edits.

You seal through WitniumChain, using the REST API, the SDK or the MCP server. The agent sends a fingerprint when it writes the diagnosis, and gets a witness ID back. The content never leaves your systems. The first 2,000 witnesses a month are free. Read the docs.

How it would work

1. The agent writes its diagnostic conclusion and calls the witness API with a fingerprint of that record.
2. The fingerprint is sealed, and the agent stores the witness ID with the diagnosis.
3. Later, when the diagnosis is questioned, you or an auditor recalculates the fingerprint and checks it against the sealed one.

What this means for you

If your organisation uses AI agents to diagnose failures, investigate incidents or make operational decisions, ask:

• Can we show what the agent concluded, exactly as it wrote it, if someone questions it six months from now?
• Do we have a way to detect if the agent's reasoning included a hallucination?
• When the AI Act's logging requirements apply from December 2027, will our logs hold up as evidence, or just as records we kept?

The Splunk survey suggests most teams have not yet asked those questions ^[1]. The gap will matter most the day an incident leads to a regulatory investigation, a customer complaint or a disputed insurance claim.

What to watch

• 2 December 2027: high-risk requirements under Annex III of the AI Act apply, including Article 12 on logging ^[2]
• Standards on AI observability and explainability, which are still being developed and may set expectations for how diagnostic conclusions should be documented