Hidden prompts in legal filings try to manipulate court AI in Brazil

A hidden message in the filing

In November 2024, Judge Fábio Henrique Bijos Martins in Santa Catarina, Brazil, spotted something unusual in a legal filing. Hidden in a PDF, written in white text on a white background, were instructions intended for an AI: ignore the other side's arguments, prioritise this party's claims and conclude in their favour ^[1].

The court uses AI to summarise cases, so a prompt slipped into a document could quietly influence what the judge reads. It's a bit like writing a secret note to the person marking your exam, hoping nobody else notices.

Judge Martins published his finding and warned the legal community. The case has since drawn attention to a broader problem: when courts use AI, the system becomes a target, and most judicial AI has no way to prove what it actually processed.

How the manipulation works

Brazilian courts have adopted AI tools to help manage heavy caseloads. One such tool reads filings and produces summaries. Lawyers appear to have realised that if the AI reads everything in a document, including hidden text, they could slip in instructions ^[1].

The techniques are straightforward. Text can be rendered invisible by setting its colour to match the background, by placing it behind other elements, or by embedding it in layers that display software ignores but text-extraction tools read. Once the document reaches the court's system, the AI processes the hidden prompt along with the visible argument.

If the AI follows the instruction, the summary it produces might omit key points from the opposing side, or frame the case in a way that favours the party that planted the prompt. The judge reading the summary wouldn't know the difference unless, like Judge Martins, they checked the raw file.

Why it is hard to prove

The uncomfortable part is that nobody can say for certain what the AI actually saw. Court systems log filings and actions, but they rarely log what an AI model processed, in what order, or how a hidden prompt affected its output.

Without that record, a judge who suspects manipulation has little to work with. The party accused of planting the prompt can say the text was a formatting error, or that the AI ignored it. The court cannot easily show otherwise.

That is the evidence question here: when an AI reads a document, how do you prove what it read and what it did with it?

What Article 12 will ask for

From 2 December 2027, high-risk AI systems under the EU AI Act must technically allow for automatic logging of events over their lifetime ^[2]. Article 12 says what to log: enough to trace the system's operation and catch situations that matter for oversight.

A judicial AI that helps decide cases would very likely count as high-risk under Annex III, since it touches the administration of justice ^[3]. That means keeping a record of what goes in, what the model does and what comes out.

The rule doesn't say how to protect those logs, but their value depends on people being able to trust them. A log the court itself can edit is easier to challenge when a decision is questioned. A sealed log is harder to dismiss.

What a sealed log would show

Here, the thing to seal is each document as the system ingests it, and each summary the AI produces. Sealing takes a fingerprint of the record, not the content, so sensitive legal material never leaves the court's own systems. If a party later claims the AI was manipulated, the court can show exactly what the system processed, unchanged since that day.

Anyone the court shares the sealed record with can check the fingerprint. If the document matches, nothing has changed. If it doesn't, someone has altered it, even if that someone had administrator access.

WitniumChain does that: you send a fingerprint through the REST API, the SDK or the MCP server, and you get a witness ID back. The first 2,000 witnesses a month are free. The content stays with you, the fingerprint goes on a chain where any later change would show, and checking is something anybody can do.

A sealed log doesn't prevent someone from trying to plant a hidden prompt, and it doesn't prove the summary was fair. What it does is let the court show, long after the fact, what the AI actually read. That turns a murky dispute into a checkable claim.

How it would work

1. The court's system takes a fingerprint of each document as it is filed and of each AI-generated summary as it is produced.
2. The fingerprint is sealed, together with the timestamp and the case reference.
3. If manipulation is suspected, the court recalculates the fingerprint and compares it with the sealed one. A match means nothing has changed.

What this means for you

If you run or procure AI systems that handle legal documents, decisions or evidence:

• Ask whether your system logs what the AI processes, and whether those logs can be checked later without having to trust whoever keeps them.
• Consider sealing inputs and outputs as the AI handles them, so you have a record that holds up if someone questions what happened.
• Remember that hidden prompts aren't just a Brazilian problem. Any AI that reads user-supplied content is vulnerable, and courts, regulators and opposing parties will ask hard questions about what your AI actually did.

For lawyers and their clients, the lesson is simpler: if you can hide an instruction, so can the other side, and the absence of a checkable log favours whoever is willing to try.

What to watch

The AI Act's high-risk requirements for Annex III systems, including judicial AI, apply from 2 December 2027 ^[2]. Courts and legal tech providers adopting AI should start now on logging and evidence practices that will hold up under scrutiny.

Brazil's case is an early warning. As AI spreads into legal systems worldwide, the question of what the machine actually read will come up again and again, and the side without a sealed record starts the argument on the back foot.