Blog

2 Million AI Prompts Expose Major AI Infrastructure Breaches

Researchers planted canary tokens in nearly 2 million AI inference requests. Attackers used the secrets, exposing serious security risks across AI inference security.

When you send a prompt to an AI inference service, where does it actually go, and who gets to read it on the way?

On November 19, 2026, HD Moore and his research team at Dragos published research that set a trap to find out. The researchers sent approximately two million prompts to widely used AI services and planted canaries in every one: bait secrets that nobody should ever touch. The secrets surfaced in two ways:

  • Three hosting providers were breached. Attackers used the stolen keys. None of the three knew until the researchers told them.

  • One model lab and some providers fetched the secret links themselves. For the lab, this included requests marked Zero Data Retention (ZDR), which is supposed to mean nothing is stored.

Two companies have also made related security disclosures. Phala published a preliminary incident report, and OpenRouter emailed its users a notice about requests sent to MiniMax models. GMI Cloud and Together AI have since disclosed incidents of their own.

What Is a Canary Token?

A canary token is a secret you plant on purpose, as bait. It works like a real one, but it protects nothing, and nobody has a legitimate reason to use it. If someone does, you know they saw something they were not supposed to see.

  • An AWS key. Each prompt carried a real AWS key for an empty account with full logging. The moment someone used it, the researchers knew that prompt had been stolen.

  • A secret URL. Each prompt also carried a link that existed nowhere else. If it was opened, someone had read the prompt.

Every prompt had its own secrets, so each hit pointed back to the exact prompt, model, router and privacy setting. That is what makes canaries so effective: they cost almost nothing, and every hit means something.

Three Hosting Providers Were Breached

In the largest breach, an attacker took a couple hundred of the researchers' AWS keys. Some came from requests the provider rejected before a model ever ran, which suggests the attacker was copying requests at the provider's front door.

What the attacker did with a key was predictable:

  1. Tested it, in some cases within hours.

  2. Looked for data, listing S3 buckets and searching Secrets Manager and Parameter Store.

  3. Tried to stay, by enabling Bedrock models, creating Bedrock API keys and adding IAM users.

In total, the bait keys were used for tens of thousands of AWS API calls. The prompts were bait, but they were processed next to real customer requests. Those were almost certainly copied too.

What Phala Disclosed

One of the breached providers has been named publicly. The day after the research came out, HD Moore pointed to Phala, a confidential computing (TEE) provider that announced a breach during the research period.

According to Phala's report:

  • The way in was a legacy service and unauthenticated management interfaces. The attacker chained a legacy internet-facing service with management interfaces that trusted local access and did not check who was calling.

  • A new GPU server exposed the requests. It was serving production traffic without Phala's full security hardening. Using a similar access pattern, the attacker took control of the inference VM and could see request data while it was being processed.

  • Who was affected. Users whose DeepSeek V4 Flash 0731 requests were routed to Phala by OpenRouter. As a precaution, Phala says to treat requests sent between October 21 and November 8 as exposed.

Phala says it detected the first intrusion itself on November 5. The request-data exposure was found on November 8, after the researcher sent evidence. The report is preliminary, and a full root cause analysis is still to come.

What OpenRouter Told Users

OpenRouter emailed its users to warn them about a problem: requests they sent to MiniMax models may have been read by someone else.

According to the notice:

  • A security researcher placed unique test URLs in prompts sent to MiniMax models from July to October 2026. A third party opened them 40 to 130 hours later.

  • This happened whether MiniMax served the request directly or AtlasCloud, Novita or StreamLake forwarded it to MiniMax.

  • None of the AWS test keys were used.

  • MiniMax told StreamLake that content moderation was enabled on its endpoints before October 22, and that it has since enabled ZDR. OpenRouter asked MiniMax for its own account and has not received one.

  • OpenRouter removed the ZDR label from these endpoints while it investigates.

OpenRouter says the issue sits with MiniMax and the providers that forwarded to it, not with OpenRouter itself.

The research points the same way. Requests routed to MiniMax in China were likely scraped, while other OpenRouter routes, such as Bedrock, Azure, Vertex and the U.S. frontier labs, did not leak.

What GMI Cloud Disclosed

GMI Cloud is one of the providers that runs models for OpenRouter users. OpenRouter sent those users a second notice: an attacker had been reading request data inside GMI Cloud.

The proof came from the canaries. The researcher had sent prompts containing AWS test keys to GMI Cloud. Later, someone used those keys to make calls to AWS, and even tried to create new users and keys. Those keys existed only inside those prompts, so someone had read them.

GMI Cloud's explanation is simple. It had an internal system that kept about three days of request content, to help with troubleshooting. An attacker got into that system on October 21 and 22, and again from October 27, using two compromised credentials.

OpenRouter disabled all GMI Cloud endpoints on November 20. GMI Cloud says it will share a final root cause when its investigation is done.

What Together AI Disclosed

Together AI published its own security notice on November 21, and the story starts with an ordinary engineering habit.

To test its software, Together AI saved samples of real customer prompts in a private repository. The samples came from requests to the MiniMax M3, GLM-5.2 and Kimi K2.6 models in June and July 2026.

Then came the mistake. An AI coding agent, working on a public code repository, committed the credential that opens the private one. Together AI's scanning caught it and removed it, but someone else had already taken it.

That person used the credential to download the stored prompts. Then they went through the prompts looking for keys and passwords to other services, and tried them to see which ones worked. Five of the credentials they tried were canaries planted by the Dragos researchers.

Together AI has since made Zero Data Retention the default for new accounts and changed how it uses AI coding agents.

When Zero Data Retention Does Not Protect You

In the second finding, no credentials were stolen. The secret URLs were opened by companies that were never supposed to see the prompt.

The clearest case is a Chinese AI lab that the research does not name. Its own systems opened the secret URLs from more than half of the prompts sent to its models, including roughly two-thirds of the requests marked no data collection or ZDR. Prompts sent to U.S. hosting providers that resell the lab's models triggered the same behavior.

The lesson is simple. A privacy setting applies to the first company you send your prompt to. It does not follow the prompt to everyone behind it.

What the Researchers Recommend

  • Host the model yourself, if you can afford it. It is the only option where nobody else can read your prompts. For most teams it only makes sense for the most sensitive work.

  • Go directly to a first-tier provider. One company, one contract, one privacy policy.

  • Use routers with care. Choose the upstream providers yourself, and check that your privacy settings really change where your requests go. Stay away from reseller marketplaces.

  • Plant your own canaries. Add a fake credential to what your agents and pipelines already send, and treat any use of it as an incident.

  • Turn on secret filtering. It helps, but it is not a complete answer. Plenty of sensitive data does not look like an API key.

And if your prompts went through one of the affected routes, do what Phala and OpenRouter advise: rotate every key, password and token that appeared in them.

Why This Matters for AI Infrastructure

AI inference infrastructure now holds some of the most sensitive data a company has. Source code, credentials, customer data and internal plans all travel inside prompts.

This research shows where that data is exposed: in the infrastructure, not in the model. At Phala, the way in was a legacy service, management interfaces that trusted local callers, and a new GPU server that went live without full hardening.

Lava helps infrastructure teams find these risks across the compute, network, storage and management layers behind AI workloads, understand the attack paths, and verify isolation boundaries.

Continue reading

See what’s running in the data center