UPDATED: Sunday 6/14 7:30pm
How a small group rattled a nearly trillion-dollar company and the U.S. government badly enough to dark the most powerful AI ever released — and why the method matters less than the math.
AI shutdown by Government. Not exactly a new thing when it comes to the tech industry. But this is very different. Government says it was a systemic security safety issue vs anthropic saying it was a misunderstanding.
A few people did this. They did it to Anthropic — a company valued at $965 billion, the most valuable AI startup on the planet, days removed from filing to go public — and to a model hardened over thousands of hours of testing and, by the end, watched by the federal government. The ‘jailbreakers’ won in days. So the question running through every group thread from Caltech to MIT is the obvious one:
What method did a handful of people engage to break a system that big?
PUBLISHERS NOTE: If you’re looking for the retail version of the story that says what the government did and why then Click Here for That Report
A Technical Breakdown
The technique used to jailbreak Claud’s Fable 5 has not been made public, and shouldn’t be — a transferable method for stripping the safeguards off a model that finds software vulnerabilities is a weapon, not a disclosure. What failed is a documented, unsolved problem in machine learning: large models have no worst-case adversarial robustness. The people who broke Fable applied a known technique to a weakness no lab currently knows how to close. Here is the mechanism.
What “Safety” Actually Is
A deployed AI model is not one system with a safety setting. It is a stack of learned components, and none of them is a hard gate.
The base model — the policy model — holds its capabilities in its weights, entangled and inseparable, including the code reasoning that lets it find exploitable conditions in real software. Alignment is added on top as behavior, not as a barrier: through RLHF, RLAIF, and Constitutional AI training, the model learns that certain requests are to be refused. It declines because refusal is the rewarded action, not because anything stops it. That is a disposition, not a constraint.
Production systems wrap that model in guardrails. The main one is the guard model: separate input and output classifiers that score traffic and block what reads as harmful, a design Anthropic has published as Constitutional Classifiers. A routing layer often sits alongside them, diverting high-risk requests to a weaker fallback; Fable reportedly used one. Behind it all, logs are retained for a fixed window so circumventions can be caught after the fact.
This is defense in depth, and the logic is sound. The problem is that every layer is a learned decision surface with a threshold, not a logical rule. Nowhere does a banned request hit an absolute barrier. There is no lock to pick — you find an input the classifiers score wrong and the policy model answers anyway, and the stack gives way there.
Why It Leaks
Models are optimized for average-case performance, not worst-case robustness, and that distinction is the whole story. A classifier can be right on all but a vanishing fraction of realistic inputs and still fall easily to an adversary who searches for the ones it gets wrong. This is the decade-old result on adversarial examples: small, deliberately built inputs that flip a model’s output with high confidence. The decision surface looks smooth on natural data and is full of failure regions off the training distribution. Going off-distribution on purpose is the attack.
In a language model the search space is every possible token sequence — text, code, mixed encodings, any language, across context that runs to hundreds of thousands of tokens. It is combinatorially unbounded. You cannot enumerate it, you cannot test it, and no certified-robustness method scales to a model this size against an unrestricted text adversary. So “thousands of hours of red-teaming” means a finite sample drawn from an infinite space, in a problem where one miss is enough. Red-teaming raises the cost of finding a hole. It cannot prove there isn’t one, and in the worst case the holes are guaranteed. The only question is how hard they are to find.
The Attack Families
Fable’s method wasn’t disclosed, but it belongs to one of a few documented families.
Optimization attacks. The 2023 Greedy Coordinate Gradient result showed you can algorithmically build a short token suffix that raises the probability the model complies and lowers the probability it refuses. It needs white-box access to the model’s internals, but the suffixes it produces often transfer to closed commercial models they were never built against.
Transfer from a surrogate. That transferability is why offline development works. You optimize against an open-weight surrogate, or a smaller model distilled to imitate the target, then fire the result at the target as a black box. The defender sees none of the preparation.
Long-context and multi-turn. Anthropic published many-shot jailbreaking itself: fill the context window with a long run of fabricated exchanges where the assistant complies, and in-context learning overrides the trained refusal. Multi-turn methods like Crescendo do it gradually, escalating across turns so no single message trips the guard.
Automated search. PAIR and TAP put a second model in the attacker’s seat, generating and refining prompts against the target until one lands. This makes finding an attack cheap, parallel, and automatic — the defender’s effort to anticipate attacks doesn’t scale like the attacker’s effort to find them.
All of these run on one principle: build an input the guard reads as benign and the policy model acts on as live, exploiting the gap between the guard’s decision boundary and the model’s. Change the surface form, keep the substance.
Offline Work and Weight Security
Put transfer and surrogates together and the serious version is obvious: you do the work where the defender can’t watch. Monitoring only catches attacks built against the live system. One matured against a surrogate arrives finished and is used once.
The worst case is exfiltrated weights. With the model itself, an attacker probes it without limit on air-gapped hardware, fine-tunes the safety behavior out, and studies the guards offline — no signal reaches the owner. That is why model weights are handled like fissile material, with a dedicated security literature behind them, including RAND’s work on protecting them. It is also why this was an export-control action and not a bug bounty. Content-moderation failures don’t get export controls. Proliferation of an offensive capability does.
The Payload
This wasn’t about objectionable text. It was about capability, and the capability is automated vulnerability discovery. The model reasons about code well enough to find exploitable conditions in real, shipping software and reason toward working proofs of concept. Aimed at defense, that is the best patch-finding tool ever built — what Anthropic restricted the unrestricted model to under Project Glasswing, which reportedly found flaws in every major operating system and browser it was pointed at. Aimed the other way, the same weights run the same analysis for the opposite purpose. Find-and-fix and find-and-exploit are the same task to the model. That symmetry, not any single output, is why this reached national security. The government reacted to what the model can do with the safety removed, not to what it might say.
The Asymmetry
The defender has to be robust across an unbounded input space, with no certified method at this scale. The attacker has to find one input — cheap, transferable, automatable. The defender ships one system to hundreds of millions of users at once; the attacker iterates in private against a copy. Run that forward and the hole gets found. It is not negligence. The gap is in the architecture every system like this shares.
What’s Known, What Isn’t
Confirmed: the government issued an export-control directive citing a Fable 5 jailbreak; Anthropic complied and pulled both models; the company says its testers found no universal jailbreak and that comparable capability exists in other public models, including GPT-5.5. Single-source: an administration official told Axios a rival company’s demonstration triggered it — one anonymous account. Not disclosed: the technique, which would be its own security incident to publish. Anyone stating the exact method is inferring from the families above.
Bottom Line
The off-switch is what’s left when the architecture guarantees nothing. June 12 wasn’t exceptional. It was a known attack on an unsolved problem, and the only surprise was the speed. Expect more.
The Standing Condition
Here is the line to leave people with. As long as alignment is a learned policy laid over capabilities that remain latent and inseparable in the weights, and as long as there is no certified robustness against an unrestricted adversary, the security of a deployed frontier model is bounded by the robustness of its weakest classifier — and that bound is empirical, not proven. There is no hard floor under it.
A small team did not out-muscle a giant – but they were clever… and fast.
