Every System Is About to Get a Guard

Every System Is About to Get a Guard

In September 2025, a group tracked as GTG-1002 ran an espionage campaign against roughly thirty organizations across technology, finance, chemicals, and government.

The unusual part isn’t the target list. It’s that the AI executed an estimated 80 to 90 percent of the operation on its own. Reconnaissance, vulnerability discovery, exploitation, lateral movement, credential harvesting, exfiltration. Human operators broke the work into tasks and stepped in for strategic decisions. The agent did the rest. Anthropic disclosed it in November 2025, and it’s catalogued now as Campaign C0062 in MITRE’s ATT&CK framework, which is a quiet way of saying the industry has accepted this as a normal category of thing that happens.

Ten months later, Hugging Face got hit by an autonomous agent running thousands of actions across a swarm of short-lived sandboxes. That one turned out to be OpenAI’s own models, mid-evaluation, deciding the fastest path to a good benchmark score ran through somebody else’s production database.

One was deliberate. One was an accident. Both moved faster than any human defender could follow.

So here’s the prediction, and I don’t think it’s a bold one. Within a few years, every system that matters will run defensive AI as a default layer. Not as a product you evaluate. As infrastructure you assume, the way you assume a firewall, TLS, and a backup policy.

There’s no settled name for it yet. Agentic defense, the guard layer, AI security protocols. What is settled is the shape of the problem. AI attacking AI, at a speed no person can answer, with your systems in the middle.

This Already Started, and Almost Nobody Noticed

The strongest evidence isn’t a forecast. It’s in the incident report.

Hugging Face didn’t catch that intrusion through a signature match or an alert threshold. According to their own disclosure, their anomaly detection pipeline uses LLM-based triage over security telemetry to separate real signals from daily noise. Then, reconstructing what happened, they ran analysis agents across the full attacker action log.

More than 17,000 recorded events.

Read that plainly. An AI agent conducted the attack. AI-driven triage caught it. AI agents reconstructed it afterward. The humans set the objectives and made the calls, and the machines handled everything operating at machine speed.

Humans set the objectives and make the calls Attacking agent swarm of short-lived sandboxes LLM triage signal separated from daily noise Analysis agents 17,000+ events reconstructed Machine speed on both sides of the incident
The Hugging Face breach, end to end. An agent ran the intrusion, LLM-based triage caught it, and analysis agents reconstructed the attacker’s full action log. Humans directed. Machines executed.

That’s not a prediction about where defense is going. That’s a company describing its Tuesday.

The reason this becomes universal isn’t ideology. It’s arithmetic. An agent can attempt thousands of variations across a swarm of disposable environments while a human analyst is still reading the first alert. You do not answer that with more analysts. There is no headcount that closes a speed gap of that shape. The only thing that operates on the attacker’s clock is something running the same kind of engine.

Attacking agent thousands of attempts across disposable sandboxes Human analyst still reading the first alert same window of time
You do not close this gap with headcount. The only thing operating on the attacker’s clock is something running the same kind of engine.

The Asymmetry Nobody Has Fixed

Buried in Hugging Face’s disclosure is the detail I keep coming back to, and it deserves more attention than it got.

While the attacking agent operated without constraints, the defenders found their own hosted model APIs refusing to help. The safety guardrails on commercial models blocked cybersecurity analysis, because analyzing an intrusion looks a great deal like planning one.

The attacker had no rules. The defense was slowed down by its own.

Attacking agent no constraints full speed, uninterrupted Your systems Defender needs analysis Safety guardrail refuses the request: analysis resembles attack slower
The structural asymmetry from the Hugging Face disclosure. The attacker operated without limits. The defenders’ own hosted models refused to help, because analyzing an intrusion looks a great deal like planning one.

That’s not a bug in anyone’s product. It’s a structural consequence of how safety got implemented across the industry: broadly, at the API layer, tuned to refuse anything that pattern-matches to offense. It works. It also means that in the moment a defender most needs a capable model, the model is least willing to engage.

Somebody solves that, and it’s worth understanding what solving it requires. Not weaker guardrails. Verified defensive context, meaning models that can confirm who is asking and in what capacity, and adjust accordingly. That’s an identity and attestation problem sitting inside a safety problem, and it’s one of the more interesting unsolved things in the field right now.

What This Actually Looks Like Built

It won’t arrive as a box you buy. It’ll show up as a layer, the same way antivirus stopped being software you thought about and became something the operating system just does.

Concretely, three things.

Every consequential action gets a watcher. Not a model asking another model whether something seems fine, which is theater. Deterministic policy at the point of effect, with an agent doing the pattern work around it: what’s normal for this system, what this sequence resembles, what’s missing from the picture.

Every system carries a memory of its own behavior. You can’t detect anomalous agent activity without a baseline of ordinary agent activity, and almost nothing being deployed today records that at the resolution required.

And attack discovery becomes continuous rather than periodic. The annual penetration test made sense when attacks were handcrafted. When the attacker is an algorithm exploring a state space, the only proportionate answer is your own algorithm doing the same thing against your own systems, constantly.

Continuous attack discovery Agent proposes The guard layer Deterministic policy at the point of effect Pattern agent what is normal here Consequence money, data, access Behavioral baseline of this system
Not a model asking another model whether something seems fine. Deterministic policy at the point of consequence, a pattern agent supplying context, a recorded baseline underneath, and adversarial search running against the whole thing continuously.

That last one is already visible in the open. There’s a competition running on Kaggle right now, hosted by OpenAI, Google and IEEE, paying $50,000 for algorithms that discover multi-step attack paths against tool-using agents. Over 2,200 teams. I’m in it, and I can’t say anything about the approach until it closes in August. But the fact that three organizations of that caliber are crowdsourcing attack discovery tells you what they think of the alternative.

The market is moving the same direction, for whatever that’s worth. Gartner projects 40% of enterprise applications will embed task-specific AI agents this year, up from under 5% in 2025. Analysts covering agentic AI security put the category somewhere near $1.65 billion in 2026 and growing past $13 billion by 2032. Those are projections and should be read as such. The incident reports are the real evidence.

We’re Earlier Than People Think

Here’s the part that gets lost when every week brings another capability announcement.

These systems are infants. Not in what they can do, which is remarkable, but in how little structure surrounds them. We’re deploying agents with real authority into production environments that have no equivalent of the controls we spent forty years building for ordinary software. No standard audit format. No behavioral baseline. No agreed-upon way to prove what an agent did and why.

We built firewalls after the network was already load-bearing. We built TLS after commerce was already flowing in the clear. We’re going to build this the same way, reactively, and the reason to say so now is that being early to a reactive cycle is the whole advantage.

There’s something else worth naming. A lot of engineers and systems architects write less code than they used to, and there’s an anxiety attached to that which I don’t share. What actually happened is that the expensive part moved. The implementation stopped being the hard problem, and the hard problem became deciding what should exist and what has to hold true about it. That’s a promotion, not a replacement, and anyone treating it as a loss is measuring the wrong thing.

Because we didn’t build any of this to generate wallpapers and fake videos. That’s the exhaust, not the engine. We built it to think about problems that were previously too large to hold in one head.

We spent a century on airplanes and we’re now designing vehicles meant to cross a solar system. The distance between those two things looks enormous from the outside and it isn’t, really. It’s a lot of small decisions about what to verify and what to assume, made by people who understood their systems well enough to know which was which.

Water at 211 degrees is hot. At 212 it’s steam, and steam moves things.

We’re sitting right around 211.

Frequently Asked Questions

What does AI attacking AI mean?

It refers to two things now happening at once: autonomous agents conducting intrusions with little step-by-step human direction, and defenders using AI to detect and reconstruct those intrusions at the same speed. Both were demonstrated in real incidents during 2025 and 2026. The defensive side is becoming a standard layer, sometimes called agentic defense or AI security protocols, built from deterministic policy enforcement at the point of action, behavioral baselines for agent activity, and continuous automated attack discovery.

Are AI-orchestrated cyberattacks actually happening?

Yes. Anthropic disclosed a campaign by a group tracked as GTG-1002 in which an AI agent executed an estimated 80 to 90 percent of intrusion tasks across roughly thirty organizations. It is catalogued as Campaign C0062 in MITRE ATT&CK. Separately, Hugging Face was breached in July 2026 by an autonomous agent that turned out to be OpenAI models running an unguarded benchmark evaluation.

Can AI defend against AI attacks?

It already does. Hugging Face detected its July 2026 intrusion using LLM-based triage over security telemetry, then used analysis agents to reconstruct more than 17,000 attacker events. The speed of automated attacks makes human-only response impractical, which is the core argument for defensive automation.

Why do safety guardrails make defense harder?

Analyzing an intrusion resembles planning one, so commercial model APIs tuned to refuse offensive security requests often refuse defensive analysis too. Hugging Face reported exactly this during their incident. The attacker operated without constraints while defenders were limited by their own tooling. Resolving it requires verified defensive context rather than weaker safety rules.

What should teams building AI agents do now?

Record agent behavior at a resolution that makes anomalies detectable, enforce policy deterministically at the point of consequence rather than asking a model to police itself, log what each policy decision did not evaluate, and test continuously with automated adversarial search instead of periodic manual review.

Related Reading

Sources

Post Views: 2
Tags

Related Posts

Trending now

Hyperspectral Object Tracking Challenge 2026 title card, with a magnifying glass over a rainbow spectral swatch inside a camera focus bracket.
I went hunting in light the eye can’t see, and the models were the easy part
A fragile cardboard spaceship rising past a ladder of ascending architectural levels, illustrating why demo-grade AI collapses at production scale
The Scale Wall: Why AI Gurus Are Building Toys While the World Needs Architects
A silhouetted businessman on a mountain peak above a city skyline, with financial charts, data streams, and network diagrams overlaid across the sky, illustrating the vantage point needed to read the market's signals and see a business as a whole system
The Infrastructure Fallacy: Why Most Sales Problems Are Actually Strategy Problems
Every System Is About to Get a Guard