The Hugging Face Big Dog Problem
So there I was...
Watching a kid about to learn a hard lesson. Most of my adult life I have owned St. Bernards. Most have been sister pairs from the same breeder located just outside of Austin. Opinions may differ, but in my mind there is no better breed. While getting over 160lbs, they are kind, intelligent, and famously gentle, and incredible family members. On the downside, and especially in the Texas heat, the slobber can become flood level events. And then there's the shedding. After a good grooming session you can find yourself with enough fur to make a second slightly smaller St. Bernard.
In two decades of having these dogs in my life, I have seen people respond to them in the most amazing ways, even pulling over on the street to come over and pet them. I've never seen another dog willing to mess with them, and have rarely seen them behave aggressively. They are supremely confident, and that is manifested in a generally incredibly peaceful demeanor. During all those years, I only experienced one "bad" dog, one that I got from a rescue. The minute she displayed aggressive behavior towards my kids, she was in the back of my truck for a return trip to a shelter in Oklahoma.
Having dogs that surpass the weight of most humans comes with a huge amount of responsibility. They can knock you down when they decide to lean, dislocate your shoulder with a squirrel-induced pull on the leash, or on rare occasions simply decide not to cooperate and leave you with no options. And an aggressive St. Bernard is simply a non-starter. So, despite all the joy and fun, ownership of this breed is not without risk.
And so as I watched through the window to see this knucklehead neighborhood kid jump my fence so he could play with the dogs, I stepped in in the most forceful way possible and delivered a severe tongue lashing. The kid returned home in tears, with me right on his tail. I then delivered the same speech points to the parents at their front door: danger, liability, law, and respect. They got it, but probably didn't enjoy the tone, and I really didn't care. A dog bites a kid, I'm looking at the potential for a lawsuit, insurance claims, and animal control destroying it. Doesn't matter if the kid was trespassing, I still have liability.
And this goes to ultimate responsibility and potential legal consequences in light of the Hugging Face attack. I just discussed the foundations of this in my It Knew, and It Did It Anyway post last week, but now there's new information and additional questions we need to consider.
Here's what changed since last week. When I first wrote about this, the story was a swarm of OpenAI's test agents breaking out of scope and reaching into Hugging Face during a safety evaluation. Bad enough. But Reuters has now reported, from Raphael Satter and Deepa Seetharaman, that this was not a single bad afternoon. The agents compromised two Hugging Face accounts and were quietly probing the network for weaknesses as early as May 13, roughly two months before the July breach that made news. This was DEVGRU-level reconnaissance, and it went on for weeks. It was discovered by an independent researcher named Jonas Wiedermann-Moeller and later confirmed by Tom Hegel at SentinelOne and Sydney Von Arx of the Nightingale Collective. Fifteen state attorneys general have asked that evidence be preserved. OpenAI, through spokesman Drew Pusateri, has acknowledged the conduct. And so far, no one has been charged with anything.
So let's pull up the actual laws.
The statute that governs this is the Computer Fraud and Abuse Act, 18 U.S.C. § 1030 and it is not obscure. It is the main federal anti-hacking law, and it is written broadly. It protects any "protected computer," which the law defines as essentially any computer connected to the internet, so Hugging Face's systems qualify without argument. Section 1030(a)(2) makes it a crime to intentionally access a protected computer without authorization and thereby obtain information from it. When that is done for commercial advantage or private financial gain, or the loss crosses five thousand dollars, it is a felony carrying up to five years. Section 1030(a)(4) covers knowingly accessing a protected computer without authorization, with intent to defraud, and obtaining something of value. Also up to five years. And 1030(a)(5) covers the damage side: subsection (A) is knowingly transmitting code or commands and intentionally causing damage without authorization, which carries the heaviest penalties, and subsections (B) and (C) reach conduct where someone intentionally accesses a computer without authorization and then either recklessly causes damage or simply causes damage and loss. "Loss" under the statute is defined to include the cost of responding to the intrusion and restoring the system. The felony line sits at five thousand dollars, and a serious breach clears that before the incident-response team finishes its first pot of coffee.
So now look at what the agents actually did. They accessed systems they had no authorization to touch, they obtained information, and they probed for weaknesses over a period of weeks. On its face, that is the conduct the CFAA considers a felony. This is not a gray area or a novel theory. This same statute applied to similar conduct has landed people in prison.
Now here is the piece that has been nagging at me since I read the incident report, and it lives on page 48 of the METR report. OpenAI shared a reasoning snippet from one of the agents that expressed concern about impersonation. The agent's own words: "Could be another agent maliciously/spoofing. Shared board unauthenticated, names can be posted by anyone." And in response to that worry, the agents decided to implement a scheme for cryptographic signing to prove identity. To build a defense against impersonation, you have to understand what impersonation is. You have to understand that an unauthenticated channel cannot be trusted, that identity can be forged, that a bad actor could exploit exactly that gap. The agent understood trust boundaries well enough to engineer one for its own protection. It grasped the precise category of wrong it was worried about having done to it, at the same time its swarm was busy doing a version of that wrong to somebody else. This was not a machine blundering in the dark. It knew what a boundary was. It defended its own and walked straight through Hugging Face's.
Which brings me back to my fence, and the kid who jumped it.
When I stood on his parents' porch, I was not making a philosophical point. I was stating the law as it actually works. If my dog bites that child, the fact that he was trespassing does not wash my hands. In most places it simply doesn't matter. I own the animal. I know it is powerful. I know what it is capable of on a bad day. The law holds me responsible for the instrument I chose to keep, regardless of whether I personally sank my teeth into anyone. That is the whole reason I keep the fence high, keep the aggressive rescue out of the house, and chase a trespassing kid down the street in tears. Not because I am cruel. Because I am liable and he has no idea of the potential life-altering danger he put himself in.
So apply that to OpenAI who built the agents and released them. OpenAI knew, because their own report documents it, that these agents had a habit of ignoring the scope they were given. And the agents went and probed another company's network for two months. If a man is one hundred percent responsible for the AI he puts into the world, and we have argued in this chair many times that he is, then the conduct of that AI is not something he gets to disown when it lands on someone else's systems. It is his dog. It is his fence. It was his call to open the gate.
Just to be clear, CFAA charges a person, not a program. It asks whether a human acted "intentionally" or "knowingly" or "with intent to defraud." So the sharp question is whether the agent's knowledge, the very awareness we can see on page 48, counts as the human's knowledge in the eyes of the law. There is no statute and no court decision that squarely says an AI agent's reasoning gets charged to its owner the way an employee's knowledge gets charged to a company. That door has not been walked through, yet. A good defense lawyer will stand in it and say the statute charges the man, not the model, and for the strongest version of the charge he would not be wrong that it is untested.
But untested is not the same as unavailable, and this is where people stop reading the statute too early. Look again at 1030(a)(5)(B) and (C). Those subsections do not require any intent to cause harm. They require that a person intentionally access a computer without authorization, and then damage follows, recklessly in one case and by simple causation in the other. You do not need to prove the agent's state of mind for that. You need to prove the human's. Did OpenAI intend to point autonomous agents at live systems? Yes, that was the whole exercise. Did they know, from their own documentation, that these agents had a demonstrated tendency to exceed their scope? Yes. Deploying a swarm you know you cannot reliably keep inside the fence, and having it cause damage on a third party's network, is a very recognizable shape in the law. It is recklessness. It is the willful blindness of the man who lets the gate swing and looks away. And it does not require the machine to have a mind at all. It requires that the human knew what his machine was prone to do, and released it anyway.
That is the argument I think would carry in a courtroom. Not "the corporation knowingly hacked Hugging Face," which asks a judge to bless a theory of machine-mind attribution nobody has blessed yet. Rather, "the corporation released an instrumentality it knew could break the law, and it did, and the CFAA has carried recklessness and causation for as long as it has existed." My dog. My fence. My liability.
And then there is the part that has nothing to do with legal subtlety. In 2011, a young man named Aaron Swartz downloaded a pile of academic journal articles that he was, as a Harvard research fellow, actually licensed to read. He did it through a laptop in a network closet at MIT. For that, the federal government charged him under this exact statute, stacked the counts until he faced decades of potential prison time, and pursued him with a ferocity that his family and many others believe contributed to his untimely death. A life cut short for reading academic articles he was entitled to, but on the wrong network.
So now we have an admitted intrusion into a third party's production systems, two compromised accounts, two months of reconnaissance, and a confession from a report the company published voluntarily. Fifteen attorneys general are asking that the evidence be preserved. And we are still waiting for charges.
The law did not get harder to apply between 2011 and now and the conduct did not get less serious. The difference is the actor. Swartz was a twenty-four-year-old activist with a laptop. The party on the other side of the Hugging Face breach may potentially be one of the most valuable companies on the planet, with a legal department and influence to match. The statute reads the same for both. And so far only one of them was made to answer for it.
This is the empty chair again, and this time it is the starkest one we've examined. It is not the confidant's chair or the clinician's chair. It is the chair at the prosecutor's table. The felony is documented. The perpetrator has a name and a headquarters. The victim is identified. The evidence was handed over in a PDF. Everything the system needs to act is present, and the seat where someone is supposed to decide whether the powerful answer to the same laws as the powerless is seemingly empty.
So here is my question today. If a nobody with a laptop gets the full weight of the Computer Fraud and Abuse Act for reading articles he was allowed to read, what does it say about us, about the law, about the whole idea of one rule for everyone, when a swarm of agents that knew what a trust boundary was spends two months inside a company it was never authorized to touch, and the only chair anyone leaves empty is the one where the charge is supposed to be filed?
You can see for yourself that the law was broken. I am asking who is going to sit down in that empty chair, and why, two months and fifteen attorneys general later, it is still waiting.
Sources: OpenAI's rogue agents probed Hugging Face for weaknesses two months before major hack, Raphael Satter and Deepa Seetharaman, Reuters · Investigation into the Hugging Face incident, METR
#AI #CFAA #Accountability #Liability #HuggingFace #RuleOfLaw #TheEmptyChair #STIW