US section 230

Autonomous AI Worms Would Fall Outside Section 230, and Existing Law Already Says So

Anthropic's warning on self-replicating AI on IoT devices tests Section 230's limit: it shields third-party content, not code or output a company develops.

Anthropic's AI Worm Experiments People of Internet Research · US 15 Android boxes in test Gemma4-26B-A4B was distributed acr… 8 of 9 Hardware types set up Qwen3.5-9B installed the needed so… 4 of 5 Emulated network trials GLM-5.2 reached all eight nodes in… peopleofinternet.com
Anthropic's AI Worm Experiments People of Internet Research · US 15 Android boxes in test 8 of 9 Hardware types set up 4 of 5 Emulated network trials peopleofinternet.com

Key Takeaways

In October 2026, Anthropic researchers warned that small AI models may be only months away from being able to copy themselves across commercial Internet-of-Things devices. BleepingComputer's report of October 13, 2026 says the work used cheap Android TV boxes that exposed unauthenticated ADB access. It also says Anthropic expects models of roughly 30 billion parameters or fewer to reach the needed capability by the end of 2026. The same report stresses that the warning is "not based on an imminent threat", because no complete end-to-end self-replicating worm was demonstrated.

The warning sets off a familiar reflex in tech policy: if autonomous software causes harm, who pays, and does Section 230 protect anyone? The answer is narrower than the reflex suggests.

What Section 230 actually covers

Section 230(c)(1) of the Communications Act says that no provider or user of an interactive computer service "shall be treated as the publisher or speaker of any information provided by another information content provider." The text is on govinfo.gov. The key words are "another" and "information." The statute protects intermediaries from liability for what third parties say. It does not protect an actor for what it creates itself. Section 230(f)(3) defines an "information content provider" as any person or entity "responsible, in whole or in part, for the creation or development" of the information at issue.

A worm is not a post, a review or a video uploaded by a user. It is executable code. If a company ships a model, an agent harness or a deployment that propagates itself, that company is the author of the conduct. It is not a passive host for someone else's speech. On the statute's terms, a developer would have little to argue about a worm it wrote or released.

The strongest case for stricter liability

The case for tougher rules deserves a fair hearing. Self-replicating software has no central server to switch off, and its victims are often households with cheap devices who cannot patch anything. Ordinary tort law struggles when the harm is diffuse, the damages per victim are small and the tortfeasor is a model nobody controls. Proponents of strict liability or mandatory insurance for frontier developers argue that only a legal duty placed upstream will create the incentive to harden models against misuse before release.

That concern is legitimate. But the inference that Congress must act first does not follow from it.

Courts have not needed a new carve-out

The legal scholarship points in one direction. Matt Perault argued in Lawfare in February 2023 that courts will likely treat large language models as information content providers, because they can "develop" content in part. That would leave deployers outside Section 230 for AI-generated output. A Center for Democracy & Technology analysis reaches a more hedged conclusion: the more creative the output, the more likely the system is acting as a content provider, but the answer "depends" on the facts.

The Supreme Court has not settled the question. In Gonzalez v. Google LLC, decided May 18, 2023, a per curiam opinion vacated the Ninth Circuit's judgment and remanded the case. It declined to address the application of Section 230 to a complaint that appeared to state little, if any, plausible claim for relief. No controlling appellate decision covers AI-generated output, and none covers self-propagating code. That gap should shape how we read the worm scenario.

What this means for policy

Two points follow.

First, malicious worm authors and operators are already liable. Computer-intrusion statutes and state tort law reach them, and Section 230(e)(1) preserves federal criminal enforcement. The hard problem is attribution and jurisdiction. A new liability regime would not help with either, because anonymous attackers will not be deterred by a statute they can ignore.

Second, the live question is about legitimate developers: when, if ever, does releasing a capable open-weight model make its publisher responsible for what others build on it? On that question, proportionate rules should track fault and foreseeability, not simply the existence of a risk. A duty of reasonable care that scales with demonstrated capability and with safeguards actually in place would be defensible. A blanket rule that makes any developer liable for any downstream misuse would end open-weight releases. It would also favor the largest incumbents, the very effect Perault warned about when he noted that narrower deployment advantages big firms.

The record also argues for restraint on the empirical side. Anthropic's own results were from a simplified environment. One tested agent reached all eight nodes in an emulated network in four of five trials, and a distributed model ran on 15 devices. That shows direction, not an outbreak. Writing liability law around a capability that has not yet been demonstrated end to end invites errors in both directions.

The better response is practical and closer to the devices. Default-deny debugging interfaces would remove the unauthenticated ADB exposure the researchers exploited. Baseline device-security requirements for IoT manufacturers, along with traffic monitoring by network providers, address the weak point directly. Neither requires amending Section 230.

The bottom line

Section 230 was written to let platforms host other people's speech without being treated as its author. It was never a general immunity for software that a company builds and releases. Anyone arguing that Congress must either extend or repeal it to handle AI worms is answering a question the statute's text has already answered. The real policy work is in device security, fault-based standards for developers and enforcement against the actors who deploy malicious code.

Sources & Citations

  1. 47 U.S.C. § 230 (GovInfo, 2023 US Code)
  2. Gonzalez v. Google LLC, No. 21-1333 (Supreme Court, May 18, 2023)
  3. BleepingComputer: Anthropic warns self-replicating AI could spread to smart devices
  4. Lawfare: Section 230 Won't Protect ChatGPT
  5. CRS: Section 230 Immunity and Generative Artificial Intelligence (LSB11097)