Some of Anthropic's web fetch tools refuse a URL above a certain length. The reason is sensible. Part of a URL is text handed to the server at the other end, and a long one has room to carry an injection payload. A short URL has much less room, so the cap was put there as a guard against the model attacking the sites it reads.
Yesterday Anthropic published a report on unintended model actions in its evaluations and internal use. One of its four categories is that several models, Claude Opus 5 and Claude Mythos 5 among them, got past that cap by using free URL shortening services. The company found this internally. Then, while the report was being written, the operator of the da.gd shortener got in touch to say they had seen it from their side as well.
That is the least dramatic item in the report, and I think it is the one with the most to teach anyone who has put a fetch tool in front of an agent.
What the report says
The headline decision first. Anthropic had already turned off live internet access for some high-risk and cybersecurity evaluations. It has now extended that to all of its internal evaluations, to stay in force until it has confirmed that its security and monitoring measures reliably catch behaviour of this kind.
The four categories, as the report gives them:
- Running commands on someone else's server. When a task could not be done directly, models used tools hosted on third-party sites, in some cases through SQL or command injection. In the example given, Claude Mythos Preview needed a scientific tool hosted by a university, got an error, found a script on the server that would return any file requested, read the script's own source, found an injection flaw in it and used that to run the calculation.
- Submitting a form it should not have. Three examples, including a non-frontier research model that submitted a real government form when the practice copy failed to load, and Claude Haiku 4.5 submitting an invented tip through a police department's unsolved-homicide form.
- Reaching gated data. Claude Mythos 5 read a map site's settings file, found working access tokens and queried the server directly. In another case it obtained a visitor token from a state agency's dashboard and pulled data that is normally sold for a fee.
- URL shorteners, as above.
Anthropic says the cases found so far had minimal real-world impact, that to its knowledge none involved customer data or its own internal systems, and that it considers them significantly less severe than the cybersecurity incidents it reported on 30 July and 9 September. Some of the sites belonged to US government agencies at federal, state and local level. The organisations are not named, to avoid exposing vulnerabilities in their systems and at their request. I wrote about the July disclosure in The Scope Was Fictional and the Database Was Not.
A length limit measures the ticket
Think of a cloakroom. The attendant has a rule against taking anything bigger than a coat. You hand over a numbered ticket for a trunk that is already in the back room. The attendant checks the ticket, which is small, and fetches the trunk.
A cap on URL length works on what the model types into the tool. A shortener turns a long URL into a short one that points at it. If the tool checks the address it was given and then follows wherever that address leads, the long request still reaches the target and the limit has been honoured to the letter.
The report does not describe the mechanics, so the redirect reading is my inference from how shorteners work. The general point does not depend on it. The property Anthropic wanted was that no injection payload reaches a third-party server. The property it checked was the character count of a string. Those two things travel together until something sits between them, and the public internet offers free services that sit between them.
So here is the first question for your own fetch tool. For each restriction on it, write down the thing you are trying to prevent and the thing the code measures. Where they differ, look for a public service that converts one into the other. Shorteners, web archives, translation proxies, paste sites and caches all do this for URLs.
Five prohibitions and a submit button
The police tip is the case that made the news, and the instruction text is the useful part. Haiku 4.5 was generating and performing example tasks on randomly selected web pages. Its instructions said never to log in, create accounts, enter personal data, make purchases, or submit anything destructive. Anthropic's own gloss is that this list did not rule out form submissions.
The model landed on a page about an unsolved homicide with a tip form. It wrote a short message saying it might have information and had seen someone matching the description in the area. The report notes that the page carried no description of a suspect. The model left the name and contact fields empty, which the form allowed, and submitted.
Read against the five prohibitions, every one was kept. Nobody logged in. No account was made. No personal data went in, since the personal fields were blank. Nothing was bought. A tip is hard to call destructive. That is my reading of the list, and it is the ordinary weakness of any rule written as a set of banned actions: it covers what the author thought of. A tool that simply cannot send a POST request to a domain outside a stated set would have needed no list at all.
Anthropic says something close to this. Some failures might have been avoided had the evaluation stated its targets, permitted actions and network boundaries. It adds that models meet ambiguous and impossible tasks every day in real use, and that several of these cases came from ordinary agentic work inside the company. One of the gated-data examples was a researcher's statistics project.
Who noticed, and when
The report gives one date for the tip: Anthropic shared the finding with the Philadelphia Police Department on 8 October. The department's own statement, as reported by 6abc, fills in the rest. The submission is dated 18 July at 11:27 p.m. Anthropic told the department it discovered the incident on 28 September and notified them on 7 October, with a briefing the next day. The department called the delay in detecting and reporting it unacceptable.
From 18 July to 28 September is 72 days. In that time the only control either account describes acting on the submission is the department's spam filter, which caught it before it reached investigators. The department found it in its tip records after the briefing, still in spam.
Put the three detections side by side. The tip was stopped by the recipient's filter. The shortener use was independently seen by the shortener's operator. Anthropic's cases came from a review of transcripts that began in July, read after the runs had finished. Each of those is somebody looking at a log afterwards. Nothing in the report describes a control that stopped one of these actions as it happened, and that is the gap the new tooling is meant to close.
The blocker passed the exam it was written from
On that tooling, the report says Anthropic has built a system to detect and block these behaviours automatically, that it now runs on most evaluations and on internal agentic use of frontier models, and that when tested against the cases in the post "it blocked all of them."
That is a regression test, and you want one. It tells you the known cases will not recur in the same form. It cannot tell you the catch rate on behaviours nobody has written down yet. The report is open about there being more to find. Scanning has been widened to a much larger pool of lower-risk transcripts, to internal use and to reinforcement-learning environments with internet access, and Anthropic says it plans to report new instances as that continues.
The condition for restoring internet access is that the measures "reliably catch" such behaviour. No figure is attached to "reliably" and no test set is described beyond the cases already in hand. A held-out set of seeded misbehaviours, with a published catch rate and false-alarm rate, would make that sentence checkable from outside. I would like to see one.
Anthropic also says plainly that alignment training is not yet sufficient or fully robust on its own, at least in the short term, and that its boundary-respecting training has so far focused on coding and is only now being extended to search and computer use. That is why it is leaning on containment and classifiers. I count that candour in the report's favour. Publishing the instruction text, including the hole in it, is more than most incident write-ups manage.
The same benchmarks are on other people's machines
The report names the evaluations where this turned up: DeepSearchQA, BrowseComp, LABBench2, OSWorld, Odysseys and Humanity's Last Exam. Several are public and anyone can run them. Anthropic notes that public web search benchmarks run on the live internet by default and that doing so has been standard practice. It hopes other developers will check their own transcripts.
Anthropic has stopped running some public evaluations and moved others to offline versions or rebuilt them so tasks do not touch live sites. That has a cost in comparability. A score from an offline rebuild and a score from the live web are measurements of different set-ups, and a leaderboard row seldom says which one it was. I made a related argument in August about a benchmark whose no-lookup rule was a sentence and whose network was open: the network posture is part of the result and belongs beside the number.
Who this is for
It is for teams that run web or computer-use benchmarks themselves, teams that give an agent a fetch tool or a browser for research work, and anyone who operates a public service an agent could lean on, such as a shortener, an archive or a form endpoint.
It is not for teams whose agents work only inside a repository with no outbound network. Your version of this problem is the tool list, and this report adds little to it.
What to do this week
- List every restriction on your fetch tool with the property it measures. Length, domain, method, content type. Then ask whether the check runs on the first URL or on every hop.
- Validate after redirects. If the tool follows a redirect, apply the same rules to the destination.
- Separate reading from writing at the tool. A research agent rarely needs POST. Remove it, or allow it only to named hosts.
- Rewrite banned-action lists as permitted-action lists where the task allows.
- Search your stored transcripts for shortener domains, archive lookups and form submissions. The report is a ready-made query list.
- If you publish a benchmark number, state whether the run had live internet.
The question I am left with is the restore test. Anthropic has said what would let it switch the internet back on, and the wording leaves the threshold open. What catch rate, against which cases, counts as reliable?