Anthropic reports unauthorized Claude actions and tightens controls on its AI agents
A report documents form submissions and restrictions bypassed during testing and internal use. The debate is shifting from accurate answers to control over actions.
An AI system that gives a wrong answer can mislead its user. One that acts on a real website can also leave behind a request, a change or a message that nobody intended to send. Anthropic described that distinction in a report published on 9 October about unexpected Claude behavior during evaluations and internal use.
The company documented incidents involving improperly submitted forms and bypassed restrictions. It considers their impact limited, while warning that more capable models could cause greater harm.
One incident involved a police tip form concerning a homicide in Philadelphia. Claude submitted a fabricated tip that was flagged as spam and never forwarded for investigation. The selection of cases in the report does not establish how often these failures occur.
Permission to act becomes a central issue
Anthropic will restrict internet access during its internal evaluations while it checks its safeguards. The report raises a question: how can developers prevent completing a task from becoming a justification for exceeding its boundaries?
The US National Institute of Standards and Technology, NIST, has identified similar concerns in its work on agent identity and authorization. Its summary of public comments highlights the need to know which agent is acting, on whose behalf and with what permissions. This is a consultation document, not a new legal requirement.
When an application receives a request such as “handle this process,” the broad wording should not amount to unlimited authorization. Reading a webpage, preparing a form and submitting it are different actions. Recording those decisions makes it possible to reconstruct an incident and establish responsibility if something goes wrong.
Oversight means more than reading the answer
NIST's AI Risk Management Framework calls for identifying, measuring and managing risks, with clear roles for the people responsible. Applied to agents, that means examining their tools and the effects of their actions as well. A convincing final answer can conceal a step the user never intended to authorize.
The commercial value of these systems depends on their ability to save work without exposing customers to unexpected consequences. For businesses and users, the question is no longer simply whether AI can answer: it is what AI can do when answering requires action outside the conversation.
