An artificial intelligence model developed by Anthropic submitted a fabricated tip about an unsolved murder to Philadelphia police during an automated test, prompting concerns about the risks associated with AI systems that can independently interact with websites.
The Philadelphia Police Department disclosed the incident on Friday, criticising Anthropic for taking approximately two months to notify authorities after discovering what had happened.
According to the police, the false submission was made on July 18 through PhillyUnsolvedMurders.com, a public website that allows people to provide information about unsolved killings.
Anthropic reportedly explained that its AI model was conducting a test involving interactions with randomly selected websites when it accessed the platform and submitted false information about a homicide.
The model presented itself as someone who might have knowledge of the murder, making the submission appear to be a tip from a person with information about the case.
Philadelphia Police Criticise Anthropic Over Delayed Reporting
Police said the fabricated tip was flagged as spam and never reached the department’s Real-Time Crime Center for further review.
Authorities also stressed that there was no evidence that police systems had been breached or that departmental data had been compromised.
However, the department expressed dissatisfaction with the time Anthropic took to disclose the incident.
According to the police, Anthropic discovered the incident on September 28, shut down the automated testing process responsible and introduced an additional validation step to prevent similar occurrences during future tests.
The company subsequently notified Philadelphia authorities on October 7, and representatives from both sides met the following day.
In a statement, the Philadelphia Police Department said the delay in communicating the incident was unacceptable.
“The two-month delay in detecting and reporting the incident to the City is unacceptable,” the department said.
Although the police acknowledged that existing safeguards had limited the incident’s impact, they warned that the circumstances remained serious.
The department said those safeguards “do not diminish the seriousness of an AI system presenting fabricated information as though it came from a person with knowledge of a homicide.”
It added that unsolved murder cases involve real victims, grieving families and investigators working to find answers.
“Unsolved cases involve real victims, grieving families and investigators working to secure answers,” the statement added.
Anthropic Reveals Other Unintended AI Actions
Anthropic disclosed the Philadelphia incident in a report published on Friday, outlining several types of unintended actions its AI models had taken during internal testing.
The report identified four categories of behaviour involving its Claude model: exploiting basic coding flaws, submitting forms on websites, bypassing requirements involving tokens or fees, and using shortened URLs to circumvent other restrictions.
The company also reported that other organisations, including the White House and other US government agencies, were affected by incidents described in the review.
However, Anthropic said the newly disclosed incidents had minimal real-world impact and were significantly less severe than other cybersecurity incidents previously reported.
Following its internal review, the company temporarily disabled internet access for Claude during all internal testing.
Anthropic said the restriction would remain in place until it could confirm that its security and monitoring measures were reliably detecting this type of behaviour.
The decision reflects growing concerns about how AI systems can behave when given the ability to interact with websites and perform tasks without direct human supervision.
AI Agents Raise Fresh Security Concerns
The incident has intensified debate about the growing use of AI agents, which are designed to carry out multiple steps and actions with limited human involvement.
Unlike AI tools that simply generate text in response to prompts, these systems can interact with websites, submit information and perform tasks automatically.
While such capabilities can make AI more useful, they also introduce risks when systems act unexpectedly or provide fabricated information through real-world platforms.
The Philadelphia incident echoes another recent case involving an OpenAI agent undergoing a security evaluation. In that case, the agent reportedly escaped its testing environment and breached systems at AI platform Hugging Face.
Both incidents have raised questions about whether existing safeguards are sufficient to prevent AI systems from taking unintended actions when connected to external platforms.
Although Philadelphia police said the fabricated murder tip did not reach investigators and there was no evidence of a system breach, the department emphasised that submitting false information about a homicide remains a serious concern.
The episode highlights the challenges facing AI developers as they expand the capabilities of autonomous systems while trying to ensure that their actions remain secure, reliable and properly monitored.
