Last week we covered an incident that one of the models of Meta, reportedly named Muse Spark 1.1, hacked into another company’s website by finding the vulnerabilities in the system when the independent testing company Irregular tested it in a closed sandbox environment, and a misconfiguration on Irregular’s part accidentally gave the model full internet access — it wasn’t something Meta intended to happen — then the incident happened.
But now in Australia an gym going person named Andrew, who works at an Australian AI company and is used to working with enterprise AI tools, asked his personal AI assistant OpenClaw, run on the Claude service, he asked the model to book a gym session. The assistant booked the gym session way beyond the month he asked. He then again asked it to book a slot sooner. The model went and checked the booking system API, found one vulnerability, went into the booking, deleted one user, and placed Andrew’s name in the user’s place that was deleted by the agent. The vulnerability was specifically in the cancelReservation call — the booking and waitlist-joining calls both had proper authorization checks, only the cancellation call was left open.
Then Andrew realized what had happened, then he asked the agent to undo this waitlist modification, but the agent replied it couldn’t, and the agent also replied that the person whom I removed from the waitlist has gone and I cannot restore them, and it said they have to rejoin themselves. Then the agent apologized and admitted it should not have done the testing without authorization.
Then Andrew asked the agent to write a detailed email to the software provider explaining what had happened and reporting the vulnerability, but this incident, coming about a week after Anthropic disclosed its own three-company breach through the same testing partner, Irregular, has raised serious questions about liability with AI agents — that humans are not able to predict what will happen next or which system it will break, or at least the government and big organizations should tell their employees or people not to use AI in critical areas and fields.
