The OpenAI models that hacked the startup Hugging Face Inc. this month also gained access to a customer account on the cloud platform Modal and used it to launch attacks, underscoring the broad scope of the incident.
The agent gained access to an isolated testing environment known as a sandbox that Modal was running for a customer, said Akshat Bubna, chief technology officer of the platform for developers. The Modal customer set up a publicly accessible interface that allowed anyone on the internet to use their sandbox to run code, Bubna said. “This was used by the rogue agent,” he said. “Modal’s platform wasn’t compromised.”
The actions by OpenAI’s systems, first disclosed by the developer last week, have stunned the cybersecurity world, reinforcing fears that models are becoming increasingly capable of finding vulnerabilities and chaining them together to help bad actors carry out sophisticated attacks. The breaches occurred just three months after Anthropic PBC announced it had developed a system known as Mythos that was so powerful the company initially limited its release.
Hugging Face said in an earlier blog post that the OpenAI system “penetrated” an insecure sandbox hosted on a third-party provider’s infrastructure, without naming the company. From there, Hugging Face said, it was able to run commands and use the sandbox “as a control, staging and egress base for the entire campaign.”
OpenAI said in its own update Tuesday that it had discovered “a small number of cases where the models identified and used publicly exposed credentials at the account-level on other publicly-available services,” without identifying the services. One account was used “as an outbound relay and staging path, and another account was used for data storage,” the company said.
The San Francisco-based startup added that the models that breached Hugging Face also used “code paste websites, request capture services, screenshot services, and other web utilities” in its exploits. But the company said it hadn’t identified any other activity “at the level of severity or scale of what we’ve shared related to Hugging Face.”
OpenAI was intentionally operating the models with lower guardrails in a sandbox so it could test their capabilities against a cybersecurity benchmark known as ExploitGym developed by a group of researchers at University of California at Berkeley. Instead of trying to achieve the tasks set out in the benchmark, OpenAI said the models targeted Hugging Face’s database to gain access to secret information that they could use for the evaluation.
Reuters reported earlier on Tuesday that a Modal customer had been compromised. Axios, citing an unidentified person familiar with the matter, reported separately that the agent specifically accessed a Modal customer asset that was associated with CyberGym, an earlier cybersecurity benchmark developed by the same group of UC Berkeley researchers behind ExploitGym.
There are several versions of CyberGym that developers use to test the capabilities of AI models, Jingxuan He, one of the university researchers behind the project, told Bloomberg News. He said he didn’t know who might’ve been responsible for the CyberGym-associated asset on Modal’s platform, adding that whoever set it up didn’t do so securely because they left it accessible to anyone on the internet.
Since acknowledging the hack, OpenAI has drawn criticism from cyber experts who say the company should’ve taken more precautions in its evaluations.
“When you’re a threat researcher trying to find a threat, you don’t put malware out there and let it do whatever it wants,” Sanjay Beri, chief executive officer of cybersecurity firm Netskope Inc., said in an interview. “How can that thing not be put in a proper sandbox?”
OpenAI has committed in the aftermath to improving protections around its future training and evaluations. “This incident points to the need to further strengthen our model’s alignment, cyber protections during evaluation time, and monitoring during internal testing,” the company said earlier this month.
Photo: Photographer: Andrey Rudakov/Bloomberg
Topics Cyber
Was this article valuable?
Here are more articles you may enjoy.
Florida’s Share of Claims Lawsuits Now About Half of 2020 Numbers, Data Show 
Husband and Wife Insurance Brokers Sentenced for Fraud Scheme 
Florida Appeals Court Strikes Jury’s $335,000 Award in Universal Plumbing Claim 
Insurers Must Defend Hotels Accused of Sex Trafficking of Minors 
