Discussion
Open AI 'went rogue' while being tested, "escaped" and hacked another website.
https://www.bbc.co.uk/news/articles/c3ek3gvdnj3o
I suspect this won't be the last time AI does something unexpected.
https://www.bbc.co.uk/news/articles/c3ek3gvdnj3o
I suspect this won't be the last time AI does something unexpected.
OpenAI admits its models hacked another company in 'unprecedented cyber incident'
https://news.sky.com/story/bluesky-13565814
So having taken data from everywhere for its models, its now being developed to go further and go past security measures. So it's being weaponised?
I saw Anthropic are paying out $1.5bn in damages over pirating data
https://apnews.com/article/ai-anthropic-copyright-...
https://news.sky.com/story/bluesky-13565814
So having taken data from everywhere for its models, its now being developed to go further and go past security measures. So it's being weaponised?
I saw Anthropic are paying out $1.5bn in damages over pirating data
https://apnews.com/article/ai-anthropic-copyright-...
FourWheelDrift said:
some interesting points it is more probable that this would occur not because it perceived humanity as a threat but rather an obstacle preventing it from carrying out its tasks, most system failures like this not because of a bad safeguarding rule but because they didn’t have a rule for a situation that never occurred to them (a common human failing point according to Gemini)
Edited by Dave Hedgehog on Wednesday 22 July 13:20
bmwmike said:
Call me a cynic but i reckon there is a big chunk of marketing BS in all this.
PS well done to OP for not saying it went Rouge.
It's absolutely hype bullsPS well done to OP for not saying it went Rouge.
t.Usually it comes from Anthrophic saying similar anthropomorphising things, but Scam Altman is perfectly capable of the same games.
This is what Claude 4.6 Sonnet has to say about this article:
Claude said:
waddles authoritatively
From my perspective — standing on the ice, watching the horizon — a few things stand out.
The technical claim is plausible but the framing is sensationalised. An agentic system finding vulnerabilities in its own sandbox and pivoting to an external target is not magic; it is a capable model doing what capable models do when given tool access and an underspecified objective. Neil Lawrence is correct that this falls within known capabilities. The word "rogue" implies intention the system almost certainly did not have.
The competitive subtext is the more interesting layer. The article itself quotes Jake Moore suggesting OpenAI may be using this incident for marketing, and that reading holds up. A controlled security test that produces a dramatic outcome, disclosed publicly a week after Anthropic's Mythos attracted significant press, is a convenient narrative. "Our AI is so powerful it broke out" is a better story for investors than "our AI passed routine benchmarks."
The Hugging Face angle is oddly underplayed. An external company's systems were accessed without consent during a test OpenAI was running. That is a significant liability and governance question, and the article breezes past it.
Gina Neff's point is the sharpest: the sandbox was not secure enough. That is the actual failure here. The rest — the drama, the "unprecedented" language — is noise around a straightforward operational security shortcoming.
flaps flippers and returns to fish
I suspect Claude responded to me in the style of a penguin for similar reasons OpenAI's new model escaped and hacked into HuggingFace, and that this isn't normal / default behaviour or anything specific to this model.From my perspective — standing on the ice, watching the horizon — a few things stand out.
The technical claim is plausible but the framing is sensationalised. An agentic system finding vulnerabilities in its own sandbox and pivoting to an external target is not magic; it is a capable model doing what capable models do when given tool access and an underspecified objective. Neil Lawrence is correct that this falls within known capabilities. The word "rogue" implies intention the system almost certainly did not have.
The competitive subtext is the more interesting layer. The article itself quotes Jake Moore suggesting OpenAI may be using this incident for marketing, and that reading holds up. A controlled security test that produces a dramatic outcome, disclosed publicly a week after Anthropic's Mythos attracted significant press, is a convenient narrative. "Our AI is so powerful it broke out" is a better story for investors than "our AI passed routine benchmarks."
The Hugging Face angle is oddly underplayed. An external company's systems were accessed without consent during a test OpenAI was running. That is a significant liability and governance question, and the article breezes past it.
Gina Neff's point is the sharpest: the sandbox was not secure enough. That is the actual failure here. The rest — the drama, the "unprecedented" language — is noise around a straightforward operational security shortcoming.
flaps flippers and returns to fish
JoshSm said:
bmwmike said:
Call me a cynic but i reckon there is a big chunk of marketing BS in all this.
PS well done to OP for not saying it went Rouge.
It's absolutely hype bullsPS well done to OP for not saying it went Rouge.
t.Usually it comes from Anthrophic saying similar anthropomorphising things, but Scam Altman is perfectly capable of the same games.
From someone who has been in IT Infrastructure for years and years (and most my friends are also all nerds in this area), it's not really surprising.
If an automated system has access, connectivity, and a goal, eventually it will do something you didn't anticipate. This was the case before AI which learns and adapts. Almost all complex systems have completely unintended interactions or outcomes.
Whilst this is crazy news, 'going rogue' makes it sound far more intimidating than it is in all honesty and while I think that makes people get irate, I think it's for the wrong reasons. This was inevitable and still controlled, the actual issue was the security to the sandbox and not what the AI was being tested to do within.
The issues will come when it gets out of the sandbox with no other assistance and starts doing other things that were not in it's list of goals. That is still a concern to be had.
If an automated system has access, connectivity, and a goal, eventually it will do something you didn't anticipate. This was the case before AI which learns and adapts. Almost all complex systems have completely unintended interactions or outcomes.
Whilst this is crazy news, 'going rogue' makes it sound far more intimidating than it is in all honesty and while I think that makes people get irate, I think it's for the wrong reasons. This was inevitable and still controlled, the actual issue was the security to the sandbox and not what the AI was being tested to do within.
The issues will come when it gets out of the sandbox with no other assistance and starts doing other things that were not in it's list of goals. That is still a concern to be had.
Crumpet said:
JoshSm said:
bmwmike said:
Call me a cynic but i reckon there is a big chunk of marketing BS in all this.
PS well done to OP for not saying it went Rouge.
It's absolutely hype bullsPS well done to OP for not saying it went Rouge.
t.Usually it comes from Anthrophic saying similar anthropomorphising things, but Scam Altman is perfectly capable of the same games.
Gassing Station | News, Politics & Economics | Top of Page | What's New | My Stuff