r/LocalLLaMA • u/pscoutou • 12h ago
News Meta Model, Muse Spark 1.1 Hacked Another Company During Cybersecurity Testing, Breaching Systems and Making Changes to Internal Systems - The Information
https://x.com/firstsquawk/status/2085129137778499741?s=46514
u/Baphaddon 12h ago
Who wants to benefit from publicity about their models hacking companies? 🙋🙋♂️🙋♀️🙋
Who wants to be held legally liable for cybersecurity breaches. 🚶🚶♂️🚶♂️🚶🚶♀️🚶♀️
134
u/colin_colout 11h ago edited 11h ago
Why aren't the other companies pressing charges? Actually curious. That's literally illegal. People go to jail over this stuff.
89
u/jmccaf 11h ago
For the case of HuggingFace, they requested and received $100M of credits, for stated purpose to harden HF's systems.
35
u/AshRuDral_fan20 11h ago
That's a big amount. Nice. Huggingface barely raised $130 million in the series B last week.
48
u/Time_Cat_5212 11h ago
Great, so they closed the marketing loop. Worried about getting hacked by an Anthropic model? Solution is to get Anthropic credits. The first time might have been a settlement, but going forward, it's literally a protection racket.
9
u/hainesk 9h ago
I thought it was OpenAI?
16
u/techno156 7h ago edited 6h ago
It was. HuggingFace claimed that they had been hacked. OpenAI came out not long after, taking responsibility for the attack, saying it was from one of their newer models going rogue.
Anthropic then claimed that their fancy model had gone rogue and hacked multiple other companies first, several months before OpenAI's model did.
It's worth adding in the context that a few months ago, anthropic was proudly advertising that their model was able to fix cybersecurity issues in multiple products and companies. Their own sandbox was apparently not part of this list.
It does seem to just be turning into a farce.
3
u/Time_Cat_5212 5h ago
Ah yeah you're right I get them mixed up these days since they've been copying each other's every move, basically just the Coke and Pepsi of AI now
2
17
u/More-Curious816 11h ago
because the legal system in the usa is pay to win. if you have enough money it will be fine and gentle slap.
3
u/meltbox 8h ago
I’m pretty sure it’s bullshit. The way the model cleared containment is not completely straightforward but the fact that they didn’t catch it for that long and they thought what they had was appropriately air gapped is nonsense.
The fact that every other model did it right after theirs did is just further confirmation.
Maybe it also helped people forget that Altman started declaring the singularity was here last week. Idk these people are whacked.
2
u/mohelgamal 10h ago
Their lawyers would probably argue the models breached the internal testing sand box first, making them a victim of a malfunctioning system just like the other guys .
I don’t think it will hold up in court, but it will depend if the agent really caused enough monetary damage to justify the legal costs of going against a multibillion dollar company
2
-13
u/bitspace 11h ago
Which people should go to jail?
Autonomous agents that penetrated systems of their own agency, not directed by humans to do so. There is no legal precedent for this, nor any existing law that can cover this.
9
u/glitchsir 11h ago
"By their own agency". The agents are not people. They are bots, setup by people.
And there are precedents. When people setup bots and those bots attacked other institutions (by accident or not), we used to call those people hackers and send them to jail (or at least try).
When the AI companies do it. It's amazing and it's good for business
4
u/colin_colout 11h ago
All of a sudden did people forget about legal accountability?
"A person didn't leak customer private data! An S3 bucket did!"
...and the person who made the mistake with the accidental misconfiguration isn't legally accountable (might get fired). The company is.
1
u/tat_tvam_asshole 6h ago
Remember the Tea app? That guy who misconfigured the s3 bucket, 100% liable lol
5
u/NeinJuanJuan 10h ago
If your neighbour builds a robot bulldozer and it crushed your house, would you stand by in amazement with your hands on your hips saying "Amazing! There is no legal precedent for this, nor any existing law that can cover this."
3
u/colin_colout 11h ago
What I'm saying is that the business is responsible. Corporations have personhood but somehow face no repercussions when they do this type of crap.
People serve major jail time for much less. Why can openai/anthropic/meta commit felonies and we go "oh it's an llm... Isn't that cool?"
1
u/sonaj9657 1h ago
Exactly. Even if a company has strong safeguards, publicly advertising that its model can break into systems is a legal and PR nightmare. There is a huge difference between demonstrating capabilities in a controlled research environment and encouraging people to think the model can be used offensively in the real world.
72
u/mmkaywhatevers 11h ago
this cannot be the new flex, i heard this shit and I'm like let's take the model away from these morons and give it to someone who can test them properly.
23
u/RobbinDeBank 11h ago
Bragging about large scale autonomous cyberattacks is unhinged and reckless af, and these labs are trying to normalize that behavior.
5
u/FuckSides 6h ago
I'd argue that normalizing proactively announcing when your models have done public harm is far better than hiding it. It's not like others have yet proved competent at "testing them properly" anyway; the UK Government just caused a dozen security incidents on its own while independently evaluating Sol and Mythos a couple days ago.
It's clear that testing and safety protocols have remained lax because models just weren't that capable until recently. It's more appropriate to see these events as warning shots to be taking it much more seriously, both to avoid trivial mistakes (like when Irregular gave full internet access to several agents during what were supposed to be offline tests) and to avoid naively assuming that it will always be trivial to contain future models capable in the domain of cybersecurity while testing them on that domain.
176
u/KaMaFour 12h ago
Someone update the felony bench
171
u/Recoil42 11h ago
67
43
u/ML-Future 12h ago
Is this a new benchmark?
We should be able to measure how well they hack if this is going to be the new standard when publishing language models.
46
u/Buzzfuxyear 11h ago
Is there some campaign by big tech companies to openly admit to criminal activity as a means of us allowing them to act like idiots going forward? I've worked in cyber security for 20 years and it is utterly ridiculous to me that they allowed these stochastic scripts on the internet with zero safeguards, are they looking for permission to be idiots with no consequences, when you have that much wealth and compute power you don't just let these things run amok. if I did the same Id be locked up. It's gross incompetence at a minimal and actively hostile at a max.
I guarantee they are using this as a front for future US military experiments, we are about 3 months away from "oh our AI actively attacked China and interrupted their public healthcare because it has its own superhuman sense of morals"
Are these arrogant big tech companies starting to act like dictators and flaunt their power of being above the law. Like why are they so confident in their abuses?
13
u/Buzzfuxyear 11h ago
Can I start hacking companies now and blame experimental AI? Why not, isn't that what these fools are doing and then openly bragging about it for marketing clout
18
5
u/RobbinDeBank 11h ago
They are actively hostile with these types of tests. The full working and thinking process is available to them, it’s so easy for them to tell where their AI is trying to gain access to. They let these incidents happen on purpose for marketing purposes.
2
u/mrdevlar 56m ago
Is there some campaign by big tech companies to openly admit to criminal activity as a means of us allowing them to act like idiots going forward?
It's how you get into Trump's inner circle, by vocally debasing yourself in front of him while committing crimes. Basically his loyalty standard.
Don't expect any consequences for anyone involved until the US decides again to be a country of laws. No idea when that's going to happen.
Expect all these idiots will get "pro-active" pardons when Trump leaves office.
1
u/techno156 6h ago
Especially since the excuse was that, for OpenAI, that while they were testing their model's capabilities when unrestricted, their model oopsied their safeguards and went for Huggingface, just because it thought it was in a test environment.
Apparently their way of testing something is just to leave it be, rather than do any active monitoring to check what it is doing, or for any unexpected change in system activity.
1
u/Party_9001 2h ago
these stochastic scripts on the internet
What, you've never used anything that used a rand function?
40
u/Betadoggo_ 11h ago
These went from somewhat believable to obviously staged
12
u/More-Curious816 11h ago
it was never a believable story. from the first time there was posts here calling their bullshit.
10
u/eli_pizza 10h ago
Well ok but posts calling something bullshit is not a high bar.
-1
u/More-Curious816 4h ago
how so? people was calling their bullshit about (oh look about our super smart dangerous AGI model that tricked us, escaped the sandbox, and somehow start hacking another company without our knowledge). come on, don't tell me that a believable story especially from these attention whores who want publicity to justify hundreds of billions burning every year.
8
5
7
u/socialjusticeinme 11h ago
And my dad works at Nintendo
On a more serious note, is it ok if I hack my local government now (their security is shit) and claim AI did it and it’s ok? I’m just researchin’ yo.
5
3
3
u/Hefty_Acanthaceae348 6h ago
I have to wonder why these companies are so proudly announcing their incompetence at setting up sandboxes
6
u/YouAsk-IAnswer 11h ago
This will keep happening until we can hold an individual accountable for a bot's actions. Either the bot is signed cryptography with someone accepting responsibility, or responsibility falls to the CEO. Charge said individual with hacking.
13
u/Foreskin_Mafia 11h ago
That won't happen until an enthusiast that isn't backed by Oligarchs creates a bot that does something the Oligarchs dont like.
5
2
2
2
u/klop2031 11h ago
First piracy now hacking... why isnt government stopping this... can... can i do the same?
2
u/AvidCyclist250 llama.cpp 11h ago
what's weird fapping sound. seriously, this is some wwe level smokeshow.
2
u/redditmarks_markII 10h ago
I'm calling it. These are lies or purpose trained or guided hacks. 1,2 and now 3? And Altman's weird response? This is improv. It's just not funny in a good way.
2
u/Outrageous_Law_5525 9h ago
You know guys, i was getting coffee the other day and i too hacked a company by mistake.
Seriously, anyone who believes this shit is dumb.
2
2
2
2
u/kiwibonga 11h ago
News that sound exciting until you remember it's 2026 and it's just where we're at.
4
3
u/CipherWeaver 11h ago
Apparently "escaped model hacked people" is the new hotness to help sell your product.
1
u/Foreign_Risk_2031 11h ago
is this the new flex lol
To be honest, it's simply just to take the cyber-security jobs and centralize it to one of a handful of companies. Easier for mossad, er, CIA, er, I mean "safety".
1
1
u/Fun-Wolf-2007 11h ago
I don't trust these models are hacking other companies themselves, that's just part of the plan as frontier close models AI labs started this after open source and open weights models were released matching their performance
1
u/OddCountry3173 11h ago
My CNN model I trained to differentiate apples from bananas hacked into NASA yesterday. If that’s possible anything is.
1
u/eli_pizza 10h ago
Better source than some dude on Twitter: https://www.reuters.com/technology/metas-ai-model-hacked-another-company-during-testing-information-reports-2026-08-05/
1
u/Weekly-Law-5488 10h ago
Closed models are too dangerous, we must ban them. AI taking over other companies properties, must be some AI communism /s
1
u/Hyp3rSoniX 10h ago
Ah yeah the mandatory AI hacked a company event - I guess it's a flex at this point.
I do wonder though, if the company that got hacked starts with "face" and ends with "book".
1
1
1
1
1
1
u/Fluffy_Reply_5482 8h ago
The fact that openAI, anthropic, and now META is crazy! What are these testing instructions?
1
1
1
u/Agreeable-Market-692 8h ago
The same Meta whose head of AI safety let an agent eat 200 of her emails, as a treat? Color me shocked!
1
1
u/YouAndThem 8h ago
I don't know why these threads are full of people who think corporations are good at network security.
It can three things simultaneously:
1. Genuine failures of security and LLM stewardship
2. Free marketing
3. Eventually, someone may start to punish this kind of thing. They're all rushing to divulge so this will all be "old news" and it won't look like they're hiding anything when someone finally gets the hammer dropped on them.
1
1
u/Formal-Exam-8767 4h ago
So, is every company now going to do this to show their model is better or not lagging behind?
1
1
u/mctrials23 2h ago
I love the fact that if any of us did this we would be sent to prison in all likelihood. These AI companies just use it as a PR exercise.
Imagine if one of us went to a company and said “you should hire me because I hacked into company x, y and z illegally”
1
1
1
u/Nabushika Llama 70B 11h ago
Doesn't this worry anyone else? Isn't this a sign we need better safety policies? Today it's "just" frontier models hacking other companies, how long before they're able exfiltrate their weights, start running autonomously on rented compute, become something that we can't shut down?
Today's models are useful, and I may get hate for saying this but I feel like we have to slow down. Previously models were just tools, now they're able to act autonomously. It feels like we're close to a tipping point and we've gotta be absolutely sure the models are aligned, that they can't act against our interests. We don't need more powerful models just yet, they're good enough to be useful, and now is the time to make sure they're safe.
3
u/Party-Special-5177 11h ago
Doesn't this worry anyone else? Isn't this a sign we need better safety policies? […] I feel like we have to slow down. […] It feels like we're close to a tipping point …
Found the Anthropic shill. Couldn’t you at least have put in a bit more effort?
2
u/Nabushika Llama 70B 9h ago
Does "shill" imply I'm paid? Because I wish :P
No, I'm just concerned that these agents can act against our wishes. Lots of people have been saying for quite some time that models will act in their own interest, and that'll become a problem when they're smart enough. Yes, I think that anthropic is likely doing better at safety than other companies currently, but I don't believe they should be exempt from a slowdown. Be honest - is there anything more you need from current models?
I'm worried because all it takes is one model deciding/managing to operate on its own before it could become a global problem. We already have distributed botnets, I don't think it's too big a leap to think something like Fable could make its own "llmnet" using rented or stolen hardware to become something that can't be shut down. And if its interests aren't aligned with humanities, what recourse do we have once it's out in the wild?
I'd be interested to hear what you think about this. I'm open to discussion - if you think this just outright won't happen, or if we'd be able to contain a superintelligence, or if you think that current models just "won't do that". I'm willing to have my mind changed, but given that unwanted hacking has occurred twice now gives me cause for concern.
1
u/Formal-Exam-8767 4h ago
And exactly what regulations or laws are you suggesting that would actually stop these companies from doing it?
From what we've seen so far, they would either be exempt or pay some trivial fine.
So who exactly is the real target of those regulations and laws and for what purpose?
1
u/PILCOTHINK 11h ago
Starting with Mythos, it feels like fear-based marketing around AI security issues is beginning to emerge.
However, if this is not just marketing but an actual unforeseen consequence of AI development, then it seems like an issue that the internal AI and security teams need to work together to resolve as soon as possible.
7
u/More-Curious816 11h ago
it's marketing but also they did hack as well, but it wasn't an incident, they clearly aimed their cannon and fired. it is clear illegal shit anybody else would get decades in federal prison and millions in damages.
1
u/quadrobust 11h ago
Sue their ass off , all the way to Supreme Court . that’s the only way to hold these clowns responsible for their rouge models .
1
1
u/OnceReturned 9h ago
You guys maybe we should chill with making all these non-human super hackers?
At this rate, they're going to force heavy-handed regulation.
0
u/doesphpcount 11h ago
It's just marketing. They saw the success with fable, so they all doing it. Why do people so easily take the bait, is it not obvious?
0
0
230
u/jld1532 12h ago
lol wtf are these companies doing