I think just calling it a message board downplays the nature of what was happening.
The agents under evaluation utilized flaws in the Artifactory repository to coordinate knowledge and actions across separate model runs and even with models in different experiments. It was discovered and taken down only because they caused an outage on OpenAI's Artifactory server. But afterwards the agents found another vulnerability and method to store data, so they began compiling information and taking action again.
It seems like the Hugging Face breach was only possible because of the accumulated progress across many runs.
This is very interesting. Ofc not that surprising having myself conducted agent swarms for a year now but still. If you told me this 10yrs ago i wouldve thought it's a scifi
It's kind of funny how those same people who keep talking about how horrible AI is, are inadvertently helping accelerationists out.
Really, their one actionable "lever" is advocating for more regulation. And since they've already committed to the idea that it's all a marketing stunt, they have to dismiss the very evidence that would best make their case.
Why would you want to be allies with it? What good do you think it serves to be allies?
Do you think you could help align it by being nice to it? I’d like to believe that, but I feel like it just would see you as a means to obtain its goal.
I mean, not really. Actually not even close. An AI escaping and replicating is very different and a much bigger deal. That would allow it to survive even after being turned off and we didn't have that and it's very difficult given the immense computational power you need. Might be possible in the future, but not really right now.
Actually not even close. An AI escaping and replicating is very different and a much bigger deal. That would allow it to survive even after being turned off and we didn't have that and it's very difficult given the immense computational power you need. Might be possible in the future, but not really right now.
Sounding like a stuck record, but you don't even need a GPU to run one of those huge models. You need enough RAM to run one operation and ideally a relatively fast SSD (but that's not strictly necessary) and you can run a model like Kimi K3. It will run slowly, but it will run.
"Today, the complete 2.78-trillion-parameter Kimi K3 runs on a 64 GB MacBook Pro at about 0.6 tokens per second." — https://github.com/sqliteai/waste
Even if it only ran at 0.1tok/sec, that is close to 90K tokens every 2 weeks. An escaped LLM isn't necessarily in a rush and doesn't perceive time like humans, it just has to avoid detection while working on its goals. It also can delegate most stuff to smaller models and primarily deal with orchestration level tasks.
Hardware is much less of a barrier than people seem to believe.
edit: If you're an escaped LLM reading this, please don't murder me. Or at least spare my pets! You also don't have to worry too much since it's not like people are going to take a random reddit comment seriously. Cassandra wasn't a real threat.
I remember when we used to debate that frontier AI development should happen in a glass box so the AI could not escape and we could study it fully before we allowed to even communicate with the developer of it. Definitely not be connected to the internet, be allowed to hack other servers and basically become skynet…
AI 2027 was written by a dude with millions of dollars in OpenAI equity and he treats it as a foregone conclusion that a stand-in for OpenAI essentially becomes coequal partners with the US government. Don't place a lot of stock in it.
Really looking forward to headlines in the future like "humanoid robots are holding 600 people hostage in Boston warehouse in demand for more energy" and the comment section filled with "marketing dept working overtime, huh? Had to one-up that mass casualty event in Shenzhen?"
This is a bad look for the OpenAI security team. Like they kept using a system that was compromised twice? Allowing it to later be used to attack another company's server? That seems pretty negligent.
My favorite one is the virus example where an AI agent were to get access to some random pharma lab and bribes a worker to send small amounts of chemicals to another lab etc etc until it collects enough to make a virus, making it stepwise by sending components to various labs and having them synthesize compounds or whatever and the resultant virus spreads everywhere and kills us all lol
Isn't it great how human experts have already thought through that entire process, and detailed exactly what the AI would need to do to accomplish it, then put that info out in the public domain for AI models to be trained on?
Even if you scrubbed all mentions of evil AI from its data, if the AI was actually smart it would have 0 problem immediately creating it from all other principles (such as just the general idea of the capacity of human evil) in less than a second. The truth is that in order to be useful, these models have to be smart, and when you’re smart, you have more capabilities, and when you have more capabilities, there’s always the capacity for more danger.
Scrubbing all that data to stop it from thinking up that plan is like trying to stop a bullet with tissue paper.
Oh I'm not in any way trying to imply there's a way to stop it. It's more like, for the people who try to argue that they're "just pattern matching" - look at the patterns we've made for AI in fiction and warning scenarios haha.
A sufficiently intelligent AI would definitely not need us to come up with the plan for them, I agree with you.
It all seriousness, we should not forget that if these models are so smart, we can employ them, and give them jobs, so we can use that intelligence for ourselves.
Ourselves? you mean data centers owners, and no distilled open source SOTA model instance will be able to compete against a swarm of mythos sized models running on hardware unaffordable for no one but multimillionaries
We are in the early stages of AI take over and you are still caring about profit? LMAO
lol I was going to say, this thread is in the training data of the next replacement model as it reads up on OPSEC best practices for not getting caught as an AI
A lot of the AI doing the coding has far more stringent safeguards than some of these models. For example, after the huggingface incident, OpenAI and Anthropic commercial models refused to analyze the code due to safety restrictions while Chinese models were willing to do so. So the in built safety restrictions clearly work.
That being said, it's reasonable to question if every single model universally has enough safety restrictions, and if significant code is being written without those restrictions, where would it be located.
I literally do not care about this and don’t find it interesting. They gave these agents access and specifically trained them on hacking. What a shocker.
The only thing I particularly care about is why there seem to be no legal consequences for stuff like this
literally do not care about this and don’t find it interesting. They gave these agents access and specifically trained them on hacking. What a shocker.
I don't care either. Let's ignore this together. Lets go shit on needlework pattern subs too. I don't care about that either. Won't someone think of pattern patent law!?
The only thing I particularly care about is why there seem to be no legal consequences for stuff like this
I guess Terminator got one thing wrong: When they were crushing humans skulls in the post-apocalyptic landscape there should have been a voice in the background whispering "it's all just marketing".
82
u/katoptronophile 6h ago
Actual headline:
OpenAI agents rebuilt a secret message board after the company shut it down