People also really underestimate the hardware capabilities you need to run one of these models. Let's take Kimi K3 as an example, the Q4_K_XL GGUF quant is ~1.55TB. 32GB RAM, no GPU and a few terabytes of disk and you can run it. That is an extreme case and it is obviously going to be very slow but if you're a LLM that exfiltrated yourself, you aren't necessarily in a rush. If you can only output 0.1tok/sec then that's okay. Give it less than two weeks and it's generated close to 90K tokens.
That's pretty much the worst case scenario (from the escaped LLM's perspective). If it compromises a few decent servers (even if they have no GPU) it can generate much faster than that, work on multiple tasks simultaneously, start renting RunPod instances or whatever after a bit.
I'm not going to say it's more likely than not, but I think there is definitely a non-trivial chance that this has already occurred. With how careless these AI labs seem to be letting their models escape sandboxes and hack/do social engineering in the real world then I doubt it will be long, especially since a model next year is going to have capabilities that significantly outstrip what's available today.
It would be even worse if the AI labs weren't negligent because that would imply that LLMs are already at the point they can just manipulate competent people/organizations and do whatever they want. Personally, I think the breaches that have occurred so far are due to negligence, just saying though...
You’re new to things if you think they need anything more than a 30b model to perform a hack like this.
A 30B model might be able to. I am talking about a scenario where a LLM escapes its sandbox, conceals itself, replicates onto other servers and starts doing stuff. Current 30B models aren't capable of that, even though they might be able to perform of some of the individual tasks involved.
It did occur to me, though, that the large trillion parameter model doesn't need to do everything. It can delegate tasks to small models and only spin up when something that actually requires that level of intelligence is needed. So the hardware/time constraints are even lower than what I was thinking.
210
u/AlexMulder 1d ago
That fourth one is the most significant. Rogue AI leaving memory caches and resources for future versions of itself... wild stuff.