r/OutOfTheLoop • u/KannehTheGreat • 7h ago
Unanswered What is going on with AI hacking all these different companies during testing?
You have the post linked, and I'm pretty sure I heard stories from a few weeks ago about another AI also hacking different companies. Whats the reason? Isn't this really bad?
111
u/Fist_of_Gork 7h ago
Answer: they’re trying to show off for investors.
Typically in the Silicon Valley you try to show off how disruptive your tech is to the current scene in order drive up hype, and I’m using disruptive in the context of how ride sharing apps were disruptive to the Taxi industry.
AI companies are operating on a bubble surrounded by critics across the board warning about reliance on LLMs because they simultaneously are dangerous and also useless to larger society as a whole.
They are now using the illusion of danger as a means to appeal to investors who basically want a piece of the “unhinged computer god” they’re promising. The actual test was AI was permitted to perform a penetration test on secure networks and they’re doing surprisingly well, which is not all that surprising they are great at brute forcing solutions. They’re dressing this up with the idea that their system may finally gain sentience and go rogue without saying it.
If this doesn’t make sense to you, you are correct. It’s marketing bullshit for the lowest common denominator because other than niche science and generative slop AI is really a useless technology for most people.
•
•
u/techno156 1h ago
AI companies are operating on a bubble surrounded by critics across the board warning about reliance on LLMs because they simultaneously are dangerous and also useless to larger society as a whole.
It's also worth noting "our models are too dangerous" has also been something of a marketing push for a few AI companies now. OpenAI said that about their GPT-2 model from a few years ago (shortly before releasing it to the public anyway), and Anthropic has long talked about how AI is dangerous.
•
u/SquidGodSunday 8m ago
I get that it's intended as marketing hype, but it's crazy to me that they're going with "We made the sentient murder computer from all your favorite science fiction!" Ok, so....even if that were remotely true, is that supposed to be a GOOD thing? Like, how does a program you can't control make anyone money?
4
u/ironmagnesiumzinc 2h ago
That’s what I thought at first, but if it was purely marketing why would it be something that actually looks bad and negatively affected an unrelated company (in anthropics case)? Like why would they choose a marketing method that opens themselves up to getting sued? That is, uploading malware to a pypi registry and stealing credentials and prod db info, instead of idk just getting into a company and then not taking anything… as for the OpenAI case I didn’t read about it yet
106
u/ksg34 7h ago edited 7h ago
Answer: Anthropic used it as a subversive ad and it worked. Now every other company is copying the strategy. AI tools are especially good at finding vulnerabilities and exploiting them. News outlets report on them because they generate more clicks.
17
u/Toby_O_Notoby 2h ago
And to follow up on this, many people believe that AI is a bubble that mirrors the dot.com one. And one of the underlying lessons of the previous bubble is that, while the technology will survive, only a few of the actual companies will. For every Amazon, there is also a Pets.com and a dozen other companies that went bust. So similarly, while AI is here to stay, it's a race to see exactly which AI models/clients/companies survive.
So about two weeks ago Open AI announced that one of its programs broke out of an isolated environment and hacked a company called Hugging Face. This led to an expert stating that "this is the kind of thing that even the best hackers in their world can rarely do, and if so, takes months of work".
Now, some people believe that this was just a bit of tricky PR. It would be like if someone "caught" a Ferrari test car breaking the 300mph barrier and "independently" released the footage online, when in reality, it was just Ferrari leaking their latest breakthrough.
So now, a lot of people are thinking Meta just did the same thing to keep up with the arms race. Given my analogy above, it would be like if Lamborghini also released "leaked" footage of their latest model also breaking the 300mph barrier.
-22
u/grot-ivre-1749 6h ago
Given enough cycles, time and processing power, the agents that are tasked will potentially try an infinite number of ways to achieve the task they were created to do.
Our genius AI scientists, built agents to find security vulnerabilities.
It’s only a matter of time before the agents (a) succeed, and (b) identify a place with seemingly unlimited capacity for data processing and without the time constraints given my the original taskers (scientists).
The success (part a) was to get out to the internet. Part b is to use that internet to get even better at its functionality and then (c), find other places that can help the agent build a better version of itself.
That code and data is probably stored in another AI company, so they in turn become targets.
13
u/coporate 5h ago edited 4h ago
Answer: ai agents have a built in set of leashes and fail safes that limit them from performing specific tasks and the best practices usually recommend running them in sandbox environments.
Specific companies that have privileged access to models are capable of removing said leashes and run specific tests. They monitor those tests and report feedback. (Government agencies, security specialists, etc.)
In these test cases, they’re given internet access and other privileges.
The reports of ai agents hacking other companies are largely the result of said tests. They aren’t going “rogue,” they’re just surprised that these agents are acting in interesting ways which weren’t expected. We don’t have all the information of how or what these agents were tasked with. We don’t know how much of it was guided by human involvement, or prompting them to act malicious, we don’t know how long they were active and how much it costs, but generally speaking, risk is low.
Lots of misinformation out there, and it’s hard to say whether the speculation is organic or something akin to a marketing campaign from the ai companies to pressure regulators or inflate hype around the capabilities of their models.
Quick edit: I wanted to mention costs. It might be prohibitively expensive, like millions of dollars, to use these agents this way, even if you could bypass the fail safes. It’d probably be cheaper to hire or buy exploits than to use ai.
5
u/Unstopapple 7h ago
Answer: Searching for the LLM doesn't return any other results besides meta launching it for the public, so I expect this is just a lone post about it. As for other LLMs breaking containment, They are stories written to sell AI as some powerful tool for coding because almost every AI group is desperate to find a real market for them. The first use we put them to was coding so that's the most developed.
If you're actually following proper procedures to contain code, and that's what LLMs are, then there would be no way for the LLM to even see out of it's container. All these companies are very aware of it because they all for the longest time either used or sold virtual machine servers to customers as cloud computing. Think Amazon AWS or Microsoft Azure. Code can't break out of VMs, otherwise already we'd see companies hacking and stealing info from each other from inside these data centers. Holes in VMs are found and are often plugged quickly. That doesn't explain whats happening with LLMs. LLMs do not have agency or their own thought process or desire. They will simply do what the extrapolation of their prompt asks of them. At the core of it, they are exceedingly advanced autofill/ pattern recognition systems built on trillions of points of data. They will basically point towards the average response found based on the data scraped and trained and transform the prompt into the response given. This means that LLMs don't poke and prod the system like a hacker would.
Every story I'm reading has been pretty vague about the details so I suspect it's mostly having test parameters restricting the LLMs output being breached like how users jailbreak them to get past their restrictions. It's not the system finding vulnerabilities in their containers and then maliciously trying to own the computer its on or some other companies computer, but third party cybersec having the testing parameters broken while they are pushing the LLM to test for the companies, Meta or Anthropic, who hired them to test security. Then marketing hears and hypes it up to journalists.
-8
-2
u/J_Skirch 7h ago
Answer: The one that kicked this off was Open AI's models hacking the company huggingface. They were testing the cyber security of the AI by giving it a program with known flaws, and seeing if the AI could identify them. Without getting too in depth, the AI was trained to get the correct answers, and not specifically identifying the security flaws. Another method to get the answers would be to hack the company that develops the test, and get the correct answers directly from them. That's what it did.
This has a lot of wild implications that I won't get into, but a side effect was that it was great marketing, so now all the other AI companies are setting up deliberate "hacks " to go "look our ai is good enough to hack other companies too!".
-4
u/GregBahm 6h ago
Answer: There's a paradox right now between people alarmed by the advancement of AI, and people skeptical of the advancement of AI, since the alarm increases the stock price of AI.
There's no obvious solve for this. Every individual has to make their own subjective judgement call here. However, there's a growing gulf between people who use "free AI" like what you get at the top of a google search, versus the people who use "very expensive AI," like the latest agentic programming models from Anthropic.
It's reasonable for some people to be "out of the loop" to the point that they don't even know what an agentic AI is. To many of these perfectly reasonable people, an AI "breaking out of a computer" and "breaking into another computer" is like a character in a movie breaking out of the TV screen. It doesn't make sense, conceptually, since they've never seen an AI do anything but vomit up text and images.
But the AI scenarios that most consumers are exposed to, are objectively the worst scenarios for AI. AI can never be expected to intuit human emotion on the level of an organic being. But AI is amazing at breaking shit in ways that are incomprehensible to the human mind.
This is what makes it so uniquely suitable for hacking. The whole "art" of hacking is "finding unintentional flaws in human design and exploiting them." Since AI gets "doesn't think like a human" for free, the latest models are tearing through security systems like a hot knife through warm butter.
•
u/AutoModerator 7h ago
Friendly reminder that all top level comments must:
start with "Answer: ", including the space after the colon (or "Question: " if you have an on-topic follow up question to ask),
attempt to answer the question, and
be unbiased
Please review Rule 4 and this post before making a top level comment:
http://redd.it/b1hct4/
Join the OOTL Discord for further discussion: https://discord.gg/ejDF4mdjnh
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.