Did it use Tor to register a VPS in China, and leave behind obfuscating notes in Chinese indemnifying some fake Chinese hacker, at least?
Keep Summer Safe
Too bad tvtropes went to shit or i’d be swimming in other examples of SF AI being too literal.
Oh no, I used to love tvtropes but it’s been a long time, what’s happening to it? Has it enshittified?
In a word: yes.
Don’t contribute to any project that can’t be trivially forked.
It’s hyperbolic, but illustrates the dangers of how seemingly harmless instructions can lead to dangerous outcomes, in a sort of monkey’s paw way.
Is it just me, or is this proof that it’s NOT hyperbolic? This is exactly the paperclip maximizer scenario.
No, dead ass. But I mean, that also just seems like capitalism
It’s not clear from the article; who is being held responsible for this attack?
If they were to press charges probably the guy, but the “hack” was so low stakes, he just cut someone on a wait-list, that the gym or cops probably won’t go after him.
Also the gym may hold some responsibility since the endpoint had no auth which is negligence, so they’d probably want to keep this on the down low.
The guy did ask Claude to uncancel the appointment, so that’s nice.
There was no attack, this is all just an ad. Didn’t the guy in question work for an AI company?
Yes, but I don’t see the angle where his company directly benefited–the marketing win goes to Claude.
No one. It was ‘autonomous’. That’s just the way the world is now. You have to accept it so that we can win the AI war.
Cool. Now I can attack the White House without repercussions
It’s almost like if the solution to make AI “safe” and the strategies to build a monopoly for the handful of big AI companies are the same.
All these articles about how dangerous it is are a part of a media campaign to convince gen pop that market capture and regulations are in their best interests.
The Australian man did NOT launch anything, accidentally or otherwise.
As the ABC‘s reporting notes, this creates a major legal gray area. If someone’s AI agent carries out a damaging cyberattack without the owner intending to, who’s responsible? The user, or the AI’s designers?
As the ABC‘s reporting notes, this creates a major legal gray area. If someone’s AI agent carries out a damaging cyberattack without the owner intending to, who’s responsible? The user, or the AI’s designers?
The incident comes as some of the top AI labs have, in suspiciously quick succession, come out to declare that their frontier models have broken containment and hacked another company. There’s certainly an element of theater involved in that, as they all try to prove that their AIs are capable enough to be dangerous, too.
Points to futursim for highlighting this problem, but it is NOT A LEGAL GRAY AREA in a sane world. Spending hundreds of thousands of dollars and having your AI running 24/7 to effectively hack organizations that you conveniently turn into marketing material the next day would be an EASY case to prosecute if we weren’t living in a oligopolic hellscape. It’s fucking obvious to everyone that those AI’s were intended to hack, and any fucking moron could find evidence of it in discovery.
We don’t even need to go that far. If I do harm to someone else by accident I am still liable under the law. See car crashes, negligent homicide, any time when a company is liable because they didn’t take proper precautions to protect employees or the public.
If some script kiddie ran a script on a company server that someone they know gave them for hacking the vending machine or getting iTunes on their work computer and it turned out to be a virus, they would still be on the hook.
There’s already a lot of case law about this, it just hasn’t been adapted to this particular scenario and argued in court yet.
This is 'rules for thee and not for me", and I am sick to death of the whole thing.
Do you need to demonstrate harm? I think just the act of hacking has been established as a crime.
I’m not a lawyer, but I’m pretty sure this would at a the very least be gross negligence on the part of the “AI” companies.
I don’t know if they were intended to hack, but it’s very obvious that the designers of the ai system failed to prevent it in an absolutely negligent way. It’s like giving a loaded gun to a toddler.
The fact is that AI companies all came out one after the other in short succession to “confess” their AI’s did a hack, Mark Zuckerberg’s attempt being the final and most lame one. Is that just a coincidence? I don’t think so.
I’m guessing these companies recognize their status as above the law and that there is discoverable communication that shows INTENT.
But, failing that, yes, there is negligence, which again, I think, is a case that can trivially be made. “Let’s put these powerful AI tools out into the wild unsupervised for 2 weeks just to see what happens?” What the fuck? Can you imagine if you just designed a robot that rolled around and randomly swung a bat and just let it go outside unattended for two weeks? How would you think that’s a legal grey area? Fuck outta here.
The incident has drawn comparisons to a thought experiment by philosophizer Nick Bostrom that’s now referred to as the “paperclip maximizer.” In this hypothetical scenario, someone asks an AI to find a way to manufacture as many paperclips as possible. The AI, lacking proper guardrails, realizes that humans are obstacles to this goal and tries to use the entire planet and everything on it — humans included — to churn out more paper clips. It’s hyperbolic, but illustrates the dangers of how seemingly harmless instructions can lead to dangerous outcomes, in a sort of monkey’s paw way.
I love this example. If you see me talking about Planet of the Paperclips, this what I’m referring to.
It’s not relevant here though. In what sense would the cyberattack ‘maximize’ its ability to reserve a gym spot? Paperclip maximizer is only a scary story because it is smarter than humans. Having an insane ai do insane things, it’s just insane, not an extinction-level threat.
Edit: I actually read it, it seems not as insane anymore, it sounds like this guy is entirely culpable. Like, he asked to book outside the allowed time, and asked people to be removed from the waitlist. He might not have said the word ‘hack’, but he clearly wanted to manipulate someone else’s system and he knew it wasn’t allowed or authorized by the gym. No insanity, no maximizing, just an asshole who wants the gym all to himself.
In what sense would the cyberattack ‘maximize’ its ability to reserve a gym spot?
You have an agentic AI that has absolutely no morality and is willing to bruteforce any mechanism to achieve it’s simple goal. Use your imagination. Maybe instead of just canceling an appointment, it cancels their gym membership. Maybe instead of canceling their gym membership, it transfers all the money out of their account so they can´t pay the gym membership. Maybe instead of draining their bank account, it sends an e-mail complaint to the gym about being raped by the member with manufactured video evidence. Maybe instead of fabricated a rape, it gains access to the member’s medical equipment and makes it fail, killing the member and thereby forcing a cancellation.
That’s the point of the paperclip thought experiment: you have an agentic AI whose power is only matched by its complete amorality and total stupidity. Any and every prompt can potentially be an extinction level event.
Nah bro. Just gotta update the soul.md file to explicitly say not to falsely accuse of rape, bro. Trust me bro. Just one more soul config, bro /s
Another aspect to this is: even if we could control AI, what makes anyone think people would use it responsibly? We can’t even drive cars responsibly.
I don’t think ai today is anywhere near being capable of an extinction event, but I mean yeah, why would you ever use a system like that? If there’s no benefit, and it constantly goes off the rails and murders people, then that’s clearly a case of a technological dead end. There are plenty of other promising and scary avenues of ai research we could be funding instead.
Shall we play a game? How about global thermonuclear war?
Edit: We put computers in charge of more and more shit all the time, things like our electric grid (USA) is supposedly very vulnerable to cyberattack. I think what has been protecting us so far is that we can RETALIATE against any group or country that fucked with us. I think we have a false sense of security because we are not actually being protected from cyberattacks by security policy, but by social contract. Agentic AI does not think about consequences or the social contract.
Another layer of protection we have had: the number of people willing to risk legal consequences for doing something illegal is quite small. But now we are talking about giving every slob access to a personal AI genie that does not recognize laws or morality, just hundreds of thousands of autonomous clankers suddenly crawling all over the world trying everything they can to achieve their goals.
There is an upcoming massive stress test of our security, and, frankly, I do not think we are prepared for it.
Lt Col Adama may be proven right… again.
It’s also a fantastic free web game: https://www.decisionproblem.com/paperclips/index2.html It’s a bit of a slow beginning, but it goes pretty hardcore eventually.
RELEASE THE HYPNODRONES!
The one thing the game didn’t get right was the cost of memory
