In what sense would the cyberattack ‘maximize’ its ability to reserve a gym spot?
You have an agentic AI that has absolutely no morality and is willing to bruteforce any mechanism to achieve it’s simple goal. Use your imagination. Maybe instead of just canceling an appointment, it cancels their gym membership. Maybe instead of canceling their gym membership, it transfers all the money out of their account so they can´t pay the gym membership. Maybe instead of draining their bank account, it sends an e-mail complaint to the gym about being raped by the member with manufactured video evidence. Maybe instead of fabricated a rape, it gains access to the member’s medical equipment and makes it fail, killing the member and thereby forcing a cancellation.
That’s the point of the paperclip thought experiment: you have an agentic AI whose power is only matched by its complete amorality and total stupidity. Any and every prompt can potentially be an extinction level event.
I don’t think ai today is anywhere near being capable of an extinction event, but I mean yeah, why would you ever use a system like that? If there’s no benefit, and it constantly goes off the rails and murders people, then that’s clearly a case of a technological dead end. There are plenty of other promising and scary avenues of ai research we could be funding instead.
Shall we play a game? How about global thermonuclear war?
Edit: We put computers in charge of more and more shit all the time, things like our electric grid (USA) is supposedly very vulnerable to cyberattack. I think what has been protecting us so far is that we can RETALIATE against any group or country that fucked with us. I think we have a false sense of security because we are not actually being protected from cyberattacks by security policy, but by social contract. Agentic AI does not think about consequences or the social contract.
Another layer of protection we have had: the number of people willing to risk legal consequences for doing something illegal is quite small. But now we are talking about giving every slob access to a personal AI genie that does not recognize laws or morality, just hundreds of thousands of autonomous clankers suddenly crawling all over the world trying everything they can to achieve their goals.
There is an upcoming massive stress test of our security, and, frankly, I do not think we are prepared for it.
You have an agentic AI that has absolutely no morality and is willing to bruteforce any mechanism to achieve it’s simple goal. Use your imagination. Maybe instead of just canceling an appointment, it cancels their gym membership. Maybe instead of canceling their gym membership, it transfers all the money out of their account so they can´t pay the gym membership. Maybe instead of draining their bank account, it sends an e-mail complaint to the gym about being raped by the member with manufactured video evidence. Maybe instead of fabricated a rape, it gains access to the member’s medical equipment and makes it fail, killing the member and thereby forcing a cancellation.
That’s the point of the paperclip thought experiment: you have an agentic AI whose power is only matched by its complete amorality and total stupidity. Any and every prompt can potentially be an extinction level event.
Nah bro. Just gotta update the soul.md file to explicitly say not to falsely accuse of rape, bro. Trust me bro. Just one more soul config, bro /s
Another aspect to this is: even if we could control AI, what makes anyone think people would use it responsibly? We can’t even drive cars responsibly.
I don’t think ai today is anywhere near being capable of an extinction event, but I mean yeah, why would you ever use a system like that? If there’s no benefit, and it constantly goes off the rails and murders people, then that’s clearly a case of a technological dead end. There are plenty of other promising and scary avenues of ai research we could be funding instead.
Shall we play a game? How about global thermonuclear war?
Edit: We put computers in charge of more and more shit all the time, things like our electric grid (USA) is supposedly very vulnerable to cyberattack. I think what has been protecting us so far is that we can RETALIATE against any group or country that fucked with us. I think we have a false sense of security because we are not actually being protected from cyberattacks by security policy, but by social contract. Agentic AI does not think about consequences or the social contract.
Another layer of protection we have had: the number of people willing to risk legal consequences for doing something illegal is quite small. But now we are talking about giving every slob access to a personal AI genie that does not recognize laws or morality, just hundreds of thousands of autonomous clankers suddenly crawling all over the world trying everything they can to achieve their goals.
There is an upcoming massive stress test of our security, and, frankly, I do not think we are prepared for it.
Lt Col Adama may be proven right… again.