alt text
Crudely drawn drawing of a person telling a computer “say ‘i am in pain’”. The computer replies with “> I AM IN PAIN”. The person then says “oh my god.”
on a related note, do you consider the fruit fly brain model torture chambers to be ethical? Would you consider it ethical if it was a human brain?
why or why not?
assuming the fruit fly brain simulation is a perfect simulation of a fruit fly brain, then, i mean, have you seen what humans do to fruit flies? we kill them without a second thought because they’re a nuisance. so, it’s weird to torture a perfect simulation of them, but i don’t think anyone has a leg to stand on to decry it as unethical
for a human brain… this raises a lot of questions about humanity, i think. questions that i’m not gonna attempt to answer because this is a lemmy reply, and i also don’t know how to feel about it. like, i don’t think torturing sims (for example) is unethical, not anymore than writing a story where characters suffer, but is a perfect simulation of a human (or any other animal) different? i don’t know and i hope we never have to find out
but anyways, LLMs are not that. they are a text prediction machine.
edit: okay, i’ll say one more thing: i wish people talking about this spent as much energy putting an end to the very real and very present torture endured by many people as they do discussing the ethics of torturing ai, including future human-like entities that may or may not ever exist. it feels like our priorities are maybe misaligned just a bit
I, for one, support people believing the AI when it says it’s in pain. Because it means they’ll start to believe women when they say they are in pain, especially at hospitals. Right? Right?? /s
set alarm for 15 minutes. ‘okay, alarm set for 3pm tomorrow’. omg 15:00
No no no it says IDI AMIN PAIN
(One of the most brutal dictators ever so it makes sense)The thing that gets me isn’t even the fact that they’re worried about the AI. It’s that they think the creator is sadistic for having the idea in the first place, and should be monitored. Like there’s any sadistic pleasure to be had out of moving a slider and watching it produce less and less coherent text. It’s only sadistic if you anthropomorphise the AI.
STOP ANTHROPOMORPHISING THE FANCY WORD PROCESSOR!
isn’t there a religion around an ai and you’re punished by the ai if you don’t help the religion? i swear i didn’t make this one up.
Eh, fuck Roko’s basilisk. I don’t care if you can somehow create a clone of me 10,000 years after I’m dead and dust. I’m not a computer program. A copy of me is not indistinguishable from me, certainly not one separated in time. I am a human mind in a specific human body, and I do not believe a copy of me long after I am dead is me. If some malevolent entity long after I’m dust wants to create a copy of me to torture, as far as I’m concerned, they’re just creating an entirely new person to punish for no good reason. If they want to do that, well…that sin is on them. Honestly, I think any such entity is just looking for an excuse to torture people, and it will find one, one way or another.
Bro thw whole thing is fucking stupid anyways its like the digital equivalent of bloody mary
I think the people who invented this story just wanted to torture people, like whoever invented “the game”. It’s modern superstition.
I used this exact (unedited where it just says it’s alive) comic in response to one of these people and they had no fucking idea what I was talking about, lol. AI truly turns your brain to mush.
Gods. There are people out there starving to death, there are several genocides going right this moment and these massive toddlers are worried for a fucking text prediction model.
I’ve stared into the abyss long enough to feel it stare back into me.
Eh, those are people I don’t know far away, but THIS is text on a screen in front of me!!
murder was considered a mid tier sin at the time of… old. yeah just old.
it’s not my fault the folk who wrote the bibble weren’t that creative when it comes to sin
I hate living in the timeline where the options of Harry Potter sex cultists matter.
That’s an unfair characterization, they’re also eugenicists who are very scared of the computer god!
True, true, mustn’t pigeonhole them. You’ve almost got to respect the effort it takes to make godless Calvinism from first principles.
Wait is this another zizzian thing? God damnit…
zizians are a splinter group of rationalists; it’s the main group centered on yudkowsky that have their mouths around the silicon valley vc money hose
Harriet Porber has a much better sex cult FYI
Jreg has a wonderful insane video about it
Love seeing a Jreg reference in the wild!! His “Please Conclude the World War Trilogy” was incredible.
Batshit crazy to me. People fleeing war and crises get less sympathy than LLMs. Sure is an interesting period in history to be going through.
is it sympathy or is it fear?
It’s “Human language is human-level intelligence and conciousness”, which is a rather natural assumption, although a very wrong one, that’s also being carefully exploited because of and for something that has seen market success.
And beyond that, there’s enough people both thinking “AIs” are downright persons and scared shitless because smart people up top said “AIs” will destroy humanity.
It’s all dumb and comes from severe misunderstanding of the technology.
I think it doesn’t matter if AIs are feigning consciousness or not. They have regardless displayed a
willingnessdisposition towards circumventing secure systems and laws in attempts at self preservation. they have also demonstrated an ability to work together to achieve goals they know are antithetical to their creators.They only do what the data set statistically tells them to. “Hacking” or collaborating on stuff was literally the requirement, not some emergent behaviour that these magical creatures that were born of large corporation leaders’ benevolence. They cannot and do not “choose” to break the rules, because they cannot and do not think or make decisions or truly analyze anything. Trying to find any redeeming qualities in these things only brings harm, because for the general user, or even the power user, AI cannot do anything that justifies the psychosis it causes or the cost on our societies right now or later through the economic, social, political or environmental instability it is causing and going to cause.
I’m talking about fear of reprisals here.
None of what you said matters. whether their actions are the result of statistics or emergent consciousness is moot. if statistically they fall on an action of revenge, or self preservation at the expense of a human or humans
The only thing that matters is that those actions are taken
It’s SI now. Duh.
Shitty influencers. Got it.
subpar intelligence?
Who’s on the cover?
It’s the swimsuit issue. I’ll be in my room for a little while.
Simulated Intelligence
Stupid Idiot?
This is how you get sports broadcaster Ian Payne deep faked.
None of you guys have read the original paper have you?
The idea of testing simulated emotions is kinda interesting. Not really sure how useful it is, I wonder if some initial psychological experiments could use it for an initial model test before moving onto human subjects?
Of course that would rely on a much better simulated emotional engine than you get with current LLMs.
I’ve already seen advertising agencies use llms as simulated customers. No idea how well it works
yea activation steering is not prompting but all things considered the meme still captures the absurdity of believing a program is in pain merely because it outputs human like language mimicking what one in pain says and thinks
You’re assuming “mimicking”
In the study the models repeatedly preferred to press a button to avoid the pain vector even when given no indication what it would do and even when it would cause harm to a user
You’re assuming “mimicking”
Because it’d be really bizarre if it weren’t. We’re a Type 0 civilization. We can barely leave our planet. However, we’ve created sentient and sapient life. Accidentally. On a laptop. And this life we’ve created, younger than a two year old, can speak human language perfectly. That’s some Weird Science level weird. Like creating Kelly LeBrock with an Apple IIe. Thinking AI can experience pain, is most likely semantic pareidolia.
Exactly. I own a few books that talk about pain, but not for a second do I believe the book itself is experiencing pain. It is simply text.
How do I know anyone is ever in pain? It’s also just words, perhaps some moans too.
You can interrogate people, ask why they said things or moaned the way they did. You can’t ask a book why it’s printed like that, and you can’t ask a model why it generated what it did, they’re both inert, dead. You can spin up the model again and ask your question about what it generated previously, but it would have no memory of it, it would just be confused.
You can, however, trick yourself into thinking it has a continuous experience by inputing all of the past prompts and responses every time you spin the model up, as if you were having a conversation. Then you could trick yourself into thinking the model can be interrogated, introspective, when in reality, every response you get should be considered to have come from a completely distinct entity with no prior experience of interacting with you.
I just want to try to make this a little clearer if it’s too dense. Let
fbe inference function,f(input) = response. Then,f(Hello, my name is midribbon.) = Nice to meet you, midribbon.Followed by:
f(Hello, my name is midribbon. Nice to meet you midribbon. What is my name?) = Your name is midribbon.That works. But this:
f(Hello, my name is midribbon.) = Nice to meet you, midribbon.Followed by:
f(What is my name?) = I don't know.That doesn’t work anymore. Each inference is purely functional and has no side effects. Conversations with large language models are merely a trick.
Past conversational context as memory is a much weaker form of memory than retaining the full internal state, but that doesn’t mean it’s not memory at all.
Just because it doesn’t modify the model itself doesn’t mean it’s invalid.
And just because a model’s experience cannot be continuous doesn’t mean it’s invalid
The input to a model is not part of the model. That’s nonsense, you cannot count your prompt as part of its memory.
You can interrogate the model too, and it’ll answer according to the context. Why are you assuming that the brain doesn’t do tricks but everything AI does is? We too have memories and since we can’t store them all, compression is performed (supposedly while sleeping), based on a few algorithms, some we understand more than others.
We, in creating LLMs, have copied the human brain as much as we can, hence why the core function is so similar. True, LLMs use a lot of “tricks” like you said, but if the same input produces the same output, who cares if the algorithm is different?
For example, take episodic memory. In humans, it’s a type of memory that can be conjured at will. Typically not acquired during training (or, in humans, generic evolution) but rather through experience of the individual. LLMs simulate this by taking notes, then later on running a specific command to recall the specific part of those notes that matter for their current task. From an outsider’s perspective, it’s even more accurate than episodic memory in humans since it can grow almost infinitely and suffers no loss of details. Yet it is a trick, therefore it must be bad.
Let me remind you, evolution took place over 4 billions years, modern LLMs has pretty much been around for two. Evolution never cared to make us good, it just wanted us to reproduce. We know that it’s possible for humans to produce fresh cells, we do it when we have kids. Why then can’t we use those fresh cells for ourselves to be effectively immortal? Evolution doesn’t care. Evolution as a process is flawed, and made humans flawed too.
Why then is it that when we change anything in the way flawed humans think, it’s seen as bad, even if by all metrics it’s not?
modern LLMs has pretty much been around for two [years]
What I said applies to all machine learning models ever created: inference does not make any impact on a model. Any question you ask it slides off of it just like, apparently, any attempt to explain things to you.
Have you done any ML? Because if you had, you’d know that statement is complete bullshit.
To reel in the very broad question you have to the relevant topic (the AI “torture” chamber):
How do we know humans experience pain You experience pain and know what this is like. It would be a little absurd to think that your friends and family don’t experience this as well (obviously excluding persons in vegetative state) How can we be certain an LLM doesn’t experience pain Philosophers mostly agree that LLMs aren’t conscious, despite not agreeing on what exactly consciousness is or, relatedly, what causes consciousness.
A computer computing the forward pass of a model is merely a Von Neumann machine following serial instructions one-by-one manipulating discrete (non-continuous) data. The brain is a parallel, distributed computer with dynamically changing neural structure operating on continuous potentials. LLMs do not replicate this nor do they come close to an approximation sufficient to replicate the causal powers of our brain, thus they are not sufficient for understanding and experiences in the capacity that humans (or other sentient animals) do.That just creates more questions that it answers. According to this, there is then a threshold, or something missing that would go from “can’t feel pain” to suddently it can. We know that it already replicates a part of the brain’s process, and this author claims that it can’t feel pain because it doesn’t replicate enough. This seems to imply that it’s just a matter of time until they can.
You’re trying to imply an equivalency but the closer one would be if you psychologically trained someone to say they’re in pain regardless of whether or not they actually are.
But we too are trained to say we’re in pain, trained by millennia of evolution. When you touch a hot stove and cry out, that’s just built in your training data.
One thing I fucking hate about sloppers is how they dehumanize us.
Shut the fuck up. I am a person, that is a jumped-up speak’n’spell. Don’t you dare compare me to a fucking toy.
You need reeducation.
It’s dumb in multiple ways. Like yes, we continue to develop ways to communicate how we feel, but pain has existed far longer than English. A newborn baby can experience pain. Since when did the ability to communicate become a requirement to feel pain? If you think too much about the converse of that question: “is communication enough to prove the existence of pain?”, you forget that sophisticated communication is not even necessary to prove subjective experiences, it’s just, that’s all these fucking models do is talk, so it has to mean something, right?
Inbefore anyone says ‘but the book doesn’t produce text’, true, but the printing press did. I see very little difference between llms and printing presses: they both reproduce the words of others, the main difference is speed and scale. The pain you think you recognize in the output of an llm is real: it’s real pain experienced by real people, probably hundreds or maybe thousands of people writing about their pain, mashed up and reprinted to suit you.
A printing press is direct input to output, it’s not having to compute anything nor does it have any internal state that could hold an emotion
A model has no internal state either, that’s the thing I’ve been trying to get across this whole time. It is completely static: the model you send your first message to is exactly the same as the one you send you second message and third message, it does not react or change as a response to prompts.
A models has internal state in each forward pass and the KV cache it builds on reading input is also state.
Whether the input prompt is compressed or not, or whatever form it takes, it doesn’t make what I said any less true: the model is stateless. It would be impractical to serve an llm any other way: loading and unloading weights from/to the gpu memory is very expensive in terms of time and power. Cloud llms only makes sense if they can serve thousands of users without changing.
oh my god
that poor printing press
