In March, a girl’s stepfather took his own life after cops discovered that he had used Grok to create 7,000 sexually explicit images using one photo taken when his stepdaughter was 11 years old, the amended complaint alleged.
Grok allowed the man to generate extreme images depicting incest and rape without flagging any harmful behavior, the complaint said. Seemingly, xAI’s child safety system only intervened after the man input a prompt for “gang rape.” That request sent a CyberTip to the National Center for Missing and Exploited Children (NCMEC), which alerted law enforcement to the AI CSAM.
Yet the harm was not stopped then, either. Despite mandatory reporting requirements to share information like a user’s IP address when CSAM is flagged, xAI repeatedly refused to help cops or NCMEC identify the user, the complaint alleged. For weeks, xAI allegedly “obstructed this investigation at every turn” and made it harder for “law enforcement efforts to locate, identify, and apprehend the perpetrator.”
Eventually, the stepfather was arrested after cops obtained a warrant to seize his devices. That’s when “a forensic review revealed approximately 7,000 AI-generated images and videos” depicting his stepdaughter, which were allegedly produced using Grok. Without Grok providing users with easy access to “undressing” capabilities, his family doubts he ever would have generated the harmful images, which he allegedly trafficked online in trade for “CSAM produced by other child sex predators.”
What a fucking headline. At least it had a happy ending.
xAI founder Elon Musk has denied that Grok has ever been used to generate child sex images.
Well, that’s a relief! Of course they wouldn’t do something like that. Because that would be super illegal!
did anyone save any of them? Is there a mega upload?
Captain?
They are making their pedo dream land, with no laws or rules.
This is their so-called effective altruism.
We all know what to do.
How do you even generate ONE of those images and NOT immediately go to prison?
“FoR thE chIlDrEn” when privacy is on the line but they can’t put a weirdo in prison after generating SEVEN THOUSAN CSAM images?!
Because it’s never ever about the children.
“Think of the children”
And this is what happens when they do that a little too much
So, if xAI did that, then did they get obstruction charges?
You mean a slap on the wrist for the richest man in the world?
Horrors of this aside… is Grok using a keyword filter for CSAM detection?
I cannot even begin to describe how technically negligent that is.
Image based CSAM detectors have been around for ~2 decades, and any image host with an ounce of self preservation uses them. I’m pretty use some Lemmy instances use them. Twitter must have already been using them, even before Musk bought them, because it would be legal suicidal not to.
That they wouldn’t implement a strong image-based checker in Grok is mind-boggling.
is Grok using a keyword filter for CSAM detection?
At some point in the last ten years, everyone apparently forgot why this doesn’t work
A pedophile would remove a CSAM checker? Color me shocked.
I cannot even begin to describe how technically negligent that is.
Tbh, this is par for the course for most of Elon’s businesses
But even unintentionally serving CSAM is unique in that it can get you in legal hot water reeeal quick. It’s why web services are so strict about filtering.
I just can’t wrap my head around this. Maybe I’m misreading what Grok did, or maybe the story isn’t quite right, but if it is, the company should be getting raked over hot coals by the feds right about now.
Yeah, but remember who’s running the FBI right now.
O.O
More likely the Epstein class will be offered given “premium private” accounts.
xAI did more to protect a pedophile than it does to protect its regular users from law enforcement.
when the ‘leader’ of the country is a known csam perpetrator, i’m sure it emboldens those members of the public with similar mental leanings
Techbros in the past were infamous of having a “free speech absolutist” view, and saw CSAM as “recordings of crimes”. While this view fell out of mainstream (alongside with nonconsensual nudes being more ostracized), it wasn’t without a lot of opposition, and an almost irreparable harm to free speech advocacy.
Yeah, this guy sort of digitally pimped his wife’s kid.
Which is a lot.
I’m thinking this has to be one of the most anti social behaviors.
However, there’s also, this Grok LLM. Which, tbh has said some shady shit on the internet and was abetting(?!).
That’s somehow more than a lot.
Well that is what happens when you train an AI on the likeness of Elon
Nothing dystopic about this headline.
Every image Grok generates of CSAM is another reason to rightfully call Elon a pedophile for knowingly allowing it to happen.
I really hope France fucks them up. In France, there is no question about the illegal status of fictional csam, it is illegal.
Those idiots think this should be legal, but not when the civilians do it. Tim Sweeny probably is only piping up because he insist on posting the stupidest thing his brain can come up with every day.
In no way shape or form has elon tried to dampen or stop this from happening to this day. He just made it so only paid users not in the united States can keep producing csam.
And trump wants this. This is the mysterious Ai progress we keep hearing about, next to price fixing, and survalience. States are not allowed to regulate Ai, and people will tell you that you are a dangerous person for trying to suggest that elon musk should be held accountable.
We are not all here looking for the guy who did it or confused on why this is happening. Elon musk has created a csam generator and doesnt have to stop it. Instead he just occasionally has to send one of his super fans to prison for using it the way he intended to. All to say hes doing something to stop this problem.
7k images and videos is a lot to go unnoticed. Its almost like this is allowed on their platform and encouraged. This isn’t a “we can’t be held accountable for what our users post” problem. This is a elon musk left a loaded gun out on a table, and he should be held liable for not putting it in a gun safe. Especially when every single other mainstream llm doesnt have this problem. Its negligence to the point of being willfully ignorant. You can’t tell me they can ban you for a single comment about Israel but all the sudden their moderation isn’t the greatest when 7k images and videos where being produced?
All the people who bought SpaceX stock invested in this. Grok is owned by SpaceX. It would be nice if journalists would put that in their articles.
It’s not just owned by SpaceX, it accounts for the vast majority of the “value” of the IPO stock. The part of the company that sends things to space was a footnote compared to the portion made of the projected value of their AI business.
Lawsuits for this need to start including shareholders as defendants. Make investing in it a civil and reputational liability.
Lawsuits for this need to start including shareholders as defendants.
I get the sentiment, but … they kind of already are.
Being a shareholder means you own part of the company. If the company stands to lose money from this lawsuit, then all the shareholders do as well. (That is, at least, on the somewhat naive assumption that a company’s stock price has any relation whatsoever to how much money it has and can make.)
When the stock value falls, they’re not immediately losing money. If they sell the stock, that’s when they either profit or lose. As long as they hold on to their stocks, there are no actualised losses.
For shareholders and investors, the worst thing that can happen to them right now is the stock going down and having to take a loss on it. It’s bounded by however much money they put into it.
If investing in 10 shitty companies knowing that 1 would fail manages to be more profitable than investing in 20 companies that don’t degrade the moral fabric of society, there’s a perverse incentive to keep investing in companies creating Groks. Making shareholders financially liable for the debts of their investments would introduce an unpredictable risk that could help change the equation.
If you’re driving a Tesla, you are funding CSAM.
The CSAM generator is in the index funds 🤢
Best thing that ever happened to me was getting banned off Twatter in October of 2019. (after a decade and 30 poasts, no less!)
It was delicious that I got to refuse the “Come back to X” message that arrived in my mailbox after Musk took over and offered to reinstate the banned accounts.
Thanks, Elon, but no.
I’d rather have a root canal w/o anaesthesia.
Every US taxpayer invested in this, involuntarily.
