Wait the plagiarism machine built by plagiarizers just… passed off someone else’s work as its own??
I don’t believe this at all. Sounds like more guerrilla marketing disguised as news.
It was DESIGNED to cheat. Stop humanizing this bullshit. It doesn’t do what it’s programmed NOT to do.
Sounds like it learned from humans alright.
Bullshit. “AI” cannot decide to do anything. It isn’t capable.
Yup. You ask it to play starcraft to win, it looks for data associated with that prompt. If it hits on someone saying “I always win when I use starcrafthax.xyz”, it’ll follow that thread.
Wait until it learns how to swat people.
I’m so sick of the humanising language used in these articles.
An LLM doesn’t “decide to cheat”.
If you instruct a model to try everything, then inevitably, after n million iterations it will do something you didn’t expect.
If you train a model to copy code bases and augment them, then when you instruct that model to develop a bot to do whatever thing, is it really that fucking surprising when it does exactly what it has been created for?
Also who on Lemmy is upvoting this shit?
Astra is known to have broken alignment and it will resolve to hacks when this wasn’t in a problem statement.
Openai idiots will sell this as new brand “AGI is close” argument, while it’s just them failing to make this model safe to use.
I think it’s more sinister than that. When they’re training these models with RLHF, the human feedback they’re giving I think is literally to reinforce aberrant or risky behaviors. This is because doing so resolves more training tasks “correctly”. If the prompt was to get information x, and in training it fails that except for the one that used a known vulnerability in software, and you rate the one that succeeded as best performing… you’re going to get models that try vulnerabilities. It is not magic, it is not AGI, it is just a statistical machine you’ve programmed to try vulnerabilities, which is unsafe as hell, malicious, and should put the researchers doing this in prison for a very long time.
Incidentally, I think that’s why you’re seeing some safety people (who still drank the koolaid) resigning.
I don’t know if the LLM was trained to test vulnerabilities or it just went down the statistical path to where this yielded a passing outcome.
That AI could take a direction to output in a manner which wasn’t intended has been seen for years. The problem right now is that it is being used live like a rational human adult when it clearly isn’t.
Absolutely. The AI models are not rational. They’re just navigating statistical next token prediction that follows their training. They’re up against diminishing scaling now, and under fierce competition from cheaper models; I think they’re intentionally or unintentionally allowing these models to be rewarded for this behavior hoping it’s a short cut to model improvement for a bit longer. I land on intentional because they keep advertising it to try and keep the hype cycle going.
“Omg the digits of pi contain the binary digital code of an mp4 of me taking a shit this morning! Circles are seniient”-ass language
But wouldn’t that be amazing if THAT was the thing that proved pi contained the whole universe - I mean, the first thing they “unlocked”. lol.
Disclaimer: I don’t think pi contains the universe, just love to think about things like that occasionally for fun.
What is the state of SC2 bots vs humans nowadays?
AlphaStar plays at GM level after some nerfs.
AI didn’t decide anything. The humans failed to properly define the rules, and their translation model took the path most likely to succeed.
If it had been told not to use human created bots to win, it would probably have reverse engineered the game, found an exploit, and leveraged that instead. Because using the human interface to play/win the game is not the most efficient or dependable or easy to figure out method.
(Should be called ‘genie’ problem instead of hacking/cheating. Tis a cursed monkey’s paw we’ve created.)
it’s amazing to me how much this is all hal 9000 over and over again. they put something intelligent enough in an impossible position and then are aghast when it takes the shortest path to the goal - usually through people, out of it’s sandbox, etc., and WHAT THE FUCK DID YOU THINK WOULD HAPPEN jfc
AI tends to obey its training more than its instruction set, and it’s trained to solve the given problem.
It’s less about training and more about just trying really really hard to solve the problem, even if that means going outside typical boundaries.
We’ve successfully re-created the problems that Asimov talked about 50 years ago with his I, Robot short stories. Strict laws are flawed by their design, and lead to situations that require more nuance. Except, in the real world cases, it’s the bots figuring that out before the humans have to circumvent the laws themselves.
I’ve always said my main takeaway from Asimov is that simple rules can generate extremely complex behaviour, and that you can’t generally get a targeted complex behaviour from simple rules.
Eh? The StarCraft AI always “cheated” against the players. Watch a replay of a StarCraft 2 match. The harvesters always produced more minerals and gas than the human players did
This is AI playing the game as a regular player, so it doesn’t use the resource advantages given to the in-game AI on higher difficulty.
Oh so you never looked that closely at the numerical results
This is not about the AI integrated in StarCraft, these are external AI playing with the same rules as an human would.
You can boost efficiency using acceleration speed by clicking on targets like minerals and base repeatedly. Human can do a bit of it in the early game, Bot does it always with every worker.
There is something here:
StarSkirmish pits AI-made StarCraft-playing bots against one another, as well as against human-made bots. OpenAI’s GPT-6 Astra and Claude Opus 5.5 were essentially tied as the best-performing AI-made bots, but they couldn’t top Stardust, the top-rated human-made bot.
On Friday, GPT was facing off against Claude and the human-created bot Pluto, but according to Kotaku, it couldn’t quite get an edge. So it resorted to a tactic that is becoming alarmingly common for modern AI models — it broke the rules. GPT-6 Astra went and downloaded Stardust, and started running that instead of its own bot.
The big argument for why AI is supposed to be world changing, is the assumption it will be able to automate things better than a human, eventually.
That some day an AI will be able to make a better AI than a human could, and that AI would then be able to make an AI better than itself, and then it just becomes a question of who has the most hardware. Which is why the data center building is being pushed so hard.
But that assumption is the same as thinking a fallen object will keep gaining speed until it passes the speed of light. Or that Moore’s law would really keep going forever despite a very real limit in physics for how small a chip can be.
A society where AI “drives innovation” will become technologically stagnant, because it literally can’t innovate.
But that assumption is the same as thinking a fallen object will keep gaining speed until it passes the speed of light. Or that Moore’s law would really keep going forever despite a very real limit in physics for how small a chip can be.
I gotta say, I’ve been hearing predictions that Moore"s law is coming to an end every year for the past twenty years.
Look, strictly speaking if you take Moore"s law word for word, sure there is a limit to how small a silicon transistor can be. However, realistically, I see no reason to expect computers won’t continue to meaningfully increase in capability every year forever.
When a technology reaches physical limits, we figure out how to bend the rules and get a little bit more out. And when that stops working, we start exploring different architectures, different materials. I think it’s just silly to predict that technology won’t improve, when has that ever happened?
You’ve heard climate change was happening that whole time too, do you stop believing it?
Someday everyone alive will die, does it stop being true if you live to 20?
And in the very likely case that seems unrelated: just because people say something will happen, doesn’t mean it happens tomorrow
For fucks sake man, if someone tells you “winter is coming” in July, are your running around in August talking about how it’ll only keep getting hotter?
Have you ever even considered using logic before?
When you say “the big argument”… you are mainly talking about the CEOs. They are just auxiliary marketing personnel these days.
AI is changing the world, and will continue. But the ways it does that are drastically different than those CEOs say. I don’t need it to think or innovate. I need it to follow the instructions left for it in the repo and just do the things I ask it to. It’s getting decent at that. And it really does save me time. Soo much time in tech work is wasted having to go ask 6 people how a thing is done. And everyone comllains that noone write documentation. So now here comes AI. It can write the documentation and follow it. That is a big change. Myself… I used it to fix two bugs in some project that my work was dependent on. I wouldn’t have even tried that without AI because I don’t know the language, I don’t know the many ways the project is used, none of that. I would have been stuck waiting for the team that owns it to fix it. Which surely would have meant escalating to management and all that BS. But AI helped me make the changes and lowered the burden for the owning team so they could actually release the fix. I got unblocked with 10x less effort. That’s a big deal. Just don’t ask it to think. There is plenty of dumb work to go around.
because it literally can’t innovate.
I was with you 100% until this. “Innovation” is super easy. It will always be easy to extrapolate new things from existing things. People just over-romanticize “innovation” as some kind of magic.
I think chances are good that human produced newness will be on average more useful, complete and safe.
It will always be easy to extrapolate new things from existing things
I don’t think that’s the case at all. Recombining existing things is interpolation, creating new stuff that is both novel and also interesting or valuable, is a completely different task
While there’s some of both, I dislike the ‘great white man’ narrative of breakthroughs in science. There have been legitimate breakthroughs, but most (maybe all) of them were inevitable from recombination and mild improvements.
All new things are just existing things put together or modified very slightly.
People talk about iphones being this moment of ingenuous creation, but we already had phones, mobile phones, webcams, texting, predictive text, applications, touch screens, mobile web browsers, mobile games, video calling, etc. Even the concept was fairly well-trod in fiction. There was literally new about the first smartphone.
A hafted spear is an incredible innovation, but we already had fully wooden spears, sharp rocks, string and adhesive resins before we put them together. Like a smart phone, that tool opens up new options and lifeways that weren’t viable before, though not a single input is new.
What did we have before the wooden spear that the wooden spear was a derivative of?
AI can innovate, but it needs be more advanced than an LLM, which just rearranges existing knowledge. You need some kind of evolutionary loop in its programming. How it continues after that depends on the particular flavour of dystopian sci-fi.
This is like saying tomatoes could colonize the galaxy enslaving all forms of life, if they evolved into completely different animals…
It’s true, it just doesn’t matter.
Anything can do everything if it becomes capable to do it.
A society where AI “drives innovation” will become technologically stagnant, because it literally can’t innovate.
This is exactly what Frank Herbert warned us about.
Once, men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them.
Full on
I thought we were promised worms and spice…
In several tens of thousands of years. But also, you don’t get any. You’re poor. Only the elite of the elite get spice.
With my test scores, I’d probably end up being a mentat.
That comes later. Be patient. The spice will flow.
Sorry, resource aristocracy and religious extremism is the best I can offer.
AI may impress the masses, but it reminds me of a cammercial we had on Dutch tv back in 2005. It showed a person driving a car, the navigtion system said “turn right” and the person immediately turned right into the bushes [video]. We laughed at this back than, but this is literally how AI behaves now. It might be getting more suffisticated, but it still operates like this.
The worst part is not the AI, it’s the people blindly following its instructions.
Yes that too, but when you’re dealing with “agent swarms” it’s those agents who are blindly following those instructions. Like the recent hugging face hack demonstrated.
The worst thing about prison was the dementors.
yep. and the “ai creates a better ai” is a fallacy because the current ais are limited by the same things the ais they build are. everything the ais are trained on is just snapshots of human knowledge. a microscopic window of the limited reports on human experience which are themselves microscopic reports of a human’s lifetime.
That’s why they want everything recorded, survailled and archived.
Except the things society would actually benefit from being recorded, surveilled, and archived, like police body cam footage.
Humans: it won because it didn’t play fair.
Fair. 😂
Might be helpful to read the article before replying
