The engineer, who posted anonymously on X under the username voxium for fear of reprisal from his employer, called his new role at an undisclosed company "soul-sucking."
AI code assistants currently are like that senior dev who knows everything, has experience in everything, and always likes to help everyone. But sadly he does a lot of shrooms in his spare time and sees things that aren’t there. He also has quite the opinionated ideas about what good code should look like.
That implies that you just can’t let him do the quality assurance. He can write code. You can ask him stuff. And he excels at debugging.
But you can’t let him plan the architecture. You can ask him about his ideas and select the ones that are actually good, though.
And you definitely can’t let him review his own code. Someone else has to do that.
So if they literally just let the AI do everything, there is absolutely no way anyone has a chance to ever review the result because AI code is harder to review (for now), and AI loves to repeat itself (for now).
And sure, they can tell it to write tests. But AI is dogshit at writing tests. It tends to forget the important cases, test the wrong thing, and/or obsess about completely irrelevant implementation details.
I also tried the multi-round approach with a committee of agents trying to iteratively find and fix flaws. It only works well for finding bugs, not fixing them (for now).
But while AI can’t replace me, it is really nice to have as a tool - just like I have syntax highlighting in the IDE, I have a code assistant that can autocomplete a few lines of code, find out why my code misbehaves, or tell me what some library method is supposed to do. Sure, I have to still verify the results. But often, finding the bug or information was the hard part.
Coding assistants are great. Just don’t let them write too much code at once. You need someone who still actually understands the codebase. That sadly can’t be an AI (yet).
I really love when I ask the AI to do something, and it gives me 40-50 lines to do it, and I tell it there’s no way something like that should take that much code, and it’s like oh, ya you’re right, here’s a 5 line version using something from from the standard sdk.
I’ve tried to put in its memory to not do things like that, but it keeps doing it.
It’s because the average quality of code that they are trained on is like that. Most code that is written by humans is slop, and the models represent that.
Producing overly complicated solutions to uncomplicated problems
It often can do it better if you prod it though. Its just so weird that its really hard to get it to do that. It knows how even if it is trained on poor over complicated stuff.
Its usually just 1 or 2 messages saying i dont like this so its only the time it takes to read it and say try again. A minute or two?
Its faster to just ask to do it again in most cases.
The times itd probably be quicker to do it on my own are the more complicated ones where it gets something wrong but it wasnt so blatantly obviously bad that I just tell it to try again, so I spend time with the wrong stuff, see its wrong with a little work, then toss it and try again, or re work it myself from that.
AI code assistants currently are like that senior dev who knows everything, has experience in everything, and always likes to help everyone. But sadly he does a lot of shrooms in his spare time and sees things that aren’t there. He also has quite the opinionated ideas about what good code should look like.
That implies that you just can’t let him do the quality assurance. He can write code. You can ask him stuff. And he excels at debugging.
But you can’t let him plan the architecture. You can ask him about his ideas and select the ones that are actually good, though.
And you definitely can’t let him review his own code. Someone else has to do that.
So if they literally just let the AI do everything, there is absolutely no way anyone has a chance to ever review the result because AI code is harder to review (for now), and AI loves to repeat itself (for now).
And sure, they can tell it to write tests. But AI is dogshit at writing tests. It tends to forget the important cases, test the wrong thing, and/or obsess about completely irrelevant implementation details.
I also tried the multi-round approach with a committee of agents trying to iteratively find and fix flaws. It only works well for finding bugs, not fixing them (for now).
But while AI can’t replace me, it is really nice to have as a tool - just like I have syntax highlighting in the IDE, I have a code assistant that can autocomplete a few lines of code, find out why my code misbehaves, or tell me what some library method is supposed to do. Sure, I have to still verify the results. But often, finding the bug or information was the hard part.
Coding assistants are great. Just don’t let them write too much code at once. You need someone who still actually understands the codebase. That sadly can’t be an AI (yet).
More like a junior dev.
I really love when I ask the AI to do something, and it gives me 40-50 lines to do it, and I tell it there’s no way something like that should take that much code, and it’s like oh, ya you’re right, here’s a 5 line version using something from from the standard sdk.
I’ve tried to put in its memory to not do things like that, but it keeps doing it.
Possibly motivated to do more complex problems to spend more tokens?
I can believe this. I’ve been put on slop duty, and had the slopmachine go all
And I’m here thinking, if you’re gonna run the whole suite anyway, why’d you run an individual test? That’s just redundant.
Even without external libraries, it routinely comes up with solutions that are at least 3-4x the complexity and LOC than how I would implement it.
It’s because the average quality of code that they are trained on is like that. Most code that is written by humans is slop, and the models represent that.
Producing overly complicated solutions to uncomplicated problems
It often can do it better if you prod it though. Its just so weird that its really hard to get it to do that. It knows how even if it is trained on poor over complicated stuff.
How much time does that prodding take vs how long would it take you to just write the (better) code yourself?
Its usually just 1 or 2 messages saying i dont like this so its only the time it takes to read it and say try again. A minute or two?
Its faster to just ask to do it again in most cases.
The times itd probably be quicker to do it on my own are the more complicated ones where it gets something wrong but it wasnt so blatantly obviously bad that I just tell it to try again, so I spend time with the wrong stuff, see its wrong with a little work, then toss it and try again, or re work it myself from that.