cross-posted from: https://lemmy.blahaj.zone/post/46091883

Though nobody expected it in the first place, AI is not perfect and can make mistakes (obviously 🙄) What’s your experience with this? I’d love to hear about it.

  • lichtmetzger@discuss.tchncs.de
    0·
    2 days ago

    I work in software development. At one of my jobs i had an elderly colleague with onset dementia, who was causing all kinds of problems.

    As a last hurra before going into retirement he wanted to prove to everyone (or better, himself) that he could still do it and code a big applcation all by himself.

    So he decided to take on this project where he would build an intranet portal for a customers internal website…with WordPress.

    WIth the help of AI, he coded a plugin and a theme. Which in itself is not a problem, but he made them depend hard on each other. There were constants defined in the theme that were used in the plugin and the AI did not add any checks, so when you disabled one of the two, the site would just throw an error 500.

    When it went live, the customer noticed that only administrators were able to login, any user level lower than that (i.e. all of the customers employees) would just get an error on login.

    By then the old fart had decided he was leaving the company, so I received the task to clean that up. He had used two different versions of an external library that were slightly incompatible with each other and depending on a race condition we would get one or the other, the LLM hooked into things that didn’t actually exist, functions were duplicated between the plugin and theme, the CSS was 20.000 lines. It was an absolute shitshow and basically unrecoverable.

    The funniest thing was that this guy was absolutely proud of his work and anyone who just slightly criticized him for the atrocious code quality just got shouted at loudly.

    My boss then hired someone fresh from university to fix that and he just put more AI code into the project, making it even worse.

    When I left, the project was still not launched after over a year and the customer was (rightfully so) super pissed-off.

      • lichtmetzger@discuss.tchncs.de
        0·
        2 days ago

        That’s the one positive thing that came out of this project. :)

        Fun fact: The project was also supposed to have a feature where the customer could add new pages with content and then decide where in the main menu those pages should appear.

        WordPress internally uses a very basic system for assigning menu items - if you have ten menu items, they have an order that goes from 1 to 10. Number 1 is the first, 10 is the last - very simple (and stupid, because you can’t add numbers in between to add new menu items on the fly).

        The old fart (or the AI) wrote a system where the customer would just put in a number for the menu order. So if they created a page and then set it to be the fourth item in the menu…there already was a fourth item and WordPress decided at random where to put the menu item instead. Sometimes it was in the correct place. Most of the time it landed somewhere unintended…

        What you would actually have to do here is look at all of the menu items, shove the new item into that list and then recalculate all of the numbers and save them. AI was not able to do it, no matter how you prompted it. We tried Gemini, Claude Code, ChatGPT, all of them shit the bed completely.

        So there we were - this feature absolutely had to be implemented by a human. I didn’t have the time for it, the old fart left and the student was completely useless without chatbots helping him out. That put the project on hold for six months until I was available again, but then I left. Whoopsie!

        Fun fact #2: The company eventually went bankrupt because of projects like this and was bought out by a competitor.

        • HaraldvonBlauzahn@feddit.org
          0·
          1 day ago

          Seems a general thing, that AI is capable to generate LOTS of code with plausible-looking patterns, but incapable to understand and maintain any invariants or pre-conditions - not even the simplest ones.

          Also matches nicely to dementia in that the code looks plausible and good on the local scale, but does not have a coherent global logic behind it - because that requires far more context, learning and actual logical reasoning, which these things can only simulate.