cross-posted from: https://lemmy.blahaj.zone/post/46091883

Though nobody expected it in the first place, AI is not perfect and can make mistakes (obviously 🙄) What’s your experience with this? I’d love to hear about it.

  • luciferofastora@feddit.org
    link
    fedilink
    arrow-up
    0
    ·
    2 days ago

    the test data was generated specifically to satisfy the tests

    I once saw someone claim that slopcoded bugs would be prevented by the AI testing its code and tried to reason that no test from within the system could fully prevent errors within that same system. This is basically the kind of thing I had in mind: Manipulating the tests to fit the results instead of altering the code to fit the tests.

    Testing has to come from outside, and with real data. Otherwise it’s just a meme of the AI giving itself a medal for passing its own tests.