The Generative AI Learning Penalty: Evidence from Chinese Secondary Education

Using 30 months of panel data on 26,811 Chinese students in grades 7-12, we study how generative AI affects homework productivity and learning. The data combine monthly closed-book exams, high-school and college entrance exams, and homework scores and completion time across nine subjects. We exploit staggered AI adoption in a difference-in-differences design. AI adoption raises homework scores by 18% and reduces completion time by 30%, but lowers monthly exam scores by 20% within six months. High-stakes entrance-exam scores fall by 18 and 24%, with the full penalty emerging only after about two years. The losses are largest in social science subjects, followed by STEM and languages, and are especially large for junior students, high-achieving students, and boys. The learning losses are concentrated among roughly 80% of AI users whose behavior is consistent with homework outsourcing, as indicated by exceptionally short homework completion time coupled with high homework scores. AI users who maintain similar homework completion time as non-AI users experience small learning losses.

Edit: moving my comment up here

Just in case as it’s formatted a bit weirdly

X-Axis: Homework scores

Y-Axis: Exam scores

    • alphabethunter@lemmy.world
      0·
      2 days ago

      EDIT: nevermind, I misread. I thought you were complaining about the one you posted… However, the image used by op is still a fairly obvious graph.

      Edit 2: the axis of the graph posted by op are also clearly defined, vertical is exam scores, horizontal is homework. The definition shows above and below the graph itself.

      It seems fairly obvious to me. It shows the clear mismatch between homework scores and test scores of AI users, where higher homework scores using AI correlate to lower test scores, clearly indicating that there is no learning, when compared to non-ai users, where higher homework scores directly translate to higher test scores as well.

      • Viceversa@lemmy.world
        0·
        1 day ago

        Y label is bold
        has the prefix “China*,”
        has additional row of text between the label and the diagram.
        Not even remotely near Y axis.

        X label is regular
        doesn’t have the prefix “China*,”
        but has the suffix “, average without Al=100”

        How is it clearly defined again?

      • brucethemoose@lemmy.world
        0·
        2 days ago

        Yeah I get it.

        Looking at the OP graph again, I get it now, but initially I found the top label to be confusing. I thought it was part of the title, and that the Y axis was unlabeled.

      • brucethemoose@lemmy.world
        0·
        2 days ago

        Side tangent, but Lemmy is the worst with sources.

        If I were admin, I’d make all users, new and extant, take an 8-minute course on clicking through reposts to finding original sources before they are allowed to submit posts.

        That’s my rant.

        • kingofras@lemmy.world
          0·
          2 days ago

          I don’t think you can prevent this from being posted, though I agree that resource should be available. It’s just that our attention spans have evaporated, everything must be instant and with LLMs it’s only accelerating.

          This is about having a brief second of critical thinking and then not upvote or downvote the low effort post and upvote the first critical thinking comment.

          Another thing on Lemmy is that we have half a dozen “users” who don’t care about the content, they just automatically post high engagement content from other platforms and post it here (without declaring themselves as a bot account too). And the mindless scrollers (at least 60-70%) going by this post, just take the bait.

          Not having karma didn’t help much in hindsight.