• schipelblorp@sh.itjust.works
    link
    fedilink
    English
    arrow-up
    0
    ·
    16 hours ago

    With the added wrinkle that there are expensive things that may be necessary-ish, but when you exclusively use it for those things, the price jumps up to insanity.

    Now that people are shunting easier requests to local models and sending the harder problems out, the hyper scalers have less income, so they have to charge more.

    Because eventually the only real limitation will be hardware, that kind of AI will be a rarely-used commodity priced only slightly above what it would cost you to recreate it inside your business.

    Not to mention the privacy concerns you’d have sending your expensive and necessary compute to the cloud.

    • Voroxpete@sh.itjust.works
      link
      fedilink
      English
      arrow-up
      0
      ·
      16 hours ago

      Now that people are shunting easier requests to local models and sending the harder problems out, the hyper scalers have less income, so they have to charge more.

      While you’re 100% correct about the fact that this is happening, and it is a serious problem for the hyper-scalers, it’s really, really important to note that even before this started happening, LLMs were already hilariously unprofitable, both on the software and the hardware side. For the AI companies, there is simply no way to charge people what the models actually cost to run and still turn a profit. On the data-centre side, the turnover on burned out GPUs is going to annihilate any hope of profitability. They’re treating as capex what is essentially a consumable and their business models simply do not account for this reality.

      The behaviour you’re identifying here is basically just a swift kick in the guts to someone who was already bleeding to death from several bullet wounds. Seriously unhelpful, but not remotely the impending cause of death.