Why AI Tokens are so Expensive - Computerphile

This reminds me of the "12 Days of Christmas" song. Each verse, you have to reiterate all the previous verses!

We're in a golden age of British guys explaining LLM limitations.

I feel like I've been going insane the last 6 months, seeing people extravagantly spend millions of tokens to do simple tasks they could code themselves. As you say, this has been clear to anyone who bothers to understand the arithmetic behind this. Computerphile as usual is a voice of reason in a sea of hype.

A couple of a years ago Sam Altman said that users using free access GPT saying “please” and “thank you” costs Open AI a few million dollars a year.

So.. have you guys been working through a surplus of dot matrix printer paper for 30+ years?

The rising cost pressure results in funny remedies, like a plugin that makes your AI talk like a caveman (or a lite version, which just makes it blunt) or a CLI tool that removes irrelevant terminal command output, all to save tokens. In a way it's funny that we are mechanically summarising text for the magic text summarizing machine

The company I work for recently switched to basically all agentic coding. I have no idea how we afford it. I have a sneaking suspicion that it's going to end quite soon.

Well that makes Kitboga's antics with AI scam calls potentially expensive for the scammers, _Albuquerque New Mexico._

It’s Mike! We click!

What's most impressive is that this man still has lined continuous feed printer paper AND is up on the latest in software development. Rarely seen together in the same place and moment in all of the space time continuum.

"it thinks to itself, right? why wouldn't it?" hilarious

My biggest annoyance is why everybody acts surprised that all these AI tools do not actually cost a flat fee of $10/$20/$30 a month.

"...and don't make any mistakes..." ....Another million tokens 🤣

If you're worried about token costs, one hack is to tell your agent to respond like a caveman

I feel like the basic answer is, that it was never a viable business model to begin with. We had the introductory period where costs were being covered by investors and companies were offering it for lower prices to get people hooked, and they hyped it up to get businesses on board. And now that they're charging what they need to to pay for everything and make their unrealistic profit expectations, people are finally realizing the garbage product is nowhere NEAR useful or worth it.

Imagine a company selling you a machine while having full control over how much consumables it uses and being the only source of said consumables. Nobody would.. oh wait printers exist already.

I hope the prices of tokesn skyrocket so much that this bubble will finally burt and we can have normal RAM prices and PC Hardware...

This can only get worse after the IPOs. When the AI companies go public and shareholders are demanding profitability, the free/20-a-month subscription will be a thing of the past and token cost will sky rocket.I recall Jensen Huang saying earlier this year that token budget will be a perk used to entice the best developers to Silicon Valley. I guess the rest of us will be stuck trying to do something with our local Qwen model on a Mac-mini!

I think the video makes it quite clear that the future of coding agents has to be locally hosted LLMs

Finally I understand AI. Thank You Mike.