plain, Thoughts on LLMs and the coming AI backlash

I find Large Language Models fascinating. They are a very different approach to AI than most of the 60 years of AI research and show great promise. At the same time they are just technology. They aren’t magic. They aren’t even very good technology yet. LLM hype has vastly outpaced reality and I think we are due for a correction, possibly even a bubble pop. Furthermore, I think future AI progress is going to happen on the app / UX side, not on the core models, which are already starting to show their scaling limits. Let’s dig in. Better pour a cup of coffee. This could be a long one.

plain, Cut through the hype

First I want to cut through all the hype. I’m dismayed that so much funding is going to ML and almost nothing else. What happened to all of the AR/VR startups? My beloved WebXR? But I digress. Back to ML.

A lot of the hype around LLMs stems from saying that we are just moments away from AGI. This is false. I get that the arguments are seductive. Breathless takes like ‘the fastest adoption of technology ever’ make us think that AGI is just around the corner. Moore’s Law lets technology scale exponentially and LLMs were only possible thanks to fast chips, therefore Moore’s Law gets us AGI. Right? No. No it doesn’t.

We have to remember that Moore’s Law doesn’t apply to anything other than transistors. Other tech will not scale the way chips did, unless you can turn it into a chip problem (which is why the mobile revolution seemed so fast and then petered out). Humans are very good at making the same thing over and over again. LLMs are not like transistors. LLMs require huge computational power to do their thing, but they also require massive amounts of data and clever algorithms to apply them. Those don’t scale exponentially. We are running out of public data. And even exponential growth can’t go on forever in the real world. There are always limits. Nothing is forever.

plain, AGI beyond LLMs

LLMs are powerful but are already starting to show their limits. I don’t think a few more rounds of Moore's law is going to get us to AGI, despite what Sam Altman has said. Real AGI is going to take a different approach, combining LLMs with other kinds of systems. LLMs can’t correct themselves because they don’t have logical reasoning. They can generate something, but not know it’s correct beyond a statistical correlation with the corpus of the web it has ingested. These are called hallucinations (though perhaps bullshit is a better term). Humans make these problems too, but we can then check our thoughts: is the thing I just thought I saw likely to be real? LLMs don’t have the ability to say how confident they are. Humans can. Self correction is possible with code generation, because code can be compiled. You can run it to see if it actually does what it should. Anything where the answer can be easily verified is a good use of an LLM.

plain, The Backlash is Coming

I see a backlash coming in the AI industry, and to avoid it we need to focus more on fail-safes and real value. Companies are making bold claims which will not only disappoint the bulls, but also anger the bears. Along the way we’re going to see the web filled with AI slop. Anything free will be flooded. Systems which depend on determining who is a real person or not are starting to break down. I don’t know of a solution beyond charging money. Maybe that’s a good thing. Massive unemployment due to LLMs probably won’t happen (long term we are in for a human labor shortage as the boomers retire) but if a recession is coming AI companies may get a lot of the blame. The backlash is coming.

A few weeks ago we saw a stockmarket correction that saw a sell off of AI and Nvidia stocks triggered by Deep Seek. This was inevitable. I never bought the idea that early AI companies could build a moat around their models. If one company can strap a bunch of GPUs together to train a model from the web then so can another company. LLMs are a technology that is destined to be commoditized. That doesn’t mean there isn’t money to be made; databases are commodities and Oracle is doing fine.

I think the real problem is that commoditization happened faster than people expected, and at the same time current AI apps just aren’t as useful and profitable as expected. This is okay. It has happened many, many times before in tech. It’s a bubble popping, or at least deflating.

I’m old enough to remember the dot-com bust. That didn’t mean the web was a bad idea. Just that these companies were too early and invested too much in technology that was about to be commoditized (fiber, servers, software). The two decades following the dot-com bust were the most productive our industry had ever seen. I think the same will happen with LLMs. It’s a new tool in the toolbox that we haven’t figured out how to use yet.

This doesn’t mean there isn’t value there. It’s simply that AI is a technology, not a product. It’s going to take longer to find the value than expected. The current LLMs just aren’t reliable enough. Over half of LLM-written news summaries have “significant issues” according to a recent BBC analysis.

plain, Things I’m Not Worried About

I’m genuinely not worried about excessive power usage from AI. Deep seek is showing that LLMs can be trained with less computation and energy. Overall power usage for the economy will continue to go up over the next decade, but at a rate consistent with historical averages. AI is a blip and the power company stocks will fall back to earth soon. Power usage is an engineering problem. It’s the kind of problem we know how to solve. Very little work has been put into making these AI systems power efficient yet. I expect that to change over the next few years.

I’m not actually too worried about LLMs being controlled by just a small number of companies. The technology is proving easy to duplicate at scale. Even if we don’t have open models from the big US companies, we likely will have them floating on the Internet sourced by Chinese companies, and efficiency gains will let us run them locally.

I’m also not worried about AI alignment and robots taking over. It’s important to remember that these things aren’t alive or conscious in any meaningful way. I’m not worried about ‘alignment’ because the question itself assumes intent and intelligence that these things don’t actually have. I am worried about ‘bias’, though, the same as I’m worried about bias in psychological studies whose core sample population is Ivy League psych majors. We already have alignment problems with other non-human processes. They are called corporations. What does it mean for a business to be ‘aligned’ with human interests? AIs will need be subject to regulation the same way businesses are. Imperfect, to be sure, but not an existential crisis.

plain, What we should be working on

LLMs are a technology, just like transistors and lasers. They are not products. We haven’t seen the real AI products yet. I feel like the current problems are largely UI and product definition issues. LLMs work best where the cost of failure is low and where a human can review the results.

The problems come when we trust them to work without supervision. We aren’t designing these things to fail, meaning to properly handle failure. Failure will always come. If you don’t handle it you are just doing bad engineering. Consider a company replacing a call center with a chat bot which can’t actually solve the 5% of problems people call human support centers for. It’s not allowed to. Or insurance review boards replacing reviewers with LLMs that fail in some cases, and no way to go back to a human to address the failures. LLMs work best where the cost of failure is low. Is the generated essay too wordy? Just generate it again. Does this scan indicate cancer? Maybe, but we need to delegate to a human for the next step.

Right now we put too much faith in these systems. The computer should never make a promise it can’t keep. Above all, the engineers need humility. Solve the small part of the problem that you can, and be humble that you can’t solve it all and account for it. It’s going to require a lot of work. A lot of experimentation to find the cases where AI provides real value instead of cheap-to-generate slop. Things may get a lot worse before they get better.

plain, Conclusion

We haven’t seen the real AI products yet. Most tools are still just generative. They can’t advise me on what I already have. They don’t have judgment. Where are the tools which analyze my code dependencies and figure out what to swap out with slimmer replacements. But we will get there. Where is the LLM powered security analyzer that can find bugs in my code beyond the heuristics. What would it look like to have a word processor for thinking? Something helps me organize my thoughts, not just rewrite them. An AI that was a trusted research partner. AI as a collaborator, not a replacement. We are a ways away from that still. We need to augment humans, not replace them.