I had to finish this one on my phone, so apologies ahead of time for formatting or spelling errors!

The duality of the AI bubble

Much has been said in the last several months about whether or not the bubble for investment in AI is going to pop—particularly following the acquihires-slash-acquilicenses-slash-whatever of previously hyped AI startups Character.AI, Inflection, and Adept. These are companies that were going after whole generative AI experiences that required massive GPU clusters—the kinds of products that have sent Nvidia rocketing toward a valuation that at one point passed $1 trillion.

The list seems to continue to grow every week. One of the most-hyped AI-first products, the Humane Ai Pin, has seemingly gone… not well. The whole industry in and around AI rode a colossal hype wave to the point that companies even adjacent to “core AI” were picking up valuations upwards of hundreds of times annual recurring revenue.

Now, the big question is, where’s the revenue? Where is the business? Where are the killer apps? Is all this just a complete waste of time with a very limited potential industry that requires colossal upfront investments? Is OpenAI just going to go out of business?

Well, two things can be true at the same time: the foundation model providers building out various colossi and dreaming of artificial general intelligence aren’t living up to the hype right now; and AI is actually really, really useful and already realizing returns within some organizations—just perhaps not in particularly exciting ways.

Instead, the answer to whether or not we’re in an AI bubble is probably more disappointing and frustrating: it’s complicated.

The less difficult, and more valuable, use cases for AI right now

I’ve written about those types of use cases before, but let’s talk about them again: batch processing and robotic process automation (or RPA). These types of use cases are really straightforward and the kinds of things we were doing that pre-date ChatGPT and its neighbors. Summarization, entity extraction, classification, and sentiment analysis are just a few examples—but the point is that you don’t need some massively powerful foundation model that costs $10 per million tokens to do it.

In most of these use cases companies can get away with using models like a customized version of GPT-3.5 Turbo with a system in place to receive relevant information in a specialized database through a process called retrieval augmented generation, or RAG for short. Most of the foundation model providers have these kinds of “workhorse” models for a reason—they accomplish the vast majority of needed tasks and perform at a relatively high level at a relatively low cost.

This extends to what we’d consider “agents” as well. The kind of one-size-fits-all-self-reasoning-autonomous-replace-us-all digital entity seems not only far away, but in most of these very achievable cases completely unnecessary. There are some potential gains to be had through an increasing ability to self-orchestrate, but the near term also points to networks of agents that are again customized to complete a closed number of tasks before passing them off to the next “agent.”

When we talk about “AI in prod,” a which is an umbrella so comically large you could fit a small city under it, the two get bunched together. AGI is the goal to accomplish all these tasks that can… already be accomplished by what’s currently on the shelf. Developers I talk to say customer service ticket routing and escalation is a relatively straightforward mechanism through these models—a very time intensive process that requires a lot of people doing a lot of mundane work. And, to be clear, it doesn’t even replace these representatives. (One source even found some of these super complicated approaches to fine tuning and deploying small models like Llama 3 overkill, which could easily be handled by a fine-tuned GPT-3.5 Turbo.)...