The Price Per Token Is Dropping. Will It Stay That Way?
LLMs are at the heart of the current AI movement. An LLM’s ability to respond to prompts with human-like responses is astonishing. All of our first interactions with ChatGPT in 2022 were amazing.
The first time I used GitHub Copilot to generate a basic database access method, it was clear that something significant was about to happen. This wasn’t just a fade or a new tool that enabled a little bit more value.
Since the release of ChatGPT a few years ago, we have seen AI systems impact almost all elements of software, not just software development.
But with LLMS, there has always been a little bit of uneasiness around any solution. What happens to the price of tokens? Well, Epoch AI has an interesting chart. That shows the price of tokens falling over time.
As we build solutions on top of LLMs, we are tightly coupling ourselves to their success. If our solution requires a tight coupling of an LLM to build out the solution, the price per token could determine our success or failure in the long term. Luckily, according to the chart, the price per token seems to drop. Now it is a little unclear as to why the price per token is dropping, and I am still a little worried that the price per token will, at some point, go up.
I have heard many people say that at some point, tokens will start to cost what they really cost. But the chart above makes me wonder. Maybe the prices are what they should be. The stories about the lack of profitability in LLMs make me still wonder, though.
Also, as an aside, it is becoming more and more expensive to train an LLM.
According to another graph at Epoch AI, the cost to train a model doubles every eight months. This tends to align with the common wisdom that we are getting a really good deal right now, while everyone is fighting to build market share.
Regardless of how this turns out, it is certain that this is a fun ride.
