What will businesses build with AI Language Models?

So, what are the economics of implementing AI in businesses? The good news is that you don’t actually, need a team of PhDs and a warehouse of GPUs to make an app which uses AI. All you need to do is have your app send a prompt to OpenAI’s computers in the cloud, and they will send back a response from their GPT-3.5 or GPT-4 language models. This can then be displayed in your app as if you wrote all the clever AI code by yourself. This is achieved by using what is called an API, (or application programming interface) – which is essentially a service that allows software to access other software, as described above.

For many, ChatGPT now stands beside Google as one of the main places to source information on the internet. However, perhaps is it API services we should be talking about. API services allow businesses to build their state-of-the-art AI apps with relatively low expertise required.

Realistically, companies which introduce AI into their products will do so through API services like OpenAI or those of competitors like Google Bard/Gemini, Claude or Perplexity. This aligns with the trend of companies relying more on software-as-a-service solutions, where software is hosted by a third-party provider and made available to customers through the Internet.

Of course, this doesn’t come for free. OpenAI charges a fixed rate based on the number of tokens used (a token is around 4 characters of English text). The GPT-3.5 model is priced at $0.001 per thousand tokens of input and $0.002 per thousand tokens of output. Essentially what this means is that processing an article, such as the one you are reading right now, would cost around $0.005 with GPT-3.5 assuming the same length of input and output. GPT-4 is significantly more expensive, meaning that the same task would cost around $0.07.

So how will companies utilize this? There are two main ways we can approach this question: how will companies use AI APIs to change the way they work, and what new products and services will they create?

How will AI APIs change the way we work?

The first thought that will probably pass through the minds of many is that the cost of using LLMs is far cheaper than human salaries, even at minimum wage. Moreover, humans will be far slower at processing a thousand tokens of text, no matter what the task is. So, does this mean that companies are soon going to be replacing human salaries with API costs?

It’s easy to get the impression that this could happen soon. Language models seemingly know almost everything and can write both code and prose at a level above many humans. It seems that they can do everything that knowledge workers can do. Furthermore, we have just witnessed a brutal round of layoffs in the tech industry. Have companies started doing this already?

The answer is probably not – for now at least. Current AI models just don’t work well enough on their own – they badly need human guidance. On a first pass, they will often misunderstand tasks, get things wrong (hallucination) or use writing cliches. Furthermore, they need to be fed the right information, which for a knowledge worker, is usually quite diverse and not always in text form: websites, emails, charts, word of mouth, etc. The tasks of workers like journalists or software engineers are necessarily multi-prompt tasks – which means that there will need to be someone to do the prompting itself.

The solution may be to let LLMs prompt themselves. This is reminiscent of what humans do – humans don’t get things right the first time either – they need to work things out step by step, as well as write and re-write. Research has shown that ‘flow engineering’ – a technique which designs a pathway for an LLM model to prompt itself can greatly improve the performance of LLM models on coding problems.

But this also introduces complexity, and complexity begets its own problems. A widely acknowledged concept in automation is the Paradox of Automation, which posits that the more complex an automated system becomes, the more crucial the human involvement becomes. This certainly applies here. Implementing ideas like flow engineering requires expertise. Teaching AI models to do tasks beyond just text generation may turn out to be as tricky as training a human in a new job. In the end, it may be worthwhile to do this training. But it will require an initial amount of investment and research. 

So, it’s unlikely that the recent layoffs in the technology sector had much to do with software engineers being replaced with AI. The actual reason appears to be an attempt to ‘right-size’ companies after they over-hired during the pandemic and the preceding decade of low interest rates. Developers are not being replaced by AI, but rather by other developers who have AI expertise – it is these people who are more in demand than ever.

But this is not to say that our jobs will always be safe. The above points just highlight that automation is more likely to happen gradually – easier tasks will be automated first, and harder tasks later. In the immediate future, automation from AI is most likely to come simply from users copying and pasting from the ChatGPT website. Admittedly, this is less interesting than humans being replaced by APIs, but this should be very consequential. Job losses have already occurred where large teams could be replaced by smaller teams which use AI to complement their output. Duolingo is an example of a company where this may have already happened – 10% of contractors at the company have been laid off recently due to the use of generative AI.

What new products and services will come out of AI?

Then what about the second question? Since the API allows any company to easily integrate AI into their apps, the applications of AI are likely to be as diverse as companies are diverse. However, there are some characteristics which could make AI integration more suitable for a certain application:

• Small tasks – meaning less chance of mistakes without human intervention.
• Specific tasks – meaning that an initial piece of prompt engineering will work predictably.
• Low stakes – where mistakes won’t get you fired.
• Unique data – to provide value over what is already in ChatGPT.

Tasks which fit these criteria might include data labelling, summarization of information, and teaching. Many companies have already added AI capabilities to their premium products, which include AI assistants, AI language teachers and AI writing assistants.

The biggest challenge with building apps which use AI is probably competition from ChatGPT itself. Since API costs are not free, AI-integrated apps will often need to come with an additional subscription fee (which will often have to come on top of an existing fee). The price-sensitive user will often find it more worthwhile to simply use ChatGPT themselves, for free. The winners will therefore be those who can feed uniquely useful data to AI models. In many cases, this could just mean personal data – which will be uniquely useful to the user.

In the end, ChatGPT itself is the “killer app” of large language models – if it even needed one.

What about the future?

It is very difficult to predict the future when it comes to AI. This article has therefore tried to reason about what (e.g. flow engineering) is needed, rather than when things will happen. Hype in the AI world moves from model to model: a few years ago, it was self-playing systems like AlphaGo; now it is LLMs. The next model could be something other than LLMs. For LLMs, the big question will be whether additional scaling (more data, more parameters etc) will result in better and better performance. This is still a debate in the AI world, with some believing that LLMs are simply “stochastic parrots”, and others believing that larger and larger models could give rise to more and more human-like intelligence.

Tao Yu, Investment Analyst