What GPT-4o mini Says About OpenAI's Next Model

Jiang Wu · 2024-07-19

*Originally published in Chinese on July 19, 2024.*

OpenAI has released GPT-4o mini, a small model priced far below its predecessors. The first part of this piece lists what the model offers, as the original article laid it out. The second part asks what a cheaper, better model means for OpenAI and for the companies that buy its API.

What GPT-4o mini offers

GPT-4o mini scores 82% on MMLU and currently beats GPT-4 on chat preference on the LMSYS leaderboard.

It is priced at 15 cents per million input tokens and 60 cents per million output tokens, an order of magnitude cheaper than earlier frontier models and more than 60% cheaper than GPT-3.5 Turbo.

Where it fits

With low cost and low latency, GPT-4o mini supports a wide range of tasks. These include chaining or parallelizing multiple model calls (for example, calling several APIs), passing a large amount of context to the model (for example, a full codebase or conversation history), and talking to customers through fast, real-time text responses (for example, customer support chatbots).

Technical features

For now GPT-4o mini supports text and vision in the API, and it will support text, image, video and audio inputs and outputs in the future. The model has a 128K-token context window, supports up to 16K output tokens per request, and has knowledge up to October 2023. Thanks to an improved tokenizer shared with GPT-4o, handling non-English text is now more cost-effective.

What a cheaper, better model means for OpenAI

First, what does a cheaper and better model mean for OpenAI? There are a few angles.

Pushing model capability up and preparing a new model

The first is that the stronger a model is, the stronger its ability to obtain data, and the smarter and more useful it is in every dimension. With cost under control, OpenAI wants users on the best product it can offer, which also pushes the capability of the whole model line forward. To some extent it also suggests that its next-generation model is already ready.

Lowering the cost of trying, widening the user base

The second is that, besides being cheap, this release gives developers free credits, which lowers customers' cost of trying it further. Highly internet-savvy companies have a clear expectation of paying and a habit of paying. For companies with no clear sense of the product or its functions, and for small indie developers, paying is still a barrier. The issue is not the amount but the return on investment. So giving developers a small free quota to try, or a free quota every day, lets them connect at no cost, much as many cloud providers give every developer a free allowance today. It is a very good choice.

The line between small and large models blurs

Beyond cheaper and better results, the boundary between small models and large models is getting less and less clear. Whether a model is large or small may not be obvious to users in the end. For users, cost that is low enough and results that are good enough are the major considerations.

What price cuts and free credits do for data intelligence

There is a consideration and effect beyond the price cut and the free credits: it lets some companies enter data intelligence first, and lets more companies make their data intelligent. In the past, data intelligence meant finding many excellent technical experts to close the loop of a whole data system. Today a very cheap model can start data intelligence inside a company's own data system and may close the data loop. What remains to be seen is when it will actually be free for companies below a certain data volume, or below a certain number of calls.

My feeling is that this day is not far off. It is a bit like the spread of broadband, which got more people (and their data) online first.

Home · All articles · Topics · 简体中文