AI Today: Cheap per word, costly when wrong
MiniMax's new model is one of the cheapest of its kind to run. An independent test found it also guesses far more often than it admits it does not know.
On Monday, the Chinese AI company MiniMax revealed who was behind Space Bunny, a model that had been running anonymously and, by MiniMax's count, had already processed more than 200 trillion tokens. It was an early version of MiniMax's new model, M3.1 Flash.
AI Today is a series from Fíor, a natural language classifier that predicts whether a piece of text was more likely written by a person or by AI. Each piece explains one development in AI and shows where the claims come from.
M3.1 Flash is built for AI agents: software that carries out multi-step jobs such as writing code, using computer programs or working through business systems. A token is a fragment of a word, and a million tokens is several novels' worth of text. MiniMax charges $0.15 per million tokens going in and $0.60 per million coming out. It says the model is nearly three times faster than the one before and cuts the cost of agent tasks by 70%. Those figures are the company's own, and Reuters reported them as claims rather than results it had tested.
On price per token, it is among the cheapest models of its kind. Artificial Analysis, which tests AI models independently, scores its intelligence well above the median for its price range.
Cheap per word
The catch is in how it gets there. On Artificial Analysis's test set, the model produced 180 million tokens of output, against a median of 100 million for comparable models. A model that is cheap per word but uses nearly twice as many words is not as cheap as its price list suggests. Measured that way, each test task cost about $0.22.
Speed is less clear-cut than the launch suggests too. MiniMax quotes typical output of more than 170 tokens a second. Artificial Analysis measured about 105 on MiniMax's own service. The conditions differ, so the two are not directly comparable.
Costly when wrong
The bigger question is reliability. VentureBeat reported that, on Artificial Analysis's knowledge test, the model had a 93% hallucination rate. That does not mean 93% of its answers are wrong. It means that of the questions it did not get right, it gave a wrong answer instead of saying it did not know 93% of the time.
For an agent working through a company's systems, that matters more than speed. A cheap answer that has to be checked and redone is not cheap.
MiniMax would point to its benchmark results, and the independent intelligence score backs up much of that. Both things can be true. A model can be capable and still unwilling to say when it is not sure.
MiniMax's own launch announcement, run through Fíor, read as AI and carried five writing-habit flags. The four news reports about it all read as human.
Space Bunny ran through 200 trillion tokens before anyone was told whose model it was. Anyone building on its successor faces a similar question. What matters is not what each answer costs, but how many of them can be trusted without checking.
Try the browser extension at Fíor.