AI Today: Amazon promises no rewrite. Its showcase customer took months.
Amazon's new tool lets AI code run on its own chips with one setting changed. Getting that code to run cheaply is a different job.
For its launch, Amazon ran the same AI training script twice: once on a conventional chip, and once on its own Trainium chip. The only difference, the company says, was a single setting telling the code which chip to use. The model learned the same things at the same rate.
AI Today is a series from Fíor, a natural language classifier that predicts whether a piece of text was more likely written by a person or by AI. Each piece explains one development in AI and shows where the claims come from.
The tool is called TorchNeuron, and Amazon announced it on 19 October. Most AI code is written with PyTorch, a free software toolkit. Amazon puts the share of AI developers using it at 85%. Until now, moving PyTorch code onto Amazon's chips meant adapting it first. TorchNeuron plugs Trainium into PyTorch's standard way of adding new hardware, so the code, and the tools around it, carry over. Amazon says it will release it as open source on 20 October, and Reuters reports that more than 30 companies used it in a private beta.
The reason this matters is Nvidia. It dominates the chips that train and run AI, and part of its hold is software. Years of code and tools are built around Nvidia's own platform, CUDA, and moving away means rework. If code can move freely between chips, chipmakers have to compete on price and performance instead.
The months behind "no rewrite"
On the same day, Amazon published a technical case study with Deepgram, a speech recognition company and one of its showcase customers. It is the most detailed public account so far of what moving to Trainium involves.
Deepgram's model did start as standard PyTorch code, and TorchNeuron got it running on Trainium. Then came several months of work. Engineers from both companies replaced parts of the model with specialised code, wrote new components from scratch where none existed, and built a separate program in another language to serve the model in production. The result, by the companies' own measure, was 11 to 70% more audio transcribed per dollar on Deepgram's batch workloads.
None of this contradicts Amazon's claim. Running the code took one setting. Making it cheaper took a team. Those are different promises, and the second is the one that decides whether anyone switches.
There are limits on the first promise too. The Next Web reported that the initial release does not yet support every PyTorch feature.
"No rewrite" describes the first step. Amazon's own showcase shows where the work went: not into making the code run, but into making it pay.
Try the browser extension at Fíor.