AWS releases TorchNeuron to move PyTorch AI apps onto Trainium

Amazon Web Services has released TorchNeuron, a developer tool designed to help AI startups run PyTorch applications on its Trainium chips. Amazon says developers can make the switch with a single configuration-line change rather than rewriting their code.

The launch challenges Nvidia’s software position in AI computing. Nvidia has worked for years to integrate its CUDA software with PyTorch, an open-source toolkit for building, training and running AI models. Amazon estimates that about 85% of AI applications use PyTorch.

Amazon also plans to submit changes to the main PyTorch project that would reduce its dependence on particular chips. The company says it is working with the PyTorch community on those changes, with the aim of expanding hardware compatibility beyond Trainium.

AWS argues that access to its broader cloud services, alongside Trainium’s ability to handle both model training and inference, can lower startups’ overall costs even when its per-token pricing is not the cheapest. Colin Brace, VP of Technology Software at AWS Annapurna Labs, said AI voice application provider Deepgram moved to Trainium using native PyTorch support and processes 70% more audio per dollar at production-grade accuracy.

TorchNeuron is based on PyTorch, making it familiar to developers who already use the toolkit. Other hardware options also support configuration-based switches: PyTorch applications can be moved to AMD GPUs or Apple chips.