Amazon opens SGLang-Neuron so Trainium runs models on the familiar SGLang interface

Amazon announced on November 12 that SGLang-Neuron is publicly available. Developers can now use the familiar SGLang interface to serve models and generate reinforcement-learning (RL) rollouts on AWS Trainium, the company's custom AI chips, through an AWS Deep Learning Container released by RadixArk in collaboration with Annapurna Labs.

How the plugin works

SGLang-Neuron is an out-of-tree plugin that connects the open-source model-serving framework SGLang with Trainium, built on TorchNeuron, AWS's native PyTorch backend, which registers Trainium as a native PyTorch device. Developers keep SGLang's existing server interface and generation APIs and select Trainium support by setting SGLANG_PLATFORM=neuron when the server starts.

Coverage and limits

Public availability follows a limited beta that began on October 22 for a small group of teams, supporting Trainium2 and Trainium3, with initial model coverage including Qwen3, DeepSeek-V2-Lite and GLM models. AWS's walkthrough runs openai/gpt-oss-120b on a Trainium3 EC2 instance with a 10,240-token context and up to 64 concurrent requests — example deployment settings, not comparative performance results. Amazon has published no performance or price comparisons against other chips. (Photo: file photo.)