NVIDIA Broadens Open AI Platform With Alpamayo 2 Super For Level 4 Autonomous Driving Infrastructure And Advanced Situational Awareness
NVIDIA is broadening its open AI platform with its new 32 billion parameter vision language action model for reasoning, Alpamayo 2 Super. Alpamayo 2 Super will assist with Level 4 autonomous robotaxis with advanced understanding and prediction of complex driving environments and safe trajectory decisions. Through the release of this open model, NVIDIA aims to reduce the effort required for developers to develop foundational autonomous vehicle infrastructure from scratch.
The official NVIDIA product release stated that Alpamayo family can now scale up to 32 billion parameters which shows it is an upgrade to the 10 billion parameter Alpamayo 1 Nano model. Alpamayo 2 Super is built upon Cosmos foundation models to allow for enhanced spatial reasoning, trajectory prediction in rare driving scenarios, and provides 360 degree situational awareness to take data from surround cameras instead of front facing cameras to help execute safer highway merges and intersection crossing.
One key addition is the use of Meta Actions, where in addition to physical paths, Alpamayo 2 Super makes high level decisions such as yield, stop, change lane, in addition to improved chain of causation trace to allow the system to explain its decisions. To help with data pipelines required for these models NVIDIA integrated reasoning auto labeling with 2D grounding. This new technology can automatically label raw driving footage, significantly reducing data preparation time from months to days without human supervision.
NVIDIA designed the model to be used as a teacher model; this teacher model can distill its knowledge into highly optimized smaller models that will run directly in the car on hardware such as the DRIVE AGX Thor. NVIDIA CEO and founder Jensen Huang highlighted this model as a significant advancement to the automotive industry:
Alpamayo is the moment cars begin to safely reason, not just drive. Only NVIDIA makes available open models, simulation, real world data and agent skills so the entire global robotaxi ecosystem can develop level 4 capabilities that understand edge cases, explain decisions, earn trust and scale safely to millions of vehicles.
In addition to this model NVIDIA has also made its open source closed loop reinforcement learning framework called AlpaGym publicly available. Unlike open loop training where an AI is tested against pre recorded driving data, AlpaGym trains the AI on a continuous loop where all the decision making goes through AlpaSim which makes each steering, braking, and acceleration decision directly impact the surrounding world in the simulation to compound errors, and discover edge case failures.
The platform also uses OmniDreams to allow developers to create photorealistic scenes, and Omniverse NuRec to let them reconstruct real world driving logs into 3D synthetic training environments with high detail. The model weights are scheduled to be available on Hugging Face and the inference code available on GitHub to allow for open source development of Alpamayo 2 Super. This open source development comes after a strong reception from the previous two versions of Alpamayo, receiving over 400,000 downloads.
