NVIDIA Unveils Nemotron 3 Super, a 120-Billion-Parameter Agent Model Challenging Claude Opus 4.6

NVIDIA has introduced Nemotron 3 Super, a 120-billion-parameter open-source model engineered for large-scale AI agents. The company states its performance approaches that of Claude Opus 4.6 and GPT-5.4, with inference speeds up to three times faster and throughput surging by a factor of five.

NVIDIA logo prominently displayed with abstract AI graphics.

The model features 12 billion active parameters and a 1-million-token context window. NVIDIA indicates Nemotron 3 Super is designed to address common challenges in multi-agent applications, such as context explosion and the computational burden of complex reasoning.

Nemotron 3 Super Architecture and Capabilities

Nemotron 3 Super incorporates a Mamba-MoE hybrid architecture, which NVIDIA claims resolves performance bottlenecks in multi-agent collaboration. Key architectural advancements include native pre-training with NVFP4 precision, a LatentMoE hybrid expert architecture for optimizing accuracy per unit of compute and parameter, and a Multi-Token Prediction (MTP) layer enabling native "speculative decoding" for faster inference.

Visual representation of a Mamba-MoE hybrid AI architecture with glowing components.

On the Pinchbench benchmark, Nemotron 3 Super reportedly holds the top position among open-source models, achieving an 85.6% success rate in OpenClaw tasks. The model's pre-training and post-training datasets, totaling over 10 trillion tokens, along with its training methodology and 15 reinforcement learning environments, have been open-sourced.

Addressing Agent Application Challenges

Multi-agent applications often face two primary hurdles: context explosion and high computational costs. Context explosion occurs because multi-agent workflows can generate up to 15 times more tokens than standard conversations, leading to increased costs and potential goal drift. The "thinking tax" refers to the high cost and slow processing when a large language model is called for every subtask in complex agent reasoning.

Abstract visualization of 'context explosion' with overwhelming data streams.

NVIDIA states that Nemotron 3 Super's 1-million-token context window allows AI agents to maintain the entire workflow state in memory, ensuring logical consistency. The model has set a new state-of-the-art (SOTA) on Artificial Analysis for efficiency and open-source performance. The NVIDIA AI-Q research AI agent, powered by Nemotron 3 Super, also secured first place on the DeepResearch Bench and DeepResearch Bench II leaderboards.

Architectural Innovations for Performance

Nemotron 3 Super's underlying structure features an 88-layer network with a periodically alternating arrangement of Mamba-2 layers for efficient sequence modeling and Transformer attention layers acting as "global anchors" for long-range information routing. This design reportedly boosts throughput by up to five times and accuracy by up to two times compared to the previous Nemotron Super model.

Compared to GPT-OSS-120B and Qwen3.5-122B, Nemotron 3 Super achieved higher scores in internal evaluations. With an input sequence length of 8k and an output sequence length of 64k, its throughput was up to 2.2 times higher than GPT-OSS-120B and 7.5 times higher than Qwen3.5-122B.

Visual explanation of LatentMoE and Multi-Token Prediction processes in AI.

LatentMoE and Multi-Token Prediction

The introduction of "LatentMoE" in Nemotron 3 Super involves projecting tokens to a smaller latent dimension before routing and expert computation. This reduces the number of expert parameters loaded and inter-card communication, allowing for scaling the total number of experts and activated experts per step without significant changes in inference cost. NVIDIA describes this as activating four experts for the computational cost of one.

Multi-Token Prediction (MTP) aims to improve both model quality and inference efficiency. Unlike traditional methods that predict the next token, MTP requires the model to predict several future tokens simultaneously. This approach reportedly enhances the model's understanding of causal relationships and longer-range text structures. MTP also enables native speculative decoding, where additional prediction heads act as a built-in "draft model," generating candidate tokens that the main model then verifies in a single forward pass, reducing generation latency.

Training and Agent Capabilities

Nemotron 3 Super underwent pre-training on the Blackwell platform using NVFP4 precision, which NVIDIA states significantly reduces memory requirements and offers four times faster inference speed than FP8 on the Hopper architecture without accuracy loss.

The model was trained on 25 trillion tokens of text data across two stages. The first stage, using 80% of the data, focused on diversity and broad knowledge from 16 categories. The second stage, using the remaining 20%, consisted of high-quality data with increased weighting for Wikipedia, high-quality PDFs, and STEM reasoning data to boost accuracy. The resulting base model achieved scores of 86.01 on MMLU, 75.65 on MMLU-Pro, and 84.84 on MATH.

NVIDIA heavily invested in AI agent capabilities during post-training, with over 7 million samples and 80 billion tokens used in the SFT stage. Agent-related tasks accounted for 36% of the data mix. The RL stage involved four steps: multi-environment RLVR across 21 environments, SWE-RL for software engineering capabilities, RLHF using a GenRM reward model for identity cognition and safety, and MTP recovery to re-align speculative decoding accuracy.

These training efforts resulted in Nemotron 3 Super achieving 60.47% on SWE-Bench (OpenHands), 91.75% in RULER@1M long-context testing, and 90.21% on AIME25 mathematical reasoning.

Applications and Enterprise Focus

Nemotron 3 Super's tool invocation capabilities are designed to enhance OpenClaw agents in various domains. In software development, it can load entire codebases into context for end-to-end code generation and debugging. For financial analysis, it can process thousands of pages of reports in memory, improving efficiency. In cybersecurity, it supports autonomous security orchestration by enabling agents to navigate function libraries reliably.

Several companies are already integrating Nemotron 3 Super. Perplexity has incorporated it for user search. Software development AI agent providers like CodeRabbit, Factory, and Greptile have integrated it into their agents. Edison Scientific and Lila Sciences are using Nemotron 3 Super to power agents for literature retrieval, data science, and molecular structure understanding.

Software developers collaborating on AI projects in a modern office.

NVIDIA is also reportedly developing an open-source AI agent platform called NemoClaw for the enterprise market. The platform aims to provide built-in security and privacy tools, addressing enterprise concerns about existing open-source agent frameworks. NemoClaw is designed to be open-source and compatible with various systems, regardless of NVIDIA chip usage.