Nemotron: NVIDIA’s open AI models for agents and self-hosting
NVIDIA doesn’t just make graphics cards. Under the Nemotron brand it releases a family of open AI models — with public weights, training data and recipes — designed mainly for long-running AI agents. It is not a single file to download. It is a range from the small Nano 4B on a laptop to Ultra with roughly 550 billion parameters.
Four things worth knowing
- 1Nemotron is a family. Lightning, Nano, Super and Ultra are roles, not just sizes. Alongside them are Omni, Parse, speech and safety.
- 230B-A3B means about 30 billion parameters in total and 3 billion active per token. It is not a dense 30B model.
- 3The architecture mixes Mamba (cheap long flow), attention (finding a specific fact) and MoE (only a fraction of experts).
- 4The licence differs by release. Ultra and Lightning 3.5 have OpenMDW-1.1; Nano and Super the older NVIDIA licence. Always read the model card.
Mini-glossary: ten terms we will use
A working meaning is enough. Details are in our articles on model names, Mixture of Experts and reasoning models.
- Nemotron
- NVIDIA’s family of open models. Don’t confuse it with NeMo — that is a toolkit for training, customising and routing models.
- Weights, roughly a checkpoint
- The model’s stored learned numbers. Public weights mean you can download the file and run it yourself.
- Active parameters
- How many weights the model actually computes per token. With MoE it is a fraction of the total.
- AI agent
- A program in which the model not only answers but also calls tools, checks the result and takes the next step.
- Mamba
- A layer that reads the sequence linearly. Long text costs it less than classic attention.
- Attention
- A mechanism that finds relationships between tokens. It can pull one specific fact out of a long context.
- MoE
- Mixture of Experts: the model has many internal sub-networks and switches on only a few of them per token.
- NVFP4
- An NVIDIA format that stores weights in roughly 4 bits. Super and Ultra are also trained in it, not just quantised afterwards.
- NIM
- NVIDIA Inference Microservices: a packaged service that deploys the model as an API. The weights themselves are not tied to it — they also run via vLLM or Ollama.
- Token
- A short piece of text. A 1M-token context is a shared budget for input and the new answer — not “a million extra words”.
Nemotron is not one model
The most common mistake is to say “just run Nemotron” and expect one file. NVIDIA’s official wording is twofold and both fit: a family of open models with open weights, data and recipes — and at the same time a family built for long-running, specialised agents.
The current generation is Nemotron 3, supplemented since August 2026 by the 3.5 addition. 3.5 is not a new generation of Nano / Super / Ultra. It is Lightning (the fast execution layer) and specialised models for safety and speech recognition. Older names like Llama-3.1-Nemotron-70B are something else: back then NVIDIA was fine-tuning Llama. Nemotron 3 is now trained by NVIDIA from scratch.
The second common mistake: Nemotron are the models, NeMo is the toolkit (data, training, RL, routing). NeMo can also work with other open and closed models. When someone says “we run it on NeMo”, they are not saying which model is answering.
Why NVIDIA opens the weights
The argument is not “we have the smartest model on the leaderboard”. It is economic. In a long-running agent, most of the time is not consumed by one big deliberation but by thousands of small steps: call a tool, check the output, format the result, hand work to a sub-agent. If you run an expensive frontier model for every such step, the bill and the latency grow faster than the quality.
That is why Nemotron comes out as a library of roles. The big model plans. The small, fast model executes. And since NVIDIA also sells chips, it makes sense to release weights, data and recipes: a company can fine-tune the model, run it on its own network and still stay on the CUDA stack.
For readers in the Czech Republic there is a practical consequence. When you need self-hosting — internal data, air-gap, control over the licence — Nemotron is a candidate alongside Qwen, Llama or DeepSeek. It is not a replacement for ChatGPT for a single prompt.
Nano, Super, Ultra and Lightning
The series names sound like T-shirt sizes. They are more like job positions. Lightning and Nano have a similar number of weights but a different purpose. Super fits on one powerful node. Ultra is the planner for demanding agents.
NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4
We broke down the same logic in the article How to read AI model names. 30B-A3B is MoE shorthand: you store roughly 30 billion parameters and compute about 3 billion per token. Official cards state approximately 31.6 billion in total for Nano and Lightning.
Scroll the table sideways →
| Series | Parameters | Context | Purpose | Release |
|---|---|---|---|---|
| Lightning 3.5 | 30B / 3B | up to 1M | fast agent steps, local deployment | 11. 8. 2026 |
| Nano 30B-A3B | 31.6B / 3.2B | up to 1M | sub-agents, summaries, tools | 15. 12. 2025 |
| Nano 4B | dense ~4B | 262K | edge, Jetson, GeForce, DGX Spark | Nemotron 3 |
| Super | 120B / 12B | up to 1M | multiple agents, tools, one node | 11. 3. 2026 |
| Ultra | 550B / 55B | up to 1M | planning, demanding reasoning | 4. 6. 2026 |
| Nano Omni | 30B / 3B | 256K | image, video, audio and text → text | 28. 4. 2026 |
Fast execution layer
Lightning 30B-A3B has been, since 11 August 2026, the smallest member of the Nemotron 3 family intended for a high number of short steps. NVIDIA positions it as the workhorse under agents such as OpenClaw or Hermes. Licence: OpenMDW-1.1.
Two Nanos, two worlds
Nano 30B-A3B is an open MoE for sub-agents. Nano 4B is a dense small model for the edge — NVIDIA officially publishes a GGUF for it too. It is not a shrunken Lightning.
One node, multiple agents
Super 120B-A12B was released on 11 March 2026 at GTC. The NVFP4 variant lists a minimum of 1× B200 or 1× DGX Spark in its card; BF16 wants roughly 8× H100 80 GB. Licence: NVIDIA Nemotron Open Model License.
The family’s planner
Ultra 550B-A55B has been the largest model of the series since 4 June 2026. NVIDIA presents it as frontier reasoning for complex agents. According to the card, BF16 needs 8× B200 or 16× H100. Licence: OpenMDW-1.1.
Hybrid architecture: Mamba, attention and MoE
A classic transformer computes attention over the whole sequence. That is expensive when the context grows to hundreds of thousands of tokens. Pure state-space models handle a sequence more cheaply but are worse at hitting one specific fact. Nemotron 3 therefore alternates layers.
Mamba-2 carries the long flow cheaply. Without it, a million tokens would remain more of a marketing limit than a usable window. Attention is inserted at selected depths because an agent needs to find one error message, one function in the code or one paragraph of a contract. MoE raises total capacity without the whole model running for every token. That is why 30B-A3B is not a “small Llama 30B”.
Super and Ultra go even further. LatentMoE routes tokens in a compressed space, so more experts fit at the same compute cost. MTP (multi-token prediction) teaches the model to estimate several following tokens at once — in training it gives a longer hint, at inference it serves as built-in speculative decoding. Lightning adds the DSpark and DFlash draft models to MTP.
More advanced practice: NVFP4, thinking and the million-token context
Super and Ultra are also pre-trained in NVFP4, not just quantised afterwards. NVIDIA promises the same file on Blackwell, Hopper and Ampere (on Ampere often via W4A16). Lightning comes out in both NVFP4 and BF16: NVFP4 for deployment, BF16 for further training.
With Nemotron 3, reasoning is not a separate model. It is switched on in the chat template via enable_thinking and, for a number of releases, also via reasoning_budget. Thinking turned off is faster and cheaper; on harder tasks quality drops. More on what happens during “thinking” is in the article Reasoning models.
The “up to 1M” context is the native limit of the Lightning, Nano, Super and Ultra series. Omni has 256K, Nano 4B 262K. The limit does not mean the model uses every million tokens equally well, nor that 1M fits on one consumer card. A long window inflates the KV cache. In the Lightning card NVIDIA states that BF16 on a single H100 is memory-wise closer to 256K; the full 1M wants more GPUs or GB200 / B200.
Two roles in an agent
If you imagine an agent that fixes tickets or writes code all day, it is not reasoning all the time. Most of the time it does routine. NVIDIA draws it as a division of labour: Ultra (or another frontier model) assembles the plan, Lightning or Nano carries the plan out.
NeMo Switchyard builds on this: an open-source router that, according to policy, sends the request to the cheapest model that can handle the task. You don’t have to use it. The same principle can be assembled by hand — what matters is not to burn small steps on a model that belongs only on decisions.
Omni, Parse, speech and safety
The text series are not all of Nemotron. Under the same brand NVIDIA releases narrowly focused models. They are worth knowing so you don’t download Ultra for a task a half-billion-parameter specialist can handle.
Image, video, audio and text
Nano Omni 30B-A3B takes video, audio, image and text and answers in text. Inside is the Nano language backbone, the C-RADIO visual encoder and the Parakeet speech model. Context 256K. Suited to documents, meeting recordings and working with the screen.
Document to structure
Nemotron Parse reads a document page and returns text, tables, bounding boxes and reading order. It is an extractor, not a chat model. The current series is Parse 2.0; the older v1.1 / v1.2 still hang on Hugging Face.
Streaming ASR and TTS
The Speech series handles fast speech recognition, synthesis and translation. 3.5 added a 0.6B streaming ASR. When building a voice agent, don’t use Ultra for transcription.
Guardrails and retrieval
Content Safety is a separate model for moderation and jailbreaks — not a replacement for legal assessment. Retriever / Embed are embeddings for retrieval in RAG, not a generative LLM.
Openness and licences
“Open” with Nemotron means more than a downloadable GGUF from the community. For the main series NVIDIA publishes weights, part of the pre-training and post-training data, RL environments and recipes in NVIDIA-NeMo/Nemotron. That is useful when you want to reproduce the result or specialise the model.
But the licence is not a single one. Nano 3 and Super 3 remain under the NVIDIA Nemotron Open Model License. Ultra and all Nemotron 3.5 (Lightning, Content Safety, ASR) have OpenMDW-1.1. The cards say “ready for commercial use”, but NVIDIA did not relicense the older series retroactively. Moreover, the identifiers are not uniform: sometimes NVIDIA-Nemotron-3-…, other times Nemotron-3-Nano-Omni-…, lowercase for speech. Copy the exact ID from the card.
When to choose Nemotron — and when not
Nemotron makes sense when
You are building an agent with many tools and want to separate planning from execution.
You need self-hosting, data control or fine-tuning on your own tasks.
You already live in the NVIDIA stack (vLLM / TensorRT-LLM / NIM / DGX Spark) and want a model made for it.
You need a long context and the Mamba + MoE hybrid, not just a “big Llama”.
Rather choose something else when
You want a single prompt in the browser — Claude, GPT or Gemini are enough for that.
You need the broadest community of GGUFs and tutorials. Qwen and Llama still lead there.
You only have a consumer card and want painless Ollama. Start with Nano 4B or a smaller Qwen, not Ultra.
The task is purely a document, audio or safety — reach for the Parse / Speech / Safety specialists, not the chat Ultra.
Scroll the table sideways →
| Checkpoint | Minimum per the card | Note |
|---|---|---|
| Lightning NVFP4 | 1× DGX Spark or 1× H100 | 1M context wants more memory than the weights alone |
| Nano 4B | Jetson / GeForce / Spark | Official GGUF, the easiest entry |
| Super NVFP4 | 1× B200 or 1× DGX Spark | BF16: approximately 8× H100 80 GB |
| Ultra NVFP4 | 4× B200 or 8× H100 | BF16: 8× B200 or 16× H100 |
| Nano Omni NVFP4 | 1× RTX 5090 or Spark | BF16 rather 1× H100 80 GB |
How to try it
Frequently asked questions
Is Nemotron better than Llama or Qwen?
There is no single answer. Nemotron is strong where you want open weights plus data and recipes and you are building agents on the NVIDIA stack. Llama and Qwen have a broader ecosystem of conversions, tutorials and community quantisations. Compare a specific checkpoint on your task, not the brand.
Is Nemotron a reasoning model?
With Nemotron 3, thinking is a mode, not a separate series. It is switched on in the template. Thinking off is faster; on harder tasks, on tends to be more accurate. It is not the same as the separate Instruct / Thinking in Qwen3.
Can I use it commercially?
The cards of the main series say so. Ultra and Lightning 3.5 go under OpenMDW-1.1, Nano 3 and Super 3 under the NVIDIA Nemotron Open Model License. Before production, read the current text of the specific repository and have lawyers approve your scenario — especially for fine-tuning and redistribution.
Which Nemotron for a first try?
On a consumer computer, Nano 4B. On one powerful NVIDIA card, Lightning NVFP4 or Nano 30B-A3B. Leave Super and Ultra for the server. If you just want chat without installing anything, use build.nvidia.com.
Why does the same model report 30B one time and 31.6B another?
Marketing rounds. The Nano and Lightning cards state 31.6 billion in total; the active count moves around 3.2–3.6B depending on whether you count embeddings. Read the card, not the nickname.
Want to build agents, not just call APIs?
The AI Agent Developer course teaches the plan → tool → check loop that Nemotron targets. The fundamentals of models and self-hosting are in AI for Developers, Science and Data Analytics and in the shorter Vibe Coding and AI in Development.
Sources and further reading
- NVIDIA: Nemotron 3 Family — Nano, Super, Ultra, hybrid MoE and 1M context
- NVIDIA: Nemotron 3 Ultra — 550B / 55B, LatentMoE, MTP, NVFP4
- NVIDIA Tech Blog: Nemotron 3 Super — released 11 Mar 2026
- NVIDIA Tech Blog: Nemotron 3.5 Lightning — 30B-A3B, agents, OpenMDW-1.1
- Hugging Face: Lightning 30B-A3B NVFP4
- Hugging Face: Nano 30B-A3B BF16
- Hugging Face: Nano 4B BF16
- Hugging Face: Super 120B-A12B NVFP4
- Hugging Face: Ultra 550B-A55B BF16
- Hugging Face: Nano Omni 30B-A3B
- NVIDIA Developer: Nemotron — Parse, Retriever, Speech, Safety
- GitHub: NVIDIA-NeMo/Nemotron — recipes and cookbooks
- Turing Institut: How to read AI model names
- Turing Institut: Mixture of Experts
- Turing Institut: How to host an open-source LLM
- Turing Institut: Reasoning models
