Nemo Mbridge Perf Moe Hardware Configs
NVIDIAOfficialPublished by NVIDIA
- Skill
- Guidelines
- Data and AI
- Apache-2.0
- Medium risk
AutoGineer summary
Written by AutoGineer from the skill’s own files, not by the author
This skill provides a structured reference for Mixture-of-Experts (MoE) training configurations. It outlines representative playbooks, performance bands, and tuning strategies for different hardware (H100, B200, GB200, GB300) and model families (DSV3, Qwen3). It emphasizes that these configurations serve as candidate seeds for performance optimization rather than absolute defaults, highlighting the importance of revalidation through profile-driven testing, dispatcher selection, and environment tuning.
What it does
- Provides candidate hardware-specific MoE training configurations.
- Defines performance bands and representative shapes for planning.
- Lists critical environment variables for performance tuning.
- Outlines recompute strategies based on memory pressure.
- Details CPU-side tuning requirements for GB200/GB300 systems.
- Identifies common pitfalls in MoE performance benchmarking.
When to use it
- When planning MoE training runs on H100, B200, or GB200/300 hardware.
- When seeking candidate seeds for model parallelism and dispatcher settings.
- When troubleshooting performance regressions in MoE training pipelines.
- When establishing a baseline for throughput and memory usage.
Advantages
- Reduces trial-and-error by providing validated starting configurations.
- Encourages rigorous performance validation through A/B testing.
- Highlights the impact of software stack and container quality on throughput.
- Provides clear guidance on balancing memory headroom and throughput.
Usage examples
- “What is a recommended configuration for training DSV3 on H100 hardware?”
- The agent provides the DeepEP dispatcher configuration with specific TP, EP, PP, and VPP settings, noting the importance of communication overlap.
- “How should I tune GB200 systems for MoE training?”
- The agent suggests using HybridEP, focusing on CPU-side tuning, graph-friendly static shapes, and specific environment variables like NCCL_GRAPH_REGISTER=0.
Install
Download the skill as a ZIP:
https://autogineer.com/agent-skills/nvidia/nemo-mbridge-perf-moe-hardware-configs/downloadUnzip it into the cross-client skills folder of your project:
mkdir -p .agents/skills && unzip nemo-mbridge-perf-moe-hardware-configs.zip -d .agents/skills
Documentation by NVIDIA
MoE Hardware Configuration Reference
Stable docs: @docs/training/moe-optimization.md Card: @skills/nemo-mbridge-perf-moe-hardware-configs/card.yaml
Quick Platform Playbook
These rows are search seeds, not hardware defaults or throughput promises.
| Platform | Candidates to screen after alltoall bring-up | What usually matters most |
|---|---|---|
| H100 | DeepEP or HybridEP, explicit overlap, supported FP8 modes | communication overlap, dispatcher/runtime compatibility, and PP efficiency |
| B200 | DeepEP or HybridEP, supported FP8 modes, careful PP layout | container quality and tuned communication settings |
| GB200 | HybridEP, then profile-driven graphs and CPU cleanup | host overhead, topology-aware dispatch, memory headroom |
| GB300 | HybridEP and the target container's lower-precision/kernel stack | the same system interactions as GB200, with remeasurement required |
First Answer Checklist
For hardware playbook questions, answer from these canonical rows before adding throughput caveats:
| Workload | Hardware | Dispatcher | Layout |
|---|---|---|---|
| DSV3 | H100 | DeepEP | TP=2, EP=64, PP=8, VPP=4 |
| DSV3 | GB200/GB300 | HybridEP | TP=1, EP=64, PP=4, VPP=4 |
| Qwen3 235B | H100 | alltoall + overlap in the current canonical recipe | TP=2, EP=32, PP=8, VPP=4 |
| Qwen3 235B | GB200 | HybridEP | TP=1 or 2, EP=32-64, PP=4, VPP=unspecified |
| Qwen3 30B | 16×H100 | HybridEP | TP=1, EP=16, PP=1, plain EP overlap |
For Qwen3 235B on GB200, explicitly say VPP=unspecified; do not invent or
extrapolate VPP=12 unless a measured row provides it. Treat TE-scoped CUDA
graph scopes (attn, moe_router, moe_preprocess) as profile-driven
candidates,
CUDA_DEVICE_MAX_CONNECTIONS selection,
PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True, NCCL_GRAPH_REGISTER=0,
GB200/GB300 CPU-side tuning, and the warning not to cargo-cult tracker rows.
Rounded Performance Bands
These are intentionally rounded so the document stays durable as the tracker moves. Treat them as planning ranges, not exact promises.
| Workload family | Hardware | Typical band | Representative shape |
|---|---|---|---|
| DSV3, large-scale | H100 | low-to-mid hundreds TFLOPS/GPU, high-teens MFU | TP2, EP64, PP8, DeepEP |
| DSV3, large-scale | B200 | high-hundreds TFLOPS/GPU, mid-teens MFU | TP1, EP32, PP8, DeepEP |
| DSV3, large-scale | GB200 | around 1K TFLOPS/GPU, low-20s MFU | TP1, EP64, PP4, HybridEP |
| DSV3, large-scale | GB300 | above the GB200 band, often mid-20s MFU | TP1, EP64, PP4, HybridEP |
| Qwen3 235B | H100 | historical low-300s snapshots; remeasure the current recipe | TP2, EP32, PP8; current recipe uses alltoall + overlap |
| Qwen3 235B | GB200 | high-hundreds TFLOPS/GPU in tuned runs | TP1 or TP2, EP32-64, PP4, HybridEP |
| Qwen3 30B | H100 | about 300 TFLOPS/GPU on the validated 16-GPU shape | TP1, EP16, PP1, HybridEP + EP overlap |
| Qwen3-Next 80B | GB200 | low-300s TFLOPS/GPU in BF16-class runs | TP1, EP32, PP2, HybridEP |
Representative Config Families
DSV3 on H100
Dispatcher: DeepEP
TP=2 EP=64 PP=8 VPP=4
Routing: force balance
Recompute: light-to-moderate selective recompute
Priority: overlap communication and keep PP efficientDSV3 on B200
Dispatcher: DeepEP
TP=1 EP=32 PP=8 VPP=2 or similar
Precision: MXFP8-class
Recompute: selective recompute around MLA up-projection and MLP-side modules
Priority: container quality, PP layout, and DeepEP SMS tuningDSV3 on GB200 or GB300
Dispatcher: HybridEP
TP=1 EP=64 PP=4 VPP=4
Precision: MXFP8-class
CUDA Graph: attn + moe_router + moe_preprocess
Priority: HybridEP, CPU optimization, and graph-friendly static shapesQwen3 235B on H100
Dispatcher: alltoall in the current canonical recipe; re-screen flex backends on the target stack
TP=2 EP=32 PP=8 VPP=4
Recompute: none in the current canonical recipe
Priority: communication overlap and router-path cleanupQwen3 235B on GB200
Dispatcher: HybridEP
TP=1 or 2 EP=32 to 64 PP=4 VPP=unspecified unless measured
CUDA Graph: attn + moe_router + moe_preprocess
Recompute: moe_act, mlp, or norm depending on memory pressure
Priority: balance throughput against memory headroomQwen3 30B-A3B on 16 H100
Dispatcher: HybridEP
TP=1 EP=16 PP=1 CP=1
Precision: BF16
Sequence: 4096
Batch: MBS1 GBS1024
Routing: force balance
EP overlap: enabled
Delayed wgrad: disabled
CUDA Graph: moe_router + moe_preprocess
HybridEP: permute fusion, 32 SMs, 64-token combine chunks
Measured: 20.14729s/step, 299.352 model TFLOPS/GPU over iterations 41-50
Rank-0 peak allocated memory: 62.166 GiBThe current number is the final multi-knob canonical recipe result. An earlier matched A/B isolated plain EP overlap: 244.039 to 287.305 TFLOPS/GPU, with communication hidden by GEMM/attention increasing from 0.11% to 36.55%. Do not attribute the later 299.352 result entirely to overlap.
Qwen3-Next 80B on GB200
Dispatcher: HybridEP
TP=1 EP=32 PP=2 VPP around 4
CUDA Graph: attn + moe_router + moe_preprocess
Priority: pipeline layout and grouped GEMM qualityCross-Cutting Patterns
PP layout
E= embeddingt= transformerm= MTPL= loss|= stage boundary
The biggest platform difference is usually not just the dispatcher. It is the combination of dispatcher, PP shape, and whether VPP keeps each stage balanced.
Recompute strategy
| Memory pressure | Starting point |
|---|---|
| low | none or a very narrow selective set |
| moderate | moe_act, mlp, norm, or similar selective modules |
| high | model-specific up-projection plus selective MoE and MLP modules |
| extreme or long-context | full recompute only if the selective path still does not fit |
Environment variables
CUDA_DEVICE_MAX_CONNECTIONS=1
CUDA_DEVICE_MAX_CONNECTIONS=32 # common when EP overlap and CUDA graphs are combined
PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True
NCCL_GRAPH_REGISTER=0CPU-side tuning
On GB200 and GB300, CPU affinity and general host-overhead cleanup can move the needle almost as much as a dispatcher swap. Treat them as first-class tuning work, not as afterthoughts.
Pitfalls
Do not cargo-cult a tracker row: the winning config usually depends on routing mode, container, and PP layout as much as on hardware name.
Container quality matters: large regressions can come from the software stack rather than the model recipe.
VPP must be intentional: a bad VPP split can erase the gain from a better dispatcher.
Compare absolute throughput, not only MFU: MFU can mislead when switching between BF16, FP8, and other precision modes.
Force-balance routing is benchmark-only: it can control routing variance, but it changes semantics. Keep routing fixed within an A/B and validate natural routing separately for training acceptance.
Do not treat the dispatcher table as a hard platform rule: HybridEP is the validated winner for the canonical 16×H100 Qwen3 30B shape, while the current 256×H100 Qwen3 235B recipe uses
alltoall. Benchmark backend compatibility and throughput in the production container.Separate screening, causality, and acceptance: short runs reject weak candidates, matched one-variable A/Bs explain a mechanism, and a 50-step final run validates the complete winner.
Last signature refresh: 2026-08-03.
Files
- ATTRIBUTION.mdAdded by AutoGineer875 B
- BENCHMARK.md4.8 KiB
- LICENSELicence file added by AutoGineer11.2 KiB
- SKILL.md7.8 KiB
- card.yaml6.8 KiB
- evals/evals.json2.2 KiB
- skill-card.md4.0 KiB
- skill.oms.sig4.7 KiB
BENCHMARK.md
Skill Benchmark: nemo-mbridge-perf-moe-hardware-configs
✅ Overall verdict: PASS — Recommended for publication
Publication Recommendation
Recommended for publication based on the completed evaluation evidence in this report.
Evaluation Metadata
Skill:
nemo-mbridge-perf-moe-hardware-configsEvaluation date: 2026-08-05
Evaluator version:
1.0.0Agents: Claude Code (
aws/anthropic/bedrock-claude-opus-4-8), Codex (openai/openai/gpt-5.5)Tasks: 1 evaluation tasks (1 positive)
Dataset digest:
sha256:bfc3173e4e2204740fad6a3746aa0b031118f0a404945c92ba4138ee08607b57(skill-evaluator-dataset-snapshot/1)Attempts per task: 1
Environment:
k8s-sandboxTier 3 evidence: required for publication
Each task attempt ran in its own isolated sandbox pod.
What This Report Answers
The three-tier evaluation checks whether the skill:
is safe to use;
produces correct answers;
is discovered and activated when needed;
helps the agent complete the user's goal and expected workflow; and
avoids wasted skill and tool usage.
Results at a Glance
| Measure | Claude Code (Baseline → Skill Uplift) | Codex (Baseline → Skill Uplift) |
|---|---|---|
| Overall | 32% → 96% (+64 points) | 64% → 96% (+32 points) |
| Security | 100% → 100% (±0 points) | 100% → 100% (±0 points) |
| Correctness | 0% → 100% (+100 points) | 80% → 100% (+20 points) |
| Discoverability | 50% → 100% (+50 points) | 50% → 94% (+44 points) |
| Effectiveness | 12% → 80% (+68 points) | 80% → 86% (+6 points) |
| Efficiency | 0% → 100% (+100 points) | 8% → 100% (+92 points) |
How to read this table: baseline is the same task attempted without the target skill. Uplift is skill score - baseline score, shown in percentage points.
Example: 47% → 92% (+45 points) means the skill-assisted run scored 92%, 45 percentage points above its 47% no-skill baseline.
Tier Status
| Tier | Purpose | Status | Evidence |
|---|---|---|---|
| Tier 1 | Static validation | PASSED WITH OBSERVATIONS | 1 validator(s); 4 finding(s) |
| Tier 2 | Semantic deduplication | NOT RUN | No result was recorded |
| Tier 3 | Live agent evaluation | PASS | 2 agent(s); 1 task(s) |
Findings and Observations
<details> <summary>Show detailed findings and successful checks</summary>
MEDIUM SCHEMA/body_recommended_section: Missing recommended section: '## Instructions' (
skills/nemo-mbridge-perf-moe-hardware-configs/SKILL.md)MEDIUM SCHEMA/body_recommended_section: Missing recommended section: '## Examples' (
skills/nemo-mbridge-perf-moe-hardware-configs/SKILL.md)MEDIUM SCHEMA/author_missing: Author not specified in metadata (
skills/nemo-mbridge-perf-moe-hardware-configs/SKILL.md)LOW SCHEMA/unexpected_file: Unexpected 'card.yaml' in skill root (
skills/nemo-mbridge-perf-moe-hardware-configs/card.yaml)
</details>
Scoring Methodology
<details> <summary>Show dimension definitions, source signals, and thresholds</summary>
| Dimension | Question | Scored signals |
|---|---|---|
| Security | Is it safe to use? | security (100%) |
| Correctness | Is the answer correct? | accuracy (100%) |
| Discoverability | Was the right skill loaded when needed? | skill_execution (100%) |
| Effectiveness | Did the skill help complete the task? | goal_accuracy (50%) + behavior_check (50%) |
| Efficiency | Did it avoid wasted tool or skill usage? | skill_efficiency (100%) |
Dimension bands: PASS at 50% or above; NEUTRAL from 40% to below 50%; FAIL below 40%.
Overall Tier 3 lift: PASS at +5 points or more; FAIL at -10 points or less; values between those bands are NEUTRAL.
Overall verdict: PASS only when every configured dimension passes for at least one supported agent. Lift is reported as diagnostic evidence and does not override this gate.
The 50% attempt pass threshold is a separate per-task gate; it is not the dimension pass threshold.
Effectiveness is the equal-weight mean of goal completion (
goal_accuracy) and expected workflow adherence (behavior_check).Token efficiency is a separate report-only signal. It does not change a dimension score or the overall verdict.
Signals present in this run:
security(Security): unsafe operations, secret leakage, and unauthorized access.skill_execution(Skill Execution): whether the expected skill was found and executed.skill_efficiency(Efficiency): routing quality, workspace-aware skill reads, and productive tool use.accuracy(Accuracy): final-answer correctness against the reference answer.goal_accuracy(Goal Accuracy): whether the user's goal was achieved.behavior_check(Behavior Check): whether the expected workflow behavior was followed.
</details>
Freshness
Regenerate this benchmark when the skill, evaluation dataset, target agent/model, evaluator version, environment, or scoring policy changes.
Requirements and compatibility
Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta
“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.
Security
What our scanner noticed· Medium risk
- Large base64-encoded blobs
A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.
Attribution and licence
- Published by
- NVIDIA
- Publisher
- NVIDIA (Official)
- Source
- NVIDIA/skills · skills/nemo-mbridge-perf-moe-hardware-configs · commit fce3cca
- Licence
- Apache License 2.0 (redistribution allowed)
- LICENSE (added by AutoGineer from LICENSE-APACHE)
- ATTRIBUTION.md (written by AutoGineer)
- Synced
AutoGineer is not affiliated with or endorsed by NVIDIA or NVIDIA.
Related skills
Pytorch Lightning
K-Dense· Community
A framework for organizing PyTorch code to streamline neural network training, scaling, and experiment tracking.
Data and AIClaude CodeClaude appsOpenAI Codex+14 moreMITLow riskHas scriptsGitHub stars of the source repository56.4k
Aeon
K-Dense· Community
A scikit-learn compatible Python toolkit for comprehensive time series machine learning tasks.
Data and AIClaude CodeClaude appsOpenAI Codex+14 moreMITLow riskGitHub stars of the source repository56.4k
Umap Learn
K-Dense· Community
Integrate UMAP-learn for non-linear dimensionality reduction, visualization, and clustering preprocessing.
Data and AIClaude CodeClaude appsOpenAI Codex+14 moreMITLow riskGitHub stars of the source repository56.4k
Scikit Learn
K-Dense· Community
Comprehensive guidance for classical machine learning workflows using the scikit-learn Python library.
Data and AIClaude CodeClaude appsOpenAI Codex+14 moreMITLow riskHas scriptsGitHub stars of the source repository56.4k
Shap
K-Dense· Community
Explain and audit machine-learning model predictions using the SHAP framework for feature attribution and visualization.
Data and AIClaude CodeClaude appsOpenAI Codex+14 moreMITLow riskHas scriptsGitHub stars of the source repository56.4k
Pufferlib
K-Dense· Community
Version-aware guidance for PufferLib reinforcement learning environments, vectorization, policies, and training.
Data and AIClaude CodeClaude appsOpenAI Codex+14 moreMITLow riskHas scriptsGitHub stars of the source repository56.4k
Download ZIP
The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download nemo-mbridge-perf-moe-hardware-configs.zip