Nemotron Speech

NVIDIAOfficialCreated by Nemotron Speech TeamDistributed by NVIDIA

Skill
Tool integration
Data and AI
Apache-2.0
Medium risk
Has scripts

From the author

Routes NVIDIA Nemotron Speech (Formerly Riva) NIM tasks — deploys, runs, and tests ASR, TTS, and NMT NIMs on build.nvidia.com or self-hosted.

— the description by Nemotron Speech Team

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

Nemotron Speech acts as a central command interface for managing NVIDIA's speech AI NIMs. It provides a structured workflow for users to navigate deployment options, whether using cloud-hosted inference via build.nvidia.com or self-hosted Docker containers. The skill handles the routing of tasks across ASR, TTS, and NMT modalities, ensuring users follow the correct procedures for model selection, pipeline configuration, and system readiness checks.

What it does

Routes users to specific workflows for ASR, TTS, and NMT deployment.
Provides guidance on self-hosted Docker deployment and cloud-hosted inference.
Assists with custom model builds using riva-build and riva-deploy.
Facilitates ASR pipeline tuning including VAD, diarization, and language models.
Supports TTS customization such as SSML, zero-shot voice cloning, and pronunciation dictionaries.
Directs users to authoritative documentation for version-specific model and hardware requirements.

When to use it

When you need to deploy or test NVIDIA Riva/Nemotron Speech NIMs.
When configuring custom ASR or TTS pipelines for specific accuracy or vocabulary needs.
When troubleshooting deployment readiness, GPU compatibility, or container health.
When integrating speech-to-text, text-to-speech, or translation into an application.

Advantages

Centralizes complex speech AI workflows into a single, navigable interface.
Ensures adherence to official NVIDIA documentation and best practices.
Simplifies the transition between cloud-hosted and self-hosted environments.
Provides clear, step-by-step procedures for custom model deployment.

Usage examples

“Deploy a Parakeet ASR NIM”: The agent loads the ASR reference file and guides the user through the self-hosted deployment steps.
“How do I fix the pronunciation of NIM in Riva TTS?”: The agent loads the TTS pronunciation reference, suggests IPA candidates, and provides the necessary gRPC snippet.
“Can my GPU run this?”: The agent initiates the deployment readiness check workflow to verify system requirements.
“Translate English to German”: The agent loads the NMT reference file and walks the user through the translation deployment flow.

Install

Download the skill as a ZIP:
https://autogineer.com/agent-skills/nvidia/nemotron-speech/download

Unzip it into the cross-client skills folder of your project:

mkdir -p .agents/skills && unzip nemotron-speech.zip -d .agents/skills

Documentation by Nemotron Speech Team

Nemotron Speech Skills

Note: "Nemotron Speech" is the public-facing name for what NVIDIA documents today as Riva / Riva NIM. All commands, container images, gRPC APIs, Python imports, and documentation URLs still use "Riva" — the rename is brand-only. Do not rename commands, images, or doc URLs.
Agent: When walking the user through a multi-step workflow, announce each step before presenting it: Step N/M — Step Title (e.g., "Step 1/4 — Deploy the Container").

Purpose

Single entry point for all NVIDIA Nemotron Speech (Riva) NIM workflows: ASR (speech-to-text), TTS (text-to-speech), and NMT (translation). Covers cloud-hosted inference via build.nvidia.com, self-hosted Docker deployment, client-protocol choice for ASR (gRPC, HTTP, WebSocket), custom NeMo model deployment via riva-build, ASR pipeline tuning (VAD, diarization, language models), and the prerequisite Docker / NGC / driver setup.

When to Use This Skill

Use this skill for any Nemotron Speech / Riva NIM task — deployment, testing, custom model build, system requirements check, or model selection across ASR / TTS / NMT modalities.

Workflow

Identify the user's task type, then load the corresponding reference file from references/. The reference files contain the detailed per-workflow content; this SKILL.md is a routing surface. Load only the reference relevant to the task at hand.

Prerequisites

For self-hosted deployment: NVIDIA AI Enterprise (NVAIE) entitlement, then complete the environment setup — NVIDIA drivers, Docker, Container Toolkit, NGC API key, Riva Python client. See references/setup.md.
For cloud-hosted inference: pip install -U nvidia-riva-client and a valid NVIDIA_API_KEY from https://build.nvidia.com.
Treat NVIDIA_API_KEY and NGC_API_KEY as secrets: never print, paste, commit, or log real key values. Prefer --password-stdin for Docker login and store persistent keys in a credential manager or a chmod 600 env file rather than world-readable shell startup files.
For self-hosted Docker model caching: host directories mounted at /opt/nim/.cache must be writable by the container user (the NIM container runs as nvs:1000 internally), not just the host user. Run sudo chown 1000:1000 $LOCAL_NIM_CACHE after creating the directory so the container can write to it. Avoid world-writable modes — they let any local user replace cached model artifacts. Also avoid -u "$(id -u):$(id -g)" on the docker run — /opt/nim/workspace inside the container isn't writable to arbitrary UIDs. If you see I/O error Permission denied (os error 13) during model download, the host directory ownership is the issue.

Instructions

Match the user's task to one reference file and load only that file; the references are detailed, so progressive disclosure keeps context tight.
Route setup requests for drivers, Docker, Container Toolkit, and NGC to references/setup.md.
Route GPU compatibility, deployment readiness, and container health checks to references/deployment-readiness-checks.md.
Route model choice across ASR, TTS, and NMT to references/model-selection.md.
Route ASR deployment or inference for Parakeet, Canary, Whisper, and Nemotron ASR Streaming to references/asr.md.
Route custom-trained NeMo ASR deployment (.nemo → RMIR → NIM) to references/asr-custom.md.
Route ASR pipeline configuration for VAD, diarization, language models, and chunk size to references/pipelines.md.
Route TTS deployment or inference for Magpie to references/tts.md.
Route custom or fine-tuned TTS model deployment (.nemo → RMIR → NIM) to references/tts-custom.md.
Route TTS synthesis pipeline configuration — SSML, zero-shot voice cloning, applying an existing pronunciation dictionary, audio encoding, sample rate, custom_configuration keys — to references/tts-pipelines.md. (For discovering and constructing the pronunciation itself, use tts-pronunciation.md.)
Route TTS pronunciation discovery — finding, testing, and applying IPA pronunciations for specific words or phrases — to references/tts-pronunciation.md.
Route NMT deployment or inference for Riva Translate, language pairs, and DNT tags to references/nmt.md.

Source of truth

For per-release detail — current model catalog, container IDs, function IDs, voice lists, VRAM minimums, per-model feature support — fetch or open the canonical NVIDIA doc rather than relying on text in this SKILL.md or the references. Each reference file includes its own routing table to the relevant doc pages.

Top-level landing pages:

Topic	URL
ASR support matrix	https://docs.nvidia.com/nim/speech/latest/reference/support-matrix/asr.html
TTS support matrix	https://docs.nvidia.com/nim/speech/latest/reference/support-matrix/tts.html
NMT support matrix	https://docs.nvidia.com/nim/speech/latest/reference/support-matrix/nmt.html
Prerequisites (driver / GPU / OS)	https://docs.nvidia.com/nim/speech/latest/get-started/prerequisites.html
ASR pipeline configuration	https://docs.nvidia.com/nim/speech/latest/asr/customization/pipeline-configuration.html
ASR runtime customization	https://docs.nvidia.com/nim/speech/latest/asr/customization/customization.html
TTS custom deployment (`.nemo` / `.riva`, `riva-build`, RMIR)	https://docs.nvidia.com/nim/speech/latest/tts/custom-deployment.html
TTS request-time customization (SSML, pronunciation dictionaries, `custom_configuration`)	https://docs.nvidia.com/nim/speech/latest/tts/customization.html
TTS voices and emotional styles	https://docs.nvidia.com/nim/speech/latest/tts/voices.html
TTS zero-shot voice cloning	https://docs.nvidia.com/nim/speech/latest/tts/voice-cloning.html
TTS IPA phone set	https://docs.nvidia.com/nim/speech/latest/tts/phoneme-support.html
Cloud function IDs (per model)	`https://build.nvidia.com/<org>/<model>/api`
NGC model catalog	https://catalog.ngc.nvidia.com/models

Examples

"Deploy a Parakeet ASR NIM" → load references/asr.md, follow Option B (self-hosted), Steps 1–4.

"Synthesize speech with Magpie" → load references/tts.md, follow Option A (cloud) or Option B (self-hosted).

"Translate English to German" → load references/nmt.md, follow the 4-step flow.

"Convert my fine-tuned .nemo to a NIM" → load references/asr-custom.md for the 4-phase pipeline and references/pipelines.md for build-time config.

"Deploy a custom fine-tuned TTS voice as a NIM" → load references/tts-custom.md for the 4-phase pipeline.

"Use zero-shot voice cloning with Magpie" → load references/tts-pipelines.md.

"Add SSML emphasis tags to my TTS request" → load references/tts-pipelines.md.

"'NVIDIA' sounds wrong in my Magpie TTS output — suggest a few IPA options to test" → load references/tts-pronunciation.md, generate IPA candidates, synthesize variants, then output all three delivery formats.

"How do I fix the pronunciation of 'NIM' in Riva TTS with a custom_dictionary in gRPC Python?" → load references/tts-pronunciation.md, propose IPA for 'NIM', show wire format and gRPC snippet.

"Can my GPU run this?" → load references/deployment-readiness-checks.md and run the 6-step system check.

"Which Riva model should I use?" → load references/model-selection.md, apply the decision framework, then fetch the support matrix for the specific current model name.

Naming & Terminology

Skill brand: Nemotron Speech (public-facing name).
Internal naming preserved: commands (riva-build, riva-deploy, riva_streaming_asr_client), Python client (riva.client), gRPC namespace (nvidia.riva.asr.*), container registry (nvcr.io/nim/nvidia/*), and all NVIDIA documentation URLs still use "Riva". Do not rename these in code, commands, or docs.

Troubleshooting

For task-specific runtime or modality issues, use the relevant reference file (references/<task>.md). Cross-cutting readiness checks:

Container does not become ready → references/deployment-readiness-checks.md (system check + health check table)
Health check fails → references/deployment-readiness-checks.md
docker pull from nvcr.io returns 403 → references/setup.md (Step 5 — Docker login)
Wrong base image / model architecture mismatch → references/asr-custom.md (Phase 2 base image)
VRAM / GPU compatibility → references/deployment-readiness-checks.md, then verify on the support matrix

Limitations

x86_64 architecture only — WSL2 on Windows requires Podman and supports a subset of NIMs (see references/setup.md)
Self-hosted deployment requires an NVIDIA AI Enterprise license
Cloud-hosted inference requires an active NVIDIA_API_KEY and internet access
Public skill branding is "Nemotron Speech"; commands, container images, Python imports (riva.client), gRPC services (nvidia.riva.*), and NVIDIA documentation URLs still use "Riva" — follow official docs and catalogs for naming, do not rename these in commands or code

Next Steps

Verify hardware compatibility: references/deployment-readiness-checks.md
Set up the environment: references/setup.md
Pick a model: references/model-selection.md
Deploy: references/asr.md, references/tts.md, or references/nmt.md

Files

ATTRIBUTION.mdAdded by AutoGineer917 B
BENCHMARK.md10.7 KiB
LICENSELicence file added by AutoGineer11.2 KiB
SKILL.md11.7 KiB
evals/EVAL.md1.2 KiB
evals/evals.json17.3 KiB
references/asr-custom.md14.6 KiB
references/asr.md26.4 KiB
references/deployment-readiness-checks.md7.8 KiB
references/model-selection.md19.2 KiB
references/nmt.md15.5 KiB
references/pipelines.md16.3 KiB
references/setup.md7.2 KiB
references/tts-custom.md14.1 KiB
references/tts-pipelines.md21.5 KiB
references/tts-pronunciation.md13.3 KiB
references/tts.md17.1 KiB
scripts/main.pyScript13.3 KiB
skill-card.md4.1 KiB
skill.oms.sig7.5 KiB

references/deployment-readiness-checks.md

Riva NIM Deployment Readiness Checks

Agent: When running the step-by-step system check, announce each step before presenting it: Step N/6 — Step Title (e.g., "Step 1/6 — Check Architecture").
Source of truth. This skill describes the system-check workflow and shell commands, which are stable. For per-release minimums — driver version, compute capability, glibc, supported GPUs, OS list, WSL2 constraints — fetch or open the canonical doc page and answer from that. See Looking up current information below.

Purpose

Verify host compatibility before deploying a Riva NIM. Covers hardware checks (architecture, GPU driver, VRAM, Container Toolkit), NGC access, and basic container health verification. This is a readiness reference, not an ASR/TTS/NMT troubleshooting guide; use the modality references for inference behavior, model options, and client-specific errors.

Looking up current information

Question type	Fetch this page
Minimum driver version, compute capability, glibc, supported OSes, WSL2 constraints	https://docs.nvidia.com/nim/speech/latest/get-started/prerequisites.html
VRAM minimums + supported GPUs per model	https://docs.nvidia.com/nim/speech/latest/reference/support-matrix/asr.html (and `/tts.html`, `/nmt.html`)
Latency / throughput per GPU	https://docs.nvidia.com/nim/speech/latest/reference/performances/asr/performance.html (and TTS / NMT)
Current valid container image to test the registry pull	https://docs.nvidia.com/nim/speech/latest/reference/support-matrix/asr.html (pick any model you have access to)

Do not infer driver minimums, compute-capability minimums, supported GPUs, or VRAM thresholds from this skill's text. The prerequisites page and support matrix are the contract.

Prerequisites

Linux x86_64 system
nvidia-smi accessible (NVIDIA driver installed)
Docker and NVIDIA Container Toolkit installed (or being verified as part of this check)

System Requirements

Always fetch the prerequisites page for current minimums (driver, compute capability, glibc, OS list). The checklist below is the categories to verify, not the version numbers — those rotate per release.

Check	How to Verify
CPU: x86_64	`uname -m`
NVIDIA driver installed	`nvidia-smi` → check "Driver Version" meets prerequisites page minimum
GPU compute capability	`nvidia-smi --query-gpu=compute_cap --format=csv` — compare to prerequisites page
VRAM sufficient	`nvidia-smi --query-gpu=memory.total --format=csv` — compare to support matrix per model
OS / glibc	`ld -v` — compare to prerequisites page
Docker installed	`docker info`
NVIDIA Container Toolkit	`docker run --rm --gpus all ubuntu nvidia-smi`
NGC credentials	`[ -n "$NGC_API_KEY" ] && echo set` (non-empty), logged into `nvcr.io`
NVAIE license	Required for self-hosting

Instructions

Run the 6-step system check below to verify your hardware and environment before deploying any Riva NIM. All 6 steps must pass. Use the Container Health Check after deployment to confirm the NIM is ready for inference.

Step-by-Step System Check

1. Check Architecture

uname -m
# Must output: x86_64

2. Check Driver Version

nvidia-smi
# Compare "Driver Version" against the minimum on the prerequisites page

3. Check GPU Compute Capability

nvidia-smi --query-gpu=name,compute_cap --format=csv,noheader
# Example output: NVIDIA A100-SXM4-80GB, 8.0
# Minimum required: see prerequisites page

4. Check Available VRAM

nvidia-smi --query-gpu=name,memory.total,memory.free --format=csv
# Ensure memory.free >= required for your model (per support matrix)

5. Verify Container Toolkit

docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi
# Should show the same GPU info as host nvidia-smi
# If this fails, reinstall NVIDIA Container Toolkit

6. Verify NGC Authentication

[ -n "$NGC_API_KEY" ] && echo "NGC_API_KEY is set" || echo "NGC_API_KEY is NOT set"
# Never echo or log your API key value — use this non-printing check instead

# Test the registry pull with any current NIM image
# (fetch a current model name from the support matrix; the example below uses a placeholder)
docker pull nvcr.io/nim/nvidia/<container-id-from-support-matrix>:latest 2>&1 | head -3
# Should start "latest: Pulling from..." not "Error response from daemon"

Container Health Check

After starting a NIM container, verify it is ready before sending inference requests:

# Wait for ready status (poll until "ready")
until curl -sf http://localhost:9000/v1/health/ready | grep -q '"ready"'; do
  echo "Waiting for NIM to be ready..."; sleep 10
done
echo "NIM is ready"

Single check:

curl -X GET http://localhost:9000/v1/health/ready
# Expected: {"status":"ready"}

Examples

Full quick system check:

uname -m                                                        # must be x86_64
nvidia-smi                                                      # verify driver
docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi   # verify Container Toolkit
[ -n "$NGC_API_KEY" ] && echo "NGC_API_KEY is set" || echo "NOT set"

Poll until NIM is ready:

until curl -sf http://localhost:9000/v1/health/ready | grep -q '"ready"'; do
  echo "Waiting..."; sleep 10
done && echo "NIM is ready"

Lookup flow — agent question "can my GPU run Whisper Large v3?":

Fetch or open the ASR support matrix
Locate the row for the requested model
Compare the listed VRAM and compute-capability minimum against the user's GPU (nvidia-smi --query-gpu=name,memory.total,compute_cap --format=csv)
Answer with the comparison

Do not answer hardware-compatibility questions from this skill's text alone.

Common Readiness Failures

Symptom	Likely Cause	Fix
Container exits immediately	Wrong `NIM_TAGS_SELECTOR` — no matching profile	Check support matrix for correct tag values
`docker pull` returns 403	Missing NGC credentials or no NVAIE license	Re-run `docker login nvcr.io`; verify `NGC_API_KEY`
Container stuck at "Downloading model" for >30 min	Large model (normal); slow network	Use model caching (`-v $LOCAL_NIM_CACHE:/opt/nim/.cache`)
`nvidia-smi` not found in container	Container Toolkit not configured	Reinstall/reconfigure NVIDIA Container Toolkit
Health check returns 503	Model still loading	Wait and retry; first load can take 10–30 min
OOM error / container killed	Insufficient VRAM	Use a profile with lower VRAM (per support matrix) or upgrade GPU
gRPC connection refused	Container not ready, or wrong port	Wait for health check; verify `-p 50051:50051` flag
HTTP 404 on inference endpoint	Wrong API path	Use `curl http://localhost:9000/v1/health/ready` to verify container is up

WSL2 (Windows) Notes

WSL2 has stricter requirements than Linux — fetch the prerequisites page for current minimums and the supported model subset, both rotate per release.

Stable WSL2 conventions:

Use Podman instead of Docker
Only a subset of NIMs is supported on WSL2 (verify on the prerequisites page)
Adjust WSL memory in .wslconfig if a container OOMs

If any check fails, report which requirement is unmet. Driver, glibc, and compute-capability minimums change with releases — always verify against the prerequisites page before deploying.

Limitations

System checks apply to x86_64 Linux only — WSL2 has additional constraints (fetch prerequisites page).
VRAM requirements are model-specific — always consult the support matrix for the NIM being deployed.
Health check polling assumes default port 9000; adjust if a custom port is configured.

Requirements and compatibility

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

Needs: Needs an API key, Needs Python

Security

What our scanner noticed· Medium risk

Large base64-encoded blobs
Instructions that use the network
Scripts that your agent may run

This skill includes scripts that your agent may run. Read them before you use it.

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Attribution and licence

Created by

Nemotron Speech Team

Publisher

NVIDIA (Official)

Distributed by

NVIDIA

Source

NVIDIA/skills · skills/nemotron-speech · commit fce3cca

Licence

Apache License 2.0 (redistribution allowed)

LICENSE (added by AutoGineer from LICENSE-APACHE)
ATTRIBUTION.md (written by AutoGineer)

Synced

2026-11-09

AutoGineer is not affiliated with or endorsed by Nemotron Speech Team or NVIDIA.

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download nemotron-speech.zip

From the author

AutoGineer summary

What it does

When to use it

Advantages

Usage examples

Install

Documentation by Nemotron Speech Team

Nemotron Speech Skills

Purpose

When to Use This Skill

Workflow

Prerequisites

Instructions

Source of truth

Examples

Naming & Terminology

Troubleshooting

Limitations

Next Steps

Files

references/deployment-readiness-checks.md

Riva NIM Deployment Readiness Checks

Purpose

Looking up current information

Prerequisites

System Requirements

Instructions

Step-by-Step System Check

1. Check Architecture

2. Check Driver Version

3. Check GPU Compute Capability

4. Check Available VRAM

5. Verify Container Toolkit

6. Verify NGC Authentication

Container Health Check

Examples

Common Readiness Failures

WSL2 (Windows) Notes

Limitations

Requirements and compatibility

Security

Attribution and licence

Related skills

Claude API

Transformers

Open Notebook

Vision Sft

LLM Evaluation

Lora Qlora Recipes

Download ZIP