Tao Setup Nvidia Gpu Host
NVIDIAOfficialCreated by NVIDIA Corporation
- Skill
- Workflow
- Cloud and DevOps
- Apache-2.0
- Medium risk
- Has scripts
AutoGineer summary
Written by AutoGineer from the skill’s own files, not by the author
This skill provides a robust mechanism to check, install, and verify the essential NVIDIA GPU software stack required for running TAO workflows on Docker or Kubernetes backends. It ensures that the host meets minimum version requirements for the NVIDIA driver, CUDA Toolkit, and NVIDIA Container Toolkit. The tool supports automated installation on major Linux distribution families (Debian, RHEL, and SUSE) and provides clear, actionable guidance for manual setup on unsupported distributions. It is designed to be used as a preflight check before executing GPU-intensive tasks.
What it does
- Probes the host for NVIDIA driver, CUDA, and Container Toolkit versions.
- Automates the installation of required GPU packages on Debian, RHEL, and SUSE-based systems.
- Configures the NVIDIA Docker runtime for seamless GPU container integration.
- Validates host readiness against TAO-wide or model-specific runtime requirements.
- Provides manual installation instructions for unsupported Linux distributions.
- Manages Docker engine installation and user group permissions for non-root access.
When to use it
- Before running TAO workflows on local Docker or Kubernetes backends.
- When troubleshooting GPU runtime issues in a TAO environment.
- To prepare a new GPU worker node for a Kubernetes cluster.
- To verify that a host meets the minimum version requirements for a specific model.
Advantages
- Ensures consistent GPU environments across different Linux distributions.
- Reduces manual configuration errors by automating repository and package setup.
- Provides a safe, read-only check mode to preview system status without modification.
- Supports model-specific runtime overrides for specialized requirements.
- Offers clear, actionable feedback for manual intervention when automation is not possible.
Usage examples
- “Check if my local Docker host is ready for TAO.”
- The agent runs the script in --check-only mode, reporting the status of the NVIDIA driver, CUDA, and Container Toolkit, and confirms if the environment meets TAO minimums.
- “Set up my Ubuntu host for TAO GPU training.”
- The agent executes the installation workflow, adding necessary NVIDIA repositories, installing required packages, and configuring the Docker runtime after user approval.
- “Verify my Kubernetes GPU worker node for TAO.”
- The agent checks the worker node's GPU runtime and reports whether it is correctly configured to support TAO workloads.
Install
Download the skill as a ZIP:
https://autogineer.com/agent-skills/nvidia/tao-setup-nvidia-gpu-host/downloadUnzip it into the cross-client skills folder of your project:
mkdir -p .agents/skills && unzip tao-setup-nvidia-gpu-host.zip -d .agents/skills
Documentation by NVIDIA Corporation
NVIDIA GPU Host Setup
Standalone install? If this session was not initialized by the TAO skill bank plugin, run the
tao-setupskill first (host preflight, credentials, cross-skill discovery).
Use this setup skill before TAO workflows run on the docker, local-docker,
or kubernetes backend. The TAO-wide default minimums are:
NVIDIA driver
>=580(open kernel module preferred)CUDA Toolkit
>=13.0NVIDIA Container Toolkit
>=1.19.0Docker engine — only installed for
docker/local-dockerbackends and only when Docker is missing. The package picked depends on the distro family (docker.ioon Debian-family by default,moby-engine/docker-cefromdownload.docker.comon RHEL-family,dockeron SUSE-family). Pass--skip-docker-installto opt out.
The check is safe and read-only by default — it works on any Linux
distribution because it only probes nvidia-smi, the CUDA toolkit path,
the installed container-toolkit package version (via dpkg/rpm/the
nvidia-ctk binary version), and the Docker daemon's NVIDIA runtime.
Installation must be explicitly authorized by the user and rerun with
--install. The install path is automated for these distro families:
| Family | Tested distros | Manager | Notes |
|---|---|---|---|
| debian | Ubuntu 22.04 / 24.04, Debian 12 (and derivatives Pop!_OS, Mint, Zorin, Raspbian, KDE Neon, etc. via UBUNTU_CODENAME / VERSION_CODENAME) | apt-get | Adds NVIDIA cuda-keyring + Container Toolkit .list. Docker via docker.io (override $DOCKER_PACKAGE_DEBIAN). |
| rhel | Fedora 39+, RHEL / Rocky / AlmaLinux 9 and 10 | dnf (or yum) | Adds NVIDIA cuda-<distro>.repo + Container Toolkit .repo. Docker via Fedora moby-engine when available, otherwise docker-ce from download.docker.com. |
| suse | openSUSE Leap 15, SLES 15 | zypper | Adds the same NVIDIA .repo files. Docker via the distribution docker package. |
| other (Arch, Alpine, Gentoo, NixOS, FreeBSD, …) | n/a | n/a | --install exits with a clear error listing the version targets and the NVIDIA install-guide URLs. Install manually, then rerun --check-only. |
Quick Start
From the skill bank root:
# Check the local Docker backend host.
bash skills/platform/tao-setup-nvidia-gpu-host/scripts/setup-nvidia-gpu-host.sh --backend docker --check-only
# Install or repair after user approval.
bash skills/platform/tao-setup-nvidia-gpu-host/scripts/setup-nvidia-gpu-host.sh --backend docker --install
# Check a Kubernetes GPU worker host.
bash skills/platform/tao-setup-nvidia-gpu-host/scripts/setup-nvidia-gpu-host.sh --backend kubernetes --check-only⚠️ Note — running non-interactively (agent/skill runs): a skill run has no terminal, so the installer's
Continue? [y/N]prompt cannot be answered. After running--check-onlyto preview and getting the user's approval, append the assume-yes flag (--yes) to the--installcommand so it proceeds without a prompt — this auto-confirms installation of system packages (NVIDIA driver, CUDA Toolkit, NVIDIA Container Toolkit, and Docker for Docker backends) and modifies the host, so only do this on a host you control. A person running--installdirectly at a terminal gets the prompt instead.
Workflow Contract
Docker and Kubernetes workflows must run the check before submitting GPU work:
SB="${TAO_SKILL_BANK_PATH:-${TAO_SKILL_BANK_ROOT:-$PWD}}"
SETUP_SCRIPT="${SB}/skills/platform/tao-setup-nvidia-gpu-host/scripts/setup-nvidia-gpu-host.sh"
bash "$SETUP_SCRIPT" --backend docker --check-only || {
echo "MISSING: TAO GPU host runtime is not ready."
echo "After user approval, run (append --yes for non-interactive agent runs):"
echo " bash \"$SETUP_SCRIPT\" --backend docker --install"
exit 1
}Never install silently. If the check fails, explain what is missing, ask the user to authorize the fix, then run the install command and rerun the check.
Model Runtime Override Contract
Platform defaults apply when a model has no override. A model that needs a
different validated host stack declares it in
references/skill_info.yaml:
runtime_requirements:
gpu_host:
min_driver_version: '<version>'
min_cuda_version: '<version>'
min_container_toolkit_version: '<version>'Read those values before the final platform preflight and pass them to the
matching --min-*-version flags. Model minimums take precedence for that
workflow only; do not rewrite the platform defaults or requirements for other
models. Version checks use numeric lower bounds, so later compatible releases
pass. Always retain the selected-image GPU smoke test because a version bound
cannot prove support for a particular GPU architecture.
What The Installer Does
The installer dispatches on the detected distribution family. On every
supported family it adds NVIDIA's CUDA and Container Toolkit repositories
(if missing), installs packages that satisfy the active minimums, optionally
installs Docker, wires the NVIDIA Docker runtime, and adds the invoking user to
the docker group.
Common steps (all families):
Adds NVIDIA's CUDA repository if missing (apt
cuda-keyringdeb,cuda-<distro>.repofor dnf/zypper).Adds NVIDIA's Container Toolkit repository if missing (
.listfor apt,.repofor dnf/zypper).Installs the matching kernel header / devel package for the running kernel.
Installs the current open-driver and Container Toolkit packages from the configured repositories plus the CUDA Toolkit package selected by
--min-cuda-version, then verifies all three against the active minimums.For Docker backends and when Docker is missing, installs Docker (override / opt-out flags below), enables/starts the daemon, then runs
nvidia-ctk runtime configure --runtime=dockerand restarts Docker whensystemctlis available.Adds the invoking user (
$SUDO_USERif available, else$USER) to thedockergroup so subsequent shells can rundockerwithoutsudo— opt out with--skip-docker-group. The new group membership does not take effect in the current shell: log out and back in, or runnewgrp dockerin each new shell.Attempts
modprobe nvidiaso verification can pass before reboot.
Family-specific package selections:
| Step | debian-family | rhel-family | suse-family |
|---|---|---|---|
| Kernel headers | linux-headers-$(uname -r) | kernel-devel-$(uname -r), kernel-headers-$(uname -r) | kernel-default-devel |
| Driver | current nvidia-open (override: $NVIDIA_DRIVER_PACKAGE_DEBIAN) | current nvidia-driver-cuda, kmod-nvidia-open-dkms (override: $NVIDIA_DRIVER_PACKAGE_RHEL, $NVIDIA_DRIVER_KMOD_RHEL) | current nvidia-open-driver-G06-signed-kmp-default (override: $NVIDIA_DRIVER_PACKAGE_SUSE) |
| CUDA toolkit | package derived from the active minimum, such as cuda-toolkit-13-0 | same | same |
| Container Toolkit | current nvidia-container-toolkit + base/tools/libs, then minimum-version validation | same | same |
| Docker | docker.io (override: $DOCKER_PACKAGE_DEBIAN) | moby-engine+moby-cli on Fedora when available, else docker-ce docker-ce-cli containerd.io from download.docker.com | docker |
Verification
After installation, verify:
nvidia-smi
nvcc --version
docker info --format '{{json .Runtimes}}' | grep nvidia
sudo docker run --rm --runtime=nvidia --gpus all "$TAO_IMAGE" nvidia-smi -LThe detected driver, CUDA Toolkit, and Container Toolkit versions must meet the active TAO-wide or model-specific minimums. Then run the selected image's GPU smoke test; version comparison alone is not sufficient compatibility proof.
For a Cosmos backend, extend that smoke with the backend contract's Python and
entrypoint checks. Cosmos Framework must execute
/workspace/.venv/bin/python as a non-root UID, import
cosmos_framework.callbacks.tao_status, find native torchrun, and verify the
A100 PatchEmbed compatibility marker when the host reports compute capability
8.0. Cosmos-RL must resolve its requested action executable, import
system PyAV for video workflows, resolve the restricted FFmpeg h264_cuvid
decoder, load libnvcuvid.so.1, verify the backward-safe linear Qwen3-VL
PatchEmbed marker, and verify its checkpoint loader accepts the prepared
qwen3_vl directory. A container that only passes nvidia-smi is not ready
for Cosmos training.
Kubernetes Notes
For self-managed Kubernetes clusters, run the host installer on every GPU worker node or bake the same package set into the node image before installing the NVIDIA GPU Operator or device plugin.
The workflow check also warns if kubectl is available but the cluster reports
no nvidia.com/gpu allocatable capacity. In that case, install/configure the
NVIDIA GPU Operator after the worker host runtime is ready:
helm repo add nvidia https://helm.ngc.nvidia.com/nvidia
helm repo update
helm install --wait gpu-operator -n gpu-operator --create-namespace nvidia/gpu-operatorManaged Kubernetes providers may own driver installation through node images or GPU Operator policy. Do not overwrite a provider-managed GPU node without user approval and a rollback plan.
Failure Modes
Unsupported distribution family: --install automates debian-, rhel-,
and suse-family hosts. On Arch, Alpine, Gentoo, NixOS, FreeBSD, or anything
without /etc/os-release (e.g. macOS), the script exits with a clear error
that lists the four version targets and the upstream NVIDIA install-guide
URLs:
https://docs.nvidia.com/cuda/cuda-installation-guide-linux/https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.htmlhttps://docs.docker.com/engine/install/
Install those four pieces using your distribution's package manager and
rerun the script with --check-only to verify. The check is universally
portable — it only queries the binaries / package databases — so once the
runtime is in place the workflow contract is satisfied regardless of the
underlying distro.
Unsupported Ubuntu/Debian derivative: When ID is e.g. pop, mint,
zorin, raspbian, or another debian-family derivative, the script maps
the host onto the upstream Ubuntu/Debian CUDA repo via UBUNTU_CODENAME /
VERSION_CODENAME (focal/jammy/noble → Ubuntu 20.04/22.04/24.04;
bullseye/bookworm/trixie → Debian 11/12/12). If the host's codename
doesn't match a known upstream release, --install exits with the same
manual-install guidance described above.
Docker not installed: --check-only reports MISSING: Docker is not installed and prints the exact rerun command appropriate to the detected
distro family. The default --install path installs Docker (docker.io /
moby-engine / docker-ce / docker depending on family), enables/starts
the daemon, configures the NVIDIA runtime, and adds the invoking user to
the docker group. If you prefer to manage Docker yourself, install it
before rerunning the script or pass --skip-docker-install.
Docker installed but docker run still needs sudo: The script adds the
invoking user to the docker group, but Linux only refreshes group
membership on a new login session. Log out and back in, or run
newgrp docker in each new shell, until the new membership is active.
Docker runtime still missing: Restart Docker, then rerun
nvidia-ctk runtime configure --runtime=docker.
Detected version is below the active minimum: Rerun the same command with
--install after approval, preserving any model-specific --min-*-version
flags. Package-name environment overrides select distribution-specific driver
packages but do not weaken the minimum-version checks.
Driver installed but nvidia-smi fails: Load the module with
sudo modprobe nvidia or reboot. Secure Boot may require MOK enrollment on
systems where it is enabled.
Kubernetes still has no GPU capacity: Confirm the driver works on each GPU
node with nvidia-smi, then check the GPU Operator/device plugin pods and node
labels.
Files
- ATTRIBUTION.mdAdded by AutoGineer945 B
- BENCHMARK.md6.9 KiB
- LICENSELicence file added by AutoGineer11.2 KiB
- SKILL.md13.2 KiB
- config/skillspector-baseline.yaml1.5 KiB
- evals/evals.json804 B
- references/skill_info.yaml2.6 KiB
- scripts/setup-nvidia-gpu-host.shScript27.4 KiB
- skill-card.md4.3 KiB
- skill.oms.sig5.2 KiB
LICENSE
Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Requirements and compatibility
Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta
“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.
Needs: Declares allowed tools, Needs a command-line tool, Needs Python, Needs Node.js
The author says: Runs `--check-only` on any Linux distribution. `--install` automates Ubuntu 22.04/24.04 + Debian 12 (apt), Fedora + RHEL/Rocky/AlmaLinux 9/10 (dnf), and openSUSE Leap / SLES 15 (zypper). Requires sudo/root, internet access to NVIDIA package repositories (and download.docker.com on rhel-family), and an x86_64 or aarch64 (sbsa) host. Other distributions (Arch, Alpine, Gentoo, NixOS, …) get a clear error that names the version targets and the NVIDIA install-guide URL.
Security
What our scanner noticed· Medium risk
- Large base64-encoded blobs
- Scripts that your agent may run
This skill includes scripts that your agent may run. Read them before you use it.
A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.
Attribution and licence
- Created by
- NVIDIA Corporation
- Publisher
- NVIDIA (Official)
- Source
- NVIDIA/skills · skills/tao-setup-nvidia-gpu-host · commit fce3cca
- Licence
- Apache License 2.0 (redistribution allowed)
- LICENSE (added by AutoGineer from LICENSE-APACHE)
- ATTRIBUTION.md (written by AutoGineer)
- Synced
AutoGineer is not affiliated with or endorsed by NVIDIA Corporation or NVIDIA.
Download ZIP
The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download tao-setup-nvidia-gpu-host.zip