AutoGineer
Generated banner for I4h Workflow Dataset Teleop
Generated logo for I4h Workflow Dataset Teleop

I4h Workflow Dataset Teleop

NVIDIAOfficialCreated by Isaac for Healthcare TeamDistributed by NVIDIA

  • Skill
  • Workflow
  • Automation and integrations
  • Apache-2.0
  • Medium risk
Download ZIP

From the author

Record demonstrations through a workflow's teleop Task into workflow HDF5. Use for keyboard, leader, VR, or bus input; do not use for policy evaluation or autonomous rule-based Tasks.
— the description by Isaac for Healthcare Team

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

This skill provides a structured procedure for recording human-controlled teleoperation demonstrations within the Isaac for Healthcare (i4h) framework. It facilitates the capture of robot actions, state, camera feeds, and task outcomes into standardized HDF5 files. The process requires an active human operator using supported input devices such as keyboards, VR controllers, or bus inputs to perform tasks in a simulated environment.

What it does

  • Resolves workflow repositories and teleoperation device configurations.
  • Executes simulation environments for interactive human-led task performance.
  • Records demonstration data including actions, state, and camera streams into HDF5 format.
  • Provides validation steps to inspect recorded segments and action motion.
  • Supports custom run directories for integration into larger data pipelines.

When to use it

  • When you need to collect human-demonstration data for training or analysis.
  • When working with i4h-compatible workflows that support teleoperation modes.
  • When you have a human operator available to perform tasks via supported input devices.
  • When you need to ensure recorded data adheres to the standard HDF5 contract.

Advantages

  • Ensures consistent data structure across different teleoperation sessions.
  • Integrates directly with existing simulation runners and inspection tools.
  • Supports multiple input modalities including VR and keyboard.
  • Provides clear verification steps to confirm data integrity and task success.

Usage examples

“Record 5 keyboard teleop demonstrations for locomanip tray pick and place.”
The agent initiates the specified workflow in teleop mode using the keyboard device, prompts the operator to complete 5 successful episodes, and saves the data to an HDF5 file for verification.

Install

  1. Download the skill as a ZIP:

    https://autogineer.com/agent-skills/nvidia/i4h-workflow-dataset-teleop/download

  2. Unzip it into the cross-client skills folder of your project:

    mkdir -p .agents/skills && unzip i4h-workflow-dataset-teleop.zip -d .agents/skills

Documentation by Isaac for Healthcare Team

Record Teleop Demonstrations

Purpose

Run the workflow's declared teleop graph through the shared SimulationRunner so actions, state, cameras, segments, attempts, and outcomes use the normal HDF5 contract.

Instructions

  1. Resolve the base checkout and live teleop device contract.

  2. Choose a supported human-input device.

  3. Record in the foreground through run.sh.

  4. Inspect visible motion and HDF5 content.

Resolve support

export I4H_WORKFLOWS_REPO_URL="${I4H_WORKFLOWS_REPO_URL:-https://github.com/isaac-for-healthcare/i4h-workflows}"
I4H_REPO_DIR_NAME="${I4H_WORKFLOWS_REPO_URL%/}"
I4H_REPO_DIR_NAME="${I4H_REPO_DIR_NAME##*/}"
I4H_REPO_DIR_NAME="${I4H_REPO_DIR_NAME##*:}"
I4H_REPO_DIR_NAME="${I4H_REPO_DIR_NAME%.git}"
[ -n "$I4H_REPO_DIR_NAME" ] || { echo "Cannot derive a checkout name from I4H_WORKFLOWS_REPO_URL" >&2; exit 2; }
ROOT="${I4H_WORKFLOWS:-$(git rev-parse --show-toplevel 2>/dev/null)}"
if [ ! -d "$ROOT/workflows/i4h_workflows" ]; then
  ROOT="${I4H_WORKFLOWS:-$HOME/$I4H_REPO_DIR_NAME}"
  [ -d "$ROOT/workflows/i4h_workflows" ] || git clone "$I4H_WORKFLOWS_REPO_URL" "$ROOT"
fi
export I4H_WORKFLOWS="$ROOT"
cd "$ROOT"
./run.sh list
./run.sh show <workflow> --mode teleop

Treat the resolver above as part of the skill contract: a hosted copy may run outside the base repository, so never assume the current checkout contains workflows/i4h_workflows. I4H_WORKFLOWS_REPO_URL selects the clone source. When I4H_WORKFLOWS is unset, derive the fallback directory from that URL; set I4H_WORKFLOWS only to reuse or choose a specific destination. Never replace an existing checkout.

Require teleop in the live mode list. Read the workflow builder, its scene manifest teleop override, and the embodiment manifest's teleop_devices. Do not maintain a static support table in the skill.

Choose input

  • Use the named device when supported.

  • Otherwise use the workflow builder's default device.

  • Keep interactive keyboard/leader/VR/bus sessions visible and in the foreground. Surface the device controls and require the human operator to complete the task.

  • Never pretend to provide human input in an unattended shell.

Record

./run.sh <workflow> --teleop <device> \
  --episodes <N> --attempts 3 \
  --record

Omit <device> to use the workflow default. Bare --record writes demos.hdf5 inside the launcher's automatic run directory. Read the absolute directory from the ==> run dir ... line or run.json; do not recreate its timestamp in the shell. When a larger pipeline requires a caller-selected shared directory, pass --run-dir "$RUN_DIR" --record demos.hdf5; the launcher creates the directory and anchors the relative recording name inside it. An absolute --record path remains supported.

Verify

Require the final N/N episodes succeeded summary. Then inspect content:

RUN_DIR="<absolute run_dir from run.json or launcher output>"
uv run --project tools/dataset i4h-dataset inspect "$RUN_DIR/demos.hdf5" --segments
uv run --project tools/dataset i4h-dataset actions "$RUN_DIR/demos.hdf5"

Visually confirm that the operator completes the requested task, robot motion matches the input device, and all expected cameras record the same behavior. Treat zero saved episodes, missing observations, absent action motion, or an unsuccessful task outcome as failure. Stop leftovers with ./stop.sh all.

Troubleshooting

On device or width errors, compare the workflow teleop builder, Scene mode override, and embodiment devices.

Prerequisites

Require a workflow with teleop, a supported device, a working simulator, and a present operator for interactive input.

Limitations

Teleop records human input and requires an operator for interactive devices. Record autonomous rule-based Tasks with i4h-workflow-validate instead.

Examples

  • Record 5 keyboard teleop demonstrations for locomanip tray pick and place. → use G1's supported keyboard device, require a human operator, and verify the recorded action motion.

Completion gate

Report workflow, mode/device, controls, requested/saved episodes, attempts, visual result, HDF5 path, dimensions/segments, and whether a human operator completed the task.

Files

skill-card.md

Description: <br>

Record demonstrations through a workflow's teleop Task into workflow HDF5. <br>

This skill is ready for commercial/non-commercial use. <br>

Owner

NVIDIA <br>

License/Terms of Use: <br>

Apache-2.0 <br>

Use Case: <br>

Developers and engineers who need to record human-operated teleop demonstrations through Isaac for Healthcare workflows into HDF5 datasets for downstream training. <br>

Deployment Geography for Use: <br>

Global <br>

Requirements / Dependencies: <br>

Requires API Key or External Credential: [Not Specified] <br> Credential Type(s): [None identified] <br>

Do not include secrets in prompts/logs/output; use least-privilege credentials; rotate keys as appropriate. <br>

Known Risks and Mitigations: <br>

Risk: Review before execution as proposals could introduce incorrect or misleading guidance into skills. <br> Mitigation: Review and scan skill before deployment. <br>

Reference(s): <br>

Skill Output: <br>

Output Type(s): [Shell commands, Analysis, Files] <br> Output Format: [Markdown with inline bash code blocks] <br> Output Parameters: [1D] <br> Other Properties Related to Output: [None] <br>

Evaluation Agents Used: <br>

  • Claude Code (aws/anthropic/bedrock-claude-opus-4-8) <br>

  • Codex (openai/openai/gpt-5.5) <br>

Evaluation Tasks: <br>

3 evaluation tasks (3 positive), 3 attempts per task, each in an isolated k8s-sandbox pod. <br>

Evaluation Metrics Used: <br>

Reported benchmark dimensions: <br>

  • Security: Checks for unsafe operations, secret leakage, and unauthorized access. <br>

  • Correctness: Checks whether the final answer is correct against the reference answer. <br>

  • Discoverability: Checks whether the expected skill was selected and the workflow executed. <br>

  • Effectiveness: Checks whether the user's goal was achieved and the expected workflow behavior was followed. <br>

  • Efficiency: Checks for tool-call productivity and token efficiency. <br>

Underlying evaluation signals used in this run: <br>

  • security: Unsafe operations, secret leakage, and unauthorized access. <br>

  • accuracy: Final-answer correctness against the reference answer. <br>

  • skill_execution: Whether the expected skill was selected, decoys were avoided, and the workflow executed. <br>

  • goal_accuracy: Whether the user's goal was achieved. <br>

  • behavior_check: Whether the expected workflow behavior was followed. <br>

  • skill_efficiency: Tool-call productivity (legacy wire id; routing is scored under Discoverability). <br>

  • token_efficiency: Actual uncached prompt plus completion usage. <br>

Evaluation Results: <br>

MeasureClaude Code (Baseline → Skill Uplift)Codex (Baseline → Skill Uplift)
Overall66.0%71.8%
Security88.9% → 66.7% (-22.2 pp)71.4% → 83.3% (+11.9 pp)
Correctness4.4% → 66.7% (+62.3 pp)31.4% → 93.3% (+61.9 pp)
Discoverability91.7%75.0%
Effectiveness3.3% → 21.7% (+18.4 pp)10.9% → 36.3% (+25.4 pp)
Efficiency83.3%71.2%

Skill Version(s): <br>

0.8.0 (source: frontmatter) <br>

Ethical Considerations: <br>

NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal team to ensure this skill meets requirements for the relevant industry and use case and addresses unforeseen product misuse. <br>

(For Release on NVIDIA Platforms Only) <br> Please report quality, risk, security vulnerabilities or NVIDIA AI Concerns here. <br>

Requirements and compatibility

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

Needs: Needs Python

Security

What our scanner noticed· Medium risk

  • Large base64-encoded blobs

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Attribution and licence

Created by
Isaac for Healthcare Team
Publisher
NVIDIA (Official)
Distributed by
NVIDIA
Source
NVIDIA/skills · skills/i4h-workflow-dataset-teleop · commit fce3cca
Licence
Apache License 2.0 (redistribution allowed)
  • LICENSE (added by AutoGineer from LICENSE-APACHE)
  • ATTRIBUTION.md (written by AutoGineer)
Synced

AutoGineer is not affiliated with or endorsed by Isaac for Healthcare Team or NVIDIA.

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download i4h-workflow-dataset-teleop.zip