Omniverse Realtime Viewer

OpenAIOfficialCreated by NVIDIA OmniverseDistributed by OpenAI

Skill
Meta (authoring, orchestration, planning, memory)
Software development
Apache-2.0
Medium risk

Download ZIP

From the author

Use as the top-level router for Omniverse Realtime Viewer USD app requests and focused viewer reference documents.

— the description by NVIDIA Omniverse

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

The Omniverse Realtime Viewer skill acts as a central orchestrator for developing USD-based viewer applications. It provides a structured approach to selecting implementation recipes, enforcing architectural conventions, and managing dependencies. By utilizing a set of focused reference documents, it guides developers through the lifecycle of building, configuring, and validating viewer applications, ensuring consistent behavior across different delivery paths such as local workstations or streaming environments.

What it does

Routes development requests to specific implementation recipes based on the target delivery path.
Enforces architectural rules for USD rendering, stage management, and UI integration.
Provides guidance on camera selection, input handling, and viewport interaction.
Standardizes the integration of AOV switching and render product configuration.
Facilitates the capture of validation evidence for viewer readiness.
Maintains separation between user-provided USD files and application-specific session state.

When to use it

When initiating the development of a new USD viewer application within the Omniverse ecosystem.
When determining the appropriate delivery path for a viewer, such as local native or streaming.
When needing to implement consistent camera, selection, or UI behavior across viewer projects.
When integrating advanced rendering features like AOV switching into an existing viewer.

Advantages

Ensures architectural consistency across diverse viewer implementations.
Provides a clear, modular path for adding specific capabilities like input routing or data display.
Promotes clean separation of concerns between stage data and viewer-owned state.
Standardizes the validation process for consistent delivery.

Usage examples

“I need to build a browser-based viewer for USD files.”: The agent will route you to the streaming viewer recipe, providing guidance on WebRTC stream setup, UI integration, and viewport interaction.
“How do I implement AOV switching in my local USD viewer?”: The agent will provide the AOV switching reference, detailing how to map render vars and update the display state on the server.

Install

Download the skill as a ZIP:
https://autogineer.com/agent-skills/openai/omniverse-realtime-viewer/download

Unzip it into the cross-client skills folder of your project:

mkdir -p .agents/skills && unzip omniverse-realtime-viewer.zip -d .agents/skills

Documentation by NVIDIA Omniverse

Omniverse Realtime Viewer

This is the top-level entry point for the Omniverse Realtime Viewer skill package. It is self-contained: all required routing, conventions, and validation guidance live in the selected references.

Use the focused reference documents as implementation recipes. This file chooses the right recipes and preserves the architectural rules that must hold across all generated viewer apps.

Instructions

Start by classifying the requested viewer, then read only the references needed for that delivery path and feature set. Implement the render path first, layer interaction and UI behavior on top of it, and finish by capturing validation evidence from references/validation.md.

Read Order

Read references/routing.md to choose the delivery path and focused references.
Read references/conventions.md before implementing camera, input, selection, viewport, streaming protocol, scene loading, or environment behavior.
For broad viewer requests, read references/usd-viewer-app/README.md.
If the delivery path is unclear, read references/streaming-vs-local/README.md.
If the prompt includes layout, panels, controls, inspectors, status, or UX, read references/viewer-ux-workflow/README.md and then the focused viewer UI references. This applies to React/WebRTC, Tauri, Electron, ovui, ovwidgets, and Dear ImGui apps; "frontend" means user-facing UI, not only browser UI.
For viewport interaction, read references/viewer-input-routing/README.md before references/camera-controls/README.md, references/native-picking-selection/README.md, or references/object-selection/README.md.
Read only the focused capability references needed for the requested app.
Use references/validation.md to capture review evidence before handoff.

Non-Negotiables

Use ovrtx for all USD and 3D rendering.
Browser apps display an ovstream WebRTC video stream plus UI. The browser does not render USD geometry.
Do not substitute WebGL, Three.js, Babylon.js, PlayCanvas, A-Frame, model-viewer, react-three-fiber, glTF browser viewers, or other client-side 3D renderers.
If local validation cannot run because the GPU/runtime environment is absent, scaffold the ovrtx path and document the runtime requirement. Do not add a browser-renderer fallback.
Keep user USD files unmodified. Viewer cameras, render products, render vars, settings, selection metadata, and runtime state belong in session/composite layers or app state.
Keep one owner for renderer.step(), stage mutation, native picking, selection writes, and live attribute writes.
Keep dependency acquisition in references/dependencies/README.md and deployment choices in references/cloud-deployment/README.md; do not duplicate package locations or deployment setup.

Focused Reference Families

Entry points and recipes: references/usd-viewer-app/README.md, references/streaming-viewer-recipe/README.md, references/ovui-local-viewer-recipe/README.md, references/streaming-vs-local/README.md, references/electron-shm-viewer/README.md, references/ovwidgets-editor-shell/README.md.
Rendering and stage: references/ovrtx-rendering/README.md, references/stage-loading/README.md, references/stage-management/README.md, references/render-settings/README.md, references/aov-switching/README.md, references/stage-hierarchy/README.md, references/stage-queries/README.md, references/stage-attribute-reads/README.md, references/prim-transform-safety/README.md, references/usd-sample-data/README.md.
Delivery and runtime: references/streaming-server/README.md, references/streaming-client/README.md, references/streaming-messages/README.md, references/streaming-lifecycle/README.md, references/local-viewer/README.md, references/tauri-local-viewer/README.md, references/cpp-native-viewer/README.md, references/headless-shm-cli/README.md, references/viewer-backend-interface/README.md, references/webgl-shm-transport/README.md.
Viewer UI/UX: references/viewer-ux-workflow/README.md, references/viewer-layout-patterns/README.md, references/viewer-control-patterns/README.md, references/viewer-data-view-patterns/README.md, references/viewer-feedback-status/README.md.
Interaction: references/viewer-input-routing/README.md, references/camera-controls/README.md, references/object-selection/README.md, references/native-picking-selection/README.md, references/selection-feedback/README.md, references/selection-animation/README.md, references/transform-manipulator/README.md, references/gl-viewport-overlay/README.md, references/ovui-library/README.md, references/prim-pick-effects/README.md, references/prim-info-display/README.md, references/viewport-overlays/README.md.
Infrastructure: references/dependencies/README.md, references/windows-native-setup/README.md, references/cloud-assets/README.md, references/cloud-deployment/README.md, references/troubleshooting/README.md.

Build Workflow

Classify the prompt by delivery path, target user, required capabilities, runtime environment, validation needs, and explicit constraints.
Select a small reference set. Start with the recipe or routing reference, then add focused capabilities such as camera, picking, hierarchy, properties, render settings, transform tools, cloud assets, or deployment.
Read selected references before writing app code. Follow their build order, import order, data-channel contracts, and renderer ownership rules.
Implement the core render path first, then input routing and camera, then selection and data panels, then scene/settings features, then packaging or deployment.
Treat the selected references as the behavior contract for API shape, compatibility, and generated project structure.
Capture validation evidence before calling the viewer ready.

Examples

For a browser viewer request, use the streaming recipe references plus camera, picking, hierarchy, properties, render settings, and stream-status references.
For a local workstation viewer request, use the local or native delivery references plus renderer setup, stage loading, viewport input, and validation.

Completion Checklist

Selected references match the user's intent and delivery path.
No code path uses a browser-side 3D renderer for USD.
The generated app has one clear owner for render stepping and stage mutation.
User USD files remain untouched by viewer-owned session data.
Camera, input, selection, scene loading, and stream behavior follow references/conventions.md.
Setup/build/run results and visual interaction evidence are captured with references/validation.md.

Files

ATTRIBUTION.mdAdded by AutoGineer1.0 KiB
BENCHMARK.md5.5 KiB
LICENSEAdded by AutoGineer10.0 KiB
SKILL.md7.4 KiB
agents/openai.yaml315 B
evals/evals.json8.1 KiB
references/aov-switching/README.md15.2 KiB
references/camera-auto-select/README.md7.4 KiB
references/camera-controls/README.md17.1 KiB
references/camera-picker/README.md9.7 KiB
references/cloud-assets/README.md10.4 KiB
references/cloud-deployment/README.md18.0 KiB
references/conventions.md5.3 KiB
references/cpp-native-viewer/README.md2.4 KiB
references/cpp-native-viewer/interaction-features.md19.2 KiB
references/cpp-native-viewer/presentation-loop.md9.0 KiB
references/cpp-native-viewer/project-build.md6.4 KiB
references/cpp-native-viewer/renderer-session.md4.5 KiB
references/cpp-native-viewer/validation.md3.3 KiB
references/dependencies/README.md3.4 KiB
references/dependencies/electron-shm.md1.5 KiB
references/dependencies/environment-validation.md4.4 KiB
references/dependencies/frontend.md2.3 KiB
references/dependencies/local-openusd-gpu.md4.4 KiB
references/dependencies/nvidia-runtime.md9.0 KiB
references/dependencies/ovrtx.md2.4 KiB
references/dependencies/ovstream.md2.5 KiB
references/dependencies/quick-setup.md5.3 KiB
references/electron-shm-viewer/README.md2.2 KiB
references/electron-shm-viewer/architecture-project.md6.3 KiB
references/electron-shm-viewer/electron-client.md5.6 KiB
references/electron-shm-viewer/protocol-interaction-lifecycle.md6.9 KiB
references/electron-shm-viewer/python-shm-server.md4.9 KiB
references/electron-shm-viewer/validation.md2.8 KiB
references/gl-viewport-overlay/README.md7.9 KiB
references/gl-viewport-overlay/architecture.md4.1 KiB
references/gl-viewport-overlay/example-gizmo.md5.5 KiB
references/gl-viewport-overlay/interaction-pattern.md4.0 KiB
references/gl-viewport-overlay/projection-alignment.md4.3 KiB
references/headless-shm-cli/README.md12.2 KiB
references/huggingface-usd/README.md7.7 KiB
references/local-viewer/README.md15.0 KiB
references/native-picking-selection/README.md10.4 KiB
references/object-selection/README.md9.4 KiB
references/ovrtx-rendering/README.md18.2 KiB
references/ovui-library/README.md3.3 KiB
references/ovui-library/api-reference.md4.3 KiB
references/ovui-library/extending.md4.8 KiB
references/ovui-local-viewer-recipe/README.md3.7 KiB
references/ovui-local-viewer-recipe/interaction-features.md16.1 KiB
references/ovui-local-viewer-recipe/project-structure.md6.2 KiB
references/ovui-local-viewer-recipe/setup-shell-renderer.md14.3 KiB
references/ovui-local-viewer-recipe/validation-build-order.md5.5 KiB
references/ovwidgets-editor-shell/README.md8.6 KiB
references/prim-info-display/README.md9.3 KiB
references/prim-pick-effects/README.md5.9 KiB
references/prim-transform-safety/README.md8.2 KiB
references/render-settings/README.md9.9 KiB
references/routing.md16.5 KiB
references/seg-outline-highlight/README.md1.4 KiB
references/selection-animation/README.md9.7 KiB
references/selection-feedback/README.md6.6 KiB
references/stage-attribute-reads/README.md5.5 KiB
references/stage-hierarchy/README.md17.1 KiB
references/stage-hierarchy/fallback-worker-protocol.md3.5 KiB
references/stage-loading/README.md19.3 KiB
references/stage-management/README.md10.1 KiB
references/stage-queries/README.md5.3 KiB
references/streaming-client/README.md17.4 KiB
references/streaming-lifecycle/README.md8.8 KiB
references/streaming-messages/README.md16.5 KiB
references/streaming-messages/server-handler-map.md3.1 KiB
references/streaming-server/README.md18.6 KiB
references/streaming-server/frame-loop-and-continuity.md3.3 KiB
references/streaming-viewer-recipe/README.md4.8 KiB
references/streaming-viewer-recipe/client-protocol.md11.0 KiB
references/streaming-viewer-recipe/interaction-features.md13.0 KiB
references/streaming-viewer-recipe/project-structure.md7.0 KiB
references/streaming-viewer-recipe/server-runtime.md16.8 KiB
references/streaming-viewer-recipe/validation-build-order.md5.8 KiB
references/streaming-vs-local/README.md8.1 KiB
references/tauri-local-viewer/README.md18.2 KiB
references/tauri-shm-transform-gizmo/README.md11.8 KiB
references/transform-manipulator/README.md15.0 KiB
references/troubleshooting/README.md7.2 KiB
references/troubleshooting/scenario-playbooks.md18.8 KiB
references/usd-sample-data/README.md11.7 KiB
references/usd-viewer-app/README.md7.5 KiB
references/validation.md3.0 KiB
references/viewer-backend-interface/README.md17.7 KiB
references/viewer-control-patterns/README.md4.8 KiB
references/viewer-data-view-patterns/README.md4.8 KiB
references/viewer-feedback-status/README.md4.9 KiB
references/viewer-input-routing/README.md7.8 KiB
references/viewer-layout-patterns/README.md4.7 KiB
references/viewer-ux-workflow/README.md4.6 KiB
references/viewport-overlays/README.md4.8 KiB
references/viewport-resize/README.md8.5 KiB
references/webgl-shm-transport/README.md14.5 KiB
references/windows-native-setup/README.md5.7 KiB
skill-card.md2.9 KiB
skill.oms.sig28.6 KiB

references/selection-feedback/README.md

Selection Feedback

Triggers

Use this skill for highlight selected, selection outline, selection group, SelectionGroupStyle, SelectionFillMode, selection glow, selected object feedback, or outline selected prim.

Use this for renderer-visible feedback after selection. Picking and selected-path state live in object-selection. The combined ovrtx 0.3 API reference is native-picking-selection.

The default selection visual path is native ovrtx selection outlines and styles. Do not create segmentation/Warp outline systems for new ovrtx 0.3 apps.

For ovrtx selection outline, fill, or C API behavior beyond this reference, read references/dependencies for acquisition guidance and supplemental dependency documentation.

Native Selection Model

Enable the native selection outline pass when creating the renderer.
Assign non-zero selection group IDs to selected prims.
Write group 0 to clear a prim.
Configure outline width and fill mode at renderer creation.
Configure per-group outline/fill colors at runtime.
Use transparent fill color when the app wants outlines only.

Python names:

ovrtx.RendererConfig(selection_outline_enabled=True)
ovrtx.RendererConfig(selection_outline_width=...)
ovrtx.RendererConfig(selection_fill_mode=ovrtx.SelectionFillMode.GROUP_FILL_COLOR)
ovrtx.SelectionGroupStyle
Renderer.set_selection_group_styles(...)
ovrtx.OVRTX_ATTR_NAME_SELECTION_OUTLINE_GROUP

C/C++ names:

OVRTX_CONFIG_SELECTION_OUTLINE_ENABLED
ovrtx_config_entry_selection_outline_enabled(true)
ovrtx_config_entry_selection_outline_width(...)
ovrtx_config_entry_selection_fill_mode(...)
ovrtx_set_selection_group_styles(...)
ovrtx_set_selection_outline_group(...)

Renderer Setup

import ovrtx


config = ovrtx.RendererConfig(
    selection_outline_enabled=True,
    selection_outline_width=4,
    selection_fill_mode=ovrtx.SelectionFillMode.GROUP_FILL_COLOR,
)
renderer = ovrtx.Renderer(config=config)

Changing selection_outline_enabled, selection_outline_width, or selection_fill_mode requires recreating the renderer. Per-group colors can be changed while the renderer is running.

Group Styles

renderer.set_selection_group_styles({
    1: ovrtx.SelectionGroupStyle(
        outline_color=(1.0, 0.6, 0.0, 1.0),
        fill_color=(0.0, 0.0, 0.0, 0.0),
    ),
    2: ovrtx.SelectionGroupStyle(
        outline_color=(0.1, 0.55, 1.0, 1.0),
        fill_color=(0.1, 0.55, 1.0, 0.18),
    ),
})

Use stable group IDs for distinct interaction states, for example:

Group	Use
`0`	Unselected / cleared
`1`	Primary selection
`2`	Secondary or marquee preview
`3`	Hover, if the app needs hover feedback

Later style writes to the same group replace earlier styles for subsequent frames.

Assign And Clear Outlines

Python:

import numpy as np
import ovrtx


def set_selection_groups(renderer, group_by_path: dict[str, int]) -> None:
    if not group_by_path:
        return

    paths = list(group_by_path)
    groups = np.asarray([group_by_path[path] for path in paths], dtype=np.uint8)
    renderer.write_attribute(
        prim_paths=paths,
        attribute_name=ovrtx.OVRTX_ATTR_NAME_SELECTION_OUTLINE_GROUP,
        tensor=groups,
    )

C/C++:

// Set selected prims to group 1.
ovrtx_set_selection_outline_group(renderer, selected_paths, selected_count, 1);

// Clear previously selected prims.
ovrtx_set_selection_outline_group(renderer, previous_paths, previous_count, 0);

Always clear previous groups that are no longer selected before assigning the new selection. The renderer displays whatever group value is currently authored for each prim.

Update Pattern

class SelectionFeedback:
    PRIMARY_GROUP = 1

    def __init__(self, renderer):
        self._renderer = renderer
        self._outlined_paths: set[str] = set()

    def update(self, selected_mesh_paths: set[str]) -> None:
        selected_mesh_paths = set(selected_mesh_paths)

        clear_paths = self._outlined_paths - selected_mesh_paths
        set_paths = selected_mesh_paths

        writes = {path: 0 for path in clear_paths}
        writes.update({path: self.PRIMARY_GROUP for path in set_paths})
        set_selection_groups(self._renderer, writes)

        self._outlined_paths = selected_mesh_paths

    def clear(self) -> None:
        set_selection_groups(self._renderer, {path: 0 for path in self._outlined_paths})
        self._outlined_paths.clear()

Tree selection often targets an Xform or Scope. Use stage-hierarchy to expand that selected item to descendant mesh paths for outline assignment, while keeping the original selected path for the tree and info panel.

Fill Mode

Selection fill color is visible only when the renderer was created with a fill mode that uses group fill colors, such as SelectionFillMode.GROUP_FILL_COLOR / OVRTX_SELECTION_FILL_MODE_GROUP_FILL_COLOR.

For outline-only selection, either use a fill mode that disables filling or keep each group's fill_color alpha at 0.0.

Scene Lifecycle

On scene switch, reload, or renderer reset:

Stop issuing feedback writes while the renderer is resetting.
Clear the runtime selected path set.
Clear previous native selection groups by writing group 0 before discarding the old selection state when practical.
Recreate renderer-level outline configuration if the renderer is recreated.
Reapply default group styles after creating a new renderer.

Gotchas

Selection visuals require both renderer config and per-prim non-zero group IDs.
Group 0 means no native selection outline.
Per-group style writes are runtime state; keep them in app setup, not in the per-frame hot path.
Fill color does nothing unless the renderer fill mode enables it.
Outline dashing/stippling is not supported by the native RTX outline pass.
Do not use seg-outline-highlight unless the user explicitly needs a custom post-process overlay instead of native selection outlines.

See also: viewer-input-routing, native-picking-selection, object-selection, stage-hierarchy, stage-management, stage-loading, ovrtx-rendering.

Generated Module Checklist - selection_feedback.py

[ ] Renderer is created with native selection outlines enabled.
[ ] Default SelectionGroupStyle values are installed at startup.
[ ] Selected mesh paths are written to non-zero selection groups.
[ ] Previously selected mesh paths are cleared with group 0.
[ ] Tree-selected Xforms/Scopes are expanded to descendant mesh paths.
[ ] Renderer recreation reapplies outline config and group styles.
[ ] No EffectLayer material or fader behavior is implemented in this module.

Requirements and compatibility

Written for: OpenAI Codex

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

The author says: Orchestrator skill. Downstream focused references may require NVIDIA GPUs, ovrtx, ovstream, ovui, OpenUSD, Python, Node/React, Tauri, Electron, C++, or cloud GPU deployment access depending on the selected viewer path.

Security

What our scanner noticed· Medium risk

Large base64-encoded blobs
Instructions that use the network

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Attribution and licence

Created by

NVIDIA Omniverse

Publisher

OpenAI (Official)

Distributed by

OpenAI

Source

openai/plugins · plugins/nvidia/skills/omniverse-realtime-viewer · commit d420d13

Licence

Apache License 2.0 (redistribution allowed)

LICENSE (written by AutoGineer)
ATTRIBUTION.md (written by AutoGineer)

Synced

2026-11-09

AutoGineer is not affiliated with or endorsed by NVIDIA Omniverse or OpenAI.

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download omniverse-realtime-viewer.zip

From the author

AutoGineer summary

What it does

When to use it

Advantages

Usage examples

Install

Documentation by NVIDIA Omniverse

Omniverse Realtime Viewer

Instructions

Read Order

Non-Negotiables

Focused Reference Families

Build Workflow

Examples

Completion Checklist

Files

references/selection-feedback/README.md

Selection Feedback

Triggers

Native Selection Model

Renderer Setup

Group Styles

Assign And Clear Outlines

Update Pattern

Fill Mode

Scene Lifecycle

Gotchas

Generated Module Checklist - selection_feedback.py

Requirements and compatibility

Security

Attribution and licence

Related skills

Setup TS Deep Modules

To Spec

Improve Codebase Architecture

Domain Modeling

Codebase Design

Prototype

Download ZIP