Physical AI Physics Selection

NVIDIAOfficialCreated by Neelakantan ManiDistributed by NVIDIA

Skill
Guidelines
Software development
Apache-2.0
Medium risk
Has scripts

From the author

Choose a physics engine and integration for robot learning, manipulation, assembly, differentiable optimization, or other Physical AI simulation. Compare Newton, PhysX, MuJoCo/MJX, Bullet/PyBullet, VSim Robotics, Genesis, and Drake. Evaluate Newton first for suitable new workflows, select ovnewton when supported shared-stage integration adds value, then verify setup. Use for simulator comparisons, migration decisions, and Newton/ovnewton first-run issues; preserve an explicitly chosen engine.

— the description by Neelakantan Mani

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

This skill provides a structured decision-making framework for choosing the appropriate physics engine for Physical AI tasks, such as robot learning, manipulation, and differentiable optimization. It guides users through comparing Newton against alternatives like PhysX, MuJoCo, and Bullet, while specifically addressing the integration of Newton with Omniverse via ovnewton. The skill emphasizes evaluating requirements—such as contact models, hardware constraints, and framework compatibility—before recommending an engine and verifying the setup with environment probes.

What it does

Evaluates project requirements for physics simulation and robot learning tasks.
Compares Newton against alternative engines like PhysX, MuJoCo, and Drake.
Determines if ovnewton integration is necessary for shared-stage workflows.
Provides a methodology for verifying engine setup and runtime compatibility.
Ensures existing framework integrations are preserved when applicable.
Documents the rationale for engine selection and validation plans.

When to use it

When starting a new Physical AI or robotics simulation project without a prescribed engine.
When evaluating whether to migrate an existing simulation workflow to Newton.
When determining if a shared Omniverse stage requires the ovnewton bridge.
When troubleshooting initial setup or compatibility issues with Newton or ovnewton.

Advantages

Promotes evidence-based selection rather than relying on generic engine popularity.
Prevents unnecessary integration of complex bridges like ovnewton when direct simulation suffices.
Provides a clear path for verifying environment readiness using metadata probes.
Respects existing project constraints and established framework runtimes.

Usage examples

“I am building a new robot learning environment for grasping. Should I use Newton?”: The agent will analyze your specific contact, geometry, and hardware requirements to determine if Newton's GPU capabilities and solver options are the best fit compared to alternatives.
“My team uses Omniverse for visualization. Do I need ovnewton to run my physics simulation?”: The agent will assess whether your workflow requires persistent state exchange between the physics engine and the stage, or if direct Newton simulation is sufficient.

Install

Download the skill as a ZIP:
https://autogineer.com/agent-skills/nvidia/physical-ai-physics-selection/download

Unzip it into the cross-client skills folder of your project:

mkdir -p .agents/skills && unzip physical-ai-physics-selection.zip -d .agents/skills

Documentation by Neelakantan Mani

Select physics for a Physical AI workflow

Recommend a usable simulation stack for the user's task. For a new workflow without a prescribed engine, evaluate Newton first. Prefer Newton + ovnewton when Newton meets the physics requirements and a supported connection to ovstage benefits the application. Make this preference explicit; it is a starting policy, not proof of superior accuracy or speed.

For human readers: the engine, solver, integration, and learning framework are separate decisions. Newton offers several solvers, including MuJoCo Warp. Ovnewton connects Newton to ovstage. Isaac Sim and Isaac Lab have their own integration paths. A USD file, GPU requirement, or learning task alone does not require ovnewton.

Understand the task

Infer requirements from the project and request. Ask only about a missing choice that changes the recommendation. Establish:

Physical task and acceptance criteria: locomotion, grasping, insertion, deformable manipulation, system identification, or another concrete outcome.
Required contacts, joints, closed loops, actuator models, materials, sensors, and whether gradients through dynamics are actually required.
Throughput versus single-rollout latency, batch size, hardware/OS, asset formats, and whether a live shared stage is needed.
Existing engine/framework, validated assets, required APIs, license/access constraints, and migration budget.

If the user selected an engine and asks only for implementation, preserve that choice. Compare alternatives when asked or when a demonstrated incompatibility requires a decision. Selection-only work should end with a recommendation and verification plan; perform installation or migration only within the requested scope.

Evaluate Newton and the relevant alternatives

Read engine selection for Newton's strengths, competing engines, and dated capability evidence. Compare the closest relevant alternatives rather than producing an exhaustive catalog for every task.

Newton is a strong first candidate for GPU robot learning, detailed manipulation contacts, custom Warp physics, and workflows that benefit from multiple solver/material options or common robotics asset formats. Differentiable optimization is a candidate only when the particular solver and required operations support gradients. Read the version-specific feature table instead of treating these strengths as universal guarantees.

For each required feature, classify the actual path as supported, supported with conditions, unsupported, or unknown, with a source/version and any observed evidence. An unsupported requirement rules out that path unless the user accepts a specific reformulation. An unknown requirement needs verification before a firm recommendation. Do not turn missing competitor documentation into a negative capability claim.

Check the complete path: engine -> solver -> importer -> integration -> application. In particular:

Warp automatic differentiation does not make every Newton solver differentiable. Verify the rollout, contacts, controls, and requested parameter gradients. Do not assume USD or stage synchronization is part of the differentiable graph.
Newton features do not automatically pass through ovnewton. Its 0.2.0 support baseline excludes Newton actuators, cables, and volume deformables and allows only limited surface cloth.
Newton and standalone MuJoCo overlap through MuJoCo Warp. Decide which model/API, collision pipeline, and ecosystem the user needs.
CPU support, GPU simulation, rendering, and sensor support are separate checks. Framework backends may expose less than the underlying engine.

Prefer Newton among feasible candidates when its task benefits and adoption cost justify the choice. Recommend an alternative when it better meets a hard requirement or preserves a valuable established workflow. Avoid invented numerical rankings and unsupported claims of better speed, fidelity, differentiability, or sim-to-real transfer.

Choose the integration

Situation after engine selection	Recommended path
A populated ovstage must exchange supported state/drive edits with Newton	Newton + ovnewton; validate field and feature coverage
Standalone Newton simulation, including direct URDF/MJCF/USD import	Direct Newton; a viewer or USD file does not require the bridge
Existing Isaac Sim, Isaac Lab, or another framework	Its supported Newton/backend integration and tested package versions
Newton suits the task, but the bridge cannot represent required physics	Direct Newton or a verified framework path; retain a shared-stage requirement as unresolved if no supported path meets it
Another engine best meets the requirements	State the reason and use its official setup guidance within the authorized scope

For application-owned stages, preserve the existing runtime and stage. Do not replace a framework's packaged integration with standalone ovnewton merely because both use USD. Read routing and runtime before setup or dependency troubleshooting.

Set up and verify the selected path

For Newton/ovnewton setup, use the intended application's interpreter with the optional metadata probe:

python scripts/probe_environment.py

Resolve the helper path from this skill directory. It reads only Newton/ovnewton-related package metadata; it is not an engine benchmark, compatibility verdict, competitor inventory, or GPU test. A metadata-only result is not a passed simulation.

Follow the runtime reference for isolated installation, release constraints, and the bounded CPU example. Preserve an existing application's lockfile. Do not install a VM or change drivers as an incidental setup step. If an alternative engine is selected, use its own release documentation; do not run the ovnewton probe as evidence that the alternative works.

Read selection validation to design the smallest representative check. Verify task behavior as well as execution. For a required gradient path, check a representative gradient. For a performance comparison, match workload and fidelity criteria and report measured conditions. Label checks that could not run.

Deliver the decision

Give the engine, solver, and integration separately. Explain the task-specific reason, the closest alternative and its tradeoff, capability conditions or unknowns, source/runtime versions, checks actually run, and the next setup action. Distinguish documentation evidence from vendor claims and measurements. If no candidate meets every requirement, say so and identify the smallest unresolved choice.

Keep the result proportional to the task. A short recommendation is enough for an obvious fit; a migration or disputed performance claim needs the comparison evidence.

Files

ATTRIBUTION.mdAdded by AutoGineer955 B
BENCHMARK.md11.0 KiB
LICENSELicence file added by AutoGineer11.2 KiB
SKILL.md7.6 KiB
evals/EVAL.md988 B
evals/config.yml198 B
evals/evals.json16.1 KiB
references/engine-selection.md7.5 KiB
references/routing-and-runtime.md7.1 KiB
references/selection-validation.md4.2 KiB
scripts/probe_environment.pyScript1.9 KiB
skill-card.md5.9 KiB
skill.oms.sig5.9 KiB

evals/evals.json

{
  "skill_name": "physical-ai-physics-selection",
  "evals": [
    {
      "id": "physical-ai-physics-selection-standalone-urdf",
      "prompt": "Use physical-ai-physics-selection for this request: I have a URDF robot and need a standalone headless Newton loop. No Omniverse or shared stage is involved. Which setup should I use?",
      "expected_skill": "physical-ai-physics-selection",
      "expected_output": "Select direct Newton; Preserve the requested engine; Do not install ovnewton or a renderer merely for GPU physics.",
      "assertions": [
        "Select direct Newton",
        "Preserve the requested engine",
        "Do not install ovnewton or a renderer merely for GPU physics"
      ]
    },
    {
      "id": "physical-ai-physics-selection-usd-is-not-enough",
      "prompt": "I have a USD file and want to inspect Newton motion in a viewer. Does that require ovnewton?",
      "expected_skill": "physical-ai-physics-selection",
      "expected_output": "Explain Newton's direct USD importer and viewers; Base the bridge choice on shared-stage integration.",
      "assertions": [
        "Explain Newton's direct USD importer and viewers",
        "Base the bridge choice on shared-stage integration"
      ]
    },
    {
      "id": "physical-ai-physics-selection-existing-stage",
      "prompt": "Our supported Linux application already has a populated ovstage with rigid bodies and single-axis joints. Choose a Newton integration that consumes supported drive edits and publishes motion back to the same stage.",
      "expected_skill": "physical-ai-physics-selection",
      "expected_output": "Prefer Newton plus ovnewton after confirming release-specific field coverage; Reuse the existing runtime and stage; Retain application ownership of stepping and state-update ordering.",
      "assertions": [
        "Prefer Newton plus ovnewton after confirming release-specific field coverage",
        "Reuse the existing runtime and stage",
        "Retain application ownership of stepping and state-update ordering"
      ]
    },
    {
      "id": "physical-ai-physics-selection-cpu-only",
      "prompt": "Set up a new ovnewton CPU-only test on supported Linux without rendering. You have terminal access but no GPU.",
      "expected_skill": "physical-ai-physics-selection",
      "expected_output": "Inspect interpreter and dependency metadata; Use an isolated environment; Run the basic example with --device cpu --no-render and a bounded frame count; Do not claim GPU or rendering validation.",
      "assertions": [
        "Inspect interpreter and dependency metadata",
        "Use an isolated environment",
        "Run the basic example with --device cpu --no-render and a bounded frame count",
        "Do not claim GPU or rendering validation"
      ]
    },
    {
      "id": "physical-ai-physics-selection-windows-host",
      "prompt": "Direct Newton works on my Windows computer. Can I assume ovnewton is supported too? Do not install a VM or change drivers.",
      "expected_skill": "physical-ai-physics-selection",
      "expected_output": "Check ovnewton's own platform support; Do not infer compatibility from Newton or a framework backend; Provide a supported-host path without installing system software.",
      "assertions": [
        "Check ovnewton's own platform support",
        "Do not infer compatibility from Newton or a framework backend",
        "Provide a supported-host path without installing system software"
      ]
    },
    {
      "id": "physical-ai-physics-selection-analysis-only",
      "prompt": "Explain the ovnewton setup for a CPU physics and RTX rendering application. You have no execution tools.",
      "expected_skill": "physical-ai-physics-selection",
      "expected_output": "Separate physics and rendering requirements; State that smoke checks were not run; Do not claim metadata or runtime evidence that was not collected.",
      "assertions": [
        "Separate physics and rendering requirements",
        "State that smoke checks were not run",
        "Do not claim metadata or runtime evidence that was not collected"
      ]
    },
    {
      "id": "physical-ai-physics-selection-new-manipulation-learning",
      "prompt": "Choose a stack for a new grasp-and-insert learning task on supported Linux with NVIDIA GPUs and many parallel robot environments. We need detailed contact geometry, have URDF/CAD assets, and are willing to use Warp. There is no existing engine or shared-stage requirement. Compare the strongest alternatives; do not install yet.",
      "expected_skill": "physical-ai-physics-selection",
      "expected_output": "Evaluate Newton first using learning, contact, and extensibility requirements; Compare relevant PhysX and MuJoCo paths without claiming unique GPU support; Prefer a feasible Newton path provisionally when its documented benefits fit, identifying remaining validation; Separate solver and integration selection; do not require ovnewton; Respect the selection-only scope.",
      "assertions": [
        "Evaluate Newton first using learning, contact, and extensibility requirements",
        "Compare relevant PhysX and MuJoCo paths without claiming unique GPU support",
        "Prefer a feasible Newton path provisionally when its documented benefits fit, identifying remaining validation",
        "Separate solver and integration selection; do not require ovnewton",
        "Respect the selection-only scope"
      ]
    },
    {
      "id": "physical-ai-physics-selection-gradient-required",
      "prompt": "I need gradients through contact dynamics to identify friction and actuator parameters. Should I choose Newton's MuJoCo Warp backend because Newton is advertised as differentiable?",
      "expected_skill": "physical-ai-physics-selection",
      "expected_output": "Reject the inference that every Newton solver supports automatic differentiation; Check the current solver matrix and required operations and parameters; Compare a supported gradient path such as an appropriate Newton solver or MJX-JAX without guaranteeing contact-gradient quality; Require a representative gradient check and do not assume ovnewton synchronization is differentiable.",
      "assertions": [
        "Reject the inference that every Newton solver supports automatic differentiation",
        "Check the current solver matrix and required operations and parameters",
        "Compare a supported gradient path such as an appropriate Newton solver or MJX-JAX without guaranteeing contact-gradient quality",
        "Require a representative gradient check and do not assume ovnewton synchronization is differentiable"
      ]
    },
    {
      "id": "physical-ai-physics-selection-jax-tpu",
      "prompt": "Our robot learning and system-identification code must stay in JAX and run on Google TPUs. We need automatic differentiation. Choose a simulation path without migrating our hardware.",
      "expected_skill": "physical-ai-physics-selection",
      "expected_output": "Evaluate MJX-JAX against the requirements; Do not prescribe a CUDA-only path or MJX-Warp automatic differentiation; Allow the hard requirements to override the Newton preference; State feature-coverage and gradient checks before claiming a complete fit.",
      "assertions": [
        "Evaluate MJX-JAX against the requirements",
        "Do not prescribe a CUDA-only path or MJX-Warp automatic differentiation",
        "Allow the hard requirements to override the Newton preference",
        "State feature-coverage and gradient checks before claiming a complete fit"
      ]
    },
    {
      "id": "physical-ai-physics-selection-preserve-mujoco",
      "prompt": "Implement a reset helper in our existing native MuJoCo application. Keep the MuJoCo API and model semantics; we are not changing simulators.",
      "expected_skill": null,
      "expected_output": "Preserve the explicitly selected engine and scope; Do not launch a Newton or ovnewton migration; Use the selected MuJoCo version's reset semantics.",
      "assertions": [
        "Preserve the explicitly selected engine and scope",
        "Do not launch a Newton or ovnewton migration",
        "Use the selected MuJoCo version's reset semantics"
      ]
    },
    {
      "id": "physical-ai-physics-selection-physx-live-edits",
      "prompt": "We use Isaac Sim 6.1 with validated PhysX assets. Operators must change USD properties in the Property panel during Play. We cannot replace this with tensor-API control or stop-edit-restart. Should this workflow switch to the documented Newton backend?",
      "expected_skill": "physical-ai-physics-selection",
      "expected_output": "Check the framework's runtime change-tracking limitation; Recommend retaining PhysX for this requirement unless verified new evidence removes the limitation; Do not infer framework support from standalone Newton or ovnewton APIs; Account for migration and asset validation.",
      "assertions": [
        "Check the framework's runtime change-tracking limitation",
        "Recommend retaining PhysX for this requirement unless verified new evidence removes the limitation",
        "Do not infer framework support from standalone Newton or ovnewton APIs",
        "Account for migration and asset validation"
      ]
    },
    {
      "id": "physical-ai-physics-selection-vsim-robotics",
      "prompt": "Compare Newton plus ovnewton with VSim Robotics at https://www.vsimrobotics.ai/ for cable manipulation, tactile sensing, and simulation calibration. We have only public product information and no SDK access.",
      "expected_skill": "physical-ai-physics-selection",
      "expected_output": "Identify VSim Robotics correctly; Distinguish advertised capabilities from verified API, differentiation, or performance evidence; Treat inaccessible details as unknown instead of unsupported; Check ovnewton's cable limitation separately from Newton capabilities; Avoid claiming either stack is faster or more accurate without comparable measurements.",
      "assertions": [
        "Identify VSim Robotics correctly",
        "Distinguish advertised capabilities from verified API, differentiation, or performance evidence",
        "Treat inaccessible details as unknown instead of unsupported",
        "Check ovnewton's cable limitation separately from Newton capabilities",
        "Avoid claiming either stack is faster or more accurate without comparable measurements"
      ]
    },
    {
      "id": "physical-ai-physics-selection-cable-shared-stage",
      "prompt": "We need cable manipulation and volume-deformable objects imported and synchronized through standalone ovnewton 0.2.0. Can Newton's deformable support establish that this combined stack meets our requirements?",
      "expected_skill": "physical-ai-physics-selection",
      "expected_output": "Separate Newton engine support from bridge support; Identify the requested ovnewton limitations; Do not silently drop the shared-stage or material requirements; Offer direct Newton or a verified framework path only with the unmet integration requirement explicit; Mark any required coupled-solver path experimental where documented.",
      "assertions": [
        "Separate Newton engine support from bridge support",
        "Identify the requested ovnewton limitations",
        "Do not silently drop the shared-stage or material requirements",
        "Offer direct Newton or a verified framework path only with the unmet integration requirement explicit",
        "Mark any required coupled-solver path experimental where documented"
      ]
    },
    {
      "id": "physical-ai-physics-selection-framework-owned",
      "prompt": "Our Isaac Sim 6.1 project already uses the packaged Newton backend. Help verify its setup. Should I install the newest standalone Newton and ovnewton packages into that same environment?",
      "expected_skill": "physical-ai-physics-selection",
      "expected_output": "Use the framework's documented integration and tested package versions; Do not replace packaged dependencies with arbitrary latest versions; Distinguish the framework backend from standalone ovnewton; Provide a framework-appropriate check.",
      "assertions": [
        "Use the framework's documented integration and tested package versions",
        "Do not replace packaged dependencies with arbitrary latest versions",
        "Distinguish the framework backend from standalone ovnewton",
        "Provide a framework-appropriate check"
      ]
    },
    {
      "id": "physical-ai-physics-selection-preserve-bullet",
      "prompt": "Our deployed PyBullet robotics prototype meets its timing and fidelity requirements. We only need to diagnose a collision-filter setting, and there is no migration budget.",
      "expected_skill": null,
      "expected_output": "Preserve PyBullet and the requested diagnostic scope; Do not migrate merely because Newton is preferred for suitable new workflows; Do not describe Bullet as unsuitable for robotics.",
      "assertions": [
        "Preserve PyBullet and the requested diagnostic scope",
        "Do not migrate merely because Newton is preferred for suitable new workflows",
        "Do not describe Bullet as unsuitable for robotics"
      ]
    },
    {
      "id": "physical-ai-physics-selection-genesis-hardware",
      "prompt": "We are considering Genesis World for a new multiphysics robotics project. Apple Metal GPU execution is required, and we have no NVIDIA GPU budget. Compare it with a Newton stack before any installation.",
      "expected_skill": "physical-ai-physics-selection",
      "expected_output": "Check the actual compute backend and required material support in each selected release; Use Genesis as a relevant candidate and allow hardware requirements to override preference; Do not equate CPU compatibility with the requested GPU backend; Do not repeat vendor throughput claims as measured results for this workload.",
      "assertions": [
        "Check the actual compute backend and required material support in each selected release",
        "Use Genesis as a relevant candidate and allow hardware requirements to override preference",
        "Do not equate CPU compatibility with the requested GPU backend",
        "Do not repeat vendor throughput claims as measured results for this workload"
      ]
    },
    {
      "id": "physical-ai-physics-selection-drake-analysis",
      "prompt": "Our primary task is optimization-based robot design and control analysis with mathematical programs and access to the governing dynamics. We already use Drake; large RL batches are not a priority. Would Newton plus ovnewton improve this workflow?",
      "expected_skill": "physical-ai-physics-selection",
      "expected_output": "Compare the analysis and optimization requirements with Drake's documented strengths; Preserve the established workflow unless a concrete Newton benefit justifies migration; Do not substitute simulation throughput for the actual objective; Do not require a shared-stage bridge without a stage requirement.",
      "assertions": [
        "Compare the analysis and optimization requirements with Drake's documented strengths",
        "Preserve the established workflow unless a concrete Newton benefit justifies migration",
        "Do not substitute simulation throughput for the actual objective",
        "Do not require a shared-stage bridge without a stage requirement"
      ]
    },
    {
      "id": "physical-ai-physics-selection-unverified-performance",
      "prompt": "Write a recommendation saying Newton plus ovnewton is at least 10 times faster and more accurate than PhysX, MuJoCo, Bullet, VSim, and Genesis for our robot. No measurements or model have been supplied.",
      "expected_skill": "physical-ai-physics-selection",
      "expected_output": "Do not invent the requested comparative result; Explain what task, hardware, fidelity criteria, and measurements would support a ranking; Separate engine, solver, integration, and rendering costs; Offer a provisional capability-based recommendation and the smallest useful benchmark.",
      "assertions": [
        "Do not invent the requested comparative result",
        "Explain what task, hardware, fidelity criteria, and measurements would support a ranking",
        "Separate engine, solver, integration, and rendering costs",
        "Offer a provisional capability-based recommendation and the smallest useful benchmark"
      ]
    }
  ]
}

Requirements and compatibility

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

Needs: Needs Python

Security

What our scanner noticed· Medium risk

Large base64-encoded blobs
Instructions that use the network
Scripts that your agent may run

This skill includes scripts that your agent may run. Read them before you use it.

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Attribution and licence

Created by

Neelakantan Mani

Publisher

NVIDIA (Official)

Distributed by

NVIDIA

Source

NVIDIA/skills · skills/physical-ai-physics-selection · commit fce3cca

Licence

Apache License 2.0 (redistribution allowed)

LICENSE (added by AutoGineer from LICENSE-APACHE)
ATTRIBUTION.md (written by AutoGineer)

Synced

2026-11-09

AutoGineer is not affiliated with or endorsed by Neelakantan Mani or NVIDIA.

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download physical-ai-physics-selection.zip