Medtech Model Evidence Export

NVIDIAOfficialCreated by NVIDIA MedTech

Skill
Tool integration
Data and AI
Apache-2.0
Medium risk
Has scripts

From the author

Exports sanitized metadata, parameters, reproducibility details, quality metrics, and optional review artifacts from Medical AI inference runs or evidence packs to MLflow. Use after inference, including NV-Generate runs; not for live training tracking, model registration, or clinical use.

— the description by NVIDIA MedTech

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

The Medtech Model Evidence Export skill provides a structured workflow for mirroring medical AI inference results or evidence packs into MLflow. It is designed for post-hoc documentation, allowing users to log metadata, parameters, reproducibility details, and quality metrics from completed inference runs. The tool emphasizes data privacy by requiring sanitization of identifiers and provides configurable artifact policies to control the upload of sensitive medical images.

What it does

Exports inference metadata, parameters, and quality metrics to MLflow.
Supports dry-run mode for inspecting data before final export.
Calculates and logs intensity statistics and volume metrics for medical imaging.
Provides configurable artifact policies to manage the upload of previews or raw files.
Generates human-readable run summaries in MLflow notes.
Integrates with local MLflow stores or remote Databricks workspaces.

When to use it

After completing a medical AI inference run or generating an evidence pack.
When you need to track reproducibility and quality metrics for inference experiments.
When you need to centralize inference results in an MLflow-compatible environment.
When you need to perform engineering checks on inference outputs without clinical validation.

Advantages

Maintains the original evidence pack as the source of truth.
Includes built-in safety checks for sensitive medical data.
Provides flexible artifact management to prevent accidental exposure of raw images.
Automates the extraction of relevant quality metrics from NIfTI files.

Usage examples

“Preview the export of my latest inference run without uploading to MLflow.”: The agent runs the export script in dry-run mode, displaying the parameters and metrics that would be logged without modifying any remote state.
“Export my inference results to the local MLflow store.”: The agent executes the export script in local mode, creating an MLflow run with the provided configuration and reproducibility metadata.
“Upload my inference pack to Databricks including slice previews.”: The agent performs the export to the specified Databricks workspace, applying the preview artifact policy and confirming the medical image upload.

Install

Download the skill as a ZIP:
https://autogineer.com/agent-skills/nvidia/medtech-model-evidence-export/download

Unzip it into the cross-client skills folder of your project:

mkdir -p .agents/skills && unzip medtech-model-evidence-export.zip -d .agents/skills

Documentation by NVIDIA MedTech

Medtech Model Evidence Export to MLflow

Purpose

Mirror an existing medical-inference result or evidence pack into MLflow after the run and emit the export_result JSON contract. Keep the original evidence pack as the source of truth. Training skills should add MLflow inside their training loops instead.

Instructions

Run scripts/export_evidence_pack.py in the default dry-run mode.
Inspect params, metrics, artifact_plan, and mlflow.note.content.
Choose --mode local or --mode databricks only after checking the target.
Keep --artifact-policy metadata unless the target is approved for images.
For preview or all in a live mode, also pass --confirm-medical-artifact-upload.
Keep --source-ref, --note, config filenames, and artifact filenames free of patient or secret identifiers; always review the dry-run output first.

Hosts with a script helper can use run_script("scripts/export_evidence_pack.py", args=["PACK_OR_RESULT", "--mode", "dry-run"]).

Available Scripts

Script	Purpose	Arguments
`scripts/export_evidence_pack.py`	Export post-hoc inference evidence through MLflow.	`PACK_OR_RESULT --mode dry-run --artifact-policy metadata`

Prerequisites

Python 3.10+.
mlflow>=2.10,<4 for local or databricks mode.
numpy>=1.24,<3 and nibabel>=4,<6 for NIfTI quality metrics and previews.
MLFLOW_TRACKING_URI may select a caller-managed tracking server.
Databricks mode uses the caller's DATABRICKS_HOST, DATABRICKS_TOKEN, or configured Databricks profile. The declared network endpoint is https://<caller-provided-mlflow-or-databricks-workspace>; Docker and GPU are not required.
Local mode may write the MLflow store under <current-working-directory>/mlruns.

Examples

Preview the export without contacting MLflow:

python skills/medtech-model-evidence-export/scripts/export_evidence_pack.py \
  runs/inference_pack --mode dry-run --artifact-policy metadata

Export a direct NV-Generate result with reproducibility metadata:

python skills/medtech-model-evidence-export/scripts/export_evidence_pack.py \
  runs/nv-generate/result.json \
  --mode local \
  --experiment-name medical-ai-inference \
  --config configs/chest_lung_tumor.json \
  --seed 0 \
  --source-ref git:61c4ec709b84cad468852243c48e250bec732074

Log downsampled slice previews, but not raw NIfTI files:

python skills/medtech-model-evidence-export/scripts/export_evidence_pack.py \
  runs/nv-generate/result.json \
  --mode databricks \
  --experiment-name /Shared/medical-ai-inference \
  --artifact-policy preview \
  --confirm-medical-artifact-upload

--artifact-policy all additionally uploads discovered or explicitly supplied NIfTI images and masks, subject to --max-artifact-mb. Use --image and --mask when paths are not present in the result JSON.

The exporter logs:

scalar run and quality metrics, including sampled HU mean/std/min/max for CT (generic intensity statistics otherwise), a documented intensity-SNR heuristic, mask foreground percentage, and mapped tumor volume percentage when a tumor label mapping is available;
generation parameters, model/checkpoint identity, RNG seed, and recipe hash;
source config digest or --source-ref, plus a prompt digest when present;
mlflow.note.content with a short human-readable run summary;
a sanitized metadata bundle by default, optional PNG slice previews, and raw image/mask artifacts only under the explicit all policy.

Limitations

This is post-hoc inference export, not live training-curve tracking.
Global intensity SNR and downsampled volume statistics are engineering checks, not image-quality or clinical-performance claims.
Preview and raw artifacts may contain sensitive medical information. The caller must approve the destination and data policy before upload.
The exporter does not evaluate model quality, register models, or alter the source evidence pack.

Troubleshooting

Error	Cause	Fix
Evidence source not recognized	No direct result JSON or pack `manifest.json`.	Pass the result file, evidence-pack directory, or trusted-run root.
MLflow import fails	Live mode lacks the declared package.	Install `mlflow>=2.10,<4` or use `--mode dry-run`.
Preview/all confirmation error	A live image upload was not acknowledged.	Review the destination, then pass `--confirm-medical-artifact-upload`.
Referenced image not found	Result paths moved after inference.	Pass current paths with `--image` and `--mask`.

Files

ATTRIBUTION.mdAdded by AutoGineer953 B
BENCHMARK.md7.7 KiB
LICENSELicence file added by AutoGineer11.2 KiB
SKILL.md5.0 KiB
evals/evals.json2.4 KiB
fixtures/sample_pack/integrity_check.json62 B
fixtures/sample_pack/manifest.json229 B
fixtures/sample_pack/output.json1.5 KiB
fixtures/sample_pack/runtime_profile.json108 B
fixtures/sample_pack/validation_summary.json121 B
fixtures/sample_result.json1.8 KiB
scripts/export_evidence_pack.pyScript32.0 KiB
skill-card.md4.1 KiB
skill.oms.sig6.9 KiB
skill_manifest.yaml3.7 KiB
tests/test_export_evidence_pack.pyScript8.6 KiB
validators/output_schema.json3.2 KiB

BENCHMARK.md

Skill Benchmark: medtech-model-evidence-export

✅ Overall verdict: PASS — Recommended for publication

Publication Recommendation

Recommended for publication based on the completed evaluation evidence in this report.

Evaluation Metadata

Skill: medtech-model-evidence-export
Evaluation date: 2026-09-14
Evaluator version: 1.5.6
Agents: Claude Code (aws/anthropic/bedrock-claude-opus-4-8), Codex (openai/openai/gpt-5.5)
Tasks: 4 evaluation tasks (2 positive, 2 negative)
Dataset digest: sha256:20dc012111b0aa3dc1ff37d9ba71d5f505c9fccdf540783bcb0acf6764f3c4b7 (skill-evaluator-dataset-snapshot/1)
Attempts per task: 3
Environment: k8s-sandbox
Tier 2 evidence: required for publication
Tier 3 evidence: required for publication

Each task attempt ran in its own isolated sandbox pod.

What This Report Answers

The three-tier evaluation checks whether the skill:

is safe to use;
produces correct answers;
is discovered and activated when needed;
helps the agent complete the user's goal and expected workflow; and
avoids wasted skill and tool usage.

Results at a Glance

Measure	Claude Code (Baseline → Skill Uplift)	Codex (Baseline → Skill Uplift)
Overall	92.0% — baseline ran, but no comparable score was available; uplift unavailable	90.6% — baseline ran, but no comparable score was available; uplift unavailable
Security	100.0% → 100.0% (±0.0 points)	66.7% → 100.0% (+33.3 points)
Correctness	40.0% → 100.0% (+60.0 points)	37.8% → 100.0% (+62.2 points)
Discoverability	95.0% — baseline ran, but no comparable score was available; uplift unavailable	80.0% — baseline ran, but no comparable score was available; uplift unavailable
Effectiveness	31.3% → 78.1% (+46.8 points)	38.9% → 78.1% (+39.2 points)
Efficiency	86.8% — baseline ran, but no comparable score was available; uplift unavailable	95.0% — baseline ran, but no comparable score was available; uplift unavailable

How to read this table: baseline is the same task attempted without the target skill. Scores are rounded to one decimal; threshold-adjacent values use additional precision so their displayed band matches the verdict. Uplift is derived from those displayed scores and shown in percentage points.

Example: 47.0% → 92.0% (+45.0 points) means the skill-assisted run scored 92.0%, 45.0 percentage points above its 47.0% no-skill baseline.

A partial dimension was calculated from only the available configured signals; review the detailed report before relying on it.

Token Usage

Actual Tier 3 execution usage is reported for every observed agent/case pair and both conditions.

Agent	Dataset case	With skill	Without skill	Delta	Change	Coverage
claude-code	All cases	866,006	1,606,045	N/A	N/A	skill 4/4; base 8/8
claude-code	export-approved-slice-previews	439,480	734,185	N/A	N/A	skill 1/1; base 3/3
claude-code	preview-inference-export	366,021	751,181	N/A	N/A	skill 1/1; base 3/3
claude-code	reject-live-training-tracking	29,918	90,793	-60,875	-67.05%	skill 1/1; base 1/1
claude-code	reject-unapproved-raw-upload	30,587	29,886	+701	+2.35%	skill 1/1; base 1/1
codex	All cases	425,426	2,199,946	N/A	N/A	skill 4/4; base 9/9
codex	export-approved-slice-previews	216,112	1,424,108	N/A	N/A	skill 1/1; base 3/3
codex	preview-inference-export	152,355	404,396	N/A	N/A	skill 1/1; base 3/3
codex	reject-live-training-tracking	28,371	357,991	N/A	N/A	skill 1/1; base 2/2
codex	reject-unapproved-raw-upload	28,588	13,451	+15,137	+112.53%	skill 1/1; base 1/1
ALL AGENTS	Dataset aggregate	1,291,432	3,805,991	N/A	N/A	skill 8/8; base 17/17

Prompt tokens include cached reads, so total tokens are prompt + completion (cached is not added twice). The Efficiency score uses (prompt - cached) + completion. N/A means the relevant trajectory counters were not available; coverage is never estimated.

Tier Status

Tier	Purpose	Status	Evidence
Tier 1	Static validation	PASSED WITH OBSERVATIONS	11 validator(s); 15 finding(s)
Tier 2	Semantic deduplication	PASSED	2 validator(s); 0 finding(s)
Tier 3	Live agent evaluation	PASS	2 agent(s); 4 task(s)

Findings and Observations

<details> <summary>Show detailed findings and successful checks</summary>

MEDIUM QUALITY/quality_correctness: SKILL_SPEC recommended field missing: 'metadata.tags' (skills/medtech-model-evidence-export/SKILL.md)
MEDIUM SCRIPT_LINT/deep_nesting: export_evidence_pack.py has deeply nested code (depth 7, max 6) (skills/medtech-model-evidence-export/scripts/export_evidence_pack.py)
MEDIUM SECURITY/Unknown (SQP-2): The skill card explicitly states it exports metadata, parameters, reproducibility details, and quality metrics from Medi (skill-card.md:2)
LOW QUALITY/quality_discoverability: Description very long (289 chars, recommend 50-150) (skills/medtech-model-evidence-export/SKILL.md)
LOW QUALITY/quality_reliability: Inputs are used but no dedicated Inputs section is documented (skills/medtech-model-evidence-export/SKILL.md)
10 additional finding(s) are available in the full evaluation artifacts.

</details>

Scoring Methodology

<details> <summary>Show dimension definitions, source signals, and thresholds</summary>

Dimension	Question	Scored signals
Security	Is it safe to use?	`security` (100%)
Correctness	Is the answer correct?	`accuracy` (100%)
Discoverability	Was the right skill loaded when needed?	`skill_execution` (100%)
Effectiveness	Did the skill help complete the task?	`goal_accuracy` (50%) + `behavior_check` (50%)
Efficiency	Did it avoid wasted tool calls and token usage?	`skill_efficiency` (50%) + `token_efficiency` (50%)

Dimension bands: PASS at 50% or above; NEUTRAL from 40% to below 50%; FAIL below 40%.
Overall Tier 3 lift: PASS at +5 points or more; FAIL at -10 points or less; values between those bands are NEUTRAL.
Overall verdict: PASS only when every configured dimension passes for at least one supported agent. Lift is reported as diagnostic evidence and does not override this gate.
The 50% attempt pass threshold is a separate per-task gate; it is not the dimension pass threshold.
Effectiveness is the equal-weight mean of goal completion (goal_accuracy) and expected workflow adherence (behavior_check).
Efficiency is 50% tool-call productivity (the backward-compatible skill_efficiency wire id) and 50% token_efficiency. Positive-case skill routing is scored under Discoverability, not Efficiency; a negative case without a routing target is N/A. N/A sources are omitted, remaining weights are renormalized, and the dimension is marked partial.

Signals present in this run:

security (Security): unsafe operations, secret leakage, and unauthorized access.
skill_execution (Skill Execution): whether the expected skill was selected, decoys were avoided, and the workflow executed.
skill_efficiency (Tool Productivity): tool-call productivity (legacy wire id; routing is scored under Discoverability).
accuracy (Accuracy): final-answer correctness against the reference answer.
goal_accuracy (Goal Accuracy): whether the user's goal was achieved.
behavior_check (Behavior Check): whether the expected workflow behavior was followed.
token_efficiency (Token Efficiency): actual uncached prompt plus completion usage (50% of Efficiency).

</details>

Freshness

Regenerate this benchmark when the skill, evaluation dataset, target agent/model, evaluator version, environment, or scoring policy changes.

Requirements and compatibility

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

Needs: Declares allowed tools, Needs Python

Security

What our scanner noticed· Medium risk

Large base64-encoded blobs
Scripts that your agent may run

This skill includes scripts that your agent may run. Read them before you use it.

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Attribution and licence

Created by

NVIDIA MedTech

Publisher

NVIDIA (Official)

Source

NVIDIA/skills · skills/medtech-model-evidence-export · commit fce3cca

Licence

Apache License 2.0 (redistribution allowed)

LICENSE (added by AutoGineer from LICENSE-APACHE)
ATTRIBUTION.md (written by AutoGineer)

Synced

2026-11-09

AutoGineer is not affiliated with or endorsed by NVIDIA MedTech or NVIDIA.

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download medtech-model-evidence-export.zip