Nv Segment Ct Finetune

NVIDIAOfficialCreated by NVIDIA MedTech

Skill
Workflow
Data and AI
Apache-2.0
Medium risk
Has scripts

From the author

Runs standard or fixed-channel softmax finetuning of NV-Segment-CT VISTA3D on CT NIfTI image/label datasets, with optional MONAI-native MLflow tracking and checkpoint evidence. Uses softmax for predefined, mutually exclusive classes; keeps the standard workflow when point prompts or runtime-variable classes are needed. Not for clinical validation.

— the description by NVIDIA MedTech

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

This skill provides a structured workflow for fine-tuning the NV-Segment-CT VISTA3D model on CT NIfTI image and label datasets. It supports two primary training modes: a standard continual-learning workflow and a fixed-channel softmax workflow for mutually exclusive class segmentation. The skill includes automated configuration generation, optional MLflow tracking for experiment management, and sanity-check presets for reproducing MSD Task06 Lung Tumor results. It is designed for research and development purposes and is not intended for clinical validation or autonomous diagnosis.

What it does

Fine-tunes VISTA3D models on custom CT NIfTI datasets.
Supports fixed-channel softmax training for mutually exclusive labels.
Provides a sanity-check preset for MSD Task06 Lung Tumor reproduction.
Integrates with MLflow for experiment tracking and metric logging.
Generates structured output including performance metrics and checkpoint recommendations.
Validates input datasets and provides audit reports.

When to use it

When you need to adapt a pre-trained VISTA3D model to a specific CT anatomy.
When labels are mutually exclusive and require a fixed-channel softmax output.
When reproducing standard benchmark results for validation purposes.
When tracking training progress and metrics via MLflow.

Advantages

Automates complex bundle configuration and environment setup.
Provides clear guidance on choosing between standard and softmax workflows.
Includes built-in sanity checks to verify environment and model performance.
Standardizes output reporting for easier model evaluation.

Usage examples

“Fine-tune the model on my lung tumor dataset using the standard workflow.”: The agent executes the fine-tuning script with the provided dataset, target anatomy, and default settings, outputting a recommended checkpoint and performance metrics.
“Run a softmax fine-tuning for my mutually exclusive labels.”: The agent configures the training for fixed-channel output using the specified label mapping and initiates the training process.
“Perform a sanity check using the Task06 Lung Tumor dataset.”: The agent runs the predefined sanity preset, comparing results against reference scores to verify the environment and model behavior.
“Fine-tune with MLflow tracking enabled.”: The agent initiates the training process and logs metrics to the specified MLflow experiment.

Install

Download the skill as a ZIP:
https://autogineer.com/agent-skills/nvidia/nv-segment-ct-finetune/download

Unzip it into the cross-client skills folder of your project:

mkdir -p .agents/skills && unzip nv-segment-ct-finetune.zip -d .agents/skills

Documentation by NVIDIA MedTech

NV-Segment-CT Finetune

Purpose

Used for smoke or dataset finetuning of NV-Segment-CT VISTA3D on CT NIfTI labels, including the upstream fixed-channel softmax workflow and optional MLflow tracking. Not for clinical validation.
Wraps the upstream MONAI bundle entrypoint; do not replace it with handwritten training or inference code.
Manifest inputs are dataset_dir, datalist, target_anatomy, label_mapping, smoke, sanity, auto_seg, softmax, skip_formal_eval, mlflow_tracking_uri, mlflow_experiment_name, and mlflow_run_name.
Manifest outputs are finetuned_ckpt and schema-checked result_json.

Instructions

Run only the caller-requested preset, dataset, output directory, and compute budget. Dependency setup and remote tracking require the caller's approval. After reporting the result, stop; further training, publishing, or deployment is a separate request.
Run scripts/run_finetune.py; do not patch files under bundle/ or upstream checkouts during normal skill use.
For standalone Bash, include the fresh-environment setup line before the wrapper; benchmark venvs start empty.
Run the committed script in place from the repo root. Do not copy this skill to a runtime directory, and do not use rm or cleanup commands in generated invocations.
If a host exposes run_script, use run_script("scripts/run_finetune.py", args=[...]); otherwise run from the repo root.
For the shortest workflow check, use --smoke; for MSD Task06 Lung Tumor reproduction, use --sanity.
Choose between the standard and --softmax workflows using the criteria below. Do not combine --softmax with --auto-seg or --sanity.
Set --mlflow-experiment-name to enable MLflow for the training phase of either workflow. --mlflow-tracking-uri and --mlflow-run-name require an experiment name. Formal pre/post evaluation does not receive MLflow credentials.
Read references/task06-and-results.md only when you need Task06 reference details, output-field definitions, or manual bundle setup notes.

Choosing the Workflow

Use --softmax only when all of these conditions hold:

The complete class set is known before training and will not vary between inference requests.
Labels are mutually exclusive: each voxel is background or exactly one foreground class.
Every foreground dataset label maps to an existing VISTA3D class ID, and a conventional fixed-channel output is desired.

Keep the standard workflow if point prompts must remain available, classes are selected dynamically at inference, labels can overlap, or the Task06 --sanity reproduction is required.

For --label-mapping '[[1,3],[2,13]]', channel 0 is background, channel 1 represents dataset label 1 initialized from VISTA3D class 3, and channel 2 represents dataset label 2 initialized from VISTA3D class 13. Preserve the entries and their order when using the resulting model_softmax.pt with upstream configs/inference_softmax.json. The nv-segment-ct and nv-segment-ctmr inference skills do not currently expose that fixed-channel inference path.

Available Scripts

Script	Purpose	Arguments
`scripts/run_finetune.py`	Primary entrypoint declared by `skill_manifest.yaml`; stages configs, runs MONAI, and writes `output.json`.	`[FIXTURE_OR_DATASET] --output-dir OUT_DIR [--smoke] [--sanity] [--auto-seg] [--softmax] [--dataset-dir DIR] [--datalist JSON] [--target-anatomy TEXT] [--label-mapping JSON] [--patch-size JSON] [--mlflow-experiment-name NAME] [--mlflow-tracking-uri URI] [--mlflow-run-name NAME]`

Prerequisites

Python 3.10+ with CUDA-capable Torch for GPU runs.
Runtime packages from skill_manifest.yaml, especially monai==1.4.0, numpy<2, nibabel, scipy, typer, PyYAML, fire, pytorch-ignite, einops, and huggingface_hub. Install mlflow>=2.10,<4 when MLflow tracking is enabled.
Optional environment variables: CUDA_VISIBLE_DEVICES restricts visible GPUs; NPROC_PER_NODE overrides GPU count and values >=2 select multi-GPU mode for non-sanity runs; NVSEG_FINETUNE_AUTO_VENV=0 disables the cached MONAI 1.4 compatibility environment. Remote tracking may use DATABRICKS_CONFIG_PROFILE, DATABRICKS_HOST, DATABRICKS_TOKEN, MLFLOW_TRACKING_CLIENT_CERT_PATH, MLFLOW_TRACKING_INSECURE_TLS, MLFLOW_TRACKING_PASSWORD, MLFLOW_TRACKING_SERVER_CERT_PATH, MLFLOW_TRACKING_TOKEN, or MLFLOW_TRACKING_USERNAME; these variables are forwarded only when MLflow is explicitly enabled, and unrelated credentials are not forwarded.
--softmax also needs the pinned NVIDIA-Medtech source checkout. Set NV_SEGMENT_CT_ROOT to its NV-Segment-CT directory, or set NV_SEGMENT_CTMR_ROOT to the sibling NV-Segment-CTMR directory. The wrapper reads the official softmax config and implementation in place and writes generated overrides only under --output-dir.
Run outputs: generated bundle configs under skills/nv-segment-ct-finetune/bundle/configs/, including auto_override.json, train_continual_task06_lung.json, and dfw_no_logging.json; checkpoints/evidence under --output-dir; and local tracking data under <output-dir>/mlruns when enabled.
Dependency cache locations: ~/.cache/nvidia-skills/venvs/nv-segment-ct-finetune-monai14/ for MONAI compatibility packages and ~/.cache/huggingface/ for model assets. These are reusable runtime files, not agent instructions or authorization for another run. Set NVSEG_FINETUNE_AUTO_VENV=0 when compatibility-environment setup is not approved; then use a caller-provided compatible environment.
Network access: model/config downloads use https://huggingface.co and https://raw.githubusercontent.com; remote tracking contacts only the caller-approved MLflow or Databricks destination when explicitly enabled. The label-dictionary download accepts HTTPS on the pinned source host and rejects redirects.

Fresh environment setup:

python -m pip install "monai==1.4.0" "numpy<2" pytorch-ignite einops nibabel scipy typer PyYAML fire huggingface_hub

When MLflow tracking is enabled, also install:

python -m pip install "mlflow>=2.10,<4"

Known upstream compatibility constraints:

DFW Task06 reference: Python 3.10.16, MONAI 1.4.0, Torch 2.7.0+cu126.
Use exact monai==1.4.0 for smoke, sanity, and evidence runs; MONAI 1.5.x can crash the upstream finetune loss on boolean labels.
Do not float the dependency as monai>=1.4,<1.6 in generated commands.
The softmax workflow keeps the upstream defaults of 100 epochs and learning rate 1e-4 unless the caller overrides them.

One-time source setup for --softmax:

export NV_SEGMENT_CTMR_COMMIT=cb921f5c58837c0f42a713855d68b32af88e1cdd
export NV_SEGMENT_CTMR_CHECKOUT="$HOME/.cache/nvidia-skills/upstreams/NV-Segment-CTMR-cb921f5"
if [ ! -d "$NV_SEGMENT_CTMR_CHECKOUT/.git" ]; then
  git clone https://github.com/NVIDIA-Medtech/NV-Segment-CTMR.git "$NV_SEGMENT_CTMR_CHECKOUT"
fi
git -C "$NV_SEGMENT_CTMR_CHECKOUT" checkout --detach "$NV_SEGMENT_CTMR_COMMIT"
export NV_SEGMENT_CT_ROOT="$NV_SEGMENT_CTMR_CHECKOUT/NV-Segment-CT"

Usage

Smoke-scale workflow check:

python -m pip install "monai==1.4.0" "numpy<2" pytorch-ignite einops nibabel scipy typer PyYAML fire huggingface_hub && \
python skills/nv-segment-ct-finetune/scripts/run_finetune.py \
  PATH_TO_DATASET \
  --smoke \
  --patch-size '[64,64,64]' \
  --output-dir runs/nvseg_smoke

Use the staged dataset as PATH_TO_DATASET. For the micro fixture, use skills/nv-segment-ct-finetune/fixtures/spleen_micro. Smoke mode proves wiring, config generation, checkpoint loading, and runtime compatibility; it is not a quality bar.

MSD Task06 Lung Tumor sanity reproduction:

python skills/nv-segment-ct-finetune/scripts/run_finetune.py \
  /path/to/Task06 \
  --sanity \
  --output-dir runs/nvseg_task06_sanity

The sanity preset follows the single-GPU DFW recipe: fold-0 validation, label mapping [[1, 23]] for lung tumor, automatic class-prompt segmentation, patch [128,128,128], 5 epochs, and original-spacing configs/evaluate.json scoring before and after training. Expected reference range is pretrained Dice about 0.6697, training-best Dice about 0.6905, and fine-tuned formal Dice about 0.6836.

User-data finetune:

python skills/nv-segment-ct-finetune/scripts/run_finetune.py \
  --dataset-dir /path/to/dataset \
  --datalist /path/to/datalist.json \
  --target-anatomy "lung tumor" \
  --auto-seg \
  --epochs 5 \
  --patch-size '[128,128,128]' \
  --output-dir runs/nvseg_user_finetune

Use --label-mapping '[[1, 23]]' when local label values are custom or the anatomy name is ambiguous.

Optional local MLflow tracking:

python skills/nv-segment-ct-finetune/scripts/run_finetune.py \
  --dataset-dir /path/to/dataset \
  --datalist /path/to/datalist.json \
  --target-anatomy "lung tumor" \
  --epochs 5 \
  --mlflow-experiment-name nvseg-finetune \
  --mlflow-run-name trial-01 \
  --output-dir runs/nvseg_mlflow

This uses MONAI's documented --tracking mlflow path and built-in rank-zero handlers. With no --mlflow-tracking-uri, data stays in <output-dir>/mlruns. Pass a caller-approved remote URI, including databricks, only when remote tracking is intended. MLflow does not change patch size, transforms, optimizer values, DataLoader settings, or other training configuration.

Fixed-channel softmax finetune for mutually exclusive labels:

export NV_SEGMENT_CT_ROOT="$HOME/.cache/nvidia-skills/upstreams/NV-Segment-CTMR-cb921f5/NV-Segment-CT"
python skills/nv-segment-ct-finetune/scripts/run_finetune.py \
  --dataset-dir /path/to/dataset \
  --datalist /path/to/datalist.json \
  --label-mapping '[[1,3],[2,13]]' \
  --softmax \
  --epochs 100 \
  --output-dir runs/nvseg_softmax

This delegates to upstream configs/train_continual_softmax.json. It produces checkpoints/model_softmax.pt; the source model.pt initializes the network but is not compatible with configs/inference_softmax.json. The wrapper therefore recommends the produced softmax checkpoint after a successful run.

Examples

Smoke run on a staged tiny dataset:

python skills/nv-segment-ct-finetune/scripts/run_finetune.py \
  runs/with_vs_without_nv/_inputs/nv_segment_ct_finetune/input_dataset \
  --smoke \
  --patch-size '[64,64,64]' \
  --output-dir runs/nvseg_smoke

Task06 sanity run on a local MSD cache:

python skills/nv-segment-ct-finetune/scripts/run_finetune.py \
  .workbench_data/datasets/Task06_Lung \
  --sanity \
  --output-dir runs/nvseg_task06_sanity

Data Contract

Preferred layout: dataset/imagesTr/*.nii.gz and dataset/labelsTr/*.nii.gz.
Labels must align one-to-one with images by basename.
The target label value must be present in the training labels.
Use a datalist when patient-level splitting matters. The bundle default fold is 0, so fold: 0 entries are validation and all other folds are training.
Every trained foreground label must map to an existing VISTA3D global class id from bundle/label_dict.json; this skill cannot invent a new class.
In --softmax mode, the first mapping column is the saved dataset label and the second is the pretrained VISTA class ID. Mapping order fixes the channel layout and must remain unchanged during inference.

Results

Check output.json in the run directory first:

formal_pretrained_val_dice and formal_finetuned_val_dice: original-spacing pre/post scores when formal eval is enabled.
training_start_val_dice, val_dice_per_epoch, and training_best_val_dice: training-time validation trace.
finetuned_ckpt_matches_pretrained_weights: detects the standard workflow's epoch-0 checkpoint trap when val_at_start=true; softmax uses a different checkpoint architecture.
recommended_ckpt: checkpoint recommendation derived from the recorded workflow and metrics. Inspect those records before selecting a checkpoint; the last epoch or a filename alone is not evidence of improvement. Report the recommendation to the caller; deployment is outside this skill's scope.
invocation.mlflow_tracking: selected tracking URI, experiment name, and optional run name, or null when tracking was disabled.
runtime.oom, runtime.peak_gpu_mb, and phase logs: distinguish OOM, slow validation, and process failure.

Decision rule: prefer formal original-spacing pre/post scores when present; reject tensor-identical "fine-tuned" checkpoints for sanity recovery; treat improved: false as valid evidence rather than a wrapper failure.

Limitations

Thin wrapper. Training, validation, transforms, and checkpointing are delegated to the upstream bundle in bundle/.
Tensor comparison uses restricted weights_only=True checkpoint loading. Unsupported serialized objects produce a comparison error; they are not retried with unrestricted pickle loading. This comparison is not a security audit of the upstream training/checkpoint loader.
Reproduction record only: the successful five-epoch Task06 run used Python 3.12.3, PyTorch 2.12.0+cu130 with CUDA 13.0, MONAI 1.4.0, NumPy 1.26.4, PyTorch-Ignite 0.5.4, NiBabel 5.4.2, SciPy 1.16.0, einops 0.8.2, Fire 0.7.1, Hugging Face Hub 0.36.2, Transformers 4.57.6, Typer 0.25.1, PyYAML 6.0.3, and MLflow 3.14.0 on one NVIDIA RTX 6000 Ada 48 GB GPU. These versions document the evidence environment; they are not additional package constraints or a claim that other versions cannot work.
The auto-derived plan is heuristic; caller-provided --patch-size, --cache-rate, --epochs, and --learning-rate win.
--softmax is not compatible with --sanity: the Task06 reference scores and original-spacing pre/post evaluation belong to the standard VISTA3D continual-learning workflow. Softmax runs record the training validation trajectory but need a separate task-specific evaluation before quality claims.
The Task06 sanity recipe intentionally forces single-GPU execution to match the DFW reference. Multi-GPU mode for other datasets requires host torchrun support.
The paired verifier is CPU-only and audits the evidence pack; it does not re-run GPU segmentation.
MLflow support is optional and uses MONAI's built-in tracking handlers. Tracking errors are part of the upstream MONAI run and can therefore fail the finetune command.
Not for clinical deployment, clinical interpretation, autonomous diagnosis, or regulatory submission.

Troubleshooting

Error	Cause	Fix
Missing dependency or import error	Runtime drift from `skill_manifest.yaml`.	Install the packages above or use the documented environment.
Low Task06 pretrained Dice	Wrong config, wrong checkpoint, data split drift, or dependency drift.	Compare environment fields and staged configs before changing training logic.
`model_finetune.pt` matches pretrained	`val_at_start=true` selected epoch 0 as best.	Use `recommended_ckpt`; treat sanity recovery as failed unless a changed checkpoint improves formal Dice.
Missing formal Dice fields	Formal eval failed or was skipped.	Inspect `eval_pretrained.log`, `eval_finetuned.log`, and `metrics.csv`.
GPU out of memory	Patch/cache settings too large.	Reduce `--patch-size`, lower `--cache-rate`, or reduce workers.
No validation cases	Datalist lacks `fold: 0`.	Provide at least one validation entry.
`--softmax requires the pinned ... checkout`	The August softmax config/implementation is absent or the checkout is at a different commit.	Check out `cb921f5c58837c0f42a713855d68b32af88e1cdd` and set `NV_SEGMENT_CT_ROOT` or `NV_SEGMENT_CTMR_ROOT`.
MLflow tracking fails	MLflow is absent, credentials are invalid, or the experiment is inaccessible.	Inspect `finetune.log`, fix the MLflow client configuration, and rerun; omit `--mlflow-experiment-name` to disable tracking.

Verification

Run the implemented verifier when quality gates matter:

python -m eval_engine.run_trusted skills/nv-segment-ct-finetune \
  --fixture skills/nv-segment-ct-finetune/fixtures/spleen_micro \
  --out runs/nvseg_trusted

Files

ATTRIBUTION.mdAdded by AutoGineer932 B
BENCHMARK.md8.0 KiB
LICENSELicence file added by AutoGineer11.2 KiB
SKILL.md16.4 KiB
evals/evals.json4.5 KiB
fixtures/spleen_micro/datalist.json507 B
references/task06-and-results.md3.8 KiB
scripts/run_finetune.pyScript79.8 KiB
skill-card.md4.3 KiB
skill.oms.sig5.9 KiB
skill_manifest.yaml14.9 KiB
tests/test_run_finetune.pyScript18.7 KiB
validators/output_schema.json7.0 KiB

skill_manifest.yaml

# SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

id: medagent.nv_segment_ct_finetune
version: 0.6.0
upstream_refs:
  - kind: huggingface_repo
    name: nvidia/NV-Segment-CT
    repo_id: nvidia/NV-Segment-CT
    revision: afb51518689f71e6abb367ee6301b2cd0225c66a
  - kind: github_repo
    name: NVIDIA-Medtech/NV-Segment-CTMR
    repo_url: https://github.com/NVIDIA-Medtech/NV-Segment-CTMR
    git_commit: cb921f5c58837c0f42a713855d68b32af88e1cdd
    notes: >
      Compatibility source for the standard training configs and the official
      fixed-channel softmax training implementation.
license: Apache-2.0
intended_use:
  summary: >
    Auto-configuring wrapper around NVIDIA-Medtech NV-Segment-CT's
    continual-learning finetune entries, with optional MONAI-native MLflow
    tracking and the optional official fixed-channel softmax workflow. For
    the MSD Task06 sanity recipe it follows the DFW single-GPU setting: one
    GPU, fold-0 validation, label mapping `[[1, 23]]`, automatic class-prompt
    segmentation, patch `[128,128,128]`, 5 epochs, and original-spacing
    evaluate.json scoring before and after fine-tuning. Anatomy name ->
    label_mappings is resolved against `bundle/label_dict.json` so the caller
    never hand-codes the vista3d global index.
  scope: development
  not_for:
    - clinical deployment
    - clinical interpretation
    - autonomous diagnosis
    - regulatory submission
    - non-CT modalities
inputs:
  - name: dataset_dir
    type: dir_path
    description: Root directory referenced by relative image/label paths in datalist.
    optional_when: smoke == true
  - name: datalist
    type: file_path
    formats: [json]
    description: MONAI-bundle datalist with `training[].fold` and optional `testing[]`.
    optional_when: smoke == true
  - name: target_anatomy
    type: string
    description: Anatomy name resolved against bundle/label_dict.json (e.g. "lung tumor").
    optional: true
  - name: label_mapping
    type: json
    description: Literal `[[user_idx, vista3d_idx], ...]`. Overrides --target-anatomy.
    optional: true
  - name: smoke
    type: bool
    description: Run one iteration on the bundled spleen_micro fixture. Executable plumbing oracle.
    optional: true
    default: false
  - name: sanity
    type: bool
    description: >
      Tutorial-recipe verification on cached MSD06 Lung Tumor. Mirrors the
      published DFW NV-Segment-CT finetune tutorial: full 5-fold split
      (seed 0) on the 63 labeled training cases, label mapping `[[1, 23]]`
      (MSD cancer label -> vista3d global class index for `lung tumor`),
      `drop_label_prob=0.0`, `drop_point_prob=1.0` (automatic segmentation),
      patch `[128,128,128]`, resample `1.5 mm isotropic`, lr=5e-5, 5 epochs.
      It writes `configs/train_continual_task06_lung.json` and
      `configs/dfw_no_logging.json`, uses no multi-GPU configs, runs
      `configs/evaluate.json` on original spacing before and after training,
      and expects DFW-reference scores near pretrained 0.6697,
      fine-tuned 0.6836, and training-best 0.6905. The sanity gate fails
      when reference recovery is not demonstrated or when the best checkpoint
      is still tensor-identical to the pretrained checkpoint; inspect recorded
      Python, MONAI, Torch, and CUDA versions before trusting low scores.
      Requires `.workbench_data/datasets/Task06_Lung/` to be populated unless
      a Task06 path is supplied.
    optional: true
    default: false
  - name: auto_seg
    type: bool
    description: >
      Force automatic class-prompt finetuning with `drop_label_prob=0.0` and
      `drop_point_prob=1.0`, matching the DFW Task06 recipe. Enabled
      automatically by --sanity; optional for caller-provided datasets.
    optional: true
    default: false
  - name: softmax
    type: bool
    description: >
      Use the upstream fixed-channel softmax workflow for mutually exclusive
      semantic-segmentation labels. The mapping order defines output channels;
      interactive point prompts and dynamic label prompts are unavailable.
    optional: true
    default: false
  - name: skip_formal_eval
    type: bool
    description: Skip the pre/post original-spacing evaluate.json passes.
    optional: true
    default: false
  - name: mlflow_tracking_uri
    type: string
    description: Optional MLflow tracking URI; defaults to a local store under the output directory.
    optional: true
  - name: mlflow_experiment_name
    type: string
    description: MLflow experiment name; setting it enables MONAI-native live tracking.
    optional: true
  - name: mlflow_run_name
    type: string
    description: Optional MLflow run name.
    optional: true
outputs:
  - name: finetuned_ckpt
    type: file_path
    formats: [pytorch]
    description: >
      Best-val-dice checkpoint produced by the bundle's `validate#handlers`:
      model_finetune.pt for the standard workflow or model_softmax.pt for
      --softmax. With val_at_start=True the standard checkpoint can be a
      reserialized copy of the pretrained weights; inspect
      output.finetuned_ckpt_matches_pretrained_weights.
    optional_when: smoke == true
  - name: result_json
    type: json
    schema: validators/output_schema.json
runtime:
  language: python
  python: ">=3.10"
  entrypoint: scripts/run_finetune.py
  args:
    - "${python}"
    - "${script}"
    - "${fixture}"
    - "--output-dir"
    - "${out}/artifacts"
  dependencies:
    # Upstream NV-Segment-CT metadata and requirements pin MONAI 1.4.0.
    monai: "==1.4.0"
    torch: ">=2.0"
    transformers: ">=4.40,<5"
    nibabel: ">=4.0"
    numpy: ">=1.23"
    scipy: ">=1.10"
    huggingface_hub: "*"
    safetensors: ">=0.4"
    mlflow: ">=2.10,<4"
    typer: ">=0.9"
  side_effects:
    pip_packages:
      - monai==1.4.0
      - torch>=2.0
      - "transformers>=4.40,<5"
      - huggingface_hub
      - "mlflow>=2.10,<4"
      - nibabel>=4.0
      - numpy>=1.23
      - safetensors>=0.4
      - typer>=0.9
      - scipy>=1.10
    local_writes:
      - {path: "skills/nv-segment-ct-finetune/bundle/", approx_mb_max: 1000}
      - {path: "skills/nv-segment-ct-finetune/bundle/configs/auto_override.json", approx_mb_max: 1}
      - {path: "skills/nv-segment-ct-finetune/bundle/configs/train_continual_task06_lung.json", approx_mb_max: 1}
      - {path: "skills/nv-segment-ct-finetune/bundle/configs/dfw_no_logging.json", approx_mb_max: 1}
      - {path: "<caller-provided --output-dir>", approx_mb_max: 20000}
      - {path: "<caller-provided --output-dir>/mlruns", approx_mb_max: 100, optional: true}
    home_writes:
      - {path: "~/.cache/huggingface/", approx_mb_max: 1500}
      - {path: "~/.cache/nvidia-skills/venvs/nv-segment-ct-finetune-monai14/", approx_mb_max: 500}
    network_endpoints:
      - https://huggingface.co
      - https://raw.githubusercontent.com
      - "https://<caller-provided-mlflow-or-databricks-workspace>"
    requires_docker: false
    requires_gpu: cuda
    gpu_fallback: cpu
    environment:
      clean_environment_required: false
      clean_environment_recommended: true
      modifies_active_python_environment: true
      user_environment_modification_ok: true
      recommended_isolation: fresh venv or container for benchmarks; caller-selected env for interactive use
      notes: >
        Runtime setup commands may install packages into the active Python
        environment. That is acceptable only when the caller chooses that
        environment; benchmark and evidence runs should use a fresh per-run
        environment.
    env_required: []
    env_optional:
      - NPROC_PER_NODE   # overrides auto-detected GPU count (>=2 selects multi-GPU)
      - CUDA_VISIBLE_DEVICES
      - NV_SEGMENT_CT_ROOT
      - NV_SEGMENT_CTMR_ROOT
      - NVSEG_FINETUNE_AUTO_VENV
      - DATABRICKS_CONFIG_PROFILE
      - DATABRICKS_HOST
      - DATABRICKS_TOKEN
      - MLFLOW_TRACKING_CLIENT_CERT_PATH
      - MLFLOW_TRACKING_INSECURE_TLS
      - MLFLOW_TRACKING_PASSWORD
      - MLFLOW_TRACKING_SERVER_CERT_PATH
      - MLFLOW_TRACKING_TOKEN
      - MLFLOW_TRACKING_USERNAME
  external_assets:
    - kind: huggingface_repo
      repo_id: nvidia/NV-Segment-CT
      size_mb_approx: 832
      install_path: bundle/
      install_command: hf download nvidia/NV-Segment-CT --revision afb51518689f71e6abb367ee6301b2cd0225c66a --local-dir skills/nv-segment-ct-finetune/bundle/
      contains:
        - configs/train.json
        - configs/train_continual.json
        - configs/multi_gpu_train.json
        - configs/evaluate.json
        - configs/metadata.json
        - label_dict.json
        - vista3d_pretrained_model/model.safetensors
    - kind: upstream_repo
      repo_url: https://github.com/NVIDIA-Medtech/NV-Segment-CTMR
      install_path: ~/.cache/nvidia-skills/upstreams/NV-Segment-CTMR-cb921f5/NV-Segment-CT
      install_command: >
        git clone https://github.com/NVIDIA-Medtech/NV-Segment-CTMR.git $HOME/.cache/nvidia-skills/upstreams/NV-Segment-CTMR-cb921f5 &&
        git -C $HOME/.cache/nvidia-skills/upstreams/NV-Segment-CTMR-cb921f5 checkout --detach cb921f5c58837c0f42a713855d68b32af88e1cdd &&
        export NV_SEGMENT_CT_ROOT=$HOME/.cache/nvidia-skills/upstreams/NV-Segment-CTMR-cb921f5/NV-Segment-CT
      contains:
        - configs/train_continual_softmax.json
        - configs/inference_softmax.json
        - scripts/vista3d_softmax.py
cost:
  # Per-invocation agent-overhead token cost measured by NeMo Agent Toolkit
  # (NAT) profiler - the cost an LLM-driven agent pays to call this skill
  # once (here: --smoke preset, ~30 s GPU). The skill itself emits zero
  # tokens. See tools/nat_audit/README.md for methodology.
  token_estimate:
    common:
      model: meta/llama-3.3-70b-instruct
      agent_type: tool_calling_agent
      measured_at: 2026-05-16
      methodology: tools/nat_audit/README.md
    isolated_tool_call:
      prompt_tokens: 2279
      completion_tokens: 87
      total_tokens: 2366
      llm_calls: 2
      n_tools_in_workflow: 9
    end_to_end_workflow:                 # SKILL.md is the largest in Medical AI Skills (~3300 tok)
      prompt_tokens: 12668
      completion_tokens: 151
      total_tokens: 12819
      llm_calls: 3
      n_tools_in_workflow: 11
      scenario: realistic_user_workflow

paired_verifiers:
  - id: medagent.verifiers.ct_segmentation_finetune_quality_v1
    status: implemented
    consumes: evidence_pack_dir
    purpose: >
      Audits the finetuned checkpoint at output.finetuned_ckpt (size + cpu
      torch.load when available), the training trajectory
      (val_dice_per_epoch, train_loss_finite, no OOM, improvement_over_baseline,
      sanity_recovery_demonstrated under --sanity), checkpoint tensor comparison
      against the pretrained checkpoint when present, and the recorded
      input.dataset_audit (shape / orientation consistency, CT HU range,
      anatomy-specific volume bounds, label coverage against
      label_mappings#default). CPU-only; does not re-run training.
limitations:
  - Thin wrapper. Training, validation, transforms, and checkpointing are
    delegated entirely to the upstream bundle in `bundle/`. Do not modify
    code under `bundle/`.
  - The auto-derived plan is a heuristic. Caller-provided `--patch-size`,
    `--cache-rate`, `--epochs`, `--learning-rate` always win.
  - The Task06 sanity recipe intentionally forces single-GPU execution to
    match the DFW reference setting. Multi-GPU mode for other datasets requires
    the host to be able to launch `torchrun` with
    the chosen `--nproc_per_node`. The wrapper does not allocate or
    fence GPUs.
  - The fixed-channel softmax workflow is additive and opt-in. It disables
    point prompts and dynamic label prompts, requires mapping order to remain
    stable between training and inference, cannot be combined with `--auto-seg`
    or `--sanity`, and does not use the standard Task06 pre/post evaluation
    baseline.
  - Optional MLflow support uses MONAI's built-in rank-zero tracking handlers.
    Tracking failures can fail the upstream MONAI run.
  - Smoke mode is an executable oracle for "pipeline plumbs end to end",
    not a quality bar - it cannot prove the full-dataset finetune will
    converge.
validation:
  expected_runtime_seconds:
    # smoke_only on H100 ~= 5s; CPU fallback ~= 120s. Real finetune is
    # bounded only by the caller's epoch budget - caller should pass a
    # custom benchmark manifest with wider envelopes for long runs.
    min: 1.0
    max: 1800.0
    inference_path: runtime.wall_seconds
  sanity_checks:
    - path: plan.patch_size
      length_eq: 3
    - path: invocation.config_stack
      contains: "configs/train.json"
    - path: invocation.config_stack
      contains: "configs/train_continual.json"
    - path: invocation.command_prefix
      contains: "monai.bundle"
    - path: output.oom
      eq: false
    - path: output.train_loss_finite
      eq: true
    - path: output.finetuned_ckpt_exists
      eq: true
    - path: output.regressed
      eq: false
  expected_cost:
    wall_seconds:        {max: 1800}
    cpu_seconds:         {max: 7200}
    rss_mb_peak:         {min: 500, max: 32000}
    gpu_seconds:         {max: 1800}
    gpu_memory_mb_peak:  {max: 80000}
  # The wrapper's patch_size ladder (scripts/run_finetune.py::PATCH_LADDER)
  # is calibrated against measurements on RTX 6000 Ada 48 GB with monai
  # 1.4.0 / torch 2.11+cu130 and real CT finetune workloads,
  # cache_rate 0.0, resample_to_spacing 1.5 mm isotropic:
  #   [64,64,64]    ->  5,510 MiB peak
  #   [96,96,96]    ->  8,132 MiB
  #   [128,128,128] -> 13,220 MiB   (bundle default)
  #   [160,160,160] -> 21,560 MiB
  #   [192,192,128] -> 24,208 MiB   (finetune.md "larger memory")
  #   [224,224,224] -> 47,354 MiB   (within 1 GB of OOM on 48 GB)
  #   [256,256,256] ->  OOM
  # Values are sub-cubic up to [160,160,160] then super-linear; do not
  # interpolate naively outside the measured range.
  # DFW Task06 reference: single GPU, patch [128,128,128], formal pre/post
  # eval plus 5-epoch finetune peaked at 10381 MiB (10.14 GiB).
  env_pin:
    monai: "==1.4.0"
    torch: ">=2.0,<3"
    transformers: ">=4.40,<5"
    huggingface_hub: ">=0.20,<1"
  reproducibility:
    mode: preflight
    fixture: fixtures/spleen_micro
    runs: 2
    reason: >
      Full finetune repeatability must run in the declared MONAI 1.4.0
      environment and compare the emitted checkpoint hash plus trajectory.
      The repository audit repeats the fixture/env boundary check so a host
      with drifted dependencies cannot be mistaken for a verified finetune
      reproduction.

Requirements and compatibility

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

Needs: Declares allowed tools, Needs a command-line tool, Needs Python

Security

What our scanner noticed· Medium risk

Large base64-encoded blobs
Instructions that use the network
Scripts that your agent may run

This skill includes scripts that your agent may run. Read them before you use it.

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Attribution and licence

Created by

NVIDIA MedTech

Publisher

NVIDIA (Official)

Source

NVIDIA/skills · skills/nv-segment-ct-finetune · commit fce3cca

Licence

Apache License 2.0 (redistribution allowed)

LICENSE (added by AutoGineer from LICENSE-APACHE)
ATTRIBUTION.md (written by AutoGineer)

Synced

2026-11-09

AutoGineer is not affiliated with or endorsed by NVIDIA MedTech or NVIDIA.

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download nv-segment-ct-finetune.zip