Tao Train Fast Foundation Stereo
NVIDIAOfficialCreated by NVIDIA Corporation
- Skill
- Tool integration
- Data and AI
- Apache-2.0
- Medium risk
AutoGineer summary
Written by AutoGineer from the skill’s own files, not by the author
The tao-train-fast-foundation-stereo skill enables users to work with the FastFoundationStereo (FFS) model, a distilled variant of FoundationStereo designed for stereo depth estimation. It integrates with the TAO depth_net CLI to support the full model lifecycle, including training, evaluation, export, and inference. The skill provides specific configuration requirements for model architecture parameters and dataset handling to ensure compatibility with the bp2 commercial checkpoint.
What it does
- Supports training and fine-tuning of FastFoundationStereo models.
- Facilitates model evaluation using standard stereo depth metrics.
- Enables export of models to TensorRT engines for deployment.
- Provides inference capabilities for generating disparity maps from stereo image pairs.
- Includes AutoML support for optimizing training hyperparameters.
- Manages configuration templates for consistent model deployment.
When to use it
- When you need to train or fine-tune a FastFoundationStereo model on custom stereo datasets.
- When evaluating the performance of a FastFoundationStereo model using ground truth disparity data.
- When deploying a pre-trained bp2 commercial checkpoint for real-time inference.
- When converting a FastFoundationStereo model to a TensorRT engine for optimized execution.
Advantages
- Provides a unified CLI interface for the entire model development lifecycle.
- Includes specific configuration overrides to ensure compatibility with bp2 commercial checkpoints.
- Supports both raw deployment and fine-tuning workflows.
- Integrates with TAO AutoML for hyperparameter optimization.
Usage examples
- “Train a FastFoundationStereo model using my custom dataset.”
- The agent will guide you through setting up the spec file with the required model width overrides and launching the training job via the depth_net CLI.
- “Export my FastFoundationStereo model to a TensorRT engine.”
- The agent will assist in configuring the export action and generating the optimized TensorRT engine for deployment.
Install
Download the skill as a ZIP:
https://autogineer.com/agent-skills/nvidia/tao-train-fast-foundation-stereo/downloadUnzip it into the cross-client skills folder of your project:
mkdir -p .agents/skills && unzip tao-train-fast-foundation-stereo.zip -d .agents/skills
Documentation by NVIDIA Corporation
Depth Net Fast Stereo
Standalone install? If this session was not initialized by the TAO skill bank plugin, run the
tao-setupskill first (host preflight, credentials, cross-skill discovery).
Real-time stereo depth estimation using FastFoundationStereo (FFS) — the bp2 commercial distilled variant of FoundationStereo. Predicts disparity maps from rectified stereo image pairs with per-layer pruned widths for real-time inference.
The mono / stereo / fast-stereo skills share the unified TAO depth_net CLI; FFS is selected via model.model_type: FastFoundationStereo. FFS differs from FoundationStereo only in pruned per-layer widths and a serialized forward path; everything else (entrypoint, action verbs, dataset classes, deploy chain) is identical to depth-net-stereo.
For TAO Deploy TensorRT actions (gen_trt_engine, TensorRT evaluate, TensorRT inference), read references/tao-deploy-fast-foundation-stereo.md first. The deploy spec template lives at references/spec_template_deploy.yaml.
When to Use
Use this skill to train, evaluate, export, or run inference for a TAO FastFoundationStereo model. Two supported use cases:
FFS raw-deploy and bp2-finetune flows require a pre-trained bp2 commercial checkpoint (model_best_bp2_serialize.pth). The default PyT image does not guarantee that this file is present on disk, so treat the checkpoint path as a required user/registry artifact. If no bp2 checkpoint is available, scratch training is still usable for workflow validation, but the resulting metrics are not representative of the bp2 model.
Raw deploy — use the bp2 ckpt as-is. Skip
train; runinference/evaluate/export/gen_trt_enginedirectly with the bp2 file as the action's checkpoint.Finetune on user data — set
train.pretrained_model_pathto the bp2 file, train on user data, then verify + deploy on the resulting ckpt. The full 7-action sequence (train → evaluate pyt → inference pyt → export → gen_trt_engine → inference deploy → evaluate deploy) is supported.
Train Action Policy
This model is AutoML-enabled at the model layer. Before handling any train-stage request, read references/skill_info.yaml and resolve the run override from either an explicit automl_policy value or the user's workflow request. Use automl_policy: on by default and only expose on / off in new launch prompts. Treat phrases like "turn off AutoML", "disable AutoML", "no HPO", or "plain training" as automl_policy: off for this run only. When automl_policy: on, automl_enabled: true, and both schemas/train.schema.json and references/spec_template_train.yaml are packaged, route the train action through tao-skill-bank:tao-run-automl by default with this model's skill_dir. Preserve workflow/application overrides for datasets, specs, output directories, GPU/platform settings, parent checkpoints, and automl_policy. Use direct model training only when automl_policy: off or the packaged train schema/template is missing; in the missing-schema case, report that AutoML is enabled but not runnable for this model until schemas are generated.
FFS shares the depth_net_stereo schema but its bp2 architecture widths are fixed invariants. For default AutoML, search only train.optim.lr and train.optim.lr_decay unless the user explicitly requests a wider search. Do not include FFS architecture fields such as model.volume_dim, model.hidden_dims, or other bp2 width settings in the default search space.
Non-train actions such as evaluate, inference, export, and deploy flows stay in this model skill. The per-run automl_policy override does not change model metadata.
Workflow
Prerequisites — data accessibility
Your dataset (left + right images + GT disparity for train / evaluate, left + right only for inference) must be reachable from inside the container:
SDK runner: place files at the S3 paths the runner resolves (
S3_TRAIN/S3_EVALplaceholders shown in spec overrides).Direct
docker run(e.g. local testing): mount the host dataset root read-only at the same in-container path:
docker run ... -v <host_data_root>:<host_data_root>:ro <container> ...The same accessibility requirement applies to the <output_dir> written by all actions, and to the bp2 checkpoint path.
Step 1 — Annotation file
Per-line annotation file referenced by data_sources[*].data_file. Schema is identical to depth-net-stereo:
| Columns | Format | Use |
|---|---|---|
| 2 | <left> <right> | Stereo inference (no GT) |
| 3 | <left> <right> <disparity> | Stereo with GT |
| 4 | <left> <right> <disparity> <occlusion_mask> | Stereo with GT and occlusion mask |
Generate via depth_net convert if needed; see the depth-net-stereo skill for convert_spec.yaml template.
Step 2 — Pair model_type and dataset_name based on your data
Use model_type: FastFoundationStereo for FFS. The dataset_name choice mirrors the stereo skill — pick the dataset-specific class when your layout matches a registered one, otherwise GenericDataset.
| Data category | model_type | dataset_name |
|---|---|---|
| Middlebury | FastFoundationStereo | Middlebury |
| KITTI | FastFoundationStereo | Kitti |
| ETH3D | FastFoundationStereo | Eth3d |
| FSD synthetic | FastFoundationStereo | FSD |
| IsaacReal synthetic | FastFoundationStereo | IsaacRealDataset |
| Crestereo synthetic | FastFoundationStereo | Crestereo |
| Other / non-canonical | FastFoundationStereo | GenericDataset |
For inference with 2-column annotations (left + right, no GT), use dataset_name: GenericDataset regardless of layout.
Step 3 — Set the bp2 distilled width overrides
FFS requires 15 model-section width override fields whose values match the bp2 commercial checkpoint exactly. Omitting any field falls back to TAO defaults that do not match the bp2 ckpt and produce shape-mismatch errors at forward time. See references/setup-and-run.md for the full copy-as-is model: block and notes. The spec templates at references/spec_template_*.yaml carry this block as the canonical source.
Step 4 — Write spec yaml from spec overrides
Copy the action block from references/spec-overrides.md. Replace:
model.model_type: FastFoundationStereo(already set)dataset.<...>.data_sources[*].dataset_namefrom Step 2dataset.<...>.data_sources[*].data_filewith the path from Step 1For raw deploy use cases (no train): set
<action>.checkpointto the bp2 file pathFor finetune use cases: set
train.pretrained_model_pathto the bp2 file path
For chained train → next-action checkpoint path resolution and shape-consistency notes, see references/setup-and-run.md. SDK-runner deploys resolve handoff automatically via parent_job_id — see references/parent-model-inference.md.
Step 5 — Run
Create writable home/cache directories inside the mounted output path before using --user, then launch docker run ... depth_net <action> -e <spec.yaml>. See references/setup-and-run.md for the full mkdir + docker run command, the --user rationale, and the local bind-mount __pycache__ tip.
Step 6 — Verify
Check container exit code 0 and a populated status.json kpi block. For train inspect per-step train_loss directly (the entrypoint reports Execution status: PASS even when loss is NaN); for evaluate rely on epe / bp1 / bp2 / bp3 / d1 / rmse; for inference check artifacts under results_dir. The pyt-vs-deploy KPI namespace difference and the expected deploy drift are detailed in references/setup-and-run.md.
7-action deploy flow
train (optional) → finetuned ckpt
evaluate (pyt) → PyT eager EPE / bp on val GT
inference (pyt) → PyT eager disparity samples (visual sanity)
export → static fp32 ONNX (recommended at 480×736 or 320×736)
gen_trt_engine → fp16 TRT engine on static ONNX path
inference (deploy) → TRT disparity samples
evaluate (deploy) → TRT EPE / bp drift vs PyT eager fp32Skip train for raw-bp2 deploy. The remaining 6 actions (or the 4 deploy-only verbs starting from export) cover both use cases.
Training Requirements
Valid
dataset_namevalues for stereodata_sources(case-insensitive):FSD,IsaacRealDataset,Crestereo,Middlebury,Eth3d,Kitti,GenericDatasetMonitoring metric: val/loss
Per-Action Dataset Requirements
| Action | Spec Key | Source | Files | List? |
|---|---|---|---|---|
| evaluate | dataset.test_dataset.data_sources | eval_dataset | data_file: annotations.txt + dataset_name | Yes |
| inference | dataset.infer_dataset.data_sources | inference_dataset | data_file: annotations.txt + dataset_name | Yes |
| train | dataset.train_dataset.data_sources | train_datasets | data_file: annotations.txt + dataset_name | Yes |
| train | dataset.val_dataset.data_sources | eval_dataset | data_file: annotations.txt + dataset_name | Yes |
Typical Spec Overrides
Data source overrides are mandatory for every action. Each data_sources entry is a dict with two mandatory fields: data_file and dataset_name. The model.* width fields are also mandatory — see Step 3. See references/spec-overrides.md for the FFS_MODEL_BLOCK and per-action (train / evaluate / inference / export) Python override dicts.
Eval Dataset
Optional. Val dataset configured via dataset.val_dataset.data_sources (each entry needs data_file and dataset_name).
Important Parameters
Key knobs include model.model_type (FastFoundationStereo), model.encoder (vitl), model.max_disparity (set 192 explicitly — schema default 416 causes severe drift), model.mixed_precision (false), model.gwc_feature_normalize (true), model.volume_dim (28), model.valid_iters (8), and per-split batch_size / workers / crop_size / data_sources. Full parameter reference, evaluation metrics, multi-GPU / multi-node spec keys, export / TRT defaults, the export use-case matrix, and hardware guidance are in references/important-parameters.md.
Error Patterns
For shape mismatch, gwc_feature_normalize schema errors, max_disparity drift, negative disparity, depth_net_stereo: not found, the pyt-evaluate crop_size asymmetry, the Failed to import SAM3 warning, and the dynamic-engine stride-incompatible silent failure, see references/error-patterns.md.
Spec Param / Parent Model Inference
Model-specific inference mappings (per-action spec field → inference function) for train / evaluate / inference / export / gen_trt_engine, plus parent_job_id / parent_model resolution and raw-bp2 explicit-checkpoint handling, are in references/parent-model-inference.md. Generated runners should read that section and apply the mappings with SDK helpers before create_job().
Deployment
Files
- ATTRIBUTION.mdAdded by AutoGineer966 B
- BENCHMARK.md7.3 KiB
- LICENSELicence file added by AutoGineer11.2 KiB
- SKILL.md11.6 KiB
- config/skillspector-baseline.yaml1.0 KiB
- evals/evals.json839 B
- references/error-patterns.md3.2 KiB
- references/important-parameters.md7.3 KiB
- references/parent-model-inference.md3.0 KiB
- references/setup-and-run.md5.5 KiB
- references/skill_info.yaml4.4 KiB
- references/spec-overrides.md3.4 KiB
- references/spec_template_deploy.yaml2.7 KiB
- references/spec_template_evaluate.yaml1.8 KiB
- references/spec_template_export.yaml1.8 KiB
- references/spec_template_inference.yaml1.8 KiB
- references/spec_template_train.yaml2.6 KiB
- references/tao-deploy-fast-foundation-stereo.md19.3 KiB
- references/tao-deploy-fast-foundation-stereo.skill_info.yaml3.7 KiB
- schemas/train.schema.json99.2 KiB
- skill-card.md4.7 KiB
- skill.oms.sig8.2 KiB
schemas/train.schema.json
{
"automl_default_parameters": [
"train.optim.lr",
"train.optim.lr_decay"
],
"automl_disabled_parameters": [
"quantize",
"train.cudnn",
"dataset.val_dataset.data_sources",
"quantize.backend_kwargs",
"dataset.train_dataset.augmentation",
"dataset.test_dataset.augmentation.input_std",
"train.gpu_ids",
"wandb.tags",
"dataset.infer_dataset.augmentation",
"dataset.train_dataset.data_sources",
"dataset.train_dataset.augmentation.color_aug_saturation",
"dataset.infer_dataset.augmentation.color_aug_hue_range",
"dataset.train_dataset",
"quantize.skip_names",
"dataset.infer_dataset.data_sources",
"dataset.val_dataset.augmentation.input_std",
"inference",
"evaluate",
"train",
"dataset.val_dataset.augmentation.input_mean",
"dataset.test_dataset.data_sources",
"gen_trt_engine",
"dataset.train_dataset.augmentation.input_std",
"dataset.train_dataset.augmentation.input_mean",
"dataset.test_dataset",
"gen_trt_engine.tensorrt.layers_precision",
"dataset",
"gen_trt_engine.tensorrt",
"dataset.val_dataset",
"dataset.val_dataset.augmentation.gamma",
"quantize.layers",
"dataset.infer_dataset",
"dataset.test_dataset.augmentation.color_aug_saturation",
"dataset.infer_dataset.augmentation.input_mean",
"dataset.test_dataset.augmentation",
"dataset.train_dataset.augmentation.crop_size",
"dataset.infer_dataset.augmentation.gamma",
"dataset.infer_dataset.augmentation.crop_size",
"dataset.quant_calibration_dataset",
"dataset.infer_dataset.augmentation.input_std",
"model.stereo_backbone",
"model.hidden_dims",
"dataset.train_dataset.augmentation.color_aug_hue_range",
"model",
"train.optim.lr_steps",
"dataset.test_dataset.augmentation.gamma",
"dataset.val_dataset.augmentation.color_aug_saturation",
"evaluate.gpu_ids",
"dataset.test_dataset.augmentation.crop_size",
"train.optim",
"dataset.val_dataset.augmentation.crop_size",
"dataset.val_dataset.augmentation",
"dataset.test_dataset.augmentation.input_mean",
"model.mono_backbone",
"dataset.train_dataset.augmentation.gamma",
"dataset.test_dataset.augmentation.color_aug_hue_range",
"export",
"wandb",
"dataset.val_dataset.augmentation.color_aug_hue_range",
"dataset.infer_dataset.augmentation.color_aug_saturation",
"inference.gpu_ids",
"train.tile_min_overlap"
],
"default": {
"dataset": {
"baseline": 0.193001,
"dataset_name": "StereoDataset",
"focal_x": 1998.842,
"infer_dataset": {
"augmentation": {
"color_aug_brightness": 0.4,
"color_aug_contrast": 0.4,
"color_aug_hue_range": [
-0.027777777777777776,
0.027777777777777776
],
"color_aug_prob": 0.2,
"color_aug_saturation": [
0.0,
1.4
],
"crop_min_valid_disp_ratio": 0.0,
"crop_size": [
518,
518
],
"do_flip": false,
"eraser_aug_prob": 0.5,
"gamma": [
1,
1,
1,
1
],
"h_flip_prob": 0.5,
"hshift_prob": 0.5,
"input_mean": [
0.485,
0.456,
0.406
],
"input_std": [
0.229,
0.224,
0.225
],
"max_scale": 0.4,
"max_stretch": 0.2,
"min_scale": -0.2,
"spatial_aug_prob": 1.0,
"stretch_prob": 0.8,
"v_flip_prob": 0.5,
"yjitter_prob": 1.0
},
"batch_size": 1,
"data_sources": [
{
"data_file": "",
"dataset_name": ""
}
],
"pin_memory": true,
"workers": 8
},
"max_disparity": 416,
"normalize_depth": false,
"quant_calibration_dataset": {
"images_dir": ""
},
"test_dataset": {
"augmentation": {
"color_aug_brightness": 0.4,
"color_aug_contrast": 0.4,
"color_aug_hue_range": [
-0.027777777777777776,
0.027777777777777776
],
"color_aug_prob": 0.2,
"color_aug_saturation": [
0.0,
1.4
],
"crop_min_valid_disp_ratio": 0.0,
"crop_size": [
518,
518
],
"do_flip": false,
"eraser_aug_prob": 0.5,
"gamma": [
1,
1,
1,
1
],
"h_flip_prob": 0.5,
"hshift_prob": 0.5,
"input_mean": [
0.485,
0.456,
0.406
],
"input_std": [
0.229,
0.224,
0.225
],
"max_scale": 0.4,
"max_stretch": 0.2,
"min_scale": -0.2,
"spatial_aug_prob": 1.0,
"stretch_prob": 0.8,
"v_flip_prob": 0.5,
"yjitter_prob": 1.0
},
"batch_size": 1,
"data_sources": [
{
"data_file": "",
"dataset_name": ""
}
],
"pin_memory": true,
"workers": 8
},
"train_dataset": {
"augmentation": {
"color_aug_brightness": 0.4,
"color_aug_contrast": 0.4,
"color_aug_hue_range": [
-0.027777777777777776,
0.027777777777777776
],
"color_aug_prob": 0.2,
"color_aug_saturation": [
0.0,
1.4
],
"crop_min_valid_disp_ratio": 0.0,
"crop_size": [
518,
518
],
"do_flip": false,
"eraser_aug_prob": 0.5,
"gamma": [
1,
1,
1,
1
],
"h_flip_prob": 0.5,
"hshift_prob": 0.5,
"input_mean": [
0.485,
0.456,
0.406
],
"input_std": [
0.229,
0.224,
0.225
],
"max_scale": 0.4,
"max_stretch": 0.2,
"min_scale": -0.2,
"spatial_aug_prob": 1.0,
"stretch_prob": 0.8,
"v_flip_prob": 0.5,
"yjitter_prob": 1.0
},
"batch_size": 1,
"data_sources": [
{
"data_file": "",
"dataset_name": ""
}
],
"pin_memory": true,
"workers": 8
},
"val_dataset": {
"augmentation": {
"color_aug_brightness": 0.4,
"color_aug_contrast": 0.4,
"color_aug_hue_range": [
-0.027777777777777776,
0.027777777777777776
],
"color_aug_prob": 0.2,
"color_aug_saturation": [
0.0,
1.4
],
"crop_min_valid_disp_ratio": 0.0,
"crop_size": [
518,
518
],
"do_flip": false,
"eraser_aug_prob": 0.5,
"gamma": [
1,
1,
1,
1
],
"h_flip_prob": 0.5,
"hshift_prob": 0.5,
"input_mean": [
0.485,
0.456,
0.406
],
"input_std": [
0.229,
0.224,
0.225
],
"max_scale": 0.4,
"max_stretch": 0.2,
"min_scale": -0.2,
"spatial_aug_prob": 1.0,
"stretch_prob": 0.8,
"v_flip_prob": 0.5,
"yjitter_prob": 1.0
},
"batch_size": 1,
"data_sources": [
{
"data_file": "",
"dataset_name": ""
}
],
"pin_memory": true,
"workers": 8
}
},
"encryption_key": "",
"model": {
"corr_levels": 2,
"corr_radius": 4,
"cv_group": 8,
"encoder": "vitl",
"hidden_dims": [
128,
128,
128
],
"low_memory": 0,
"max_disparity": 416,
"mixed_precision": false,
"model_type": "MetricDepthAnything",
"mono_backbone": {
"pretrained_path": "",
"use_bn": false,
"use_clstoken": false
},
"n_downsample": 2,
"n_gru_layers": 3,
"stereo_backbone": {
"depth_anything_v2_pretrained_path": "",
"edgenext_pretrained_path": "",
"use_bn": false,
"use_clstoken": false
},
"train_iters": 22,
"valid_iters": 22,
"volume_dim": 32
},
"model_name": "",
"quantize": {
"algorithm": "minmax",
"backend": "torchao",
"backend_kwargs": {},
"device": "cuda",
"layers": [],
"mode": "weight_only_ptq",
"model_path": "",
"results_dir": "",
"skip_names": []
},
"results_dir": "",
"train": {
"activation_checkpoint": false,
"checkpoint_interval": 1,
"checkpoint_interval_unit": "epoch",
"clip_grad_norm": 0.1,
"cudnn": {
"benchmark": false,
"deterministic": true
},
"dataloader_visualize": false,
"distributed_strategy": "ddp",
"gpu_ids": [
0
],
"inference_tile": false,
"is_dry_run": false,
"log_every_n_steps": 500,
"num_epochs": 10,
"num_gpus": 1,
"num_nodes": 1,
"optim": {
"lr": 0.0001,
"lr_decay": 0.1,
"lr_scheduler": "MultiStepLR",
"lr_step_size": 1000,
"lr_steps": [
1000
],
"min_lr": 1e-07,
"momentum": 0.9,
"monitor_name": "val_loss",
"optimizer": "AdamW",
"warmup_steps": 20,
"weight_decay": 0.0001
},
"precision": "fp32",
"pretrained_model_path": "",
"results_dir": "",
"resume_training_checkpoint_path": "",
"seed": 1234,
"tile_min_overlap": [
16,
16
],
"tile_wtype": "gaussian",
"validation_interval": 1,
"vis_step_interval": 10
},
"wandb": {
"enable": true,
"entity": "",
"group": "",
"name": "TAO Toolkit Training",
"project": "TAO Toolkit",
"reinit": false,
"run_id": "",
"save_code": false,
"sync_tensorboard": false,
"tags": [
"tao-toolkit"
]
}
},
"popular": {
"evaluate": {
"gpu_ids": [
0
],
"num_gpus": 1,
"num_nodes": 1
},
"gen_trt_engine": {
"batch_size": -1,
"gpu_id": 0,
"tensorrt": {
"max_batch_size": 1,
"min_batch_size": 1,
"opt_batch_size": 1
}
},
"inference": {
"gpu_ids": [
0
],
"num_gpus": 1,
"num_nodes": 1
},
"train": {
"checkpoint_interval": 1,
"gpu_ids": [
0
],
"num_epochs": 10,
"num_gpus": 1,
"num_nodes": 1,
"validation_interval": 1
}
},
"properties": {
"automl_disabled_parameters": [
"wandb",
"dataset",
"model",
"inference",
"evaluate",
"train",
"export",
"gen_trt_engine",
"quantize"
],
"dataset": {
"automl_disabled_parameters": [
"dataset.train_dataset",
"dataset.val_dataset",
"dataset.test_dataset",
"dataset.infer_dataset",
"dataset.quant_calibration_dataset"
],
"automl_enabled": false,
"default": {
"baseline": 0.193001,
"dataset_name": "StereoDataset",
"focal_x": 1998.842,
"infer_dataset": {
"augmentation": {
"color_aug_brightness": 0.4,
"color_aug_contrast": 0.4,
"color_aug_hue_range": [
-0.027777777777777776,
0.027777777777777776
],
"color_aug_prob": 0.2,
"color_aug_saturation": [
0.0,
1.4
],
"crop_min_valid_disp_ratio": 0.0,
"crop_size": [
518,
518
],
"do_flip": false,
"eraser_aug_prob": 0.5,
"gamma": [
1,
1,
1,
1
],
"h_flip_prob": 0.5,
"hshift_prob": 0.5,
"input_mean": [
0.485,
0.456,
0.406
],
"input_std": [
0.229,
0.224,
0.225
],
"max_scale": 0.4,
"max_stretch": 0.2,
"min_scale": -0.2,
"spatial_aug_prob": 1.0,
"stretch_prob": 0.8,
"v_flip_prob": 0.5,
"yjitter_prob": 1.0
},
"batch_size": 1,
"data_sources": [
{
"data_file": "",
"dataset_name": ""
}
],
"pin_memory": true,
"workers": 8
},
"max_disparity": 416,
"normalize_depth": false,
"quant_calibration_dataset": {
"images_dir": ""
},
"test_dataset": {
"augmentation": {
"color_aug_brightness": 0.4,
"color_aug_contrast": 0.4,
"color_aug_hue_range": [
-0.027777777777777776,
0.027777777777777776
],
"color_aug_prob": 0.2,
"color_aug_saturation": [
0.0,
1.4
],
"crop_min_valid_disp_ratio": 0.0,
"crop_size": [
518,
518
],
"do_flip": false,
"eraser_aug_prob": 0.5,
"gamma": [
1,
1,
1,
1
],
"h_flip_prob": 0.5,
"hshift_prob": 0.5,
"input_mean": [
0.485,
0.456,
0.406
],
"input_std": [
0.229,
0.224,
0.225
],
"max_scale": 0.4,
"max_stretch": 0.2,
"min_scale": -0.2,
"spatial_aug_prob": 1.0,
"stretch_prob": 0.8,
"v_flip_prob": 0.5,
"yjitter_prob": 1.0
},
"batch_size": 1,
"data_sources": [
{
"data_file": "",
"dataset_name": ""
}
],
"pin_memory": true,
"workers": 8
},
"train_dataset": {
"augmentation": {
"color_aug_brightness": 0.4,
"color_aug_contrast": 0.4,
"color_aug_hue_range": [
-0.027777777777777776,
0.027777777777777776
],
"color_aug_prob": 0.2,
"color_aug_saturation": [
0.0,
1.4
],
"crop_min_valid_disp_ratio": 0.0,
"crop_size": [
518,
518
],
"do_flip": false,
"eraser_aug_prob": 0.5,
"gamma": [
1,
1,
1,
1
],
"h_flip_prob": 0.5,
"hshift_prob": 0.5,
"input_mean": [
0.485,
0.456,
0.406
],
"input_std": [
0.229,
0.224,
0.225
],
"max_scale": 0.4,
"max_stretch": 0.2,
"min_scale": -0.2,
"spatial_aug_prob": 1.0,
"stretch_prob": 0.8,
"v_flip_prob": 0.5,
"yjitter_prob": 1.0
},
"batch_size": 1,
"data_sources": [
{
"data_file": "",
"dataset_name": ""
}
],
"pin_memory": true,
"workers": 8
},
"val_dataset": {
"augmentation": {
"color_aug_brightness": 0.4,
"color_aug_contrast": 0.4,
"color_aug_hue_range": [
-0.027777777777777776,
0.027777777777777776
],
"color_aug_prob": 0.2,
"color_aug_saturation": [
0.0,
1.4
],
"crop_min_valid_disp_ratio": 0.0,
"crop_size": [
518,
518
],
"do_flip": false,
"eraser_aug_prob": 0.5,
"gamma": [
1,
1,
1,
1
],
"h_flip_prob": 0.5,
"hshift_prob": 0.5,
"input_mean": [
0.485,
0.456,
0.406
],
"input_std": [
0.229,
0.224,
0.225
],
"max_scale": 0.4,
"max_stretch": 0.2,
"min_scale": -0.2,
"spatial_aug_prob": 1.0,
"stretch_prob": 0.8,
"v_flip_prob": 0.5,
"yjitter_prob": 1.0
},
"batch_size": 1,
"data_sources": [
{
"data_file": "",
"dataset_name": ""
}
],
"pin_memory": true,
"workers": 8
}
},
"description": "Configurable parameters to construct the dataset for a DepthNet experiment.",
"properties": {
"baseline": {
"default": 0.193001,
"description": "The baseline for stereo datasets",
"maximum": Infinity,
"minimum": 0.0,
"title": "Stereo baseline",
"type": "float"
},
"dataset_name": {
"default": "StereoDataset",
"description": "Dataset Name",
"enum": [
"MonoDataset",
"StereoDataset"
],
"title": "dataset mame",
"type": "categorical"
},
"focal_x": {
"default": 1998.842,
"description": "The focal length along x-axis",
"maximum": Infinity,
"minimum": 0.0,
"title": "The focal length along x-axis",
"type": "float"
},
"infer_dataset": {
"automl_disabled_parameters": [
"dataset.infer_dataset.data_sources",
"dataset.infer_dataset.augmentation"
],
"automl_enabled": false,
"default": {
"augmentation": {
"color_aug_brightness": 0.4,
"color_aug_contrast": 0.4,
"color_aug_hue_range": [
-0.027777777777777776,
0.027777777777777776
],
"color_aug_prob": 0.2,
"color_aug_saturation": [
0.0,
1.4
],
"crop_min_valid_disp_ratio": 0.0,
"crop_size": [
518,
518
],
"do_flip": false,
"eraser_aug_prob": 0.5,
"gamma": [
1,
1,
1,
1
],
"h_flip_prob": 0.5,
"hshift_prob": 0.5,
"input_mean": [
0.485,
0.456,
0.406
],
"input_std": [
0.229,
0.224,
0.225
],
"max_scale": 0.4,
"max_stretch": 0.2,
"min_scale": -0.2,
"spatial_aug_prob": 1.0,
"stretch_prob": 0.8,
"v_flip_prob": 0.5,
"yjitter_prob": 1.0
},
"batch_size": 1,
"data_sources": [
{
"data_file": "",
"dataset_name": ""
}
],
"pin_memory": true,
"workers": 8
},
"description": "Configurable parameters to construct the infer dataset for a DepthNet experiment.",
"properties": {
"augmentation": {
"automl_default_parameters": [
"dataset.infer_dataset.augmentation.yjitter_prob",
"dataset.infer_dataset.augmentation.color_aug_prob",
"dataset.infer_dataset.augmentation.eraser_aug_prob",
"dataset.infer_dataset.augmentation.spatial_aug_prob",
"dataset.infer_dataset.augmentation.stretch_prob",
"dataset.infer_dataset.augmentation.h_flip_prob",
"dataset.infer_dataset.augmentation.v_flip_prob",
"dataset.infer_dataset.augmentation.hshift_prob",
"dataset.infer_dataset.augmentation.crop_min_valid_disp_ratio"
],
"automl_disabled_parameters": [
"dataset.infer_dataset.augmentation.input_mean",
"dataset.infer_dataset.augmentation.input_std",
"dataset.infer_dataset.augmentation.crop_size",
"dataset.infer_dataset.augmentation.gamma",
"dataset.infer_dataset.augmentation.color_aug_saturation",
"dataset.infer_dataset.augmentation.color_aug_hue_range"
],
"automl_enabled": false,
"default": {
"color_aug_brightness": 0.4,
"color_aug_contrast": 0.4,
"color_aug_hue_range": [
-0.027777777777777776,
0.027777777777777776
],
"color_aug_prob": 0.2,
"color_aug_saturation": [
0.0,
1.4
],
"crop_min_valid_disp_ratio": 0.0,
"crop_size": [
518,
518
],
"do_flip": false,
"eraser_aug_prob": 0.5,
"gamma": [
1,
1,
1,
1
],
"h_flip_prob": 0.5,
"hshift_prob": 0.5,
"input_mean": [
0.485,
0.456,
0.406
],
"input_std": [
0.229,
0.224,
0.225
],
"max_scale": 0.4,
"max_stretch": 0.2,
"min_scale": -0.2,
"spatial_aug_prob": 1.0,
"stretch_prob": 0.8,
"v_flip_prob": 0.5,
"yjitter_prob": 1.0
},
"description": "Configuration parameters for data augmentation",
"properties": {
"color_aug_brightness": {
"default": 0.4,
"description": "The color jitter brightness",
"maximum": 1.0,
"minimum": 0.0,
"title": "The color jitter brightness",
"type": "float"
},
"color_aug_contrast": {
"default": 0.4,
"description": "The color jitter contrast",
"maximum": 1.0,
"minimum": 0.0,
"title": "The color jitter contrast",
"type": "float"
},
"color_aug_hue_range": {
"automl_enabled": false,
"default": [
-0.027777777777777776,
0.027777777777777776
],
"description": "The hue range in data augmentation",
"title": "hue range augmentaiton",
"type": "list"
},
"color_aug_prob": {
"automl_enabled": true,
"default": 0.2,
"description": "The probability for asymmetric color augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for asymmetric color augmentation",
"type": "float"
},
"color_aug_saturation": {
"automl_enabled": false,
"default": [
0.0,
1.4
],
"description": "The color jitter saturation",
"title": "The color jitter saturation",
"type": "list"
},
"crop_min_valid_disp_ratio": {
"automl_enabled": true,
"default": 0.0,
"description": "The probability for minimum crop valid disparity ratio",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for min crop valid disparity ratio",
"type": "float"
},
"crop_size": {
"automl_enabled": false,
"default": [
518,
518
],
"description": "The crop size for input RGB images [height, width]",
"title": "augmentation crop size",
"type": "list"
},
"do_flip": {
"default": false,
"description": "A flag specifying whether to perform flip in data augmentation",
"title": "do flip",
"type": "bool"
},
"eraser_aug_prob": {
"automl_enabled": true,
"default": 0.5,
"description": "The probability for eraser augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for eraser augmentation",
"type": "float"
},
"gamma": {
"automl_enabled": false,
"default": [
1,
1,
1,
1
],
"description": "Gamma range in data augmentation",
"title": "gamma range",
"type": "list"
},
"h_flip_prob": {
"automl_enabled": true,
"default": 0.5,
"description": "The probability for horizontal flip augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for horizontal flip augmentation",
"type": "float"
},
"hshift_prob": {
"automl_enabled": true,
"default": 0.5,
"description": "The probability for horizontal shift augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for horizontal flip augmentation",
"type": "float"
},
"input_mean": {
"automl_enabled": false,
"default": [
0.485,
0.456,
0.406
],
"description": "The input mean for RGB frames",
"title": "input mean per pixel",
"type": "list"
},
"input_std": {
"automl_enabled": false,
"default": [
0.229,
0.224,
0.225
],
"description": "The input standard deviation per pixel for RGB frames",
"title": "input std per pixel",
"type": "list"
},
"max_scale": {
"default": 0.4,
"description": "The maximum scale in data augmentation",
"maximum": 1.0,
"minimum": -0.2,
"title": "max scale",
"type": "float"
},
"max_stretch": {
"default": 0.2,
"description": "The maximum stretch augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The maximum stretch augmentation",
"type": "float"
},
"min_scale": {
"default": -0.2,
"description": "The minimum scale in data augmentation",
"maximum": 1.0,
"minimum": 0.2,
"title": "min scale",
"type": "float"
},
"spatial_aug_prob": {
"automl_enabled": true,
"default": 1.0,
"description": "The probability for spatial augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for spatial augmentation",
"type": "float"
},
"stretch_prob": {
"automl_enabled": true,
"default": 0.8,
"description": "The probability for stretch augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for stretch augmentation",
"type": "float"
},
"v_flip_prob": {
"automl_enabled": true,
"default": 0.5,
"description": "The probability for vertical flip augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for vertical flip augmentation",
"type": "float"
},
"yjitter_prob": {
"automl_enabled": true,
"default": 1.0,
"description": "The probability for y jitter",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for y jitter",
"type": "float"
}
},
"title": "augmentation",
"type": "collection"
},
"batch_size": {
"default": 1,
"description": "The batch size for training and validation",
"maximum": Infinity,
"minimum": 1,
"title": "batch size",
"type": "int"
},
"data_sources": {
"automl_enabled": false,
"default": [
{
"data_file": "",
"dataset_name": ""
}
],
"description": "The list of data sources for training:\n * dataset_name : The type of the dataset\n * data_file : The path of the data file",
"title": "train data sources",
"type": "list"
},
"pin_memory": {
"default": true,
"description": "Flag to enable the dataloader to allocated pagelocked memory for faster\n of data between the CPU and GPU.",
"title": "pin_memory",
"type": "bool"
},
"workers": {
"default": 8,
"description": "The number of parallel workers processing data",
"maximum": Infinity,
"minimum": 1,
"title": "batch size",
"type": "int"
}
},
"type": "collection"
},
"max_depth": {
"description": "The maximum depth in meters in MetricDepthAnythingV2",
"maximum": Infinity,
"minimum": 1.0,
"title": "max depth in meters",
"type": "float"
},
"max_disparity": {
"default": 416,
"description": "The maximum allowed disparity for which we compute losses during training",
"maximum": 416,
"minimum": 1,
"title": "maximum dispairty",
"type": "int"
},
"min_depth": {
"description": "The minimum depth in meters in MetricDepthAnythingV2",
"maximum": Infinity,
"minimum": 0.0,
"title": "min depth in meters",
"type": "float"
},
"normalize_depth": {
"default": false,
"description": "Normalize depth",
"title": "normalize depth",
"type": "bool"
},
"quant_calibration_dataset": {
"automl_enabled": false,
"default": {
"images_dir": ""
},
"description": "Configurable parameters for quantization calibration dataset.",
"properties": {
"images_dir": {
"default": "",
"description": "Path to the directory containing calibration images.",
"title": "calibration images directory",
"type": "string"
}
},
"type": "collection"
},
"test_dataset": {
"automl_disabled_parameters": [
"dataset.test_dataset.data_sources",
"dataset.test_dataset.augmentation"
],
"automl_enabled": false,
"default": {
"augmentation": {
"color_aug_brightness": 0.4,
"color_aug_contrast": 0.4,
"color_aug_hue_range": [
-0.027777777777777776,
0.027777777777777776
],
"color_aug_prob": 0.2,
"color_aug_saturation": [
0.0,
1.4
],
"crop_min_valid_disp_ratio": 0.0,
"crop_size": [
518,
518
],
"do_flip": false,
"eraser_aug_prob": 0.5,
"gamma": [
1,
1,
1,
1
],
"h_flip_prob": 0.5,
"hshift_prob": 0.5,
"input_mean": [
0.485,
0.456,
0.406
],
"input_std": [
0.229,
0.224,
0.225
],
"max_scale": 0.4,
"max_stretch": 0.2,
"min_scale": -0.2,
"spatial_aug_prob": 1.0,
"stretch_prob": 0.8,
"v_flip_prob": 0.5,
"yjitter_prob": 1.0
},
"batch_size": 1,
"data_sources": [
{
"data_file": "",
"dataset_name": ""
}
],
"pin_memory": true,
"workers": 8
},
"description": "Configurable parameters to construct the test dataset for a DepthNet experiment.",
"properties": {
"augmentation": {
"automl_default_parameters": [
"dataset.test_dataset.augmentation.yjitter_prob",
"dataset.test_dataset.augmentation.color_aug_prob",
"dataset.test_dataset.augmentation.eraser_aug_prob",
"dataset.test_dataset.augmentation.spatial_aug_prob",
"dataset.test_dataset.augmentation.stretch_prob",
"dataset.test_dataset.augmentation.h_flip_prob",
"dataset.test_dataset.augmentation.v_flip_prob",
"dataset.test_dataset.augmentation.hshift_prob",
"dataset.test_dataset.augmentation.crop_min_valid_disp_ratio"
],
"automl_disabled_parameters": [
"dataset.test_dataset.augmentation.input_mean",
"dataset.test_dataset.augmentation.input_std",
"dataset.test_dataset.augmentation.crop_size",
"dataset.test_dataset.augmentation.gamma",
"dataset.test_dataset.augmentation.color_aug_saturation",
"dataset.test_dataset.augmentation.color_aug_hue_range"
],
"automl_enabled": false,
"default": {
"color_aug_brightness": 0.4,
"color_aug_contrast": 0.4,
"color_aug_hue_range": [
-0.027777777777777776,
0.027777777777777776
],
"color_aug_prob": 0.2,
"color_aug_saturation": [
0.0,
1.4
],
"crop_min_valid_disp_ratio": 0.0,
"crop_size": [
518,
518
],
"do_flip": false,
"eraser_aug_prob": 0.5,
"gamma": [
1,
1,
1,
1
],
"h_flip_prob": 0.5,
"hshift_prob": 0.5,
"input_mean": [
0.485,
0.456,
0.406
],
"input_std": [
0.229,
0.224,
0.225
],
"max_scale": 0.4,
"max_stretch": 0.2,
"min_scale": -0.2,
"spatial_aug_prob": 1.0,
"stretch_prob": 0.8,
"v_flip_prob": 0.5,
"yjitter_prob": 1.0
},
"description": "Configuration parameters for data augmentation",
"properties": {
"color_aug_brightness": {
"default": 0.4,
"description": "The color jitter brightness",
"maximum": 1.0,
"minimum": 0.0,
"title": "The color jitter brightness",
"type": "float"
},
"color_aug_contrast": {
"default": 0.4,
"description": "The color jitter contrast",
"maximum": 1.0,
"minimum": 0.0,
"title": "The color jitter contrast",
"type": "float"
},
"color_aug_hue_range": {
"automl_enabled": false,
"default": [
-0.027777777777777776,
0.027777777777777776
],
"description": "The hue range in data augmentation",
"title": "hue range augmentaiton",
"type": "list"
},
"color_aug_prob": {
"automl_enabled": true,
"default": 0.2,
"description": "The probability for asymmetric color augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for asymmetric color augmentation",
"type": "float"
},
"color_aug_saturation": {
"automl_enabled": false,
"default": [
0.0,
1.4
],
"description": "The color jitter saturation",
"title": "The color jitter saturation",
"type": "list"
},
"crop_min_valid_disp_ratio": {
"automl_enabled": true,
"default": 0.0,
"description": "The probability for minimum crop valid disparity ratio",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for min crop valid disparity ratio",
"type": "float"
},
"crop_size": {
"automl_enabled": false,
"default": [
518,
518
],
"description": "The crop size for input RGB images [height, width]",
"title": "augmentation crop size",
"type": "list"
},
"do_flip": {
"default": false,
"description": "A flag specifying whether to perform flip in data augmentation",
"title": "do flip",
"type": "bool"
},
"eraser_aug_prob": {
"automl_enabled": true,
"default": 0.5,
"description": "The probability for eraser augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for eraser augmentation",
"type": "float"
},
"gamma": {
"automl_enabled": false,
"default": [
1,
1,
1,
1
],
"description": "Gamma range in data augmentation",
"title": "gamma range",
"type": "list"
},
"h_flip_prob": {
"automl_enabled": true,
"default": 0.5,
"description": "The probability for horizontal flip augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for horizontal flip augmentation",
"type": "float"
},
"hshift_prob": {
"automl_enabled": true,
"default": 0.5,
"description": "The probability for horizontal shift augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for horizontal flip augmentation",
"type": "float"
},
"input_mean": {
"automl_enabled": false,
"default": [
0.485,
0.456,
0.406
],
"description": "The input mean for RGB frames",
"title": "input mean per pixel",
"type": "list"
},
"input_std": {
"automl_enabled": false,
"default": [
0.229,
0.224,
0.225
],
"description": "The input standard deviation per pixel for RGB frames",
"title": "input std per pixel",
"type": "list"
},
"max_scale": {
"default": 0.4,
"description": "The maximum scale in data augmentation",
"maximum": 1.0,
"minimum": -0.2,
"title": "max scale",
"type": "float"
},
"max_stretch": {
"default": 0.2,
"description": "The maximum stretch augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The maximum stretch augmentation",
"type": "float"
},
"min_scale": {
"default": -0.2,
"description": "The minimum scale in data augmentation",
"maximum": 1.0,
"minimum": 0.2,
"title": "min scale",
"type": "float"
},
"spatial_aug_prob": {
"automl_enabled": true,
"default": 1.0,
"description": "The probability for spatial augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for spatial augmentation",
"type": "float"
},
"stretch_prob": {
"automl_enabled": true,
"default": 0.8,
"description": "The probability for stretch augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for stretch augmentation",
"type": "float"
},
"v_flip_prob": {
"automl_enabled": true,
"default": 0.5,
"description": "The probability for vertical flip augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for vertical flip augmentation",
"type": "float"
},
"yjitter_prob": {
"automl_enabled": true,
"default": 1.0,
"description": "The probability for y jitter",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for y jitter",
"type": "float"
}
},
"title": "augmentation",
"type": "collection"
},
"batch_size": {
"default": 1,
"description": "The batch size for training and validation",
"maximum": Infinity,
"minimum": 1,
"title": "batch size",
"type": "int"
},
"data_sources": {
"automl_enabled": false,
"default": [
{
"data_file": "",
"dataset_name": ""
}
],
"description": "The list of data sources for training:\n * dataset_name : The type of the dataset\n * data_file : The path of the data file",
"title": "train data sources",
"type": "list"
},
"pin_memory": {
"default": true,
"description": "Flag to enable the dataloader to allocated pagelocked memory for faster\n of data between the CPU and GPU.",
"title": "pin_memory",
"type": "bool"
},
"workers": {
"default": 8,
"description": "The number of parallel workers processing data",
"maximum": Infinity,
"minimum": 1,
"title": "batch size",
"type": "int"
}
},
"type": "collection"
},
"train_dataset": {
"automl_disabled_parameters": [
"dataset.train_dataset.data_sources",
"dataset.train_dataset.augmentation"
],
"automl_enabled": false,
"default": {
"augmentation": {
"color_aug_brightness": 0.4,
"color_aug_contrast": 0.4,
"color_aug_hue_range": [
-0.027777777777777776,
0.027777777777777776
],
"color_aug_prob": 0.2,
"color_aug_saturation": [
0.0,
1.4
],
"crop_min_valid_disp_ratio": 0.0,
"crop_size": [
518,
518
],
"do_flip": false,
"eraser_aug_prob": 0.5,
"gamma": [
1,
1,
1,
1
],
"h_flip_prob": 0.5,
"hshift_prob": 0.5,
"input_mean": [
0.485,
0.456,
0.406
],
"input_std": [
0.229,
0.224,
0.225
],
"max_scale": 0.4,
"max_stretch": 0.2,
"min_scale": -0.2,
"spatial_aug_prob": 1.0,
"stretch_prob": 0.8,
"v_flip_prob": 0.5,
"yjitter_prob": 1.0
},
"batch_size": 1,
"data_sources": [
{
"data_file": "",
"dataset_name": ""
}
],
"pin_memory": true,
"workers": 8
},
"description": "Configurable parameters to construct the train dataset for a DepthNet experiment.",
"properties": {
"augmentation": {
"automl_default_parameters": [
"dataset.train_dataset.augmentation.yjitter_prob",
"dataset.train_dataset.augmentation.color_aug_prob",
"dataset.train_dataset.augmentation.eraser_aug_prob",
"dataset.train_dataset.augmentation.spatial_aug_prob",
"dataset.train_dataset.augmentation.stretch_prob",
"dataset.train_dataset.augmentation.h_flip_prob",
"dataset.train_dataset.augmentation.v_flip_prob",
"dataset.train_dataset.augmentation.hshift_prob",
"dataset.train_dataset.augmentation.crop_min_valid_disp_ratio"
],
"automl_disabled_parameters": [
"dataset.train_dataset.augmentation.input_mean",
"dataset.train_dataset.augmentation.input_std",
"dataset.train_dataset.augmentation.crop_size",
"dataset.train_dataset.augmentation.gamma",
"dataset.train_dataset.augmentation.color_aug_saturation",
"dataset.train_dataset.augmentation.color_aug_hue_range"
],
"automl_enabled": false,
"default": {
"color_aug_brightness": 0.4,
"color_aug_contrast": 0.4,
"color_aug_hue_range": [
-0.027777777777777776,
0.027777777777777776
],
"color_aug_prob": 0.2,
"color_aug_saturation": [
0.0,
1.4
],
"crop_min_valid_disp_ratio": 0.0,
"crop_size": [
518,
518
],
"do_flip": false,
"eraser_aug_prob": 0.5,
"gamma": [
1,
1,
1,
1
],
"h_flip_prob": 0.5,
"hshift_prob": 0.5,
"input_mean": [
0.485,
0.456,
0.406
],
"input_std": [
0.229,
0.224,
0.225
],
"max_scale": 0.4,
"max_stretch": 0.2,
"min_scale": -0.2,
"spatial_aug_prob": 1.0,
"stretch_prob": 0.8,
"v_flip_prob": 0.5,
"yjitter_prob": 1.0
},
"description": "Configuration parameters for data augmentation",
"properties": {
"color_aug_brightness": {
"default": 0.4,
"description": "The color jitter brightness",
"maximum": 1.0,
"minimum": 0.0,
"title": "The color jitter brightness",
"type": "float"
},
"color_aug_contrast": {
"default": 0.4,
"description": "The color jitter contrast",
"maximum": 1.0,
"minimum": 0.0,
"title": "The color jitter contrast",
"type": "float"
},
"color_aug_hue_range": {
"automl_enabled": false,
"default": [
-0.027777777777777776,
0.027777777777777776
],
"description": "The hue range in data augmentation",
"title": "hue range augmentaiton",
"type": "list"
},
"color_aug_prob": {
"automl_enabled": true,
"default": 0.2,
"description": "The probability for asymmetric color augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for asymmetric color augmentation",
"type": "float"
},
"color_aug_saturation": {
"automl_enabled": false,
"default": [
0.0,
1.4
],
"description": "The color jitter saturation",
"title": "The color jitter saturation",
"type": "list"
},
"crop_min_valid_disp_ratio": {
"automl_enabled": true,
"default": 0.0,
"description": "The probability for minimum crop valid disparity ratio",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for min crop valid disparity ratio",
"type": "float"
},
"crop_size": {
"automl_enabled": false,
"default": [
518,
518
],
"description": "The crop size for input RGB images [height, width]",
"title": "augmentation crop size",
"type": "list"
},
"do_flip": {
"default": false,
"description": "A flag specifying whether to perform flip in data augmentation",
"title": "do flip",
"type": "bool"
},
"eraser_aug_prob": {
"automl_enabled": true,
"default": 0.5,
"description": "The probability for eraser augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for eraser augmentation",
"type": "float"
},
"gamma": {
"automl_enabled": false,
"default": [
1,
1,
1,
1
],
"description": "Gamma range in data augmentation",
"title": "gamma range",
"type": "list"
},
"h_flip_prob": {
"automl_enabled": true,
"default": 0.5,
"description": "The probability for horizontal flip augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for horizontal flip augmentation",
"type": "float"
},
"hshift_prob": {
"automl_enabled": true,
"default": 0.5,
"description": "The probability for horizontal shift augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for horizontal flip augmentation",
"type": "float"
},
"input_mean": {
"automl_enabled": false,
"default": [
0.485,
0.456,
0.406
],
"description": "The input mean for RGB frames",
"title": "input mean per pixel",
"type": "list"
},
"input_std": {
"automl_enabled": false,
"default": [
0.229,
0.224,
0.225
],
"description": "The input standard deviation per pixel for RGB frames",
"title": "input std per pixel",
"type": "list"
},
"max_scale": {
"default": 0.4,
"description": "The maximum scale in data augmentation",
"maximum": 1.0,
"minimum": -0.2,
"title": "max scale",
"type": "float"
},
"max_stretch": {
"default": 0.2,
"description": "The maximum stretch augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The maximum stretch augmentation",
"type": "float"
},
"min_scale": {
"default": -0.2,
"description": "The minimum scale in data augmentation",
"maximum": 1.0,
"minimum": 0.2,
"title": "min scale",
"type": "float"
},
"spatial_aug_prob": {
"automl_enabled": true,
"default": 1.0,
"description": "The probability for spatial augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for spatial augmentation",
"type": "float"
},
"stretch_prob": {
"automl_enabled": true,
"default": 0.8,
"description": "The probability for stretch augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for stretch augmentation",
"type": "float"
},
"v_flip_prob": {
"automl_enabled": true,
"default": 0.5,
"description": "The probability for vertical flip augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for vertical flip augmentation",
"type": "float"
},
"yjitter_prob": {
"automl_enabled": true,
"default": 1.0,
"description": "The probability for y jitter",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for y jitter",
"type": "float"
}
},
"title": "augmentation",
"type": "collection"
},
"batch_size": {
"default": 1,
"description": "The batch size for training and validation",
"maximum": Infinity,
"minimum": 1,
"title": "batch size",
"type": "int"
},
"data_sources": {
"automl_enabled": false,
"default": [
{
"data_file": "",
"dataset_name": ""
}
],
"description": "The list of data sources for training:\n * dataset_name : The type of the dataset\n * data_file : The path of the data file",
"title": "train data sources",
"type": "list"
},
"pin_memory": {
"default": true,
"description": "Flag to enable the dataloader to allocated pagelocked memory for faster\n of data between the CPU and GPU.",
"title": "pin_memory",
"type": "bool"
},
"workers": {
"default": 8,
"description": "The number of parallel workers processing data",
"maximum": Infinity,
"minimum": 1,
"title": "batch size",
"type": "int"
}
},
"type": "collection"
},
"val_dataset": {
"automl_disabled_parameters": [
"dataset.val_dataset.data_sources",
"dataset.val_dataset.augmentation"
],
"automl_enabled": false,
"default": {
"augmentation": {
"color_aug_brightness": 0.4,
"color_aug_contrast": 0.4,
"color_aug_hue_range": [
-0.027777777777777776,
0.027777777777777776
],
"color_aug_prob": 0.2,
"color_aug_saturation": [
0.0,
1.4
],
"crop_min_valid_disp_ratio": 0.0,
"crop_size": [
518,
518
],
"do_flip": false,
"eraser_aug_prob": 0.5,
"gamma": [
1,
1,
1,
1
],
"h_flip_prob": 0.5,
"hshift_prob": 0.5,
"input_mean": [
0.485,
0.456,
0.406
],
"input_std": [
0.229,
0.224,
0.225
],
"max_scale": 0.4,
"max_stretch": 0.2,
"min_scale": -0.2,
"spatial_aug_prob": 1.0,
"stretch_prob": 0.8,
"v_flip_prob": 0.5,
"yjitter_prob": 1.0
},
"batch_size": 1,
"data_sources": [
{
"data_file": "",
"dataset_name": ""
}
],
"pin_memory": true,
"workers": 8
},
"description": "Configurable parameters to construct the val dataset for a DepthNet experiment.",
"properties": {
"augmentation": {
"automl_default_parameters": [
"dataset.val_dataset.augmentation.yjitter_prob",
"dataset.val_dataset.augmentation.color_aug_prob",
"dataset.val_dataset.augmentation.eraser_aug_prob",
"dataset.val_dataset.augmentation.spatial_aug_prob",
"dataset.val_dataset.augmentation.stretch_prob",
"dataset.val_dataset.augmentation.h_flip_prob",
"dataset.val_dataset.augmentation.v_flip_prob",
"dataset.val_dataset.augmentation.hshift_prob",
"dataset.val_dataset.augmentation.crop_min_valid_disp_ratio"
],
"automl_disabled_parameters": [
"dataset.val_dataset.augmentation.input_mean",
"dataset.val_dataset.augmentation.input_std",
"dataset.val_dataset.augmentation.crop_size",
"dataset.val_dataset.augmentation.gamma",
"dataset.val_dataset.augmentation.color_aug_saturation",
"dataset.val_dataset.augmentation.color_aug_hue_range"
],
"automl_enabled": false,
"default": {
"color_aug_brightness": 0.4,
"color_aug_contrast": 0.4,
"color_aug_hue_range": [
-0.027777777777777776,
0.027777777777777776
],
"color_aug_prob": 0.2,
"color_aug_saturation": [
0.0,
1.4
],
"crop_min_valid_disp_ratio": 0.0,
"crop_size": [
518,
518
],
"do_flip": false,
"eraser_aug_prob": 0.5,
"gamma": [
1,
1,
1,
1
],
"h_flip_prob": 0.5,
"hshift_prob": 0.5,
"input_mean": [
0.485,
0.456,
0.406
],
"input_std": [
0.229,
0.224,
0.225
],
"max_scale": 0.4,
"max_stretch": 0.2,
"min_scale": -0.2,
"spatial_aug_prob": 1.0,
"stretch_prob": 0.8,
"v_flip_prob": 0.5,
"yjitter_prob": 1.0
},
"description": "Configuration parameters for data augmentation",
"properties": {
"color_aug_brightness": {
"default": 0.4,
"description": "The color jitter brightness",
"maximum": 1.0,
"minimum": 0.0,
"title": "The color jitter brightness",
"type": "float"
},
"color_aug_contrast": {
"default": 0.4,
"description": "The color jitter contrast",
"maximum": 1.0,
"minimum": 0.0,
"title": "The color jitter contrast",
"type": "float"
},
"color_aug_hue_range": {
"automl_enabled": false,
"default": [
-0.027777777777777776,
0.027777777777777776
],
"description": "The hue range in data augmentation",
"title": "hue range augmentaiton",
"type": "list"
},
"color_aug_prob": {
"automl_enabled": true,
"default": 0.2,
"description": "The probability for asymmetric color augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for asymmetric color augmentation",
"type": "float"
},
"color_aug_saturation": {
"automl_enabled": false,
"default": [
0.0,
1.4
],
"description": "The color jitter saturation",
"title": "The color jitter saturation",
"type": "list"
},
"crop_min_valid_disp_ratio": {
"automl_enabled": true,
"default": 0.0,
"description": "The probability for minimum crop valid disparity ratio",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for min crop valid disparity ratio",
"type": "float"
},
"crop_size": {
"automl_enabled": false,
"default": [
518,
518
],
"description": "The crop size for input RGB images [height, width]",
"title": "augmentation crop size",
"type": "list"
},
"do_flip": {
"default": false,
"description": "A flag specifying whether to perform flip in data augmentation",
"title": "do flip",
"type": "bool"
},
"eraser_aug_prob": {
"automl_enabled": true,
"default": 0.5,
"description": "The probability for eraser augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for eraser augmentation",
"type": "float"
},
"gamma": {
"automl_enabled": false,
"default": [
1,
1,
1,
1
],
"description": "Gamma range in data augmentation",
"title": "gamma range",
"type": "list"
},
"h_flip_prob": {
"automl_enabled": true,
"default": 0.5,
"description": "The probability for horizontal flip augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for horizontal flip augmentation",
"type": "float"
},
"hshift_prob": {
"automl_enabled": true,
"default": 0.5,
"description": "The probability for horizontal shift augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for horizontal flip augmentation",
"type": "float"
},
"input_mean": {
"automl_enabled": false,
"default": [
0.485,
0.456,
0.406
],
"description": "The input mean for RGB frames",
"title": "input mean per pixel",
"type": "list"
},
"input_std": {
"automl_enabled": false,
"default": [
0.229,
0.224,
0.225
],
"description": "The input standard deviation per pixel for RGB frames",
"title": "input std per pixel",
"type": "list"
},
"max_scale": {
"default": 0.4,
"description": "The maximum scale in data augmentation",
"maximum": 1.0,
"minimum": -0.2,
"title": "max scale",
"type": "float"
},
"max_stretch": {
"default": 0.2,
"description": "The maximum stretch augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The maximum stretch augmentation",
"type": "float"
},
"min_scale": {
"default": -0.2,
"description": "The minimum scale in data augmentation",
"maximum": 1.0,
"minimum": 0.2,
"title": "min scale",
"type": "float"
},
"spatial_aug_prob": {
"automl_enabled": true,
"default": 1.0,
"description": "The probability for spatial augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for spatial augmentation",
"type": "float"
},
"stretch_prob": {
"automl_enabled": true,
"default": 0.8,
"description": "The probability for stretch augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for stretch augmentation",
"type": "float"
},
"v_flip_prob": {
"automl_enabled": true,
"default": 0.5,
"description": "The probability for vertical flip augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for vertical flip augmentation",
"type": "float"
},
"yjitter_prob": {
"automl_enabled": true,
"default": 1.0,
"description": "The probability for y jitter",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for y jitter",
"type": "float"
}
},
"title": "augmentation",
"type": "collection"
},
"batch_size": {
"default": 1,
"description": "The batch size for training and validation",
"maximum": Infinity,
"minimum": 1,
"title": "batch size",
"type": "int"
},
"data_sources": {
"automl_enabled": false,
"default": [
{
"data_file": "",
"dataset_name": ""
}
],
"description": "The list of data sources for training:\n * dataset_name : The type of the dataset\n * data_file : The path of the data file",
"title": "train data sources",
"type": "list"
},
"pin_memory": {
"default": true,
"description": "Flag to enable the dataloader to allocated pagelocked memory for faster\n of data between the CPU and GPU.",
"title": "pin_memory",
"type": "bool"
},
"workers": {
"default": 8,
"description": "The number of parallel workers processing data",
"maximum": Infinity,
"minimum": 1,
"title": "batch size",
"type": "int"
}
},
"type": "collection"
}
},
"type": "collection"
},
"encryption_key": {
"default": "",
"description": "Key for encrypting model checkpoints",
"title": "Encryption key",
"type": "string"
},
"model": {
"automl_default_parameters": [
"model.corr_radius",
"model.cv_group",
"model.volume_dim"
],
"automl_disabled_parameters": [
"model.mono_backbone",
"model.stereo_backbone",
"model.hidden_dims"
],
"automl_enabled": false,
"default": {
"corr_levels": 2,
"corr_radius": 4,
"cv_group": 8,
"encoder": "vitl",
"hidden_dims": [
128,
128,
128
],
"low_memory": 0,
"max_disparity": 416,
"mixed_precision": false,
"model_type": "MetricDepthAnything",
"mono_backbone": {
"pretrained_path": "",
"use_bn": false,
"use_clstoken": false
},
"n_downsample": 2,
"n_gru_layers": 3,
"stereo_backbone": {
"depth_anything_v2_pretrained_path": "",
"edgenext_pretrained_path": "",
"use_bn": false,
"use_clstoken": false
},
"train_iters": 22,
"valid_iters": 22,
"volume_dim": 32
},
"description": "Configurable parameters to construct the model for a DepthNet experiment.",
"properties": {
"corr_levels": {
"default": 2,
"description": "The number of levels in the correlation pyramid",
"maximum": 2,
"minimum": 1,
"title": "number of correlation pyramid levels",
"type": "int"
},
"corr_radius": {
"automl_enabled": true,
"default": 4,
"description": "The width of the correlation pyramid",
"maximum": 8,
"minimum": 2,
"title": "correlation pyramid width",
"type": "int"
},
"cv_group": {
"automl_enabled": true,
"default": 8,
"description": "cv group",
"maximum": 16,
"minimum": 4,
"title": "cv group",
"type": "int"
},
"encoder": {
"default": "vitl",
"description": "DepthAnythingV2 Encoder options",
"enum": [
"vits",
"vitb",
"vitl",
"vitg"
],
"type": "categorical"
},
"hidden_dims": {
"automl_enabled": false,
"default": [
128,
128,
128
],
"description": "The hidden dimensions.",
"title": "The hidden dimensions.",
"type": "list"
},
"low_memory": {
"default": 0,
"description": "reduce memory usage",
"maximum": 4,
"minimum": 0,
"title": "reduce memory usage",
"type": "int"
},
"max_disparity": {
"default": 416,
"description": "\n The maximum disparity of the model used in the training of a stereo model\n ",
"title": "max disparity",
"type": "int"
},
"mixed_precision": {
"default": false,
"description": "A flag specifying whether to use mixed precision training",
"title": "Mixed Precision Training",
"type": "bool"
},
"model_type": {
"default": "MetricDepthAnything",
"description": "Network name",
"enum": [
"FoundationStereo",
"MetricDepthAnything",
"RelativeDepthAnything"
],
"type": "categorical"
},
"mono_backbone": {
"automl_enabled": false,
"default": {
"pretrained_path": "",
"use_bn": false,
"use_clstoken": false
},
"description": "Network defined paths for Monocular DepthNet Backbone",
"properties": {
"pretrained_path": {
"default": "",
"description": "Path to load depth anything v2 as an encoder for Monocular DepthNet",
"title": "Pretrained path for mono backbone",
"type": "string"
},
"use_bn": {
"default": false,
"description": "A flag specifying whether to use batch normalization in Monocular DepthNet",
"title": "Batch normalization in Monocular DepthNet",
"type": "bool"
},
"use_clstoken": {
"default": false,
"description": "A flag specifying whether to use class token",
"title": "Class token in Monocular DepthNet",
"type": "bool"
}
},
"title": "Mono backbone configuration",
"type": "collection"
},
"n_downsample": {
"default": 2,
"description": "resolution of the disparity field (1/2^K)",
"maximum": 2,
"minimum": 1,
"title": "disparity field resoultion",
"type": "int"
},
"n_gru_layers": {
"default": 3,
"description": "The number of hidden GRU levels",
"maximum": 3,
"minimum": 1,
"title": "number of hidden GRU levels",
"type": "int"
},
"stereo_backbone": {
"automl_enabled": false,
"default": {
"depth_anything_v2_pretrained_path": "",
"edgenext_pretrained_path": "",
"use_bn": false,
"use_clstoken": false
},
"description": "Network defined paths for Edgenext and Depthanythingv2",
"properties": {
"depth_anything_v2_pretrained_path": {
"default": "",
"description": "Path to load depth anything v2 as an encoder for Stereo DepthNet (FoundationStereo)",
"type": "string"
},
"edgenext_pretrained_path": {
"default": "",
"description": "Path to load edgenext encoder for Stereo DepthNet (FoundationStereo)",
"type": "string"
},
"use_bn": {
"default": false,
"description": "A flag specifying whether to use batch normalization in DepthAnythingV2",
"title": "batch normalization in DepthAnythingV2",
"type": "bool"
},
"use_clstoken": {
"default": false,
"description": "A flag specifying whether to use class token",
"title": "class token in DepthAnythingV2",
"type": "bool"
}
},
"title": "Stereo backbone configuration",
"type": "collection"
},
"train_iters": {
"default": 22,
"description": "Train Iteration",
"minimum": 1,
"title": "train iteration",
"type": "int"
},
"valid_iters": {
"default": 22,
"description": "Validation Iteration",
"minimum": 1,
"title": "Validation iteration",
"type": "int"
},
"volume_dim": {
"automl_enabled": true,
"default": 32,
"description": "Volume dimension",
"maximum": 64,
"minimum": 16,
"title": "volume dimension",
"type": "int"
}
},
"type": "collection"
},
"model_name": {
"default": "",
"description": "Name of model if invoking task via :code:`model_agnostic`",
"title": "Model name",
"type": "string"
},
"quantize": {
"automl_disabled_parameters": [
"quantize.layers",
"quantize.skip_names",
"quantize.backend_kwargs"
],
"automl_enabled": false,
"default": {
"algorithm": "minmax",
"backend": "torchao",
"backend_kwargs": {},
"device": "cuda",
"layers": [],
"mode": "weight_only_ptq",
"model_path": "",
"results_dir": "",
"skip_names": []
},
"description": "Configurable parameters for model quantization.",
"properties": {
"algorithm": {
"default": "minmax",
"description": "Calibration/optimization algorithm. Used by ModelOpt backends (modelopt.pytorch and modelopt.onnx). Ignored by torchao backend.",
"enum": [
"minmax",
"max",
"entropy",
"awq_clip",
"awq_lite",
"awq_full",
"rtn_dq"
],
"title": "Calibration algorithm",
"type": "categorical"
},
"backend": {
"default": "torchao",
"description": "The quantization backend to use",
"enum": [
"modelopt.pytorch",
"torchao",
"modelopt.onnx"
],
"title": "Quantization backend",
"type": "categorical"
},
"backend_kwargs": {
"automl_enabled": false,
"description": "Additional keyword arguments to pass to the backend",
"title": "Backend kwargs",
"type": "collection"
},
"device": {
"default": "cuda",
"description": "Device to use for calibration. Accepts 'cuda' (uses default GPU), 'cpu', 'trt' (TensorRT for ONNX backend), or specific GPU device like 'cuda:0', 'cuda:1', etc. If 'cuda' is specified but no GPU is available, the framework will automatically fall back to 'cpu'. Note: 'trt' is only supported by the modelopt.onnx backend for ONNX Runtime with TensorRT execution provider.",
"pattern": "^(cuda|cpu|trt|cuda:[0-9]+)$",
"title": "Calibration device",
"type": "string"
},
"layers": {
"automl_enabled": false,
"default": [],
"description": "List of per-module quantization configurations. Each entry specifies which modules to quantize and their data types. This is the primary way to configure quantization.",
"title": "Layer quantization configs",
"type": "list"
},
"mode": {
"default": "weight_only_ptq",
"description": "The quantization mode to use",
"enum": [
"static_ptq",
"weight_only_ptq"
],
"title": "Quantization mode",
"type": "categorical"
},
"model_path": {
"default": "",
"description": "Path to the model to be quantized. For ONNX backend, path to ONNX file.",
"title": "Model path",
"type": "string"
},
"results_dir": {
"default": "",
"description": "Path to where all the assets generated from a task are stored.",
"title": "Results directory",
"type": "string"
},
"skip_names": {
"automl_enabled": false,
"default": [],
"description": "List of module/layer names or patterns to exclude from quantization",
"title": "Skip names",
"type": "list"
}
},
"required": [
"model_path",
"results_dir"
],
"type": "collection"
},
"results_dir": {
"default": "",
"description": "\n Path to where all the assets generated from a task are stored.\n ",
"title": "Results directory",
"type": "string"
},
"train": {
"automl_disabled_parameters": [
"train.gpu_ids",
"train.cudnn",
"train.optim",
"train.tile_min_overlap"
],
"automl_enabled": false,
"default": {
"activation_checkpoint": false,
"checkpoint_interval": 1,
"checkpoint_interval_unit": "epoch",
"clip_grad_norm": 0.1,
"cudnn": {
"benchmark": false,
"deterministic": true
},
"dataloader_visualize": false,
"distributed_strategy": "ddp",
"gpu_ids": [
0
],
"inference_tile": false,
"is_dry_run": false,
"log_every_n_steps": 500,
"num_epochs": 10,
"num_gpus": 1,
"num_nodes": 1,
"optim": {
"lr": 0.0001,
"lr_decay": 0.1,
"lr_scheduler": "MultiStepLR",
"lr_step_size": 1000,
"lr_steps": [
1000
],
"min_lr": 1e-07,
"momentum": 0.9,
"monitor_name": "val_loss",
"optimizer": "AdamW",
"warmup_steps": 20,
"weight_decay": 0.0001
},
"precision": "fp32",
"pretrained_model_path": "",
"results_dir": "",
"resume_training_checkpoint_path": "",
"seed": 1234,
"tile_min_overlap": [
16,
16
],
"tile_wtype": "gaussian",
"validation_interval": 1,
"vis_step_interval": 10
},
"description": "Configurable parameters to construct the trainer for a DepthNet experiment.",
"popular": [
"validation_interval",
"checkpoint_interval",
"num_gpus",
"num_nodes",
"num_epochs",
"gpu_ids"
],
"properties": {
"activation_checkpoint": {
"default": false,
"description": "\n A True value instructs train to recompute in backward pass to save GPU memory,\n rather than storing activations.",
"title": "enable activation checkpointing",
"type": "bool"
},
"checkpoint_interval": {
"default": 1,
"description": "The interval (in epochs) at which a checkpoint will be saved. Helps resume training.",
"minimum": 1,
"popular": true,
"title": "Checkpoint interval",
"type": "int"
},
"checkpoint_interval_steps": {
"description": "The number of steps to save the checkpoint.",
"title": "checkpoint interval steps",
"type": "int"
},
"checkpoint_interval_unit": {
"default": "epoch",
"description": "The unit of the checkpoint interval.",
"enum": [
"epoch",
"step"
],
"title": "Checkpoint interval unit",
"type": "categorical"
},
"clip_grad_norm": {
"default": 0.1,
"description": "\n Amount to clip the gradient by L2 Norm.\n A value of 0.0 specifies no clipping.",
"math_cond": "> 0.0",
"title": "clip gradient norm",
"type": "float"
},
"cudnn": {
"automl_enabled": false,
"default": {
"benchmark": false,
"deterministic": true
},
"properties": {
"benchmark": {
"default": false,
"type": "bool"
},
"deterministic": {
"default": true,
"type": "bool"
}
},
"type": "collection"
},
"dataloader_visualize": {
"default": false,
"description": "Whether to visualize the dataloader.",
"title": "dataloader visualize",
"type": "bool"
},
"distributed_strategy": {
"default": "ddp",
"description": "\n The multi-GPU training strategy.\n DDP (Distributed Data Parallel) and Fully Sharded DDP are supported.",
"enum": [
"ddp",
"fsdp"
],
"title": "distributed_strategy",
"type": "categorical"
},
"gpu_ids": {
"automl_enabled": false,
"default": [
0
],
"description": "\n List of GPU IDs to run the training on. The length of this list\n must be equal to the number of gpus in train.num_gpus.",
"popular": true,
"title": "GPU IDs",
"type": "list"
},
"inference_tile": {
"default": false,
"description": "Use tiled inference, particularly for transformers\n which expect fixed size of sequences.\n ",
"title": "tile inference",
"type": "bool"
},
"is_dry_run": {
"default": false,
"description": "\n Whether to run the trainer in Dry Run mode. This serves\n as a good means to validate the spec file and run a sanity check on the trainer\n without actually initializing and running the trainer.",
"title": "Is dry run",
"type": "bool"
},
"log_every_n_steps": {
"default": 500,
"description": "\n Interval steps of logging training results and running validation numbers within 1 epoch",
"title": "log steps",
"type": "int"
},
"num_epochs": {
"default": 10,
"description": "Number of epochs to run the training.",
"maximum": Infinity,
"minimum": 1,
"popular": true,
"title": "Number of epochs",
"type": "int"
},
"num_gpus": {
"default": 1,
"description": "The number of GPUs to run the train job.",
"minimum": 1,
"popular": true,
"title": "Number of GPUs",
"type": "int"
},
"num_nodes": {
"default": 1,
"description": "Number of nodes to run the training on. If > 1, then multi-node is enabled.",
"minimum": 1,
"popular": true,
"title": "Number of nodes",
"type": "int"
},
"optim": {
"automl_default_parameters": [
"train.optim.lr",
"train.optim.momentum",
"train.optim.weight_decay",
"train.optim.lr_step_size",
"train.optim.lr_decay",
"train.optim.min_lr"
],
"automl_disabled_parameters": [
"train.optim.lr_steps"
],
"automl_enabled": false,
"default": {
"lr": 0.0001,
"lr_decay": 0.1,
"lr_scheduler": "MultiStepLR",
"lr_step_size": 1000,
"lr_steps": [
1000
],
"min_lr": 1e-07,
"momentum": 0.9,
"monitor_name": "val_loss",
"optimizer": "AdamW",
"warmup_steps": 20,
"weight_decay": 0.0001
},
"description": "Hyper parameters to configure the optimizer.",
"properties": {
"lr": {
"automl_enabled": true,
"default": 0.0001,
"description": "The initial learning rate for training the model, excluding the backbone.",
"math_cond": "> 0.0",
"maximum": 0.01,
"minimum": 1e-06,
"title": "learning rate",
"type": "float"
},
"lr_decay": {
"automl_enabled": true,
"default": 0.1,
"description": "The decreasing factor for the learning rate scheduler.",
"math_cond": "> 0.0",
"maximum": 1.0,
"minimum": 0.0,
"title": "learning rate decay",
"type": "float"
},
"lr_scheduler": {
"default": "MultiStepLR",
"description": "The learning scheduler:\n * MultiStepLR : Decrease the lr by lr_decay from lr_steps\n * StepLR : Decrease the lr by lr_decay at every lr_step_size.",
"enum": [
"MultiStep",
"StepLR",
"CustomMultiStepLRScheduler",
"LambdaLR",
"PolynomialLR",
"OneCycleLR",
"CosineAnnealingLR"
],
"title": "Learning rate scheduler",
"type": "categorical"
},
"lr_step_size": {
"automl_enabled": true,
"default": 1000,
"description": "The number of steps to decrease the learning rate in the StepLR.",
"math_cond": "> 0",
"maximum": 10000,
"minimum": 1,
"title": "learning rate step size",
"type": "int"
},
"lr_steps": {
"automl_enabled": false,
"default": [
1000
],
"description": "The steps at which the learning rate must be decreased.\n This is applicable only with the MultiStep LR.",
"title": "learning rate decay steps",
"type": "list"
},
"min_lr": {
"automl_enabled": true,
"default": 1e-07,
"description": "The minimum learning rate value for the learning rate scheduler.",
"math_cond": "> 0.0",
"maximum": 0.001,
"minimum": 1e-08,
"title": "minimum learning rate",
"type": "float"
},
"momentum": {
"automl_enabled": true,
"default": 0.9,
"description": "The momentum for the AdamW optimizer.",
"math_cond": "> 0.0",
"maximum": 1.0,
"minimum": 0.0,
"title": "momentum - AdamW",
"type": "float"
},
"monitor_name": {
"default": "val_loss",
"description": "The metric value to be monitored for the :code:`AutoReduce` Scheduler.",
"enum": [
"val_loss",
"train_loss"
],
"title": "monitor_name",
"type": "categorical"
},
"optimizer": {
"default": "AdamW",
"description": "Type of optimizer used to train the network.",
"enum": [
"AdamW",
"SGD"
],
"type": "categorical"
},
"warmup_steps": {
"default": 20,
"description": "The number of steps to perform linear learning rate\" warm-up before engaging a learning rate scheduler",
"maximum": Infinity,
"minimum": 0,
"title": "Warm up steps",
"type": "int"
},
"weight_decay": {
"automl_enabled": true,
"default": 0.0001,
"description": "The weight decay coefficient.",
"math_cond": "> 0.0",
"maximum": 0.01,
"minimum": 1e-06,
"title": "weight decay",
"type": "float"
}
},
"title": "optimizer",
"type": "collection"
},
"precision": {
"default": "fp32",
"description": "Precision to run the training on.",
"enum": [
"bf16",
"fp32",
"fp16"
],
"title": "precision",
"type": "categorical"
},
"pretrained_model_path": {
"default": "",
"description": "Path to a pre-trained DepthNet model to initialize the current training from.",
"type": "string"
},
"results_dir": {
"default": "",
"description": "\n Path to where all the assets generated from a task are stored.\n ",
"title": "Results directory",
"type": "string"
},
"resume_training_checkpoint_path": {
"default": "",
"description": "Path to the checkpoint to resume training from.",
"title": "Resume checkpoint path",
"type": "string"
},
"seed": {
"default": 1234,
"description": "The seed for the initializer in PyTorch. If < 0, disable fixed seed.",
"maximum": Infinity,
"minimum": -1,
"title": "Seed for randomization",
"type": "int"
},
"tile_min_overlap": {
"automl_enabled": false,
"default": [
16,
16
],
"description": "Use tiled inference weight type",
"title": "tile weight type",
"type": "list"
},
"tile_wtype": {
"default": "gaussian",
"description": "Use tiled inference weight type",
"title": "tile weight type",
"type": "string"
},
"validation_interval": {
"default": 1,
"description": "\n The interval (in epochs) at which a evaluation\n will be triggered on the validation dataset.",
"minimum": 1,
"popular": true,
"title": "Validation interval",
"type": "int"
},
"vis_step_interval": {
"default": 10,
"description": "The visualization interval in step.",
"title": "visualization interval",
"type": "int"
}
},
"type": "collection"
},
"wandb": {
"automl_disabled_parameters": [
"wandb.tags"
],
"automl_enabled": false,
"default": {
"enable": true,
"entity": "",
"group": "",
"name": "TAO Toolkit Training",
"project": "TAO Toolkit",
"reinit": false,
"run_id": "",
"save_code": false,
"sync_tensorboard": false,
"tags": [
"tao-toolkit"
]
},
"description": "Configurable parameters to construct the wandb client for a DepthNet experiment.",
"properties": {
"enable": {
"default": true,
"type": "bool"
},
"entity": {
"default": "",
"type": "string"
},
"group": {
"default": "",
"type": "string"
},
"name": {
"default": "TAO Toolkit Training",
"type": "string"
},
"project": {
"default": "TAO Toolkit",
"type": "string"
},
"reinit": {
"default": false,
"type": "bool"
},
"run_id": {
"default": "",
"type": "string"
},
"save_code": {
"default": false,
"type": "bool"
},
"sync_tensorboard": {
"default": false,
"type": "bool"
},
"tags": {
"automl_enabled": false,
"default": [
"tao-toolkit"
],
"type": "list"
}
},
"type": "collection"
}
},
"required": [
"quantize"
],
"type": "object",
"x_tao_schema": {
"action": "train",
"core_module": "depth_net",
"model": "depth-net-fast-stereo",
"network_arch": "depth_net_stereo",
"schema_action": "train",
"schema_version": 1,
"source": "tao-core dataclass config"
}
}
Requirements and compatibility
Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta
“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.
Needs: Declares allowed tools, Needs a command-line tool, Needs Node.js
The author says: Requires docker + nvidia-container-toolkit.
Security
What our scanner noticed· Medium risk
- Large base64-encoded blobs
A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.
Attribution and licence
- Created by
- NVIDIA Corporation
- Publisher
- NVIDIA (Official)
- Source
- NVIDIA/skills · skills/tao-train-fast-foundation-stereo · commit fce3cca
- Licence
- Apache License 2.0 (redistribution allowed)
- LICENSE (added by AutoGineer from LICENSE-APACHE)
- ATTRIBUTION.md (written by AutoGineer)
- Synced
AutoGineer is not affiliated with or endorsed by NVIDIA Corporation or NVIDIA.
Related skills
Pytorch Lightning
K-Dense· Community
A framework for organizing PyTorch code to streamline neural network training, scaling, and experiment tracking.
Data and AIClaude CodeClaude appsOpenAI Codex+14 moreMITLow riskHas scriptsGitHub stars of the source repository56.4k
Aeon
K-Dense· Community
A scikit-learn compatible Python toolkit for comprehensive time series machine learning tasks.
Data and AIClaude CodeClaude appsOpenAI Codex+14 moreMITLow riskGitHub stars of the source repository56.4k
Umap Learn
K-Dense· Community
Integrate UMAP-learn for non-linear dimensionality reduction, visualization, and clustering preprocessing.
Data and AIClaude CodeClaude appsOpenAI Codex+14 moreMITLow riskGitHub stars of the source repository56.4k
Scikit Learn
K-Dense· Community
Comprehensive guidance for classical machine learning workflows using the scikit-learn Python library.
Data and AIClaude CodeClaude appsOpenAI Codex+14 moreMITLow riskHas scriptsGitHub stars of the source repository56.4k
Shap
K-Dense· Community
Explain and audit machine-learning model predictions using the SHAP framework for feature attribution and visualization.
Data and AIClaude CodeClaude appsOpenAI Codex+14 moreMITLow riskHas scriptsGitHub stars of the source repository56.4k
Pufferlib
K-Dense· Community
Version-aware guidance for PufferLib reinforcement learning environments, vectorization, policies, and training.
Data and AIClaude CodeClaude appsOpenAI Codex+14 moreMITLow riskHas scriptsGitHub stars of the source repository56.4k
Download ZIP
The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download tao-train-fast-foundation-stereo.zip