Tao Train Depth Anything V2
NVIDIAOfficialCreated by NVIDIA Corporation
- Skill
- Tool integration
- Data and AI
- Apache-2.0
- Medium risk
AutoGineer summary
Written by AutoGineer from the skill’s own files, not by the author
The tao-train-depth-anything-v2 skill provides a command-line interface for managing the lifecycle of monocular depth estimation models based on the Metric Depth Anything v2 or Relative Depth Anything architectures. It supports training, evaluation, inference, export, and quantization of models that predict per-pixel depth from single RGB images. The skill integrates with the TAO toolkit's AutoML capabilities and requires specific dataset configurations to handle relative or metric depth data.
What it does
- Trains monocular depth estimation models using Metric or Relative Depth Anything architectures.
- Evaluates model performance on test datasets using specific metrics like Delta-1 accuracy.
- Exports trained models to formats suitable for deployment.
- Supports AutoML for automated hyperparameter optimization during training.
- Performs inference on single RGB images to generate depth maps.
- Handles quantization for optimized model execution.
When to use it
- When you need to train a monocular depth estimation model from scratch or finetune an existing checkpoint.
- When you have RGB image datasets with corresponding ground truth depth data.
- When you need to convert trained models for deployment using TensorRT.
- When you want to leverage AutoML to optimize training hyperparameters for depth estimation.
Advantages
- Provides a unified interface for the entire model lifecycle within the TAO ecosystem.
- Supports both relative and metric depth estimation variants.
- Includes built-in support for AutoML to streamline the training process.
- Offers flexible configuration for various dataset types and model architectures.
Usage examples
- “Train a new relative depth model using my custom dataset.”
- The agent will guide you through setting up the annotation file, selecting the RelativeDepthAnything model type, and executing the training command with the appropriate spec file.
- “How do I evaluate my trained metric depth model?”
- The agent will provide the necessary steps to configure the evaluation spec, point to your test dataset, and run the evaluation action to generate performance metrics.
Install
Download the skill as a ZIP:
https://autogineer.com/agent-skills/nvidia/tao-train-depth-anything-v2/downloadUnzip it into the cross-client skills folder of your project:
mkdir -p .agents/skills && unzip tao-train-depth-anything-v2.zip -d .agents/skills
Documentation by NVIDIA Corporation
Depth Net Mono
Standalone install? If this session was not initialized by the TAO skill bank plugin, run the
tao-setupskill first (host preflight, credentials, cross-skill discovery).
Monocular depth estimation using Metric Depth Anything v2 or Relative Depth Anything architectures. Predicts per-pixel depth from single RGB images.
Pretrained checkpoint loading varies by model variant and use case — see the Pretrained checkpoint loading — use case matrix in references/parameters.md.
The mono and stereo skills both invoke the unified TAO depth_net CLI inside the container; the mono/stereo family is selected via model.model_type (see references/parameters.md).
For TAO Deploy TensorRT actions (gen_trt_engine, TensorRT evaluate, and TensorRT inference), read references/tao-deploy-depth-anything-v2.md first. The deploy spec template lives in this skill's references/spec_template_deploy.yaml.
PyT actions packaged by this model skill: train, evaluate, inference, export, and quantize. The PyT depth_net entrypoint does not accept a PyT-side gen_trt_engine action in the current TAO image. The gen_trt_engine action metadata must run with the TAO Deploy container, and the deploy workflow remains the deploy-specific entrypoint.
Train Action Policy
This model is AutoML-enabled at the model layer. Before handling any train-stage request, read references/skill_info.yaml and resolve the run override from either an explicit automl_policy value or the user's workflow request. Use automl_policy: on by default and only expose on / off in new launch prompts. Treat phrases like "turn off AutoML", "disable AutoML", "no HPO", or "plain training" as automl_policy: off for this run only. When automl_policy: on, automl_enabled: true, and both schemas/train.schema.json and references/spec_template_train.yaml are packaged, route the train action through tao-skill-bank:tao-run-automl by default with this model's skill_dir. Preserve workflow/application overrides for datasets, specs, output directories, GPU/platform settings, parent checkpoints, and automl_policy. Use direct model training only when automl_policy: off or the packaged train schema/template is missing; in the missing-schema case, report that AutoML is enabled but not runnable for this model until schemas are generated.
Non-train actions such as evaluate, inference, export, and deploy flows stay in this model skill. The per-run automl_policy override does not change model metadata.
Workflow
Prerequisites — data accessibility
Your dataset (RGB images + GT depth files) must be reachable from inside the container:
SDK runner: place files at the S3 paths the runner resolves (the
S3_TRAIN/S3_EVALplaceholders shown in Typical Spec Overrides). The runner handles S3 → container-path mounting transparently.Direct
docker run(e.g. local testing): mount the host dataset root read-only at the same in-container path:
docker run ... -v <host_data_root>:<host_data_root>:ro <container> ...The same accessibility requirement applies to the <output_dir> written by all actions.
Step 1 — Annotation file
Per-line annotation file referenced by data_sources[*].data_file:
| Columns | Format | Use |
|---|---|---|
| 1 | <image> | Mono inference (no GT) |
| 2 | <image> <gt_depth> | Mono with GT |
Do not pass stereo annotation rows such as <left_image> <right_image> <gt_depth> directly to mono train/evaluate/inference. If only a stereo depth
dataset is available, derive a mono annotation file by keeping the left image
and GT depth columns, then mount or stage the image/depth archive at the same
container paths referenced by that derived annotation file.
If you already have one, point to it. Otherwise generate via depth_net convert:
depth_net convert -e <convert_spec.yaml>convert_spec.yaml template:
results_dir: <directory where generated annotation files are written>
data_root: <directory whose immediate children are scene/sample folders that contain your image+depth files; convert walks data_root recursively but expects per-scene subdirectories at one level below>
image_dir_pattern: [<substring matching left/RGB image paths>]
depth_dir_pattern: [<substring matching GT depth paths>]
image_extension: '' # optional .endswith filter, e.g. '.jpg'
depth_extension: '' # optional, swapped during depth derivation, e.g. '.png'
split_ratio: 0.0 # 0.0/1.0 = test-only; 0.8 = 80/20 train+valconvert walks data_root recursively, selects paths whose path-string contains all substrings in image_dir_pattern (AND-filter), then derives the depth path by replacing image_dir_pattern[0] with depth_dir_pattern[0] and image_extension with depth_extension. Inspect your dataset's directory layout and identify the substring distinguishing RGB images from depth files (e.g. rgb_ vs sync_depth_).
data_root must point at the parent that contains the per-scene subdirectories (e.g. for NYU eval, use /data/nyu_v2/eval/test, not /data/nyu_v2/eval/test/bathroom — the latter limits the walk to a single scene). Always include the leading dot in image_extension / depth_extension (e.g. '.jpg' not 'jpg'); the substring swap is form-sensitive and a mismatch silently corrupts derived paths.
Step 2 — Pair model_type and dataset_name based on your data
Default — generic class for each task:
| Data category | model_type | dataset_name |
|---|---|---|
| Disparity-encoded data (pixels) | RelativeDepthAnything | RelativeMonoDataset |
| Metric depth (meters) | MetricDepthAnything | MetricMonoDataset |
| Mono inference (no GT, any image) | matches train choice | RelativeMonoDataset or MetricMonoDataset |
Dataset-specific class — switch when the data needs preprocessing the generic class does not perform:
| Special case | model_type | dataset_name | What the class adds |
|---|---|---|---|
NYU sync_depth_*.png (raw uint16 millimetres) — relative | RelativeDepthAnything | NYUDV2Relative | mm→m unit conversion + Eigen evaluation crop |
NYU sync_depth_*.png (raw uint16 millimetres) — metric | MetricDepthAnything | NYUDV2 | same |
Using a generic class on data that requires unit conversion (e.g. raw NYU uint16 PNGs) results in an empty valid mask and silent train_loss = NaN. Match the class to your data's encoding.
For relative mono data (RelativeMonoDataset or NYUDV2Relative), leave dataset.min_depth and dataset.max_depth unset or set both to null. Non-null metric depth ranges are passed into the relative dataset constructor and fail with BaseRelativeMonoDataset.__init__() got an unexpected keyword argument 'min_depth'.
Step 3 — Write spec yaml from Typical Spec Overrides
Copy the action block from Typical Spec Overrides (references/spec-overrides.md). Replace:
model.model_typefrom Step 2dataset.<...>.data_sources[*].dataset_namefrom Step 2data_sources[*].data_filewith the path from Step 1 (S3 path under SDK runner, host path for direct docker)For metric finetune: additionally apply the Metric Variant Finetuning Recipe in
references/finetuning-recipes.md.
For mono training set train.precision: fp32 (recommended) or bf16 (Ampere SM80+, alternative).
Step 4 — Run
Create writable home/cache directories inside the mounted output path before using
--user. Some TAO containers do not have an /etc/passwd entry for the host UID,
and PyTorch / matplotlib need writable cache paths when running as that UID.
mkdir -p <output_dir>/home \
<output_dir>/.cache/matplotlib \
<output_dir>/.cache/torchinductor \
<output_dir>/.cache/xdgdocker run --gpus 'device=0' --shm-size 16G --shm-size=8g \
--user "$(id -u):$(id -g)" \
-e USER="$(id -un)" \
-e LOGNAME="$(id -un)" \
-e HOME=<output_dir>/home \
-e MPLCONFIGDIR=<output_dir>/.cache/matplotlib \
-e TORCHINDUCTOR_CACHE_DIR=<output_dir>/.cache/torchinductor \
-e XDG_CACHE_HOME=<output_dir>/.cache/xdg \
-v <data_root>:<data_root>:ro \
-v <output_dir>:<output_dir> \
<container> \
depth_net <action> -e <spec.yaml>Without --user "$(id -u):$(id -g)" the container writes outputs as nobody:nogroup, blocking host-side cleanup and retry.
Step 5 — Verify
Container exit code 0
status.jsonkpiblock populatedFor
train: inspect per-steptrain_lossdirectly — the entrypoint reportsExecution status: PASSeven whentrain_loss = NaN(see the Metric Variant Finetuning Recipe → Sanity-run PASS criteria inreferences/finetuning-recipes.md)For
evaluate/inference: artifacts underresults_dir
For TAO Deploy TensorRT actions (gen_trt_engine, TensorRT evaluate, and TensorRT inference), read references/tao-deploy-depth-anything-v2.md first. Deploy spec templates live in this skill's references/ folder with the spec_template_deploy_*.yaml prefix.
Training Requirements
Valid
dataset_namevalues for monodata_sources(case-insensitive):ThreeDVLM,FSD,NvCLIP,IssacStereo,Crestereo,Middlebury,NYUDV2,NYUDV2Relative,RelativeMonoDataset,MetricMonoDataset.NYUDV2carries metric depth GT (meters) — pair withMetricDepthAnything;NYUDV2Relativeis the same data with relative-depth conventions — pair withRelativeDepthAnything.Monitoring metric:
val/d1(maximize),val/loss(minimize).AutoML metric contract: for AutoML sanity runs on the packaged relative-depth smoke data, use
val/d1as the primary monitor and maximize it. Monod1is Delta-1 accuracy: the fraction of valid pixels wheremax(pred/target, target/pred) < 1.25. Do not apply the stereod1error-rate direction to this mono metric.val/losscan be emitted asNaNeven when the trainer exits successfully and writes a usable checkpoint, so it is not a reliable AutoML objective unless the run's status metrics show a finite value.Mono AutoML search allowlist: use
train.optim.lrandtrain.optim.weight_decayfor the default relative-depth train workflow. Do not searchdataset.val_dataset,dataset.test_dataset, ordataset.infer_datasetaugmentation fields because they change scoring/non-train behavior rather than training. Do not searchmodel.corr_radius,model.cv_group, ormodel.volume_dimfor mono; those fields belong to the stereo architecture and are inert forRelativeDepthAnything.
Per-Action Dataset Requirements
| Action | Spec Key | Source | Files | List? |
|---|---|---|---|---|
| evaluate | dataset.test_dataset.data_sources | eval_dataset | data_file: annotations.txt + dataset_name | Yes |
| inference | dataset.infer_dataset.data_sources | inference_dataset | data_file: annotations.txt + dataset_name | Yes |
| quantize | dataset.train_dataset.data_sources | train_datasets | data_file: annotations.txt + dataset_name | Yes |
| quantize | dataset.val_dataset.data_sources | eval_dataset | data_file: annotations.txt + dataset_name | Yes |
| quantize | dataset.quant_calibration_dataset.images_dir | train_datasets | images.tar.gz | No |
| train | dataset.train_dataset.data_sources | train_datasets | data_file: annotations.txt + dataset_name | Yes |
| train | dataset.val_dataset.data_sources | eval_dataset | data_file: annotations.txt + dataset_name | Yes |
Typical Spec Overrides
Data source overrides are mandatory for every action — construct data source paths from the Per-Action Dataset Requirements table above and include them in spec_overrides. Each data_sources entry is a dict with two mandatory fields: data_file and dataset_name. See references/spec-overrides.md for the full per-action override blocks (train, evaluate, export, inference, quantize), the S3_TRAIN / S3_EVAL placeholders, the relative-variant precision recommendation, and the quantize known-issue note.
Eval Dataset
Optional. Val dataset configured via dataset.val_dataset.data_sources (each entry needs data_file and dataset_name).
Important Parameters
See references/parameters.md for the full parameter glossary (model, train, dataset, export, and inference keys with options, defaults, and sources) and the Pretrained checkpoint loading — use case matrix.
Finetuning Recipes
See references/finetuning-recipes.md for:
Relative Variant Finetuning Recipe — finetune from a TAO-trained
RelativeDepthAnythingcheckpoint (lr5e-6,LambdaLR, sanity-vs-convergent guidance, deploy LSQ alignment note).Metric Variant Finetuning Recipe — checkpoint compatibility, required overrides, the dataset normalization block (
normalize_depth/min_depth/max_depth) required in train AND export specs, trainer-enforced defaults, precision, the 1-epoch sanity-run override, and the Sanity-run PASS criteria with the NaN-mitigation order.
Multi-GPU / Multi-Node
Launch method: Lightning-managed (single python process, Lightning spawns workers).
| Spec Key | Description | Default |
|---|---|---|
train.num_gpus | Number of GPUs | 1 |
train.gpu_ids | GPU device indices | [0] |
train.num_nodes | Number of nodes | 1 |
train.distributed_strategy | ddp or fsdp | ddp |
ddpwith activation checkpointing:find_unused_parameters=Falseddpwithout:find_unused_parameters=Truefsdpforces precision to FP16
Multi-node env vars (set by orchestrator): WORLD_SIZE, NODE_RANK, MASTER_ADDR, MASTER_PORT, NUM_GPU_PER_NODE.
Export / TRT Defaults
TRT data types: FP32, BF16 (Ampere SM80+). FP16 is not supported for the ViT-L mono backbone.
Fresh-install TRT precision:
fp32. BF16 is supported on Ampere SM80+ hardware, but keep smoke tests on FP32 unless the user explicitly requests BF16.
Hardware
Minimum 1 GPU(s), recommended 2 GPU(s). 24GB+ VRAM per GPU. ViT-Large encoder is memory intensive. Use fp32 (recommended) or bf16 (Ampere SM80+, alternative) for training. Activation checkpointing is available for larger inputs.
Error Patterns
See references/troubleshooting.md for the full error-pattern catalog (depth range mismatch, relative dataset rejecting min_depth, missing pretrained weights, encoder key location, dataset_name not in struct, depth_net_mono not found, metric variant hyperparameter sourcing, and export ONNX overwrite).
Spec Param / Parent Model Inference
See references/spec-param-inference.md for the model-specific inference mappings (the TAO Core depth_net_mono.config.json action table), checkpoint-file naming under <results_dir>/train/, the dn_model_latest.pth policy, the parent-gen_trt_engine rationale, and the parent_model / parent_job_id resolution rules.
Deployment
Files
- ATTRIBUTION.mdAdded by AutoGineer951 B
- BENCHMARK.md7.0 KiB
- LICENSELicence file added by AutoGineer11.2 KiB
- SKILL.md15.3 KiB
- config/skillspector-baseline.yaml1010 B
- evals/evals.json871 B
- references/finetuning-recipes.md7.1 KiB
- references/parameters.md5.4 KiB
- references/skill_info.yaml4.4 KiB
- references/spec-overrides.md4.6 KiB
- references/spec-param-inference.md3.4 KiB
- references/spec_template_deploy.yaml2.1 KiB
- references/spec_template_evaluate.yaml5.6 KiB
- references/spec_template_export.yaml5.6 KiB
- references/spec_template_gen_trt_engine.yaml5.6 KiB
- references/spec_template_inference.yaml5.6 KiB
- references/spec_template_quantize.yaml5.4 KiB
- references/spec_template_train.yaml5.4 KiB
- references/tao-deploy-depth-anything-v2.md7.3 KiB
- references/tao-deploy-depth-anything-v2.skill_info.yaml2.4 KiB
- references/troubleshooting.md2.1 KiB
- schemas/evaluate.schema.json104.5 KiB
- schemas/export.schema.json105.1 KiB
- schemas/gen_trt_engine.schema.json107.0 KiB
- schemas/inference.schema.json104.9 KiB
- schemas/manifest.json29.9 KiB
- schemas/quantize.schema.json101.4 KiB
- schemas/train.schema.json101.4 KiB
- skill-card.md4.9 KiB
- skill.oms.sig10.1 KiB
schemas/inference.schema.json
{
"automl_default_parameters": [
"train.optim.lr_step_size",
"dataset.infer_dataset.augmentation.yjitter_prob",
"dataset.test_dataset.augmentation.crop_min_valid_disp_ratio",
"model.corr_radius",
"train.optim.weight_decay",
"train.optim.lr_decay",
"dataset.infer_dataset.augmentation.hshift_prob",
"dataset.infer_dataset.augmentation.crop_min_valid_disp_ratio",
"dataset.train_dataset.augmentation.eraser_aug_prob",
"dataset.val_dataset.augmentation.color_aug_prob",
"dataset.val_dataset.augmentation.yjitter_prob",
"dataset.train_dataset.augmentation.stretch_prob",
"dataset.train_dataset.augmentation.yjitter_prob",
"model.volume_dim",
"dataset.train_dataset.augmentation.spatial_aug_prob",
"dataset.infer_dataset.augmentation.spatial_aug_prob",
"dataset.val_dataset.augmentation.h_flip_prob",
"dataset.train_dataset.augmentation.color_aug_prob",
"dataset.test_dataset.augmentation.h_flip_prob",
"dataset.train_dataset.augmentation.hshift_prob",
"train.optim.momentum",
"dataset.val_dataset.augmentation.crop_min_valid_disp_ratio",
"dataset.test_dataset.augmentation.hshift_prob",
"dataset.val_dataset.augmentation.v_flip_prob",
"dataset.infer_dataset.augmentation.h_flip_prob",
"dataset.val_dataset.augmentation.hshift_prob",
"dataset.test_dataset.augmentation.stretch_prob",
"dataset.val_dataset.augmentation.stretch_prob",
"dataset.infer_dataset.augmentation.eraser_aug_prob",
"train.optim.min_lr",
"model.cv_group",
"dataset.infer_dataset.augmentation.stretch_prob",
"dataset.train_dataset.augmentation.h_flip_prob",
"dataset.test_dataset.augmentation.eraser_aug_prob",
"dataset.infer_dataset.augmentation.color_aug_prob",
"train.optim.lr",
"dataset.test_dataset.augmentation.spatial_aug_prob",
"dataset.test_dataset.augmentation.yjitter_prob",
"dataset.test_dataset.augmentation.v_flip_prob",
"dataset.train_dataset.augmentation.v_flip_prob",
"dataset.val_dataset.augmentation.spatial_aug_prob",
"dataset.train_dataset.augmentation.crop_min_valid_disp_ratio",
"dataset.test_dataset.augmentation.color_aug_prob",
"dataset.infer_dataset.augmentation.v_flip_prob",
"dataset.val_dataset.augmentation.eraser_aug_prob"
],
"automl_disabled_parameters": [
"quantize",
"train.cudnn",
"dataset.val_dataset.data_sources",
"quantize.backend_kwargs",
"dataset.train_dataset.augmentation",
"dataset.test_dataset.augmentation.input_std",
"train.gpu_ids",
"wandb.tags",
"dataset.infer_dataset.augmentation",
"dataset.train_dataset.data_sources",
"dataset.train_dataset.augmentation.color_aug_saturation",
"dataset.infer_dataset.augmentation.color_aug_hue_range",
"dataset.train_dataset",
"quantize.skip_names",
"dataset.infer_dataset.data_sources",
"dataset.val_dataset.augmentation.input_std",
"inference",
"evaluate",
"train",
"dataset.val_dataset.augmentation.input_mean",
"dataset.test_dataset.data_sources",
"gen_trt_engine",
"dataset.train_dataset.augmentation.input_std",
"dataset.train_dataset.augmentation.input_mean",
"dataset.test_dataset",
"gen_trt_engine.tensorrt.layers_precision",
"dataset",
"gen_trt_engine.tensorrt",
"dataset.val_dataset",
"dataset.val_dataset.augmentation.gamma",
"quantize.layers",
"dataset.infer_dataset",
"dataset.test_dataset.augmentation.color_aug_saturation",
"dataset.infer_dataset.augmentation.input_mean",
"dataset.test_dataset.augmentation",
"dataset.train_dataset.augmentation.crop_size",
"dataset.infer_dataset.augmentation.gamma",
"dataset.infer_dataset.augmentation.crop_size",
"dataset.quant_calibration_dataset",
"dataset.infer_dataset.augmentation.input_std",
"model.stereo_backbone",
"model.hidden_dims",
"dataset.train_dataset.augmentation.color_aug_hue_range",
"model",
"train.optim.lr_steps",
"dataset.test_dataset.augmentation.gamma",
"dataset.val_dataset.augmentation.color_aug_saturation",
"evaluate.gpu_ids",
"dataset.test_dataset.augmentation.crop_size",
"train.optim",
"dataset.val_dataset.augmentation.crop_size",
"dataset.val_dataset.augmentation",
"dataset.test_dataset.augmentation.input_mean",
"model.mono_backbone",
"dataset.train_dataset.augmentation.gamma",
"dataset.test_dataset.augmentation.color_aug_hue_range",
"export",
"wandb",
"dataset.val_dataset.augmentation.color_aug_hue_range",
"dataset.infer_dataset.augmentation.color_aug_saturation",
"inference.gpu_ids",
"train.tile_min_overlap"
],
"default": {
"dataset": {
"baseline": 0.193001,
"dataset_name": "StereoDataset",
"focal_x": 1998.842,
"infer_dataset": {
"augmentation": {
"color_aug_brightness": 0.4,
"color_aug_contrast": 0.4,
"color_aug_hue_range": [
-0.027777777777777776,
0.027777777777777776
],
"color_aug_prob": 0.2,
"color_aug_saturation": [
0.0,
1.4
],
"crop_min_valid_disp_ratio": 0.0,
"crop_size": [
518,
518
],
"do_flip": false,
"eraser_aug_prob": 0.5,
"gamma": [
1,
1,
1,
1
],
"h_flip_prob": 0.5,
"hshift_prob": 0.5,
"input_mean": [
0.485,
0.456,
0.406
],
"input_std": [
0.229,
0.224,
0.225
],
"max_scale": 0.4,
"max_stretch": 0.2,
"min_scale": -0.2,
"spatial_aug_prob": 1.0,
"stretch_prob": 0.8,
"v_flip_prob": 0.5,
"yjitter_prob": 1.0
},
"batch_size": 1,
"data_sources": [
{
"data_file": "",
"dataset_name": ""
}
],
"pin_memory": true,
"workers": 8
},
"max_disparity": 416,
"normalize_depth": false,
"quant_calibration_dataset": {
"images_dir": ""
},
"test_dataset": {
"augmentation": {
"color_aug_brightness": 0.4,
"color_aug_contrast": 0.4,
"color_aug_hue_range": [
-0.027777777777777776,
0.027777777777777776
],
"color_aug_prob": 0.2,
"color_aug_saturation": [
0.0,
1.4
],
"crop_min_valid_disp_ratio": 0.0,
"crop_size": [
518,
518
],
"do_flip": false,
"eraser_aug_prob": 0.5,
"gamma": [
1,
1,
1,
1
],
"h_flip_prob": 0.5,
"hshift_prob": 0.5,
"input_mean": [
0.485,
0.456,
0.406
],
"input_std": [
0.229,
0.224,
0.225
],
"max_scale": 0.4,
"max_stretch": 0.2,
"min_scale": -0.2,
"spatial_aug_prob": 1.0,
"stretch_prob": 0.8,
"v_flip_prob": 0.5,
"yjitter_prob": 1.0
},
"batch_size": 1,
"data_sources": [
{
"data_file": "",
"dataset_name": ""
}
],
"pin_memory": true,
"workers": 8
},
"train_dataset": {
"augmentation": {
"color_aug_brightness": 0.4,
"color_aug_contrast": 0.4,
"color_aug_hue_range": [
-0.027777777777777776,
0.027777777777777776
],
"color_aug_prob": 0.2,
"color_aug_saturation": [
0.0,
1.4
],
"crop_min_valid_disp_ratio": 0.0,
"crop_size": [
518,
518
],
"do_flip": false,
"eraser_aug_prob": 0.5,
"gamma": [
1,
1,
1,
1
],
"h_flip_prob": 0.5,
"hshift_prob": 0.5,
"input_mean": [
0.485,
0.456,
0.406
],
"input_std": [
0.229,
0.224,
0.225
],
"max_scale": 0.4,
"max_stretch": 0.2,
"min_scale": -0.2,
"spatial_aug_prob": 1.0,
"stretch_prob": 0.8,
"v_flip_prob": 0.5,
"yjitter_prob": 1.0
},
"batch_size": 1,
"data_sources": [
{
"data_file": "",
"dataset_name": ""
}
],
"pin_memory": true,
"workers": 8
},
"val_dataset": {
"augmentation": {
"color_aug_brightness": 0.4,
"color_aug_contrast": 0.4,
"color_aug_hue_range": [
-0.027777777777777776,
0.027777777777777776
],
"color_aug_prob": 0.2,
"color_aug_saturation": [
0.0,
1.4
],
"crop_min_valid_disp_ratio": 0.0,
"crop_size": [
518,
518
],
"do_flip": false,
"eraser_aug_prob": 0.5,
"gamma": [
1,
1,
1,
1
],
"h_flip_prob": 0.5,
"hshift_prob": 0.5,
"input_mean": [
0.485,
0.456,
0.406
],
"input_std": [
0.229,
0.224,
0.225
],
"max_scale": 0.4,
"max_stretch": 0.2,
"min_scale": -0.2,
"spatial_aug_prob": 1.0,
"stretch_prob": 0.8,
"v_flip_prob": 0.5,
"yjitter_prob": 1.0
},
"batch_size": 1,
"data_sources": [
{
"data_file": "",
"dataset_name": ""
}
],
"pin_memory": true,
"workers": 8
}
},
"encryption_key": "",
"inference": {
"batch_size": -1,
"checkpoint": "???",
"conf_threshold": 0.5,
"gpu_ids": [
0
],
"num_gpus": 1,
"num_nodes": 1,
"results_dir": "",
"save_raw_pfm": false,
"trt_engine": ""
},
"model": {
"corr_levels": 2,
"corr_radius": 4,
"cv_group": 8,
"encoder": "vitl",
"hidden_dims": [
128,
128,
128
],
"low_memory": 0,
"max_disparity": 416,
"mixed_precision": false,
"model_type": "MetricDepthAnything",
"mono_backbone": {
"pretrained_path": "",
"use_bn": false,
"use_clstoken": false
},
"n_downsample": 2,
"n_gru_layers": 3,
"stereo_backbone": {
"depth_anything_v2_pretrained_path": "",
"edgenext_pretrained_path": "",
"use_bn": false,
"use_clstoken": false
},
"train_iters": 22,
"valid_iters": 22,
"volume_dim": 32
},
"model_name": "",
"quantize": {
"algorithm": "minmax",
"backend": "torchao",
"backend_kwargs": {},
"device": "cuda",
"layers": [],
"mode": "weight_only_ptq",
"model_path": "",
"results_dir": "",
"skip_names": []
},
"results_dir": "",
"train": {
"activation_checkpoint": false,
"checkpoint_interval": 1,
"checkpoint_interval_unit": "epoch",
"clip_grad_norm": 0.1,
"cudnn": {
"benchmark": false,
"deterministic": true
},
"dataloader_visualize": false,
"distributed_strategy": "ddp",
"gpu_ids": [
0
],
"inference_tile": false,
"is_dry_run": false,
"log_every_n_steps": 500,
"num_epochs": 10,
"num_gpus": 1,
"num_nodes": 1,
"optim": {
"lr": 0.0001,
"lr_decay": 0.1,
"lr_scheduler": "MultiStepLR",
"lr_step_size": 1000,
"lr_steps": [
1000
],
"min_lr": 1e-07,
"momentum": 0.9,
"monitor_name": "val_loss",
"optimizer": "AdamW",
"warmup_steps": 20,
"weight_decay": 0.0001
},
"precision": "fp32",
"pretrained_model_path": "",
"results_dir": "",
"resume_training_checkpoint_path": "",
"seed": 1234,
"tile_min_overlap": [
16,
16
],
"tile_wtype": "gaussian",
"validation_interval": 1,
"vis_step_interval": 10
},
"wandb": {
"enable": true,
"entity": "",
"group": "",
"name": "TAO Toolkit Training",
"project": "TAO Toolkit",
"reinit": false,
"run_id": "",
"save_code": false,
"sync_tensorboard": false,
"tags": [
"tao-toolkit"
]
}
},
"popular": {
"evaluate": {
"gpu_ids": [
0
],
"num_gpus": 1,
"num_nodes": 1
},
"gen_trt_engine": {
"batch_size": -1,
"gpu_id": 0,
"tensorrt": {
"max_batch_size": 1,
"min_batch_size": 1,
"opt_batch_size": 1
}
},
"inference": {
"gpu_ids": [
0
],
"num_gpus": 1,
"num_nodes": 1
},
"train": {
"checkpoint_interval": 1,
"gpu_ids": [
0
],
"num_epochs": 10,
"num_gpus": 1,
"num_nodes": 1,
"validation_interval": 1
}
},
"properties": {
"automl_disabled_parameters": [
"wandb",
"dataset",
"model",
"inference",
"evaluate",
"train",
"export",
"gen_trt_engine",
"quantize"
],
"dataset": {
"automl_disabled_parameters": [
"dataset.train_dataset",
"dataset.val_dataset",
"dataset.test_dataset",
"dataset.infer_dataset",
"dataset.quant_calibration_dataset"
],
"automl_enabled": false,
"default": {
"baseline": 0.193001,
"dataset_name": "StereoDataset",
"focal_x": 1998.842,
"infer_dataset": {
"augmentation": {
"color_aug_brightness": 0.4,
"color_aug_contrast": 0.4,
"color_aug_hue_range": [
-0.027777777777777776,
0.027777777777777776
],
"color_aug_prob": 0.2,
"color_aug_saturation": [
0.0,
1.4
],
"crop_min_valid_disp_ratio": 0.0,
"crop_size": [
518,
518
],
"do_flip": false,
"eraser_aug_prob": 0.5,
"gamma": [
1,
1,
1,
1
],
"h_flip_prob": 0.5,
"hshift_prob": 0.5,
"input_mean": [
0.485,
0.456,
0.406
],
"input_std": [
0.229,
0.224,
0.225
],
"max_scale": 0.4,
"max_stretch": 0.2,
"min_scale": -0.2,
"spatial_aug_prob": 1.0,
"stretch_prob": 0.8,
"v_flip_prob": 0.5,
"yjitter_prob": 1.0
},
"batch_size": 1,
"data_sources": [
{
"data_file": "",
"dataset_name": ""
}
],
"pin_memory": true,
"workers": 8
},
"max_disparity": 416,
"normalize_depth": false,
"quant_calibration_dataset": {
"images_dir": ""
},
"test_dataset": {
"augmentation": {
"color_aug_brightness": 0.4,
"color_aug_contrast": 0.4,
"color_aug_hue_range": [
-0.027777777777777776,
0.027777777777777776
],
"color_aug_prob": 0.2,
"color_aug_saturation": [
0.0,
1.4
],
"crop_min_valid_disp_ratio": 0.0,
"crop_size": [
518,
518
],
"do_flip": false,
"eraser_aug_prob": 0.5,
"gamma": [
1,
1,
1,
1
],
"h_flip_prob": 0.5,
"hshift_prob": 0.5,
"input_mean": [
0.485,
0.456,
0.406
],
"input_std": [
0.229,
0.224,
0.225
],
"max_scale": 0.4,
"max_stretch": 0.2,
"min_scale": -0.2,
"spatial_aug_prob": 1.0,
"stretch_prob": 0.8,
"v_flip_prob": 0.5,
"yjitter_prob": 1.0
},
"batch_size": 1,
"data_sources": [
{
"data_file": "",
"dataset_name": ""
}
],
"pin_memory": true,
"workers": 8
},
"train_dataset": {
"augmentation": {
"color_aug_brightness": 0.4,
"color_aug_contrast": 0.4,
"color_aug_hue_range": [
-0.027777777777777776,
0.027777777777777776
],
"color_aug_prob": 0.2,
"color_aug_saturation": [
0.0,
1.4
],
"crop_min_valid_disp_ratio": 0.0,
"crop_size": [
518,
518
],
"do_flip": false,
"eraser_aug_prob": 0.5,
"gamma": [
1,
1,
1,
1
],
"h_flip_prob": 0.5,
"hshift_prob": 0.5,
"input_mean": [
0.485,
0.456,
0.406
],
"input_std": [
0.229,
0.224,
0.225
],
"max_scale": 0.4,
"max_stretch": 0.2,
"min_scale": -0.2,
"spatial_aug_prob": 1.0,
"stretch_prob": 0.8,
"v_flip_prob": 0.5,
"yjitter_prob": 1.0
},
"batch_size": 1,
"data_sources": [
{
"data_file": "",
"dataset_name": ""
}
],
"pin_memory": true,
"workers": 8
},
"val_dataset": {
"augmentation": {
"color_aug_brightness": 0.4,
"color_aug_contrast": 0.4,
"color_aug_hue_range": [
-0.027777777777777776,
0.027777777777777776
],
"color_aug_prob": 0.2,
"color_aug_saturation": [
0.0,
1.4
],
"crop_min_valid_disp_ratio": 0.0,
"crop_size": [
518,
518
],
"do_flip": false,
"eraser_aug_prob": 0.5,
"gamma": [
1,
1,
1,
1
],
"h_flip_prob": 0.5,
"hshift_prob": 0.5,
"input_mean": [
0.485,
0.456,
0.406
],
"input_std": [
0.229,
0.224,
0.225
],
"max_scale": 0.4,
"max_stretch": 0.2,
"min_scale": -0.2,
"spatial_aug_prob": 1.0,
"stretch_prob": 0.8,
"v_flip_prob": 0.5,
"yjitter_prob": 1.0
},
"batch_size": 1,
"data_sources": [
{
"data_file": "",
"dataset_name": ""
}
],
"pin_memory": true,
"workers": 8
}
},
"description": "Configurable parameters to construct the dataset for a DepthNet experiment.",
"properties": {
"baseline": {
"default": 0.193001,
"description": "The baseline for stereo datasets",
"maximum": Infinity,
"minimum": 0.0,
"title": "Stereo baseline",
"type": "float"
},
"dataset_name": {
"default": "StereoDataset",
"description": "Dataset Name",
"enum": [
"MonoDataset",
"StereoDataset"
],
"title": "dataset mame",
"type": "categorical"
},
"focal_x": {
"default": 1998.842,
"description": "The focal length along x-axis",
"maximum": Infinity,
"minimum": 0.0,
"title": "The focal length along x-axis",
"type": "float"
},
"infer_dataset": {
"automl_disabled_parameters": [
"dataset.infer_dataset.data_sources",
"dataset.infer_dataset.augmentation"
],
"automl_enabled": false,
"default": {
"augmentation": {
"color_aug_brightness": 0.4,
"color_aug_contrast": 0.4,
"color_aug_hue_range": [
-0.027777777777777776,
0.027777777777777776
],
"color_aug_prob": 0.2,
"color_aug_saturation": [
0.0,
1.4
],
"crop_min_valid_disp_ratio": 0.0,
"crop_size": [
518,
518
],
"do_flip": false,
"eraser_aug_prob": 0.5,
"gamma": [
1,
1,
1,
1
],
"h_flip_prob": 0.5,
"hshift_prob": 0.5,
"input_mean": [
0.485,
0.456,
0.406
],
"input_std": [
0.229,
0.224,
0.225
],
"max_scale": 0.4,
"max_stretch": 0.2,
"min_scale": -0.2,
"spatial_aug_prob": 1.0,
"stretch_prob": 0.8,
"v_flip_prob": 0.5,
"yjitter_prob": 1.0
},
"batch_size": 1,
"data_sources": [
{
"data_file": "",
"dataset_name": ""
}
],
"pin_memory": true,
"workers": 8
},
"description": "Configurable parameters to construct the infer dataset for a DepthNet experiment.",
"properties": {
"augmentation": {
"automl_default_parameters": [
"dataset.infer_dataset.augmentation.yjitter_prob",
"dataset.infer_dataset.augmentation.color_aug_prob",
"dataset.infer_dataset.augmentation.eraser_aug_prob",
"dataset.infer_dataset.augmentation.spatial_aug_prob",
"dataset.infer_dataset.augmentation.stretch_prob",
"dataset.infer_dataset.augmentation.h_flip_prob",
"dataset.infer_dataset.augmentation.v_flip_prob",
"dataset.infer_dataset.augmentation.hshift_prob",
"dataset.infer_dataset.augmentation.crop_min_valid_disp_ratio"
],
"automl_disabled_parameters": [
"dataset.infer_dataset.augmentation.input_mean",
"dataset.infer_dataset.augmentation.input_std",
"dataset.infer_dataset.augmentation.crop_size",
"dataset.infer_dataset.augmentation.gamma",
"dataset.infer_dataset.augmentation.color_aug_saturation",
"dataset.infer_dataset.augmentation.color_aug_hue_range"
],
"automl_enabled": false,
"default": {
"color_aug_brightness": 0.4,
"color_aug_contrast": 0.4,
"color_aug_hue_range": [
-0.027777777777777776,
0.027777777777777776
],
"color_aug_prob": 0.2,
"color_aug_saturation": [
0.0,
1.4
],
"crop_min_valid_disp_ratio": 0.0,
"crop_size": [
518,
518
],
"do_flip": false,
"eraser_aug_prob": 0.5,
"gamma": [
1,
1,
1,
1
],
"h_flip_prob": 0.5,
"hshift_prob": 0.5,
"input_mean": [
0.485,
0.456,
0.406
],
"input_std": [
0.229,
0.224,
0.225
],
"max_scale": 0.4,
"max_stretch": 0.2,
"min_scale": -0.2,
"spatial_aug_prob": 1.0,
"stretch_prob": 0.8,
"v_flip_prob": 0.5,
"yjitter_prob": 1.0
},
"description": "Configuration parameters for data augmentation",
"properties": {
"color_aug_brightness": {
"default": 0.4,
"description": "The color jitter brightness",
"maximum": 1.0,
"minimum": 0.0,
"title": "The color jitter brightness",
"type": "float"
},
"color_aug_contrast": {
"default": 0.4,
"description": "The color jitter contrast",
"maximum": 1.0,
"minimum": 0.0,
"title": "The color jitter contrast",
"type": "float"
},
"color_aug_hue_range": {
"automl_enabled": false,
"default": [
-0.027777777777777776,
0.027777777777777776
],
"description": "The hue range in data augmentation",
"title": "hue range augmentaiton",
"type": "list"
},
"color_aug_prob": {
"automl_enabled": true,
"default": 0.2,
"description": "The probability for asymmetric color augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for asymmetric color augmentation",
"type": "float"
},
"color_aug_saturation": {
"automl_enabled": false,
"default": [
0.0,
1.4
],
"description": "The color jitter saturation",
"title": "The color jitter saturation",
"type": "list"
},
"crop_min_valid_disp_ratio": {
"automl_enabled": true,
"default": 0.0,
"description": "The probability for minimum crop valid disparity ratio",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for min crop valid disparity ratio",
"type": "float"
},
"crop_size": {
"automl_enabled": false,
"default": [
518,
518
],
"description": "The crop size for input RGB images [height, width]",
"title": "augmentation crop size",
"type": "list"
},
"do_flip": {
"default": false,
"description": "A flag specifying whether to perform flip in data augmentation",
"title": "do flip",
"type": "bool"
},
"eraser_aug_prob": {
"automl_enabled": true,
"default": 0.5,
"description": "The probability for eraser augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for eraser augmentation",
"type": "float"
},
"gamma": {
"automl_enabled": false,
"default": [
1,
1,
1,
1
],
"description": "Gamma range in data augmentation",
"title": "gamma range",
"type": "list"
},
"h_flip_prob": {
"automl_enabled": true,
"default": 0.5,
"description": "The probability for horizontal flip augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for horizontal flip augmentation",
"type": "float"
},
"hshift_prob": {
"automl_enabled": true,
"default": 0.5,
"description": "The probability for horizontal shift augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for horizontal flip augmentation",
"type": "float"
},
"input_mean": {
"automl_enabled": false,
"default": [
0.485,
0.456,
0.406
],
"description": "The input mean for RGB frames",
"title": "input mean per pixel",
"type": "list"
},
"input_std": {
"automl_enabled": false,
"default": [
0.229,
0.224,
0.225
],
"description": "The input standard deviation per pixel for RGB frames",
"title": "input std per pixel",
"type": "list"
},
"max_scale": {
"default": 0.4,
"description": "The maximum scale in data augmentation",
"maximum": 1.0,
"minimum": -0.2,
"title": "max scale",
"type": "float"
},
"max_stretch": {
"default": 0.2,
"description": "The maximum stretch augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The maximum stretch augmentation",
"type": "float"
},
"min_scale": {
"default": -0.2,
"description": "The minimum scale in data augmentation",
"maximum": 1.0,
"minimum": 0.2,
"title": "min scale",
"type": "float"
},
"spatial_aug_prob": {
"automl_enabled": true,
"default": 1.0,
"description": "The probability for spatial augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for spatial augmentation",
"type": "float"
},
"stretch_prob": {
"automl_enabled": true,
"default": 0.8,
"description": "The probability for stretch augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for stretch augmentation",
"type": "float"
},
"v_flip_prob": {
"automl_enabled": true,
"default": 0.5,
"description": "The probability for vertical flip augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for vertical flip augmentation",
"type": "float"
},
"yjitter_prob": {
"automl_enabled": true,
"default": 1.0,
"description": "The probability for y jitter",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for y jitter",
"type": "float"
}
},
"title": "augmentation",
"type": "collection"
},
"batch_size": {
"default": 1,
"description": "The batch size for training and validation",
"maximum": Infinity,
"minimum": 1,
"title": "batch size",
"type": "int"
},
"data_sources": {
"automl_enabled": false,
"default": [
{
"data_file": "",
"dataset_name": ""
}
],
"description": "The list of data sources for training:\n * dataset_name : The type of the dataset\n * data_file : The path of the data file",
"title": "train data sources",
"type": "list"
},
"pin_memory": {
"default": true,
"description": "Flag to enable the dataloader to allocated pagelocked memory for faster\n of data between the CPU and GPU.",
"title": "pin_memory",
"type": "bool"
},
"workers": {
"default": 8,
"description": "The number of parallel workers processing data",
"maximum": Infinity,
"minimum": 1,
"title": "batch size",
"type": "int"
}
},
"type": "collection"
},
"max_depth": {
"description": "The maximum depth in meters in MetricDepthAnythingV2",
"maximum": Infinity,
"minimum": 1.0,
"title": "max depth in meters",
"type": "float"
},
"max_disparity": {
"default": 416,
"description": "The maximum allowed disparity for which we compute losses during training",
"maximum": 416,
"minimum": 1,
"title": "maximum dispairty",
"type": "int"
},
"min_depth": {
"description": "The minimum depth in meters in MetricDepthAnythingV2",
"maximum": Infinity,
"minimum": 0.0,
"title": "min depth in meters",
"type": "float"
},
"normalize_depth": {
"default": false,
"description": "Normalize depth",
"title": "normalize depth",
"type": "bool"
},
"quant_calibration_dataset": {
"automl_enabled": false,
"default": {
"images_dir": ""
},
"description": "Configurable parameters for quantization calibration dataset.",
"properties": {
"images_dir": {
"default": "",
"description": "Path to the directory containing calibration images.",
"title": "calibration images directory",
"type": "string"
}
},
"type": "collection"
},
"test_dataset": {
"automl_disabled_parameters": [
"dataset.test_dataset.data_sources",
"dataset.test_dataset.augmentation"
],
"automl_enabled": false,
"default": {
"augmentation": {
"color_aug_brightness": 0.4,
"color_aug_contrast": 0.4,
"color_aug_hue_range": [
-0.027777777777777776,
0.027777777777777776
],
"color_aug_prob": 0.2,
"color_aug_saturation": [
0.0,
1.4
],
"crop_min_valid_disp_ratio": 0.0,
"crop_size": [
518,
518
],
"do_flip": false,
"eraser_aug_prob": 0.5,
"gamma": [
1,
1,
1,
1
],
"h_flip_prob": 0.5,
"hshift_prob": 0.5,
"input_mean": [
0.485,
0.456,
0.406
],
"input_std": [
0.229,
0.224,
0.225
],
"max_scale": 0.4,
"max_stretch": 0.2,
"min_scale": -0.2,
"spatial_aug_prob": 1.0,
"stretch_prob": 0.8,
"v_flip_prob": 0.5,
"yjitter_prob": 1.0
},
"batch_size": 1,
"data_sources": [
{
"data_file": "",
"dataset_name": ""
}
],
"pin_memory": true,
"workers": 8
},
"description": "Configurable parameters to construct the test dataset for a DepthNet experiment.",
"properties": {
"augmentation": {
"automl_default_parameters": [
"dataset.test_dataset.augmentation.yjitter_prob",
"dataset.test_dataset.augmentation.color_aug_prob",
"dataset.test_dataset.augmentation.eraser_aug_prob",
"dataset.test_dataset.augmentation.spatial_aug_prob",
"dataset.test_dataset.augmentation.stretch_prob",
"dataset.test_dataset.augmentation.h_flip_prob",
"dataset.test_dataset.augmentation.v_flip_prob",
"dataset.test_dataset.augmentation.hshift_prob",
"dataset.test_dataset.augmentation.crop_min_valid_disp_ratio"
],
"automl_disabled_parameters": [
"dataset.test_dataset.augmentation.input_mean",
"dataset.test_dataset.augmentation.input_std",
"dataset.test_dataset.augmentation.crop_size",
"dataset.test_dataset.augmentation.gamma",
"dataset.test_dataset.augmentation.color_aug_saturation",
"dataset.test_dataset.augmentation.color_aug_hue_range"
],
"automl_enabled": false,
"default": {
"color_aug_brightness": 0.4,
"color_aug_contrast": 0.4,
"color_aug_hue_range": [
-0.027777777777777776,
0.027777777777777776
],
"color_aug_prob": 0.2,
"color_aug_saturation": [
0.0,
1.4
],
"crop_min_valid_disp_ratio": 0.0,
"crop_size": [
518,
518
],
"do_flip": false,
"eraser_aug_prob": 0.5,
"gamma": [
1,
1,
1,
1
],
"h_flip_prob": 0.5,
"hshift_prob": 0.5,
"input_mean": [
0.485,
0.456,
0.406
],
"input_std": [
0.229,
0.224,
0.225
],
"max_scale": 0.4,
"max_stretch": 0.2,
"min_scale": -0.2,
"spatial_aug_prob": 1.0,
"stretch_prob": 0.8,
"v_flip_prob": 0.5,
"yjitter_prob": 1.0
},
"description": "Configuration parameters for data augmentation",
"properties": {
"color_aug_brightness": {
"default": 0.4,
"description": "The color jitter brightness",
"maximum": 1.0,
"minimum": 0.0,
"title": "The color jitter brightness",
"type": "float"
},
"color_aug_contrast": {
"default": 0.4,
"description": "The color jitter contrast",
"maximum": 1.0,
"minimum": 0.0,
"title": "The color jitter contrast",
"type": "float"
},
"color_aug_hue_range": {
"automl_enabled": false,
"default": [
-0.027777777777777776,
0.027777777777777776
],
"description": "The hue range in data augmentation",
"title": "hue range augmentaiton",
"type": "list"
},
"color_aug_prob": {
"automl_enabled": true,
"default": 0.2,
"description": "The probability for asymmetric color augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for asymmetric color augmentation",
"type": "float"
},
"color_aug_saturation": {
"automl_enabled": false,
"default": [
0.0,
1.4
],
"description": "The color jitter saturation",
"title": "The color jitter saturation",
"type": "list"
},
"crop_min_valid_disp_ratio": {
"automl_enabled": true,
"default": 0.0,
"description": "The probability for minimum crop valid disparity ratio",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for min crop valid disparity ratio",
"type": "float"
},
"crop_size": {
"automl_enabled": false,
"default": [
518,
518
],
"description": "The crop size for input RGB images [height, width]",
"title": "augmentation crop size",
"type": "list"
},
"do_flip": {
"default": false,
"description": "A flag specifying whether to perform flip in data augmentation",
"title": "do flip",
"type": "bool"
},
"eraser_aug_prob": {
"automl_enabled": true,
"default": 0.5,
"description": "The probability for eraser augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for eraser augmentation",
"type": "float"
},
"gamma": {
"automl_enabled": false,
"default": [
1,
1,
1,
1
],
"description": "Gamma range in data augmentation",
"title": "gamma range",
"type": "list"
},
"h_flip_prob": {
"automl_enabled": true,
"default": 0.5,
"description": "The probability for horizontal flip augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for horizontal flip augmentation",
"type": "float"
},
"hshift_prob": {
"automl_enabled": true,
"default": 0.5,
"description": "The probability for horizontal shift augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for horizontal flip augmentation",
"type": "float"
},
"input_mean": {
"automl_enabled": false,
"default": [
0.485,
0.456,
0.406
],
"description": "The input mean for RGB frames",
"title": "input mean per pixel",
"type": "list"
},
"input_std": {
"automl_enabled": false,
"default": [
0.229,
0.224,
0.225
],
"description": "The input standard deviation per pixel for RGB frames",
"title": "input std per pixel",
"type": "list"
},
"max_scale": {
"default": 0.4,
"description": "The maximum scale in data augmentation",
"maximum": 1.0,
"minimum": -0.2,
"title": "max scale",
"type": "float"
},
"max_stretch": {
"default": 0.2,
"description": "The maximum stretch augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The maximum stretch augmentation",
"type": "float"
},
"min_scale": {
"default": -0.2,
"description": "The minimum scale in data augmentation",
"maximum": 1.0,
"minimum": 0.2,
"title": "min scale",
"type": "float"
},
"spatial_aug_prob": {
"automl_enabled": true,
"default": 1.0,
"description": "The probability for spatial augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for spatial augmentation",
"type": "float"
},
"stretch_prob": {
"automl_enabled": true,
"default": 0.8,
"description": "The probability for stretch augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for stretch augmentation",
"type": "float"
},
"v_flip_prob": {
"automl_enabled": true,
"default": 0.5,
"description": "The probability for vertical flip augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for vertical flip augmentation",
"type": "float"
},
"yjitter_prob": {
"automl_enabled": true,
"default": 1.0,
"description": "The probability for y jitter",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for y jitter",
"type": "float"
}
},
"title": "augmentation",
"type": "collection"
},
"batch_size": {
"default": 1,
"description": "The batch size for training and validation",
"maximum": Infinity,
"minimum": 1,
"title": "batch size",
"type": "int"
},
"data_sources": {
"automl_enabled": false,
"default": [
{
"data_file": "",
"dataset_name": ""
}
],
"description": "The list of data sources for training:\n * dataset_name : The type of the dataset\n * data_file : The path of the data file",
"title": "train data sources",
"type": "list"
},
"pin_memory": {
"default": true,
"description": "Flag to enable the dataloader to allocated pagelocked memory for faster\n of data between the CPU and GPU.",
"title": "pin_memory",
"type": "bool"
},
"workers": {
"default": 8,
"description": "The number of parallel workers processing data",
"maximum": Infinity,
"minimum": 1,
"title": "batch size",
"type": "int"
}
},
"type": "collection"
},
"train_dataset": {
"automl_disabled_parameters": [
"dataset.train_dataset.data_sources",
"dataset.train_dataset.augmentation"
],
"automl_enabled": false,
"default": {
"augmentation": {
"color_aug_brightness": 0.4,
"color_aug_contrast": 0.4,
"color_aug_hue_range": [
-0.027777777777777776,
0.027777777777777776
],
"color_aug_prob": 0.2,
"color_aug_saturation": [
0.0,
1.4
],
"crop_min_valid_disp_ratio": 0.0,
"crop_size": [
518,
518
],
"do_flip": false,
"eraser_aug_prob": 0.5,
"gamma": [
1,
1,
1,
1
],
"h_flip_prob": 0.5,
"hshift_prob": 0.5,
"input_mean": [
0.485,
0.456,
0.406
],
"input_std": [
0.229,
0.224,
0.225
],
"max_scale": 0.4,
"max_stretch": 0.2,
"min_scale": -0.2,
"spatial_aug_prob": 1.0,
"stretch_prob": 0.8,
"v_flip_prob": 0.5,
"yjitter_prob": 1.0
},
"batch_size": 1,
"data_sources": [
{
"data_file": "",
"dataset_name": ""
}
],
"pin_memory": true,
"workers": 8
},
"description": "Configurable parameters to construct the train dataset for a DepthNet experiment.",
"properties": {
"augmentation": {
"automl_default_parameters": [
"dataset.train_dataset.augmentation.yjitter_prob",
"dataset.train_dataset.augmentation.color_aug_prob",
"dataset.train_dataset.augmentation.eraser_aug_prob",
"dataset.train_dataset.augmentation.spatial_aug_prob",
"dataset.train_dataset.augmentation.stretch_prob",
"dataset.train_dataset.augmentation.h_flip_prob",
"dataset.train_dataset.augmentation.v_flip_prob",
"dataset.train_dataset.augmentation.hshift_prob",
"dataset.train_dataset.augmentation.crop_min_valid_disp_ratio"
],
"automl_disabled_parameters": [
"dataset.train_dataset.augmentation.input_mean",
"dataset.train_dataset.augmentation.input_std",
"dataset.train_dataset.augmentation.crop_size",
"dataset.train_dataset.augmentation.gamma",
"dataset.train_dataset.augmentation.color_aug_saturation",
"dataset.train_dataset.augmentation.color_aug_hue_range"
],
"automl_enabled": false,
"default": {
"color_aug_brightness": 0.4,
"color_aug_contrast": 0.4,
"color_aug_hue_range": [
-0.027777777777777776,
0.027777777777777776
],
"color_aug_prob": 0.2,
"color_aug_saturation": [
0.0,
1.4
],
"crop_min_valid_disp_ratio": 0.0,
"crop_size": [
518,
518
],
"do_flip": false,
"eraser_aug_prob": 0.5,
"gamma": [
1,
1,
1,
1
],
"h_flip_prob": 0.5,
"hshift_prob": 0.5,
"input_mean": [
0.485,
0.456,
0.406
],
"input_std": [
0.229,
0.224,
0.225
],
"max_scale": 0.4,
"max_stretch": 0.2,
"min_scale": -0.2,
"spatial_aug_prob": 1.0,
"stretch_prob": 0.8,
"v_flip_prob": 0.5,
"yjitter_prob": 1.0
},
"description": "Configuration parameters for data augmentation",
"properties": {
"color_aug_brightness": {
"default": 0.4,
"description": "The color jitter brightness",
"maximum": 1.0,
"minimum": 0.0,
"title": "The color jitter brightness",
"type": "float"
},
"color_aug_contrast": {
"default": 0.4,
"description": "The color jitter contrast",
"maximum": 1.0,
"minimum": 0.0,
"title": "The color jitter contrast",
"type": "float"
},
"color_aug_hue_range": {
"automl_enabled": false,
"default": [
-0.027777777777777776,
0.027777777777777776
],
"description": "The hue range in data augmentation",
"title": "hue range augmentaiton",
"type": "list"
},
"color_aug_prob": {
"automl_enabled": true,
"default": 0.2,
"description": "The probability for asymmetric color augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for asymmetric color augmentation",
"type": "float"
},
"color_aug_saturation": {
"automl_enabled": false,
"default": [
0.0,
1.4
],
"description": "The color jitter saturation",
"title": "The color jitter saturation",
"type": "list"
},
"crop_min_valid_disp_ratio": {
"automl_enabled": true,
"default": 0.0,
"description": "The probability for minimum crop valid disparity ratio",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for min crop valid disparity ratio",
"type": "float"
},
"crop_size": {
"automl_enabled": false,
"default": [
518,
518
],
"description": "The crop size for input RGB images [height, width]",
"title": "augmentation crop size",
"type": "list"
},
"do_flip": {
"default": false,
"description": "A flag specifying whether to perform flip in data augmentation",
"title": "do flip",
"type": "bool"
},
"eraser_aug_prob": {
"automl_enabled": true,
"default": 0.5,
"description": "The probability for eraser augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for eraser augmentation",
"type": "float"
},
"gamma": {
"automl_enabled": false,
"default": [
1,
1,
1,
1
],
"description": "Gamma range in data augmentation",
"title": "gamma range",
"type": "list"
},
"h_flip_prob": {
"automl_enabled": true,
"default": 0.5,
"description": "The probability for horizontal flip augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for horizontal flip augmentation",
"type": "float"
},
"hshift_prob": {
"automl_enabled": true,
"default": 0.5,
"description": "The probability for horizontal shift augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for horizontal flip augmentation",
"type": "float"
},
"input_mean": {
"automl_enabled": false,
"default": [
0.485,
0.456,
0.406
],
"description": "The input mean for RGB frames",
"title": "input mean per pixel",
"type": "list"
},
"input_std": {
"automl_enabled": false,
"default": [
0.229,
0.224,
0.225
],
"description": "The input standard deviation per pixel for RGB frames",
"title": "input std per pixel",
"type": "list"
},
"max_scale": {
"default": 0.4,
"description": "The maximum scale in data augmentation",
"maximum": 1.0,
"minimum": -0.2,
"title": "max scale",
"type": "float"
},
"max_stretch": {
"default": 0.2,
"description": "The maximum stretch augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The maximum stretch augmentation",
"type": "float"
},
"min_scale": {
"default": -0.2,
"description": "The minimum scale in data augmentation",
"maximum": 1.0,
"minimum": 0.2,
"title": "min scale",
"type": "float"
},
"spatial_aug_prob": {
"automl_enabled": true,
"default": 1.0,
"description": "The probability for spatial augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for spatial augmentation",
"type": "float"
},
"stretch_prob": {
"automl_enabled": true,
"default": 0.8,
"description": "The probability for stretch augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for stretch augmentation",
"type": "float"
},
"v_flip_prob": {
"automl_enabled": true,
"default": 0.5,
"description": "The probability for vertical flip augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for vertical flip augmentation",
"type": "float"
},
"yjitter_prob": {
"automl_enabled": true,
"default": 1.0,
"description": "The probability for y jitter",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for y jitter",
"type": "float"
}
},
"title": "augmentation",
"type": "collection"
},
"batch_size": {
"default": 1,
"description": "The batch size for training and validation",
"maximum": Infinity,
"minimum": 1,
"title": "batch size",
"type": "int"
},
"data_sources": {
"automl_enabled": false,
"default": [
{
"data_file": "",
"dataset_name": ""
}
],
"description": "The list of data sources for training:\n * dataset_name : The type of the dataset\n * data_file : The path of the data file",
"title": "train data sources",
"type": "list"
},
"pin_memory": {
"default": true,
"description": "Flag to enable the dataloader to allocated pagelocked memory for faster\n of data between the CPU and GPU.",
"title": "pin_memory",
"type": "bool"
},
"workers": {
"default": 8,
"description": "The number of parallel workers processing data",
"maximum": Infinity,
"minimum": 1,
"title": "batch size",
"type": "int"
}
},
"type": "collection"
},
"val_dataset": {
"automl_disabled_parameters": [
"dataset.val_dataset.data_sources",
"dataset.val_dataset.augmentation"
],
"automl_enabled": false,
"default": {
"augmentation": {
"color_aug_brightness": 0.4,
"color_aug_contrast": 0.4,
"color_aug_hue_range": [
-0.027777777777777776,
0.027777777777777776
],
"color_aug_prob": 0.2,
"color_aug_saturation": [
0.0,
1.4
],
"crop_min_valid_disp_ratio": 0.0,
"crop_size": [
518,
518
],
"do_flip": false,
"eraser_aug_prob": 0.5,
"gamma": [
1,
1,
1,
1
],
"h_flip_prob": 0.5,
"hshift_prob": 0.5,
"input_mean": [
0.485,
0.456,
0.406
],
"input_std": [
0.229,
0.224,
0.225
],
"max_scale": 0.4,
"max_stretch": 0.2,
"min_scale": -0.2,
"spatial_aug_prob": 1.0,
"stretch_prob": 0.8,
"v_flip_prob": 0.5,
"yjitter_prob": 1.0
},
"batch_size": 1,
"data_sources": [
{
"data_file": "",
"dataset_name": ""
}
],
"pin_memory": true,
"workers": 8
},
"description": "Configurable parameters to construct the val dataset for a DepthNet experiment.",
"properties": {
"augmentation": {
"automl_default_parameters": [
"dataset.val_dataset.augmentation.yjitter_prob",
"dataset.val_dataset.augmentation.color_aug_prob",
"dataset.val_dataset.augmentation.eraser_aug_prob",
"dataset.val_dataset.augmentation.spatial_aug_prob",
"dataset.val_dataset.augmentation.stretch_prob",
"dataset.val_dataset.augmentation.h_flip_prob",
"dataset.val_dataset.augmentation.v_flip_prob",
"dataset.val_dataset.augmentation.hshift_prob",
"dataset.val_dataset.augmentation.crop_min_valid_disp_ratio"
],
"automl_disabled_parameters": [
"dataset.val_dataset.augmentation.input_mean",
"dataset.val_dataset.augmentation.input_std",
"dataset.val_dataset.augmentation.crop_size",
"dataset.val_dataset.augmentation.gamma",
"dataset.val_dataset.augmentation.color_aug_saturation",
"dataset.val_dataset.augmentation.color_aug_hue_range"
],
"automl_enabled": false,
"default": {
"color_aug_brightness": 0.4,
"color_aug_contrast": 0.4,
"color_aug_hue_range": [
-0.027777777777777776,
0.027777777777777776
],
"color_aug_prob": 0.2,
"color_aug_saturation": [
0.0,
1.4
],
"crop_min_valid_disp_ratio": 0.0,
"crop_size": [
518,
518
],
"do_flip": false,
"eraser_aug_prob": 0.5,
"gamma": [
1,
1,
1,
1
],
"h_flip_prob": 0.5,
"hshift_prob": 0.5,
"input_mean": [
0.485,
0.456,
0.406
],
"input_std": [
0.229,
0.224,
0.225
],
"max_scale": 0.4,
"max_stretch": 0.2,
"min_scale": -0.2,
"spatial_aug_prob": 1.0,
"stretch_prob": 0.8,
"v_flip_prob": 0.5,
"yjitter_prob": 1.0
},
"description": "Configuration parameters for data augmentation",
"properties": {
"color_aug_brightness": {
"default": 0.4,
"description": "The color jitter brightness",
"maximum": 1.0,
"minimum": 0.0,
"title": "The color jitter brightness",
"type": "float"
},
"color_aug_contrast": {
"default": 0.4,
"description": "The color jitter contrast",
"maximum": 1.0,
"minimum": 0.0,
"title": "The color jitter contrast",
"type": "float"
},
"color_aug_hue_range": {
"automl_enabled": false,
"default": [
-0.027777777777777776,
0.027777777777777776
],
"description": "The hue range in data augmentation",
"title": "hue range augmentaiton",
"type": "list"
},
"color_aug_prob": {
"automl_enabled": true,
"default": 0.2,
"description": "The probability for asymmetric color augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for asymmetric color augmentation",
"type": "float"
},
"color_aug_saturation": {
"automl_enabled": false,
"default": [
0.0,
1.4
],
"description": "The color jitter saturation",
"title": "The color jitter saturation",
"type": "list"
},
"crop_min_valid_disp_ratio": {
"automl_enabled": true,
"default": 0.0,
"description": "The probability for minimum crop valid disparity ratio",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for min crop valid disparity ratio",
"type": "float"
},
"crop_size": {
"automl_enabled": false,
"default": [
518,
518
],
"description": "The crop size for input RGB images [height, width]",
"title": "augmentation crop size",
"type": "list"
},
"do_flip": {
"default": false,
"description": "A flag specifying whether to perform flip in data augmentation",
"title": "do flip",
"type": "bool"
},
"eraser_aug_prob": {
"automl_enabled": true,
"default": 0.5,
"description": "The probability for eraser augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for eraser augmentation",
"type": "float"
},
"gamma": {
"automl_enabled": false,
"default": [
1,
1,
1,
1
],
"description": "Gamma range in data augmentation",
"title": "gamma range",
"type": "list"
},
"h_flip_prob": {
"automl_enabled": true,
"default": 0.5,
"description": "The probability for horizontal flip augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for horizontal flip augmentation",
"type": "float"
},
"hshift_prob": {
"automl_enabled": true,
"default": 0.5,
"description": "The probability for horizontal shift augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for horizontal flip augmentation",
"type": "float"
},
"input_mean": {
"automl_enabled": false,
"default": [
0.485,
0.456,
0.406
],
"description": "The input mean for RGB frames",
"title": "input mean per pixel",
"type": "list"
},
"input_std": {
"automl_enabled": false,
"default": [
0.229,
0.224,
0.225
],
"description": "The input standard deviation per pixel for RGB frames",
"title": "input std per pixel",
"type": "list"
},
"max_scale": {
"default": 0.4,
"description": "The maximum scale in data augmentation",
"maximum": 1.0,
"minimum": -0.2,
"title": "max scale",
"type": "float"
},
"max_stretch": {
"default": 0.2,
"description": "The maximum stretch augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The maximum stretch augmentation",
"type": "float"
},
"min_scale": {
"default": -0.2,
"description": "The minimum scale in data augmentation",
"maximum": 1.0,
"minimum": 0.2,
"title": "min scale",
"type": "float"
},
"spatial_aug_prob": {
"automl_enabled": true,
"default": 1.0,
"description": "The probability for spatial augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for spatial augmentation",
"type": "float"
},
"stretch_prob": {
"automl_enabled": true,
"default": 0.8,
"description": "The probability for stretch augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for stretch augmentation",
"type": "float"
},
"v_flip_prob": {
"automl_enabled": true,
"default": 0.5,
"description": "The probability for vertical flip augmentation",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for vertical flip augmentation",
"type": "float"
},
"yjitter_prob": {
"automl_enabled": true,
"default": 1.0,
"description": "The probability for y jitter",
"maximum": 1.0,
"minimum": 0.0,
"title": "The probability for y jitter",
"type": "float"
}
},
"title": "augmentation",
"type": "collection"
},
"batch_size": {
"default": 1,
"description": "The batch size for training and validation",
"maximum": Infinity,
"minimum": 1,
"title": "batch size",
"type": "int"
},
"data_sources": {
"automl_enabled": false,
"default": [
{
"data_file": "",
"dataset_name": ""
}
],
"description": "The list of data sources for training:\n * dataset_name : The type of the dataset\n * data_file : The path of the data file",
"title": "train data sources",
"type": "list"
},
"pin_memory": {
"default": true,
"description": "Flag to enable the dataloader to allocated pagelocked memory for faster\n of data between the CPU and GPU.",
"title": "pin_memory",
"type": "bool"
},
"workers": {
"default": 8,
"description": "The number of parallel workers processing data",
"maximum": Infinity,
"minimum": 1,
"title": "batch size",
"type": "int"
}
},
"type": "collection"
}
},
"type": "collection"
},
"encryption_key": {
"default": "",
"description": "Key for encrypting model checkpoints",
"title": "Encryption key",
"type": "string"
},
"inference": {
"automl_disabled_parameters": [
"inference.gpu_ids"
],
"automl_enabled": false,
"default": {
"batch_size": -1,
"checkpoint": "???",
"conf_threshold": 0.5,
"gpu_ids": [
0
],
"num_gpus": 1,
"num_nodes": 1,
"results_dir": "",
"save_raw_pfm": false,
"trt_engine": ""
},
"description": "Configurable parameters to construct the inferencer for a DepthNet experiment.",
"popular": [
"num_gpus",
"num_nodes",
"gpu_ids"
],
"properties": {
"batch_size": {
"default": -1,
"description": "The batch size of the input Tensor. This is important if batch_size > 1 for large dataset.",
"minimum": -1,
"title": "batch size",
"type": "int"
},
"checkpoint": {
"default": "???",
"description": "Path to the checkpoint used for inference.",
"title": "Checkpoint path",
"type": "string"
},
"conf_threshold": {
"default": 0.5,
"description": "The value of the confidence threshold to be used when\n filtering out the final list of boxes.",
"title": "confidence threshold",
"type": "float"
},
"gpu_ids": {
"automl_enabled": false,
"default": [
0
],
"description": "\n List of GPU IDs to run the inference on. The length of this list\n must be equal to the number of gpus in inference.num_gpus.",
"popular": true,
"title": "GPU IDs",
"type": "list"
},
"input_height": {
"description": "Height of the input image tensor.",
"minimum": 1,
"title": "input height",
"type": "int"
},
"input_width": {
"description": "Width of the input image tensor.",
"minimum": 1,
"title": "input width",
"type": "int"
},
"num_gpus": {
"default": 1,
"description": "The number of GPUs to run the inference job.",
"minimum": 1,
"popular": true,
"title": "Number of GPUs",
"type": "int"
},
"num_nodes": {
"default": 1,
"description": "Number of nodes to run the inference on. If > 1, then multi-node is enabled.",
"minimum": 1,
"popular": true,
"title": "Number of nodes",
"type": "int"
},
"results_dir": {
"default": "",
"description": "\n Path to where all the assets generated from a task are stored.\n ",
"title": "Results directory",
"type": "string"
},
"save_raw_pfm": {
"default": false,
"description": "Whether to save the raw pfm output during inference.",
"title": "Save PFM Output",
"type": "bool"
},
"trt_engine": {
"default": "",
"description": "Path to the TensorRT engine to be used for inference.\n This only works with :code:`tao-deploy`.",
"title": "TensorRT Engine",
"type": "string"
}
},
"type": "collection"
},
"model": {
"automl_default_parameters": [
"model.corr_radius",
"model.cv_group",
"model.volume_dim"
],
"automl_disabled_parameters": [
"model.mono_backbone",
"model.stereo_backbone",
"model.hidden_dims"
],
"automl_enabled": false,
"default": {
"corr_levels": 2,
"corr_radius": 4,
"cv_group": 8,
"encoder": "vitl",
"hidden_dims": [
128,
128,
128
],
"low_memory": 0,
"max_disparity": 416,
"mixed_precision": false,
"model_type": "MetricDepthAnything",
"mono_backbone": {
"pretrained_path": "",
"use_bn": false,
"use_clstoken": false
},
"n_downsample": 2,
"n_gru_layers": 3,
"stereo_backbone": {
"depth_anything_v2_pretrained_path": "",
"edgenext_pretrained_path": "",
"use_bn": false,
"use_clstoken": false
},
"train_iters": 22,
"valid_iters": 22,
"volume_dim": 32
},
"description": "Configurable parameters to construct the model for a DepthNet experiment.",
"properties": {
"corr_levels": {
"default": 2,
"description": "The number of levels in the correlation pyramid",
"maximum": 2,
"minimum": 1,
"title": "number of correlation pyramid levels",
"type": "int"
},
"corr_radius": {
"automl_enabled": true,
"default": 4,
"description": "The width of the correlation pyramid",
"maximum": 8,
"minimum": 2,
"title": "correlation pyramid width",
"type": "int"
},
"cv_group": {
"automl_enabled": true,
"default": 8,
"description": "cv group",
"maximum": 16,
"minimum": 4,
"title": "cv group",
"type": "int"
},
"encoder": {
"default": "vitl",
"description": "DepthAnythingV2 Encoder options",
"enum": [
"vits",
"vitb",
"vitl",
"vitg"
],
"type": "categorical"
},
"hidden_dims": {
"automl_enabled": false,
"default": [
128,
128,
128
],
"description": "The hidden dimensions.",
"title": "The hidden dimensions.",
"type": "list"
},
"low_memory": {
"default": 0,
"description": "reduce memory usage",
"maximum": 4,
"minimum": 0,
"title": "reduce memory usage",
"type": "int"
},
"max_disparity": {
"default": 416,
"description": "\n The maximum disparity of the model used in the training of a stereo model\n ",
"title": "max disparity",
"type": "int"
},
"mixed_precision": {
"default": false,
"description": "A flag specifying whether to use mixed precision training",
"title": "Mixed Precision Training",
"type": "bool"
},
"model_type": {
"default": "MetricDepthAnything",
"description": "Network name",
"enum": [
"FoundationStereo",
"MetricDepthAnything",
"RelativeDepthAnything"
],
"type": "categorical"
},
"mono_backbone": {
"automl_enabled": false,
"default": {
"pretrained_path": "",
"use_bn": false,
"use_clstoken": false
},
"description": "Network defined paths for Monocular DepthNet Backbone",
"properties": {
"pretrained_path": {
"default": "",
"description": "Path to load depth anything v2 as an encoder for Monocular DepthNet",
"title": "Pretrained path for mono backbone",
"type": "string"
},
"use_bn": {
"default": false,
"description": "A flag specifying whether to use batch normalization in Monocular DepthNet",
"title": "Batch normalization in Monocular DepthNet",
"type": "bool"
},
"use_clstoken": {
"default": false,
"description": "A flag specifying whether to use class token",
"title": "Class token in Monocular DepthNet",
"type": "bool"
}
},
"title": "Mono backbone configuration",
"type": "collection"
},
"n_downsample": {
"default": 2,
"description": "resolution of the disparity field (1/2^K)",
"maximum": 2,
"minimum": 1,
"title": "disparity field resoultion",
"type": "int"
},
"n_gru_layers": {
"default": 3,
"description": "The number of hidden GRU levels",
"maximum": 3,
"minimum": 1,
"title": "number of hidden GRU levels",
"type": "int"
},
"stereo_backbone": {
"automl_enabled": false,
"default": {
"depth_anything_v2_pretrained_path": "",
"edgenext_pretrained_path": "",
"use_bn": false,
"use_clstoken": false
},
"description": "Network defined paths for Edgenext and Depthanythingv2",
"properties": {
"depth_anything_v2_pretrained_path": {
"default": "",
"description": "Path to load depth anything v2 as an encoder for Stereo DepthNet (FoundationStereo)",
"type": "string"
},
"edgenext_pretrained_path": {
"default": "",
"description": "Path to load edgenext encoder for Stereo DepthNet (FoundationStereo)",
"type": "string"
},
"use_bn": {
"default": false,
"description": "A flag specifying whether to use batch normalization in DepthAnythingV2",
"title": "batch normalization in DepthAnythingV2",
"type": "bool"
},
"use_clstoken": {
"default": false,
"description": "A flag specifying whether to use class token",
"title": "class token in DepthAnythingV2",
"type": "bool"
}
},
"title": "Stereo backbone configuration",
"type": "collection"
},
"train_iters": {
"default": 22,
"description": "Train Iteration",
"minimum": 1,
"title": "train iteration",
"type": "int"
},
"valid_iters": {
"default": 22,
"description": "Validation Iteration",
"minimum": 1,
"title": "Validation iteration",
"type": "int"
},
"volume_dim": {
"automl_enabled": true,
"default": 32,
"description": "Volume dimension",
"maximum": 64,
"minimum": 16,
"title": "volume dimension",
"type": "int"
}
},
"type": "collection"
},
"model_name": {
"default": "",
"description": "Name of model if invoking task via :code:`model_agnostic`",
"title": "Model name",
"type": "string"
},
"quantize": {
"automl_disabled_parameters": [
"quantize.layers",
"quantize.skip_names",
"quantize.backend_kwargs"
],
"automl_enabled": false,
"default": {
"algorithm": "minmax",
"backend": "torchao",
"backend_kwargs": {},
"device": "cuda",
"layers": [],
"mode": "weight_only_ptq",
"model_path": "",
"results_dir": "",
"skip_names": []
},
"description": "Configurable parameters for model quantization.",
"properties": {
"algorithm": {
"default": "minmax",
"description": "Calibration/optimization algorithm. Used by ModelOpt backends (modelopt.pytorch and modelopt.onnx). Ignored by torchao backend.",
"enum": [
"minmax",
"max",
"entropy",
"awq_clip",
"awq_lite",
"awq_full",
"rtn_dq"
],
"title": "Calibration algorithm",
"type": "categorical"
},
"backend": {
"default": "torchao",
"description": "The quantization backend to use",
"enum": [
"modelopt.pytorch",
"torchao",
"modelopt.onnx"
],
"title": "Quantization backend",
"type": "categorical"
},
"backend_kwargs": {
"automl_enabled": false,
"description": "Additional keyword arguments to pass to the backend",
"title": "Backend kwargs",
"type": "collection"
},
"device": {
"default": "cuda",
"description": "Device to use for calibration. Accepts 'cuda' (uses default GPU), 'cpu', 'trt' (TensorRT for ONNX backend), or specific GPU device like 'cuda:0', 'cuda:1', etc. If 'cuda' is specified but no GPU is available, the framework will automatically fall back to 'cpu'. Note: 'trt' is only supported by the modelopt.onnx backend for ONNX Runtime with TensorRT execution provider.",
"pattern": "^(cuda|cpu|trt|cuda:[0-9]+)$",
"title": "Calibration device",
"type": "string"
},
"layers": {
"automl_enabled": false,
"default": [],
"description": "List of per-module quantization configurations. Each entry specifies which modules to quantize and their data types. This is the primary way to configure quantization.",
"title": "Layer quantization configs",
"type": "list"
},
"mode": {
"default": "weight_only_ptq",
"description": "The quantization mode to use",
"enum": [
"static_ptq",
"weight_only_ptq"
],
"title": "Quantization mode",
"type": "categorical"
},
"model_path": {
"default": "",
"description": "Path to the model to be quantized. For ONNX backend, path to ONNX file.",
"title": "Model path",
"type": "string"
},
"results_dir": {
"default": "",
"description": "Path to where all the assets generated from a task are stored.",
"title": "Results directory",
"type": "string"
},
"skip_names": {
"automl_enabled": false,
"default": [],
"description": "List of module/layer names or patterns to exclude from quantization",
"title": "Skip names",
"type": "list"
}
},
"required": [
"model_path",
"results_dir"
],
"type": "collection"
},
"results_dir": {
"default": "",
"description": "\n Path to where all the assets generated from a task are stored.\n ",
"title": "Results directory",
"type": "string"
},
"train": {
"automl_disabled_parameters": [
"train.gpu_ids",
"train.cudnn",
"train.optim",
"train.tile_min_overlap"
],
"automl_enabled": false,
"default": {
"activation_checkpoint": false,
"checkpoint_interval": 1,
"checkpoint_interval_unit": "epoch",
"clip_grad_norm": 0.1,
"cudnn": {
"benchmark": false,
"deterministic": true
},
"dataloader_visualize": false,
"distributed_strategy": "ddp",
"gpu_ids": [
0
],
"inference_tile": false,
"is_dry_run": false,
"log_every_n_steps": 500,
"num_epochs": 10,
"num_gpus": 1,
"num_nodes": 1,
"optim": {
"lr": 0.0001,
"lr_decay": 0.1,
"lr_scheduler": "MultiStepLR",
"lr_step_size": 1000,
"lr_steps": [
1000
],
"min_lr": 1e-07,
"momentum": 0.9,
"monitor_name": "val_loss",
"optimizer": "AdamW",
"warmup_steps": 20,
"weight_decay": 0.0001
},
"precision": "fp32",
"pretrained_model_path": "",
"results_dir": "",
"resume_training_checkpoint_path": "",
"seed": 1234,
"tile_min_overlap": [
16,
16
],
"tile_wtype": "gaussian",
"validation_interval": 1,
"vis_step_interval": 10
},
"description": "Configurable parameters to construct the trainer for a DepthNet experiment.",
"popular": [
"validation_interval",
"checkpoint_interval",
"num_gpus",
"num_nodes",
"num_epochs",
"gpu_ids"
],
"properties": {
"activation_checkpoint": {
"default": false,
"description": "\n A True value instructs train to recompute in backward pass to save GPU memory,\n rather than storing activations.",
"title": "enable activation checkpointing",
"type": "bool"
},
"checkpoint_interval": {
"default": 1,
"description": "The interval (in epochs) at which a checkpoint will be saved. Helps resume training.",
"minimum": 1,
"popular": true,
"title": "Checkpoint interval",
"type": "int"
},
"checkpoint_interval_steps": {
"description": "The number of steps to save the checkpoint.",
"title": "checkpoint interval steps",
"type": "int"
},
"checkpoint_interval_unit": {
"default": "epoch",
"description": "The unit of the checkpoint interval.",
"enum": [
"epoch",
"step"
],
"title": "Checkpoint interval unit",
"type": "categorical"
},
"clip_grad_norm": {
"default": 0.1,
"description": "\n Amount to clip the gradient by L2 Norm.\n A value of 0.0 specifies no clipping.",
"math_cond": "> 0.0",
"title": "clip gradient norm",
"type": "float"
},
"cudnn": {
"automl_enabled": false,
"default": {
"benchmark": false,
"deterministic": true
},
"properties": {
"benchmark": {
"default": false,
"type": "bool"
},
"deterministic": {
"default": true,
"type": "bool"
}
},
"type": "collection"
},
"dataloader_visualize": {
"default": false,
"description": "Whether to visualize the dataloader.",
"title": "dataloader visualize",
"type": "bool"
},
"distributed_strategy": {
"default": "ddp",
"description": "\n The multi-GPU training strategy.\n DDP (Distributed Data Parallel) and Fully Sharded DDP are supported.",
"enum": [
"ddp",
"fsdp"
],
"title": "distributed_strategy",
"type": "categorical"
},
"gpu_ids": {
"automl_enabled": false,
"default": [
0
],
"description": "\n List of GPU IDs to run the training on. The length of this list\n must be equal to the number of gpus in train.num_gpus.",
"popular": true,
"title": "GPU IDs",
"type": "list"
},
"inference_tile": {
"default": false,
"description": "Use tiled inference, particularly for transformers\n which expect fixed size of sequences.\n ",
"title": "tile inference",
"type": "bool"
},
"is_dry_run": {
"default": false,
"description": "\n Whether to run the trainer in Dry Run mode. This serves\n as a good means to validate the spec file and run a sanity check on the trainer\n without actually initializing and running the trainer.",
"title": "Is dry run",
"type": "bool"
},
"log_every_n_steps": {
"default": 500,
"description": "\n Interval steps of logging training results and running validation numbers within 1 epoch",
"title": "log steps",
"type": "int"
},
"num_epochs": {
"default": 10,
"description": "Number of epochs to run the training.",
"maximum": Infinity,
"minimum": 1,
"popular": true,
"title": "Number of epochs",
"type": "int"
},
"num_gpus": {
"default": 1,
"description": "The number of GPUs to run the train job.",
"minimum": 1,
"popular": true,
"title": "Number of GPUs",
"type": "int"
},
"num_nodes": {
"default": 1,
"description": "Number of nodes to run the training on. If > 1, then multi-node is enabled.",
"minimum": 1,
"popular": true,
"title": "Number of nodes",
"type": "int"
},
"optim": {
"automl_default_parameters": [
"train.optim.lr",
"train.optim.momentum",
"train.optim.weight_decay",
"train.optim.lr_step_size",
"train.optim.lr_decay",
"train.optim.min_lr"
],
"automl_disabled_parameters": [
"train.optim.lr_steps"
],
"automl_enabled": false,
"default": {
"lr": 0.0001,
"lr_decay": 0.1,
"lr_scheduler": "MultiStepLR",
"lr_step_size": 1000,
"lr_steps": [
1000
],
"min_lr": 1e-07,
"momentum": 0.9,
"monitor_name": "val_loss",
"optimizer": "AdamW",
"warmup_steps": 20,
"weight_decay": 0.0001
},
"description": "Hyper parameters to configure the optimizer.",
"properties": {
"lr": {
"automl_enabled": true,
"default": 0.0001,
"description": "The initial learning rate for training the model, excluding the backbone.",
"math_cond": "> 0.0",
"maximum": 0.01,
"minimum": 1e-06,
"title": "learning rate",
"type": "float"
},
"lr_decay": {
"automl_enabled": true,
"default": 0.1,
"description": "The decreasing factor for the learning rate scheduler.",
"math_cond": "> 0.0",
"maximum": 1.0,
"minimum": 0.0,
"title": "learning rate decay",
"type": "float"
},
"lr_scheduler": {
"default": "MultiStepLR",
"description": "The learning scheduler:\n * MultiStepLR : Decrease the lr by lr_decay from lr_steps\n * StepLR : Decrease the lr by lr_decay at every lr_step_size.",
"enum": [
"MultiStep",
"StepLR",
"CustomMultiStepLRScheduler",
"LambdaLR",
"PolynomialLR",
"OneCycleLR",
"CosineAnnealingLR"
],
"title": "Learning rate scheduler",
"type": "categorical"
},
"lr_step_size": {
"automl_enabled": true,
"default": 1000,
"description": "The number of steps to decrease the learning rate in the StepLR.",
"math_cond": "> 0",
"maximum": 10000,
"minimum": 1,
"title": "learning rate step size",
"type": "int"
},
"lr_steps": {
"automl_enabled": false,
"default": [
1000
],
"description": "The steps at which the learning rate must be decreased.\n This is applicable only with the MultiStep LR.",
"title": "learning rate decay steps",
"type": "list"
},
"min_lr": {
"automl_enabled": true,
"default": 1e-07,
"description": "The minimum learning rate value for the learning rate scheduler.",
"math_cond": "> 0.0",
"maximum": 0.001,
"minimum": 1e-08,
"title": "minimum learning rate",
"type": "float"
},
"momentum": {
"automl_enabled": true,
"default": 0.9,
"description": "The momentum for the AdamW optimizer.",
"math_cond": "> 0.0",
"maximum": 1.0,
"minimum": 0.0,
"title": "momentum - AdamW",
"type": "float"
},
"monitor_name": {
"default": "val_loss",
"description": "The metric value to be monitored for the :code:`AutoReduce` Scheduler.",
"enum": [
"val_loss",
"train_loss"
],
"title": "monitor_name",
"type": "categorical"
},
"optimizer": {
"default": "AdamW",
"description": "Type of optimizer used to train the network.",
"enum": [
"AdamW",
"SGD"
],
"type": "categorical"
},
"warmup_steps": {
"default": 20,
"description": "The number of steps to perform linear learning rate\" warm-up before engaging a learning rate scheduler",
"maximum": Infinity,
"minimum": 0,
"title": "Warm up steps",
"type": "int"
},
"weight_decay": {
"automl_enabled": true,
"default": 0.0001,
"description": "The weight decay coefficient.",
"math_cond": "> 0.0",
"maximum": 0.01,
"minimum": 1e-06,
"title": "weight decay",
"type": "float"
}
},
"title": "optimizer",
"type": "collection"
},
"precision": {
"default": "fp32",
"description": "Precision to run the training on.",
"enum": [
"bf16",
"fp32",
"fp16"
],
"title": "precision",
"type": "categorical"
},
"pretrained_model_path": {
"default": "",
"description": "Path to a pre-trained DepthNet model to initialize the current training from.",
"type": "string"
},
"results_dir": {
"default": "",
"description": "\n Path to where all the assets generated from a task are stored.\n ",
"title": "Results directory",
"type": "string"
},
"resume_training_checkpoint_path": {
"default": "",
"description": "Path to the checkpoint to resume training from.",
"title": "Resume checkpoint path",
"type": "string"
},
"seed": {
"default": 1234,
"description": "The seed for the initializer in PyTorch. If < 0, disable fixed seed.",
"maximum": Infinity,
"minimum": -1,
"title": "Seed for randomization",
"type": "int"
},
"tile_min_overlap": {
"automl_enabled": false,
"default": [
16,
16
],
"description": "Use tiled inference weight type",
"title": "tile weight type",
"type": "list"
},
"tile_wtype": {
"default": "gaussian",
"description": "Use tiled inference weight type",
"title": "tile weight type",
"type": "string"
},
"validation_interval": {
"default": 1,
"description": "\n The interval (in epochs) at which a evaluation\n will be triggered on the validation dataset.",
"minimum": 1,
"popular": true,
"title": "Validation interval",
"type": "int"
},
"vis_step_interval": {
"default": 10,
"description": "The visualization interval in step.",
"title": "visualization interval",
"type": "int"
}
},
"type": "collection"
},
"wandb": {
"automl_disabled_parameters": [
"wandb.tags"
],
"automl_enabled": false,
"default": {
"enable": true,
"entity": "",
"group": "",
"name": "TAO Toolkit Training",
"project": "TAO Toolkit",
"reinit": false,
"run_id": "",
"save_code": false,
"sync_tensorboard": false,
"tags": [
"tao-toolkit"
]
},
"description": "Configurable parameters to construct the wandb client for a DepthNet experiment.",
"properties": {
"enable": {
"default": true,
"type": "bool"
},
"entity": {
"default": "",
"type": "string"
},
"group": {
"default": "",
"type": "string"
},
"name": {
"default": "TAO Toolkit Training",
"type": "string"
},
"project": {
"default": "TAO Toolkit",
"type": "string"
},
"reinit": {
"default": false,
"type": "bool"
},
"run_id": {
"default": "",
"type": "string"
},
"save_code": {
"default": false,
"type": "bool"
},
"sync_tensorboard": {
"default": false,
"type": "bool"
},
"tags": {
"automl_enabled": false,
"default": [
"tao-toolkit"
],
"type": "list"
}
},
"type": "collection"
}
},
"required": [
"quantize"
],
"type": "object",
"x_tao_schema": {
"action": "inference",
"core_module": "depth_net",
"model": "depth-net-mono",
"network_arch": "depth_net_mono",
"schema_action": "inference",
"schema_version": 1,
"source": "tao-core dataclass config"
}
}
Requirements and compatibility
Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta
“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.
Needs: Declares allowed tools, Needs a command-line tool, Needs Python, Needs Node.js
The author says: Requires docker + nvidia-container-toolkit.
Security
What our scanner noticed· Medium risk
- Large base64-encoded blobs
A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.
Attribution and licence
- Created by
- NVIDIA Corporation
- Publisher
- NVIDIA (Official)
- Source
- NVIDIA/skills · skills/tao-train-depth-anything-v2 · commit fce3cca
- Licence
- Apache License 2.0 (redistribution allowed)
- LICENSE (added by AutoGineer from LICENSE-APACHE)
- ATTRIBUTION.md (written by AutoGineer)
- Synced
AutoGineer is not affiliated with or endorsed by NVIDIA Corporation or NVIDIA.
Related skills
Pytorch Lightning
K-Dense· Community
A framework for organizing PyTorch code to streamline neural network training, scaling, and experiment tracking.
Data and AIClaude CodeClaude appsOpenAI Codex+14 moreMITLow riskHas scriptsGitHub stars of the source repository56.4k
Aeon
K-Dense· Community
A scikit-learn compatible Python toolkit for comprehensive time series machine learning tasks.
Data and AIClaude CodeClaude appsOpenAI Codex+14 moreMITLow riskGitHub stars of the source repository56.4k
Umap Learn
K-Dense· Community
Integrate UMAP-learn for non-linear dimensionality reduction, visualization, and clustering preprocessing.
Data and AIClaude CodeClaude appsOpenAI Codex+14 moreMITLow riskGitHub stars of the source repository56.4k
Scikit Learn
K-Dense· Community
Comprehensive guidance for classical machine learning workflows using the scikit-learn Python library.
Data and AIClaude CodeClaude appsOpenAI Codex+14 moreMITLow riskHas scriptsGitHub stars of the source repository56.4k
Shap
K-Dense· Community
Explain and audit machine-learning model predictions using the SHAP framework for feature attribution and visualization.
Data and AIClaude CodeClaude appsOpenAI Codex+14 moreMITLow riskHas scriptsGitHub stars of the source repository56.4k
Pufferlib
K-Dense· Community
Version-aware guidance for PufferLib reinforcement learning environments, vectorization, policies, and training.
Data and AIClaude CodeClaude appsOpenAI Codex+14 moreMITLow riskHas scriptsGitHub stars of the source repository56.4k
Download ZIP
The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download tao-train-depth-anything-v2.zip