AutoGineer
Generated banner for Cdf Auto Labeling
Generated logo for Cdf Auto Labeling

Cdf Auto Labeling

NVIDIAOfficialCreated by NVIDIA

  • Skill
  • Workflow
  • Automation and integrations
  • Apache-2.0
  • Medium risk
Download ZIP

From the author

Use when a user needs to get started with CDF Auto-Labeling, plan a scenario, run or debug a shipped cookbook, author prompts or cookbooks, migrate a pipeline, configure a stage, or pull the prebuilt stage containers from NGC (nvcr.io). Confirm critical inputs (data path, output path, endpoints) and ask when any are missing. This is a router: read the matching reference instead of inventing a workflow.
— the description by NVIDIA

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

CDF Auto-Labeling is a central routing skill designed to guide users through the lifecycle of automated data labeling tasks within the Cosmos Data Factory ecosystem. It provides a structured approach to onboarding, scenario planning, cookbook authoring, and pipeline execution. By acting as a router, it directs users to specialized documentation for specific tasks—such as migrating pipelines, debugging workflows, or managing containerized stages—while enforcing strict input validation and operational guardrails to ensure reproducible results.

What it does

  • Routes users to specific documentation based on their current task or stage in the pipeline.
  • Validates critical run inputs like data paths, output directories, and model endpoints.
  • Provides a standardized workflow for running, debugging, and customizing annotation cookbooks.
  • Facilitates the integration of new stages and the management of prebuilt container images.
  • Enforces operational guardrails to prevent common errors like hardcoding secrets or skipping validation steps.

When to use it

  • When you are a new user needing to get started with CDF Auto-Labeling.
  • When you need to plan a new annotation scenario or adapt an existing cookbook.
  • When you are troubleshooting a failed pipeline run or debugging a specific stage.
  • When you need to migrate an external annotation repository into the CDF framework.
  • When you need to pull or configure prebuilt stage containers from NGC.

Advantages

  • Ensures consistent and reproducible pipeline execution through standardized cookbooks.
  • Prevents configuration errors by mandating input validation before execution.
  • Simplifies complex workflows by breaking them down into manageable, modular tasks.
  • Promotes best practices by delegating specialized tasks to dedicated reference guides.

Usage examples

“I cloned the repo and have my own warehouse-safety video. How do I produce auto-labels for my domain?”
The agent will guide you to verify your environment, run a shipped example to confirm the stack, plan your scenario, adapt a cookbook, and execute a dry-run.
“How do I pull the prebuilt stage containers from NGC?”
The agent will direct you to the ngc-containers reference and provide instructions on using make pull to retrieve the necessary images.

Install

  1. Download the skill as a ZIP:

    https://autogineer.com/agent-skills/nvidia/cdf-auto-labeling/download

  2. Unzip it into the cross-client skills folder of your project:

    mkdir -p .agents/skills && unzip cdf-auto-labeling.zip -d .agents/skills

Documentation by NVIDIA

CDF Auto-Labeling

Use this skill when a user wants to kick off CDF Auto-Labeling on their own data, domain, or use case, or when the request matches a shipped cookbook, stage, authoring, or migration task. This is a router: sequence the specialized references instead of duplicating their detail.

Routing (Read First)

Request looks likeRead
New user, clean checkout, first validated run, "how do I get started"This file, then the matching reference below
Choose annotation targets / stage subset for a domainreferences/scenario-planning.md
Create, review, or adapt a cookbookreferences/cookbook-authoring.md
Write or adapt VLM/LLM prompts or question banksreferences/prompt-authoring.md
Migrate an existing annotation repo into this onereferences/pipeline-migration.md
Run the video data augmentation cookbookreferences/video-data-augmentation.md
Run or choose an EPAS / PAS cookbookreferences/event-and-person-attribute-search.md
Run event-verification reasoningreferences/event-verification-reasoning.md
Debug an already-integrated workflowreferences/workflow-runner-debugging.md
Pull prebuilt stage images from NGC, log in to nvcr.io, or fix a missing cdf-* imagereferences/ngc-containers.md
Implement or review a new stage or Dockerized servicereferences/workflow-stage-integration.md
Configure or debug one production stageThe matching file under references/stages/

Stage references: super-resolution, detection-and-tracking, captioning, visual-qa, reasoning, person-attribute-search, grounding-2d, referring-expressions, training-export.

Instructions

  1. Confirm the critical run inputs with the user before doing anything else, and ask a concise question whenever one is missing or ambiguous - never guess or silently invent a default. At minimum confirm: input data path, output path, VLM/LLM endpoint URLs and model names, model cache path, GPU ids, and (for reasoning-capable models) the max_tokens cap. Restate the confirmed values back to the user before the first execution.

  2. Verify the environment: repository cloned, make targets available, the model cache path exists, the VLM/LLM endpoints are reachable, and a GPU is available. State any missing prerequisite as a blocker instead of assuming it. If the stage images are not present locally, offer to pull the prebuilt images from NGC with make pull instead of building them. Use ngc-containers.

  3. Run a shipped example first to confirm the stack works end to end before customizing. Pick the closest operator pipeline - video data augmentation, event-and-person-attribute-search, or event-verification-reasoning - and run its committed cookbook. Use the matching operator reference.

  4. Plan the target scenario: define modality, domain, intended consumer, and required annotations, and get a minimal stage subset. Use scenario-planning.

  5. Adapt the closest shipped cookbook to the new domain rather than authoring from scratch. Use cookbook-authoring.

  6. Author the domain prompts and question banks. Use prompt-authoring.

  7. Configure the per-stage settings for the domain (detector classes or SAM3 prompts, endpoints, windowing, max_tokens). Use the relevant stage reference, starting with detection-and-tracking.

  8. Dry-run the adapted cookbook, then execute and validate the outputs. Use workflow-runner-debugging.

Adopting an existing external annotation or dataset-generation repository into CDF instead of starting from a shipped cookbook is a migration task; use pipeline-migration for that path.

Examples

New user, new domain: "I cloned the repo and have my own warehouse-safety video. How do I produce auto-labels for my domain?"

Guided path:

  • Confirm env (model cache, VLM/LLM endpoints, GPU), then prove the stack on a shipped example before customizing:

make run SCRIPT=workflow-runner:main \
  ARGS='--cookbook-file cookbooks/video_data_augmentation/configs/pipeline_video.yaml --container-dry-run'

  • Plan the domain (scenario-planning) -> subset detection_and_tracking -> captioning -> visual_qa -> reasoning -> training_export (add grounding_2d for caption→boxes or referring_expressions for boxes→phrases; use grounding-2d / referring-expressions).

  • Copy the closest cookbook to cookbooks/warehouse_safety/configs/pipeline.yaml and adapt inputs, detector classes/SAM3 prompts, prompts, and question banks.

  • Dry-run the new cookbook, then run for real and validate outputs:

make run SCRIPT=workflow-runner:main \
  ARGS='--cookbook-file cookbooks/warehouse_safety/configs/pipeline.yaml --container-dry-run'

Guardrails

  • Do not guess or fabricate the critical inputs enumerated in step 1; if any is missing or ambiguous, ask the user and confirm before executing.

  • Do not customize a cookbook before a shipped example runs clean; a broken base makes domain debugging ambiguous.

  • Keep the first custom pipeline minimal - only the stages needed for the requested annotations - and expand later.

  • Verify that every selected stage's service package and image exist in the current branch before promising an end-to-end run.

  • Do not put secrets, tokens, or absolute home paths in committed cookbooks; use placeholders such as <model-cache> and env vars for endpoint keys.

  • For reasoning-capable models (for example Gemini 3 Flash), raise max_tokens on the visual_qa and reasoning LLM substages to avoid the thinking-token tax; keep the default cap for non-reasoning models.

  • Do not rely on non-CDF pipelines, commands, or file locations. A first run must be reproducible through workflow-runner:main inside this repo.

Files

references/event-and-person-attribute-search.md

CDF Visual Attribute Search

Use this skill to operate one of the four checked-in Visual Attribute Search workflows. Read run-reference.md before generating commands.

The event-and-person-attribute-search-service is PAS-only. It consumes an explicit attribute JSON or sidecars from explicit upstream workflow nodes and uses only the text LLM for query generation. It does not caption media, run Visual QA, detect people, copy crops, or synthesize tracks.

Workflow

  1. Select one contract. Match the input and requested output to exactly one cookbook:

    • pipeline_image_attributes_pas.yaml: one attribute JSON, LLM only.

    • pipeline_image_multiview_pas.yaml: one identity across multiple images; joint captioning + Visual QA, then PAS.

    • pipeline_video_pas_reasoning.yaml: tracking, event verification and reasoning, per-track PAS, then training export.

    • pipeline_video_epas.yaml: tracking, captions, anomaly evidence, per-track attributes, PAS query buckets, and contextual output.

  2. Collect required values. Confirm input, output, and the endpoints required by the selected contract. Video runs also need a model cache and SAM3 weights; multi-view needs a representative image and local image-group directory. Never invent paths, URLs, models, or credentials. For real individuals, also confirm the documented consent or other legal basis, permitted purpose, retention period, and required privacy or legal approvals before proceeding.

  3. Create a local config. Copy the tracked config to *.local.yaml and make only deployment-specific edits. Keep its node IDs, dependencies, question banks, artifact namespaces, and PAS source lists intact.

  4. Dry-run. Run the selected config with --container-dry-run. Validate the exact node order, images, endpoint arguments, input/output mounts, path-valued node arguments, and that every PAS input has an upstream producer. Fix and retry at most three times; if it remains invalid, stop and report the failing contract.

  5. Preflight. Verify local inputs and mounts exist, model endpoints respond, and required images are available or can be built. Do not print secrets.

  6. Execution gate. A dry-run is read-only. Before a real Docker run, present the exact command, GPU use, builds/pulls, mounts, endpoints, and output writes, then require explicit user approval. If approval is not given, stop.

  7. Validate artifacts. After a successful run, validate the producer and PAS artifacts listed in the run reference. Do not validate a partial output after a failed container.

Sensitive-data disclaimer for expected artifacts

Visual Attribute Search expected artifacts, including person_attributes.json, bundle_queries.json, Visual QA results, crops, tracks, captions, and derived query files, may contain personal data, biometric-adjacent attributes, or other PII about identifiable individuals. Before running image or video person attribute search on real people, identify and document consent or another valid legal basis and confirm compliance with all applicable laws, which may include the GDPR, CCPA/CPRA, and Illinois BIPA. Obtain qualified privacy or legal review when requirements are uncertain.

Limit input and output directory access to authorized personnel and service accounts with least-privilege permissions; never use public or broadly shared storage. Encrypt output artifacts at rest. Define and enforce a purpose-limited retention schedule before execution, including secure deletion of source media, crops, sidecars, query bundles, backups, and derived copies when the retention period expires.

Contract facts

  • Attribute-only PAS uses the same JSON as the single DataEntry.media_path and --attribute-json. It requires no VLM or image processing.

  • Multi-view PAS keeps media_path as one representative file and passes the group directory to captioning and Visual QA with --max-group-images 0. It produces one consolidated identity scene.

  • Both video flows repeat visual_qa. Preserve distinct node IDs, question banks, sidecar paths, and --state-artifacts-key values.

  • Only the video configs require detection/tracking and the SAM3 weights mount.

  • The VLM serves captioning and Visual QA. The LLM serves reasoning when present and PAS query generation. PAS must not receive VLM arguments.

  • Reasoning/thinking models may need a larger max_tokens value in the Visual QA and reasoning nodes; non-reasoning cookbook defaults should remain unchanged.

Guardrails

  • Always inspect a dry-run before execution.

  • Require explicit approval before Docker execution or image builds.

  • Forward credential names such as NVIDIA_API_KEY, never secret values.

  • Do not collapse explicit producer nodes into PAS configuration and do not add image_pas, composite producer, or overwrite/reuse settings to the service. The absence of an image_pas block does not remove the privacy obligations for image-derived attributes or other person data in the pipeline.

  • Do not change one Visual QA pass to write another pass's namespace.

Requirements and compatibility

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

Security

What our scanner noticed· Medium risk

  • Large base64-encoded blobs

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Attribution and licence

Created by
NVIDIA
Publisher
NVIDIA (Official)
Source
NVIDIA/skills · skills/cdf-auto-labeling · commit fce3cca
Licence
Apache License 2.0 (redistribution allowed)
  • LICENSE (added by AutoGineer from LICENSE-APACHE)
  • ATTRIBUTION.md (written by AutoGineer)
Synced

AutoGineer is not affiliated with or endorsed by NVIDIA or NVIDIA.

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download cdf-auto-labeling.zip