Huggingface Trackio

Hugging FaceOfficialPublished by Hugging Face

Skill
Tool integration
Data and AI
Apache-2.0
Medium risk

From the author

Track and visualize ML training experiments with Trackio. Use when logging metrics during training (Python API), firing alerts for training diagnostics, or retrieving/analyzing logged metrics (CLI). Supports real-time dashboard visualization, alerts with webhooks, HF Space syncing, and JSON output for automation.

— the description by Hugging Face

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

Trackio provides a Python API and CLI for tracking machine learning experiments. It supports local metric storage, real-time dashboard visualization, and synchronization with Hugging Face Spaces. Designed with a focus on autonomous workflows, it allows developers and LLM agents to log metrics, fire diagnostic alerts, and retrieve training data programmatically via JSON output.

What it does

Logs training metrics and hyperparameters using a Python API.
Provides real-time experiment visualization via local or remote Hugging Face Space dashboards.
Enables diagnostic alerting with severity levels (INFO, WARN, ERROR) and webhook support.
Allows programmatic retrieval of metrics and alerts via CLI with JSON output.
Supports automatic integration with TRL trainers.
Syncs local experiment data to persistent remote storage.

When to use it

When you need to track and visualize metrics during ML model training.
When building autonomous agents that need to monitor training health and iterate on hyperparameters.
When you require persistent dashboards for cloud-based training runs.
When you need to integrate training diagnostics with external notification systems like Slack or Discord.

Advantages

Lightweight and local-first design.
API-compatible with existing experiment tracking tools.
Enables autonomous experiment iteration through structured alerts.
Supports both local and cloud-based monitoring workflows.
Provides structured JSON output for easy integration with automation scripts.

Usage examples

“How do I log training metrics with Trackio?”: Initialize the run with trackio.init(), use trackio.log() to record metrics during your loop, and call trackio.finish() to save the data.
“How can an agent monitor for training failures?”: The agent can use trackio.alert() in the training script to flag issues, then poll for these alerts using the CLI command 'trackio list alerts --json'.
“How do I sync my local training metrics to a Hugging Face Space?”: Pass the space_id parameter to trackio.init() during initialization, or use trackio.sync() to upload an existing local project.

Install

Download the skill as a ZIP:
https://autogineer.com/agent-skills/huggingface/huggingface-trackio/download

Unzip it into the cross-client skills folder of your project:

mkdir -p .agents/skills && unzip huggingface-trackio.zip -d .agents/skills

Documentation by Hugging Face

Trackio - Experiment Tracking for ML Training

Trackio is an experiment tracking library for logging and visualizing ML training metrics. It syncs to Hugging Face Spaces for real-time monitoring dashboards.

Three Interfaces

Task	Interface	Reference
Logging metrics during training	Python API	references/logging_metrics.md
Firing alerts for training diagnostics	Python API	references/alerts.md
Retrieving metrics & alerts after/during training	CLI	references/retrieving_metrics.md

When to Use Each

Python API → Logging

Use import trackio in your training scripts to log metrics:

Initialize tracking with trackio.init()
Log metrics with trackio.log() or use TRL's report_to="trackio"
Finalize with trackio.finish()

Key concept: For remote/cloud training, pass space_id — metrics sync to a Space dashboard so they persist after the instance terminates. Auto-created Spaces are public by default — pass private=True if the metrics should not be public.

→ See references/logging_metrics.md for setup, TRL integration, and configuration options.

Python API → Alerts

Insert trackio.alert() calls in training code to flag important events — like inserting print statements for debugging, but structured and queryable:

trackio.alert(title="...", level=trackio.AlertLevel.WARN) — fire an alert
Three severity levels: INFO, WARN, ERROR
Alerts are printed to terminal, stored in the database, shown in the dashboard, and optionally sent to webhooks (Slack/Discord)

Key concept for LLM agents: Alerts are the primary mechanism for autonomous experiment iteration. An agent should insert alerts into training code for diagnostic conditions (loss spikes, NaN gradients, low accuracy, training stalls). Since alerts are printed to the terminal, an agent that is watching the training script's output will see them automatically. For background or detached runs, the agent can poll via CLI instead.

→ See references/alerts.md for the full alerts API, webhook setup, and autonomous agent workflows.

CLI → Retrieving

Use the trackio command to query logged metrics and alerts:

trackio list projects/runs/metrics — discover what's available
trackio get project/run/metric — retrieve summaries and values
trackio list alerts --project <name> --json — retrieve alerts
trackio show — launch the dashboard
trackio sync — sync to HF Space

Key concept: Add --json for programmatic output suitable for automation and LLM agents.

→ See references/retrieving_metrics.md for all commands, workflows, and JSON output formats.

Minimal Logging Setup

import trackio

# Spaces are PUBLIC by default (good for shareable dashboards);
# pass private=True if the metrics should not be public
trackio.init(project="my-project", space_id="username/trackio", private=True)
trackio.log({"loss": 0.1, "accuracy": 0.9})
trackio.log({"loss": 0.09, "accuracy": 0.91})
trackio.finish()

Minimal Retrieval

trackio list projects --json
trackio get metric --project my-project --run my-run --metric loss --json

Autonomous ML Experiment Workflow

When running experiments autonomously as an LLM agent, the recommended workflow is:

Set up training with alerts — insert trackio.alert() calls for diagnostic conditions
Launch training — run the script in the background
Poll for alerts — use trackio list alerts --project <name> --json --since <timestamp> to check for new alerts
Read metrics — use trackio get metric ... to inspect specific values
Iterate — based on alerts and metrics, stop the run, adjust hyperparameters, and launch a new run

import trackio

trackio.init(project="my-project", config={"lr": 1e-4})

for step in range(num_steps):
    loss = train_step()
    trackio.log({"loss": loss, "step": step})

    if step > 100 and loss > 5.0:
        trackio.alert(
            title="Loss divergence",
            text=f"Loss {loss:.4f} still high after {step} steps",
            level=trackio.AlertLevel.ERROR,
        )
    if step > 0 and abs(loss) < 1e-8:
        trackio.alert(
            title="Vanishing loss",
            text="Loss near zero — possible gradient collapse",
            level=trackio.AlertLevel.WARN,
        )

trackio.finish()

Then poll from a separate terminal/process:

trackio list alerts --project my-project --json --since "2025-01-01T00:00:00"

Files

ATTRIBUTION.mdAdded by AutoGineer882 B
LICENSELicence file added by AutoGineer11.1 KiB
SKILL.md5.0 KiB
references/alerts.md5.8 KiB
references/logging_metrics.md5.4 KiB
references/retrieving_metrics.md8.5 KiB

references/retrieving_metrics.md

Retrieving Metrics with Trackio CLI

The trackio CLI provides direct terminal access to query Trackio experiment tracking data locally without needing to start the MCP server.

Quick Command Reference

Task	Command
List projects	`trackio list projects`
List runs	`trackio list runs --project <name>`
List metrics	`trackio list metrics --project <name> --run <name>`
List system metrics	`trackio list system-metrics --project <name> --run <name>`
List alerts	`trackio list alerts --project <name> [--run <name>] [--level <level>] [--since <timestamp>]`
Get project summary	`trackio get project --project <name>`
Get run summary	`trackio get run --project <name> --run <name>`
Get metric values	`trackio get metric --project <name> --run <name> --metric <name>`
Get metric at step	`trackio get metric ... --metric <name> --step <N>`
Get metric around step	`trackio get metric ... --metric <name> --around <N> --window <W>`
Get all metrics snapshot	`trackio get snapshot --project <name> --run <name> --step <N>`
Get system metrics	`trackio get system-metric --project <name> --run <name>`
Show dashboard	`trackio show [--project <name>]`
Sync to Space	`trackio sync --project <name> --space-id <space_id>`

Core Commands

List Commands

trackio list projects                                    # List all projects
trackio list projects --json                            # JSON output

trackio list runs --project <name>                      # List runs in project
trackio list runs --project <name> --json               # JSON output

trackio list metrics --project <name> --run <name>      # List metrics for run
trackio list metrics --project <name> --run <name> --json

trackio list system-metrics --project <name> --run <name>  # List system metrics
trackio list system-metrics --project <name> --run <name> --json

trackio list alerts --project <name>                       # List alerts
trackio list alerts --project <name> --run <name> --json   # Filter by run
trackio list alerts --project <name> --level error --json  # Filter by level
trackio list alerts --project <name> --json --since <ts>   # Poll since timestamp

Get Commands

trackio get project --project <name>                    # Project summary
trackio get project --project <name> --json             # JSON output

trackio get run --project <name> --run <name>           # Run summary
trackio get run --project <name> --run <name> --json

trackio get metric --project <name> --run <name> --metric <name>  # Metric values
trackio get metric --project <name> --run <name> --metric <name> --json
trackio get metric ... --metric <name> --step 200                 # At exact step
trackio get metric ... --metric <name> --around 200 --window 10   # ±10 steps
trackio get metric ... --metric <name> --at-time <ts> --window 60 # ±60 seconds

trackio get snapshot --project <name> --run <name> --step 200 --json       # All metrics at step
trackio get snapshot --project <name> --run <name> --around 200 --window 5 --json  # Window
trackio get snapshot --project <name> --run <name> --at-time <ts> --window 60 --json

trackio get system-metric --project <name> --run <name>           # All system metrics
trackio get system-metric --project <name> --run <name> --metric <name>  # Specific metric
trackio get system-metric --project <name> --run <name> --json

Dashboard Commands

trackio show                                              # Launch dashboard
trackio show --project <name>                           # Load specific project
trackio show --theme <theme>                            # Custom theme
trackio show --mcp-server                                # Enable MCP server
trackio show --color-palette "#FF0000,#00FF00"         # Custom colors

Sync Commands

trackio sync --project <name> --space-id <space_id>     # Sync to HF Space
trackio sync --project <name> --space-id <space_id> --private  # Private space
trackio sync --project <name> --space-id <space_id> --force   # Overwrite

Output Formats

All list and get commands support two output formats:

Human-readable (default): Formatted text for terminal viewing
JSON (with --json flag): Structured JSON for programmatic use

Common Patterns

Discover Projects and Runs

# List all available projects
trackio list projects

# List runs in a project
trackio list runs --project my-project

# Get project overview
trackio get project --project my-project --json

Inspect Run Details

# Get run summary with all metrics
trackio get run --project my-project --run my-run --json

# List available metrics
trackio list metrics --project my-project --run my-run

# Get specific metric values
trackio get metric --project my-project --run my-run --metric loss --json

Query System Metrics

# List system metrics (GPU, etc.)
trackio list system-metrics --project my-project --run my-run

# Get all system metric data
trackio get system-metric --project my-project --run my-run --json

# Get specific system metric
trackio get system-metric --project my-project --run my-run --metric gpu_utilization --json

Automation Scripts

# Extract latest metric value
LATEST_LOSS=$(trackio get metric --project my-project --run my-run --metric loss --json | jq -r '.values[-1].value')

# Export run summary to file
trackio get run --project my-project --run my-run --json > run_summary.json

# Filter runs with jq
trackio list runs --project my-project --json | jq '.runs[] | select(startswith("train"))'

LLM Agent Workflow

# 1. Discover available projects
trackio list projects --json

# 2. Explore project structure
trackio get project --project my-project --json

# 3. Inspect specific run
trackio get run --project my-project --run my-run --json

# 4. Query metric values
trackio get metric --project my-project --run my-run --metric accuracy --json

# 5. Poll for alerts (use --since for efficient incremental polling)
trackio list alerts --project my-project --json --since "2025-06-01T00:00:00"

# 6. When an alert fires at step N, get all metrics around that point
trackio get snapshot --project my-project --run my-run --around 200 --window 5 --json

Error Handling

Commands validate inputs and return clear errors:

Missing project: Error: Project '<name>' not found.
Missing run: Error: Run '<name>' not found in project '<project>'.
Missing metric: Error: Metric '<name>' not found in run '<run>' of project '<project>'.

All errors exit with non-zero status code and write to stderr.

Key Options

--project: Project name (required for most commands)
--run: Run name (required for run-specific commands)
--metric: Metric name (required for metric-specific commands)
--json: Output in JSON format instead of human-readable
--step: Exact step filter (for get metric, get snapshot)
--around: Center step for window filter (for get metric, get snapshot)
--at-time: Center ISO timestamp for window filter (for get metric, get snapshot)
--window: Window size: ±steps for --around, ±seconds for --at-time (default: 10)
--level: Alert level filter (info, warn, error) (for list alerts)
--since: ISO timestamp to filter alerts after (for list alerts)
--theme: Dashboard theme (for show command)
--mcp-server: Enable MCP server mode (for show command)
--color-palette: Comma-separated hex colors (for show command)
--private: Create private Space (for sync command)
--force: Overwrite existing database (for sync command)

JSON Output Structure

List Projects

{"projects": ["project1", "project2"]}

List Runs

{"project": "my-project", "runs": ["run1", "run2"]}

Project Summary

{
  "project": "my-project",
  "num_runs": 3,
  "runs": ["run1", "run2", "run3"],
  "last_activity": 100
}

Run Summary

{
  "project": "my-project",
  "run": "my-run",
  "num_logs": 50,
  "metrics": ["loss", "accuracy"],
  "config": {"learning_rate": 0.001},
  "last_step": 49
}

Metric Values

{
  "project": "my-project",
  "run": "my-run",
  "metric": "loss",
  "values": [
    {"step": 0, "timestamp": "2024-01-01T00:00:00", "value": 0.5},
    {"step": 1, "timestamp": "2024-01-01T00:01:00", "value": 0.4}
  ]
}

References

Complete CLI documentation: See docs/source/cli_commands.md
API and MCP Server: See docs/source/api_mcp_server.md

Requirements and compatibility

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

Needs: Needs Python

Security

What our scanner noticed· Medium risk

Possible data-exfiltration endpoints
Instructions that use the network

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Attribution and licence

Published by

Hugging Face

Publisher

Hugging Face (Official)

Source

huggingface/skills · skills/huggingface-trackio · commit 25910b3

Licence

Apache License 2.0 (redistribution allowed)

LICENSE (added by AutoGineer from LICENSE)
ATTRIBUTION.md (written by AutoGineer)

Synced

2026-11-09

AutoGineer is not affiliated with or endorsed by Hugging Face or Hugging Face.

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download huggingface-trackio.zip