AutoGineer
Generated banner for Huggingface Datasets
Generated logo for Huggingface Datasets

Huggingface Datasets

Hugging FaceOfficialPublished by Hugging Face

  • Skill
  • Tool integration
  • Data and AI
  • Apache-2.0
  • Low risk
Download ZIP

From the author

Use this skill for Hugging Face Dataset Viewer API workflows that fetch subset/split metadata, paginate rows, search text, apply filters, download parquet URLs, and read size or statistics.
— the description by Hugging Face

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

This skill provides a structured approach to interacting with the Hugging Face Dataset Viewer API. It enables users to programmatically explore datasets by fetching metadata, paginating through rows, applying search and filter predicates, and retrieving parquet file links. Additionally, it covers workflows for uploading datasets and managing agent session traces, ensuring efficient data discovery and management within the Hugging Face ecosystem.

What it does

  • Validates dataset availability and retrieves configuration and split information.
  • Previews and paginates through dataset rows using offset and length parameters.
  • Executes text searches and applies row-level filters with custom predicates.
  • Retrieves parquet file shards, dataset size totals, and column statistics.
  • Facilitates the upload of parquet files to dataset repositories via CLI.
  • Supports the ingestion and viewing of agent session traces in JSONL format.

When to use it

  • When you need to programmatically explore or extract data from Hugging Face datasets.
  • When building data pipelines that require specific dataset splits or filtered subsets.
  • When you need to verify dataset structure, column statistics, or parquet shard locations.
  • When uploading local parquet files to the Hugging Face Hub for public or private hosting.
  • When managing and visualizing agent session traces from local development environments.

Advantages

  • Provides direct, read-only access to dataset content without requiring full downloads.
  • Enables efficient data exploration through server-side pagination and filtering.
  • Simplifies the process of discovering parquet shards for large-scale data processing.
  • Supports secure access to private datasets via standard authorization tokens.
  • Streamlines the upload and organization of datasets and agent trace logs.

Usage examples

“Fetch the first 50 rows of the train split for the stanfordnlp/imdb dataset.”
The agent calls the /first-rows endpoint with the specified dataset, config, and split parameters to return the initial data subset.
“Filter the 'test' split of my-dataset to find rows where the 'label' column is 1.”
The agent uses the /filter endpoint with a 'where' predicate to retrieve only the rows matching the specified criteria.

Install

  1. Download the skill as a ZIP:

    https://autogineer.com/agent-skills/huggingface/huggingface-datasets/download

  2. Unzip it into the cross-client skills folder of your project:

    mkdir -p .agents/skills && unzip huggingface-datasets.zip -d .agents/skills

Documentation by Hugging Face

Hugging Face Dataset Viewer

Use this skill to execute read-only Dataset Viewer API calls for dataset exploration and extraction.

Core workflow

  1. Optionally validate dataset availability with /is-valid.

  2. Resolve config + split with /splits.

  3. Preview with /first-rows.

  4. Paginate content with /rows using offset and length (max 100).

  5. Use /search for text matching and /filter for row predicates.

  6. Retrieve parquet links via /parquet and totals/metadata via /size and /statistics.

Defaults

  • Base URL: https://datasets-server.huggingface.co

  • Default API method: GET

  • Query params should be URL-encoded.

  • offset is 0-based.

  • length max is usually 100 for row-like endpoints.

  • Gated/private datasets require Authorization: Bearer <HF_TOKEN>.

Dataset Viewer

  • Validate dataset: /is-valid?dataset=<namespace/repo>

  • List subsets and splits: /splits?dataset=<namespace/repo>

  • Preview first rows: /first-rows?dataset=<namespace/repo>&config=<config>&split=<split>

  • Paginate rows: /rows?dataset=<namespace/repo>&config=<config>&split=<split>&offset=<int>&length=<int>

  • Search text: /search?dataset=<namespace/repo>&config=<config>&split=<split>&query=<text>&offset=<int>&length=<int>

  • Filter with predicates: /filter?dataset=<namespace/repo>&config=<config>&split=<split>&where=<predicate>&orderby=<sort>&offset=<int>&length=<int>

  • List parquet shards: /parquet?dataset=<namespace/repo>

  • Get size totals: /size?dataset=<namespace/repo>

  • Get column statistics: /statistics?dataset=<namespace/repo>&config=<config>&split=<split>

  • Get Croissant metadata (if available): /croissant?dataset=<namespace/repo>

Pagination pattern:

curl "https://datasets-server.huggingface.co/rows?dataset=stanfordnlp/imdb&config=plain_text&split=train&offset=0&length=100"
curl "https://datasets-server.huggingface.co/rows?dataset=stanfordnlp/imdb&config=plain_text&split=train&offset=100&length=100"

When pagination is partial, use response fields such as num_rows_total, num_rows_per_page, and partial to drive continuation logic.

Search/filter notes:

  • /search matches string columns (full-text style behavior is internal to the API).

  • /filter requires predicate syntax in where and optional sort in orderby.

  • Keep filtering and searches read-only and side-effect free.

For CLI-based parquet URL discovery or SQL, use the hf-cli skill with hf datasets parquet and hf datasets sql.

Creating and Uploading Datasets

Use one of these flows depending on dependency constraints.

Zero local dependencies (Hub UI):

  • Create dataset repo in browser: https://huggingface.co/new-dataset

  • Upload parquet files in the repo "Files and versions" page.

  • Verify shards appear in Dataset Viewer:

curl -s "https://datasets-server.huggingface.co/parquet?dataset=<namespace>/<repo>"

Low dependency CLI flow (npx @huggingface/hub / hfjs):

  • Set auth token:

export HF_TOKEN=<your_hf_token>

  • Upload parquet folder to a dataset repo (auto-creates repo if missing):

npx -y @huggingface/hub upload datasets/<namespace>/<repo> ./local/parquet-folder data

  • Upload as private repo on creation:

npx -y @huggingface/hub upload datasets/<namespace>/<repo> ./local/parquet-folder data --private

After upload, call /parquet to discover <config>/<split>/<shard> values for querying with @~parquet.

Agent Traces

The Hub supports raw agent session traces from Claude Code, Codex, and Pi Agent. Upload them to Hugging Face Datasets as original JSONL files and the Hub can auto-detect the trace format, tag the dataset as Traces, and enable the trace viewer for browsing sessions, turns, tool calls, and model responses. Common local session directories:

  • Claude Code: ~/.claude/projects

  • Codex: ~/.codex/sessions

  • Pi: ~/.pi/agent/sessions

Default to private dataset repos because traces can contain prompts, file paths, tool outputs, secrets, or PII. Preserve the raw .jsonl files and nest them by project/cwd instead of uploading every session at the dataset root.

hf repos create <namespace>/<repo> --type dataset --private --exist-ok
hf upload <namespace>/<repo> ~/.codex/sessions codex/<project-or-cwd> --type dataset

Files

ATTRIBUTION.md

Attribution

This folder is an unmodified copy of the Agent Skill "huggingface-datasets", distributed by AutoGineer (https://autogineer.com/agent-skills).

The skill's own files are unmodified copies. AutoGineer is not affiliated with or endorsed by the author or the publisher.

Files added by AutoGineer

  • ATTRIBUTION.md: this file, written by AutoGineer

  • LICENSE: copied from LICENSE

Requirements and compatibility

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

Needs: Needs network access, Needs Node.js

Security

What our scanner noticed· Low risk

  • Instructions that use the network

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Attribution and licence

Published by
Hugging Face
Publisher
Hugging Face (Official)
Source
huggingface/skills · skills/huggingface-datasets · commit 25910b3
Licence
Apache License 2.0 (redistribution allowed)
  • LICENSE (added by AutoGineer from LICENSE)
  • ATTRIBUTION.md (written by AutoGineer)
Synced

AutoGineer is not affiliated with or endorsed by Hugging Face or Hugging Face.

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download huggingface-datasets.zip