Data Catalog

MicrosoftOfficialPublished by Microsoft

Skill
Content generator
Data and AI
CC-BY-4.0
Low risk
Has scripts

From the author

Create and enrich durable data catalogs using the native DS_CATALOG_V1 Markdown contract, declared entity relationships, privacy citation fields, and stable relationship IDs. Use when inventorying engagement data, recording semantic relationships, or preparing a catalog for ERD rendering.

— the description by Microsoft

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

The data-catalog skill provides a standardized procedure for inventorying business entities and their relationships. It utilizes the DS_CATALOG_V1 Markdown contract to ensure that data catalogs are both human-readable and machine-validatable. The workflow emphasizes data integrity by requiring stable identifiers, explicit confidence levels for relationships, and clear separation between semantic facts and narrative descriptions. It is designed to support data science and engineering engagements by providing a reliable, versioned record of data assets, lineage, and privacy classifications without embedding sensitive credentials or duplicating profile data.

What it does

Defines a structured Markdown and YAML format for data catalogs.
Requires stable, unique identifiers for all entities and relationships.
Enforces explicit confidence levels (confirmed, inferred, assumed) for all semantic links.
Validates catalog structure against a predefined JSON schema.
Separates metadata and facts from narrative documentation.
Provides a mechanism for tracking privacy standards and lineage.

When to use it

When you need to inventory engagement data for a data science or engineering project.
When recording semantic relationships between business entities for ERD rendering.
When preparing a durable, versioned data catalog that must be machine-readable.
When you need to document privacy classifications and lineage in a standardized way.

Advantages

Ensures consistency across different data catalogs using a strict contract.
Prevents the accidental leakage of credentials by prohibiting embedded secrets.
Maintains high data quality by requiring explicit evidence for relationships.
Supports automated validation to catch errors before review.
Provides a clear, audit-ready history of data relationships.

Usage examples

“Create a new data catalog for the customer engagement project.”: The agent initializes a new Markdown file using the DS_CATALOG_V1 template and guides the user through inventorying entities and defining their relationships.
“Validate the current data catalog file.”: The agent executes the validation script to ensure the YAML frontmatter adheres to the DS_CATALOG_V1 schema and reports any structural issues.

Install

Download the skill as a ZIP:
https://autogineer.com/agent-skills/microsoft/data-catalog/download

Unzip it into the cross-client skills folder of your project:

mkdir -p .agents/skills && unzip data-catalog.zip -d .agents/skills

Documentation by Microsoft

Data Catalog Workflow

Goal

Produce a customer-readable Markdown catalog whose YAML frontmatter is a valid DS_CATALOG_V1 machine contract. Preserve uncertainty explicitly so inferred or assumed relationships never appear confirmed.

Flow

Confirm the engagement name and the caller-approved durable output path.
Inventory entities at business grain. Record source access, tier, volume, profile pointer, classification, lineage, and open questions without copying column-level profile data.
Assign every relationship a stable rel-* identifier. Record endpoints, maximum cardinality, both endpoint minimums, one or more paired join-key fields, confidence, and evidence basis.
Reconcile coverage counts with the entity and relationship records.
Render the human-readable sections from the YAML facts, ending with the canonical Data Science and Engineering Coaching disclaimer footer. Narrative can explain facts but cannot redefine them.
Validate the artifact with scripts/validate_catalog.py before treating it as ready for review.

Inputs

Engagement context and a caller-approved output path
Data source inventory and access status
Business entity names, grain, and declared relationships
Existing per-dataset profile paths, when available
Privacy classifications or standards citations produced by the owning privacy workflow

Success criteria

The frontmatter declares exactly catalog_version: DS_CATALOG_V1 and validates against assets/ds-catalog-v1.schema.json.
Entity IDs and relationship IDs are unique and stable. Every endpoint and lineage reference resolves.
Every relationship declares cardinality as its maximum multiplicity plus from_minimum and to_minimum as zero or one.
Join keys use one string on both sides or paired arrays of equal length, and record field names only without primary-key, foreign-key, or uniqueness roles.
Relationship confidence is one of confirmed, inferred, or assumed, and every relationship records its basis.
Classification uses the privacy-standards citation-field names. The catalog does not invent standards identifiers.
Column statistics and feature metadata remain behind profile_ref rather than being copied into the catalog.
Every customer-facing catalog ends with the canonical Data Science and Engineering Coaching disclaimer footer.

Constraints

Treat catalog enrichment as user-driven. Offer enrichment when a source is missing, but do not change the catalog silently.
Record source locations as paths or connection references, never embedded credentials.
Keep DCAT alignment non-binding. The catalog is native YAML, not RDF, and makes no DCAT conformance claim.
Keep tier recording separate from tier behavior. data-catalog records the tier; dataops owns what that tier means.
Keep rendering separate from semantic authority. Diagram tools consume declared relationships and do not infer new ones.
End every customer-facing catalog with the canonical Data Science and Engineering Coaching disclaimer from the disclaimer-language skill.

Stop rules

Stop and request clarification when an entity grain, relationship endpoint, join-key pairing, or endpoint minimum is ambiguous.
Stop and retain inferred or assumed confidence when evidence does not support confirmed.
Stop before a durable write when the caller has not confirmed the destination.
Stop and route privacy interpretation to privacy-standards or the Privacy Planner when citation values or DPIA status are unknown.

Package resources

Resource	Use
catalog-contract.md	Read for the authoritative field, identity, multiplicity, and body-generation rules
dcat-crosswalk.md	Read when planning a DCAT, DCAT-AP, or catalog-platform export
provenance.md	Read for standards selection, licensing, and non-conformance boundaries
ds-catalog-v1.md	Copy when starting a new catalog
northwind-catalog.md	Read as a complete valid example with scalar and composite join keys
`assets/ds-catalog-v1.schema.json`	Use as the structural JSON Schema for parsed frontmatter
`scripts/validate_catalog.py`	Execute with `uv run python scripts/validate_catalog.py <catalog.md>` to validate a catalog

Attribution

The native contract, workflow, schema, template, examples, and validator are repository-original content licensed CC BY 4.0.

The crosswalk paraphrases selected concepts from W3C DCAT 3, W3C PROV-O, and Frictionless Table Schema v2. Standard names and term identifiers are factual citations, the crosswalk prose and the paired-array join-key contract are independently authored, and no upstream schema, example, table, or substantial excerpt is reproduced. Because no upstream expression is redistributed, the package remains solely CC BY 4.0. See provenance.md.

Files

ATTRIBUTION.mdAdded by AutoGineer975 B
LICENSEAdded by AutoGineer16.6 KiB
SKILL.md6.1 KiB
assets/ds-catalog-v1.schema.json5.9 KiB
examples/northwind-catalog.md8.4 KiB
pyproject.toml698 B
references/catalog-contract.md11.1 KiB
references/dcat-crosswalk.md5.4 KiB
references/provenance.md3.1 KiB
scripts/validate_catalog.pyScript15.0 KiB
templates/ds-catalog-v1.md5.3 KiB
tests/corpus/0_valid_frontmatter90 B
tests/corpus/1_empty_frontmatter8 B
tests/corpus/2_unclosed_sequence20 B
tests/corpus/3_duplicate_key24 B
tests/corpus/4_no_frontmatter15 B
tests/corpus/README.md1.2 KiB
tests/fuzz_harness.pyScript1.2 KiB
tests/test_validate_catalog.pyScript24.0 KiB
uv.lock107.7 KiB

templates/ds-catalog-v1.md

catalog_version: DS_CATALOG_V1 engagement: replace-with-engagement-slug generated_at: "2026-08-03T00:00:00Z" last_enriched: "2026-08-03T00:00:00Z" entities:

id: replace-with-parent-entity-id name: Replace with parent business name description: Replace with what this entity represents source: system: Replace with source system location: Replace with a credential-free path or connection reference format: table access_confirmed: false tier: bronze grain: Replace with what one row represents volume: row_estimate: null period_covered: null update_frequency: unknown profile_ref: null classification: sensitivity: none contains_personal_data: false data_categories: [] gdpr_article: null ccpa_section: null nist_pf_category: null nistir8062_objective: null owasp_privacy_id: null dpia_ref: null lineage: derived_from: [] transform_ref: null open_questions:
- Confirm source access and update cadence
id: replace-with-child-entity-id name: Replace with child business name description: Replace with what this entity represents source: system: Replace with source system location: Replace with a credential-free path or connection reference format: table access_confirmed: false tier: bronze grain: Replace with what one row represents volume: row_estimate: null period_covered: null update_frequency: unknown profile_ref: null classification: sensitivity: none contains_personal_data: false data_categories: [] gdpr_article: null ccpa_section: null nist_pf_category: null nistir8062_objective: null owasp_privacy_id: null dpia_ref: null lineage: derived_from: [] transform_ref: null open_questions:
- Confirm the join key that relates this entity to its parent relationships:
id: rel-replace-with-relationship-id from: replace-with-parent-entity-id to: replace-with-child-entity-id cardinality: one-to-many from_minimum: one to_minimum: zero join_keys: from_field: replace_with_parent_field to_field: replace_with_child_field confidence: assumed basis: Replace with the evidence that supports this relationship coverage: entities_catalogued: 2 entities_access_confirmed: 0 entities_classified: 0 relationships_confirmed: 0 relationships_inferred: 0

Replace with engagement name data catalog

Overview and engagement context

Replace with the engagement context and catalog scope.

Entity summary

Entity	Grain	Tier	Sensitivity	Access
Replace with parent business name	Replace with what one row represents	Bronze	None	Unconfirmed
Replace with child business name	Replace with what one row represents	Bronze	None	Unconfirmed

Entity relationship diagram

Replace with the rendered diagram or a pointer to it. The declared relationship stays assumed until evidence supports a higher confidence.

A diagram never stands alone. Accompany it with the relationship table below, which conveys the same declared facts in text so the section remains readable without seeing the image. Every relationship in the diagram appears as a row, and every row appears in the diagram.

Relationship	Endpoints	Cardinality	Minimums	Join keys	Confidence
Replace with relationship id	parent to child business name	one-to-many	from `one`, to `zero`	Replace with join columns	assumed

Entity details

Replace with parent business name

Replace with source, volume, lineage, and business-semantic context generated from the frontmatter.

Replace with child business name

Replace with source, volume, lineage, and business-semantic context generated from the frontmatter.

Coverage summary

Two entities are catalogued and one relationship is declared. Access, classification, and relationship evidence remain open.

Open questions and access gaps

Confirm source access and update cadence
Confirm the join key that relates this entity to its parent

Disclaimer

[!CAUTION] Disclaimer: This agent is an assistive data-science and data-engineering coaching tool only. It does not validate customer data, execute production pipelines, establish model fitness, or replace data owners, privacy and Responsible AI reviewers, engineering review, or business decision authority. Catalogs, feasibility findings, analyses, experiments, tests, and operational recommendations generated with this tool may be incomplete or inaccurate and must be independently reviewed against approved data sources, stakeholder evidence, and organizational controls before use. Outputs from this tool do not constitute data approval, feasibility sign-off, model approval, privacy or Responsible AI approval, or production readiness.

Requirements and compatibility

Written for: GitHub Copilot, Claude Code

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

Needs: Claude Code frontmatter, Needs Python

Security

What our scanner noticed· Low risk

Scripts that your agent may run

This skill includes scripts that your agent may run. Read them before you use it.

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Attribution and licence

Published by

Microsoft

Publisher

Microsoft (Official)

Source

microsoft/hve-core · .github/skills/data-science-engineering/data-catalog · commit f425e94

Licence

Creative Commons Attribution 4.0 International (redistribution allowed)

LICENSE (written by AutoGineer)
ATTRIBUTION.md (written by AutoGineer)

Synced

2026-11-09

AutoGineer is not affiliated with or endorsed by Microsoft or Microsoft.

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download data-catalog.zip