Debugging Failed Data Quality Checks

PostHogOfficialPublished by PostHog

Skill
Review and audit
Data and AI
MIT
No issues found

From the author

Finds out why a data quality check failed or errored, looks at the rows it matched, and recommends the fix. Also covers a materialized view refresh that the data quality gate did not publish. Use when asked why a check is red, what rows failed, why a view shows "Not published", why a view's data is stale after a refresh, or how to get a blocked refresh published. Trigger terms: failed check, failing rows, errored check, data quality failure, not published, blocked materialization, unpublished refresh, check keeps failing.

— the description by PostHog

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

This skill provides a structured approach to investigating why data quality checks fail or error out. It guides the user through reading check results, inspecting failing rows via compiled queries, and interpreting specific error messages. It also addresses issues where materialized view refreshes are blocked by data quality gates, helping users identify the root cause—whether it is bad data, an incorrect assertion, or a configuration issue—and provides steps to recover and re-run the processes.

What it does

Analyzes recent check runs to determine if a failure is due to data issues or assertion errors.
Replays compiled queries to inspect specific rows that triggered a failure.
Interprets error messages for failed or errored data quality checks.
Identifies why a materialized view refresh is blocked or shows as not published.
Provides a recovery path to fix assertions, update severity, or re-trigger refreshes.
Distinguishes between failed, errored, skipped, and passed check statuses.

When to use it

When a data quality check status is red, failed, or errored.
When a user needs to see the specific rows that caused a check to fail.
When a materialized view shows a Not published error or appears stale.
When a refresh is blocked by a data quality gate.
After being handed off from a view health audit or check authoring process.

Advantages

Systematic troubleshooting for complex data quality and materialization issues.
Clear distinction between data-related failures and configuration errors.
Direct integration with tools to re-run checks and update assertions.
Helps maintain data integrity by preventing bad data from being published.

Usage examples

“Why is my data quality check failing?”: The agent will retrieve the recent check results, identify the status, and analyze the compiled query to show you the specific rows causing the failure.
“My materialized view says Not published, what happened?”: The agent will check the view's run history, identify the failing data quality check that blocked the publication, and guide you through fixing the issue.

Install

Download the skill as a ZIP:
https://autogineer.com/agent-skills/posthog/debugging-failed-data-quality-checks/download

Unzip it into the cross-client skills folder of your project:

mkdir -p .agents/skills && unzip debugging-failed-data-quality-checks.zip -d .agents/skills

Documentation by PostHog

Debugging failed data quality checks

A check run ends in one of four states. Read the state first, because it decides where the problem is:

Status	Meaning	Where to look
`failed`	The query ran and the assertion found bad data	The data, or the assertion itself
`errored`	The query could not run, so nothing was judged	The error text (never a data problem)
`skipped`	The subject is gone	Whether the table or view still exists
`passed`	The assertion held	Nothing

row_count and freshness checks judge observed_value, not failed_row_count. A failed row_count means the count fell outside its bounds. A failed freshness means the newest row is older than the limit.

When to use this skill

A check or a subject shows failed or errored and the user wants to know why
The user wants to see the rows a check matched
A materialized view's latest run shows an error that starts with Not published:
A view kept serving old data after a refresh, and the data quality gate is on
Another skill (authoring-data-quality-checks or auditing-warehouse-view-health) found a failing check or a blocked view and handed it off

Available tools

Tool	Purpose
`posthog:data-quality-check-results`	A check's recent runs, with `compiled_query`, `error` and the config it ran
`posthog:execute-sql`	Re-run a run's `compiled_query` to see the failing rows
`posthog:data-quality-check-run`	Run a check now
`posthog:data-quality-check-update`	Fix an assertion or change its severity
`view-get`	A view's definition and status
`view-run-history`	A view's materialization runs and their errors
`view-run`	Materialize a view again

To find the failing checks, query system.information_schema.data_quality_checks for last_status IN ('failed', 'errored'), or system.information_schema.data_quality_health for a subject's verdict.

Workflow

Step 1: Read the run

Call posthog:data-quality-check-results for the check. On the newest run, read:

status and error
check_config and check_severity: what the run asserted, which can differ from the check today
compiled_query: HogQL that selects the failing rows. For row_count it returns the count instead, so judge that check by observed_value.
audited_staged_refresh: true when the run was part of a gated audit of a refresh before it was published. An errored run can carry it too, so read status first.

Retention clears compiled_query after 30 days. For an older run, run the check again.

Step 2: Look at the failing rows

Run compiled_query with posthog:execute-sql. An empty compiled_query means there is nothing to replay: retention cleared it, or a gated run could not read the view's definition. Look at what it matched before you report anything. A failure means one of two things: the data is bad, or the assertion is wrong. The rows tell you which.

When audited_staged_refresh is true, the query does not read the published table. It holds the view's definition in a WITH clause and reads the view's source tables now. The sources may have changed since the run, so the rows and the count can differ from the run. A query that now returns zero rows does not prove that the refresh was clean. Compare with failed_row_count on the run.

Step 3: Read an errored run

An errored run did not judge the data. Match the error text:

Error text	Cause and fix
`The staged files could not be read, so this data was not audited.`	The gate could not reach the refresh before publish. The refresh publishes without a verdict. Run the view again.
`Metric checks cannot audit staged data.`	A metric check was part of a gated refresh. It runs on its own schedule instead.
`The check query returned no rows.`	The aggregate query returned nothing. Run the check again and report it if it repeats.
`A check that reads another subject needs an initiator or author…`	An automatic run of a `custom_sql` or `relationships` check has no user to run as. A manual run works. An edit to the check's assertion makes the editor its author.
A HogQL or ClickHouse error	A column name typo, a type mismatch, or a query that ran out of time. Fix the config, or add `lookback_hours`.

Two messages come with skipped: The subject was deleted. and The subject no longer resolves. The table or view is gone. Delete the check or point it at the replacement.

Step 4: Understand a blocked refresh

When the team turns on "Block materialization on failing checks", a materialized view's error-severity checks run on each refresh before it is published. If one fails:

The view's run shows an error that starts with Not published: (see view-run-history).
The previous version of the view keeps serving queries.
Downstream views in the same DAG are skipped for this run.
The view's schedule is not paused. The next scheduled refresh runs the checks again.

Warn-severity checks never block a refresh.

Step 5: Recover

Fix the cause, then prove it:

Fix the upstream data, or fix the assertion with posthog:data-quality-check-update. If the failure is real but must not block the refresh, change severity to warn.
For a blocked view, call view-run to materialize it again. Otherwise call posthog:data-quality-check-run.
Read posthog:data-quality-check-results again and confirm the newest run passed.

Important notes

Do not change a check's severity or assertion only to turn it green. Confirm with the user first.
errored is never evidence of bad data, and passed on an empty table is not evidence of good data.
A check reports against its subject, but a custom_sql query can read other tables. Read the query before you blame the subject.

authoring-data-quality-checks: writing and choosing checks
auditing-warehouse-view-health: finding failing materialized views

Files

ATTRIBUTION.mdAdded by AutoGineer891 B
LICENSELicence file added by AutoGineer1.0 KiB
SKILL.md8.0 KiB

Requirements and compatibility

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

Security

No issues found by our static scan· No issues found

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Attribution and licence

Published by

PostHog

Publisher

PostHog (Official)

Source

PostHog/skills · skills/omnibus/debugging-failed-data-quality-checks · commit a5d30e7

Licence

MIT License (redistribution allowed)

LICENSE (added by AutoGineer from LICENSE)
ATTRIBUTION.md (written by AutoGineer)

Synced

2026-11-09

AutoGineer is not affiliated with or endorsed by PostHog or PostHog.

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download debugging-failed-data-quality-checks.zip