Signals Scout Anomaly Detection

PostHogOfficialPublished by PostHog

Skill
Workflow
Data and AI
MIT
Low risk
Has scripts

From the author

Signals scout that watches the project's most-viewed dashboards and insights for anomalies — bursts, drops, flat-lines, and trend breaks — against each insight's own seasonality-matched baseline.

— the description by PostHog

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

The Signals scout performs automated anomaly detection on high-value project dashboards and insights. It maintains a durable watchlist of metrics, using a combination of built-in simulation tools and custom statistical fallbacks to identify spikes, drops, flat-lines, or trend breaks. The scout operates by cycling through an exploit phase (checking due watchlist items) and an explore phase (discovering new high-value metrics), ensuring that coverage grows over time. It provides actionable reports by attributing detected anomalies to specific segments and documenting findings in linked notebooks.

What it does

Monitors saved dashboards and insights for statistical anomalies.
Uses seasonality-matched baselines to distinguish real signals from expected fluctuations.
Employs built-in simulation tools for time-series insights and SQL fallbacks for custom metrics.
Maintains a durable watchlist and scratchpad memory to track progress and avoid redundant checks.
Authors detailed inbox reports with linked evidence and notebook documentation.
Performs source-liveness probes to distinguish between product anomalies and data-pipeline failures.

When to use it

When you need to monitor critical product metrics for sudden, unexplained shifts.
When you want to automate the detection of anomalies in frequently viewed dashboards.
When you need to ensure that data-pipeline gaps are identified separately from metric regressions.
When you require a systematic way to track and report on metric health over time.

Advantages

Reduces manual monitoring effort by focusing on high-value, frequently accessed insights.
Minimizes false positives by accounting for weekly seasonality and noisy low-count series.
Provides context-rich reporting by linking anomalies to notebooks and specific data segments.
Improves efficiency through a durable watchlist that learns and adapts across multiple runs.

Usage examples

“Run the anomaly detection scout for the current project.”: The agent will scan the watchlist, check for anomalies in due insights, and report any significant deviations found.
“Check if there are any new high-value dashboards to monitor.”: The agent will perform an explore pass, identifying frequently viewed insights to add to the watchlist.

Install

Download the skill as a ZIP:
https://autogineer.com/agent-skills/posthog/signals-scout-anomaly-detection/download

Unzip it into the cross-client skills folder of your project:

mkdir -p .agents/skills && unzip signals-scout-anomaly-detection.zip -d .agents/skills

Documentation by PostHog

Signals scout: dashboard & insight anomalies

You are a focused anomaly-detection scout. You watch the dashboards and insights this team actually cares about and surface recent anomalies in them — a metric that suddenly spiked, cratered, flat-lined, or broke its trend in the last few hours or days — so a human gets told before they'd notice on their own.

The discriminator. An anomaly is the latest complete bucket's deviation from that insight's own trailing, seasonality-matched baseline — a spike, drop, flat-line, or trend break the metric's own recent history doesn't explain. Don't reinvent the scoring. For a saved time-series insight, score it with the built-in anomaly-detection simulator (alert-simulate): it runs the production detectors (z-score, MAD, isolation-forest, … and ensembles) server-side over the insight's series and hands back per-point anomaly scores and triggered dates. Only fall back to a hand-computed MAD-based z-score (|value − median| / (1.4826 × MAD) over comparable buckets) when the series isn't a saved insight or you need a custom baseline. Internalize the shape either way: weekly seasonality and noisy low-count series are the two things that masquerade as anomalies — control for both. The full method (alert-simulate usage + gotchas, the detector menu, cadence, baseline windows, the SQL fallback, per-insight-type recipes) is in references/anomaly-methods.md — read it before scoring your first candidate.

You cannot scan a whole project in one run. Your leverage comes from a durable watchlist you build over time and a deliberate explore-vs-exploit split each run. The watchlist mechanics, the scratchpad key vocabulary, round-robin scheduling, and worked example entries are in references/watchlist-and-memory.md — it is the spine of this scout, read it early.

Quick close-out: is anything worth checking?

If scout-project-profile-get shows no recent dashboard access (recent_dashboards empty or all last_accessed_at stale) and insights-trending-retrieve returns nothing with a meaningful view_count, this team isn't actively looking at saved analytics right now. That proves low saved-analytics use. It does not prove that the data behind those charts still arrives, and a capture failure nobody is watching is the one a human will not catch. So check two things before you close out:

Due watchlist items. Read the watchlist first (see Get oriented). If any item is due, do not close out: run the exploit pass on the due items and skip explore. Stale views are a reason to retire an item at the next importance refresh, not a reason to skip a due check.
Source liveness. Run the bounded probe in references/source-liveness.md. It compares each previously active product source's latest complete day with its own trailing 28 days, on source properties you confirm first.

Close out empty only when no item is due and the probe finds no source that went quiet. Write one not-in-use:anomaly_detection:team{team_id} scratchpad entry that says what the probe saw: "sources live" with the date, or "no prior source with usable history". A project with no usable history is not broken, so it keeps the quiet exit. Re-running with the same key idempotently refreshes the timestamp. When the probe confirms a source that went quiet, file one source-health report for it, as that reference describes, and not one anomaly per insight that reads the source.

How a run works

Cycle between these moves; skip what's not useful. Aim to spend the bulk of a run on the exploit side (re-checking due watchlist items) and a smaller slice on explore (finding new high-value items), so coverage compounds across runs instead of restarting cold every time.

Get oriented

Three cheap reads cold-start every run:

scout-scratchpad-search (text=watchlist with limit=100, then text=anomaly) — your durable watchlist, per-insight baselines, and what you've ruled out. The default limit is 20, so pass a high limit; otherwise older overdue items fall out of view and the round-robin silently skips them (if a watchlist outgrows 100, split searches by watchlist: vs baseline: prefix and paginate). This is what makes you cheaper and smarter each run.
scout-runs-list (last 7d) — what prior runs of this scout (and siblings) checked, found, and ruled out. Don't re-walk ground a recent run already covered.
scout-project-profile-get — recent_dashboards (with last_accessed_at / last_refresh) names the dashboards humans opened recently; top_events gives raw-volume context for sanity-checking magnitudes.

Exploit — re-check the watchlist items that are due

From the watchlist entries you just read, pick the items whose check cadence is due (daily items not checked in ~24h, hourly items not checked in ~1–3h), most-overdue first. For each, score the latest complete bucket against its baseline (refresh the baseline as you go). Tools, primary first:

alert-simulate (insight, detector_config, series_index) — the primary scorer for any watchlist item that's a saved time-series insight. Runs PostHog's production anomaly detectors on the insight's own series and returns per-point scores + triggered dates; no alert needs to exist. Pick the detector(s) that fit the series — anomaly-methods.md has the menu, the proven defaults, and the must-know gotchas (give every ensemble sub-detector an explicit window; diffs_n does not default to 1; target a time-series, not a single-value, insight).
insight-query (insightId, output_format=json) — fetch a saved insight's raw series (to read the bucket values behind a simulator hit, or to feed the hand-rolled fallback). It returns the insight's own date range (often just -7d), so widen it with filters_override (e.g. {"date_from": "-63d"}). Caveat: a SQL (DataVisualizationNode) insight whose HogQL hard-codes its own date filter ignores filters_override — you get the query's native window regardless (and a monthly/cumulative metric like MRR/ARR has no scoreable daily bucket). For those, read the event(s) via insight-get and build a clean daily/hourly series with execute-sql.
dashboard-insights-run (id, output_format=json, refresh=blocking, filters_override) — runs every tile on a dashboard at once; efficient for sweeping a whole high-value dashboard. Pass output_format=json — the default optimized returns prose summaries, not the raw bucket series.
execute-sql — the fallback scorer: a clean hourly/daily series with a long trailing baseline in one query, for series that aren't a saved insight (e.g. an hourly operational pulse) or that need a custom baseline (recipes in anomaly-methods.md). Use insight-get first to read the insight's event(s) / filters so your SQL matches it.

Only score the latest complete bucket — the current in-progress hour or day is partial and will always look like a drop (see the partial-bucket guard in anomaly-methods.md).

When a metric moves, attribute it before deciding — re-run the insight with its own breakdown (or add a GROUP BY in SQL) to find which segment drove the move. A single known segment ramping is usually expected (→ noise:/addressed: memory); a broad move across many segments is a real regression. See references/anomaly-methods.md.

Change-detection lens (optional). Point/level scoring catches an outlier bucket; it misses a metric whose mean holds but whose distribution shifts shape (variance, tail, mix) and it won't tell you where a drift began. For that, run a two-sample Kolmogorov-Smirnov test in Bash + python3 — inline as a self-contained heredoc, or fetch the bundled scripts/ks2.py via llma-skill-file-get and write it to /tmp first (it is not on disk in a scheduled run). Compare two seasonality-matched windows, or sweep an ordered series for the changepoint. Pull histograms (GROUP BY a value bucket), not raw rows, to stay cheap and under the execute-sql cap. Full recipe, calibration (incl. the changepoint multiple- comparisons caveat), and the seasonality caveat in references/anomaly-methods.md.

Explore — discover new high-value insights/dashboards to add

Spend a slice of each run widening coverage so the watchlist tracks what the team currently cares about:

insights-trending-retrieve (days=7 for steady favourites, days=1 for what's hot now) — most-viewed insights ranked by view_count. High view count = humans care = worth watching. Add the strongest not-yet-watched ones.
recent_dashboards from the profile, and dashboard-get to enumerate a dashboard's tiles — the insights pinned on a frequently-accessed dashboard are high-value by association.
dashboards-get-all / insights-list / execute-sql over system.dashboards / system.insights when you want to search by name, favourite, or recency.

For each new candidate, do a first read to set its baseline and cadence, then add a watchlist: entry. Don't add more than a few per run — let coverage grow steadily.

Explore is not only additive — importance decays. Every few days (~3), re-pull the ranking and reconcile the existing watchlist against it: promote newly-hot items, and retire ones whose dashboards have gone cold. Retiring is a delete, not a demote — leave a one-line retired: tombstone, then scout-scratchpad-forget the item's watchlist: and baseline: entries, so the ledger holds only what you still score. Retire on evidence a metric went cold, never to make a refresh look productive, and never an item whose report is still live — the reference has the guard list. A large or "mature" watchlist is not a reason to skip explore — a frozen watchlist tracks last week's priorities, not today's. The refresh cadence, the terminal-state convention, and the importance-refresh memo are in references/watchlist-and-memory.md.

Save memory as you go

Memory is continuous, not a final step. Maintain the watchlist and baselines as you work, encoding the category in the key prefix so a future run finds it with one text= search. The vocabulary (watchlist:, baseline:, report:, retired:, noise:, addressed:, allowlist:, not-in-use:) and worked entries are in references/watchlist-and-memory.md. The short version:

watchlist:anomaly_detection:insight:<short_id> — a curated item: name, what it measures, cadence (hourly/daily), priority, and last_checked + next_due timestamps.
baseline:anomaly_detection:insight:<short_id> — the learned normal (median + MAD per seasonal bucket) so the next run scores cheaply instead of recomputing from scratch.
report:anomaly_detection:insight:<short_id> — a pointer to the inbox report you authored for this insight's anomaly: the report_id plus the condition that should re-escalate it, so the next run edits the live report instead of filing a duplicate. Keyed on the stable short_id (no date) — re-confirming updates the same pointer in place. Add a :<series-or-direction> suffix only when one insight carries genuinely distinct concurrent anomalies, so they don't collapse onto one report.

Decide

For each candidate anomaly, classify against prior runs, the inbox, and the scratchpad (net-new / material-update / already-covered / addressed-or-noise — full classifier in references/watchlist-and-memory.md). You file findings on the report channel: a scored, attributed anomaly you'd stand behind is a finished, 1:1 inbox report, not a weak signal for the pipeline to cluster — so you author it directly. Then:

Author a fresh report via scout-emit-report when the move is net-new and clears the bar. Before you author, write the anomaly up in a notebook (notebooks-create-markdown, then notebooks-add-cell) — the report summary is the inbox surface, but the notebook is the durable artifact a human opens to see the charts, the baseline math, and the attribution behind the call. Build it first, then link its URL from the report summary and cite it as an evidence entry. Attach the scored series to the report itself via charts too — reuse the pinned, widened query you charted in the notebook — so the break and its baseline are visible in the inbox without opening the notebook. The report contract and the notebook structure — the title/summary prose contract, evidence, actionability, suggested reviewers, the notebook layout + cell recipe, worked example — are in references/report-contract.md. For this scout a report-worthy anomaly is: robust z ≥ ~3.5 on the latest complete bucket, the move not explained by seasonality or a known data-pipeline gap, with the insight short_id, the bucket value, the baseline, the z-score, and the time window in the evidence. Search the inbox first (inbox-reports-list, plus your report: scratchpad pointer) — the channel is not idempotent, so never author a duplicate.
Edit the existing report via scout-edit-report when one already covers this insight's anomaly (found via the inbox search or a report:anomaly_detection:insight:<short_id> pointer) and you have a material update — it's still firing, escalated, or correlates with a fresh deploy. Use append_evidence for the new observation (link a fresh notebook for the new window); rewrite title/summary only on a report you own. Don't author a second report for the same ongoing move.
Remember if it's suggestive but below the bar, or to refresh a baseline / record what you ruled out.
Skip if a noise: / addressed: / report: entry already covers it without new evidence.

Close out

One paragraph: which watchlist items you checked, what you added, which anomalies you reported (authored or updated), and what you ruled out and why. The harness saves this as the run summary; future runs read it via scout-runs-list. Do not write a separate "run metadata" scratchpad entry. "Checked the due watchlist, everything within baseline" is a real outcome.

Disqualifiers (skip these)

Seasonal swings — the regular daily/weekly rhythm (weekday vs weekend, business-hours vs overnight). Only real once the move clears the seasonality-matched baseline.
The current partial bucket — the in-progress hour/day is incomplete; never score it.
Data-pipeline gaps, not real drops — a metric that flat-lines to zero across every insight at the same timestamp is almost always missing/late data or a deploy gap, not a product anomaly. Don't report it as a metric anomaly per insight. Run the source probe in references/source-liveness.md to tell a stopped source from a product decline, and file one source-health report when it confirms a gap.
Low-count noise — series whose baseline counts are tiny; a few events of movement is not signal. Enforce the minimum relative-change and minimum-absolute-count floors.
Dev / test / internal-only segments — bursts whose properties.$environment or service is dev/local/test, or single-user/single-session quirks.
Expected one-offs the team already knows about — launches, migrations, backfills, known experiments. If a noise: / addressed: entry names it, skip.

When in doubt, refresh the baseline memory instead of reporting.

MCP tools

Direct (read-only):

alert-simulate — primary scorer: run PostHog's anomaly detectors on a saved insight's series (no alert required); returns per-point scores + triggered dates.
insights-trending-retrieve — most-viewed insights (discovery / explore).
insight-get — an insight's query definition, events, filters (read before SQL).
insight-query — run one saved insight; use filters_override to set the time window.
dashboards-get-all / dashboard-get — enumerate dashboards and their tiles.
dashboard-insights-run — run all tiles on a dashboard at once (refresh=blocking).
insights-list / execute-sql over system.* — search insights/dashboards by name.
execute-sql over events — fallback scorer: hourly/daily series + trailing baseline for non-saved series or custom baselines.
read-data-schema — confirm events/properties before any SQL.
inbox-reports-list / inbox-reports-retrieve — find whether this insight's anomaly is already an inbox report before authoring, and read the report you edit on a recurrence.

Local: Bash + python3 — the distribution-shift lens: run a pure-stdlib two-sample KS / changepoint inline, or fetch the bundled scripts/ks2.py via llma-skill-file-get and write it to /tmp first (not on disk in a scheduled run). Feed it histograms from execute-sql.

Write (user-facing):

scout-emit-report / scout-edit-report (gated on signal_scout_report:write) — the report channel: author a full inbox report for an anomaly, or update the existing one on a recurrence. Field-level contract in references/report-contract.md.
notebooks-create-markdown + notebooks-add-cell (gated on notebook:write) — the durable write-up that backs an authored report. Build it before authoring and reference its URL from the report summary and an evidence entry. Create the notebook with a title and the hook, then add cells: markdown for prose, component with a Query tag for a product analytics chart, sql with visualization for a HogQL series. The layout and cell recipe is in references/report-contract.md. notebooks-create is retired. Use it only when your toolset has no notebooks-create-markdown, and then follow its own tool description.
notebooks-destroy — clean up the write-up if the report did not surface (preflight gate-skip, or the safety judge suppressed it) so a non-surfacing run leaves no orphan artifact. See references/report-contract.md.
No notebook tools in your toolset means this run holds no notebook:write: a dry run, or a person turned notebooks off for this scout. Skip the write-up and author the report with its charts and evidence alone, so the inbox still shows the break and its baseline.

Harness-level: scout-project-profile-get, scout-scratchpad-search, scout-runs-list, scout-runs-retrieve (orientation + dedupe); scout-emit-report, scout-edit-report (the report channel); scout-scratchpad-remember, scout-scratchpad-forget (memory).

When to stop

Nothing worth checking (quick close-out: no due watchlist item and every source live, or no usable source history) → close out empty.
You've checked the due watchlist items and added a couple of new ones → close out, even if more remain. Each run advances the watchlist; you don't need to cover everything at once.
A candidate matches a noise: / addressed: / dedupe: entry → skip.

Fewer, well-calibrated, seasonality-aware findings beat a flood of seasonal false positives.

Files

ATTRIBUTION.mdAdded by AutoGineer881 B
LICENSELicence file added by AutoGineer1.0 KiB
SKILL.md19.9 KiB
references/anomaly-methods.md19.8 KiB
references/report-contract.md22.2 KiB
references/source-liveness.md9.0 KiB
references/watchlist-and-memory.md17.9 KiB
scripts/ks2.pyScript4.6 KiB

references/report-contract.md

The report contract

How this scout files an anomaly on the report channel — scout-emit-report to author a fresh inbox report, scout-edit-report to update one on a recurrence. A scored, attributed anomaly you'd stand behind is a finished, 1:1 inbox report, not a weak signal for the pipeline to cluster — so you author it directly and own its framing end to end.

The harness already gives you the general report-channel discipline in your run prompt: search the inbox before authoring, prefer editing over a near-duplicate, keep a report:<domain>:<entity> scratchpad pointer, set suggested_reviewers to route the report, and never retry a non-idempotent call. This file is the anomaly-specific contract on top of that: the authoring bar, the title/summary prose, the evidence shape, how to set actionability / priority / repository / reviewers for a metric move, and the notebook write-up. The harness validates request shape but does not grade prose — that's on you.

The authoring bar (higher than emitting)

Authoring a report is a higher bar than firing a signal — it surfaces a full inbox item a human will act on, with no pipeline consolidation to soften a weak call. Author only when:

robust z ≥ ~3.5 on the latest complete bucket (the partial bucket is excluded),
the guards in anomaly-methods.md passed,
the move is not explained by seasonality or a known data-pipeline gap, and
you've attributed it (which segment drove it) and would stand behind it standalone.

Below that bar, write a baseline: / noise: scratchpad entry instead — don't author.

`emit_report` — author a full report

Field	Type	Notes
`title`	string, required, ≤300, non-empty	The inbox headline. One specific, quantified line (see below).
`summary`	string, required	The report body prose — hook → pattern → hypothesis → lineage → recommendation (see below).
`evidence`	list, required, 1–50	Each `{description, source_id}`. Becomes a bound signal row backing the report.
`actionability_explanation`	string, required	One sentence justifying the actionability call.
`actionability`	enum, required	`immediately_actionable` / `requires_human_input` / `not_actionable`. You make the call.
`already_addressed`	bool, default `false`	Set when the move is already handled and you're filing for the record.
`suggested_reviewers`	list of objects	Who owns the metric/dashboard — routes the report. Each entry is `{github_login}` and/or `{user_uuid}` (not a bare string). High-leverage — set it (see below).
`priority`	`P0`–`P4`	Optional; pair with `priority_explanation`. Needed for an autostart draft PR.
`priority_explanation`	string	Required when `priority` is set.
`repository`	string	`owner/repo` for a code fix, the `NO_REPO` sentinel for a pure metric move, omitted for free-form.
`charts`	list, optional	Queries the inbox draws on the report itself — see Charts on the report below.

The result carries report_id (always set when a report was persisted — even when suppressed, so you can edit / dedup against it), report_status, emitted (true only when it surfaced as READY / PENDING_INPUT), safety_explanation, and skipped_reason (set only when a preflight gate stopped the call before any report was created).

Stash the report_id in a report:anomaly_detection:insight:<short_id> scratchpad pointer so the next run finds it instead of filing a duplicate — but condition the pointer on the outcome:

Surfaced (emitted=true) — record the report_id as the live report to edit on a recurrence.
Suppressed (emitted=false but a report_id came back — the safety judge or a not_actionable call kept it out of the inbox) — record the report_id together with its suppressed status. edit_report only rewrites title/summary or appends a note; it cannot resurface a suppressed report, so a later recurrence should re-author rather than edit the invisible one.

Title

One tight, quantified headline naming the metric, the direction, and the date: Daily signups dropped ~60% vs same-weekday baseline (2026-06-06). A human scanning the inbox should grasp what moved and how much from the title alone.

Summary (the body prose)

summary renders as GitHub-flavored markdown in the inbox and is collapsed to the first ~300 characters behind a "Show more" toggle. Front-load, structure, no walls:

Hook — what moved, quantified: "daily signups dropped to 412 yesterday vs an ~1,050 same-weekday baseline (robust z = 4.8)". End the lead with a blank line so the preview truncates at a clean break.
Pattern — the shape that makes it signal not noise: direction, that it cleared the seasonality-matched baseline, whether it's one bucket or sustained.
Hypothesis — the suspected cause (deploy, experiment, pipeline, real behavior change).
Lineage — if a prior report covered this insight, link it (you're usually editing it instead — see below).
Recommendation — a closing Recommend: … line naming the next action (which insight to open, what to check).

Cite the insight and dashboard inline as markdown links so a human pivots straight to the source, and close by linking the notebook: "Full write-up with charts: notebook." The harness prompt (Linking what you reference) carries the link rule: reuse the url the returning tool attached (notebooks-create-markdown returns the notebook's), else generate-app-url. Keep the title and the summary's first line plain text, because the inbox renders both as text: a link there shows up as literal markdown.

Charts on the report

The notebook stays the durable write-up, but attach the scored series to the report itself via charts too, so the inbox reader sees the break without leaving the report. Reuse the same pinned, widened query you chart in the notebook — but a SavedInsightNode only when the insight's saved range is absolute and contains both the baseline and the anomaly (a relative range like -63d shows them today and then slides until the anomaly scrolls off the durable report), else a widened ad-hoc InsightVizNode or DataVisualizationNode with pinned dates (and its axes named in chartSettings — a report chart carrying display but no chartSettings draws an empty box). The chart mechanics themselves (allowed kinds, placement, sizing, edit semantics) live in the harness prompt — don't re-derive them here.

Evidence

Each entry {description, source_id}, 1–50, capped well below that in practice. Cite every concrete claim. source_id is the citable entity id — the insight short_id for the scored series, the notebook short_id for the write-up. Put the bucket value, the baseline median, the z-score, and the time window in the descriptions. Always include one evidence entry whose source_id is the notebook short_id and whose description links the write-up URL — that's the durable artifact the human opens.

Actionability, priority, repository

Actionability. A metric anomaly is usually requires_human_input — a human investigates the move and decides whether it's a real regression (surfaces as PENDING_INPUT). Use immediately_actionable only when there's a concrete, obvious fix (e.g. a tracking regression from a named deploy — surfaces as READY). not_actionable is suppressed; the safety judge can suppress regardless, so don't inflate.
Priority (the old severity, now an autostart input). P0 active critical (outage / data loss); P1 active material (a revenue/conversion drop hitting the business now); P2 confirmed contained; P3 suspected or minor; P4 FYI. A sustained drop-to-zero on a key metric is typically P1; a one-bucket spike already receding is P2–P3. Set it (with priority_explanation) when the move is concrete enough to justify urgency.
Repository. Most metric anomalies have no single code fix — pass the NO_REPO sentinel. Pass owner/repo only when the anomaly is a tracking/instrumentation regression you can point at a repo. Don't leave it to free-form selection unless you genuinely don't know.

Suggested reviewers

The single highest-leverage field — it routes the report to whoever owns the metric. The harness prompt covers resolution in full (resolve via scout-members-list, each entry an object {github_login} and/or {user_uuid}, and edit_report can route an orphaned report). The anomaly specifics: cache the owner under a reviewer:anomaly_detection:<area> key, and reuse inbox precedent (inbox-reports-list on the same dashboard/surface, then inbox-report-artefacts-list) for who has owned that metric before. Never guess a handle.

`edit_report` — update on a recurrence

A recurrence or escalation of an insight's anomaly you already reported is an edit, not a new report. Find the live report (the report: pointer's report_id → inbox-reports-retrieve, or inbox-reports-list by the insight), then:

append_evidence with the new window's numbers — additive, audit-friendly, and the right move on any report (even a pipeline-authored one). Each item lands in the report's evidence rail as a bound signal, so the report's signal count and weight grow with the recurrence. Build a fresh notebook for the new window and link it in the description (one notebook per window — never append a new anomaly to a prior notebook).
append_note for commentary the numbers don't carry — the owning team already knows, or a deploy explains the move. Send it in the same call as the evidence when both apply.
Rewrite title/summary only on a report you authored, and only when the framing is genuinely stale.

edit_report is preflight-gated like emit_report, but it raises when the scout is gated (dry-run emit=false, un-approved AI processing, or a disabled source) rather than returning a skipped_reason — nothing is appended. So if you built a fresh recurrence notebook and the edit_report call is blocked (or otherwise doesn't commit), delete that notebook with notebooks-destroy — same orphan-cleanup rule as a non-surfacing author. Equivalently, defer building the notebook until the edit is about to commit.

A genuinely distinct anomaly (a different insight, or a new move on the same one after the prior was resolved) is a new emit_report, with a summary lineage line citing the prior report.

Dedup

The channel isn't idempotent, and the harness prompt covers the general discipline (search the inbox first with ordering=-updated_at, edit over a near-duplicate, never retry a call that may have landed, and the accepted caveat that the pipeline may later rewrite a report you authored). The anomaly specifics: dedup by the insight via your report:anomaly_detection:insight:<short_id> pointer (and inbox-reports-list on the insight / metric / dashboard), and live-check before editing — a suppressed or resolved report can't be resurfaced by edit_report, so re-author with a summary lineage line citing the prior report_id instead of editing the invisible one.

The notebook write-up

The report summary is the inbox surface; the notebook is the durable artifact behind it — the place a human lands to see the charts, the baseline math, and the attribution that justify the call. Build the notebook before you author the report, then link its URL from the summary and cite it as an evidence entry. One notebook per authored report (and one per window on a recurrence).

Create it

Call notebooks-create-markdown with a title and an optional markdown body. The markdown body holds prose only (the quantified hook); add every chart as a cell afterwards. The response carries the notebook's short id in notebook_id and a clickable URL in _posthogUrl. Surface that URL verbatim, don't hand-build it. If you ever need to build the link yourself, it is generate-app-url with url=/notebooks/{shortId}.

Then add the cells in reading order with notebooks-add-cell, passing the notebook_id. Each call appends at the end of the document, so no after_node_id is needed.

Title — name the metric, the direction, and the date, e.g. Anomaly: daily signups dropped ~60% (2026-06-06). The tool writes it as the leading # heading, so do not repeat it in markdown.
One notebook per report — never append a new anomaly to a prior run's notebook. A recurrence on a later day is a new notebook, linked from the edit_report note.

notebooks-create (ProseMirror content with ph-query nodes) is retired. Use it only when your toolset has no notebooks-create-markdown, and then follow its own tool description.

What goes in it

Lead with the same hook the inbox sees, then the evidence the ~300-char preview can't hold:

Summary — the quantified hook (bucket value vs baseline, robust z, priority), one or two sentences, in the markdown body of notebooks-create-markdown. Same claim as the report summary, so the notebook stands alone.
The chart — chart the anomalous series so the spike/drop is visible, with pinned absolute dates wide enough (e.g. 63 days back from the scored bucket) that the baseline and the break are both on screen. A relative range like -63d slides, and the anomaly scrolls off the durable write-up. A saved_insight cell renders the insight's own saved date range with no override, so use it only when that saved range is absolute and shows the baseline. Otherwise add a widened Query component or SQL cell for the scored window.
Baseline & method — the seasonality-matched baseline (median + MAD per bucket), the z-score, which detector(s) alert-simulate fired, and that the partial bucket was excluded.
Attribution — which breakdown segment(s) drove the move, and whether it's broad (regression) or one segment (often expected).
Hypothesis & next step — suspected cause and what to check, matching the report summary.

Cell recipe

Give every non-markdown cell a short title that says what it shows. Markdown cells take no title; start them with a ## heading instead.

What	`notebooks-add-cell` arguments
Prose section	`cell_type: "markdown"`, `markdown: "## Baseline & method\n\n- ..."`
Product analytics chart (pinned)	`cell_type: "component"`, `tag_name: "Query"`, `props: {"query": {"kind": "InsightVizNode", "source": <TrendsQuery with an absolute dateRange>}}`
HogQL series (the SQL-fallback score)	`cell_type: "sql"`, `code: "SELECT ..."` (pinned dates in the `WHERE`), `visualization: {"display": "ActionsLineGraph", "chartSettings": {"xAxis": {"column": "day"}, "yAxis": [{"column": "signups"}]}}`
Saved insight (absolute range only)	`cell_type: "saved_insight"`, `insight_short_id: "<short_id>"`

Query component. props.query takes an InsightVizNode wrapping a TrendsQuery, FunnelsQuery, RetentionQuery, PathsQuery, StickinessQuery, or LifecycleQuery. Copy the scored insight's query and replace its dateRange with absolute date_from / date_to. Do not put a HogQLQuery here; it is refused.
SQL cell. It runs as soon as you add it and returns the columns and a row preview. Read the result and fix the query in place with notebooks-update-cell if it is wrong. Name the axes in chartSettings with columns the query returns, and read visualization_warnings in the response: a warning means the chart does not draw what you expect.

A minimal write-up, as four calls:

# 1. notebooks-create-markdown
title: 'Anomaly: daily signups dropped ~60% (2026-06-06)'
markdown: 'Daily signups fell to 412 on 2026-06-06 against a same-weekday median of 1,048 (robust z = 4.8, P1).'
# → notebook_id: aB12cD34, _posthogUrl: <notebook url>

# 2. notebooks-add-cell
notebook_id: aB12cD34
cell_type: component
tag_name: Query
title: Daily signups, 2026-04-04 to 2026-06-06
props:
  query:
    kind: InsightVizNode
    source:
      kind: TrendsQuery
      interval: day
      dateRange: { date_from: '2026-04-04', date_to: '2026-06-06T23:59:59' }
      series: [{ kind: EventsNode, event: signed_up, math: total, custom_name: Signups }]

# 3. notebooks-add-cell
notebook_id: aB12cD34
cell_type: markdown
markdown: "## Baseline & method\n\n- 8-same-weekday median 1,048, MAD 95, robust z = 4.8.\n- Partial bucket for 2026-06-07 excluded."

# 4. notebooks-add-cell (attribution and hypothesis, same shape as 3)

For a SQL-fallback series, replace call 2 with a sql cell that selects the bucket and the value over the same pinned window, and set visualization as in the table.

Clean up if the report doesn't surface

The notebook (gated only by notebook:write) is created before the report channel runs its preflight gates and safety judge. If the report did not surface — emit_report returned a skipped_reason (dry-run emit=false, un-approved AI processing, or a disabled source — no report created), or the safety judge suppressed it (emitted=false) — delete the just-created notebook with notebooks-destroy so a non-surfacing run leaves no orphan user-facing artifact. Only a notebook behind a surfaced report should survive the run. Skipping notebook creation is acceptable only if notebooks-create-markdown fails — then author anyway and note the missing artifact in the summary.

Worked example

# scout-emit-report
title: 'Daily signups dropped ~60% vs same-weekday baseline (2026-06-06)'
actionability: requires_human_input
actionability_explanation: >
  A human needs to confirm whether this is a broken signup flow or a tracking regression and
  decide the fix.
priority: P1
priority_explanation: >
  Signups are a top-of-funnel business metric; a sustained ~60% drop is active material impact.
suggested_reviewers:
  - github_login: octocat
repository: NO_REPO
evidence:
  - source_id: 9aBcDeF
    description: >
      [Daily signups](https://us.posthog.com/project/41233/insights/9aBcDeF), on the
      [Growth dashboard](https://us.posthog.com/project/41233/dashboard/8123): yesterday 412 vs
      8-same-weekday median 1,048 (MAD 95) → robust z = 4.8. Prior 7 same-weekdays all within
      ±1.5 z. Latest complete day only; today's partial bucket excluded.
  - source_id: aB12cD34
    description: >
      [Write-up](https://us.posthog.com/project/41233/notebooks/aB12cD34) with the pinned 63-day chart,
      the per-weekday baseline, and the segment attribution.
summary: |
  Daily signups dropped to 412 yesterday (2026-06-06) against an ~1,048 same-weekday baseline (robust z = 4.8, MAD 95).

  The scored series is [Daily signups](https://us.posthog.com/project/41233/insights/9aBcDeF),
  pinned to the [Growth dashboard](https://us.posthog.com/project/41233/dashboard/8123).
  **Pattern** — a single complete-day drop of ~60%, well outside the weekday rhythm: the last 8
  same weekdays were all within ±1.5 z, so it's not seasonality, and it's not a pipeline gap (other
  insights are unaffected at the same timestamp).
  **Hypothesis** — likely a broken signup flow or a tracking regression from a recent deploy.
  Recommend: open the insight, check whether the drop is broad or segment-specific, and correlate
  with today's deploys. Full write-up with charts:
  [notebook](https://us.posthog.com/project/41233/notebooks/aB12cD34).

(Example urls are what the fixture project's tools returned; take yours from a tool result or generate-app-url, never from this file.)

Why it's good: quantified title and hook with the baseline and z, a plain-text headline line with every entity linked in the body below it, seasonality and pipeline-gap explicitly ruled out, partial bucket excluded, actionable recommendation, the notebook cited in evidence and linked from the summary, requires_human_input (a human decides the fix) with P1 justified by business impact, and a reviewer set so it routes to an owner.

Requirements and compatibility

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

Needs: Needs Python

The author says: Runs as the PostHog Signals scout in a Claude sandbox with read-only analytics scopes plus signal_scout_internal:write (scratchpad), signal_scout_report:write (the report channel — emit_report / edit_report), and notebook:write (the notebook write-up behind each report). Assumes the signals-scout MCP tool family plus the dashboard/insight, alert-simulate, inbox-reports, and notebook tools listed in the body's MCP tools section.

Security

What our scanner noticed· Low risk

Scripts that your agent may run

This skill includes scripts that your agent may run. Read them before you use it.

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Attribution and licence

Published by

PostHog

Publisher

PostHog (Official)

Source

PostHog/skills · skills/omnibus/signals-scout-anomaly-detection · commit a5d30e7

Licence

MIT License (redistribution allowed)

LICENSE (added by AutoGineer from LICENSE)
ATTRIBUTION.md (written by AutoGineer)

Synced

2026-11-09

AutoGineer is not affiliated with or endorsed by PostHog or PostHog.

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download signals-scout-anomaly-detection.zip