Working With Autoresearch

PostHogOfficialPublished by PostHog

Skill
Tool integration
Data and AI
MIT
No issues found

From the author

Use PostHog Autoresearch, agent-driven predictive models: ask "who will do X in the next N days?", let an agent train a model, then score people daily into a person property and `autoresearch_prediction` events. Use when a user wants to predict churn, activity, return, feature adoption, conversion or any event or action per person; create or validate a model from a template; start training or score now; read model quality (holdout vs realized AUC, lift, calibration, likelihood segments, coverage); steer training with suggestions; act on predictions with cohorts, feature flags, workflows or SQL; or diagnose a model that fails to train, scores everyone the same, or stops scoring. Trigger on "autoresearch", "predictive model", "propensity score", "churn prediction", "likely to convert", "who will do X", "predicted_p_", "autoresearch_prediction", "realized AUC", "champion model".

— the description by PostHog

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

Autoresearch enables users to create, train, and maintain predictive models within PostHog. By defining a target event, a population, and a time horizon, an agent-driven process trains models, selects the best-performing champion, and scores users daily. Predictions are stored as person properties and events, allowing for integration into cohorts, feature flags, and workflows. The system includes built-in validation, performance monitoring, and troubleshooting tools to ensure model quality and reliability.

What it does

Trains predictive models to forecast specific user actions or events.
Automatically selects the best model (champion) based on holdout performance.
Scores populations daily and records results as person properties and events.
Provides metrics like holdout and realized AUC to validate model quality.
Allows steering of training runs and troubleshooting of model performance.
Integrates predictions directly into PostHog cohorts, feature flags, and SQL queries.

When to use it

When you need to predict churn, feature adoption, or conversion likelihood.
When you want to automate personalized user experiences based on propensity scores.
When you need to validate if a specific user segment is likely to perform a key action.
When you need to diagnose why a predictive model is underperforming or failing to score.

Advantages

Automates the complex process of feature engineering and model selection.
Provides clear, actionable metrics for model quality and reliability.
Seamlessly connects predictions to existing product analytics and activation tools.
Supports both template-based setup and custom model definitions.

Usage examples

“Who will do X in the next 7 days?”: The agent initiates a training run to build a model predicting which users will perform the specified event within the 7-day horizon.
“Is my churn prediction model any good?”: The agent retrieves the model's holdout and realized AUC, providing a quality label and performance trend to assess if the model is reliable.

Install

Download the skill as a ZIP:
https://autogineer.com/agent-skills/posthog/working-with-autoresearch/download

Unzip it into the cross-client skills folder of your project:

mkdir -p .agents/skills && unzip working-with-autoresearch.zip -d .agents/skills

Documentation by PostHog

Working with autoresearch

Autoresearch answers one standing question per model: who in a population will do a target within a horizon? For example, "which active users will open a report in the next 7 days?".

The user names a target (an event or a PostHog action), a population and a horizon in days.
A training agent runs in a PostHog sandbox. It writes feature SQL and a model, scores each experiment on a holdout set, keeps what improves holdout AUC and discards the rest.
The server picks the champion from the kept experiments. The agent proposes; the backend decides.
Every cadence (daily by default, at 02:00 UTC), the champion scores the inference population. Each scored person gets an autoresearch_prediction event with their probability. When the model's write_person_property is on, a person who resolves to a current distinct ID also gets the model's output person property, for example predicted_p_adopt_insights_14d. It is off by default for new models. Turn it on with autoresearch-partial-update before you target people by score in a flag, cohort, workflow or insight. Turning it off removes the property from the people the model scored. Changing the setting needs the person:write scope.
When a prediction date's horizon has passed, online validation checks the scores against what really happened and records realized AUC.
Training keeps going on a schedule. On each scoring day the model starts a research run (explore) until a run does not promote a better model, or until 5 research runs have run. After that it starts one research run every research_interval_days (maintain, default 7) after the latest training run, so a run started by hand resets the clock. With research_interval_days null the model trains only when someone asks. A champion that cannot score still gets a short rescue run. training_phase and next_research_at on posthog:autoresearch-retrieve say where the model is.

In the PostHog UI the entity is a model. The API and tools call it a pipeline. Use "model" when you talk to people.

Autoresearch is behind the autoresearch feature flag. Training runs in a cloud sandbox, so it also needs PostHog Desktop access. If the autoresearch-* tools are missing, the project probably does not have access yet.

Pick the job

The user wants to	Start with	Reference
Create a model	`posthog:autoresearch-resolve-template-create` or a custom definition, then `posthog:autoresearch-validate-create`	defining-a-question.md
Know if a model is any good	`posthog:autoresearch-retrieve`, then `posthog:autoresearch-online-performance-retrieve`	reading-model-quality.md
Use the predictions	The output person property, or `posthog:execute-sql` on `autoresearch_prediction`	using-predictions.md
Fix or steer a model	`posthog:autoresearch-training-runs-list`, `posthog:autoresearch-runs-list`, `posthog:autoresearch-suggestions-create`	troubleshooting-and-steering.md
See what models exist	`posthog:autoresearch-list`	This page

Create, train, score

The usual path is short. Each tool's description names its prerequisites and refusals, so read it with info before the first call.

Resolve or define. Templates cover the common questions (activity, inactivity risk, return after first use, feature adoption, repeat behavior). A custom definition works when none fits. See defining-a-question.md.
Validate. posthog:autoresearch-validate-create with the same target, horizon, lookback and populations you will create with. It returns the base rate, positive and negative counts and warnings. Fix anything that sets can_proceed to false, and tell the user about the non-blocking warnings.
Create. posthog:autoresearch-create. The new model is a draft until training starts.
Train. posthog:autoresearch-train-create starts an agent run that takes a while (often tens of minutes). It costs real sandbox and LLM time, so confirm with the user before you start one, and never start a second run while one is live. A 403 means either the call came from inside a sandbox (ask the user to click Start training on the model's page) or the user has no PostHog Desktop access (the page will refuse too). Read the error message to tell them apart.
Wait. Poll posthog:autoresearch-training-runs-list at a relaxed pace. While the first run is in progress the model is bootstrapping, and a missing champion is normal.
Score. After a champion is promoted, the next daily sweep scores the population. posthog:autoresearch-score-create scores today now; call it once, then poll posthog:autoresearch-runs-list. A second manual score for the same date can duplicate predictions.

Pause, resume and delete are in the model's menu in the UI only. Pausing stops scoring and new training runs. A training run already in progress keeps running, and it can still promote a champion.

Model status

Status	Meaning
`draft`	Defined, never trained.
`bootstrapping`	The first training run is in progress. No champion yet.
`running`	A champion exists and scores on the cadence. Retraining can still happen.
`converged`	Treated like `running`: the champion still scores and is still validated.
`paused`	No scoring and no new training until someone resumes it in the UI. A run already in progress finishes.

The training run's iteration_count and best_holdout_score are written only when the run completes. A healthy run can show 0 iterations while its experiments are already recorded, so do not call a run stuck from the counter alone.

How to talk about results

Holdout AUC is the score on held-back training data. It says how well the model ranked people in the past. Treat it as a promise.
Realized AUC is the score against real outcomes after the horizon passes. Treat it as the proof. It wins over holdout AUC whenever it exists.
The UI labels quality Strong at AUC 0.8 and up, Fair at 0.7 and up, and Weak below that, and says whether the label comes from testing only or is confirmed by real outcomes. Use the same words so the agent and the UI agree.
Lift at 10% is the plain-language headline: "the top 10% are 3.2× more likely to do X than average".
The score is a probability, but on a rare target it is often best read as a ranking. Say so when the base rate is below a few percent.

reading-model-quality.md covers calibration, likelihood segments, coverage and what to say when the numbers disagree.

Guardrails worth keeping in mind

Changing the question means a new model. The fields a model was trained on (target, horizon, populations) are frozen once a model exists. Create a new model rather than editing one.
Autoresearch trains on identified users only. Anonymous traffic does not count toward training data, so raw event volume can overstate what is available. Trust the validation counts.
Scores are keyed on the person. A model whose scores are almost all the same value usually has an identifier problem, not a weak signal. See troubleshooting-and-steering.md.
Read the output property name, do not guess it. Take output_person_property from posthog:autoresearch-retrieve. A derived name includes the target and the horizon (predicted_p_<target>_<horizon>d), and templates use their own prefix, so two models on one target do not overwrite each other.
The research model sets most of the training cost. A training run uses the model's agent_model and agent_reasoning_effort. Null uses the default, Claude Opus 5.5 at medium effort, which costs about twice what Claude Sonnet 5.5 costs per run. effective_agent_model on posthog:autoresearch-retrieve shows what the next run uses, and posthog:autoresearch-partial-update changes it. An unknown model or an effort the model does not support returns 400 with the valid choices. posthog:autoresearch-training-runs-list shows the model each run used.
A model costs money to train and query to score. A very large population scores in rolling batches (about 45,000 people per run), so not everyone gets a fresh score every day. Validation tells you how many runs a full rescore takes.

Bring your own agent

An agent outside PostHog (on a laptop, for example) can train a model for an existing pipeline. It opens a run with posthog:autoresearch-training-runs-create, records each experiment with posthog:autoresearch-training-runs-iterations-create, uploads its bundle with posthog:autoresearch-training-runs-artifacts-upload-create, and finishes with posthog:autoresearch-training-runs-complete-create. It pulls training data with posthog:execute-sql, because posthog:autoresearch-materialize-features writes only into a PostHog sandbox. Read posthog:autoresearch-training-runs-history first to reuse what earlier runs on the same target learned. The iteration tool's description is the contract for valid feature SQL: read it in full before the first iteration.

Files

ATTRIBUTION.mdAdded by AutoGineer869 B
LICENSELicence file added by AutoGineer1.0 KiB
SKILL.md11.3 KiB
references/defining-a-question.md9.5 KiB
references/reading-model-quality.md6.3 KiB
references/troubleshooting-and-steering.md10.3 KiB
references/using-predictions.md6.6 KiB

references/using-predictions.md

Using predictions

A scored model writes its output in two places. Pick the one that fits the job.

Output	Holds	Use it for
The output person property	The newest probability for each scored person who resolves to a current distinct ID	Cohorts, feature flag targeting, workflow audiences, breakdowns
`autoresearch_prediction` events	One event per scored person per scoring run, with full history	Score history, comparisons across models, custom analysis in SQL

Take the property name from output_person_property on posthog:autoresearch-retrieve. A scored person whose identity cannot be resolved still gets an event, but no property. So a cohort on the property can hold fewer people than the run reports as scored.

Event properties

Property	Meaning
`$autoresearch_p_y`	The probability that the person does the target within the horizon, corrected for any negative sampling in training.
`$autoresearch_pipeline_id`	The model (pipeline) id. Always filter on it: a project can have several models.
`$autoresearch_model_id`	The champion that produced the score. Changes when a retrain promotes a new one.
`$autoresearch_model_role`	`champion`, or `shadow` for a challenger scored alongside it before any promotion. Shadow events set no property. Filter on `champion`.
`$autoresearch_prediction_date`	The date the score is for. The horizon starts here.
`$autoresearch_target_event` / `$autoresearch_horizon_days`	The question the score answers.
`$autoresearch_person_id`	The scored person. Use it as the person key: some prediction events have no person attached.

Common queries

Confirm the event exists with posthog:read-data-schema first. It appears only after the first scoring run.

Both queries read the newest prediction date only, so a manual score and the daily run do not mix, and a model that scores every few days still returns rows. On a large population that scores in rolling batches, the newest date holds only that day's batch.

Top people from the newest prediction date:

WITH latest AS (
    SELECT max(properties.$autoresearch_prediction_date) AS d
    FROM events
    WHERE event = 'autoresearch_prediction'
      AND properties.$autoresearch_pipeline_id = '<pipeline_id>'
      AND properties.$autoresearch_model_role = 'champion'
      AND timestamp > now() - INTERVAL 30 DAY
)
SELECT
    coalesce(nullIf(properties.$autoresearch_person_id, ''), distinct_id) AS person,
    argMax(toFloat(properties.$autoresearch_p_y), timestamp) AS p
FROM events
WHERE event = 'autoresearch_prediction'
  AND properties.$autoresearch_pipeline_id = '<pipeline_id>'
  AND properties.$autoresearch_model_role = 'champion'
  AND timestamp > now() - INTERVAL 30 DAY
  AND properties.$autoresearch_prediction_date = (SELECT d FROM latest)
GROUP BY person
ORDER BY p DESC
LIMIT 50

Score spread on the newest prediction date, one score per person:

WITH latest AS (
    SELECT max(properties.$autoresearch_prediction_date) AS d
    FROM events
    WHERE event = 'autoresearch_prediction'
      AND properties.$autoresearch_pipeline_id = '<pipeline_id>'
      AND properties.$autoresearch_model_role = 'champion'
      AND timestamp > now() - INTERVAL 30 DAY
)
SELECT
    count() AS people,
    min(p) AS min_p,
    quantiles(0.1, 0.5, 0.9, 0.99)(p) AS p10_p50_p90_p99,
    max(p) AS max_p
FROM (
    SELECT
        coalesce(nullIf(properties.$autoresearch_person_id, ''), distinct_id) AS person,
        argMax(toFloat(properties.$autoresearch_p_y), timestamp) AS p
    FROM events
    WHERE event = 'autoresearch_prediction'
      AND properties.$autoresearch_pipeline_id = '<pipeline_id>'
      AND properties.$autoresearch_model_role = 'champion'
      AND timestamp > now() - INTERVAL 30 DAY
      AND properties.$autoresearch_prediction_date = (SELECT d FROM latest)
    GROUP BY person
)

On a rare target, healthy scores sit close together near zero, so look at the quantiles rather than equal-width buckets. Quantiles that are all the same value point to an identifier problem. See troubleshooting-and-steering.md.

Acting on the scores

Cohort. A dynamic cohort on the output property, for example <output property> >= 0.6. It updates as scores change.
Feature flag. Target the flag with the same person property condition to show something only to the Likely group.
Workflow. A batch workflow with the property as its audience filter, for example an email to people at risk of going inactive (low activity score).
Insight. Break down the target event by the output property to show that higher scores really do the target more often.

The model page has shortcuts for these on the Predictions tab. They build the same filters.

Points worth telling the user:

For at_risk_of_inactivity, the at-risk group is a low score, because the model predicts activity.
Pick a threshold from the segments or the score spread, not a round number. On a rare target, 0.6 can match almost nobody.
People who were never scored have no property value. A filter like < 0.2 leaves them out, which is usually right, but check coverage on a large population.
To measure the impact of acting on a score, hold back a random part of the targeted group. The prediction says who is likely; only a holdout shows that the action changed anything.

Requirements and compatibility

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

Security

No issues found by our static scan· No issues found

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Attribution and licence

Published by

PostHog

Publisher

PostHog (Official)

Source

PostHog/skills · skills/omnibus/working-with-autoresearch · commit a5d30e7

Licence

MIT License (redistribution allowed)

LICENSE (added by AutoGineer from LICENSE)
ATTRIBUTION.md (written by AutoGineer)

Synced

2026-11-09

AutoGineer is not affiliated with or endorsed by PostHog or PostHog.

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download working-with-autoresearch.zip