Backtesting Frameworks

From the author

Build robust backtesting systems for trading strategies with proper handling of look-ahead bias, survivorship bias, and transaction costs. Use when developing trading algorithms, validating strategies, or building backtesting infrastructure.

— the description by Seth Hobson

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

This skill provides a structured approach to developing reliable backtesting infrastructure for trading algorithms. It emphasizes the mitigation of common analytical pitfalls such as look-ahead bias, survivorship bias, and overfitting. By implementing rigorous data handling, realistic transaction cost modeling, and validation techniques like walk-forward analysis, users can generate more accurate performance estimates for their trading strategies.

What it does

Defines core backtesting biases and their mitigation strategies.
Outlines a structured data pipeline for training, validation, and testing.
Introduces walk-forward analysis for temporal strategy validation.
Provides patterns for event-driven backtester architecture.
Establishes best practices for realistic cost and slippage modeling.
Encourages Monte Carlo analysis to assess strategy uncertainty.

When to use it

When developing or refining quantitative trading algorithms.
When building custom backtesting infrastructure from scratch.
When validating strategy performance to ensure statistical significance.
When comparing multiple strategy alternatives under consistent conditions.

Advantages

Reduces the risk of overfitting and selection bias.
Ensures realistic performance estimates by including transaction costs.
Promotes the use of point-in-time data to prevent look-ahead bias.
Provides a clear, modular structure for strategy and execution modeling.

Usage examples

“How do I structure my data to avoid look-ahead bias in a backtest?”: The skill explains the necessity of using point-in-time data and provides a clear separation between training, validation, and test sets to ensure future information does not leak into the model.
“What are the essential components of an event-driven backtester?”: The skill provides a Python-based pattern including classes for Orders, Fills, Portfolios, and Execution Models to simulate realistic trading environments.

Install

Download the skill as a ZIP:
https://autogineer.com/agent-skills/wshobson/backtesting-frameworks/download

Unzip it into the cross-client skills folder of your project:

mkdir -p .agents/skills && unzip backtesting-frameworks.zip -d .agents/skills

Documentation by Seth Hobson

Build robust, production-grade backtesting systems that avoid common pitfalls and produce reliable strategy performance estimates.

When to Use This Skill

Developing trading strategy backtests
Building backtesting infrastructure
Validating strategy performance
Avoiding common backtesting biases
Implementing walk-forward analysis
Comparing strategy alternatives

Core Concepts

1. Backtesting Biases

Bias	Description	Mitigation
Look-ahead	Using future information	Point-in-time data
Survivorship	Only testing on survivors	Use delisted securities
Overfitting	Curve-fitting to history	Out-of-sample testing
Selection	Cherry-picking strategies	Pre-registration
Transaction	Ignoring trading costs	Realistic cost models

2. Proper Backtest Structure

Historical Data
      │
      ▼
┌─────────────────────────────────────────┐
│              Training Set               │
│  (Strategy Development & Optimization)  │
└─────────────────────────────────────────┘
      │
      ▼
┌─────────────────────────────────────────┐
│             Validation Set              │
│  (Parameter Selection, No Peeking)      │
└─────────────────────────────────────────┘
      │
      ▼
┌─────────────────────────────────────────┐
│               Test Set                  │
│  (Final Performance Evaluation)         │
└─────────────────────────────────────────┘

3. Walk-Forward Analysis

Window 1: [Train──────][Test]
Window 2:     [Train──────][Test]
Window 3:         [Train──────][Test]
Window 4:             [Train──────][Test]
                                     ─────▶ Time

Detailed worked examples and patterns

Detailed sections (starting with ## Implementation Patterns) live in references/details.md. Read that file when the navigation summary above is insufficient.

Best Practices

Do's

Use point-in-time data - Avoid look-ahead bias
Include transaction costs - Realistic estimates
Test out-of-sample - Always reserve data
Use walk-forward - Not just train/test
Monte Carlo analysis - Understand uncertainty

Don'ts

Don't overfit - Limit parameters
Don't ignore survivorship - Include delisted
Don't use adjusted data carelessly - Understand adjustments
Don't optimize on full history - Reserve test set
Don't ignore capacity - Market impact matters

Files

ATTRIBUTION.mdAdded by AutoGineer1006 B
LICENSELicence file added by AutoGineer1.0 KiB
SKILL.md3.4 KiB
references/details.md17.8 KiB

Requirements and compatibility

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

Security

No issues found by our static scan· No issues found

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Process Letter
AnthropicOfficial
Draft professional process letters and bid instructions for sell-side M&A transactions.
Business and productivityClaude CodeClaude appsOpenAI Codex+14 more
Apache-2.0GitHub stars of the source repository41.1k
Break Trace
AnthropicOfficial
Diagnose the root cause of reconciliation breaks by tracing GL and subledger entries to their source.
Business and productivityClaude CodeClaude appsOpenAI Codex+14 more
Apache-2.0GitHub stars of the source repository41.1k
Analyzing Financial Statements
AnthropicOfficial
Calculate and interpret key financial ratios and metrics from financial statement data for investment analysis.
Business and productivityClaude CodeClaude appsOpenAI Codex+14 more
MITLow riskHas scriptsGitHub stars of the source repository54.4k
Creating Financial Models
AnthropicOfficial
A comprehensive toolkit for performing financial modeling, valuation, and risk assessment.
Business and productivityClaude CodeClaude appsOpenAI Codex+14 more
MITLow riskHas scriptsGitHub stars of the source repository54.4k
Risk Metrics Calculation
Seth Hobson· Community
A comprehensive toolkit for calculating and monitoring portfolio risk metrics including VaR, CVaR, and drawdown analysis.
Business and productivityClaude CodeClaude appsOpenAI Codex+14 more
MITGitHub stars of the source repository42.3k
Billing Automation
Seth Hobson· Community
A framework for building automated billing systems including recurring payments, invoicing, and subscription lifecycle management.
Business and productivityClaude CodeClaude appsOpenAI Codex+14 more
MITGitHub stars of the source repository42.3k

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download backtesting-frameworks.zip