Prompt Optimizer

Prompt Optimizer Specification

Intent

prompt-optimizer improves reusable prompts through a contract-first, eval-backed workflow.

It should produce shorter, more reliable prompt packages with explicit model strategy, external context inventory, eval evidence, and residual risks.

Scope

In scope:

• New agent, system, developer, and reusable prompt templates.

• Refining existing prompts from failures or examples.

• Porting prompts across OpenAI, Claude, Gemini, or unknown model families.

• Prompt eval set design, candidate comparison, and holdout checks.

• Layering stable policy, task-local context, examples, tool policy, and external file references.

Out of scope:

• Choosing model architecture or fine-tuning strategy except to flag when prompting is not the bottleneck.

• Rewriting product docs, specs, or policies referenced by a prompt.

• Creating reusable agent skills.

• Debugging repository code unrelated to prompt behavior.

• Asking models to reveal hidden reasoning or chain-of-thought.

Users And Trigger Context

• Primary users: engineers and agents maintaining prompts for coding agents, product agents, eval harnesses, and model integrations.

• Should trigger for: improve a prompt, optimize a system prompt, rewrite an agent prompt, tune prompt wording, make prompts reliable, port prompts across model families, build prompt evals.

• Should not trigger for: normal code review, PR writing, skill authoring, generic documentation editing, or parameter-only tuning.

Runtime Contract

• Capture the prompt contract before edits: target model, prompt surfaces, layer owners, objective, non-goals, inputs, tools, output shape, success criteria, failures, and hard constraints.

• Build or request a small eval set when success criteria or examples are missing.

• Inventory stable external context by exact repo-relative path.

• Reference docs/specs/policies by path; paste only necessary excerpts.

• Keep one authoritative owner per behavior rule.

• Compare candidates on the same eval slice.

• Validate the selected prompt on holdout cases.

• Return a reusable package with target, success criteria, external context, optimized prompt, adapter notes, eval set, optimization log, and residual risks.

Source And Evidence Model

Authoritative sources:

• Local prompt-optimizer runtime files and references.

• Repository instructions and skill-writer authoring rules.

• Official OpenAI, Anthropic, and Gemini prompting and eval guidance.

• Prompt optimization research and framework docs already captured in SOURCES.md.

Useful improvement sources:

• positive examples: prompts that meet eval targets with fewer tokens and cleaner layering

• negative examples: prompts with duplicated policy, vague context, stale examples, or weak tool rules

• eval outputs: failure clusters, holdout regressions, candidate scores, and optimization logs

• model changes: provider docs or release notes showing changed behavior, tool APIs, or reasoning defaults

Do not store secrets, customer data, private policy text, or long copyrighted prompt/source excerpts in examples.

Reference Architecture

• SKILL.md contains the runtime workflow and output contract.

• SPEC.md contains this maintenance contract.

• SOURCES.md stores source inventory, decisions, coverage, gaps, and changelog.

• references/core-patterns.md covers prompt structure, layering, markers, external files, tool policy, and symptom fixes.

• references/meta-optimization-loop.md covers eval-backed iteration.

• references/model-family-notes.md covers provider adapters.

• references/transformed-examples.md contains compact examples for new prompts, repairs, and anti-pattern correction.

• scripts/ and assets/ are unused unless prompt scoring automation or reusable templates become necessary.

Evaluation

• Lightweight validation: run representative prompt tasks through the contract checklist and verify the package includes external context, eval cases, and residual risks.

• Candidate validation: compare all candidates on the same working slice and at least one holdout case.

• Trigger QA: confirm prompt optimization requests trigger this skill while skill writing, code review, and parameter-only tuning do not.

• Acceptance gates: prompt is shorter or behaviorally justified, rules have one owner, external files are exact, examples are causal, and residual risks name non-prompt bottlenecks.

Known Limitations

• Prompt behavior can change across model snapshots; the skill must recommend re-running evals after model changes.

• The skill cannot prove prompt quality without representative eval cases.

• External files referenced by path are useful only when the runtime agent can access them.

• Provider-specific advice can drift; refresh official docs when model or API behavior matters.

Maintenance Notes

• Update SKILL.md when runtime workflow, output package, or failure modes change.

• Update SPEC.md when scope, evidence policy, evaluation, or reference architecture changes.

• Update SOURCES.md when source inventory, decisions, coverage, gaps, or changelog entries change.

• Keep reference files focused; split any file that mixes unrelated lookup needs.