AutoGineer
Generated banner for Phoenix Tracing
Generated logo for Phoenix Tracing

Phoenix Tracing

GitHubOfficialFirst committed by jimbobbennettDistributed by GitHub

  • Skill
  • Tool integration
  • Data and AI
  • Apache-2.0
  • Low risk
Download ZIP

From the author

OpenInference semantic conventions and instrumentation for Phoenix AI observability. Use when implementing LLM tracing, creating custom spans, or deploying to production.
— the description by jimbobbennett

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

Phoenix Tracing provides a structured framework for implementing observability in LLM-based applications. It covers the full lifecycle of trace management, including setup, auto and manual instrumentation, span type definitions, and production deployment. The skill offers specific guidance for both Python and TypeScript environments, enabling developers to capture detailed telemetry for LLM calls, retrieval processes, agent reasoning, and evaluation feedback.

What it does

  • Provides setup instructions for Python and TypeScript SDKs.
  • Defines OpenInference semantic conventions for various span types like LLM, retriever, and agent.
  • Supports both auto-instrumentation and manual custom span creation.
  • Enables feedback collection through annotations on traces, spans, documents, and sessions.
  • Offers guidance on production concerns such as batch processing and PII masking.
  • Facilitates organization of trace data by projects and user sessions.

When to use it

  • When you need to implement observability for LLM applications.
  • When you want to track multi-step workflows or agent reasoning processes.
  • When you need to add custom metadata or feedback annotations to your traces.
  • When preparing your LLM application for production monitoring.
  • When you need to ensure your instrumentation follows OpenInference standards.

Advantages

  • Standardized tracing using OpenInference conventions.
  • Supports both Python and TypeScript ecosystems.
  • Granular control over span types and custom attributes.
  • Built-in support for human, LLM, and code-based feedback annotations.
  • Comprehensive documentation for both development and production deployment.

Usage examples

“How do I set up Phoenix tracing in my Python project?”
The agent provides the installation steps for the arize-phoenix-otel package and demonstrates how to configure the endpoint to begin capturing traces.
“How can I add a custom span for an LLM call in TypeScript?”
The agent guides you through using the @arizeai/phoenix-otel library to wrap your LLM API calls with the appropriate span attributes.
“How do I add human feedback to a specific span?”
The agent shows the code pattern for using the client.spans.add_span_annotation method, including how to specify the label, score, and explanation.
“What are the best practices for production tracing?”
The agent explains how to implement batch processing and PII masking to ensure efficient and secure trace data collection in a production environment.

Install

  1. Download the skill as a ZIP:

    https://autogineer.com/agent-skills/github/phoenix-tracing/download

  2. Unzip it into the cross-client skills folder of your project:

    mkdir -p .agents/skills && unzip phoenix-tracing.zip -d .agents/skills

Documentation by jimbobbennett

Phoenix Tracing

Comprehensive guide for instrumenting LLM applications with OpenInference tracing in Phoenix. Contains reference files covering setup, instrumentation, span types, and production deployment.

When to Apply

Reference these guidelines when:

  • Setting up Phoenix tracing (Python or TypeScript)

  • Creating custom spans for LLM operations

  • Adding attributes following OpenInference conventions

  • Deploying tracing to production

  • Querying and analyzing trace data

Reference Categories

PriorityCategoryDescriptionPrefix
1SetupInstallation and configurationsetup-*
2InstrumentationAuto and manual tracinginstrumentation-*
3Span Types9 span kinds with attributesspan-*
4OrganizationProjects and sessionsprojects-*, sessions-*
5EnrichmentCustom metadatametadata-*
6ProductionBatch processing, maskingproduction-*
7FeedbackAnnotations and evaluationannotations-*

Quick Reference

1. Setup (START HERE)
2. Instrumentation
3. Span Types (with full attribute schemas)
4. Organization
5. Enrichment
6. Production (CRITICAL)
7. Feedback
Reference Files

Common Workflows

  • Quick Start: setup-{lang} → instrumentation-auto-{lang} → Check Phoenix

  • Custom Spans: setup-{lang} → instrumentation-manual-{lang} → span-{type}

  • Session Tracking: sessions-{lang} for conversation grouping patterns

  • Production: production-{lang} for batching, masking, and deployment

How to Use This Skill

Navigation Patterns:

# By category prefix
references/setup-*              # Installation and configuration
references/instrumentation-*    # Auto and manual tracing
references/span-*               # Span type specifications
references/sessions-*           # Session tracking
references/production-*         # Production deployment
references/fundamentals-*       # Core concepts
references/attributes-*         # Attribute specifications

# By language
references/*-python.md          # Python implementations
references/*-typescript.md      # TypeScript implementations

Reading Order:

  1. Start with setup-{lang} for your language

  2. Choose instrumentation-auto-{lang} OR instrumentation-manual-{lang}

  3. Reference span-{type} files as needed for specific operations

  4. See fundamentals-* files for attribute specifications

References

Phoenix Documentation:

Python API Documentation:

TypeScript API Documentation:

  • TypeScript Packages - @arizeai/phoenix-otel, @arizeai/phoenix-client, and other TypeScript packages

Files

references/annotations-typescript.md

TypeScript SDK Annotation Patterns

Add feedback to spans, traces, documents, and sessions using the TypeScript client.

Client Setup

import { createClient } from "@arizeai/phoenix-client";
const client = createClient();  // Default: http://localhost:6006

Span Annotations

Add feedback to individual spans:

import { addSpanAnnotation } from "@arizeai/phoenix-client/spans";

await addSpanAnnotation({
  client,
  spanAnnotation: {
    spanId: "abc123",
    name: "quality",
    annotatorKind: "HUMAN",
    label: "high_quality",
    score: 0.95,
    explanation: "Accurate and well-formatted",
    metadata: { reviewer: "alice" }
  },
  sync: true
});

Span Notes

Notes are a special type of annotation for free-form text — useful for open coding, where reviewers leave qualitative observations on a span before any rubric exists. Later, those notes can be aggregated and distilled into structured labels or scores.

Notes are append-only: each call auto-generates a UUIDv4 identifier, so multiple notes naturally accumulate on the same span. Structured annotations are keyed by (name, spanId, identifier) — you can have many same-named annotations on one span by supplying distinct identifiers (e.g. one per reviewer); writing the same (name, spanId, identifier) overwrites the existing entry.

import { addSpanNote } from "@arizeai/phoenix-client/spans";

await addSpanNote({
  client,
  spanNote: {
    spanId: "abc123",
    note: "This span shows unexpected behavior, needs review"
  }
});

Document Annotations

Rate individual documents in RETRIEVER spans:

import { addDocumentAnnotation } from "@arizeai/phoenix-client/spans";

await addDocumentAnnotation({
  client,
  documentAnnotation: {
    spanId: "retriever_span",
    documentPosition: 0,  // 0-based index
    name: "relevance",
    annotatorKind: "LLM",
    label: "relevant",
    score: 0.95
  }
});

Trace Annotations

Feedback on entire traces:

import { addTraceAnnotation } from "@arizeai/phoenix-client/traces";

await addTraceAnnotation({
  client,
  traceAnnotation: {
    traceId: "trace_abc",
    name: "correctness",
    annotatorKind: "HUMAN",
    label: "correct",
    score: 1.0
  }
});

Trace Notes

Notes on entire traces (multiple notes allowed per trace):

import { addTraceNote } from "@arizeai/phoenix-client/traces";

await addTraceNote({
  client,
  traceNote: {
    traceId: "abc123def456",
    note: "Needs follow-up — unexpected tool call sequence"
  }
});

Session Annotations

Feedback on multi-turn conversations:

import { addSessionAnnotation } from "@arizeai/phoenix-client/sessions";

await addSessionAnnotation({
  client,
  sessionAnnotation: {
    sessionId: "session_xyz",
    name: "user_satisfaction",
    annotatorKind: "HUMAN",
    label: "satisfied",
    score: 0.85
  }
});

RAG Pipeline Example

import { createClient } from "@arizeai/phoenix-client";
import { logDocumentAnnotations, addSpanAnnotation } from "@arizeai/phoenix-client/spans";
import { addTraceAnnotation } from "@arizeai/phoenix-client/traces";

const client = createClient();

// Document relevance (batch)
await logDocumentAnnotations({
  client,
  documentAnnotations: [
    { spanId: "retriever_span", documentPosition: 0, name: "relevance",
      annotatorKind: "LLM", label: "relevant", score: 0.95 },
    { spanId: "retriever_span", documentPosition: 1, name: "relevance",
      annotatorKind: "LLM", label: "relevant", score: 0.80 }
  ]
});

// LLM response quality
await addSpanAnnotation({
  client,
  spanAnnotation: {
    spanId: "llm_span",
    name: "faithfulness",
    annotatorKind: "LLM",
    label: "faithful",
    score: 0.90
  }
});

// Overall trace quality
await addTraceAnnotation({
  client,
  traceAnnotation: {
    traceId: "trace_123",
    name: "correctness",
    annotatorKind: "HUMAN",
    label: "correct",
    score: 1.0
  }
});

API Reference

Requirements and compatibility

Written for: GitHub Copilot

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

Needs: Needs Python

The author says: Requires Phoenix server. Python skills need arize-phoenix-otel; TypeScript skills need @arizeai/phoenix-otel.

Security

What our scanner noticed· Low risk

  • Instructions that use the network

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Attribution and licence

First committed by
GitHub avatar of jimbobbennettjimbobbennett
Publisher
GitHub (Official)
Distributed by
GitHub
Source
github/awesome-copilot · skills/phoenix-tracing · commit 797337f
Licence
Apache License 2.0 (redistribution allowed)
  • LICENSE (written by AutoGineer)
  • ATTRIBUTION.md (written by AutoGineer)
Synced
Contributors
jimbobbennett, stbenjam

AutoGineer is not affiliated with or endorsed by jimbobbennett or GitHub.

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download phoenix-tracing.zip