Dd Audit AI Activity — Agent Skill by Datadog

From the author

Audit what the Bits AI assistant (MCP server) has done in your Datadog org — tool calls by user, resources accessed, and anomaly flags for AI governance.

— the description by datadog-labs

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

This skill provides a structured approach to auditing the actions performed by the Bits AI assistant (MCP server) within a Datadog environment. By leveraging Audit Trail logs, it enables administrators to track tool calls, identify resource modifications, and detect potential anomalies. It is designed to support security reviews, compliance reporting, and organizational governance by ensuring that AI-driven operations are transparent, attributable, and within authorized scopes.

What it does

Retrieves logs of all MCP tool calls made by the AI assistant.
Identifies which users are invoking AI tools and the frequency of their usage.
Tracks specific resource modifications including creations, updates, and deletions.
Generates summary reports of AI activity over defined time windows.
Highlights potential security anomalies such as destructive actions or unusual access patterns.

When to use it

During routine security reviews to verify AI actions remain within authorized scopes.
When performing compliance audits to demonstrate accountability for AI-driven changes.
To generate governance reports on AI adoption and risk across the organization.
When investigating suspicious or unexpected modifications to monitors and dashboards.

Advantages

Provides granular visibility into AI-driven operations within Datadog.
Facilitates the identification of over-permissioned AI sessions.
Supports data-driven governance by surfacing top users and common action types.
Enables proactive detection of potentially destructive AI operations.

Usage examples

“Show me all resources modified by the AI assistant in the last week.”: The skill returns a list of timestamps, users, actions, and resource IDs for all creations, modifications, or deletions performed by the AI.
“Generate a weekly summary report of Bits AI activity.”: The skill outputs a structured report containing total tool calls, unique user counts, top users, action breakdowns, and resource type statistics.

Install

Download the skill as a ZIP:
https://autogineer.com/agent-skills/datadog-labs/dd-audit-ai-activity/download

Unzip it into the cross-client skills folder of your project:

mkdir -p .agents/skills && unzip dd-audit-ai-activity.zip -d .agents/skills

Documentation by datadog-labs

Audit Trail: AI Activity Audit

Every Datadog MCP tool call is recorded in Audit Trail under the Bits AI SRE category. This skill surfaces what the AI assistant has done in your org — which users invoked it, which tools were called, and which resources were affected.

Prerequisites

pup auth login   # OAuth2 (recommended)
# or set DD_API_KEY + DD_APP_KEY with audit_logs_read scope

Queries

All MCP tool activity in a time window

pup audit-logs search --query "@evt.name:\"MCP Server\"" --from 7d --limit 500 -o json \
  | jq '[.data[] | {
      timestamp: .attributes.timestamp,
      user: .attributes.attributes.usr.email,
      actor_type: .attributes.attributes.evt.actor.type,
      action: .attributes.attributes.action,
      resource_type: .attributes.attributes.asset.type,
      resource_id: .attributes.attributes.asset.id,
      ip: .attributes.attributes.network.client.ip,
      country: .attributes.attributes.network.client.geoip.country.name
    }]'

Activity by user (who is using the AI assistant most?)

pup audit-logs search --query "@evt.name:\"MCP Server\"" --from 30d --limit 1000 -o json \
  | jq '[.data[] | .attributes.attributes.usr.email]
    | group_by(.)
    | map({user: .[0], tool_calls: length})
    | sort_by(-.tool_calls)'

Resources modified by AI tool calls

pup audit-logs search \
  --query "@evt.name:\"MCP Server\" @action:(created OR modified OR deleted)" \
  --from 7d --limit 500 -o json \
  | jq '[.data[] | {
      timestamp: .attributes.timestamp,
      user: .attributes.attributes.usr.email,
      action: .attributes.attributes.action,
      resource_type: .attributes.attributes.asset.type,
      resource_id: .attributes.attributes.asset.id
    }]'

AI activity for a specific user

pup audit-logs search \
  --query "@evt.name:\"MCP Server\" @usr.email:[email protected]" \
  --from 30d --limit 500 -o json \
  | jq '[.data[] | {
      timestamp: .attributes.timestamp,
      action: .attributes.attributes.action,
      resource_type: .attributes.attributes.asset.type,
      resource_id: .attributes.attributes.asset.id
    }]'

Weekly summary report

pup audit-logs search --query "@evt.name:\"MCP Server\"" --from 7d --limit 1000 -o json \
  | jq '{
      total_tool_calls: (.data | length),
      unique_users: ([.data[] | .attributes.attributes.usr.email] | unique | length),
      top_users: (
        [.data[] | .attributes.attributes.usr.email]
        | group_by(.)
        | map({user: .[0], calls: length})
        | sort_by(-.calls)
        | .[:5]
      ),
      actions_breakdown: (
        [.data[] | .attributes.attributes.action]
        | group_by(.)
        | map({action: .[0], count: length})
        | sort_by(-.count)
      ),
      resource_types: (
        [.data[] | .attributes.attributes.asset.type]
        | group_by(.)
        | map({type: .[0], count: length})
        | sort_by(-.count)
      )
    }'

Anomaly Flags

Signal	Governance concern
AI performing `deleted` actions on monitors or dashboards	Review whether destructive AI operations are expected
AI acting as `SUPPORT_USER`	Datadog support using AI on behalf of org
First-time user invoking AI tools	New user accessing AI assistant
High volume of tool calls in short window	Automated/batch AI usage
AI accessing resources outside user's normal scope	Potential over-permissioned AI session

Output Format

AI Activity Audit — [Org] — [Date Range]

Total MCP tool calls: [N]
Unique users: [N]

Top users:
  [[email protected]]: [N] calls

Actions breakdown:
  accessed: [N]
  modified: [N]
  created: [N]
  deleted: [N]

Resource types affected:
  dashboard: [N]
  monitor: [N]

Anomalies:
  [List any flagged events with timestamp, user, action, resource]

Context

This skill is most useful for:

Security reviews: Verifying AI actions were authorized and within expected scope
Compliance audits: Demonstrating AI activity is logged and attributable to specific users
Governance reports: Understanding adoption and risk surface of the AI assistant across the org

No other observability vendor audits their AI assistant's actions at this level of detail.

References

Files

ATTRIBUTION.mdAdded by AutoGineer971 B
LICENSELicence file added by AutoGineer1.1 KiB
SKILL.md4.9 KiB

SKILL.md

Audit Trail: AI Activity Audit

Every Datadog MCP tool call is recorded in Audit Trail under the Bits AI SRE category. This skill surfaces what the AI assistant has done in your org — which users invoked it, which tools were called, and which resources were affected.

Prerequisites

pup auth login   # OAuth2 (recommended)
# or set DD_API_KEY + DD_APP_KEY with audit_logs_read scope

Queries

All MCP tool activity in a time window

pup audit-logs search --query "@evt.name:\"MCP Server\"" --from 7d --limit 500 -o json \
  | jq '[.data[] | {
      timestamp: .attributes.timestamp,
      user: .attributes.attributes.usr.email,
      actor_type: .attributes.attributes.evt.actor.type,
      action: .attributes.attributes.action,
      resource_type: .attributes.attributes.asset.type,
      resource_id: .attributes.attributes.asset.id,
      ip: .attributes.attributes.network.client.ip,
      country: .attributes.attributes.network.client.geoip.country.name
    }]'

Activity by user (who is using the AI assistant most?)

pup audit-logs search --query "@evt.name:\"MCP Server\"" --from 30d --limit 1000 -o json \
  | jq '[.data[] | .attributes.attributes.usr.email]
    | group_by(.)
    | map({user: .[0], tool_calls: length})
    | sort_by(-.tool_calls)'

Resources modified by AI tool calls

pup audit-logs search \
  --query "@evt.name:\"MCP Server\" @action:(created OR modified OR deleted)" \
  --from 7d --limit 500 -o json \
  | jq '[.data[] | {
      timestamp: .attributes.timestamp,
      user: .attributes.attributes.usr.email,
      action: .attributes.attributes.action,
      resource_type: .attributes.attributes.asset.type,
      resource_id: .attributes.attributes.asset.id
    }]'

AI activity for a specific user

pup audit-logs search \
  --query "@evt.name:\"MCP Server\" @usr.email:[email protected]" \
  --from 30d --limit 500 -o json \
  | jq '[.data[] | {
      timestamp: .attributes.timestamp,
      action: .attributes.attributes.action,
      resource_type: .attributes.attributes.asset.type,
      resource_id: .attributes.attributes.asset.id
    }]'

Weekly summary report

pup audit-logs search --query "@evt.name:\"MCP Server\"" --from 7d --limit 1000 -o json \
  | jq '{
      total_tool_calls: (.data | length),
      unique_users: ([.data[] | .attributes.attributes.usr.email] | unique | length),
      top_users: (
        [.data[] | .attributes.attributes.usr.email]
        | group_by(.)
        | map({user: .[0], calls: length})
        | sort_by(-.calls)
        | .[:5]
      ),
      actions_breakdown: (
        [.data[] | .attributes.attributes.action]
        | group_by(.)
        | map({action: .[0], count: length})
        | sort_by(-.count)
      ),
      resource_types: (
        [.data[] | .attributes.attributes.asset.type]
        | group_by(.)
        | map({type: .[0], count: length})
        | sort_by(-.count)
      )
    }'

Anomaly Flags

Signal	Governance concern
AI performing `deleted` actions on monitors or dashboards	Review whether destructive AI operations are expected
AI acting as `SUPPORT_USER`	Datadog support using AI on behalf of org
First-time user invoking AI tools	New user accessing AI assistant
High volume of tool calls in short window	Automated/batch AI usage
AI accessing resources outside user's normal scope	Potential over-permissioned AI session

Output Format

AI Activity Audit — [Org] — [Date Range]

Total MCP tool calls: [N]
Unique users: [N]

Top users:
  [[email protected]]: [N] calls

Actions breakdown:
  accessed: [N]
  modified: [N]
  created: [N]
  deleted: [N]

Resource types affected:
  dashboard: [N]
  monitor: [N]

Anomalies:
  [List any flagged events with timestamp, user, action, resource]

Context

This skill is most useful for:

Security reviews: Verifying AI actions were authorized and within expected scope
Compliance audits: Demonstrating AI activity is logged and attributable to specific users
Governance reports: Understanding adoption and risk surface of the AI assistant across the org

No other observability vendor audits their AI assistant's actions at this level of detail.

References

Requirements and compatibility

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

Needs: Needs an MCP server, Needs an API key

Security

No issues found by our static scan· No issues found

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Security Audit
CloudflareOfficial
A structured, source-first security audit and vulnerability research framework for codebases and services.
SecurityClaude CodeClaude appsOpenAI Codex+14 more
MITLow riskHas scriptsGitHub stars of the source repository65.3k
Sast Configuration
Seth Hobson· Community
Configure and optimize Static Application Security Testing (SAST) tools like Semgrep, SonarQube, and CodeQL for automated vulnerability detection.
SecurityClaude CodeClaude appsOpenAI Codex+14 more
MITLow riskGitHub stars of the source repository42.3k
Firmware Analyst
Seth Hobson· Community
An expert firmware analyst providing structured guidance for embedded systems security, reverse engineering, and vulnerability research.
SecurityClaude Code
MITLow riskGitHub stars of the source repository42.3k
Policy Enforcer
Seth Hobson· Community
Expert guide for authoring and auditing Cedar authorization policies to secure Claude Code agent tool interactions.
SecurityClaude Code
MITGitHub stars of the source repository42.3k
Backend Development Security Auditor
Seth Hobson· Community
A specialized security auditor for identifying vulnerabilities in backend code and architecture during feature development.
SecurityClaude Code
MITGitHub stars of the source repository42.3k
Memory Forensics
Seth Hobson· Community
Master memory forensics and incident response using the Volatility framework and industry-standard acquisition tools.
SecurityClaude CodeClaude appsOpenAI Codex+14 more
MITLow riskGitHub stars of the source repository42.3k

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download dd-audit-ai-activity.zip

From the author

AutoGineer summary

What it does

When to use it

Advantages

Usage examples

Install

Documentation by datadog-labs

Audit Trail: AI Activity Audit

Prerequisites

Queries

All MCP tool activity in a time window

Activity by user (who is using the AI assistant most?)

Resources modified by AI tool calls

AI activity for a specific user

Weekly summary report

Anomaly Flags

Output Format

Context

References

Files

SKILL.md

Audit Trail: AI Activity Audit

Prerequisites

Queries

All MCP tool activity in a time window

Activity by user (who is using the AI assistant most?)

Resources modified by AI tool calls

AI activity for a specific user

Weekly summary report

Anomaly Flags

Output Format

Context

References

Requirements and compatibility

Security

Attribution and licence

Related skills

Security Audit

Sast Configuration

Firmware Analyst

Policy Enforcer

Backend Development Security Auditor

Memory Forensics

Download ZIP