AutoGineer
Generated banner for Dd Audit AI Activity
Generated logo for Dd Audit AI Activity

Dd Audit AI Activity

DatadogOfficialCreated by datadog-labs

  • Skill
  • Review and audit
  • Security
  • MIT
  • No issues found
Download ZIP

From the author

Audit what the Bits AI assistant (MCP server) has done in your Datadog org — tool calls by user, resources accessed, and anomaly flags for AI governance.
— the description by datadog-labs

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

This skill provides a structured approach to auditing the actions performed by the Bits AI assistant (MCP server) within a Datadog environment. By leveraging Audit Trail logs, it enables administrators to track tool calls, identify resource modifications, and detect potential anomalies. It is designed to support security reviews, compliance reporting, and organizational governance by ensuring that AI-driven operations are transparent, attributable, and within authorized scopes.

What it does

  • Retrieves logs of all MCP tool calls made by the AI assistant.
  • Identifies which users are invoking AI tools and the frequency of their usage.
  • Tracks specific resource modifications including creations, updates, and deletions.
  • Generates summary reports of AI activity over defined time windows.
  • Highlights potential security anomalies such as destructive actions or unusual access patterns.

When to use it

  • During routine security reviews to verify AI actions remain within authorized scopes.
  • When performing compliance audits to demonstrate accountability for AI-driven changes.
  • To generate governance reports on AI adoption and risk across the organization.
  • When investigating suspicious or unexpected modifications to monitors and dashboards.

Advantages

  • Provides granular visibility into AI-driven operations within Datadog.
  • Facilitates the identification of over-permissioned AI sessions.
  • Supports data-driven governance by surfacing top users and common action types.
  • Enables proactive detection of potentially destructive AI operations.

Usage examples

“Show me all resources modified by the AI assistant in the last week.”
The skill returns a list of timestamps, users, actions, and resource IDs for all creations, modifications, or deletions performed by the AI.
“Generate a weekly summary report of Bits AI activity.”
The skill outputs a structured report containing total tool calls, unique user counts, top users, action breakdowns, and resource type statistics.

Install

  1. Download the skill as a ZIP:

    https://autogineer.com/agent-skills/datadog-labs/dd-audit-ai-activity/download

  2. Unzip it into the cross-client skills folder of your project:

    mkdir -p .agents/skills && unzip dd-audit-ai-activity.zip -d .agents/skills

Documentation by datadog-labs

Audit Trail: AI Activity Audit

Every Datadog MCP tool call is recorded in Audit Trail under the Bits AI SRE category. This skill surfaces what the AI assistant has done in your org — which users invoked it, which tools were called, and which resources were affected.

Prerequisites

pup auth login   # OAuth2 (recommended)
# or set DD_API_KEY + DD_APP_KEY with audit_logs_read scope

Queries

All MCP tool activity in a time window
pup audit-logs search --query "@evt.name:\"MCP Server\"" --from 7d --limit 500 -o json \
  | jq '[.data[] | {
      timestamp: .attributes.timestamp,
      user: .attributes.attributes.usr.email,
      actor_type: .attributes.attributes.evt.actor.type,
      action: .attributes.attributes.action,
      resource_type: .attributes.attributes.asset.type,
      resource_id: .attributes.attributes.asset.id,
      ip: .attributes.attributes.network.client.ip,
      country: .attributes.attributes.network.client.geoip.country.name
    }]'
Activity by user (who is using the AI assistant most?)
pup audit-logs search --query "@evt.name:\"MCP Server\"" --from 30d --limit 1000 -o json \
  | jq '[.data[] | .attributes.attributes.usr.email]
    | group_by(.)
    | map({user: .[0], tool_calls: length})
    | sort_by(-.tool_calls)'
Resources modified by AI tool calls
pup audit-logs search \
  --query "@evt.name:\"MCP Server\" @action:(created OR modified OR deleted)" \
  --from 7d --limit 500 -o json \
  | jq '[.data[] | {
      timestamp: .attributes.timestamp,
      user: .attributes.attributes.usr.email,
      action: .attributes.attributes.action,
      resource_type: .attributes.attributes.asset.type,
      resource_id: .attributes.attributes.asset.id
    }]'
AI activity for a specific user
pup audit-logs search \
  --query "@evt.name:\"MCP Server\" @usr.email:[email protected]" \
  --from 30d --limit 500 -o json \
  | jq '[.data[] | {
      timestamp: .attributes.timestamp,
      action: .attributes.attributes.action,
      resource_type: .attributes.attributes.asset.type,
      resource_id: .attributes.attributes.asset.id
    }]'
Weekly summary report
pup audit-logs search --query "@evt.name:\"MCP Server\"" --from 7d --limit 1000 -o json \
  | jq '{
      total_tool_calls: (.data | length),
      unique_users: ([.data[] | .attributes.attributes.usr.email] | unique | length),
      top_users: (
        [.data[] | .attributes.attributes.usr.email]
        | group_by(.)
        | map({user: .[0], calls: length})
        | sort_by(-.calls)
        | .[:5]
      ),
      actions_breakdown: (
        [.data[] | .attributes.attributes.action]
        | group_by(.)
        | map({action: .[0], count: length})
        | sort_by(-.count)
      ),
      resource_types: (
        [.data[] | .attributes.attributes.asset.type]
        | group_by(.)
        | map({type: .[0], count: length})
        | sort_by(-.count)
      )
    }'

Anomaly Flags

SignalGovernance concern
AI performing deleted actions on monitors or dashboardsReview whether destructive AI operations are expected
AI acting as SUPPORT_USERDatadog support using AI on behalf of org
First-time user invoking AI toolsNew user accessing AI assistant
High volume of tool calls in short windowAutomated/batch AI usage
AI accessing resources outside user's normal scopePotential over-permissioned AI session

Output Format

AI Activity Audit — [Org] — [Date Range]

Total MCP tool calls: [N]
Unique users: [N]

Top users:
  [[email protected]]: [N] calls

Actions breakdown:
  accessed: [N]
  modified: [N]
  created: [N]
  deleted: [N]

Resource types affected:
  dashboard: [N]
  monitor: [N]

Anomalies:
  [List any flagged events with timestamp, user, action, resource]

Context

This skill is most useful for:

  • Security reviews: Verifying AI actions were authorized and within expected scope

  • Compliance audits: Demonstrating AI activity is logged and attributable to specific users

  • Governance reports: Understanding adoption and risk surface of the AI assistant across the org

No other observability vendor audits their AI assistant's actions at this level of detail.

References

Files

Requirements and compatibility

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

Needs: Needs an MCP server, Needs an API key

Security

No issues found by our static scan· No issues found

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Attribution and licence

Created by
datadog-labs
Publisher
Datadog (Official)
Source
datadog-labs/agent-skills · dd-audit/ai-activity-audit · commit 8e1306a
Licence
MIT License (redistribution allowed)
  • LICENSE (added by AutoGineer from LICENSE)
  • ATTRIBUTION.md (written by AutoGineer)
Synced

AutoGineer is not affiliated with or endorsed by datadog-labs or Datadog.

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download dd-audit-ai-activity.zip