Generated banner for Fleet Management
Generated logo for Fleet Management

Fleet Management

Grafana LabsOfficialPublished by Grafana Labs

  • Skill
  • Tool integration
  • Cloud and DevOps
  • Apache-2.0
  • Low risk

From the author

Manage a fleet of Grafana Alloy collectors with Fleet Management — author Alloy pipelines once, target them via attribute matchers (`env="production"`, regex `region=~"us-.*"`), push remotely via OpAMP without restarting collectors. Covers pipeline create / update / matcher RPCs, collector attribute API, `remotecfg` bootstrap block (standalone + Helm), pre-deploy `alloy fmt` validation, the local Alloy UI at port 12345 for component health, and post-deploy `REMOTE_CONFIG_STATUS_APPLIED` verification. Use when standing up a Cloud Alloy fleet, pushing a config change to 200 collectors, hunting why one collector shows `REMOTE_CONFIG_STATUS_FAILED`, validating River syntax before saving, or wiring `discovery.kubernetes` → `prometheus.remote_write` — even when the user says "configure Alloy", "remote config the collectors", "push pipeline", "OpAMP", "collector is unhealthy", or "manage agent config centrally" without naming Fleet Management.
— the description by Grafana Labs

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

Fleet Management allows for centralized control of Grafana Alloy collectors. By defining pipelines and using attribute-based matchers, users can push configurations to specific subsets of collectors without requiring manual restarts. The skill covers the full lifecycle of remote configuration, including local syntax validation, API-driven pipeline deployment, collector attribute management, and troubleshooting status codes.

What it does

  • Distributes Alloy River configurations to collectors remotely via OpAMP.
  • Targets specific collectors using attribute matchers like environment or region.
  • Validates Alloy River syntax locally using the alloy CLI before deployment.
  • Provides API endpoints to create, update, and roll back pipeline configurations.
  • Enables monitoring of collector health and remote configuration status.
  • Supports bootstrap configuration for both standalone and Helm-based Alloy deployments.

When to use it

  • When you need to manage configurations for a large fleet of Alloy collectors centrally.
  • When you want to apply configuration changes without restarting collector processes.
  • When troubleshooting why a collector is failing to apply a remote configuration.
  • When onboarding new Alloy instances that require automated remote configuration setup.

Advantages

  • Reduces manual overhead by automating configuration distribution.
  • Ensures configuration consistency across large-scale deployments.
  • Allows for granular targeting of collectors using custom attributes.
  • Supports safe deployments through pre-deploy syntax validation.
  • Provides visibility into the application status of remote configurations.

Usage examples

“How do I push a new Alloy pipeline to all production collectors?”
The agent will guide you through creating a pipeline via the PipelineService API, setting the appropriate matchers for 'env=production', and verifying the status of the targeted collectors.
“My Alloy collector shows REMOTE_CONFIG_STATUS_FAILED, how do I fix it?”
The agent will help you list the failed collectors, retrieve the error message, validate your local River syntax with alloy fmt, and check the collector's internal health via the local UI.
“How do I set up a new Alloy instance to use remote configuration?”
The agent will provide the necessary remotecfg bootstrap block for your standalone or Helm configuration, including instructions for setting required attributes.

Install

  1. Download the skill as a ZIP:

    https://autogineer.com/agent-skills/grafana/fleet-management/download
  2. Unzip it into the cross-client skills folder of your project:

    mkdir -p .agents/skills && unzip fleet-management.zip -d .agents/skills

Documentation by Grafana Labs

Grafana Fleet Management + Alloy Configuration

Docs: https://grafana.com/docs/grafana-cloud/send-data/fleet-management/

Remote pipeline distribution to Alloy collectors via OpAMP — author once, target with matchers, hot-apply (no restart).

Prerequisites

  • Grafana Cloud stack with Fleet Management enabled

  • API token with Fleet Management access (Authorization: Bearer <STACK_ID>:<TOKEN>)

  • Alloy ≥ 1.0 installed on the targets (standalone or via grafana/alloy Helm chart)

  • alloy CLI locally for alloy fmt syntax validation

Concepts

  • Collector — Alloy instance with unique ID + attributes

  • Pipeline — named Alloy River config stored in Fleet Management

  • Matcher — selector mapping a pipeline to collectors by attribute

  • Attributes — key/value labels on a collector (env, team, region)

Common Workflows

1. Author + validate + deploy a pipeline
# 1. Save the pipeline to a local file (lint catches typos before remote)
cat > pipeline.alloy <<'EOF'
prometheus.scrape "default" {
  targets    = []
  forward_to = [prometheus.remote_write.grafana_cloud.receiver]
  scrape_interval = "60s"
}

prometheus.remote_write "grafana_cloud" {
  endpoint {
    url = "https://prometheus-prod-01-eu-west-0.grafana.net/api/prom/push"
    basic_auth {
      username = "<METRICS_USERNAME>"
      password = env("GRAFANA_CLOUD_API_KEY")
    }
  }
}
EOF

# 2. Validate syntax LOCALLY before sending to Fleet Management
alloy fmt pipeline.alloy            # rewrites in place or errors with line number
alloy validate pipeline.alloy       # full semantic check (newer Alloy releases)

# 3. Create the pipeline via API (see references/api.md for the payload schema)
BASE=https://fleet-management-prod-us-east-0.grafana.net
TOKEN=<STACK_ID>:<API_TOKEN>
PAYLOAD=$(jq -n --rawfile c pipeline.alloy '{
  name:"k8s-metrics", contents:$c,
  matchers:[{name:"env",value:"production",type:"EQUAL"}]
}')
curl -s -X POST "$BASE/pipeline.v1.PipelineService/CreatePipeline" \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d "$PAYLOAD" | jq

# 4. Verify it rolled out — every targeted collector should report APPLIED within 1-2 polls
curl -s -X POST "$BASE/collector.v1.CollectorService/ListCollectors" \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -d '{}' \
  | jq '.collectors[] | select(.attributes[]?.value=="production")
        | {name, remoteConfigStatus}'
# Expect every row: remoteConfigStatus == "REMOTE_CONFIG_STATUS_APPLIED"
2. Troubleshoot a REMOTE_CONFIG_STATUS_FAILED collector
# 1. Find failed collectors and surface the error message
curl -s -X POST "$BASE/collector.v1.CollectorService/ListCollectors" \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -d '{}' \
  | jq '.collectors[] | select(.remoteConfigStatus=="REMOTE_CONFIG_STATUS_FAILED")
        | {name, msg:.remoteConfigStatusMessage}'

# 2. Re-validate the offending pipeline locally
alloy fmt pipeline.alloy

# 3. Inspect Alloy directly — UI at port 12345 shows per-component health
#    http://<COLLECTOR_HOST>:12345 → Graph / Components / Clustering tabs
kubectl -n monitoring logs -l app.kubernetes.io/name=alloy --tail=100 | grep -iE 'remote|error'

# 4. After fixing + re-pushing, re-list collectors and confirm the row flips to APPLIED.

Failure-message decoder table: references/api.md.

3. Onboard a new Alloy with the bootstrap block

The bootstrap remotecfg block is the only local config required:

remotecfg {
  url = "https://<FLEET_MANAGEMENT_HOST>"
  basic_auth { username = "<STACK_ID>"; password = env("GRAFANA_CLOUD_API_KEY") }
  poll_frequency = "1m"
  attributes = { "env" = env("ENVIRONMENT"), "team" = "platform" }
}
# Verify after start
curl -s http://localhost:12345/api/v0/web/components \
  | jq '.[] | select(.id=="remotecfg") | {id, health:.health.state}'
# health.state == "healthy"

Full bootstrap (standalone + Helm) + Assistant tool list: references/bootstrap.md.

Resources

Files

references/api.md

Fleet Management API

Endpoints are gRPC-Web — POST JSON to <host>/<service>.<RPC>.

BASE=https://fleet-management-prod-us-east-0.grafana.net
TOKEN=<STACK_ID>:<API_TOKEN>

Collectors

# List all collectors + health
curl -s -X POST "$BASE/collector.v1.CollectorService/ListCollectors" \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{}' | jq '.collectors[] | {id, name, remoteConfigStatus}'

# Update collector attributes (matchers target these)
curl -s -X POST "$BASE/collector.v1.CollectorService/UpdateCollector" \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{
    "id": "<COLLECTOR_ID>",
    "attributes": [
      {"name":"env",   "value":"production"},
      {"name":"team",  "value":"platform"},
      {"name":"region","value":"us-east-1"}
    ]
  }'

Auto-set attributes on registration: platform, arch, alloy_version.

Pipelines

# List
curl -s -X POST "$BASE/pipeline.v1.PipelineService/ListPipelines" \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -d '{}'

# Create (plain-text Alloy config, not base64)
curl -s -X POST "$BASE/pipeline.v1.PipelineService/CreatePipeline" \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{
    "name": "k8s-metrics",
    "contents": "prometheus.scrape \"default\" {\n  targets = []\n  forward_to = []\n}",
    "matchers": [{"name":"env","value":"production","type":"EQUAL"}]
  }'

# Update — set matchers
curl -s -X POST "$BASE/pipeline.v1.PipelineService/UpdatePipeline" \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{
    "id":"<PIPELINE_ID>",
    "matchers": [
      {"name":"env","value":"production","type":"EQUAL"},
      {"name":"team","value":"platform","type":"EQUAL"}
    ]
  }'

Matcher type values: EQUAL, NOT_EQUAL, REGEX, NOT_REGEX. A pipeline with no matchers is saved but deployed to zero collectors.

Matcher selector syntax (UI form)

OpExampleMeaning
=env="production"Exact match
!=env!="dev"Not equal
=~region=~"us-.*"Regex match
!~region!~"eu-.*"Regex not match

Alloy component categories

CategoryExample components
Discoverydiscovery.kubernetes, discovery.docker, discovery.relabel
Metricsprometheus.scrape, prometheus.remote_write, prometheus.operator.*
Logsloki.source.file, loki.source.kubernetes, loki.write
Tracesotelcol.receiver.otlp, otelcol.exporter.otlp
Profilespyroscope.scrape, pyroscope.write
Transformotelcol.processor.batch, otelcol.processor.filter

Common failure messages

Status messageRoot causeFix
syntax error at line NInvalid River syntaxRun alloy fmt before saving
component not found: XAlloy version too oldUpgrade Alloy
failed to unmarshal configEncoding error in API callSend plain text, not base64
authentication failedWrong tokenRotate and re-apply
connection refusedNetwork/firewallOpen egress to Fleet Management host

Pipeline Revision History

Every UpdatePipeline call creates a new revision:

# List revisions
curl -X POST $BASE/pipeline.v1.PipelineService/ListPipelineRevisions \
  -H "Authorization: Bearer $TOKEN" -d '{"pipelineId": "pipeline-id"}'

# Roll back to a previous revision
curl -X POST $BASE/pipeline.v1.PipelineService/RollbackPipeline \
  -H "Authorization: Bearer $TOKEN" \
  -d '{"pipelineId": "pipeline-id", "revisionId": "revision-id"}'

Tenant API

Returns rate limits and quotas for your Fleet Management tenant:

curl -X POST $BASE/tenant.v1.TenantService/GetLimits \
  -H "Authorization: Bearer $TOKEN" -d '{}'

# Response:
# {
#   "maxCollectors": 1000,
#   "maxPipelines": 100,
#   "maxPipelineSizeBytes": 1048576,
#   "maxMatchersPerPipeline": 10
# }

Matcher Types (API form)

When constructing the JSON payload (vs the UI selector syntax above), the API expects the enum form:

TypeDescriptionExample value
EQUALExact match"production"
NOT_EQUALNot equal"dev"
REGEXRE2 regex match"us-.*"
NOT_REGEXDoes not match regex"eu-.*"

remotecfg Block (Alloy)

remotecfg {
  url = "https://fleet-management-prod-us-east-0.grafana.net"

  basic_auth {
    username = sys.env("FM_INSTANCE_ID")   // Stack ID / instance ID
    password = sys.env("FM_API_KEY")
  }

  // Attributes used to match pipeline matchers
  attributes = {
    "env"    = "production",
    "region" = "us-east-1",
    "team"   = "platform",
  }

  poll_interval = "1m"   // How often to check for config updates
  id            = ""     // Auto-generated; persist across restarts for stable ID
}

Advanced Patterns

Staged Rollout
# 1. Create canary pipeline targeting one collector by attribute
POST .../CreatePipeline
{ "name": "canary-v2",
  "matchers": [{"name": "canary", "value": "true", "type": "EQUAL"}],
  "contents": "# New config v2..." }

# 2. Add canary=true attribute to test collector
POST .../UpdateCollector
{ "id": "collector-id", "attributes": [{"name": "canary", "value": "true"}] }

# 3. Validate metrics; expand to more collectors
# 4. Update main pipeline, remove canary pipeline
On-Demand Debug Pipeline
# Temporarily boost log verbosity for one collector
POST .../CreatePipeline
{ "name": "debug-001",
  "matchers": [{"name": "id", "value": "collector-001", "type": "EQUAL"}],
  "contents": "logging { level = \"debug\" }" }
# Delete when done
Kubernetes DaemonSet with Auto-Attributes
# Pass K8s metadata as env vars to Alloy
env:
  - name: K8S_NODE_NAME
    valueFrom: { fieldRef: { fieldPath: spec.nodeName } }
  - name: K8S_NAMESPACE
    valueFrom: { fieldRef: { fieldPath: metadata.namespace } }
remotecfg {
  url = sys.env("FM_URL")
  basic_auth {
    username = sys.env("FM_INSTANCE_ID")
    password = sys.env("FM_API_KEY")
  }
  attributes = {
    "k8s.node.name" = sys.env("K8S_NODE_NAME"),
    "k8s.namespace" = sys.env("K8S_NAMESPACE"),
    "env"           = "production",
  }
}

Requirements and compatibility

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

Needs: Needs network access, Needs an API key

Security

What our scanner noticed· Low risk

  • Instructions that use the network

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Attribution and licence

Published by
Grafana Labs
Publisher
Grafana Labs (Official)
Source
grafana/skills · skills/grafana-cloud/fleet-management · commit 8fb72a5
Licence
Apache License 2.0 (redistribution allowed)
  • LICENSE (added by AutoGineer from LICENSE)
  • ATTRIBUTION.md (written by AutoGineer)
Synced

AutoGineer is not affiliated with or endorsed by Grafana Labs or Grafana Labs.

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download fleet-management.zip