AutoGineer
Generated banner for Managing Amazon Kinesis Data Streams
Generated logo for Managing Amazon Kinesis Data Streams

Managing Amazon Kinesis Data Streams

AWSOfficialPublished by AWS

  • Skill
  • Tool integration
  • Cloud and DevOps
  • Apache-2.0
  • No issues found
Download ZIP

From the author

Operates Amazon Kinesis Data Streams (KDS). Covers streaming tables and Amazon S3 delivery - serverless, fully managed delivery from a KDS stream to Apache Iceberg tables on S3 Tables or to general-purpose S3 buckets. Includes channel setup, IAM, schemas, output key templates, CloudWatch metrics and alarms, DLQ handling, quotas, and troubleshooting. For all other KDS topics and questions, search AWS documentation and blogs instead. Triggers: Kinesis Data Streams, KDS, streaming tables, stream to S3, stream to Iceberg, stream to S3 Tables, KDS delivery, KDS channel, CreateChannel, data channel, data freshness, dead-letter queue, KDS lakehouse, serverless Kinesis delivery, Firehose alternative for KDS, zero-ops Kinesis to S3. DO NOT USE for Kinesis Data Firehose, Kinesis Video Streams, or Managed Service for Apache Flink — use dedicated skills or search documentation instead.
— the description by AWS

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

This skill provides operational guidance for configuring and managing Amazon Kinesis Data Streams (KDS) delivery channels. It focuses on serverless, zero-ops delivery of stream data directly to Apache Iceberg tables on S3 Tables or general-purpose S3 buckets. The skill covers channel lifecycle management, IAM configuration, monitoring via CloudWatch, and troubleshooting common delivery issues, ensuring efficient data movement without the need for additional consumers or Flink applications.

What it does

  • Configures KDS channels for direct delivery to S3 or Apache Iceberg tables.
  • Provides IAM policy templates for secure service execution.
  • Explains monitoring strategies using CloudWatch metrics and delivery logs.
  • Outlines troubleshooting steps for failed channels and data freshness issues.
  • Defines hard limits and prerequisites for KDS delivery configurations.

When to use it

  • When you need to deliver KDS data to S3 or Iceberg tables without managing consumers.
  • When building a data lake or lakehouse architecture on AWS.
  • When you require a serverless, append-only delivery mechanism for streaming data.
  • When troubleshooting delivery failures or monitoring data freshness in KDS channels.

Advantages

  • Eliminates the need for managing Kinesis consumers or Flink applications.
  • Channels consume no shard capacity or enhanced fan-out slots.
  • Fully managed, serverless delivery architecture.
  • Supports direct integration with Apache Iceberg for queryable streaming data.

Usage examples

“How do I set up a channel to deliver Kinesis stream data to an S3 bucket?”
The agent provides the necessary IAM trust and permission policies, along with the configuration steps for the general-purpose S3 delivery channel.
“My Kinesis delivery channel is ACTIVE but no data is appearing in S3. How do I debug this?”
The agent guides you to check CloudWatch metrics like FailedRecordCount, verify IAM permissions, and enable delivery logging to identify potential access or schema errors.

Install

  1. Download the skill as a ZIP:

    https://autogineer.com/agent-skills/aws/managing-amazon-kinesis-data-streams/download

  2. Unzip it into the cross-client skills folder of your project:

    mkdir -p .agents/skills && unzip managing-amazon-kinesis-data-streams.zip -d .agents/skills

Documentation by AWS

Amazon Kinesis Data Streams

Overview

Domain expertise for Amazon Kinesis Data Streams. This skill covers streaming tables and Amazon S3 delivery, which provide serverless, fully managed delivery from a KDS stream directly to Apache Iceberg tables on S3 Tables or to general-purpose S3 buckets — no consumers, no Firehose, no Flink required for append-only delivery.

Delivery is configured as a channel on a stream via CreateChannel. Channels consume no shard capacity and no enhanced fan-out slots, so they do not compete with existing consumers.

The AWS MCP server is recommended for executing AWS operations. When the MCP server is not available, use the AWS CLI or shell commands instead.

Which Workflow Do You Need?

Customer IntentReference
Deliver stream data to Apache Iceberg tables on S3 Tables — setup, IAM, Glue schema, type mapping, partitioning, channel lifecyclereferences/streaming-tables.md
Deliver stream data to a general-purpose S3 bucket as objects — setup, IAM, compression, storage class, output key templates, channel lifecyclereferences/general-purpose-delivery-to-s3.md
Build a lakehouse / data lake from Kinesis; make streaming data queryable in Athenareferences/streaming-tables.md
Zero-ops, serverless alternative to Firehose or Kinesis consumers for KDS → S3 deliveryreferences/general-purpose-delivery-to-s3.md
CloudWatch metrics and alarms, delivery logging, CloudTrail, encryption, channel states, quotas, naming rules, what can be updatedreferences/monitoring-security-and-limits.md
Channel is ACTIVE but no data arrives; records landing in the DLQ; rising data freshnessStart with references/monitoring-security-and-limits.md, then the destination-specific troubleshooting table
Deliver to both S3 and S3 Tables from one streamSupported — create one channel per destination

Prerequisites

Both destinations require an ON_DEMAND stream — provisioned-mode streams are not supported. Each channel also needs an IAM service execution role trusted by kinesis.amazonaws.com, and a destination in the same Region as the stream.

Hard Limits to Check First

These rule out the feature entirely, so confirm them before designing a solution:

  • Append-only — no CDC, upserts, or deletes

  • No schema evolution — a schema change means deleting and recreating the channel

  • No backfill — only records produced after the channel is ACTIVE are delivered

  • No transformations — records are delivered as-is

  • New table per channel — cannot deliver into an existing Iceberg table

  • Data freshness is 300–900 seconds — not sub-minute

  • Same Region only — cross-account is supported, cross-Region is not

If any of these block the customer, recommend Managed Service for Apache Flink instead.

Additional Resources

Files

Requirements and compatibility

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

Needs: Needs an MCP server

Security

No issues found by our static scan· No issues found

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Attribution and licence

Published by
AWS
Publisher
AWS (Official)
Source
aws/agent-toolkit-for-aws · skills/specialized-skills/analytics-skills/managing-amazon-kinesis-data-streams · commit 0a218f8
Licence
Apache License 2.0 (redistribution allowed)
  • LICENSE (added by AutoGineer from LICENSE)
  • NOTICE (added by AutoGineer from NOTICE)
  • ATTRIBUTION.md (written by AutoGineer)
Synced

AutoGineer is not affiliated with or endorsed by AWS or AWS.

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download managing-amazon-kinesis-data-streams.zip