Dsql
AWS LabsOfficialPublished by AWS Labs
- Skill
- Tool integration
- Data and AI
- Apache-2.0
- Low risk
- Has scripts
AutoGineer summary
Written by AutoGineer from the skill’s own files, not by the author
The dsql skill provides a structured framework for interacting with Amazon Aurora DSQL. It encompasses schema management, migration strategies from MySQL and PostgreSQL, performance diagnostics, and bulk data loading. The skill integrates with MCP tools for direct database operations and provides extensive documentation on IAM authentication, multi-tenant patterns, and transaction management to ensure safe and efficient database operations.
What it does
- Executes SQL queries and DDL operations via MCP tools.
- Provides migration paths and type mapping for MySQL and PostgreSQL to DSQL.
- Offers diagnostic workflows for cluster performance and query plan explainability.
- Facilitates bulk data loading using specialized loaders.
- Enforces best practices for IAM authentication and scoped database roles.
- Validates SQL compatibility and provides linting for DSQL-specific constraints.
When to use it
- When building or migrating applications to Aurora DSQL.
- When performing schema changes or complex DDL migrations.
- When troubleshooting performance issues or query plan inefficiencies.
- When setting up multi-tenant database architectures.
- When bulk loading data into a DSQL cluster.
Advantages
- Ensures adherence to DSQL-specific transaction and concurrency limits.
- Promotes secure access patterns through IAM and scoped roles.
- Provides clear, phased workflows for complex tasks like performance tuning.
- Reduces migration friction with automated linting and type mapping guidance.
- Standardizes diagnostic reporting for query plan analysis.
Usage examples
- “How do I migrate my existing PostgreSQL schema to Aurora DSQL?”
- The agent will guide you through pg-migrations/type-mapping.md and schema-objects.md, providing steps to convert indexes, enums, and data types while ensuring compatibility.
- “My query is running slowly on DSQL, can you help?”
- The agent will initiate Workflow 9 (Query Plan Explainability) to analyze the query plan, check for full scans, and provide a structured diagnostic report.
- “How do I set up a multi-tenant table in DSQL?”
- The agent will provide instructions on using tenant_id in your schema design, enforcing it in WHERE clauses, and using separate transact calls for DDL.
- “What are the transaction limits for DSQL?”
- The agent will consult the limits-and-error-codes.md or awsknowledge tool to provide the exact limits for row modifications, data size, and duration.
Install
Download the skill as a ZIP:
https://autogineer.com/agent-skills/awslabs/dsql/downloadUnzip it into the cross-client skills folder of your project:
mkdir -p .agents/skills && unzip dsql.zip -d .agents/skills
Documentation by AWS Labs
Amazon Aurora DSQL Skill
Aurora DSQL is a serverless, PostgreSQL-compatible distributed SQL database. This skill covers direct query execution via MCP tools, schema management, migrations, multi-tenant isolation, IAM auth, and bulk data loading via aurora-dsql-loader.
Reference Files
Load these files as needed for detailed guidance:
Core:
| Reference | When to Load | Contains |
|---|---|---|
| development-guide.md | ALWAYS before schema changes or DB operations | Best practices, DDL rules, transaction limits, foreign key constraints |
| foreign-keys.md | MUST load for foreign key operations or migrations | FK syntax, actions, validation, tenant keys |
| language.md | MUST load for language-specific choices | Driver selection, DSQL Connectors, connection code |
| access-control.md | MUST load for roles, grants, or sensitive data | Scoped role setup, IAM-to-database role mapping |
| troubleshooting.md | SHOULD load for errors or unexpected behavior | OCC and 23503, FK, connections, cluster state |
| limits-and-error-codes.md | SHOULD load before sizing batches, keys or indexes, or on 54000/0A000/54011 | Every limit, rejection and quota with SQLSTATE and exact message |
| dsql-examples.md | Load for implementation examples | Multi-tenant access, batch operations, identity and sequences, connection pooling |
| onboarding.md | User requests "Get started with DSQL" | Interactive step-by-step guide |
| occ-retry-patterns.md | MUST load for OCC retry code or conflict mitigation | Connectors, 40001 retry, non-retryable 23503, FK read conflicts, idempotent design |
MCP:
| Reference | When to Load | Contains |
|---|---|---|
| mcp-setup.md | Always for MCP server guidance | Setup instructions, 2 configuration options |
| mcp-tools.md | For MCP tool syntax and examples | Tool parameters, input validation |
| dsql-lint.md | MUST load before running dsql_lint or processing external SQL | Tool reference, fix statuses, unfixable error resolution |
DDL Migrations:
| Reference | When to Load | Contains |
|---|---|---|
| ddl-migrations/overview.md | MUST load for ALTER TYPE, SET NOT NULL, MODIFY PRIMARY KEY | Direct ALTER coverage and last-resort table recreation |
| ddl-migrations/column-operations.md | DROP COLUMN, ALTER TYPE, SET/DROP NOT NULL/DEFAULT | Column-level migration patterns |
| ddl-migrations/constraint-operations.md | ADD/DROP CONSTRAINT, VALIDATE CONSTRAINT, MODIFY PRIMARY KEY | Constraint and structural changes |
| ddl-migrations/batched-migration.md | Tables exceeding 3,000 rows | Batching patterns, progress tracking |
MySQL Migrations:
| Reference | When to Load | Contains |
|---|---|---|
| mysql-migrations/type-mapping.md | MUST load for MySQL → DSQL migration | Data type mappings, feature alternatives |
| mysql-migrations/ddl-operations.md | Translating MySQL DDL to DSQL | AUTO_INCREMENT, ENUM, SET, FK patterns |
| mysql-migrations/full-example.md | Complete MySQL table migration | End-to-end example with decision summary |
PostgreSQL Migrations:
| Reference | When to Load | Contains |
|---|---|---|
| pg-migrations/type-mapping.md | MUST load for DSQL NUMERIC or PG type questions | C collation rules, NUMERIC(p,s), JSON/JSONB |
| pg-migrations/index-conversion.md | MUST load for any index conversion or index support question | GIN/GiST/BRIN → btree, operator class removal, partial + expression indexes, sort order |
| pg-migrations/schema-objects.md | MUST load for ENUM, materialized views, extensions, multi-schema, CREATE FUNCTION | ENUM → CHECK, views, role/IAM mapping, LANGUAGE sql functions |
| pg-migrations/multi-region.md | Multi-region, active-active, or HA questions | Architecture, geographic partitioning |
ORM Guides:
| Reference | When to Load | Contains |
|---|---|---|
| orm-guides/overview.md | Migrating an ORM or using ORM locking with DSQL | Adapter names, locking, and key gotchas for Django/EF Core/Hibernate/Rails/SQLAlchemy |
Data Loading:
| Reference | When to Load | Contains |
|---|---|---|
| data-loading.md | Planning or running bulk loads with aurora-dsql-loader | Fresh-vs-warm partitions, resume/retry, --on-conflict semantics, throughput diagnostics |
System Diagnostics:
| Reference | When to Load | Contains |
|---|---|---|
| system-diagnostics/workflow.md | MUST load at Workflow 12 entry — cluster performance diagnostics | Prerequisites, 5 diagnostic phases, temporal comparison, handoff |
| system-diagnostics/wait-events.md | ALWAYS load when interpreting AAS results | Canonical DSQL wait event descriptions and investigation guidance |
| system-diagnostics/promql-patterns.md | Load when constructing PromQL queries | Reusable query templates for AAS breakdown, top-SQL, temporal compare |
Query Plan Explainability:
| Reference | When to Load | Contains |
|---|---|---|
| query-plan/workflow.md | MUST load at Workflow 9 entry — gates all other files | Trigger criteria, context disambiguation, routing, phased workflow |
| query-plan/plan-interpretation.md | MUST load at Workflow 9 Phase 0 | DSQL node types, Node Duration math, estimation-error bands |
| query-plan/catalog-queries.md | MUST load at Workflow 9 Phase 0 | pg_class/pg_stats/pg_indexes SQL, correlated-predicate verification |
| query-plan/guc-experiments.md | MUST load at Workflow 9 Phase 0 | GUC experiment procedures, 30-second skip protocol |
| query-plan/report-format.md | MUST load at Workflow 9 Phase 0 | Required report structure, element checklist, support request template |
| query-plan/query-rewrites-generic.md | SHOULD load at Phase 0; sub-files on-demand | Index of 10 generic rewrite patterns |
| query-plan/query-rewrites-dsql-specific.md | SHOULD load at Phase 0; sub-files on-demand | Index of DSQL-specific rewrite patterns |
Choosing How to Connect: MCP vs CLI/psql
The aurora-dsql MCP server binds a single cluster at startup (--cluster_endpoint), so
using it for another cluster means editing .mcp.json and restarting the session.
Use the
aurora-dsqlMCP tools (readonly_query,transact,get_schema) ONLY when the server already targets the cluster you need.Otherwise — unconfigured, disabled, or bound to a different cluster — do NOT reconfigure it. Use the CLI +
psqlpath instead:scripts/psql-connect.sh<cluster-id> --region <region> --command "SELECT ..."(mints an IAM token and runs viapsql).If you cannot confirm which cluster the MCP targets, confirm first or use the CLI/psql path — running against the wrong cluster is worse than the check.
The doc-only MCP tools (dsql_lint, dsql_*_documentation, dsql_recommend) need no cluster.
The CloudWatch MCP (Workflow 12) takes region/cluster_id per call, so one running server can
query clusters in any PromQL-enabled region (pass each cluster's region on the call). Details:
connectivity-tools.md.
MCP Tools Available
The aurora-dsql MCP server provides these tools:
Database Operations:
readonly_query - Execute SELECT queries (returns list of dicts)
transact - Execute DDL/DML statements in transaction (takes list of SQL statements)
get_schema - Get table structure for a specific table
SQL Validation:
dsql_lint - Validate SQL for DSQL compatibility and optionally auto-fix issues. Use before executing externally-sourced SQL.
Documentation & Knowledge:
dsql_search_documentation - Search Aurora DSQL documentation
dsql_read_documentation - Read specific documentation pages
dsql_recommend - Get DSQL best practice recommendations
Note: There is no list_tables tool. Use readonly_query with information_schema.
See mcp-setup.md for detailed setup instructions. See mcp-tools.md for detailed usage and examples.
AWS Knowledge MCP (awsknowledge)
Consult for verifying DSQL service limits before advising users. The numeric limits below are defaults that may change — when a user's decision depends on an exact limit, verify it first:
| Limit | Default | Verify query |
|---|---|---|
| Max row modifications per transaction | 3,000 | aurora dsql transaction limits |
| Max data size per transaction | 10 MiB | aurora dsql transaction limits |
| Max transaction duration | 5 minutes | aurora dsql transaction limits |
| Max connections per cluster | 10,000 | aurora dsql connection limits |
| Auth token expiry | 15 minutes | aurora dsql authentication token |
| Max connection duration | 60 minutes | aurora dsql connection limits |
| Max indexes per table | 24 incl. PK | aurora dsql index limits |
| Max columns per index | 8 | aurora dsql index limits |
| IDENTITY/SEQUENCE CACHE values | 1 or >= 65536 | aurora dsql sequence cache |
| Supported column data types | See docs | aurora dsql supported data types |
When to verify: Before recommending batch sizes, connection pool settings, or schema designs where hitting a limit would cause failures; any time the exact number can affect user decision. For the full limit set with each SQLSTATE and its exact error message, see limits-and-error-codes.md; where it marks a value observed and the docs differ, use the observed value.
Fallback: If awsknowledge is unavailable, use the defaults above and flag that limits should be verified against DSQL documentation.
CLI Scripts Available
Bash scripts in scripts/ for cluster management (create, delete, list, cluster info), psql connection, and bulk data loading from local/s3 csv/tsv/parquet files. See scripts/README.md for usage and hook configuration.
Quick Start
Pick a connection path: confirm the
aurora-dsqlMCP targets your cluster; if not, use the CLI/psqlpath instead — see Choosing How to Connect. The steps below name MCP tools; the equivalent SQL runs the same way throughpsql-connect.sh --command "...".Explore: Use
readonly_querywithinformation_schemato list tables. Useget_schemafor table structure.Query: Use
readonly_queryfor SELECT queries. MUST includetenant_idin WHERE for multi-tenant apps. MUST build SQL withsafe_query.build().Schema changes: Use
transactwith one DDL per transaction. MUST batch DML under 3,000 rows. MUST useCREATE INDEX ASYNCin a separate call. Usedsql_lintto validate first.Bulk load data: Use
aurora-dsql-loaderfor CSV/TSV/Parquet. Load data-loading.md for details. Use--dry-runfirst.
Performance Routing
When the user reports a performance problem, use this table to select the correct workflow:
| User signal | Route to |
|---|---|
| General performance complaint, "cluster is slow", "something changed", latency regression, no specific query identified | Workflow 12 (System Diagnostics) — observe via CloudWatch first |
| Specific query or query_id to investigate, "explain this plan", "why is this query slow" | Workflow 9 (Query Plan Explainability) — direct EXPLAIN analysis |
| OCC conflicts, commit errors, retry storms | Workflow 12 (System Diagnostics) — confirm via CW metrics before investigating |
| Cost optimization, "where is compute time spent" | Workflow 12 (System Diagnostics) — identify top contributors first |
Rule: When in doubt, start with Workflow 12. It identifies specific queries to investigate and routes to Workflow 9 with context.
Common Workflows
Workflow 1: Create Multi-Tenant Schema
Create tenant-owned tables with
tenant_idusing separatetransactcallsUse composite tenant FKs for tenant-owned parents and ordinary FKs for shared parents
Create tenant and query-pattern indexes with separate
CREATE INDEX ASYNCcallsVerify with
get_schema
MUST issue each DDL in its own transact call:
transact(["CREATE TABLE ..."])MUST serialize arrays into a single-column representation — DSQL has no array column type; PREFER
JSONB(operators work directly); MAY useTEXTwhen the column is opaque to the database; ASK the user. ForJSONBarrays, expand at query time withjsonb_array_elements_text(data)
Workflow 2: Safe Data Migration
MUST validate every DDL with dsql_lint(fix=true) before executing. DML does not require linting.
Validate DDL with
dsql_lint(sql=..., fix=true)— handle diagnostics per dsql-lint.mdExecute the reviewed
fixed_sqlwhen present, otherwise the reviewed source statementAdd column in its own
transactcallPopulate existing rows with UPDATE (batched under 3,000 rows)
Verify with readonly_query COUNT
Create an index if needed: validate then execute the reviewed DDL in its own
transactcall
MUST issue each
ALTER TABLEin its owntransactcall — DSQL rejects multi-DDL transactions withmultiple ddl statements not supported in a transactionMUST add column with only name and type; apply DEFAULT via separate UPDATE
MUST batch updates under 3,000 rows in separate transact calls
Recovery: Resume failed batches by filtering WHERE new_column IS NULL.
Workflow 3: Bulk Data Loading
Use aurora-dsql-loader for CSV, TSV, or Parquet loads. MUST load data-loading.md before advising on throughput or diagnosing slow loads.
Validate with
--dry-runfirstRun with
--manifest-diron persistent storage (not/tmp— tmpfs on AL2023, lost on crash) and--headerif file has a header rowOn failure: resume with
--resume-job-id; for duplicates use--on-conflict do-nothingFor large tables: create secondary indexes after load using
CREATE INDEX ASYNC
Workflow 4: Foreign Key Constraints
MUST load and follow foreign-keys.md before creating, altering, dropping, or migrating foreign keys.
Workflow 5: Query with Tenant Isolation
MUST authorize the caller against the tenant — format validation does not establish authorization
MUST build SQL with
safe_query.build()— useallow()/regex()for values (emits'v'),ident()for table/column names (emits"v"). See input-validation.mdMUST include
tenant_idin the WHERE clause; reject cross-tenant access at the application layer
Workflow 6: Set Up Scoped Database Roles
MUST load access-control.md for role setup, IAM mapping, and schema permissions.
Workflow 7: Table Recreation DDL Migration
For ALTER COLUMN TYPE, SET NOT NULL, or MODIFY PRIMARY KEY, MUST
load ddl-migrations/overview.md. Use a direct ALTER form
when supported; otherwise, present a user-approved table-recreation plan.
Workflow 8: Validate and Migrate to DSQL
MUST load dsql-lint.md before running dsql_lint. Run dsql_lint(sql=source_sql, fix=true) to validate and auto-convert. For MySQL-origin SQL, MUST cross-check against mysql-migrations/type-mapping.md even when lint returns clean. On parse_error, fall back to manual conversion then re-lint.
Workflow 9: Query Plan Explainability
Explains why the DSQL optimizer chose a particular plan. Triggered by slow queries, high DPU, unexpected Full Scans, or plans the user doesn't understand. REQUIRES a structured Markdown diagnostic report as the deliverable.
MUST load query-plan/workflow.md at entry — it defines trigger criteria, context disambiguation, routing, and the full phased workflow (Phase 0–4). Workflow.md specifies which reference files to load at each phase.
Safety. Plan capture uses readonly_query exclusively. Rewrite DML to SELECT for plan capture. MUST NOT use transact --allow-writes for plan capture.
Workflow 10: Full PostgreSQL → DSQL Schema Migration
MUST load pg-migrations/type-mapping.md, pg-migrations/index-conversion.md, pg-migrations/schema-objects.md, and foreign-keys.md. Run dsql_lint(fix=true) first for mechanical fixes, preserve foreign-key relationships, translate unsupported source syntax or options, then apply semantic conversions from the pg-migrations references for unfixable diagnostics and patterns the linter cannot handle. Re-lint the final output before deploying.
Workflow 11: ORM Migration and Locking (Django/EF Core/Hibernate/Rails/SQLAlchemy)
Load orm-guides/overview.md for adapter names, locking guidance, and framework-specific gotchas.
Workflow 12: System Diagnostics (CloudWatch AAS)
Diagnose cluster performance by querying db.active_sessions.avg via PromQL. Detects temporal anomalies in wait event distribution, identifies regressed queries, and routes to Workflow 9 for per-query investigation.
Requires: CloudWatch MCP server (awslabs.cloudwatch-mcp-server) enabled and configured with PromQL access in the same region as the cluster — see mcp/mcp-setup.md for enabling it, region requirements, and the session restart needed for its tools to register.
MUST load system-diagnostics/workflow.md at entry — it defines prerequisites, 5 diagnostic phases, temporal baselines, and the routing to Workflow 9 for identified queries.
Error Scenarios
awsknowledgereturns no results: Use the default limits in the table above and note that limits should be verified against DSQL documentation.dsql_lintunavailable or timing out: See the Error Handling section of dsql-lint.md. Do not silently skip validation — inform the user and require explicit confirmation before proceeding with manual rules from development-guide.md.OCC serialization error (
40001): Retry with backoff, and keep(OC001)in the loop too — see occ-retry-patterns.md.Foreign key violation (
23503): Correct the relationship or referential action; MUST NOT send it through the40001retry loop — see troubleshooting.md.Transaction exceeds limits: Split batches under 3,000 rows and 10 MiB; size is checked at
COMMIT— see limits-and-error-codes.md to size them and batched-migration.md for the pattern.Token expiration mid-operation: Generate a fresh IAM token — see authentication-guide.md. See troubleshooting.md for other issues.
Additional Resources
Files
- ATTRIBUTION.mdAdded by AutoGineer868 B
- LICENSELicence file added by AutoGineer9.9 KiB
- NOTICELicence file added by AutoGineer89 B
- SKILL.md27.9 KiB
- mcp/mcp-setup.md6.7 KiB
- mcp/mcp-tools.md2.0 KiB
- mcp/platforms/claude-code.md3.6 KiB
- mcp/platforms/codex.md1.6 KiB
- mcp/platforms/gemini.md2.1 KiB
- mcp/platforms/kiro.md2.8 KiB
- mcp/tools/database-tools.md4.6 KiB
- mcp/tools/documentation-tools.md1.3 KiB
- mcp/tools/input-validation.md3.3 KiB
- mcp/tools/safe_query.pyScript10.4 KiB
- mcp/tools/workflow-patterns.md3.1 KiB
- references/access-control.md5.0 KiB
- references/auth/authentication-guide.md4.6 KiB
- references/auth/connectivity-tools.md8.7 KiB
- references/auth/scaling-guide.md3.9 KiB
- references/data-loading.md6.0 KiB
- references/ddl-migrations/batched-migration.md2.5 KiB
- references/ddl-migrations/column-operations.md5.2 KiB
- references/ddl-migrations/constraint-operations.md6.7 KiB
- references/ddl-migrations/overview.md4.8 KiB
- references/development-guide.md9.6 KiB
- references/dsql-examples.md1.5 KiB
- references/dsql-lint.md10.8 KiB
- references/examples/connection.md2.5 KiB
- references/examples/data-operations.md2.9 KiB
- references/examples/migrations.md1.6 KiB
- references/examples/patterns.md5.5 KiB
- references/examples/schema.md1.6 KiB
- references/foreign-keys.md6.0 KiB
- references/language.md8.6 KiB
- references/limits-and-error-codes.md19.2 KiB
- references/mysql-migrations/ddl-auto-increment.md4.3 KiB
- references/mysql-migrations/ddl-batching.md1.0 KiB
- references/mysql-migrations/ddl-column-changes.md1.3 KiB
- references/mysql-migrations/ddl-constraints.md782 B
- references/mysql-migrations/ddl-operations.md1.5 KiB
- references/mysql-migrations/ddl-structural.md3.7 KiB
- references/mysql-migrations/ddl-type-alternatives.md4.5 KiB
- references/mysql-migrations/full-example.md7.3 KiB
- references/mysql-migrations/type-mapping.md14.3 KiB
- references/occ-retry-patterns.md9.6 KiB
- references/onboarding.md13.8 KiB
- references/orm-guides/overview.md10.9 KiB
- references/pg-migrations/index-conversion.md15.7 KiB
- references/pg-migrations/multi-region.md2.9 KiB
- references/pg-migrations/schema-objects.md15.0 KiB
- references/pg-migrations/type-mapping.md5.3 KiB
- references/query-plan/catalog-queries.md8.0 KiB
- references/query-plan/guc-experiments.md7.5 KiB
- references/query-plan/plan-interpretation.md15.0 KiB
- references/query-plan/query-rewrites-dsql-specific.md900 B
- references/query-plan/query-rewrites-generic.md1.9 KiB
- references/query-plan/query-rewrites/cte-late-materialization.md1.4 KiB
- references/query-plan/query-rewrites/flatten-union-all.md946 B
- references/query-plan/query-rewrites/left-join-to-inner.md794 B
- references/query-plan/query-rewrites/not-in-to-not-exists.md1.4 KiB
- references/query-plan/query-rewrites/or-to-in.md889 B
- references/query-plan/query-rewrites/propagate-filter.md1.0 KiB
- references/query-plan/query-rewrites/push-computation-to-constant.md1.0 KiB
- references/query-plan/query-rewrites/push-group-by-into-subquery.md1.6 KiB
- references/query-plan/query-rewrites/reltuples-estimate.md1.2 KiB
- references/query-plan/query-rewrites/split-large-joins.md2.2 KiB
- references/query-plan/query-rewrites/subquery-unnesting-correlated.md1.6 KiB
- references/query-plan/query-rewrites/subquery-unnesting-scalar.md1.4 KiB
- references/query-plan/query-rewrites/subquery-unnesting-uncorrelated.md1.4 KiB
- references/query-plan/report-format.md10.9 KiB
- references/query-plan/workflow.md9.4 KiB
- references/system-diagnostics/promql-patterns.md8.6 KiB
- references/system-diagnostics/wait-events.md13.7 KiB
- references/system-diagnostics/workflow.md19.4 KiB
- references/troubleshooting.md11.7 KiB
references/query-plan/plan-interpretation.md
Plan Interpretation Reference
Table of Contents
DSQL Node Types
Layered Plan Structure
Calculating Node Duration
Detecting Estimation Errors
Nested Loop Amplification
Post-Scan Filter Selectivity
Hash Table Resizing
High-Loop Storage Lookups
Anomalous Values
Type Coercion and Index Bypass
Projections and Row Width
Cost Number Interpretation
DPU Interpretation
DSQL Node Types
DSQL stores all table data in B-Tree structures. Secondary indexes are also B-Tree, and contain the primary table keys for the secondary index values that make up the tree. DSQL extends standard PostgreSQL with storage-layer node types:
DSQL-Specific Nodes
| Node Type | Description |
|---|---|
| Full Scan (btree-table) | Full table scan |
| Storage Scan | Physical read of >1 rows of data from storage layer via Pushdown Compute Engine |
| B-Tree Scan | Physical read of rows from storage |
| Storage Lookup | Point lookup of a row by internal row pointer (follows index scan) |
| B-Tree Lookup | Point lookup of a table entry by key |
Standard PostgreSQL Nodes
| Node Type | Description |
|---|---|
| Nested Loop | Iterates inner side once per outer row |
| Hash Join | Builds hash table from one side, probes with the other |
| Merge Join | Merges two pre-sorted inputs |
| Index Scan | Scans an index and fetches matching rows |
| Index Only Scan | Retrieves all data from index access (no table access) |
| Seq Scan | Sequential full table scan |
| Sort | Sorts rows for Merge Join or ORDER BY |
| Aggregate | Computes GROUP BY / aggregate functions |
Layered Plan Structure
A logical scan decomposes into a Storage Scan, which itself has a B-Tree Scan child — not two siblings. Index Scan adds a second, parallel Storage Lookup branch (its own B-Tree Lookup child) for columns the index does not cover.
Full Scan (single branch):
Full Scan (btree-table) on tablename
Filter: col_a = 'v' ← query processor filter (post-transfer)
-> Storage Scan on tablename
Filters: col_b = 'v' ← storage filter (pre-transfer)
-> B-Tree Scan on tablenameIndex Scan (two parallel branches; Storage Lookup is a sibling of Storage Scan, not a child):
Index Scan using idx on tablename
Index Cond: col_a = 'v'
-> Storage Scan on idx
-> B-Tree Scan on tablename
-> Storage Lookup on tablename ← separate branch for non-covered columns
-> B-Tree Lookup on tablenameA child's timing and row counts roll up into its parent's totals — not into a sibling branch.
Three-Layer Filter Model
Every predicate is evaluated at one of three layers. The layer determines how much data crosses the network between storage and compute — the primary lever for DSQL optimization.
| Level | Filter Type | Where it appears in EXPLAIN | Data Movement | How to push predicates here |
|---|---|---|---|---|
| 1 (best) | Index Condition | Index Cond: on scan node | Minimized — only matching index entries read | Equality/range on indexed key columns; most selective column leftmost |
| 2 (moderate) | Storage Filter | Filters: inside Storage Scan or Storage Lookup node | Reduced — applied at storage before transfer | Add filter columns to index INCLUDE clause |
| 3 (worst) | Query Processor Filter | Filter: above Storage Scan (at the scan-type node level) | Maximum — all data transferred before predicate applied | Requires new index, restructured query, or schema change |
Optimization goal: Move predicates from Level 3 → Level 2 → Level 1. Each step reduces network transfer between storage and compute, directly reducing latency and DPU.
Fixing Storage Lookups (INCLUDE columns)
When a Storage Lookup node appears, the index satisfied the filter but not all projected columns. The fix: add missing columns to the index's INCLUDE clause.
-- Before: Storage Lookup fetches created_at from base table
Index Scan using idx1 on account
-> Storage Scan on idx1
-> Storage Lookup on account ← extra round trip
Projections: created_at
-- Fix: CREATE INDEX ASYNC idx2 ON account (customer_id) INCLUDE (balance, status, created_at)
-- After: Index Only Scan, no Storage Lookup
Index Only Scan using idx2 on account
-> Storage Scan on idx2
Projections: customer_id, balance, status, created_atTrade-off: INCLUDE columns are copied into every index entry, increasing index size. Only include columns that your most-queried paths actually need.
Calculating Node Duration
DSQL follows the standard PostgreSQL EXPLAIN convention: actual time is reported per iteration, not cumulative. The node's total wall-clock time is:
Node Duration = actual_time_end × loopsWhere:
actual_time_endis the per-iteration time reported for the node (in ms)loopsis the number of times the node executed (always 1 at the top level; >1 for the inner side of a Nested Loop)
Rank all nodes by total duration descending. Begin analysis from the most expensive node.
Detecting Estimation Errors
An estimation error exists when estimated rows diverge significantly from actual rows:
| Error Magnitude | Classification |
|---|---|
| 2x–5x | Minor — note but low priority |
| 5x–50x | Significant — investigate statistics |
| 50x+ | Severe — likely correlated predicates or stale statistics |
Calculate error ratio: actual_rows / estimated_rows (or inverse if estimate is higher).
For each significant error, record:
The node type and table
The estimated vs actual row count
The index or scan method used
Any filter predicates applied
Nested Loop Amplification
Flag when a Nested Loop's outer input has a significant estimation error:
Pattern:
Nested Loop (est: N rows, actual: M rows)
├── [Outer] Hash Join / Scan (est: X, actual: Y where Y >> X)
└── [Inner] Index Scan (per-loop cost × Y loops)Explanation: The planner chose Nested Loop expecting X iterations on the inner side. With Y actual iterations (where Y >> X), total inner-side cost = per-loop cost × Y. A Hash Join or Merge Join would have been more efficient at this cardinality.
Quantify:
Expected total inner time: per-loop time × estimated outer rows
Actual total inner time: per-loop time × actual outer rows
Amplification factor: actual / estimated
Post-Scan Filter Selectivity
Calculate filter waste when a node applies a post-scan filter:
Filter Selectivity = Rows Removed by Filter / (Rows Removed by Filter + Actual Rows)| Selectivity | Interpretation |
|---|---|
| <10% | Minimal waste — filter removes few rows |
| 10%–50% | Moderate — consider composite index |
| >50% | High waste — strong candidate for index pushdown |
For nodes inside loops, calculate total filter waste:
Total rows scanned = (Actual Rows + Rows Removed) × loops
Total rows filtered = Rows Removed × loopsHash Table Resizing
When a Hash Join reports Buckets: originally N, now M (where M > N):
The planner underestimated the build-side cardinality
The hash table was dynamically resized during execution
This adds memory pressure and execution overhead
Flag the build-side estimation error and trace it to the source scan node.
High-Loop Storage Lookups
When a Storage Lookup has a high loop count:
Total I/O operations = actual_rows × loopsFlag when total I/O operations exceed 10,000. Each Storage Lookup involves a point read from the storage layer — high loop counts with even modest per-loop rows create significant cumulative I/O.
Anomalous Values
Detect physically impossible row counts in DSQL plan nodes:
Detection criteria:
A node reports
actual rowsexceeding the table's known total row count by 10x or moreParticularly common on Storage Lookup nodes under high loop counts
Example: Storage Lookup reporting 7.7 trillion actual rows for a table with 379,484 rows.
Action:
Flag as a potential DSQL reporting bug
Verify query results are correct (they typically are — only EXPLAIN output is affected)
Include in support request template
These anomalous values do not affect query correctness — only diagnostic output accuracy.
Type Coercion and Index Bypass
An index may exist on a column yet not be used when the predicate value's type does not match the column's declared type and no implicit cast exists between the two types.
Detection Pattern
Flag this condition when all of the following are true:
An index exists whose leading column matches a WHERE predicate column
The plan uses a Full Scan or Seq Scan on that table instead of an Index Scan
The predicate literal's type differs from the indexed column's declared type
The
pg_amopquery in catalog-queries.md (B-Tree Cross-Type Operator Support) returns no row for the type pair
Why It Happens
DSQL (like PostgreSQL) can only use a B-Tree index when a cross-type B-Tree operator is registered in pg_amop for the (predicate-type, column-type) pair. When a predicate supplies a value of a different type:
If a cross-type B-Tree operator is registered (verify via the
pg_amopquery in catalog-queries.md), the index can be usedIf no cross-type operator is registered, the planner MUST apply a per-row cast or comparison function that cannot use the index's ordering — resulting in a full scan
This is particularly surprising to users because the query returns correct results (the cast happens at execution time, row by row) but performance degrades dramatically on large tables.
Determining Index-Compatible Type Pairs
Rather than relying on a static matrix, query pg_amop directly on the cluster to determine which cross-type comparisons the DSQL B-Tree index access method supports. See catalog-queries.md for the exact SQL.
The key insight: DSQL's B-Tree access method (amopmethod 10003) only supports index scans when a registered operator exists for the specific (left-type, right-type) pair. If no operator is registered for the pair, the index cannot be used — regardless of whether a general-purpose implicit cast exists in pg_cast.
At time of writing, cross-type index support is limited to the integer family (smallint, integer, bigint — all combinations). All other indexed types (text, numeric, uuid, timestamp, date, boolean, etc.) require an exact type match. MUST verify via the pg_amop query in catalog-queries.md before asserting this to a user, as DSQL MAY add cross-type operator families in future releases.
Quantifying Impact
When this pattern is detected:
Full Scan rows processed = actual_rows from Full Scan node
Index Scan rows (expected) = estimated rows matching the predicate (from pg_stats selectivity)
Scan amplification = Full Scan rows / Index Scan rows (expected)Recommendation Template
When a type coercion bypass is confirmed:
Explicit cast in the predicate: Rewrite
WHERE col = '42'asWHERE col = 42::integer(cast the literal to the column's declared type)Application-layer fix: Ensure the application passes parameters with the correct type rather than relying on implicit conversion
MUST keep the column type unchanged — changing it to accommodate mismatched predicates masks the real issue and MAY break other queries
Evidence Gathering
To confirm this pattern, cross-reference:
The column type from
pg_attributeorinformation_schema.columns(see catalog-queries.md)The index definition from
pg_indexesThe predicate literal in the EXPLAIN output (visible in
Filter:orIndex Cond:lines)The
pg_amopquery in catalog-queries.md (B-Tree Cross-Type Operator Support)
Projections and Row Width
Capture Projections lists from Storage Scan and Storage Lookup nodes:
Projections: [col1, col2, col3, ...]Assess row width overhead:
Count projected columns per node
Note when
SELECT *pulls all columns from wide tablesFlag tables with 50+ columns or estimated row width >5,000 bytes
Wide projections increase I/O on Storage Lookups and memory usage in Hash Joins. Impact scales with result set size.
Cost Number Interpretation
DSQL cost numbers appear much higher than equivalent PostgreSQL plans. This is expected — the cost model accounts for distributed round-trips.
Format: startup_cost..total_cost (e.g., 100.28..208.29)
Startup cost ~100 is normal — reflects fixed overhead of initiating a storage round-trip
Total cost includes incremental per-row processing, network transfer, and page access
MUST NOT compare cost numbers across queries to determine which is "better." Cost units are internal to the optimizer and non-comparable. Use DPU estimates instead.
DPU Interpretation
EXPLAIN ANALYZE VERBOSE appends a Statement DPU Estimate block:
Statement DPU Estimate:
Compute: 0.01724 DPU
Read: 0.01202 DPU
Write: 0.00000 DPU
Total: 0.02926 DPURead DPU is the primary optimization signal for read-heavy queries. High Read DPU with selective filters means those filters aren't pushed down far enough (Level 3 or 2 when they could be Level 1).
Optimization loop:
Run
EXPLAIN ANALYZE VERBOSEon the unoptimized query — note Total DPUApply fix (add index, add INCLUDE columns, restructure query)
Re-run — compare DPU delta
MUST use DPU as the before/after comparison metric, not cost numbers or execution time (which varies with load).
Requirements and compatibility
Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta
“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.
Needs: Needs an MCP server, Needs Python, Needs Node.js
Security
What our scanner noticed· Low risk
- Instructions that use the network
- Scripts that your agent may run
This skill includes scripts that your agent may run. Read them before you use it.
A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.
Attribution and licence
- Published by
- AWS Labs
- Publisher
- AWS Labs (Official)
- Source
- awslabs/agent-plugins · plugins/databases-on-aws/skills/dsql · commit fbefb37
- Licence
- Apache License 2.0 (redistribution allowed)
- LICENSE (added by AutoGineer from LICENSE)
- NOTICE (added by AutoGineer from NOTICE)
- ATTRIBUTION.md (written by AutoGineer)
- Synced
AutoGineer is not affiliated with or endorsed by AWS Labs or AWS Labs.
Download ZIP
The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download dsql.zip