Managing Amazon Msk

Configure MSK Brokers and Clusters

Broker-level and cluster-level configuration for Amazon MSK Provisioned clusters: creating and applying MSK configurations (server.properties), and setting custom domain names on the brokers with custom.advertised.listeners.

For the client-side layer that fronts a custom domain (NLB, ACM certificate, Route 53, TLS handshake, mTLS), see configure-clients.md. For per-instance-size thread tuning and durability defaults, see size-and-choose-cluster.md.

Note: The AWS MCP server is the recommended way to run the AWS API interactions in this reference (kafka describe/update calls, kafka-configs.sh, etc.) — it provides sandboxed execution, audit logging, and observability. It is not a hard requirement; when the MCP server is unavailable, fall back to the AWS CLI or shell.

Custom Domain Names (custom.advertised.listeners)

Use a custom domain name (e.g., b-1.example.com) when clients need a static, customer-controlled endpoint that survives cluster recreation, migration, or DR failover, or must align with organizational DNS/naming. MSK brokers otherwise advertise AWS-generated addresses (b-1.<cluster>.<id>.kafka.<region>.amazonaws.com) that change when a cluster is recreated. This section covers the cluster configuration that sets the advertised address; the NLB / certificate / DNS layer that must exist first is in configure-clients.md.

Set it with the custom.advertised.listeners configuration property

Recommended approach: a single, validated, cluster-wide MSK configuration property. Set it once and MSK resolves it per broker and reapplies it automatically as the cluster scales. This replaces the per-broker kafka-configs.sh --alter --add-config advertised.listeners=... override.

• Works in both ZooKeeper and KRaft mode. The per-broker advertised.listeners override could not be applied on KRaft clusters; custom.advertised.listeners works on both. If asked whether custom domains are possible on a KRaft cluster, the answer is yes — via this property. Do NOT recommend the per-broker kafka-configs.sh override or self-managed Kafka as the current solution, and do NOT claim custom domains are impossible on KRaft.

• Provisioned only (Standard and Express brokers). NOT supported on MSK Serverless (Serverless does not support custom broker configurations). If a custom domain is required on Serverless, use a Provisioned cluster.

• Reversible and IaC-friendly. Flows through CloudFormation, CDK, Terraform, and the CLI.

Value format:

custom.advertised.listeners=<LISTENER>://<hostname>:<port>

Example (three-broker IAM cluster fronted by an NLB on ports 9001-9003):

custom.advertised.listeners=CLIENT_IAM://b-{broker_id}.example.com:9000+{broker_id}

Rules that MSK validates (synchronously, before any change)

1. Client listeners only. Valid values are the client-facing listeners: CLIENT, CLIENT_SECURE, CLIENT_SECURE_PUBLIC, CLIENT_SASL_SCRAM, CLIENT_SASL_SCRAM_PUBLIC, CLIENT_IAM, CLIENT_IAM_PUBLIC. Internal listeners REPLICATION and CONTROLLER are rejected — this protects the listeners MSK manages. A rejected REPLICATION://... config is not a syntax problem with the + separator or offset; it is rejected because internal listeners are not allowed.

2. Listener must be bound (active) on the cluster. Name the client listener that matches the cluster's authentication type (e.g., CLIENT_IAM on an IAM cluster). Specifying an unbound listener (e.g., CLIENT_SECURE on an IAM-only cluster) is rejected, and the error lists the valid client listeners for your cluster.

3. {broker_id} template + uniqueness. {broker_id} is replaced with each broker's numeric ID at apply time. In a port expression like 9000+{broker_id}, the broker ID is added to the base port, so broker 1 -> 9001, broker 2 -> 9002, broker 10 -> 9010 (9000 is just an example base). Each broker's resolved host:port must be unique. {broker_id} may appear in the hostname, the port, or both — so a shared hostname with a per-broker port (CLIENT_IAM://example.com:9000+{broker_id}) is valid, but a shared hostname AND shared port for every broker fails validation. The resulting ports must match the TLS listeners provisioned on your NLB.

Apply workflow

1. Put the property in a file (leave {broker_id} literal — MSK resolves it per broker; do not substitute IDs yourself). It can live alongside other broker properties in a single configuration revision.

2. Create/update the configuration, passing the file with fileb:// (not file://) so the CLI reads it as bytes and base64-encodes it — passing inline or with file:// is fragile because of the {broker_id} braces:

   Copy

   aws kafka create-configuration --name custom-domain-iam \
     --server-properties fileb://custom-domain-config.txt

   (For an existing configuration, add the property and create a new revision with update-configuration.)

3. Apply the returned configuration ARN and revision to the cluster:

   Copy

   aws kafka update-cluster-configuration \
     --cluster-arn <arn> \
     --configuration-info arn=<config-arn>,revision=<revision> \
     --current-version <current-cluster-version>

   MSK validates, resolves the pattern per broker, and applies it via a rolling restart.

4. Track the rollout with describe-cluster-operation-v2 (or the DescribeOperation API): states go UPDATE_IN_PROGRESS -> UPDATE_COMPLETE/SUCCESS or UPDATE_FAILED/FAILED.

Auditability: Ensure CloudTrail is logging MSK API calls (kafka:UpdateClusterConfiguration, kafka:UpdateConfiguration) so configuration changes are auditable.

Rollout failure recovery. If a broker fails to start, the rollout halts at that broker and the remaining brokers keep their previous configuration — the cluster is not left half-configured, and no recreation is needed. Fix the property (common cause: the resolved host:port isn't resolvable/reachable/trusted via your NLB+DNS+cert) and re-apply.

Reverting. Fully reversible: remove custom.advertised.listeners from the configuration (new revision without it) and re-apply. MSK reverts the listener to its original AWS-generated advertised address via a rolling restart. Only the advertised address of the named listener is affected — REPLICATION, authentication, multi-VPC (CLIENT_IAM_VPCE), and PrivateLink connectivity are unaffected.

Migrate from the dynamic per-broker override to the static property

Existing custom-domain setups configured the advertised address dynamically, per broker, with kafka-configs.sh --alter --add-config advertised.listeners=[...] — repeated on every broker and re-run whenever a broker was added, hand-preserving the REPLICATION/REPLICATION_SECURE (and any multi-VPC) entries each time. Migrating to the static custom.advertised.listeners property removes that per-broker toil, adds up-front validation, works on KRaft, and flows through IaC. The networking layer you already built (NLB, DNS, certificate) stays exactly as is — only how the advertised address is configured changes.

1. Capture the current pattern. On each broker, read the existing dynamic value so you can reproduce it exactly:

   Copy

   kafka-configs.sh --bootstrap-server $BS --entity-type brokers --entity-name <broker-id> \
     --command-config client.properties --all --describe | grep advertised.listeners

   Note the client listener name and the host:port each broker advertises (e.g., CLIENT_SASL_SCRAM://b-1.example.com:9001) and confirm it fits a {broker_id} template such as b-{broker_id}.example.com:9000+{broker_id}.

2. Include only the client listener in the property. Unlike the dynamic override — where you had to include and preserve REPLICATION/REPLICATION_SECURE (and multi-VPC) entries yourself — custom.advertised.listeners manages only the named client listener, and MSK preserves the internal, multi-VPC (CLIENT_IAM_VPCE), and PrivateLink listeners for you. Do NOT put REPLICATION/CONTROLLER in the property (they are rejected).

3. Match the existing address for a zero-cutover migration. Choose the template so each broker's resolved host:port equals what it already advertises. Then the advertised address does not change and clients keep connecting with no cutover. If you deliberately change the pattern, clients cut over on their next metadata refresh — build and verify the new NLB listeners, target groups, and DNS records first (see configure-clients.md).

4. Apply via the configuration, not kafka-configs.sh. Add custom.advertised.listeners to a new MSK configuration or, if you already manage one for the cluster, add it and create a new revision with update-configuration; then apply with update-cluster-configuration (see Apply workflow above). From now on, manage the advertised address only through the MSK configuration, not with per-broker dynamic overrides.

5. Clear any stale dynamic override and verify the effective value. A per-broker dynamic config can take precedence over broker configuration in Kafka, so after applying, confirm the effective advertised.listeners with the --describe command from step 1. If a leftover dynamic override remains, delete it with kafka-configs.sh --alter --entity-type brokers --entity-name <broker-id> --delete-config advertised.listeners — do this only after step 3 guarantees the resolved address is identical, so removal changes nothing for clients.

6. Confirm and finish. Ensure the operation reaches SUCCESS (describe-cluster-operation-v2) and clients still connect (kafka-topics.sh --list --bootstrap-server <custom-bootstrap>). Going forward MSK reapplies the property to new/replaced brokers automatically, so you no longer re-run the per-broker override (you still add NLB/DNS entries for new brokers).

KRaft note: if the dynamic override was never an option because the cluster is in KRaft mode, the static property is the supported path and works on KRaft. The migration is reversible — remove the property and re-apply to return to the AWS-generated address.

Scaling and broker replacement (cluster side)

When you add brokers (or a broker is replaced during automated healing), MSK automatically applies the configuration to the new broker, resolving {broker_id} for its ID — nothing to do on the cluster side. The networking layer does NOT auto-scale, however: you must add the corresponding NLB listener, target group, and DNS record for each new broker. See configure-clients.md.

Prerequisite: the networking/trust layer must exist first

Applying this property changes what the brokers advertise, so clients cut over to the custom domain on their next metadata refresh. If the NLB, DNS, and trusted certificate are not already in place, connected clients cannot reconnect. Build that layer first and follow the safe two-phase cutover documented in configure-clients.md.

References

• Amazon MSK simplifies configuring custom domain names — the custom.advertised.listeners property

• MSK Configuration