Cloud Design Patterns

Reliability & Resilience Patterns

Ambassador Pattern

Problem: Services need proxy functionality for network requests (logging, monitoring, routing, security).

Solution: Create helper services that send network requests on behalf of a consumer service or application.

When to Use:

• Offloading common client connectivity tasks (monitoring, logging, routing)

• Supporting legacy applications that can't be easily modified

• Implementing retry logic, circuit breakers, or timeout handling for remote services

Implementation Considerations:

• Deploy ambassador as a sidecar process or container with the application

• Consider network latency introduced by the proxy layer

• Ensure ambassador doesn't become a single point of failure

Bulkhead Pattern

Problem: A failure in one component can cascade and affect the entire system.

Solution: Isolate elements of an application into pools so that if one fails, the others continue to function.

When to Use:

• Isolating critical resources from less critical ones

• Preventing resource exhaustion in one area from affecting others

• Partitioning consumers and resources to improve availability

Implementation Considerations:

• Separate connection pools for different backends

• Partition service instances across different groups

• Use resource limits (CPU, memory, threads) per partition

• Monitor bulkhead health and capacity

Circuit Breaker Pattern

Problem: Applications can waste resources attempting operations that are likely to fail.

Solution: Prevent an application from repeatedly trying to execute an operation that's likely to fail, allowing it to continue without waiting for the fault to be fixed.

When to Use:

• Protecting against cascading failures

• Failing fast when a remote service is unavailable

• Providing fallback behavior when services are down

Implementation Considerations:

• Define threshold for triggering circuit breaker (failures/time window)

• Implement three states: Closed, Open, Half-Open

• Set appropriate timeout values for operations

• Log state transitions and failures for diagnostics

• Provide meaningful error messages to clients

Compensating Transaction Pattern

Problem: Distributed transactions are difficult to implement and may not be supported.

Solution: Undo the work performed by a sequence of steps that collectively form an eventually consistent operation.

When to Use:

• Implementing eventual consistency in distributed systems

• Rolling back multi-step business processes that fail partway through

• Handling long-running transactions that can't use 2PC

Implementation Considerations:

• Define compensating logic for each step in transaction

• Store enough state to undo operations

• Handle idempotency for compensation operations

• Consider ordering dependencies between compensating actions

Retry Pattern

Problem: Transient failures are common in distributed systems.

Solution: Enable applications to handle anticipated temporary failures by retrying failed operations.

When to Use:

• Handling transient faults (network glitches, temporary unavailability)

• Operations expected to succeed after a brief delay

• Non-idempotent operations with careful consideration

Implementation Considerations:

• Implement exponential backoff between retries

• Set maximum retry count to avoid infinite loops

• Distinguish between transient and permanent failures

• Ensure operations are idempotent or track retry attempts

• Consider jitter to avoid thundering herd problem

Health Endpoint Monitoring Pattern

Problem: External tools need to verify system health and availability.

Solution: Implement functional checks in an application that external tools can access through exposed endpoints at regular intervals.

When to Use:

• Monitoring web applications and back-end services

• Implementing readiness and liveness probes

• Providing detailed health information to orchestrators

Implementation Considerations:

• Expose health endpoints (e.g., /health, /ready, /live)

• Check critical dependencies (databases, queues, external services)

• Return appropriate HTTP status codes (200, 503)

• Implement authentication/authorization for sensitive health data

• Provide different levels of detail based on security context

Leader Election Pattern

Problem: Distributed tasks need coordination through a single instance.

Solution: Coordinate actions in a distributed application by electing one instance as the leader that manages collaborating task instances.

When to Use:

• Coordinating distributed tasks

• Managing shared resources in a cluster

• Ensuring single-instance execution of critical tasks

Implementation Considerations:

• Use distributed locking mechanisms (Redis, etcd, ZooKeeper)

• Handle leader failures with automatic re-election

• Implement heartbeats to detect leader health

• Ensure followers can become leaders quickly

Saga Pattern

Problem: Maintaining data consistency across microservices without distributed transactions.

Solution: Manage data consistency across microservices in distributed transaction scenarios using a sequence of local transactions.

When to Use:

• Long-running business processes spanning multiple services

• Distributed transactions without 2PC support

• Eventual consistency requirements across microservices

Implementation Considerations:

• Choose between orchestration (centralized) or choreography (event-based)

• Define compensating transactions for rollback scenarios

• Handle partial failures and rollback logic

• Implement idempotency for all saga steps

• Provide clear audit trails and monitoring

Sequential Convoy Pattern

Problem: Process related messages in order without blocking independent message groups.

Solution: Process a set of related messages in a defined order without blocking other message groups.

When to Use:

• Message processing requires strict ordering within groups

• Independent message groups can be processed in parallel

• Implementing session-based message processing

Implementation Considerations:

• Use session IDs or partition keys to group related messages

• Process each group sequentially but process groups in parallel

• Handle message failures within a session appropriately