Asynchronous messaging is the backbone of scalable, decoupled distributed architectures. When services communicate via events rather than synchronous HTTP/RPC, downstream services can fail, scale, or deploy without halting upstream producers.
However, event-driven architectures introduce subtle distributed data consistency hazards. The most infamous is the Dual-Write Problem: if a service writes to an ACID database and attempts to publish an event to a message queue (Kafka, RabbitMQ, SQS), a failure between the two operations leaves the database and the event broker in an inconsistent state.
This module deconstructs event-driven architecture and guaranteed delivery:
- The Dual-Write Dilemma: Why distributed transactions (XA/2PC) fail across disparate databases and brokers, and how the Transactional Outbox pattern guarantees atomicity.
- Log-Based Change Data Capture (CDC): Extracting events directly from database write-ahead logs (PostgreSQL WAL, MySQL binlog) via Debezium without polling tables.
- Delivery Guarantees & Idempotency: Why exactly-once delivery over an unreliable network is an illusion, and how at-least-once delivery combined with idempotent consumer deduplication achieves effective exactly-once processing.
- Poison Pills & Error Recovery: Dead-letter queues, exponential backoff with full jitter, and automated replay workflows.