Read-Heavy Systems
Workloads where reads vastly outnumber writes. Strict tail-latency requirements, high cacheability of popular items, and susceptibility to cache stampedes.
Never select technologies or draw architectural boxes before characterizing the workload. Deconstruct complex systems into their constituent workload families, evaluate access shapes, and map profiles directly to optimal data structures and storage engines.
Configure your target system's parameters to calculate its primary workload archetype, required storage engines, and architectural traps.
Requires absolute financial-grade invariants, zero balance drift, and strict serializability or snapshot isolation.
Every production workload family forces a distinct set of non-negotiable engineering questions before choosing storage engines, network protocols, or consistency boundaries.
| Workload Family | Required Design Questions | Canonical Reference Lessons |
|---|---|---|
| Read-Heavy | Cache hierarchy, read replicas, cache invalidation races, hot-key stampedes, and freshness bounds | Distributed Caching · URL Shortener |
| Write-Heavy | Partitioning keys, client/edge batching, write amplification, LSM compaction, and backpressure | LSM-Trees · Kafka Commit Log |
| Low-Latency | Tail-latency amplification, queueing delay, CPU/memory locality, network hops, and admission control | io_uring & XDP · Real-Time Chat |
| Event-Driven | Delivery semantics, per-key ordering, log replay, idempotent deduplication, and schema evolution | Outbox & CDC · Stream Watermarks |
| Analytical (OLAP) | Columnar layouts, vectorized scans, distributed aggregations, distributed joins, and resource isolation | ClickHouse OLAP · Distributed Analytics |
| Geospatial | Spatial indexes, Geohash vs Uber H3 vs Google S2 trade-offs, R-trees, proximity queries, and live location updates | Disk & Spatial Indexing · Geo-Routing |
| Logging & Observability | Ingestion velocity, label cardinality explosions, tiered retention, inverted vs label indexing, sampling, and query cost | Distributed Observability · eBPF Profiling |
| Large-Object & Media | Multipart chunked upload, Reed-Solomon erasure coding, CDN byte-range delivery, transcoding DAGs, and lifecycle policies | Object Storage & CDN · Video Transcoding |
| Stateful Coordination | Time-bounded leases, monotonic fencing tokens, quorum consensus, leader election failover, and state recovery | etcd Leases & Fencing · Raft Consensus |
| AI & ML Infrastructure | Training collective communication, GPU gang scheduling, continuous inference batching, KV-cache capacity, retrieval, and evaluation | 3D Parallelism · PagedAttention · Vector HNSW |
Each workload archetype defines unique constraints on concurrency, access patterns, storage layouts, and failure modes.
Workloads where reads vastly outnumber writes. Strict tail-latency requirements, high cacheability of popular items, and susceptibility to cache stampedes.
Massive continuous write streams dominated by append-only operations. Immediate read consistency is rarely required, but sustained write throughput and low write amplification are paramount.
Workloads requiring absolute data integrity, zero drift, and strict serializability or snapshot isolation. Double-entry ledgers, payments, and inventory allocation.
Normal baseline operation interrupted by unpredictable, extreme demand surges that threaten to saturate CPU, memory, and database connection pools simultaneously.
Systems where every microsecond, CPU context switch, and network hop directly impacts business outcomes. In-memory execution, non-blocking I/O, and lock-free concurrency.
Unbounded event streams processed incrementally with stateful aggregations, sliding time windows, watermarks, and exactly-once semantics.
High-volume unstructured and semi-structured telemetry combining immense ingestion throughput, label-based filtering, and multi-tier retention economics.
High-frequency timestamped numeric metrics arriving at regular intervals. Requires extreme compression and continuous automated rollups.
Two-dimensional or three-dimensional coordinates subject to constant mobile updates and low-latency radius or bounding-box proximity queries.
Natural language keyword and prefix queries over massive document corpora. Requires linguistic tokenization, BM25 scoring, and distributed score merging.
Interconnected entity networks where relationships (edges) are first-class citizens and queries traverse multiple hops rather than performing tabular joins.
Analytical aggregations (SUM, AVG, GROUP BY) across billions of historical rows. Columnar data layout, compression, and vectorized execution.
Large payload sizes (10 MB to 50 GB) requiring petabyte-scale storage durability, Reed-Solomon erasure coding, and byte-range streaming.
Multi-master concurrent edits on shared document state without central locking. Requires mathematical convergence guarantees.
Shared cloud infrastructure serving thousands of independent enterprise tenants with strict isolation, fair-share scheduling, and encryption boundaries.
Large-scale offline computations scheduled during off-peak windows or on recurring intervals. Fault-tolerant task checkpointing and speculative execution.
Real production systems are composites of multiple distinct workloads. Never force one database or pattern across the entire system.
| Subsystem | Primary Function | Workload Family | Optimal Architectural Pattern | Storage Engine |
|---|---|---|---|---|
| 1. Driver GPS Stream | Ingest GPS coordinates from 2M active drivers every 3s | Family B (Write-Heavy) | Partitioned log buffer, zero-copy socket transfers | Apache Kafka + Mem-ring |
| 2. Driver Discovery | Find 10 nearest available drivers within 3 km | Family I (Geospatial) | H3 hexagonal spatial tessellation, cell replication | Redis GEO / Uber H3 |
| 3. Dispatch Matcher | Pair driver and rider in < 500 ms with surge pricing | Family E (Low-Latency) | In-memory state machine, strict deadline hedging | Go/Rust In-Memory Engine |
| 4. Ride Booking Ledger | Debit customer, credit driver, and create immutable ledger | Family C (Transactional) | ACID serializable transaction, Outbox CDC, Sagas | PostgreSQL / CockroachDB |
| 5. Location Push | Stream live vehicle position to rider phone at 1 Hz | Family F (Streaming Pub/Sub) | Stateful WebSocket connection pinning, Redis pub/sub | Distributed Gateway Mesh |
| 6. Driver Settlement | Calculate weekly payouts, tax withholdings, and bank transfers | Family P (Batch Offline) | Idempotent MapReduce, checkpointed distributed shuffle | Apache Spark Batch |
| 7. Surge Pricing Analytics | Compute dynamic demand heatmaps across metropolitan zones | Family L (Analytical OLAP) | Columnar vectorized scan, streaming materialized views | ClickHouse OLAP |