INTERACTIVE DECISION ENGINE

System Workload Characterizer

Configure your target system's parameters to calculate its primary workload archetype, required storage engines, and architectural traps.

FAMILY C · TRANSACTIONAL

Mixed Transactional Systems (ACID)

Requires absolute financial-grade invariants, zero balance drift, and strict serializability or snapshot isolation.

Primary PatternMVCC + Sagas + Outbox + Idempotency
Recommended EnginesPostgreSQL, CockroachDB, TiDB / TiKV
Top Architectural TrapDistributed 2PC across services holding long row locks.
ESSENTIAL DESIGN QUESTIONS

Core Workload Families & Architectural Checklists

Every production workload family forces a distinct set of non-negotiable engineering questions before choosing storage engines, network protocols, or consistency boundaries.

Workload FamilyRequired Design QuestionsCanonical Reference Lessons
Read-HeavyCache hierarchy, read replicas, cache invalidation races, hot-key stampedes, and freshness boundsDistributed Caching · URL Shortener
Write-HeavyPartitioning keys, client/edge batching, write amplification, LSM compaction, and backpressureLSM-Trees · Kafka Commit Log
Low-LatencyTail-latency amplification, queueing delay, CPU/memory locality, network hops, and admission controlio_uring & XDP · Real-Time Chat
Event-DrivenDelivery semantics, per-key ordering, log replay, idempotent deduplication, and schema evolutionOutbox & CDC · Stream Watermarks
Analytical (OLAP)Columnar layouts, vectorized scans, distributed aggregations, distributed joins, and resource isolationClickHouse OLAP · Distributed Analytics
GeospatialSpatial indexes, Geohash vs Uber H3 vs Google S2 trade-offs, R-trees, proximity queries, and live location updatesDisk & Spatial Indexing · Geo-Routing
Logging & ObservabilityIngestion velocity, label cardinality explosions, tiered retention, inverted vs label indexing, sampling, and query costDistributed Observability · eBPF Profiling
Large-Object & MediaMultipart chunked upload, Reed-Solomon erasure coding, CDN byte-range delivery, transcoding DAGs, and lifecycle policiesObject Storage & CDN · Video Transcoding
Stateful CoordinationTime-bounded leases, monotonic fencing tokens, quorum consensus, leader election failover, and state recoveryetcd Leases & Fencing · Raft Consensus
AI & ML InfrastructureTraining collective communication, GPU gang scheduling, continuous inference batching, KV-cache capacity, retrieval, and evaluation3D Parallelism · PagedAttention · Vector HNSW
THE 16 CANONICAL ARCHETYPES

Workload Taxonomy Catalogue

Each workload archetype defines unique constraints on concurrency, access patterns, storage layouts, and failure modes.

FAMILY A · DATA ACCESSp99 < 15 ms

Read-Heavy Systems

Workloads where reads vastly outnumber writes. Strict tail-latency requirements, high cacheability of popular items, and susceptibility to cache stampedes.

Key PatternsMulti-tier caching, Read replicas, CDN edge distribution, Materialized views, Single-flight request coalescing.
Core EnginesRedis, Memcached, Varnish, CDN edge caches, B+ Tree primary indexes.
Critical TrapQuerying database directly for hot keys without stampede protection; cache invalidation race conditions.
Product CatalogueUser Profile ServiceContent PublishingDNS Resolution
FAMILY B · INGESTIONHigh Throughput (≥ 100k ops/s)

Write-Heavy & Ingestion Systems

Massive continuous write streams dominated by append-only operations. Immediate read consistency is rarely required, but sustained write throughput and low write amplification are paramount.

Key PatternsAppend-only logs, Write-Ahead Logging (WAL), Log-Structured Merge (LSM) Trees, Memory-mapped ring buffers, Backpressure propagation.
Core EnginesRocksDB, Apache Kafka, Apache Cassandra, ClickHouse, Apache Pulsar.
Critical TrapIn-place B+ Tree updates causing random disk I/O; synchronous unbuffered write transactions.
IoT Telemetry CollectorClickstream Analytics LakeApplication Event PipelineAudit Trails
FAMILY C · TRANSACTIONALp99 < 50 ms (100% ACID)

Mixed Transactional Systems (ACID)

Workloads requiring absolute data integrity, zero drift, and strict serializability or snapshot isolation. Double-entry ledgers, payments, and inventory allocation.

Key PatternsMulti-Version Concurrency Control (MVCC), Two-Phase Locking (2PL), Distributed Sagas, Transactional Outbox, Idempotency keys.
Core EnginesPostgreSQL, CockroachDB, TiDB / TiKV, FoundationDB, MySQL with InnoDB.
Critical TrapDistributed 2PC across microservices with long holding locks; missing idempotency tokens on retry paths.
Core Banking LedgerPayment GatewayInventory ReservationAirline Booking Engine
FAMILY D · TRAFFIC SHOCKS10x - 100x Spikes in seconds

Burst-Heavy & Spike Workloads

Normal baseline operation interrupted by unpredictable, extreme demand surges that threaten to saturate CPU, memory, and database connection pools simultaneously.

Key PatternsVirtual waiting rooms, Token Bucket / Leaky Bucket admission control, Adaptive load shedding, Priority queues, Feature degradation.
Core EnginesRedis Lua token buckets, Cloudflare Waiting Room, AWS SQS FIFO with backpressure.
Critical TrapAllowing unbounded requests directly into connection pool; uncoordinated client retry storms (dogpiling).
Flash-Sale Ticket DropsViral DropsTax Filing DeadlinesEmergency Alert Broadcasts
FAMILY E · SUB-MILLISECONDp99 < 1 - 2 ms

Low-Latency & Real-Time Systems

Systems where every microsecond, CPU context switch, and network hop directly impacts business outcomes. In-memory execution, non-blocking I/O, and lock-free concurrency.

Key PatternsCPU core pinning, In-memory state, Kernel bypass (io_uring, XDP, eBPF), Lock-free queues, Request hedging, Zero-copy serialization.
Core EnginesAerospike, Redis, In-memory C++/Rust state machines, Aeron IPC.
Critical TrapUnbounded Garbage Collection pauses; blocking I/O calls inside hot loop; inter-thread mutex lock contention.
Real-Time Bidding (RTB)High-Frequency TradingFraud Scoring EngineMultiplayer Game Loops
FAMILY F · STREAMINGSub-second Event Time

Streaming & Continuous Event Processing

Unbounded event streams processed incrementally with stateful aggregations, sliding time windows, watermarks, and exactly-once semantics.

Key PatternsEvent time vs processing time, Watermark alignment, RocksDB state backends, Asynchronous Chandy-Lamport barrier checkpointing.
Core EnginesApache Flink, Kafka Streams, Apache Spark Streaming, Apache Pulsar.
Critical TrapRelying on processing time for out-of-order events; synchronous external API lookups inside stream operators.
Live Fraud DetectionContinuous Billing AggregatorDynamic Surge PricingIoT Anomaly Detector
FAMILY G · OBSERVABILITYSeconds to Ingest, Sub-second Query

Logging & Observability Platforms

High-volume unstructured and semi-structured telemetry combining immense ingestion throughput, label-based filtering, and multi-tier retention economics.

Key PatternsChunked append-only compression, Inverted posting indexes vs label-only indexes (Loki model), Tiered storage (NVMe -> S3 -> Glacier).
Core EnginesGrafana Loki, OpenSearch, ClickHouse, Apache Kafka, Jaeger, VictoriaLogs.
Critical TrapFull-text indexing of high-cardinality payloads (UUIDs, timestamps) causing index explosion.
Enterprise Log LakeDistributed Tracing CollectorSIEM Security Event Store
FAMILY H · TELEMETRYSub-second Window Rollups

Time-Series & Metrics Systems

High-frequency timestamped numeric metrics arriving at regular intervals. Requires extreme compression and continuous automated rollups.

Key PatternsDelta-of-deltas timestamp compression, Gorilla XOR float compression, Continuous downsampling (raw -> 5m -> 1h), Time-based partitioning.
Core EnginesPrometheus TSDB, VictoriaMetrics, TimescaleDB, InfluxDB, ClickHouse.
Critical TrapHigh-cardinality label explosions creating millions of concurrent open metric series.
Datacenter TelemetryIndustrial IoT Sensor FleetApplication Performance Monitoring (APM)
FAMILY I · SPATIALp99 < 20 ms Radius Searches

Geospatial & Location-Based Systems

Two-dimensional or three-dimensional coordinates subject to constant mobile updates and low-latency radius or bounding-box proximity queries.

Key PatternsHierarchical spatial tessellation (Uber H3 hexagons, Google S2 curves, Geohash), R-Trees, QuadTrees, Cell-level sharding, GPS jitter filtering.
Core EnginesRedis GEO, PostGIS, Uber H3, Elasticsearch geospatial.
Critical TrapNaive Euclidean distance scanning over raw latitude/longitude columns without spatial index.
Ride-Sharing Driver DispatchFood Delivery TrackingGeofence Breach AlertsMaritime Fleet Tracking
FAMILY K · GRAPHSub-100 ms Multi-Hop Queries

Graph & Relationship-Heavy Systems

Interconnected entity networks where relationships (edges) are first-class citizens and queries traverse multiple hops rather than performing tabular joins.

Key PatternsIndex-free adjacency, Bidirectional breadth-first / depth-first search, Graph partitioning (vertex cut vs edge cut), Supernode mitigations.
Core EnginesNeo4j, AWS Neptune, NebulaGraph, JanusGraph.
Critical TrapExecuting 5-way recursive SQL JOINs in relational tables; ignoring supernodes (celebrities with 10M edges).
Social Follower NetworksFinancial Fraud Ring AnalysisKnowledge GraphsDependency Trees
FAMILY L · COLUMNARSeconds to Minutes (Multi-GB/s)

Analytical Data Warehousing (OLAP)

Analytical aggregations (SUM, AVG, GROUP BY) across billions of historical rows. Columnar data layout, compression, and vectorized execution.

Key PatternsColumnar storage (Parquet, ORC), SIMD vectorized execution, Column pruning, Predicate pushdown, Run-length encoding (RLE).
Core EnginesClickHouse, Apache Pinot, Apache Druid, Snowflake, Trino, Apache Iceberg.
Critical TrapAttempting OLAP aggregate queries against an OLTP transactional database during business hours.
Business Intelligence Data WarehouseProduct Usage Analytics LakeFinancial Reconciliation
FAMILY M · BLOB STORAGEHigh Throughput / Wire Speed

Large-Object, File & Media Systems

Large payload sizes (10 MB to 50 GB) requiring petabyte-scale storage durability, Reed-Solomon erasure coding, and byte-range streaming.

Key PatternsContent-Addressable Storage (CAS), Multipart chunked uploads, Reed-Solomon Erasure Coding (M+N), Metadata / Data separation, CDN edge caching.
Core EnginesCeph (RADOS), MinIO, Apache Ozone, HDFS, SeaweedFS.
Critical TrapStoring large binary blobs directly inside database row columns; single-stream full file uploads without chunking.
Cloud Object Storage (S3)Video-on-Demand (VOD) PipelineContainer Image Registry
FAMILY N · CONCURRENCYLocal Instant + Background Sync

Collaborative & Conflict-Tolerant Systems

Multi-master concurrent edits on shared document state without central locking. Requires mathematical convergence guarantees.

Key PatternsConflict-Free Replicated Data Types (CRDTs), Operational Transformation (OT), Join-Semilattices (⊔), State vector synchronization.
Core EnginesYjs, Automerge, Redis Pub/Sub, WebSocket gateways.
Critical TrapLast-Write-Wins (LWW) timestamp overwriting in collaborative documents causing silent character loss.
Real-Time Collaborative Doc EditingShared Whiteboard CanvasesOffline-First Mobile Sync
FAMILY O · TENANT ISOLATIONPer-Tenant SLA Enforcement

Multi-Tenant SaaS & Shared Infrastructure

Shared cloud infrastructure serving thousands of independent enterprise tenants with strict isolation, fair-share scheduling, and encryption boundaries.

Key PatternsPool vs Silo architecture, Hierarchical token bucket rate limiting, Noisy neighbor throttling, Row-Level Security (RLS), Per-tenant encryption (BYOK).
Core EnginesPostgreSQL with RLS, Envoy tenant rate limiting, Kubernetes soft multi-tenancy.
Critical TrapShared unmetered database pools allowing one runaway tenant to starve all other enterprise customers.
B2B Enterprise SaaSManaged Message Broker CloudShared Analytics Cloud
FAMILY P · HIGH THROUGHPUTMinutes to Hours Completion Window

Batch & Offline Processing Systems

Large-scale offline computations scheduled during off-peak windows or on recurring intervals. Fault-tolerant task checkpointing and speculative execution.

Key PatternsMap/Shuffle/Reduce, DAG dependency execution, Speculative task execution, Idempotent backfilling, Distributed data lineage.
Core EnginesApache Spark, Apache Airflow, dbt, Apache Hadoop.
Critical TrapNon-idempotent batch mutations that duplicate or corrupt financial ledger state on retry after failure.
Nightly Financial SettlementBulk Data MigrationsML Training PreprocessingOffline Search Indexing
PRACTICAL ARCHITECTURAL METHOD

Multi-Workload Decomposition: Ride-Sharing Platform

Real production systems are composites of multiple distinct workloads. Never force one database or pattern across the entire system.

SubsystemPrimary FunctionWorkload FamilyOptimal Architectural PatternStorage Engine
1. Driver GPS StreamIngest GPS coordinates from 2M active drivers every 3sFamily B (Write-Heavy)Partitioned log buffer, zero-copy socket transfersApache Kafka + Mem-ring
2. Driver DiscoveryFind 10 nearest available drivers within 3 kmFamily I (Geospatial)H3 hexagonal spatial tessellation, cell replicationRedis GEO / Uber H3
3. Dispatch MatcherPair driver and rider in < 500 ms with surge pricingFamily E (Low-Latency)In-memory state machine, strict deadline hedgingGo/Rust In-Memory Engine
4. Ride Booking LedgerDebit customer, credit driver, and create immutable ledgerFamily C (Transactional)ACID serializable transaction, Outbox CDC, SagasPostgreSQL / CockroachDB
5. Location PushStream live vehicle position to rider phone at 1 HzFamily F (Streaming Pub/Sub)Stateful WebSocket connection pinning, Redis pub/subDistributed Gateway Mesh
6. Driver SettlementCalculate weekly payouts, tax withholdings, and bank transfersFamily P (Batch Offline)Idempotent MapReduce, checkpointed distributed shuffleApache Spark Batch
7. Surge Pricing AnalyticsCompute dynamic demand heatmaps across metropolitan zonesFamily L (Analytical OLAP)Columnar vectorized scan, streaming materialized viewsClickHouse OLAP