Architecture Study: Large-Scale Distributed Storage — Vitess, ClickHouse & Ceph

Scaling storage past petabytes requires acknowledging a fundamental truth of database architecture: no single engine can optimize both high-concurrency OLTP row transactions and high-throughput OLAP columnar analytical aggregations simultaneously.

This module deconstructs two world-class open-source storage systems designed for opposite extremes:

• Vitess (OLTP Sharding): Originally built at YouTube to scale MySQL to billions of users, Vitess provides a transparent proxy layer (VTGate) that shards and routes relational SQL queries across thousands of MySQL database instances without requiring application code changes. Through VReplication, Vitess executes zero-downtime resharding and table splits under live traffic.
• ClickHouse (OLAP Columnar Engine): An open-source columnar database capable of processing hundreds of millions of rows per second per server. Through the MergeTree storage engine, physical column compression, sparse primary key indexing, and vectorized SIMD execution, ClickHouse accelerates analytical aggregations by $100\times$ to $1000\times$ compared to traditional row stores.

This architectural study contrasts horizontal OLTP sharding with vectorized OLAP data processing at petabyte scale.