Distributed Object Storage & Erasure Coding: Ceph, S3 & Reed-Solomon

Modern hyperscale cloud platforms—AWS S3, Google Cloud Storage, Ceph, MinIO—store petabytes to exabytes of unstructured objects (images, videos, machine learning training datasets, database backups).

Storing this volume of data using naive $3\times$ replication requires a ruinous 200% storage overhead: 1 Exabyte of user data demands 3 Exabytes of physical raw hard drives! Furthermore, centralized metadata servers (like early Hadoop NameNodes) become severe bottlenecks and single points of failure when tracking billions of small files.

This module deconstructs modern distributed object storage engineering:

• Erasure Coding Mathematics: How Reed-Solomon $(k+m)$ transforms $k$ data blocks into $k+m$ encoded chunks using Galois Field arithmetic, allowing the system to tolerate the loss of any $m$ disks with only $m/k$ storage overhead.
• Ceph CRUSH Algorithm: How clients calculate the exact physical drive location of any object chunk purely through algorithmic hashing against a cluster topology map, completely eliminating centralized metadata bottlenecks.
• Small-Object Needle Packing: Why creating a separate filesystem inode for every 10 KB thumbnail exhausts Linux inodes and destroys disk IOPS, and how Facebook’s Haystack and Bitcask append billions of needles into large immutable packfiles.