CHAPTER 0101
Modern accelerator hardware, HBM memory hierarchies, and multi-node parameter sharding (ZeRO / FSDP).
LEARNING PATH 04
Distributed AI and ML systems
Study distributed training, inference, GPU orchestration, vector retrieval and AI platforms.
CURRICULUM SEQUENCE
Problem-Oriented Chapters
CHAPTER 0202
Ring and Tree All-Reduce collective communication topologies, NVLink, and RoCEv2 RDMA.
CHAPTER 0303
Prefill vs. decode phases, dynamic KV cache management with PagedAttention, and continuous batching.
CHAPTER 0404
HNSW graphs, IVF clustering, and Product Quantization (PQ) for high-dimensional retrieval.
CHAPTER 0505
All-or-nothing gang scheduling for multi-node GPU clusters, locality constraints, and topology-aware placement.
CHAPTER 0606
Point-in-time correct feature stores, high-throughput streaming embedding pipelines, and data freshness guarantees.
CHAPTER 0707
Multi-model PPO/DPO policy alignment on Ray and durable state machines for multimodal tool-calling agents.
CHAPTER 0808
MFU telemetry, TTFT/ITL profiling, NCCL diagnostics, and the LLM Inference & Enterprise RAG Capstones.