By the end of this lesson
  • Deconstruct the process and threading models of NGINX (master-worker, non-blocking epoll) vs HAProxy (multi-threaded event engine with zero-copy TCP splicing)
  • Analyze HAProxy stick tables for stateful real-time tracking, rate limiting, and peer-to-peer memory synchronization
  • Master OpenResty and LuaJIT execution phases: cosockets, non-blocking coroutines, and shared memory slabs (lua_shared_dict)
  • Compare modern control plane distribution mechanisms: PostgreSQL polling (Kong) vs etcd v3 MVCC HTTP/2 streaming (Apache APISIX)

At the edge of modern distributed infrastructures, reverse proxies and API gateways are the primary entry points for planetary-scale traffic. Every incoming HTTP, gRPC, and WebSocket request traverses these edge layers before reaching internal services. Achieving sub-millisecond routing, TLS termination, load balancing, rate limiting, and authentication across hundreds of thousands of concurrent connections requires specialized operating system primitives, lock-free memory layouts, and asynchronous event loops.

This architectural study deconstructs the foundational systems powering edge routing:

  • HAProxy: The gold standard in high-performance Layer 4 and Layer 7 proxying, featuring zero-copy TCP splicing, lock-free stick tables, and socket-passing hitless reloads.
  • NGINX & OpenResty: The event-driven master-worker paradigm, epoll notification loops, and LuaJIT coroutine cosockets.
  • Kong & Apache APISIX: Next-generation programmable API gateways comparing RDBMS-backed polling and invalidation clusters against distributed etcd v3 MVCC streaming.

1. Context & Architecture Setting

Project Identity, Version & License Baseline

Project ComponentCanonical RepositoryPinned ReleaseGovernance / LicenseEdition Boundary
HAProxyhaproxy/haproxyv2.8 (LTS)Willy Tarreau / Community (GPL v2 / LGPL 2.1)Open source
NGINXnginx/nginxv1.26 (Stable)F5 / NGINX Open Source (2-clause BSD)Open source core vs NGINX Plus
Kong GatewayKong/kongv3.6Kong Inc (Apache 2.0)Open source core vs Kong Enterprise
Apache APISIXapache/apisixv3.8Apache Software Foundation (Apache 2.0)Open source

Every edge gateway faces a severe concurrency challenge known historically as the C10K (and now C10M) problem:

  1. Thread-per-connection exhaustion: Early servers (such as Apache HTTPD prefork or naive thread pools) allocated one operating system thread per connection. At concurrent connections, stack allocations alone ( per thread) consume hundreds of gigabytes of RAM, while Linux kernel O(N) context switching renders CPU caches useless.
  2. Synchronous I/O stalls: If a worker blocks waiting for an upstream database query or backend microservice response, that compute resource cannot serve other incoming connections.
  3. Dynamic policy application: Modern cloud-native architectures require modifying routing rules, rate limits, and authentication keys hundreds of times per day without dropping in-flight TCP handshakes or causing latency spikes.
+-----------------------------------------------------------------------------+
|                          EDGE ARCHITECTURE SPECTRUM                         |
+-----------------------------------------------------------------------------+
| Layer 4 (Transport)     | Layer 7 (Application)     | Programmable Gateway  |
| HAProxy / IPVS          | NGINX / Envoy             | Kong / Apache APISIX  |
|                         |                           |                       |
| - TCP/UDP stream proxy  | - HTTP/1.1, HTTP/2, gRPC  | - Dynamic Auth (JWT)  |
| - Zero-copy splice()    | - TLS Termination / SNI   | - Distributed Limits  |
| - Stick table counters  | - Header rewrite / cache  | - Lua / Wasm plugins  |
| - Kernel socket handoff | - epoll event loops       | - etcd / DB-less sync |
+-----------------------------------------------------------------------------+

2. Architectural Mental Model

SYSTEM ARCHITECTURE BLUEPRINTNETWORKING
Architecture Study: High-Throughput Ingress, Proxies and API Gateways
ARCHITECTURAL FLOW · STEP 1 OF 3

1. Layer 4 Ingress and HAProxy Core & Linux Kernel: SO_REUSEPORT

Establishes the ingress and structural boundary across Layer 4 Ingress and HAProxy Core & Linux Kernel: SO_REUSEPORT: Kernel 4-tuple hash distributes incoming SYN Zero-lock lockless connection distribution; Single-process multi-threaded event loop; Zero-copy TCP splicing (splice syscall).

Architectural Invariant: Sub-microsecond rate-limit lookup.
ARCHITECTURAL FLOW · STEP 2 OF 3

2. NGINX Core and OpenResty Architecture & Master Process (Privileged PID 1)

Coordinates core data-plane execution and state transitions through NGINX Core and OpenResty Architecture & Master Process (Privileged PID 1): Reads and validates nginx.conf; opens sockets; Forks workers; orchestrates hot reload via SIGHUP; epoll_wait() loop Non-blocking state LuaJIT VM instance 100k+ keepalive conns.

Architectural Invariant: Zero OS thread context-switch penalty.
ARCHITECTURAL FLOW · STEP 3 OF 3

3. Gateway Control Planes: Kong vs APISIX & Kong Gateway Control Plane

Enforces durability, state convergence, and output verification across Gateway Control Planes: Kong vs APISIX & Kong Gateway Control Plane: Storage: PostgreSQL / Cassandra or DB-less YAML; Invalidation: Cluster events bus over pub/sub; Worker Cache: Local memory LRU cache per node ⚠ Cache warm-up latency and DB polling overhead.

Architectural Invariant: Zero polling lag; radix-tree route matching.

To understand how high-throughput proxies operate, we must contrast the process and concurrency models of HAProxy, NGINX, and OpenResty-based Gateways.

The HAProxy Model: Multi-Threaded Single Process

HAProxy operates as a single master process running multiple worker threads pinned to CPU cores.

  • Unified Event Scheduler: Each thread runs an event loop monitoring file descriptors via Linux epoll or BSD kqueue.
  • Zero-Copy TCP Splicing: For Layer 4 proxying, HAProxy uses splice(2). Network data moves directly from the incoming socket buffer into the outgoing socket buffer inside kernel memory, never copying payload bytes into user-space buffers.
  • Stick Tables: An in-memory, lock-free key-value store using Elastic Binary Trees (ebtree) that tracks client IP connection rates, error counts, and session cookies across threads and peered cluster nodes.

The NGINX Model: Master-Worker Event Loop

NGINX isolates concurrency across independent worker processes rather than threads:

  • Master Process: Runs with privileged access, parses configuration files, binds to listening sockets (e.g., ports 80 and 443), and manages worker lifecycles.
  • Worker Processes: Unprivileged processes forked from the master. Each worker runs a single-threaded non-blocking event loop. Connections are distributed via Linux SO_REUSEPORT, allowing the kernel networking stack to load-balance incoming SYN packets across worker listening sockets without lock contention.
  • Shared Memory Slabs: Workers communicate shared state (rate limits, SSL session caches, upstream health) through POSIX shared memory segments (shm) protected by lightweight atomic spinlocks.

The OpenResty / LuaJIT Gateway Model

OpenResty embeds the LuaJIT compiler inside NGINX worker processes:

  • Cosocket Mechanism: When Lua code inside a plugin performs network I/O (e.g., querying Redis or verifying an OAuth token), OpenResty yields the Lua coroutine. The underlying socket is registered with the NGINX epoll loop. When data arrives, NGINX resumes the coroutine.
  • Developer Simplicity, Async Speed: Engineers write straightforward procedural code (local res = http:request(...)) while the underlying engine executes fully non-blocking I/O.

3. Core Mechanisms & Protocol Internals

Master-Worker Event Loops & Socket Inheritance

When NGINX initializes:

  1. The master process calls socket(), setsockopt(SO_REUSEPORT), bind(), and listen().
  2. Master forks worker processes (typically ).
  3. Each worker process inherits the file descriptors and executes:
  4. When a reload signal (SIGHUP) is sent, the master spawns new workers with the updated configuration. Old workers stop accepting new connections, finish serving active connections within a grace period, and terminate cleanly.
       [Client Traffic]
              |
              v (Kernel 4-tuple hashing)
     +-----------------+
     |  SO_REUSEPORT   |
     +--------+--------+
              |
       +------+------+
       |             |
       v             v
+--------------+ +--------------+
| Worker 0     | | Worker 1     |
| epoll_wait() | | epoll_wait() |
| LuaJIT VM    | | LuaJIT VM    |
+-------+------+ +-------+------+
        |                |
        +--------+-------+
                 |
                 v
      [lua_shared_dict Slab]
      (Atomic CAS Rate Limits)

HAProxy Zero-Copy Splicing

In standard user-space proxies, relaying a packet requires two system calls and two memory copies:

  1. read(client_fd, buf, len) copies kernel receive buffer to user memory.
  2. write(backend_fd, buf, len) copies user memory to kernel send buffer.

HAProxy avoids this through Linux pipe buffers: No payload bytes cross the kernel-user boundary. The CPU only manipulates buffer pointer descriptors, slashing memory bus traffic and CPU cache misses.

HAProxy Stick Tables & Cluster Synchronization

Stick tables provide stateful distributed intelligence at line rate:

  • Elastic Binary Trees (ebtree): O(log N) lookup and insertion with zero memory fragmentation.
  • Peering Protocol: HAProxy nodes stream stick table state changes over dedicated TCP connections (peers section) using a compact binary protocol, providing real-time global DDoS protection across a cluster.

4. Gateway Control Plane Distribution: Kong vs Apache APISIX

Programmable API gateways differ fundamentally in how they distribute configuration changes to worker data planes:

Architectural DimensionKong Gateway (Traditional)Apache APISIX
Configuration StorePostgreSQL / Cassandra or DB-less YAMLDistributed etcd v3 cluster (MVCC)
Worker SynchronizationPolling database + cluster event pub/subHTTP/2 long-polling Watch API
Propagation LatencyHundreds of milliseconds to secondsUnder 5 milliseconds
Routing AlgorithmLua-based prefix tree matchingRadix Tree (rax) implemented in C
Process ModelOpenResty NGINX wrapperOpenResty with optimized C libraries
Dynamic SSL / UpstreamsCached in lua_shared_dict via pollingStreamed via etcd watch into shared memory

Apache APISIX etcd Watch Architecture

Rather than polling a relational database, APISIX workers establish long-lived HTTP/2 streams to etcd:

  1. An administrator posts a route change to the Admin API: PUT /apisix/admin/routes/1.
  2. etcd persists the key-value revision in its Raft-backed Bbolt engine.
  3. etcd immediately pushes the revision delta over the active Watch connection to all APISIX worker nodes.
  4. Each worker updates its in-memory Radix tree atomically using pointer swaps. Zero master reloads, zero dropped connections.

5. Mathematical & Performance Analysis

Ingress Queueing & Little’s Law

The concurrency capacity of an edge proxy is governed by Little’s Law: Where:

  • = Average number of concurrent open connections inside the proxy.
  • = Ingress arrival rate (requests per second).
  • = Total connection residence time (including upstream latency and network transfer).

If an edge proxy receives and the mean upstream latency is :

However, if an upstream service degrades and climbs to under a retry storm:

If the proxy worker limit (worker_connections) is configured to , the proxy runs out of file descriptors, rejecting new client connections with HTTP 502 / 504 errors.

Memory Footprint of Concurrent Connections

The RAM requirement of an event-driven reverse proxy is calculated as: Where:

  • = Linux kernel socket buffer size (rmem + wmem tuned).
  • = Proxy user-space buffer (proxy_buffer_size ).
  • At concurrent idle keepalive connections:

An event-driven proxy handles 1M idle connections in 8 GB RAM, whereas a thread-per-connection architecture would require .


6. Concrete Failure Modes & Edge Cases

1. The OpenResty Coroutine Blocking Trap

  • Failure Mode: An engineer imports a third-party Lua library that uses standard Lua socket or C-FFI synchronous file I/O instead of ngx.socket.tcp().
  • System Impact: The entire single-threaded NGINX worker stops processing events. All concurrent requests on that worker stall until the synchronous I/O unblocks or times out, causing latency spikes that alternate across CPU cores.
  • Remediation: Enforce automated linting to ban standard Lua io.* and blocking sockets. Use OpenResty cosocket-compatible drivers (lua-resty-redis, lua-resty-http).

2. NGINX Master Reload Drop During High Connection Velocity

  • Failure Mode: Under heavy traffic (), executing nginx -s reload triggers SIGHUP.
  • System Impact: Old workers close their listen sockets and enter drain mode. If new workers take several milliseconds to initialize Lua VMs and compile regex routes, the Linux socket listen backlog (backlog parameter in listen) overflows, dropping SYN packets and causing TCP connection timeouts.
  • Remediation: Use reuseport with HAProxy hitless reload (-x socket handoff) or use dynamic control planes like APISIX/Envoy that update routes via memory pointers without process restarts.

3. lua_shared_dict Spinlock Contention

  • Failure Mode: Hundreds of concurrent requests on high-core servers () increment a single global rate-limit key in lua_shared_dict.
  • System Impact: OpenResty’s shared memory dictionary uses atomic CPU spinlocks. Severe contention across 128 cores causes CPU utilization to spike to 100% in kernel spin-lock loops without processing useful traffic.
  • Remediation: Shard shared memory keys across multiple buckets (e.g., key .. ":" .. (worker_id % 16)), or use local worker caching with periodic batch synchronization.

7. Production Trade-Offs & Anti-Patterns

Trade-Off Matrix

StrategyAdvantagesDisadvantagesOptimal Use Case
Pure HAProxy (L4)Unbeatable throughput, minimal memory, kernel splice()No payload transformation, limited dynamic routingTCP/UDP ingress, database connection pooling, TLS pass-through
NGINX / OpenRestyMature ecosystem, static asset speed, rich Lua scriptingReload cost for config changes, single-threaded worker sensitivityHigh-scale static caching, edge TLS termination, CDN nodes
Apache APISIXReal-time dynamic updates via etcd, sub-ms sync, Radix treeComplex ops (etcd quorum dependency), higher memory than raw NGINXMicroservice API Gateway, dynamic multi-tenant SaaS ingress
Kong GatewayRich enterprise plugin ecosystem, DB-less optionPostgreSQL scaling bottlenecks at high node counts if not DB-lessEnterprise developer portal, API governance, hybrid cloud ingress

8. Practical Engineering Simulation: Edge Gateway & Token Bucket Router

The following self-contained TypeScript simulation models an edge router with an L4 load-balancing distributor, an L7 URL-prefix router with dynamic Radix matching, and a non-blocking token bucket rate-limiting filter.

/**
 * Systems Notebook - Edge Gateway & Rate-Limiting Simulation
 * Models L4 connection distribution, L7 routing, and non-blocking token bucket filters.
 */

interface Request {
  id: string;
  clientIp: string;
  path: string;
  method: 'GET' | 'POST';
  timestampMs: number;
}

interface UpstreamTarget {
  id: string;
  host: string;
  activeConnections: number;
  weight: number;
  isHealthy: boolean;
}

interface RouteRule {
  prefix: string;
  upstreamPool: string;
  rateLimitQps: number;
  burstCapacity: number;
}

interface RateLimitBucket {
  tokens: number;
  lastRefillMs: number;
}

class TokenBucketFilter {
  private buckets: Map<string, RateLimitBucket> = new Map();

  constructor(
    private readonly qps: number,
    private readonly capacity: number
  ) {}

  public allowRequest(clientKey: string, nowMs: number): boolean {
    let bucket = this.buckets.get(clientKey);

    if (!bucket) {
      bucket = { tokens: this.capacity, lastRefillMs: nowMs };
      this.buckets.set(clientKey, bucket);
    } else {
      // Calculate token refill based on elapsed time
      const elapsedSeconds = (nowMs - bucket.lastRefillMs) / 1000;
      const tokensToAdd = elapsedSeconds * this.qps;
      bucket.tokens = Math.min(this.capacity, bucket.tokens + tokensToAdd);
      bucket.lastRefillMs = nowMs;
    }

    if (bucket.tokens >= 1.0) {
      bucket.tokens -= 1.0;
      return true; // Allowed
    }

    return false; // Rate-limited (HTTP 429)
  }
}

class EdgeGateway {
  private routes: RouteRule[] = [];
  private upstreams: Map<string, UpstreamTarget[]> = new Map();
  private rateLimiters: Map<string, TokenBucketFilter> = new Map();

  public registerRoute(rule: RouteRule): void {
    this.routes.push(rule);
    // Sort descending by prefix length for longest-prefix match
    this.routes.sort((a, b) => b.prefix.length - a.prefix.length);
    this.rateLimiters.set(
      rule.prefix,
      new TokenBucketFilter(rule.rateLimitQps, rule.burstCapacity)
    );
  }

  public registerUpstreamPool(poolName: string, targets: UpstreamTarget[]): void {
    this.upstreams.set(poolName, targets);
  }

  /**
   * L4 Least-Connection Load Balancing
   */
  private selectTarget(poolName: string): UpstreamTarget | null {
    const targets = this.upstreams.get(poolName);
    if (!targets || targets.length === 0) return null;

    const healthy = targets.filter((t) => t.isHealthy);
    if (healthy.length === 0) return null;

    // Pick target with minimum active connections
    return healthy.reduce((min, curr) =>
      curr.activeConnections < min.activeConnections ? curr : min
    );
  }

  /**
   * L7 Process Pipeline
   */
  public handleRequest(req: Request): {
    status: number;
    upstreamNode?: string;
    error?: string;
  } {
    // 1. Longest-Prefix L7 Route Matching
    const matchedRoute = this.routes.find((r) => req.path.startsWith(r.prefix));
    if (!matchedRoute) {
      return { status: 404, error: 'Route Not Found' };
    }

    // 2. Token Bucket Rate Limiting (access_by_lua phase)
    const limiter = this.rateLimiters.get(matchedRoute.prefix)!;
    const allowed = limiter.allowRequest(req.clientIp, req.timestampMs);
    if (!allowed) {
      return { status: 429, error: 'Rate Limit Exceeded (HTTP 429)' };
    }

    // 3. Upstream Target Selection (L4 leastconn load balancer)
    const target = this.selectTarget(matchedRoute.upstreamPool);
    if (!target) {
      return { status: 503, error: 'Service Unavailable (No Healthy Upstreams)' };
    }

    // Simulate connection acquisition
    target.activeConnections++;
    const responseNode = `${target.id} (${target.host})`;

    // Simulate immediate release for this discrete tick
    target.activeConnections--;

    return { status: 200, upstreamNode: responseNode };
  }
}

// -------------------------------------------------------------
// Verification Demonstration
// -------------------------------------------------------------
function runSimulation(): void {
  const gateway = new EdgeGateway();

  gateway.registerUpstreamPool('orders-cluster', [
    { id: 'node-1', host: '10.0.1.10:8080', activeConnections: 12, weight: 1, isHealthy: true },
    { id: 'node-2', host: '10.0.1.11:8080', activeConnections: 4,  weight: 1, isHealthy: true },
    { id: 'node-3', host: '10.0.1.12:8080', activeConnections: 25, weight: 1, isHealthy: false } // Down
  ]);

  gateway.registerRoute({
    prefix: '/api/v1/orders',
    upstreamPool: 'orders-cluster',
    rateLimitQps: 5,        // 5 requests per second
    burstCapacity: 2        // Max burst of 2 tokens
  });

  const clientIp = '198.51.100.42';
  let virtualTimeMs = 1000;

  console.log('--- Phase 1: Burst of 3 concurrent requests ---');
  for (let i = 1; i <= 3; i++) {
    const res = gateway.handleRequest({
      id: `req-${i}`,
      clientIp,
      path: '/api/v1/orders/create',
      method: 'POST',
      timestampMs: virtualTimeMs
    });
    console.log(`Req ${i}: Status ${res.status} | Routed To: ${res.upstreamNode ?? res.error}`);
  }

  console.log('\n--- Phase 2: Advance time by 500ms (Tokens refilled: 0.5 * 5 = 2.5) ---');
  virtualTimeMs += 500;
  const resAfterRefill = gateway.handleRequest({
    id: 'req-4',
    clientIp,
    path: '/api/v1/orders/status',
    method: 'GET',
    timestampMs: virtualTimeMs
  });
  console.log(`Req 4: Status ${resAfterRefill.status} | Routed To: ${resAfterRefill.upstreamNode ?? resAfterRefill.error}`);
}

runSimulation();

Simulation Execution Output & State Transitions

--- Phase 1: Burst of 3 concurrent requests ---
Req 1: Status 200 | Routed To: node-2 (10.0.1.11:8080)
Req 2: Status 200 | Routed To: node-2 (10.0.1.11:8080)
Req 3: Status 429 | Routed To: Rate Limit Exceeded (HTTP 429)

--- Phase 2: Advance time by 500ms (Tokens refilled: 0.5 * 5 = 2.5) ---
Req 4: Status 200 | Routed To: node-2 (10.0.1.11:8080)

9. Observability & SRE Metrics

To maintain edge proxy health, SREs monitor four foundational golden signals:

+-------------------------------------------------------------------------------+
|                        KEY EDGE GATEWAY TELEMETRY                             |
+-------------------------------------------------------------------------------+
| Metric Name             | Source / Exporter     | SRE Target / Alert Rule     |
+-------------------------+-----------------------+-----------------------------+
| Active Connections      | HAProxy show info     | Alert if active > 80% of    |
| (current_conn)          | NGINX stub_status     | worker_connections limit    |
| Upstream Response Time  | $upstream_response_   | p99 < 50ms; p99.9 < 200ms   |
| (p95, p99, p99.9)       | time in access log    | Spike indicates backend lag |
| Rate Limit Discards     | APISIX / Kong metric  | Spike indicates brute-force |
| (http_status 429)       | prometheus plugin     | or misconfigured client     |
| etcd Watch Lag          | APISIX etcd_sync_lag  | Alert if lag > 500ms        |
|                         | prometheus counter    | indicates config drift      |
+-------------------------+-----------------------+-----------------------------+

10. System Design Interview & Production Rubric

When presenting edge routing in a system design interview:

+------------------------------------------------------------------------------+
| LEVEL 3 (Senior Engineer)       | LEVEL 4 (Staff / Principal Architect)      |
+---------------------------------+--------------------------------------------+
| - Mentions NGINX or HAProxy as  | - Distinguishes L4 zero-copy splice from   |
|   a reverse proxy box           |   L7 TLS termination and request parsing   |
| - Configures round-robin and    | - Designs dynamic etcd-backed control      |
|   basic SSL termination         |   planes to eliminate reload packet drop   |
| - Relies on simple static IP    | - Implements multi-tier rate limiting with |
|   allow-lists                   |   sharded shared memory and Redis fallback |
| - Unaware of reload packet drop | - Explains kernel SO_REUSEPORT and CPU core|
|   under high SYN traffic        |   pinning to eliminate cache contention    |
+---------------------------------+--------------------------------------------+

11. Exercises & Deliverables

  1. HAProxy vs NGINX Benchmarking:
    • Deploy HAProxy in Layer 4 mode and NGINX in Layer 7 mode using Docker or Linux network namespaces.
    • Run a wrk or vegeta benchmark with concurrent connections. Measure CPU utilization, context switch rate (vmstat 1), and p99 latency.
  2. Hitless Reload Verification:
    • Write a client script generating continuous HTTP requests at .
    • Issue 50 successive reload signals to NGINX (kill -HUP) and HAProxy (-x socket handoff). Verify whether any TCP connections reset or return 502.
  3. Custom OpenResty Plugin:
    • Implement an OpenResty Lua filter that extracts an HMAC-SHA256 signature from an incoming header, verifies it against a key stored in lua_shared_dict, and injects a sanitized tenant ID header before proxying upstream.
View Reference Architecture & Worked Solutions (CORE-M36A-002)

1. Worked Solution — Exercise 1: HAProxy (L4) vs. NGINX (L7) Benchmarking ( Concurrent Connections)

  • Data Path & Syscall Comparison:
    • HAProxy (mode tcp + option splice-auto): Executes splice(client_fd, pipe_w, len, SPLICE_F_MOVE | SPLICE_F_NONBLOCK) followed by splice(pipe_r, backend_fd, len, SPLICE_F_MOVE | SPLICE_F_NONBLOCK). Payload pages remain pinned inside kernel sk_buff ring memory; only DMA descriptor pointers move between socket buffers. Zero bytes are copied into user-space virtual memory, keeping L1/L2 CPU caches warm for ebtree stick-table lookups.
    • NGINX (http L7 proxy): Executes epoll_wait() read(client_fd, user_buf) (kernel-to-user copy) HTTP state-machine header parsing and buffer chain allocation (ngx_buf_t) writev(upstream_fd, iov) (user-to-kernel copy).
  • Expected Quantitative Telemetry under Connections (, payload):
    • Memory Footprint: With tuned buffers ( kernel socket + user buffer), connections consume RAM—confirming event-driven per-connection overhead.
    • CPU & Context Switches (vmstat 1): With worker threads/processes pinned 1:1 to physical cores (cpu-map in HAProxy, worker_cpu_affinity in NGINX), voluntary context switches (cs) remain low (, driven purely by epoll_wait wakeups). HAProxy L4 consumes less CPU bandwidth than L7 proxying due to zero-copy splicing and bypassing HTTP header tokenization, yielding proxy latency of (L4) vs. (L7).

2. Worked Solution — Exercise 2: Hitless Reload Verification (NGINX SIGHUP vs. HAProxy -x Socket Handoff)

  • Why Traditional Reloads Drop Packets Under High SYN Velocity:
    • When SO_REUSEPORT is enabled, the Linux kernel hashes incoming SYN packets across the set of active listening socket file descriptors. During a standard kill -HUP reload, old workers close their listening FDs while new workers bind fresh SO_REUSEPORT sockets. Any connection that completed its 3-way handshake (or is mid-handshake) inside an old listening socket’s accept queue at the moment that FD closes is sent a TCP RST (or dropped) unless kernel BPF_SOCK_OPS reuseport groups drain it cleanly. At with rapid reloads, a drain gap per reload risks resetting handshakes.
  • HAProxy Hitless Reload Configuration & Mechanics:
    • Configure the master-worker stats socket in haproxy.cfg:
      global
          master-worker
          stats socket /var/run/haproxy-master.sock mode 600 level admin expose-fd listeners
    • Reload via haproxy -W -db -f /etc/haproxy/haproxy.cfg -x /var/run/haproxy-master.sock -sf $(cat /var/run/haproxy.pid).
    • The new HAProxy process connects to /var/run/haproxy-master.sock and receives the exact open listening file descriptors via UNIX domain socket sendmsg(2) with SCM_RIGHTS ancillary data. Because the kernel socket structs and their SYN/accept queues are never closed or re-allocated, zero packets are dropped across all 50 reloads.

3. Worked Solution — Exercise 3: Custom Non-Blocking OpenResty HMAC-SHA256 Plugin

-- nginx.conf (http block):
-- lua_shared_dict tenant_keys 10m;
-- init_worker_by_lua_block {
--     ngx.shared.tenant_keys:set("tenant_acme", "super-secret-signing-key-256")
-- }

-- access_by_lua_block implementation:
local resty_hmac = require("resty.hmac")
local resty_str  = require("resty.string")

local headers   = ngx.req.get_headers()
local tenant_id = headers["X-Client-Tenant"]
local signature = headers["X-Signature-HMAC"]

if not tenant_id or not signature then
    ngx.status = ngx.HTTP_UNAUTHORIZED
    ngx.say('{"error":"Missing X-Client-Tenant or X-Signature-HMAC header"}')
    return ngx.exit(ngx.HTTP_UNAUTHORIZED)
end

-- 1. O(1) lock-free/spinlock lookup from POSIX shared memory slab (zero network I/O)
local secret = ngx.shared.tenant_keys:get(tenant_id)
if not secret then
    ngx.status = ngx.HTTP_FORBIDDEN
    ngx.say('{"error":"Unknown tenant identity"}')
    return ngx.exit(ngx.HTTP_FORBIDDEN)
end

-- 2. Read request body non-blockingly and compute HMAC-SHA256
ngx.req.read_body()
local body_data = ngx.req.get_body_data() or ""
local signing_input = ngx.var.request_method .. ":" .. ngx.var.uri .. ":" .. body_data

local hmac_sha256 = resty_hmac:new(secret, resty_hmac.ALGOS.SHA256)
hmac_sha256:update(signing_input)
local expected_hex = resty_str.to_hex(hmac_sha256:final())

-- 3. Constant-time byte comparison to prevent timing side-channel attacks
if #signature ~= #expected_hex then
    return ngx.exit(ngx.HTTP_UNAUTHORIZED)
end
local diff = 0
for i = 1, #expected_hex do
    diff = bit.bor(diff, bit.bxor( string.byte(signature, i), string.byte(expected_hex, i) ))
end
if diff ~= 0 then
    return ngx.exit(ngx.HTTP_UNAUTHORIZED)
end

-- 4. Sanitize and inject verified tenant header for upstream microservice
ngx.req.clear_header("X-Signature-HMAC")
ngx.req.set_header("X-Verified-Tenant-ID", tenant_id)
  • Event-Loop Safety Invariant: Every operation (ngx.shared.DICT:get, ngx.req.read_body, OpenSSL C-FFI resty.hmac) executes either in-memory in microseconds or yields via OpenResty’s epoll-integrated coroutine scheduler, guaranteeing zero worker event-loop stalls.

12. Comprehensive Architectural Summary

Modern edge routing relies on specialized architectural divisions:

  • HAProxy reigns at Layer 4 transport proxying due to its multi-threaded event engine, zero-copy kernel splice() data relay, and lightning-fast ebtree stick tables.
  • NGINX and OpenResty dominate Layer 7 web serving and static content due to battle-tested master-worker isolation and LuaJIT non-blocking coroutines.
  • Apache APISIX and Kong represent the modern programmable API gateway tier, with APISIX leveraging etcd v3 MVCC HTTP/2 streaming to achieve zero-reload real-time configuration synchronization across globally distributed edge fleets.

13. Authoritative References & Version Context

  • Igor Sysoev (2004–2024), “NGINX Architecture Overview & Internal Event Processing” — nginx.org/en/docs/
  • Willy Tarreau (2000–2024), “HAProxy Architecture and Performance Tuning Guide” — haproxy.org
  • Kong Gateway Documentation (v3.5+) — Architecture and Declarative DB-less Configuration — docs.konghq.com/gateway/latest/
  • Version Context: Edge gateways reflect NGINX 1.26 LTS, HAProxy 2.8 / 3.0 LTS, and Kong Gateway 3.5+ LTS enterprise deployments.
MASTERY GATE · STEP 64 OF 94

Knowledge check & lesson completion

Score 4 / 4 (100%) on this check to mark this lesson Completed ✓, advance your position on the Master Spine, and add these concepts to your Spaced-Repetition Review Deck.

1. Why does HAProxy achieve significantly lower CPU utilization and higher packet throughput than standard user-space reverse proxies during pure Layer 4 TCP proxying?
2. In an OpenResty-based gateway (such as Kong or APISIX), what fatal architectural mistake occurs if an engineer executes a blocking C-library DNS resolution or standard Lua socket call inside access_by_lua?
3. How does Apache APISIX eliminate the configuration-reload latency and database bottleneck inherent in traditional PostgreSQL-backed API gateway clusters?
4. What is the primary advantage of HAProxy's hitless reload mechanism (-x socket transfer) compared to the traditional NGINX SIGHUP worker retirement model?
PERSONAL NOTEBOOK

Chapter Notes

Personal engineering insights and observations saved to this browser.

0 notes
Tip: You can also highlight text in the article above to quote directly.

Loading personal notes...