Time, Clocks, and Ordering: From Physical Drift to Vector Clocks and TrueTime

1. The Core Engineering Challenge: Time Does Not Exist

On a single machine, physical time appears orderly. When a thread executes instruction $A$ and then instruction $B$, the CPU cycle counter increments monotonically. If you save a file at 10:00:01 AM and modify it at 10:00:02 AM, the filesystem ordering is unambiguous.

In a distributed system, there is no global clock.

+-------------------------------------------------------------------------------+
|                       The Physical Clock Reality                              |
|                                                                               |
|  - Every server has its own quartz crystal oscillator on its motherboard.     |
|  - Quartz crystals drift based on ambient temperature, age, and voltage.      |
|  - A typical quartz clock drifts by ~1 to 10 parts per million (ppm).         |
|    1 ppm = 1 microsecond per second = 86 milliseconds per day!                |
|  - Virtual machine hypervisors pause guests for VM migration, freezing        |
|    time for seconds or minutes while the physical world marches forward.      |
+-------------------------------------------------------------------------------+

The Failure of the Network Time Protocol (NTP)

To synchronize clocks, servers communicate over the internet using the Network Time Protocol (NTP):

1. Server $A$ queries an authoritative Stratum-1 time server (synchronized to GPS or atomic clocks).
2. Server $A$ calculates round-trip network latency ($RTT$) and shifts its local clock toward the server’s time.

However, network latency across the internet or a cloud datacenter is inherently asymmetric and variable:

If outbound network packet delivery takes 15 ms and inbound return takes 45 ms due to switch queue buffering, NTP’s symmetry assumption fails, introducing tens of milliseconds of undetectable clock skew.

The Last-Write-Wins (LWW) Data Loss Disaster

Many distributed systems (such as Apache Cassandra or Couchbase) provide conflict resolution via Last-Write-Wins (LWW): when two updates collide, the update with the highest physical wall-clock timestamp wins, and the other is discarded.

SYSTEM ARCHITECTURE BLUEPRINTDISTRIBUTED SYSTEMS

⤢Expand

ARCHITECTURAL FLOW · STEP 1 OF 3

1. Node A (Ahead) & Node B (Lagging 15ms)

Establishes the ingress and structural boundary across Node A (Ahead) & Node B (Lagging 15ms): Clock: 10:00:00.005; Write v1 (ts=.005); Clock: 09:59:59.990.

Architectural Invariant: Local event on Node i: L_i = L_i + 1.

ARCHITECTURAL FLOW · STEP 2 OF 3

2. Message Transport & Lamport Properties

Coordinates core data-plane execution and state transitions through Message Transport & Lamport Properties: Sender attaches L_msg Receiver updates: L = max(L, L_msg) + 1; Clock Consistency: a -> b ==> L(a) < L(b) The Flaw: L(a) < L(b) does NOT imply that a caused b! Cannot detect if events a and b were concurrent; V = [v_1, v_2, ..., v_N] Node i increments V[i] V[i] = V[i] + 1.

Architectural Invariant: On receiving message: V[j] = max(V[j], Msg[j]) V[i] = V[i] + 1.

ARCHITECTURAL FLOW · STEP 3 OF 3

3. Full Causality Order & Hardware Anchors

Enforces durability, state convergence, and output verification across Full Causality Order & Hardware Anchors: V(a) < V(b) <==> a -> b Conflict Detection: If V(a) !< V(b) and V(b) !< V(a): Events are Concurrent! Signals Dynamo/Riak to preserve siblings; GPS Receivers + Atomic Rubidium oscillators in every datacenter; TT.now() returns.

Architectural Invariant: Transaction commits at t_commit = latest Coordinator sleeps until: TT.now().earliest > t_c Guarantees Linearizability: Any subsequent read is guaranteed to see t > t_c without global locks!.

Consider this production disaster:

1. 10:00:00.005: User updates their profile on Node A: set name = "Alice". Node A assigns timestamp 10:00:00.005.
2. Node A sends a notification to Node B over the network.
3. 10:00:00.010: Receiving the message, Node B immediately updates the profile: set name = "Bob".
4. However, Node B’s physical clock is lagging by 25 ms due to NTP drift! Node B records its write with local timestamp 09:59:59.985.
5. The database receives both writes and applies Last-Write-Wins: $$\text{Timestamp("Alice")}=10:00:00.005>\text{Timestamp("Bob")}=09:59:59.985$$
6. Result: The database silently deletes “Bob” and restores “Alice”!

Even though “Bob” occurred after “Alice” in physical reality and was directly caused by “Alice”, physical clock skew caused the newer update to be permanently eradicated from history.

2. Monotonic Clocks vs. Time-of-Day Wall Clocks

Operating systems expose two fundamentally different types of physical clocks:

1. Time-of-Day Wall Clock (CLOCK_REALTIME)

• Returns the current Gregorian calendar date and time (e.g., 1730344800 seconds since Unix Epoch).
• Non-Monotonic: When NTP resynchronizes, the operating system may step the clock backward by 500 ms!
• Never use System.currentTimeMillis() or Date.now() to measure elapsed time or calculate timeouts. If an NTP adjustment occurs during a database call, elapsed time can be calculated as negative, triggering infinite timeout loops or deadlock states!

2. Monotonic Clock (CLOCK_MONOTONIC)

• Returns the elapsed time since an arbitrary point in the past (e.g., since the server booted).
• Strictly Monotonic: Guaranteed to never jump backward, regardless of NTP adjustments. NTP only slews the frequency by up to 500 ppm ($0.05\%$).
• In Java: System.nanoTime(). In Node.js / Browser: performance.now(). In Go: time.Since(). In Python: time.monotonic().
• Limitation: Monotonic clocks have zero meaning across multiple servers. Server A’s monotonic clock started when Server A booted; comparing Server A’s nanoTime() to Server B’s nanoTime() is completely meaningless.

3. Logical Time: Lamport Timestamps & Causality

In 1978, Leslie Lamport published the seminal paper of distributed computing: “Time, Clocks, and the Ordering of Events in a Distributed System”.

Lamport recognized that physical time is irrelevant for software correctness. What matters is Causality: if event $A$ influenced or caused event $B$, then the system must order event $A$ before event $B$.

The “Happens-Before” Relation ($\to$)

Lamport defined the partial order Happens-Before ($a\to b$) through three axioms:

1. Program Order: If $a$ and $b$ are events in the same process, and $a$ occurs before $b$, then $a\to b$.
2. Message Transmission: If event $a$ is the sending of a message by one process, and event $b$ is the receipt of that same message by another process, then $a\to b$.
3. Transitivity: If $a\to b$ and $b\to c$, then $a\to c$.

If neither $a\to b$ nor $b\to a$, then events $a$ and $b$ are Concurrent ($a\parallel b$). Neither event had any knowledge of the other when it occurred.

Process 1:   (e1) ------------> [Send Msg 1] -------------------> (e2)
                                      \
                                       \
Process 2:                 (e3) ------> [Recv Msg 1] -----------> (e4)

Causality: e1 -> e2, e3 -> e4.
Since [Send Msg 1] -> [Recv Msg 1], by transitivity: e1 -> e4!
Events e1 and e3 are CONCURRENT (e1 || e3): neither knows about the other.

The Lamport Logical Clock Algorithm

Each node maintains a simple integer counter $L$, initialized to 0:

Algorithm: Lamport Logical Clock
---------------------------------------------------------------------------------
1. Local Event:
   Before executing a local action, node i increments its counter:
   L_i = L_i + 1

2. Send Message:
   Attach the current counter to the outbound message payload:
   msg = { data: payload, timestamp: L_i }

3. Receive Message:
   Upon receiving { data, timestamp: L_msg }, node i updates its counter:
   L_i = max(L_i, L_msg) + 1
---------------------------------------------------------------------------------

The Clock Consistency Condition

Lamport clocks satisfy the Clock Consistency Condition:

If event $a$ causally preceded event $b$, event $a$ is guaranteed to have a smaller Lamport timestamp than event $b$.

The Fundamental Flaw of Lamport Clocks

While $a\to b⟹L(a)<L(b)$, the converse is false:

If you inspect two events in a distributed log and observe $L(a)=2$ and $L(b)=5$, you cannot determine whether $a$ caused $b$, or if $a$ and $b$ were completely concurrent! Lamport timestamps establish a total order that respects causality, but they destroy the ability to detect concurrency.

4. Vector Clocks: Capturing Concurrency and Causality

To capture true causal history and distinguish concurrent conflicts from causal succession, Colin Fidge and Friedemann Mattern independently invented Vector Clocks in 1988.

Mathematical Definition

In a system of $N$ nodes, each node maintains a vector of integers of size $N$:

Where $V[k]$ represents the number of events that node $k$ has generated that are causally known to the current node.

The Vector Clock Algorithm

Algorithm: Vector Clock (for Node i in a cluster of size N)
---------------------------------------------------------------------------------
1. Local Event on Node i:
   V_i[i] = V_i[i] + 1

2. Send Message:
   Attach the entire vector to the message:
   msg = { data: payload, vector: V_i }

3. Receive Message with Vector V_msg on Node i:
   - For all k in 1..N:
     V_i[k] = max(V_i[k], V_msg[k])  // Merge knowledge of other nodes
   - V_i[i] = V_i[i] + 1             // Increment own event counter
---------------------------------------------------------------------------------

The Causal Comparison Theorem

Given two vector timestamps $V_A$ and $V_B$:

1. Dominance (Causal Ancestry): $$V_A\le V_B⟺\forall k\in [\mathrm{1..}N],V_A[k]\le V_B[k]$$ $$V_A<V_B⟺(V_A\le V_B)\wedge (V_A\ne V_B)$$

   The Theorem: $$a\to b⟺V(a)<V(b)$$

2. Concurrency Conflict Detection: $$a\parallel b⟺\neg (V(a)\le V(b))\wedge \neg (V(b)\le V(a))$$

If $V_A$ has a component greater than $V_B$, and $V_B$ has a component greater than $V_A$, the system has detected a Concurrent Branch!

Worked Example: Amazon Dynamo Cart Resolution

Initial Cart on Node 1:  V = [1, 0, 0]  (Item: Book)
                         /            \
           (Client A adds Shoes)      (Client B adds Hat)
                  |                            |
       Node 1: V = [2, 0, 0]        Node 2 receives [1, 0, 0]
                                    Node 2: V = [1, 1, 0]
                  \                            /
                   +--------------------------+
                                 |
                     (Nodes sync during read)
         Compare: V_A = [2, 0, 0]  vs  V_B = [1, 1, 0]
         - Node 1 index: V_A[1] (2) > V_B[1] (1)  --> A is ahead on Node 1!
         - Node 2 index: V_A[2] (0) < V_B[2] (1)  --> B is ahead on Node 2!

Because neither vector dominates the other, Dynamo proves that these two updates were concurrent. Rather than randomly destroying one update via Last-Write-Wins, Dynamo preserves both versions as Siblings:

• The client receives both versions: {Book, Shoes} and {Book, Hat}.
• The application merges them into {Book, Shoes, Hat} and writes back the resolved vector: $V=[2,1,0]$.

5. Google Spanner & TrueTime: Bounded Uncertainty

In 2012, Google published the design of Google Spanner, a globally distributed database that provides External Consistency (Strict Serializability) at worldwide scale.

How did Spanner achieve linearizable global transactions without bottlenecking on a single global lock manager? Through TrueTime.

The TrueTime Abstraction

Google accepted that perfect physical clock synchronization is physically impossible due to relativistic physics and network noise. Instead of pretending clock skew is zero, Spanner explicitly represents clock uncertainty:

TT.now() -> [ earliest, latest ]
Where:
- earliest = t_local - epsilon
- latest   = t_local + epsilon
- The true, absolute physical GPS time t_absolute is GUARANTEED to be inside [earliest, latest]!

The Hardware Reality

To guarantee that uncertainty $ϵ$ never exceeds 7 ms, Google installed custom hardware in every single datacenter:

• Every datacenter contains a Time Master cluster equipped with GPS Satellite Receivers.
• Every datacenter also contains Rubidium Atomic Clocks independent of GPS.
• GPS and atomic clocks fail in completely different ways (GPS fails on antenna failure or satellite jamming; atomic clocks drift slowly due to physical decay). If GPS desynchronizes, the atomic clock holds drift under 1 ms for 48 hours.

The TrueTime Commit Wait Rule

To guarantee that transaction $T_2$ initiated after transaction $T_1$ commits receives an absolute timestamp $s_2>s_1$:

1. Transaction T1 initiates commit:
   Coordinator queries TrueTime: TT.now() -> [earliest_1, latest_1]
   T1 selects commit timestamp: s_1 = latest_1

2. THE COMMIT WAIT (Crucial Step):
   The coordinator DOES NOT commit immediately.
   It SLEEPS until: TT.now().earliest > s_1 !

3. Release locks and commit T1.

Physical Timeline:
--------------------------------------------------------------------------------->
[earliest_1 ........... latest_1 (s_1)]
                        |=== COMMIT WAIT SLEEP ===|
                                                   [earliest_2 ........... latest_2]
                                                   ^
                                                   T2 starts here!
Guaranteed: earliest_2 > s_1. Therefore, s_2 > s_1 ALWAYS!

By intentionally pausing the coordinator for $2ϵ$ (~8–14 ms), Google Spanner guarantees that no transaction anywhere on Earth can ever observe an earlier timestamp than a committed transaction!

6. Hybrid Logical Clocks (HLC)

While TrueTime is elegant, most engineering organizations do not own atomic clocks or satellite antennas. Modern distributed databases (CockroachDB, MongoDB, YugabyteDB) use Hybrid Logical Clocks (HLC) (Kulkarni et al., 2014).

An HLC timestamp is a pair:

Where:

• $l$: Physical component (reflects maximum physical NTP time observed so far).
• $c$: Logical counter (increments when multiple events occur within the same physical millisecond).

HLC Invariants

1. Bounded Drift: The physical component $l$ never drifts farther than maximum NTP offset ${ϵ}_{\max }$ (e.g., 250 ms) from the local wall clock. If a node’s physical clock drifts beyond ${ϵ}_{\max }$, the node self-terminates to protect consistency.
2. Strict Monotonicity: $(l,c)$ is strictly monotonically increasing. If physical time steps backward due to an NTP adjustment, $l$ remains steady and $c$ increments, preventing time-reversal anomalies.

7. Real-World Architectural Comparison

8. Operational Failure Modes & Production Troubleshooting

Failure Mode 1: Cloud VM “Hypervisor Steal” Freeze

Symptom: A distributed consensus cluster (etcd/Consul) experiences sudden leader elections and partition dropouts.
Root Cause: The cloud provider (AWS EC2, Google Cloud) performs live migration of the virtual machine host. During migration, the guest VM is paused for 300 ms to 2 seconds. When unpaused, the physical clock instantly jumps forward by 2,000 ms. Software relying on monotonic timeouts detects an immediate heartbeat timeout, triggering false failovers.
Diagnostic: Inspect /proc/stat for steal time or run dmesg | grep -i "clocksource".
Resolution:

1. Ensure Linux uses tsc or kvm-clock as clocksource: cat /sys/devices/system/clocksource/clocksource0/current_clocksource.
2. Configure consensus heartbeats with sufficient headroom above hypervisor jitter.

Failure Mode 2: Vector Clock Payload Bloat

Symptom: Network bandwidth spikes, and database document sizes double.
Root Cause: In dynamic environments where worker nodes join, process a few writes, and terminate (e.g., Kubernetes pods), the Vector Clock array grows indefinitely ($N\to 10,000$). Each document carries thousands of dead node IDs.
Resolution:

1. Implement Dotted Version Vectors or Vector Clock Pruning: truncate vector entries older than a configured threshold (e.g., 7 days) and resolve truncated conflicts via deterministic tie-breakers.

9. Concrete Implementation: Distributed Clock Engine

The following complete TypeScript implementation demonstrates Lamport Logical Clocks, full Vector Clock dominance and concurrency comparison, and a simulated TrueTime Commit-Wait engine:

export interface Message<T> {
  payload: T;
  senderId: string;
  lamportTime?: number;
  vectorTime?: number[];
}

/**
 * 1. Lamport Logical Clock Implementation
 */
export class LamportNode {
  public readonly id: string;
  private time = 0;

  constructor(id: string) {
    this.id = id;
  }

  getTime(): number {
    return this.time;
  }

  localEvent(): number {
    this.time += 1;
    return this.time;
  }

  send<T>(payload: T): Message<T> {
    this.time += 1;
    return {
      payload,
      senderId: this.id,
      lamportTime: this.time,
    };
  }

  receive<T>(msg: Message<T>): void {
    if (msg.lamportTime === undefined) throw new Error('Missing Lamport timestamp');
    // L_i = max(L_i, L_msg) + 1
    this.time = Math.max(this.time, msg.lamportTime) + 1;
  }
}

/**
 * 2. Vector Clock Implementation with Causal Comparison
 */
export class VectorClockNode {
  public readonly id: string;
  public readonly nodeIndex: number;
  public vector: number[];

  constructor(id: string, nodeIndex: number, clusterSize: number) {
    this.id = id;
    this.nodeIndex = nodeIndex;
    this.vector = new Array(clusterSize).fill(0);
  }

  localEvent(): number[] {
    this.vector[this.nodeIndex] += 1;
    return [...this.vector];
  }

  send<T>(payload: T): Message<T> {
    this.vector[this.nodeIndex] += 1;
    return {
      payload,
      senderId: this.id,
      vectorTime: [...this.vector],
    };
  }

  receive<T>(msg: Message<T>): void {
    if (!msg.vectorTime) throw new Error('Missing Vector timestamp');
    // Merge component-wise max
    for (let k = 0; k < this.vector.length; k++) {
      this.vector[k] = Math.max(this.vector[k], msg.vectorTime[k]);
    }
    this.vector[this.nodeIndex] += 1;
  }

  /**
   * Compares two vector clocks.
   * Returns:
   *  -1 if a -> b (a causally preceded b)
   *   1 if b -> a (b causally preceded a)
   *   0 if a == b (identical)
   *  null if a || b (CONCURRENT conflict!)
   */
  static compare(a: number[], b: number[]): -1 | 1 | 0 | null {
    let aGreater = false;
    let bGreater = false;

    for (let i = 0; i < a.length; i++) {
      if (a[i] > b[i]) aGreater = true;
      if (b[i] > a[i]) bGreater = true;
    }

    if (!aGreater && !bGreater) return 0; // Exactly equal
    if (aGreater && !bGreater) return 1;  // a dominates b (b -> a)
    if (!aGreater && bGreater) return -1; // b dominates a (a -> b)
    return null; // Both have greater components: CONCURRENT! (a || b)
  }
}

/**
 * 3. Simulated Google TrueTime API with Commit-Wait
 */
export interface TrueTimeInterval {
  earliest: number;
  latest: number;
}

export class TrueTimeProvider {
  private readonly epsilonMs: number;

  constructor(epsilonMs = 5) {
    this.epsilonMs = epsilonMs;
  }

  now(): TrueTimeInterval {
    const physicalNow = Date.now();
    return {
      earliest: physicalNow - this.epsilonMs,
      latest: physicalNow + this.epsilonMs,
    };
  }

  /**
   * Enforces the Spanner Commit Wait rule:
   * Chooses commit timestamp s = latest, then pauses execution until earliest > s.
   */
  async commitWait(): Promise<number> {
    const tt = this.now();
    const commitTimestamp = tt.latest;

    // Sleep until TrueTime earliest > commitTimestamp
    while (this.now().earliest <= commitTimestamp) {
      await new Promise((resolve) => setTimeout(resolve, 1));
    }

    return commitTimestamp;
  }
}

function runDemo(): void {
  console.log('=== Logical & Physical Clocks: Lamport vs. Vector Clocks ===');
  const nodeA = new VectorClockNode('node-A', 0, 3);
  const nodeB = new VectorClockNode('node-B', 1, 3);
  nodeA.localEvent();
  console.log('Node A local event vector: [' + nodeA.vector.join(', ') + ']');
  const msg = nodeA.send('Hello from A');
  nodeB.receive(msg);
  console.log('Node B received message vector: [' + nodeB.vector.join(', ') + ']');
}
runDemo();

10. Summary & Decision Guide

+-----------------------------------------------------------------------------------+
|                        Distributed Ordering Selection Guide                       |
+--------------------------+-----------------------+--------------------------------+
| Protocol / Model         | Guarantees            | Best Use Case                  |
+--------------------------+-----------------------+--------------------------------+
| Monotonic Clocks         | Single-node duration  | Latency metrics, timeouts,     |
| (nanoTime / performance) | guaranteed monotonic  | circuit breaker state delays   |
+--------------------------+-----------------------+--------------------------------+
| Lamport Logical Clocks   | Partial causality     | Consensus term elections,      |
|                          | preservation          | distributed lock sequences     |
+--------------------------+-----------------------+--------------------------------+
| Vector Clocks            | Exact causality and   | Collaborative document edits,  |
|                          | concurrency detection | multi-master shopping carts    |
+--------------------------+-----------------------+--------------------------------+
| TrueTime / Spanner       | Strict serializability| Global multi-region banking,   |
| (Commit Wait)            | worldwide             | mission-critical SQL ledgers   |
+--------------------------+-----------------------+--------------------------------+
| Hybrid Logical Clocks    | Monotonic timestamps  | CockroachDB, YugabyteDB,       |
| (HLC)                    | bounded to NTP        | distributed transaction MVCC   |
+--------------------------+-----------------------+--------------------------------+

The Distributed Engineer’s Rules of Thumb

1. Never Trust Wall Clocks for Correctness: Assume server clocks are always skewed by tens of milliseconds. Never use timestamps to determine transaction serializability unless backed by TrueTime or HLCs.
2. If You Must Use Last-Write-Wins, Accept Data Loss: LWW is an optimistic heuristic, not a correctness guarantee. If business requirements mandate zero data loss, use Vector Clocks or strong consensus (Raft).
3. Always Check Monotonicity for Local Latencies: Never calculate duration via end - start with Date.now(). Use monotonic primitives (performance.now()) to prevent clock-slewing skew.

11. Guided Design Exercises

Exercise 1: Vector Clock Causal History & Sibling Conflict Resolution

A 3-node Dynamo-style key-value store ($N=3$, nodes [N1, N2, N3]) initializes all vector clocks to $[0,0,0]$. Trace the exact vector clock $[V_1,V_2,V_3]$ attached to each version of key cart:42 across the following sequence of operations:

1. N1 executes local write $W_1$ ({"item": "Laptop"}).
2. N1 replicates $W_1$ to N2 (which applies receive) and to N3 (which applies receive).
3. While a network partition temporarily isolates N2 and N3, a mobile client writes $W_2$ ({"item": "Laptop, Mouse"}) on N2, and a desktop client concurrently writes $W_3$ ({"item": "Laptop, Keyboard"}) on N3.
4. Prove mathematically using the Vector Clock dominance criteria why $W_2\parallel W_3$ (concurrent siblings), whereas $W_1\to W_2$ and $W_1\to W_3$. Finally, compute the reconciled vector clock when N1 reads both siblings, merges the cart to {"item": "Laptop, Mouse, Keyboard"}, and writes $W_4$.

Exercise 2: Google Spanner TrueTime $2ϵ$ Commit-Wait Throughput & Latency Math

A Google Spanner Paxos leader executes serializable read-write transactions holding an exclusive lock on a hot row account_id = 99.

1. If TrueTime uncertainty is $ϵ=4\text{ ms}$ (so TT.now() returns $[t-4\text{ ms},t+4\text{ ms}]$) and consensus replication takes $R=2\text{ ms}$ (overlapped with the Commit Wait), calculate the mandatory Commit-Wait duration $2ϵ$ and the maximum sequential single-row update throughput (${\text{TPS}}_{\max }$) achievable on that single hot row.
2. Suppose the local GPS receiver fails and the datacenter falls back to its Rubidium atomic clock, which drifts at $1\text{ ppm}$ ($1\mu \text{s/sec}$), while quartz server clocks drift at $200\text{ ppm}$ between $30\text{-second}$ time-master polls. Calculate how $ϵ$ grows during a $60\text{-second}$ time-master poll outage and quantify the resulting impact on Spanner write latency.

Exercise 3: Hybrid Logical Clocks (HLC) vs. TrueTime Read Restarts in CockroachDB

1. Trace how a Hybrid Logical Clock (l, c) on Node B (PT_B = 1000) updates its state when it receives an RPC carrying message timestamp (l_m = 1012, c_m = 4) from Node A (whose NTP clock is $12\text{ ms}$ ahead), followed immediately by a second local event on Node B within the same physical millisecond (PT_B = 1000).
2. Explain why CockroachDB (using commodity NTP + HLC with max_offset = 250ms instead of GPS/atomic TrueTime hardware) does not perform a $2ϵ=500\text{ ms}$ Commit Wait on writes, and instead shifts the uncertainty cost onto Read Restarts whenever a reader at timestamp $t_r$ encounters a value with timestamp $t_w\in (t_r,t_r+{ϵ}_{\max }]$.

12. Authoritative References & Version Context

• Leslie Lamport (1978), “Time, Clocks, and the Ordering of Events in a Distributed System”, Communications of the ACM 21(7), pp. 558–565 — doi.org/10.1145/359545.359563
• Colin J. Fidge (1988), “Timestamps in Message-Passing Systems That Preserve the Partial Ordering”, Australian Computer Science Communications — semanticscholar.org
• James C. Corbett et al. (2012), “Spanner: Google’s Globally-Distributed Database”, ACM TOCS / OSDI ‘12 (TrueTime API and Commit Wait) — usenix.org/conference/osdi12/technical-sessions/presentation/corbett
• Version Context: Clock synchronization mechanisms reflect PTP (IEEE 1588), NTP (RFC 5905), Hybrid Logical Clocks (HLC) in CockroachDB 23.x, and TrueTime commit wait.