Fix relay E2EE reconnect races (#973)

* Fix relay E2EE reconnect races

* fix(relay): use WebSocket protocol ping for daemon control keepalive

The daemon's control socket pinged the relay every 10s with a JSON
`{type:"ping"}` message. Every JSON message wakes a hibernated Cloudflare
Durable Object via `webSocketMessage`, so an idle daemon was billing
relay DO CPU 24/7 just to echo `pong`.

Switch to `socket.ping()` (RFC 6455 protocol ping frames). Cloudflare's
runtime auto-responds at the edge without waking hibernated DOs, so an
idle daemon-to-relay connection now costs zero DO CPU.

Liveness still works: the `ws` library emits a `pong` event on the
daemon when the edge replies, which refreshes `controlLastSeenAt`.
`markControlReady` stays tied to DO-originated messages (sync,
connected, disconnected), so we still detect a broken DO at connect.

The DO keeps its JSON ping handler tagged `COMPAT(relay-json-ping)` so
old daemons (< v0.1.76) continue to work.

* test(relay): assert protocol-ping pongs arrive in e2e idle test

Adds a `relay_control_pong_received` debug log to the daemon's pong
handler, then asserts in the existing idle-relay e2e test that:

  - at least one pong arrived during the 12s idle wait (proving the
    Cloudflare edge auto-responds to RFC 6455 protocol pings)
  - no stale terminations fired (proving the keepalive path is healthy)

If the daemon ever regresses to JSON-level keepalive pings (which wake
the hibernated relay Durable Object and incur CPU billing), this test
will catch it: the `pong` event would never fire because the DO's reply
would arrive via `message`, not as a protocol pong frame.

The test's capturing logger is bumped to `debug` level so the assertion
can read the new debug log; production daemons stay at `info`.

Confirmed locally against wrangler-hosted relay: 2 pongs / 0 stale per
12s window.

* test(relay): catch dual-ping regression via DO-side legacy ping log

The existing idle-test assertion catches "revert to JSON ping only" via
the missing pong event, but misses the dual-ping regression where
someone adds `socket.send(JSON.stringify({type:"ping"}))` alongside
`socket.ping()`. Both paths would work, the pong assertion still
passes, and DO CPU billing silently returns.

Add a console.log in the DO's handleControlKeepalive (which only runs
when a JSON ping reaches the DO), then assert in the idle e2e test
that no such log appeared during the 12s idle window.

Verified the assertion catches the regression: temporarily injected a
`socket.send(JSON.stringify({type:"ping"}))` next to socket.ping(),
test failed with `expected 1 to be 0` and the relay logged
`legacy_json_ping_received`. Removed the injection.

Log noise during the COMPAT window doubles as useful telemetry: it
counts how many old daemons are still pinging, which informs the
removal date for the COMPAT shim.
This commit is contained in:
Mohamed Boudra
2026-05-13 13:44:11 +08:00
committed by GitHub
parent 15cab6014c
commit d24087c10a
6 changed files with 301 additions and 11 deletions
+7
View File
@@ -192,11 +192,18 @@ export class RelayDurableObject {
}
}
// COMPAT(relay-json-ping): Old daemons (< v0.1.76) send JSON {type:"ping"} on the control
// socket and rely on a JSON {type:"pong"} reply to keep controlLastSeenAt fresh. New daemons
// use WebSocket protocol pings (auto-answered at the edge, DO stays hibernated). Remove this
// handler once the supported-daemon floor is >= v0.1.76 (target: 2026-11-13).
private handleControlKeepalive(ws: WebSocket, message: string): void {
try {
const parsed: unknown = JSON.parse(message);
const parsedRecord = isRecord(parsed) ? parsed : null;
if (parsedRecord?.type !== "ping") return;
// Logged so the daemon-side e2e idle test can assert no JSON ping reached the DO
// (which would indicate a regression to app-level pings that wake the DO).
console.log("[Relay DO] legacy_json_ping_received");
try {
ws.send(JSON.stringify({ type: "pong", ts: Date.now() }));
} catch {