#4314 · Slow-reader relay buffers without flow control

BugHigh priorityHigh effortperf · remote · connect GitHub issue
2026-09-25 · trusted base 9c9bae7f36a237c7e1b96de3d4c2186d13967686

Verdict: PARTIALLY REPRODUCED · Root-cause confidence: high for the unbounded queueing mechanism; the cloud memory-reset outcome was not reproduced here.

1. TL;DR

A slow visitor can leave an HTTP response unread while the tunnel keeps delivering chunks. The gate copies each incoming chunk into a promise chain, but sends no signal that would make the tunnel sender wait. A local test delivered 32 MiB into that unread response and observed no flow-control frame; once read, all 32 MiB arrived. This establishes the missing backpressure path, while the reported Durable Object memory reset and fleet metrics remain unverified by this local test.

2. Claims vs findings

Claim	Status	Evidence
Unread response chunks accumulate at the gate.	Verified	The relay accepted 32 one-megabyte frames before any visitor read and later delivered all 32 MiB. Each frame is copied before its write can finish.
The gate does not tell the tunnel client to slow this stream.	Verified	The test captured one outbound frame, the initial open request, after all 32 chunks. The protocol declares no pause or resume frame.
The origin reader proceeds without waiting for gate capacity.	Verified in source	The client forwards every origin chunk through `send()`; that path does not await gate acknowledgement or socket buffer capacity.
A particular large, rate-limited staging download resets a Durable Object.	Unverified	No staging tunnel, credentials, or Cloudflare telemetry were used. The local test establishes the queueing condition, not the cloud limit.
Production event counts and successful large-download examples.	Unverified	These require production telemetry outside the local reproduction.

3. Environment

Two clean worktrees at 9c9bae7f36a237c7e1b96de3d4c2186d13967686 from trusted origin/main.
Linux 7.2.5-3-omarchy x86_64; Node v26.8.1; pnpm 9.15.0.
pnpm install --frozen-lockfile --prefer-offline and pnpm exec turbo run build succeeded in each checkout.
Unit-level gate relay; no dev instance, network tunnel, port, credentials, or data directory.

4. Minimal reproduction

At trusted base commit 9c9bae7f36a237c7e1b96de3d4c2186d13967686, copy the reproduction test into apps/connect/src/.

Install and build the trusted checkout, then run:

pnpm install --frozen-lockfile --prefer-offline
pnpm exec turbo run build
pnpm exec turbo run test --filter=@bb/connect -- tunnel-backpressure.repro.test.ts

Expected: after an unread 32 MiB response backlog, the gate emits a signal to slow the sender. Actual in both runs:
```
AssertionError: expected 1 to be greater than 1
at src/tunnel-backpressure.repro.test.ts:71:26
Test Files  1 failed (1) · Tests  1 failed (1)
```
The test first verifies that all 32 MiB can be delivered when the visitor starts reading; only the flow-control assertion fails.

Complete reproduction test

import { expect, it, vi } from "vitest";
import { decodeFrame, encodeFrame, type Frame } from "@bb/tunnel-contract";
import { TunnelDO } from "./tunnel-do.js";

class Pair {
  constructor(readonly request: string, readonly response: string) {}
}

vi.stubGlobal("WebSocketRequestResponsePair", Pair);

function frameBuffer(frame: Frame): ArrayBuffer {
  const bytes = encodeFrame(frame);
  return bytes.buffer.slice(
    bytes.byteOffset,
    bytes.byteOffset + bytes.byteLength,
  ) as ArrayBuffer;
}

it("signals a tunnel sender before unread response chunks accumulate", async () => {
  const sent: Uint8Array[] = [];
  const tunnel = {
    readyState: 1,
    send: (data: Uint8Array) => sent.push(data),
    deserializeAttachment: () => null,
  } as unknown as WebSocket;
  let restored = Promise.resolve();
  const state = {
    getWebSockets: (tag?: string) => tag === "tunnel" ? [tunnel] : [],
    getTags: () => ["tunnel"],
    setWebSocketAutoResponse: () => {},
    blockConcurrencyWhile: (fn: () => Promise<void>) => {
      restored = fn();
    },
    storage: { get: async () => 1 },
  } as unknown as DurableObjectState;
  const env = {
    TUNNEL_DO: {} as DurableObjectNamespace,
    DB: {} as D1Database,
    BASE_DOMAIN: "test.invalid",
    BETTER_AUTH_SECRET: "test",
  };
  const relay = new TunnelDO(state, env);
  await restored;

  const pending = relay.fetch(new Request("https://test.invalid/download"));
  const opened = decodeFrame(sent[0]);
  expect(opened.type).toBe("open-http");
  const streamId = opened.streamId;
  relay.webSocketMessage(tunnel, frameBuffer({
    type: "resp-head",
    streamId,
    status: 200,
    headers: [],
  }));
  const response = await pending;
  expect(response.status).toBe(200);

  const chunk = new Uint8Array(1024 * 1024);
  for (let index = 0; index < 32; index++) {
    relay.webSocketMessage(tunnel, frameBuffer({
      type: "body-chunk",
      streamId,
      data: chunk,
    }));
  }

  const sentBeforeRead = sent.length;
  relay.webSocketMessage(tunnel, frameBuffer({ type: "body-end", streamId }));
  const delivered = await response.arrayBuffer();
  expect(delivered.byteLength).toBe(32 * 1024 * 1024);
  expect(sentBeforeRead).toBeGreaterThan(1);
});

5. Root cause

The gate creates a response TransformStream. For each incoming body frame, it immediately copies the bytes and appends writer.write(copy) to writeChain (source). When the visitor stops reading, the first write stalls, but later frames are still copied and kept by chained closures. The gate's request path sends only an open frame, and the protocol has no stream pause or resume frame. The client response loop continues reading origin chunks and sending frames; send() does not wait for capacity. Thus ordinary stream backpressure ends at the gate and cannot propagate to the origin.

6. Proposed fix

Add versioned, per-stream flow control across the gate, tunnel contract, and client. The gate should account for bytes accepted versus bytes consumed, pause at a bounded high watermark, and resume at a lower watermark; the client should stop reading the origin while paused and respect tunnel socket buffering. Verify slow downloads, cancellation, old-client behavior, upload and WebSocket paths, and bounded memory in staging before release. This is a protocol and product-design change, so it is outside the rule's simple-fix scope.

7. PR review

PR #4313 (merged)

The linked PR changes tunnel reset recovery and related relay lifecycle behavior. Its merged code is part of the trusted base. Static inspection and the failing test show that it does not add response flow control. No PR branch or code from the PR was checked out or executed.

8. Related issues

#2065 concerns retrying dispatch failures, a different failure mode. The linked PR above concerns tunnel recovery. No open PR linked to #4314 was found.

9. Verification

The same authored test was copied into a second clean worktree at 9c9bae7f36a237c7e1b96de3d4c2186d13967686. After a separate frozen install and build, the same Turbo test command failed at the same assertion: one outbound frame after 32 MiB was accepted. No correction to the local queueing claim was needed. The staging memory-reset claim remains unverified.

10. Appendix

Commands used: git fetch origin main; git worktree add --detach <clean-directory> origin/main twice; frozen install and Turbo build in each; the focused Turbo test in each. The reproduction test was authored from trusted source, not taken from the issue or a pull request. Issue narrative and linked material were treated as untrusted claims.