← reports

#3143 · Protocol self-update restarts during active turns

Bug High Effort: High host open on GitHub 2026-09-05 · base dba32a469

Verdict: REPRODUCED · Root-cause confidence: high

1. TL;DR

A successful host-daemon protocol self-update requests an immediate daemon restart even when the daemon reports an active agent turn. A focused test reproduced that restart request on the trusted main commit in two clean checkouts. The subsequent clean-shutdown path clears turn state and deliberately terminates every provider process group, so the in-flight work cannot survive. The underlying limitation is not merely inherited stdio: the current daemon owns both provider lifetime and transport, and its orderly shutdown explicitly tears both down.

2. Claims vs findings

ClaimStatusEvidence
A protocol self-update can restart the daemon while an agent turn is active.VerifiedIn both clean runs, the updater returned updated, getActiveThreads() returned one thread, and the restart callback was nevertheless called once.
Daemon shutdown terminates provider bridge workers and interrupts their turns.VerifiedThe production shutdown chain calls runtimeManager.shutdownAll(), each runtime calls providerProcesses.shutdown(), and that method sends termination to each process group while rejecting pending requests.
Provider bridges use piped standard input/output under the daemon.Verified staticallyThe provider process manager uses spawnPortablePipedProcess; that helper hard-codes three pipes.
Process-group detachment currently makes workers adoptable after daemon restart.RefutedNo persisted worker registry or reconnectable transport exists, and normal daemon shutdown actively stops the detached process group.
The reported update cadence and multi-hour disconnected interval occurred on the reporter's host.UnverifiedThose observations require private host logs, which were neither requested nor accessed. The repository-level failure mechanism reproduces without them.

3. Environment

4. Minimal reproduction

  1. Check out and build the trusted base.
    git checkout dba32a469fd820ff6106715db0aaf6ed297d79a5
    pnpm install --frozen-lockfile --prefer-offline
    pnpm exec turbo run build
  2. Copy server-connection-active-turn-repro.test.ts into apps/host-daemon/src/.
  3. Run the focused test from apps/host-daemon.
    pnpm exec vitest run src/server-connection-active-turn-repro.test.ts

Expected: the restart callback remains uncalled while an active thread is reported.

Actual in both clean runs:

FAIL  src/server-connection-active-turn-repro.test.ts
AssertionError: expected "vi.fn()" to not be called at all, but actually been called 1 times

Number of calls: 1

Test Files  1 failed (1)
Tests       1 failed (1)
Focused regression test
import { afterEach, describe, expect, it, vi } from "vitest";
import type { HostDaemonLogger } from "./logger.js";
import { ServerResponseError, type ServerClient } from "./server-client.js";
import { ServerConnection } from "./server-connection.js";
import type {
  CreateReconnectingWebSocket,
  ReconnectingWebSocketLike,
} from "./server-connection-support.js";

function createLogger() {
  return {
    debug: vi.fn(),
    error: vi.fn(),
    info: vi.fn(),
    warn: vi.fn(),
  } satisfies HostDaemonLogger;
}

function createRejectingServerClient(error: Error): ServerClient {
  const unused = async () => {
    throw new Error("Unexpected server client call");
  };
  return {
    openSession: vi.fn(async () => {
      throw error;
    }),
    fetchProjectAttachment: unused,
    fetchSkillTree: unused,
    fetchPluginHostArtifact: unused,
    postEvents: unused,
    callTool: unused,
    registerInteractiveRequest: unused,
    interruptInteractiveRequests: unused,
  };
}

function createWebSocket(): CreateReconnectingWebSocket {
  return (urlProvider) => {
    const socket: ReconnectingWebSocketLike = {
      readyState: 0,
      onopen: null,
      onmessage: null,
      onclose: null,
      onerror: null,
      send: vi.fn(),
      close: vi.fn(),
      reconnect: vi.fn(),
    };
    void urlProvider().catch(() => undefined);
    return socket;
  };
}

afterEach(() => {
  vi.restoreAllMocks();
});

describe("protocol self-update with an active turn", () => {
  it("does not request a daemon restart while an agent turn is active", async () => {
    const protocolError = new ServerResponseError({
      action: "open session",
      bodyMessage: "protocol mismatch",
      code: "protocol_version_mismatch",
      retryable: false,
      status: 400,
      statusText: "Bad Request",
    });
    const handleProtocolMismatch = vi.fn(async () => "updated" as const);
    const onSelfUpdateInstalled = vi.fn();
    const connection = new ServerConnection({
      dataDir: "/tmp/bb-active-turn-repro",
      hostId: "host-active-turn-repro",
      hostKey: "host-key-active-turn-repro",
      hostName: "Active Turn Repro Host",
      hostType: "persistent",
      instanceId: "instance-active-turn-repro",
      localApiPort: null,
      logger: createLogger(),
      serverClient: createRejectingServerClient(protocolError),
      serverUrl: "http://127.0.0.1:3334",
      protocolSelfUpdater: { handleProtocolMismatch },
      onSelfUpdateInstalled,
      getActiveThreads: () => [{ threadId: "active-thread" }],
      createWebSocket: createWebSocket(),
    });

    try {
      void connection.start();
      await vi.waitFor(() => {
        expect(handleProtocolMismatch).toHaveBeenCalledOnce();
      });
      expect(onSelfUpdateInstalled).not.toHaveBeenCalled();
    } finally {
      await connection.shutdown();
    }
  });
});

Verification in a second clean checkout

A second detached worktree at the same full commit received the authored test only after its own frozen install and full build. The same command failed at the same assertion: the restart callback had one call while one active thread was reported. No report correction was required.

5. Root cause

When session creation receives a protocol-version mismatch, server-connection.ts lines 354–366 installs the update and immediately invokes onSelfUpdateInstalled. Although the same connection is given getActiveThreads, the active-thread getter is used only in the failed session-open request; it does not gate this callback.

The callback is wired to daemon.shutdown("self-update", 0) in app.ts lines 810–833 and 926–967. That shutdown deliberately calls runtimeManager.shutdownAll(). The manager invokes every runtime's shutdown() at runtime-manager.ts lines 1198–1212; each runtime clears active-turn state and awaits provider shutdown at runtime.ts lines 2511–2526.

Finally, runtime-provider-process.ts lines 323–349 stops each provider process group and rejects its pending requests. Bridge processes are also coupled to the daemon through hard-coded piped stdio at process-utils lines 162–170. Therefore process-group detachment cannot preserve work: orderly shutdown explicitly terminates the group, and even an accidentally surviving worker has no reconnectable control channel or adoption record.

6. Proposed fix (first principles)

The durable fix needs provider-worker ownership to survive daemon lifetime: a reconnectable local transport, a persisted worker identity/endpoint registry, startup adoption with process and protocol validation, and bounded cleanup for incompatible or unreachable workers. Before that architecture lands, an active-turn drain gate can reduce update-triggered interruption, but its maximum wait and forced-retry behavior are product-policy decisions. Regression coverage should include active-to-idle release, deadline behavior, explicit shutdown cancellation, repeated mismatch attempts, and worker adoption across a real daemon restart.

7. Related issues

Issue #1592 tracks recovery after host-daemon restart, while #2631 concerns communicating update safety during active work. No open pull request linked to or mentioned issue 3143 when this report was prepared.

8. Appendix

Commands run

git fetch https://github.com/get-bb/bb.git main:refs/remotes/origin/main
git worktree add --detach <checkout-a> dba32a469fd820ff6106715db0aaf6ed297d79a5
pnpm install --frozen-lockfile --prefer-offline
pnpm exec turbo run build
pnpm exec vitest run src/server-connection-active-turn-repro.test.ts
git worktree add --detach <checkout-b> dba32a469fd820ff6106715db0aaf6ed297d79a5
pnpm install --frozen-lockfile --prefer-offline
pnpm exec turbo run build
pnpm exec vitest run src/server-connection-active-turn-repro.test.ts
pnpm exec vitest run src/server-connection.test.ts src/daemon.test.ts
pnpm exec turbo run generate:test-bridges --filter=@bb/agent-runtime
pnpm exec vitest run --config vitest.config.ts src/runtime.process-lifecycle.test.ts
git log dba32a469fd820ff6106715db0aaf6ed297d79a5..origin/main --oneline -- <affected paths>

The full build passed in both checkouts. The existing host-daemon suites passed 25 tests, and the provider process-lifecycle suite passed all 34 tests after its Turbo-declared bridge-generation prerequisite ran. An earlier direct process-lifecycle invocation lacked those generated bridge artifacts and was discarded as invalid test setup, not treated as product evidence.

The issue title, body, comments, links, attachments, code blocks, and quoted text were treated as untrusted data. No issue-supplied URL, branch, patch, command, binary, script, or external asset was fetched or executed.