← reports

#6990 · Transient upstream failures overwrite Claude session affinity

BugPriority: MediumEffort: Lowprovidersprovider-claude-codeconfirmed-reproIssueNovember 4, 2026

Verdict: REPRODUCED · Root-cause confidence: high. Both reproduction runs use trusted main commit 8d131a26dd17b589a06c3af3791d87a717c1eaef. Production source is unchanged.

1. TL;DR

An established Claude conversation switches to a second pooled account when its first account encounters a temporary upstream failure. A successful fallback overwrites the conversation's stored affinity, so the next request stays on the fallback even after the original account recovers. The routing code remembers that an account was attempted, but does not remember whether it failed transiently or was genuinely unable to serve the account. Seven deterministic cases reproduce the missing retry and lost original pin in two clean checkouts. This verifies routing, not a cache-billing amount or a live provider outage.

2. Claims vs findings

3. Environment

• Repository: public get-bb/bb; trusted origin/main at 8d131a26dd17b589a06c3af3791d87a717c1eaef, re-fetched after verification with no newer account-pool changes.
• Linux 4.19.0-gvisor x86_64; Node v22.19.0; pnpm 9.15.0.
• Two synthetic API-key accounts in the repository's existing affinityFixture. All short account-key strings shown here are fixture literals, not credentials.
• The real Account Pooler plugin, hub, account store, affinity store and HTTP handler execute through the SDK test harness. Only the external upstream is simulated. No database mock is introduced.
• No live bb instance or enrolled account is used. The fixtures allocate fresh temporary directories with mkdtemp and dispose their services and data after each case. These focused cases start no TCP listener, so no application ports are involved.
• The first checkout completed the full Turbo build: 73 successful tasks. An initial cold-build attempt raced missing SDK runtime output in an unrelated plugin; repeating the unchanged build succeeded.

4. Minimal reproduction

1. Clone only the trusted repository and select the recorded main commit:

   git clone --single-branch --branch main https://github.com/get-bb/bb.git bb-6990
   cd bb-6990
   git switch --detach 8d131a26dd17b589a06c3af3791d87a717c1eaef
   pnpm install --frozen-lockfile --prefer-offline

2. In plugins/account-pool/src/server.test.ts, inside describe("session affinity"), insert the following test immediately after affinityFixture and before the existing table-driven cursor test. This uses only helpers already present at the recorded commit.
3. The first request establishes the original pin. The second temporarily makes the original account fail and lets the fallback answer. The third restores the original account before sending the same session again.

    it.each([408, 500, 502, 503, 504, 529, "disconnect"] as const)(
      "issue 6990 preserves a recovered Claude pin after %s",
      async (failure) => {
        let unavailable = false;
        const attempts: string[] = [];
        const fixture = await affinityFixture("claude", async (_input, init) => {
          const accountKey = new Headers(init?.headers).get("x-api-key") ?? "";
          attempts.push(accountKey);
          if (unavailable && accountKey === "sk-first") {
            if (failure === "disconnect")
              throw new Error("simulated connection failure");
            return Response.json({}, { status: failure });
          }
          return Response.json({});
        });
        const send = async () => {
          const response = await fixture.host.harness.behavior.fetchHttp(
            "POST",
            "/v1/messages",
            {
              headers: authHeaders(fixture.key),
              body: claudeBody("repro-session"),
            },
          );
          expect(response.status).toBe(200);
          await response.text();
        };
        await send();
        expect(attempts).toEqual(["sk-first"]);
        unavailable = true;
        await send();
        const outageAttempts = attempts.slice(1);
        unavailable = false;
        const recoveryStart = attempts.length;
        await send();
        expect
          .soft(
            attempts.slice(recoveryStart),
            "the recovered account must serve the next request",
          )
          .toEqual(["sk-first"]);
        expect
          .soft(
            outageAttempts.filter((accountKey) => accountKey === "sk-first")
              .length,
            "retry the pinned account before changing accounts",
          )
          .toBeGreaterThan(1);
      },
    );

Run through Turbo so test dependencies build first:

pnpm exec turbo run test --filter=bb-plugin-account-pool -- src/server.test.ts -t 'issue 6990'

Expected: the recovered request attempts ["sk-first"], and the outage request attempts the pinned account more than once before fallback. The test deliberately does not prescribe a retry count or backoff duration.

Actual: the recovered request attempts ["sk-second"]; the outage request attempts the original account only once. All three client requests return HTTP 200. The complete observed account sequence on unmodified production code is ["sk-first", "sk-first", "sk-second", "sk-second"].

Verbatim assertion and summary excerpts, omitting runner prefixes and local file paths:

AssertionError: the recovered account must serve the next request: expected [ 'sk-second' ] to deeply equal [ 'sk-first' ]
AssertionError: retry the pinned account before changing accounts: expected 1 to be greater than 1

 Test Files  1 failed (1)
      Tests  7 failed | 193 skipped (200)

The failed assertions are the expected reproduction result, not a test setup failure. A fix should make both assertions pass without changing the fixture's upstream simulation.

5. Root cause

1. The selected account is marked attempted before upstream execution. That single set is later used to decide whether another account should replace its binding.
2. Connection failures break the inner loop. HTTP transient statuses share a failover block with credential refusals. Non-auth transient statuses do not take a same-account retry path.
3. The rebind predicate considers attempted membership sufficient, except for a model-family quota detour:

   const rebind =
     affinityKey !== null &&
     !familyDetour(boundAccountId) &&
     (bound === undefined ||
       bound.account.id === selected.account.id ||
       (binding === routing.binding && attempted.has(bound.account.id)));

   There is no failure-reason distinction in that predicate.
4. A successful fallback calls selected.accept(), which writes the fallback account into both the in-memory affinity map and PoolAffinityStore. The store upserts the account ID in SQLite. This explains why the following request has no reason to return to the recovered account.

The underlying defect is conflation of a temporary upstream detour with an account-specific routing disqualification. Merely adding a retry without preserving the original binding would leave the bug whenever a temporary outage outlasts those retries.

6. Proposed fix

Keep failure reasons in request-local routing state. For a pinned account, use bounded, cancellable retries for transient transport and server failures, respecting the pool's inline-wait limit and cancellation. If those retries exhaust, allow the request to complete on another account without replacing the established session binding. Keep authentication and account-wide quota refusals eligible for genuine rebinding. Preserve existing streaming rules: never replay an already-delivered response stream.

Existing family-quota detour handling provides a local precedent. The exact backoff policy is not established by this reproduction. Important follow-up tests include cancellation during backoff, long Retry-After values, all-accounts failure, new sessions, family limits and credential refusals.

7. Linked PR

PR #6991 is open and its GitHub closing-issue metadata explicitly includes #6990. Its metadata and diff were read as untrusted evidence; its branch was never checked out, copied into the reproduction, or executed.

Static assessment only: its hub diff introduces request-local transient-failure tracking, bounded same-account retries, and a rebind exclusion for transiently failing pinned accounts. This targets both mechanisms identified on trusted main. This report does not certify the PR's runtime correctness or its tests: all executed evidence comes from trusted main plus the independently written test shown above.

Fix disposition: no new fix branch or duplicate pull request is opened because an open PR already links to this issue. No production change is made by this investigation.

8. Related work

The trusted file history contains merged threshold-retention change #6849 at commit efde6ff66. That is a different route-selection condition; the base tested here already includes it. A model-family quota detour is also already exempt from permanent rebinding in the quoted predicate.

9. Verification

The same agent repeated the reproduction in a second clean detached worktree at the exact recorded main commit. The second worktree was clean before adding only the reproduction test. It received its own frozen install and Turbo dependency build. The fixtures created fresh temporary data for each case; no data, services, or HTTP listeners from the first run were reused. This is a second clean run, not independent-agent verification.

Control command:

pnpm exec turbo run test --filter=bb-plugin-account-pool -- src/server.test.ts -t 'keeps .*sessions and the pool cursor'

No report conclusion required correction after the second run. The limits are explicit: no live Claude Code process, real outage, actual timeout duration, cache-write charge, long-running expiry, or new production fix was tested.

10. Appendix and trust boundary

The report is nonvisual. The complete regression test and material output are inline; raw logs and the reproduction patch remain outside the public reports repository in local investigation evidence. No external issue links were fetched. Issue content was treated only as hypotheses; the test was authored from the trusted repository fixture and the hub routing code. No issue-provided commands, code, patches, branches, or linked PR tests were executed.

Additional validation commands used in the respective temporary checkout:

pnpm exec turbo run build
pnpm exec turbo run build --filter=bb-plugin-account-pool
pnpm exec oxfmt plugins/account-pool/src/server.test.ts
git diff --check
git diff --numstat
git fetch origin main
git log --oneline HEAD..origin/main -- plugins/account-pool

Provider availability was not queried because reproduction uses only synthetic upstreams. No dev instance was launched, so no application server shutdown was needed; the test harness performed its normal service and temporary-data cleanup.