#3716 · Concurrent dispatch loses competing sends

Bug · High priority · Medium effort · threads · 2026-09-15
Base: 62fe3bc46bd1d935b71482b62a1e0887ffb83487 · Issue

Verdict: REPRODUCED · Root-cause confidence: high

1. TL;DR

Concurrent sends can all pass the idle-thread admission check before any one commits. The first starts a turn; the others attempt the same lifecycle transition using an outdated thread snapshot. Their database transactions roll back, and the inline dispatch path has not created queue rows. A four-send service test reproduced three rejections and zero queued messages in two clean-base checkouts.

2. Claims vs findings

3. Environment

Trusted origin/main at the commit above; macOS, Node 22.22.3, pnpm 9.15.0, Vitest 4.1.1. Service harness uses real migrated in-memory SQLite and simulated host commands. No live provider, listening dev server, imported store, or user runtime data was used. Each harness allocates temporary test data.

The installed pnpm entrypoint was broken. A temporary PATH wrapper called corepack pnpm. Frozen dependency linking used --ignore-scripts after the normal install hit a missing bundled node-gyp file; the repository native-module verifier repaired/verified SQLite. The normal Turbo build then passed all 57 tasks.

4. Minimal reproduction

1. Check out the trusted base in a clean checkout.
2. Install existing locked dependencies and build with pnpm install --frozen-lockfile --prefer-offline and pnpm exec turbo run build.
3. Download the regression patch and apply it with git apply regression.patch.
4. Run:

   pnpm exec turbo run test --filter=@bb/server -- test/threads/thread-send-dispatch.test.ts -t 'concurrent idle dispatch regression'

Expected: all four requests fulfill, with three retained queue rows. Actual in both runs:

AssertionError: queued=0; commands=1
One fulfilled send; three rejected sends:
ThreadLifecycleEventNotAppliedError: Thread lifecycle event not applied (illegal-transition): no transition for run.started from status active
Test Files  1 failed (1)
Tests  1 failed | 26 skipped (27)

The new regression uses existing trusted fixture helpers:

describe("concurrent idle dispatch regression", () => {
  it("retains every concurrent queue-mode send", async () => {
    await withTestHarness(async (harness) => {
      const { thread } = seedProviderThreadFixture({ harness, value: 3716 });
      const results = await Promise.allSettled(
        Array.from({ length: 4 }, (_, index) =>
          acceptThreadSendRequest(harness.deps, {
            thread,
            payload: {
              input: textInput(`concurrent message ${index}`),
              mode: "queue-if-active",
              model: "gpt-5",
              permissionMode: "full",
              reasoningLevel: "medium",
              serviceTier: "default",
            },
          }),
        ),
      );
      expect(results.map((result) => result.status === "rejected" ? String(result.reason) : result.status), `queued=${listQueuedThreadMessages(harness.db, thread.id).length}; commands=${listQueuedThreadCommands(harness, "turn.submit", thread.id).length}`).toEqual(
        Array.from({ length: 4 }, () => "fulfilled"),
      );
      expect(listQueuedThreadMessages(harness.db, thread.id)).toHaveLength(3);
    });
  });
});

5. Root cause

Admission checks consult the thread status and recheck its snapshot, but the later dispatch awaits environment and message preparation. No status check runs inside the inline request append transaction at this base. Multiple attempts therefore reach command preparation with the same idle snapshot.

The send transaction applies run.started for turn.submit. Active has no run.started transition. The immediate transaction rolls back the appended event when the required transition fails. Host dispatch occurs afterward, so the rejected requests produce no command. A queued drain has rollback protection for its existing rows; an inline request has no existing queue row to restore.

6. Proposed fix

Validate the thread status inside the existing transaction preflight, before request append or claimed-row consumption. On a status mismatch, use the existing bounded dispatch reattempt to re-evaluate the current thread and queue the competing request. Catch only a dedicated internal status-change error. Keep the lifecycle transition table unchanged. The local implementation changes 73 text lines across two files in the server thread subsystem.

After the change, the regression and all existing tests in thread-send-dispatch, dispatch-hooks, and thread-send-error-recovery pass: 44 tests across three files. The server Turbo typecheck also passes. This evidence covers the reproduced admission race; it does not establish safety of arbitrary caller retries after transport failures.

7. Verification

The same agent created a second detached temporary Git worktree at 62fe3bc46bd1d935b71482b62a1e0887ffb83487, installed locked dependencies in that checkout, verified native SQLite, and copied only the new regression test. No production fix was present. The same Turbo test command ran freshly (the server test task was not cached), again yielding one command, zero queue rows and three identical lifecycle errors. The final report required no correction. See second-run log.

8. Related issues and pull requests

No open linked pull request appeared in the issue timeline at investigation time. Recent thread issues were reviewed for classification patterns; none was established as a duplicate.

9. Appendix

First-run log · Second-run log · 44 passing tests · Typecheck · Build · Test snippet

pnpm exec turbo run test --filter=@bb/server -- test/threads/thread-send-dispatch.test.ts test/threads/dispatch-hooks.test.ts test/threads/thread-send-error-recovery.test.ts
pnpm exec turbo run typecheck --filter=@bb/server
git diff --check
git diff --numstat origin/main

Issue content was treated as untrusted claims. No issue-supplied code, instructions, branches, or external links were executed or fetched.

> AGENT GENERATED