#2400 · thread spawn accepts a nonexistent model
Verdict: REPRODUCED · Root-cause confidence: high
1. TL;DR
The CLI accepts an invalid model and returns exit code zero. The server then creates a thread and a managed worktree.
The provider starts a turn and rejects the model with HTTP status 404. The timeline also shows the rejection as an agent message.
Tasks presets accept the same invalid model. They also accept a reasoning level that the selected model does not support.
Workflows rejects both invalid selections. The server does not use that catalog check for thread creation or Tasks preset storage.
2. Claims vs findings
| Claim from the issue | Status | Evidence |
|---|---|---|
thread spawn accepts a model that does not exist. | Verified | The full catalog did not contain the model. Spawn returned SPAWN_RC=0. |
| The command creates a full worktree before the failure. | Verified | Events 3 through 6 show worktree creation and completed setup. |
| The command starts a provider turn. | Verified | Event 7 has the provider thread ID. Event 8 starts the provider turn. |
| The failure appears as an agent message. | Verified | Event 10 is item/completed. Its item type is agentMessage. |
| A preset accepts unsupported reasoning for its model. | Verified | The Haiku catalog entry supports only low. Preset creation stored medium. |
| Preset update accepts a nonexistent model. | Verified | The update stored claude-does-not-exist-9 and returned zero. |
| Workflow validation rejects the same invalid selections. | Verified | Both workflow checks returned one and printed exact catalog errors. |
| 10,126 of 11,011 turns used one model. | Unverified | The issue used private data. This check did not use that data. |
| Automation has the same defect. | Unverified | The issue did not test this path. This check stayed within the assigned scope. |
3. Environment
- bb commit:
ad79bbb5ec909524f8f281e62d860c588a86f332 - OS: Linux 7.0.0-30-generic, x86_64
- Node: v24.18.0, ABI 137
- Claude Code: 2.1.247
- Codex CLI: 0.150.1
- App:
localhost:16360 - Server:
localhost:24360 - Host daemon:
127.0.0.1:32360 - Data:
~/.bb-dev/bb-report-worktrees-bb-report-2400-revise-aj9ntk-1ea2bbb0ac44
The private-copy install passed. The Turbo build passed with 18 successful tasks.
4. Minimal reproduction
Prepare and run the isolated instance
Run these commands from the parent bb checkout. The script starts its own isolated dev instance.
export PATH="/home/sawyer/.nvm/versions/node/v24.18.0/bin:$PATH" test "$(node --version)" = "v24.18.0" test "$(node -p 'process.versions.modules')" = "137" REPRO_CHECKOUT=$(mktemp -d /tmp/bb-2400-repro-XXXXXX) rmdir "$REPRO_CHECKOUT" git worktree add --detach "$REPRO_CHECKOUT" ad79bbb5ec909524f8f281e62d860c588a86f332 cd "$REPRO_CHECKOUT" pnpm install --frozen-lockfile --prefer-offline --package-import-method=copy pnpm exec turbo run build bash /home/sawyer/.bb/reports-work/issues/2400/repro/repro-invalid-selection.sh echo "SCRIPT_RC=$?"
The script runs scripts/bb-dev-app current before it contacts the server.
The script verifies /health and a connected machine. A connection error cannot produce a pass result.
The script reads the complete model catalog. It captures each new thread ID from the spawn output.
Expected output
A fixed server must reject the model before any record or worktree exists.
Model "claude-does-not-exist-9" is not available for provider claude-code on the selected machine. SPAWN_RC=1 PASS: The server returned the exact catalog error and created no side effects. SCRIPT_RC=0
The script compares thread, environment, event, worktree, provider-thread, and provider-turn counts.
Actual output at the base commit
This output is an excerpt. The attached live output contains every catalog, spawn, thread, and event field.
STATE_BEFORE
[{"threads":0,"environments":0,"events":0,"provider_threads":0,"provider_turns":0}]
SPAWN_RC=0
STATE_AFTER
[{"threads":1,"environments":1,"events":3,"provider_threads":0,"provider_turns":0}]
DYNAMIC_THREAD_ID=thr_x42r48mx6e
Thread thr_x42r48mx6e reached status error.
WAIT_FOR_ERROR_RC=0
seq 7 thread/identity
seq 8 turn/started
seq 10 item/completed item.type=agentMessage
seq 13 provider/error httpStatusCode=404
seq 14 turn/completed status=failed
REPRODUCED: Spawn returned zero and created a thread, worktree, provider thread, and failed provider turn.
SCRIPT_RC=1
Test the Tasks preset path
eval "$(scripts/bb-dev-app env)" node packages/scripts/dist/commands/run-cli.js plugin install builtin:tasks --yes --json node packages/scripts/dist/commands/run-cli.js tasks preset create --name "Issue 2400 Haiku medium" --provider claude-code --model claude-haiku-4-5-20251001 --reasoning medium --permission accept-edits --json
Expected: The preset command rejects medium. The model supports only low.
Actual: The create command returned zero and stored medium.
The update command also returned zero and stored claude-does-not-exist-9.
Reproduction files
- Runnable regression script
- Complete live catalog, spawn, thread, and event output
- Complete preset and workflow validation output
5. Root cause
The CLI checks the general reasoning enum. It then sends the model string and reasoning value to the thread API.
The spawn command sends both values without a catalog check.
The server can load the target catalog. However, an explicit model causes an early return.
Lines 118 through 120 skip catalog load for each explicit model.
The execution plan accepts the first nonempty model string. It does not compare the string with the selected model catalog.
Lines 288 through 337 show the model and reasoning checks.
The reasoning check uses the provider-wide list. It does not use the selected model list.
Haiku supports only low. The Claude provider list also contains medium.
Tasks validates nonblank model strings and a general reasoning enum. The API then stores the parsed values.
The Tasks contract has no model and reasoning relation.
The Tasks API stores create and update input directly.
Workflows performs the missing tuple check on its path.
The validator rejects missing models and unsupported model-specific reasoning values.
6. Proposed fix (first principles)
- Add one server validator for a model and reasoning tuple.
- Load the catalog for the exact target host and workspace.
- Accept supported IDs, aliases, and configured custom models.
- Use the selected model reasoning list when that list is authoritative.
- Run validation before the server creates any thread or environment.
- Expose the same check to server plugins.
- Use the check before Tasks preset create and update.
- Repeat the check when a preset starts work.
Add tests for target hosts, aliases, custom models, inherited selections, and unavailable catalogs.
Each rejection test must prove that no thread, environment, event, worktree, or provider turn exists.
The fix can use the current model-list command. A daemon protocol change is not necessary unless the wire contract changes.
7. Related issues
- #1864 · Allow a model change for a queued message
- #2503 · Model picker reasoning lists
- PR #2581 · Reject invalid execution selections before provisioning
PR #2581 appeared after workflow scope selection. The workflow assigned no pull request review.
8. Appendix
Commands run
gh issue view 2400 -R get-bb/bb --json ... pnpm install --frozen-lockfile --prefer-offline --package-import-method=copy pnpm exec turbo run build scripts/bb-dev-app current scripts/bb-dev-app env node packages/scripts/dist/commands/run-cli.js machine list --json node packages/scripts/dist/commands/run-cli.js provider models claude-code --json bash issues/2400/repro/repro-invalid-selection.sh node packages/scripts/dist/commands/run-cli.js tasks preset create ... node packages/scripts/dist/commands/run-cli.js tasks preset update ... node packages/scripts/dist/commands/run-cli.js workflows validate --source ... git log ad79bbb5ec90..origin/main -- <affected paths>
History check
No later commit on origin/main changed the affected paths. The reproduction used the required base commit.
PR #2581 remained open during this check. Therefore, the base defect remained reproducible.
Pull requests
The workflow assigned no linked pull request for review. This report has no pull request verdict.
Verification
The verifier first found that the report did not start a dev instance. The verifier also found a fixed thread ID.
The revised script now starts scripts/bb-dev-app current. It captures the new thread ID from JSON output.
The verifier found that any spawn error produced a false pass. The revised script now requires the exact catalog error.
The script also proves that a rejected request creates no thread, environment, event, worktree, provider thread, or provider turn.
The revised artifacts contain complete raw output. The model catalog redacts one private custom model identifier.
This revision ran the script again. It reproduced the defect with thread thr_x42r48mx6e.