#3911 · Pi custom model discovery investigation
Verdict: NOT REPRODUCED · Root-cause confidence: low for the reported failure; high for the narrower catalog behavior tested below.
1. TL;DR
The issue reports a local model missing from Pi and suspects an ACP authentication handshake. BB's native Pi integration instead discovers models through Pi's RPC process. A real Pi 0.84.0 catalog test hid an unauthenticated custom model, then exposed that model when a dummy key was configured, with zero HTTP requests in both cases. This provides a plausible configuration explanation, but does not establish what happened in the reporter's environment. No production bug or safe code fix has been established.
2. Claims vs findings
| Claim | Status | Evidence |
|---|---|---|
| Local model fails initialization | Unverified | No BB/Pi version, actual error, configuration file or executable bridge setup supplied. No real inference or GGUF model was run. |
| Native Pi discovery uses an ACP HTTP authentication probe | Refuted for tested native Pi path | The catalog launches Pi RPC and calls get_state/get_available_models. Both real-Pi test cases made zero HTTP requests. |
| Local model can be absent from model picker | Verified in controlled configuration | Without provider credentials it is absent; with a dummy key it appears. This is Pi's documented availability policy, not reproduction of an unexpected BB defect. |
| Server readiness, forwarded headers or session modes cause failure | Unverified | The test server always returns 503, but is never contacted during discovery. Session creation and inference were not tested. |
3. Environment
Trusted origin/main commit 3a4288bd0f34f888a5eb43f1099f7b60fe86eea4; macOS Darwin arm64; Node v22.22.3; pnpm 9.15.0; locked Pi 0.84.0; Vitest 4.1.1. Two detached checkouts were created from the same fetched main commit. Dependencies were installed with the frozen lockfile; no dependency was added. Full first-checkout build: 58/58 tasks passed. Second-checkout provider dependency build: 6/6 passed.
The host pnpm launcher was broken; a temporary PATH shim routed pnpm through Corepack 9.15.0. Installs then passed. Each test creates a fresh temporary home, Pi agent directory and ephemeral loopback listener, then closes the listener and removes its data. No BB instance, real credentials or user runtime data was used.
4. Minimal reproduction attempt
This is a discovery diagnostic against real Pi and BB catalog code, not a llama-server or inference reproduction. The model and provider IDs are synthetic and the endpoint belongs to the test.
git clone https://github.com/get-bb/bb.git bb-3911 cd bb-3911 git checkout --detach 3a4288bd0f34f888a5eb43f1099f7b60fe86eea4 pnpm install --frozen-lockfile --prefer-offline pnpm exec turbo run build --filter=bb-plugin-provider-pi... # Save the attached test to plugins/provider-pi/src/bridge/issue-3911.test.ts pnpm exec turbo run test --filter=bb-plugin-provider-pi -- --run src/bridge/issue-3911.test.ts --silent=false
Expected and actual, in both clean checkouts:
configuredKey=false modelVisible=false HTTPRequests=0 configuredKey=true modelVisible=true HTTPRequests=0 Test Files 1 passed (1) Tests 1 passed (1)
The only variable between cases is the configured dummy key. The listener returns 503 for any request; no request was received. The test asserts visibility and request counts, and restores process environment and removes temporary data.
The complete diagnostic test follows; copy it into the path given above.
Complete diagnostic test
import { mkdtempSync, mkdirSync, writeFileSync, rmSync } from "node:fs";
import { createServer } from "node:http";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { fileURLToPath } from "node:url";
import { expect, it } from "vitest";
import { BB_PI_EXTENSION_SOURCE } from "./bb-pi-extension.js";
import { closeAllPiCatalogs, getPiCatalog } from "./catalog.js";
it("discovers a configured local model without an HTTP auth probe", async () => {
const originalEnv = { ...process.env };
const root = mkdtempSync(join(tmpdir(), "bb-3911-catalog-"));
let requests = 0;
const server = createServer((_req, res) => {
requests += 1;
res.writeHead(503).end();
});
await new Promise<void>((resolve) => server.listen(0, "127.0.0.1", resolve));
const address = server.address();
if (address === null || typeof address === "string") throw new Error("No port");
const cli = fileURLToPath(new URL("cli.js", import.meta.resolve("@earendil-works/pi-coding-agent")));
try {
for (const configuredKey of [false, true]) {
const workspace = join(root, configuredKey ? "with-key" : "without-key");
const agentDir = join(workspace, "agent");
mkdirSync(agentDir, { recursive: true });
writeFileSync(join(agentDir, "models.json"), JSON.stringify({ providers: {
"local-test": {
baseUrl: `http://127.0.0.1:${address.port}/v1`,
api: "openai-completions",
...(configuredKey ? { apiKey: "local-test-value" } : {}),
models: [{ id: "test-model", name: "Local test model" }],
},
} }));
const extension = join(workspace, "extension.mjs");
writeFileSync(extension, BB_PI_EXTENSION_SOURCE);
process.env = {
PATH: originalEnv.PATH,
HOME: workspace,
TMPDIR: root,
PI_CODING_AGENT_DIR: agentDir,
BB_PI_BRIDGE_COMMAND: process.execPath,
BB_PI_BRIDGE_ARGS: JSON.stringify([cli, "--no-skills", "--no-prompt-templates", "--no-themes"]),
};
const catalog = await getPiCatalog(workspace, extension);
const models = await catalog.listModels();
const found = models.models.some((model) => model.id === "local-test/test-model");
expect(found).toBe(configuredKey);
expect(requests).toBe(0);
console.log(`configuredKey=${configuredKey} modelVisible=${found} HTTPRequests=${requests}`);
await closeAllPiCatalogs();
}
} finally {
await closeAllPiCatalogs();
process.env = originalEnv;
await new Promise<void>((resolve, reject) => server.close((error) => error ? reject(error) : resolve()));
rmSync(root, { recursive: true, force: true });
}
}, 60_000);
5. Root cause
The actual reported root cause remains unknown. The tested mechanism is local model availability filtering by Pi's credential configuration. BB's catalog starts Pi with --mode rpc; it then requests get_available_models and maps those records into BB models. No ACP handshake participates in this native Pi path.
BB's provider health code reports an empty Pi model catalog as unauthenticated. That label can therefore reflect an empty local registry rather than a rejected HTTP request. The pinned dependency's docs/models.md describes custom providers using a baseUrl, API type and model IDs, and explains that a keyless local service still needs a configured dummy credential for availability. The test verifies this distinction. The pinned dependency is declared in provider-pi/package.json.
We cannot conclude that the reporter omitted credentials, used the native provider, or configured the endpoint incorrectly. A custom ACP adapter could have a different code path; its command/configuration was not supplied.
6. Proposed next test
Obtain the BB and Pi versions, selected provider ID, redacted custom provider configuration, exact error and daemon log. Confirm whether the model appears in Pi's available models with the same host configuration. Then test session creation against the actual local inference service. Keep the API base URL and served model ID as separate configuration fields. Do not change ACP authentication or model filtering without that evidence.
No PR: the suspected defect was not reproduced on trusted main, so the simple-fix prerequisite fails. No open linked PR was found in GitHub timeline metadata or the open-PR issue-number search.
7. Verification
The same investigator repeated the test in a second fresh detached checkout at 3a4288bd0f34f888a5eb43f1099f7b60fe86eea4, with separately installed dependencies, new temporary data directories and a new ephemeral port. The exact Turbo test command above passed again; both runs executed uncached. Both production trees remained unchanged, with only the diagnostic test added. No report correction was needed. This is a repeat check by the same investigator, not an independent review.
8. Related issues
The issue cites other reports, but their suggested relationship was not used as evidence. No linked code or issue-provided commands were executed.
9. Appendix and limitations
Read-only investigation used GitHub issue, comments, type/field and linked-PR metadata; git fetch, worktree creation, source inspection, frozen installs, Turbo builds and the attached test. Focused output from both runs is included below; raw logs and the test file are retained in the local report backup. Code permalinks were checked against the recorded base. The issue had no actual diagnostic logs. This report does not verify the specific model, endpoint, startup timing, HTTP headers, or session behavior and does not claim that the reported problem is resolved.
Recorded run output
First clean checkout
bb-plugin-provider-pi:test: configuredKey=false modelVisible=false HTTPRequests=0 bb-plugin-provider-pi:test: configuredKey=true modelVisible=true HTTPRequests=0 bb-plugin-provider-pi:test: Test Files 1 passed (1) bb-plugin-provider-pi:test: Tests 1 passed (1) bb-plugin-provider-pi:test: Duration 2.64s (transform 309ms, setup 0ms, import 480ms, tests 1.93s, environment 0ms) Tasks: 5 successful, 5 total Cached: 0 cached, 5 total
Second clean checkout
bb-plugin-provider-pi:test: configuredKey=false modelVisible=false HTTPRequests=0 bb-plugin-provider-pi:test: configuredKey=true modelVisible=true HTTPRequests=0 bb-plugin-provider-pi:test: Test Files 1 passed (1) bb-plugin-provider-pi:test: Tests 1 passed (1) bb-plugin-provider-pi:test: Duration 2.41s (transform 347ms, setup 0ms, import 511ms, tests 1.79s, environment 0ms) Tasks: 5 successful, 5 total Cached: 0 cached, 5 total