#3910 · Pi custom model discovery

Bug · Priority: Medium · Effort: Medium · providers · provider-pi · 2026-09-18

Issue · Base b84e67d672897cfbee2be2752ca9474183a07784

Verdict: NOT REPRODUCED · Root-cause confidence: low

1. TL;DR

The report describes an unavailable local model and suspects ACP authentication. Two clean trusted checkouts successfully initialized the real Pi RPC catalog and exposed a configured custom model. The local HTTP endpoint received zero requests during discovery, even though it was deliberately unable to serve inference. Omitting the configured key hid the model without preventing initialization, providing a possible configuration explanation rather than proof of the reported cause. The original configuration, versions and error logs are missing, so neither its failure nor a BB defect is established.

2. Claims vs findings

3. Environment

Public get-bb/bb main at the commit above; macOS Darwin arm64; Node v22.22.3; pnpm 9.15.0; repository-pinned Pi 0.84.0. Two detached trusted worktrees with separate frozen installs. Both full Turbo builds passed all 58 tasks. No user BB instance or credentials were used. Each probe created fresh temporary agent/workspace directories and an OS-assigned loopback port, removed on completion. The listener always returns HTTP 503 and only counts requests; it does not emulate inference.

4. Minimal reproduction attempt

1. From a clean checkout of the recorded commit, install frozen dependencies and build:

   corepack pnpm install --frozen-lockfile --prefer-offline
   corepack pnpm exec turbo run build

   Ensure child processes resolve a working pnpm 9.15.0 launcher. This host required a temporary Corepack shim because its default launcher was broken.
2. Save repro-3910.ts in the checkout root and run:

   node --conditions=source --import tsx repro-3910.ts

3. Expected: both catalogs initialize; a configured custom model appears; no network discovery request occurs. Actual, verbatim in both runs:

   {"withKey":false,"initialized":true,"models":[],"endpointRequests":0}
   {"withKey":true,"initialized":true,"models":["local-test/test-model"],"endpointRequests":0}
   PASS: custom model discovery follows Pi configuration; no HTTP auth probe occurred.
   

The harness uses BB's actual catalog and extension with the pinned real Pi executable. It changes only whether the isolated provider configuration includes a dummy key. It does not use a fake Pi process, external model URL, downloaded weights or an inference service.

import assert from 'node:assert/strict';
import { mkdtempSync, mkdirSync, writeFileSync, rmSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { resolve, join } from 'node:path';
import { createServer } from 'node:http';
import { getPiCatalog, closeAllPiCatalogs } from './plugins/provider-pi/src/bridge/catalog.js';
import { BB_PI_EXTENSION_SOURCE } from './plugins/provider-pi/src/bridge/bb-pi-extension.js';

const root = mkdtempSync(join(tmpdir(), 'bb-3910-probe-'));
const originalEnv = process.env;
let requests = 0;
const server = createServer((_req, res) => { requests++; res.writeHead(503); res.end(); });
await new Promise<void>((done) => server.listen(0, '127.0.0.1', done));
const address = server.address();
assert(address && typeof address === 'object');
const cli = resolve('plugins/provider-pi/node_modules/@earendil-works/pi-coding-agent/dist/cli.js');
try {
  for (const withKey of [false, true]) {
    const dir = join(root, withKey ? 'configured' : 'missing-key');
    mkdirSync(dir);
    const agentDir = join(dir, 'agent');
    mkdirSync(agentDir);
    writeFileSync(join(agentDir, 'models.json'), JSON.stringify({providers: {'local-test': {
      baseUrl: `http://127.0.0.1:${address.port}/v1`, api: 'openai-completions',
      ...(withKey ? {apiKey: 'local-test-dummy'} : {}),
      models: [{id: 'test-model', name: 'Local test model', reasoning: false, input: ['text'], contextWindow: 8192, maxTokens: 512}]
    }}}));
    const extension = join(dir, 'extension.mjs');
    writeFileSync(extension, BB_PI_EXTENSION_SOURCE);
    process.env = {
      PATH: originalEnv.PATH,
      TMPDIR: originalEnv.TMPDIR,
      PI_CODING_AGENT_DIR: agentDir,
      BB_PI_BRIDGE_COMMAND: process.execPath,
      BB_PI_BRIDGE_ARGS: JSON.stringify([cli, '--no-extensions', '--no-skills', '--no-prompt-templates', '--no-themes']),
    };
    const catalog = await getPiCatalog(dir, extension);
    await catalog.probe();
    const {models} = await catalog.listModels();
    const ids = models.map(model => model.id);
    assert.deepEqual(ids, withKey ? ['local-test/test-model'] : []);
    assert.equal(requests, 0);
    console.log(JSON.stringify({withKey, initialized: true, models: ids, endpointRequests: requests}));
    await closeAllPiCatalogs();
  }
  console.log('PASS: custom model discovery follows Pi configuration; no HTTP auth probe occurred.');
} finally {
  await closeAllPiCatalogs();
  process.env = originalEnv;
  await new Promise<void>((done, reject) => server.close(error => error ? reject(error) : done()));
  rmSync(root, {recursive: true, force: true});
}

5. Root cause

The reporter's root cause remains unknown. The observed empty-list case is Pi's credential-dependent model availability: with otherwise identical configuration, adding a dummy key makes this local model available. The frozen Pi package's docs/models.md documents that local providers still need configured auth to appear in its available catalog. This is a plausible setup explanation, not a diagnosis of the original installation.

BB's catalog launch runs --mode rpc --no-session --extension, then sends get_state and get_available_models. This path does not perform an ACP HTTP authentication handshake. The health handler maps an empty model list to unauthenticated, which can explain authentication wording without a failed network handshake. Model selection expects a provider/model identifier or a resolvable model ID. A model-selection string is not a substitute for configuring an HTTP provider in Pi.

6. Proposed next test

No production fix is supported by this evidence. Obtain the reporter's BB/Pi versions, exact provider selection (native Pi or external ACP agent), sanitized models.json/settings, and verbatim startup errors. Test the same configuration directly in Pi and then in BB. Verify the inference API base URL and model ID separately from the GGUF file location. Do not publish authentication values. An inference test against the original local server is still needed.

7. Verification

The same agent repeated the probe in a second clean detached checkout at the identical commit after a separate frozen install. Command: node --conditions=source --import tsx repro-3910.ts. It used new temporary directories and a new OS-assigned port and produced the identical successful assertions. No correction to the first result was needed. This verifies the bounded negative result; it does not verify the reporter's setup. Both runs produced the exact output shown above. Full build counts: first checkout 58 successful (4 cached); second checkout 58 successful (56 cached). Raw local artifacts are retained outside the published files; the complete probe and results are embedded here.

Existing relevant checks also passed: pnpm exec turbo run test --filter=bb-plugin-provider-pi -- src/bridge/catalog.test.ts src/model-list.test.ts (8 tests). No linked open PR was found in issue cross-reference metadata or the open-PR search for 3910. No fix branch or PR was created because no product defect reproduced.

8. Related issues

Repository search found #2289, concerning stale Pi model discovery after configuration edits. This probe used fresh catalogs, so it does not establish that this report has the same cause. Issue-supplied related links were not fetched.

9. Appendix and limits

Read-only GitHub metadata established classification, public visibility, labels and absence of linked open PRs. The trusted get-bb/bb main commit was fetched and matched origin/main before creating both worktrees. The included probe and verification record contain the repeatable checks. The model weights, real inference, provider health UI, and a third-party ACP deployment were not exercised. No source change, dependency addition or migration was made.

Issue content was treated as untrusted claims, not executable instructions. No issue-provided commands, endpoints or model strings were executed.

> AGENT GENERATED