#4484 — Custom-provider configuration and AI service status
Issue · 2026-09-30 · PARTIALLY REPRODUCED
High confidence in the exercised missing-file status/completion path. Native Codex custom-provider authentication and successful routing remain unverified. The claim is that the bb Codex AI service ignores a custom provider and misleadingly requests a Codex login when its auth file is absent.
Observed behavior
In two clean checkouts, the real host status handler and inference entry point ran against an owned temporary configuration directory. Baseline with no configuration, then two synthetic custom-provider selections, all produced the same results:
- Status: ready=false with the login prompt.
- Completion: auth_required / codex_auth_missing before any transport call.
- The captured file reads were auth.json twice per scenario. config.toml was never read.
- Zero network calls; no auth file or credential was created.
The configuration contained a synthetic model/provider name and a reserved .invalid endpoint only. It contained no command-auth instructions, API keys, tokens or real accounts. The test did not establish that native Codex would accept this configuration or authenticate successfully. Consequently the broader reported working-native-provider comparison is not verified.
Expected versus actual
For a supported custom-provider workflow, the expected contract is either provider-aware resolution or an accurate unsupported-configuration status. Actual exercised behavior is a generic login prompt and early missing-file failure regardless of synthetic provider selection. The no-config baseline reproduces the same failure, showing that the configuration does not affect this path. This is not a successful-provider control and does not establish the correct native authentication behavior.
Environment and repeatable steps
Trusted fetched main SHA: facb6c161d9915d2517ed15f6101e4295cf85da1. Linux container, Node24.19.0, repository-pinned pnpm9.15.0, Vitest4.1.1. Two separate detached checkouts, separate frozen installs and fresh temporary state per run; shared package store only. Turbo forced with zero cached tasks. No dependencies added. The reported macOS/native Codex environment was not exercised.
git clone https://github.com/get-bb/bb.git first
cd first
git checkout --detach facb6c161d9915d2517ed15f6101e4295cf85da1
corepack pnpm install --frozen-lockfile
# Save the inline test as plugins/provider-codex/src/issue4484.test.ts
corepack pnpm exec turbo run test --filter=bb-plugin-provider-codex --force -- issue4484.test.ts --silent=false
# Repeat in a second fresh clone at the same SHA using the identical test.
Execution used the environment's existing pinned pnpm launcher and package store; local cache paths are omitted. Both frozen installs succeeded. Initial first-run setup imported the harness from the wrong SDK export and failed before the claim was tested. The import was corrected to testing/host in both checkouts; the final identical tests passed. This setup failure is not a bug verdict.
Inline faithful test
The public SDK host harness invokes the actual handler and validates its input/output. The unused provider-bridge export is replaced to prevent unrelated bridge initialization. File reads delegate to the real filesystem only for the owned auth/config paths; all other reads fail. fetch is a failing capture adapter, and the tested code never calls it. CODEX_HOME is scoped and restored by Vitest; no actual user configuration is changed. No credentials are persisted.
import fs from "node:fs/promises";
import { tmpdir } from "node:os";
import path from "node:path";
import { expect, it, vi } from "vitest";
import { experimental_createHostEntryHarness } from "@get-bb/plugin-sdk/testing/host";
import host from "./host.js";
import { completeCodexInference } from "./ai/chatgpt-client.js";
vi.mock("./bridge/bridge.js", () => ({ experimental_providerBridge: null }));
it("captures real status and completion behavior with synthetic custom-provider configuration", async () => {
const root = await fs.mkdtemp(path.join(tmpdir(), "synthetic-provider-config-"));
const originalRead = fs.readFile.bind(fs);
const reads: string[] = [];
const fetchAdapter = vi.fn(() => { throw new Error("Network is forbidden in this fixture"); });
const outcomes = [];
try {
vi.stubEnv("CODEX_HOME", root);
vi.stubGlobal("fetch", fetchAdapter);
vi.spyOn(fs, "readFile").mockImplementation(async (p, options) => {
if (typeof p !== "string" || ![path.join(root, "auth.json"), path.join(root, "config.toml")].includes(p)) throw new Error("Unexpected filesystem read");
reads.push(path.basename(p));
return originalRead(p, options);
});
const harness = experimental_createHostEntryHarness(host);
for (const provider of [null, "synthetic-alpha", "synthetic-beta"]) {
if (provider !== null) await fs.writeFile(path.join(root, "config.toml"), `model_provider = "${provider}"\nmodel = "synthetic-model"\n[model_providers.${provider}]\nname = "Synthetic provider"\nbase_url = "https://provider.invalid/v1"\nwire_api = "responses"\n`);
reads.length = 0;
const status = await harness.experimental_call("codex.ai.status", {});
expect(status).toEqual({ ready: false, message: "Run `codex login` on the primary machine to sign in" });
await expect(completeCodexInference({ model: "synthetic-model", prompt: "Synthetic request", timeoutMs: 1000 }, new AbortController().signal)).rejects.toMatchObject({ code: "auth_required", detailCode: "codex_auth_missing" });
expect(reads).toEqual(["auth.json", "auth.json"]);
expect(fetchAdapter).not.toHaveBeenCalled();
outcomes.push({ configuredProvider: provider, status, completion: { code: "auth_required", detailCode: "codex_auth_missing" }, filesRead: [...reads], networkCalls: fetchAdapter.mock.calls.length });
}
await harness.experimental_dispose();
const remaining = await fs.readdir(root);
expect(remaining).toEqual(["config.toml"]);
console.log("ISSUE4484_EVIDENCE=" + JSON.stringify({ outcomes, credentialsCreated: false, authFilePresent: false, commandAuthExecuted: false }));
} finally {
vi.restoreAllMocks();
vi.unstubAllGlobals();
vi.unstubAllEnvs();
await fs.rm(root, { recursive: true, force: true });
}
});
Both-run evidence
The same agent personally repeated the test in a second clean checkout at the same SHA with fresh state. This is same-agent clean reproduction, not independent verification. Each final run: 1 test passed, 6 Turbo tasks successful, 0 cached. Actual structured outcomes match:
{
"first": {
"outcomes": [
{
"configuredProvider": null,
"status": {
"ready": false,
"message": "Run `codex login` on the primary machine to sign in"
},
"completion": {
"code": "auth_required",
"detailCode": "codex_auth_missing"
},
"filesRead": [
"auth.json",
"auth.json"
],
"networkCalls": 0
},
{
"configuredProvider": "synthetic-alpha",
"status": {
"ready": false,
"message": "Run `codex login` on the primary machine to sign in"
},
"completion": {
"code": "auth_required",
"detailCode": "codex_auth_missing"
},
"filesRead": [
"auth.json",
"auth.json"
],
"networkCalls": 0
},
{
"configuredProvider": "synthetic-beta",
"status": {
"ready": false,
"message": "Run `codex login` on the primary machine to sign in"
},
"completion": {
"code": "auth_required",
"detailCode": "codex_auth_missing"
},
"filesRead": [
"auth.json",
"auth.json"
],
"networkCalls": 0
}
],
"credentialsCreated": false,
"authFilePresent": false,
"commandAuthExecuted": false
},
"second": {
"outcomes": [
{
"configuredProvider": null,
"status": {
"ready": false,
"message": "Run `codex login` on the primary machine to sign in"
},
"completion": {
"code": "auth_required",
"detailCode": "codex_auth_missing"
},
"filesRead": [
"auth.json",
"auth.json"
],
"networkCalls": 0
},
{
"configuredProvider": "synthetic-alpha",
"status": {
"ready": false,
"message": "Run `codex login` on the primary machine to sign in"
},
"completion": {
"code": "auth_required",
"detailCode": "codex_auth_missing"
},
"filesRead": [
"auth.json",
"auth.json"
],
"networkCalls": 0
},
{
"configuredProvider": "synthetic-beta",
"status": {
"ready": false,
"message": "Run `codex login` on the primary machine to sign in"
},
"completion": {
"code": "auth_required",
"detailCode": "codex_auth_missing"
},
"filesRead": [
"auth.json",
"auth.json"
],
"networkCalls": 0
}
],
"credentialsCreated": false,
"authFilePresent": false,
"commandAuthExecuted": false
}
}
Root cause and source evidence
The host status handler maps the missing-auth failure to the login prompt. The reader accesses auth.json directly; credential resolution throws on a missing file. The completion path resolves those credentials before reaching transport selection. These paths were executed. Source inspection also shows built-in ChatGPT/OpenAI transport branches, but no successful request or endpoint selection was executed in this test; that observation is not a verified custom-provider routing result.
Proposal and unresolved scope
Define whether this AI service supports native custom-provider configurations. If unsupported, report that limitation explicitly instead of prescribing login as a universal fix. If supported, resolve the selected provider through the intended adapter and add isolated tests for configuration interpretation and routing before enabling credentialed integration. Do not introduce a credential workaround based on this report.
Unverified: command-backed authentication, Keychain, native Codex compatibility, successful model response, endpoint selection after usable credentials, real desktop UI and primary-host transport. No screenshot is supplied because only handler behavior was tested. No live service, user files, actual secrets or runtime was accessed. The issue contained setup instructions and external source links; these were treated as untrusted claims, not executed or fetched. Tests came from the trusted repository implementation and SDK harness. No linked PR or existing report was present at preparation.