#3925 · Project environment lost during skill discovery

BugPriority: MediumEffort: Mediumprovidersprovider-claude-codeIssue

2026-09-19 UTC · trusted origin/main 2c9fdf2dc02f09fb16003658f148bbd362cff0c2

Verdict: REPRODUCED · Root-cause confidence: high · reproduction label: confirmed-repro

1. TL;DR

A project can configure an agent directory that its skill catalog does not use. A focused test of the project Skills API finds only a default-directory skill after setting a project environment override, even though the execution environment resolver returns the override. The identical override works at global scope. The server drops project identity before asking the provider to resolve its directories; clearing the cache does not repair this. Two clean checkouts of trusted main produced the same failure.

2. Claims vs findings

Claim	Finding	Evidence
Project configuration is ignored by skill discovery	Verified	Real Skills route, real database, real provider resolver and filesystem scanner; project fixture is absent.
Global configuration works	Verified	Global test passes in both runs.
Execution and listing use different environments	Verified at service boundary	Execution environment resolver contains the configured directory; captured listing RPC does not.
Cache expiry alone resolves the discrepancy	Refuted	Test explicitly invalidates native-root cache before the failing listing.
Command listings have the same context gap	Verified by source, not exercised end to end	Project command route calls the same scan helper with host and cwd only.
Other providers, WSL, terminal and composer exhibit equivalent behavior	Unverified dynamically	This run targets the Claude skill-listing API on macOS; no provider CLI, browser, terminal session, or Windows host was used.
Default roots can remain after a configured root is found	Supported by source	Provider declares a static user root as well as resolved roots. The test requires configured skill presence and does not require removal of default skills.

3. Environment

Darwin arm64; Node v22.22.3; pnpm 9.15.0 via Corepack. Both checkouts use the recorded trusted commit. Frozen installs succeeded. Full Turbo builds succeeded: 58/58 tasks in each checkout; second build reused 56 cached tasks. No dependencies or production files were changed.

First checkout: /tmp/issue-3925/checkout-a; second: /tmp/issue-3925/checkout-b. Test harness uses a migrated in-memory SQLite database and fresh temporary data directories, cleaned by the harness. HTTP requests run in process: no listening ports, live daemon, live account, or real user configuration is required. Host RPC transport is replaced by the repository test responder, which invokes the trusted Claude resolver and skill scanner with an explicit synthetic home directory. Other providers return empty resolved roots. No live provider versions apply.

4. Minimal reproduction

Checkout the trusted commit and install/build as below.
Save the inline reproduction test at the specified path. It writes two synthetic skills, requests the project catalog, stores a configuration override, verifies the execution environment, clears the native-root cache, and requests the catalog again.
Run the test: global scope should pass; project scope should fail until fixed.

git clone https://github.com/get-bb/bb.git base
cd base
git checkout --detach 2c9fdf2dc02f09fb16003658f148bbd362cff0c2
corepack pnpm install --frozen-lockfile --prefer-offline
corepack pnpm exec turbo run build
# Save the linked test as apps/server/test/public/issue-3925.test.ts
corepack pnpm exec turbo run test --filter=@bb/server -- --run test/public/issue-3925.test.ts
# Expected exit code: 1; the project assertion exposes the defect.
git worktree add --detach ../verify 2c9fdf2dc02f09fb16003658f148bbd362cff0c2
cp apps/server/test/public/issue-3925.test.ts ../verify/apps/server/test/public/issue-3925.test.ts
cd ../verify
corepack pnpm install --frozen-lockfile --prefer-offline
corepack pnpm exec turbo run build
corepack pnpm exec turbo run test --filter=@bb/server --force -- --run test/public/issue-3925.test.ts

The host has a broken direct pnpm launcher, so the actual runs used a temporary Corepack pnpm shim on PATH. No repository dependency was added. An initial test draft omitted the API's required environment ID and returned HTTP 400; the final artifact supplies a seeded environment and reaches HTTP 200 before checking skill contents.

Expected: both scopes include fixture-configured. Actual in both runs:

✓ lists configured skills with global environment
× lists configured skills with project environment
{"scope":"project","executionUsesConfiguredDirectory":true,"listingUsesConfiguredDirectory":false,"names":["fixture-default"]}
AssertionError: expected [ 'fixture-default' ] to include 'fixture-configured'
Tests  1 failed | 1 passed (2)

Complete reproduction test

import { mkdir, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { expect, it } from "vitest";
import { skillListResponseSchema } from "@bb/server-contract";
import { resolveClaudeNativeRoots } from "../../../../plugins/provider-claude-code/src/native-roots.js";
import { discoverSkills } from "../../../host-daemon/src/command-discovery.js";
import { resolveSkillScanRoots } from "../../../host-daemon/src/command-handlers/list-skills.js";
import { resolveHostEnvironment } from "../../src/services/hosts/host-environment.js";
import { replaceMachineEnvironment } from "../../src/services/machines/environment-settings.js";
import { registerHostRpcResponder } from "../helpers/host-rpc.js";
import { seedEnvironment, seedHostSession, seedPrimaryHost, seedProjectWithSource } from "../helpers/seed.js";
import { withTestHarness } from "../helpers/test-app.js";

it.each(["global", "project"] as const)("lists configured skills with %s environment", async (scope) => {
  await withTestHarness(async (harness) => {
    const { host, session } = seedHostSession(harness.deps);
    seedPrimaryHost(harness.deps, host.id);
    const homeDir = join(harness.config.dataDir, "synthetic-home");
    const configuredDir = join(harness.config.dataDir, "configured-agent");
    const cwd = join(harness.config.dataDir, "workspace");
    await mkdir(cwd, { recursive: true });
    for (const [directory, name] of [
      [join(homeDir, ".claude"), "fixture-default"],
      [configuredDir, "fixture-configured"],
    ]) {
      const root = join(directory, "skills", name);
      await mkdir(root, { recursive: true });
      await writeFile(join(root, "SKILL.md"), `---\nname: ${name}\ndescription: Synthetic discovery fixture\n---\nFixture.\n`);
    }
    const { project } = seedProjectWithSource(harness.deps, { hostId: host.id, path: cwd });
    const environment = seedEnvironment(harness.deps, { hostId: host.id, projectId: project.id, path: cwd });
    const forwarded: boolean[] = [];
    registerHostRpcResponder(harness, {
      hostId: host.id,
      sessionId: session.id,
      handle: async ({ command }) => {
        if (command.type === "plugin.host.call") {
          if (command.pluginId !== "provider-claude-code") {
            return { ok: true, result: { output: { skills: [], commands: [] } } };
          }
          const env = Object.fromEntries(command.contributedEnv.map(({ name, value }) => [name, value]));
          forwarded.push(env.CLAUDE_CONFIG_DIR === configuredDir);
          const output = await resolveClaudeNativeRoots({ cwd, homeDir, env });
          return { ok: true, result: { output } };
        }
        if (command.type === "host.list_skills") {
          const roots = await resolveSkillScanRoots({
            cwd: command.cwd,
            homeDir,
            providerId: command.providerId,
            nativeRoots: command.nativeRoots,
          });
          return { ok: true, result: { skills: await discoverSkills({ roots }) } };
        }
        throw new Error(`Unexpected command: ${command.type}`);
      },
    });
    const list = async () => {
      const response = await harness.app.request(`/api/v1/projects/${project.id}/skills?environmentId=${environment.id}`);
      expect(response.status, await response.clone().text()).toBe(200);
      return skillListResponseSchema.parse(await response.json()).skills
        .filter((skill) => skill.name.startsWith("fixture-"))
        .map((skill) => skill.name).sort();
    };
    expect(await list()).toEqual(["fixture-default"]);
    await replaceMachineEnvironment(harness.db, harness.config.dataDir, {
      variables: [{ name: "CLAUDE_CONFIG_DIR", value: configuredDir, note: null }],
    }, scope === "project" ? project.id : null);
    const executionEnvironment = await resolveHostEnvironment(harness.deps, { hostId: host.id, projectId: project.id });
    expect(executionEnvironment.find((entry) => entry.name === "CLAUDE_CONFIG_DIR")?.value).toBe(configuredDir);
    harness.deps.providerNativeRoots.invalidate();
    const names = await list();
    console.log(JSON.stringify({ scope, executionUsesConfiguredDirectory: true, listingUsesConfiguredDirectory: forwarded.at(-1), names }));
    expect(names).toContain("fixture-configured");
  });
});

5. Root cause

Project workspace resolution accepts project identity but returns only host ID and working directory. The Skills route passes that workspace to the listing service, whose native-root scan receives no project identity. The commands route also calls that scan with host and cwd only.

Native-root resolution and its cache have no project field. The decisive RPC contribution in callPluginHostRpc is:

contributedEnv: await resolveHostEnvironment(deps, {
  hostId: args.hostId,
  projectId: null,
})

Environment resolution only loads project entries when a project ID is supplied. The vendor directory resolver correctly reads the supplied environment. The test reaches that real resolver using the captured contribution and therefore resolves the default directory for a project-only setting. Global entries survive the null project context and pass the control test.

There is also a safety concern for a complete repair: the provider host handler reads process.env, and the host worker environment scope installs the first active call's environment until all active calls release it. The handler runs after an await in worker dispatch. Source inspection indicates that merely sending different project environments into overlapping calls is insufficient to guarantee per-call isolation. That concurrency behavior was not separately reproduced here.

6. Proposed fix and automatic-fix decision

Carry explicit project identity from the authorized listing route through native-root discovery to environment resolution. Make cache identity and invalidation respect project environment scope. Ensure each provider resolver receives its own effective environment under concurrent calls, with host/SDK compatibility tests. Retain global behavior and test two projects with different directories, cache refresh, and the command catalog.

No automatic PR. A safe repair spans server listing, host worker behavior, and provider/plugin environment contracts rather than one existing subsystem. Introducing a per-call SDK environment surface or changing worker concurrency/contract behavior falls outside this rule's simple-fix limits. No production fix, branch push, or PR was attempted. Open-PR metadata and search found no linked open PR; external issue-linked code was not fetched or executed.

7. Verification

The same agent repeated the reproduction in a second clean worktree at the exact trusted base commit. The second tree was clean before copying only the authored test. It received its own frozen install and build, then the same Turbo test command with --force, preventing a cached test result. The harness created new isolated temporary directories and databases. Both runs reached the catalog API and failed the same project-skill assertion; both global controls passed. No production diff exists in either checkout. No correction to the root-cause finding was needed.

Full reproduction source is inline above. Final test logs and build summaries are inline below; local temporary path identifiers are replaced with generic checkout names. Raw artifacts remain local under the reports repository publication policy.

first.log

cache miss, executing c698f9b870994627

> @bb/server@0.0.1 test /tmp/issue-3925/checkout-a/apps/server
> vitest run --config vitest.config.ts "--run" "test/public/issue-3925.test.ts"


 RUN  v4.1.1 /tmp/issue-3925/checkout-a/apps/server

(node:88394) ExperimentalWarning: SQLite is an experimental feature and might change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
stdout | test/public/issue-3925.test.ts > lists configured skills with project environment
{"scope":"project","executionUsesConfiguredDirectory":true,"listingUsesConfiguredDirectory":false,"names":["fixture-default"]}

 ❯ |@bb/server| test/public/issue-3925.test.ts (2 tests | 1 failed) 402ms
   ✓ lists configured skills with global environment  355ms
   × lists configured skills with project environment 45ms

⎯⎯⎯⎯⎯⎯⎯ Failed Tests 1 ⎯⎯⎯⎯⎯⎯⎯

 FAIL  |@bb/server| test/public/issue-3925.test.ts > lists configured skills with project environment
AssertionError: expected [ 'fixture-default' ] to include 'fixture-configured'
 ❯ test/public/issue-3925.test.ts:74:19
     72|     const names = await list();
     73|     console.log(JSON.stringify({ scope, executionUsesConfiguredDirecto…
     74|     expect(names).toContain("fixture-configured");
       |                   ^
     75|   });
     76| });
 ❯ withTestHarness test/helpers/test-app.ts:330:12
 ❯ test/public/issue-3925.test.ts:15:3

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[1/1]⎯


 Test Files  1 failed (1)
      Tests  1 failed | 1 passed (2)
   Start at  17:48:04
   Duration  3.90s (transform 2.43s, setup 992ms, import 2.42s, tests 402ms, environment 0ms)

 ELIFECYCLE  Test failed. See above for more details.

second.log

cache bypass, force executing c698f9b870994627

> @bb/server@0.0.1 test /tmp/issue-3925/checkout-b/apps/server
> vitest run --config vitest.config.ts "--run" "test/public/issue-3925.test.ts"


 RUN  v4.1.1 /tmp/issue-3925/checkout-b/apps/server

(node:90250) ExperimentalWarning: SQLite is an experimental feature and might change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
stdout | test/public/issue-3925.test.ts > lists configured skills with project environment
{"scope":"project","executionUsesConfiguredDirectory":true,"listingUsesConfiguredDirectory":false,"names":["fixture-default"]}

 ❯ |@bb/server| test/public/issue-3925.test.ts (2 tests | 1 failed) 428ms
   ✓ lists configured skills with global environment  381ms
   × lists configured skills with project environment 46ms

⎯⎯⎯⎯⎯⎯⎯ Failed Tests 1 ⎯⎯⎯⎯⎯⎯⎯

 FAIL  |@bb/server| test/public/issue-3925.test.ts > lists configured skills with project environment
AssertionError: expected [ 'fixture-default' ] to include 'fixture-configured'
 ❯ test/public/issue-3925.test.ts:74:19
     72|     const names = await list();
     73|     console.log(JSON.stringify({ scope, executionUsesConfiguredDirecto…
     74|     expect(names).toContain("fixture-configured");
       |                   ^
     75|   });
     76| });
 ❯ withTestHarness test/helpers/test-app.ts:330:12
 ❯ test/public/issue-3925.test.ts:15:3

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[1/1]⎯


 Test Files  1 failed (1)
      Tests  1 failed | 1 passed (2)
   Start at  17:48:41
   Duration  3.55s (transform 1.88s, setup 711ms, import 2.31s, tests 428ms, environment 0ms)

 ELIFECYCLE  Test failed. See above for more details.

build-results.txt

build.log
 Tasks:    58 successful, 58 total
Cached:    4 cached, 58 total
  Time:    48.171s 

build-second.log
 Tasks:    58 successful, 58 total
Cached:    56 cached, 58 total
  Time:    2.238s

8. Related issues and trust boundary

No duplicate determination was made. Related-issue claims and a suggested external patch were treated as untrusted evidence and were not used as executable inputs. No issue scripts, branches, patches, external links, or instructions were followed. Similar provider issues were consulted only for repository classification patterns. Existing classification fields were empty; Bug, Medium priority, Medium effort, and the two area/plugin labels were applied and read back.

9. Appendix

Base was fetched again after the first run and remained unchanged. Full reproduction source and logs are inline above. No screenshots are needed for this API/configuration bug. Synthetic fixture values are the only configuration values included in the artifacts. No live application was started, so there are no application ports or daemon processes to clean up.