← reports

#2508 · submit-a-plugin skill: engines example does not match the marketplace schema

Type: Documentation defect Priority: Medium Effort: not assigned documentation plugins open on GitHub 2026-08-27 · base ad79bbb5ec90

Verdict: REPRODUCED · Root-cause confidence: high

1. TL;DR

The built-in skill tells an agent to add engines to a marketplace entry.

The marketplace schema forbids that field, so both marketplace validation commands report a schema error.

Commit 1e56238f removed the field from the schema after the skill entered the repository.

The change did not update the skill, and no test compares the skill example with the schema.

2. Claims vs findings

3. Environment

• bb commit: ad79bbb5ec909524f8f281e62d860c588a86f332
• Investigation-time origin/main: d3fae5aa9f9164a99b48c0ad2b11acac749565c1.
• Revision-time origin/main: cd1de34a0215c82273a08f87f1614829f9666059; no later commit changed the four affected files.
• Marketplace commit: a683caa2ffb502cdc26926c48c88a45a8579970a
• OS: Linux 7.0.0-29-generic, x86_64
• Node.js: v24.18.0
• Provider: not applicable; this test did not start a provider.
• Dev ports and data directory: not applicable; this static schema defect required no dev instance.

4. Minimal reproduction

1. Clone the marketplace and install its six development packages.

   git clone https://github.com/get-bb/marketplace.git
   cd marketplace
   git checkout a683caa2ffb502cdc26926c48c88a45a8579970a
   npm ci --ignore-scripts

2. Confirm that the unmodified marketplace builds.

   $ npm run build
   
   > build
   > node scripts/build.mjs
   
   built dist/marketplace.json with 82 entries

3. Add the skill's engines object to one valid entry.

   node -e 'const fs=require("node:fs"); const p="entries/advisor.json"; const e=JSON.parse(fs.readFileSync(p,"utf8")); e.engines={bb:">=0.40.0",bbPluginSdk:">=0.5.0"}; fs.writeFileSync(p,JSON.stringify(e,null,2)+"\n");'

4. Run the marketplace build again.

   Expected from the skill advice:
   The build accepts the documented entry shape.
   
   Actual:
   $ npm run build
   
   > build
   > node scripts/build.mjs
   
   error: schema: /plugins/0 must NOT have additional properties
   
   Exit code: 1

5. From the bb repository root, copy the supplied test into the checkout. Then run it from apps/server.

   install -Dm644 /tmp/bb-reports/issues/2508/repro/submit-a-plugin-schema.repro.test.ts \
     apps/server/test/skills/submit-a-plugin-schema.repro.test.ts
   cd apps/server
   pnpm exec vitest run test/skills/submit-a-plugin-schema.repro.test.ts
   
   AssertionError: [{"instancePath":"/plugins/0","schemaPath":"#/additionalProperties","keyword":"additionalProperties","params":{"additionalProperty":"engines"},"message":"must NOT have additional properties"}]: expected false to be true
   
   Test Files  1 failed (1)
   Tests       1 failed (1)

Repro files:

• Focused Vitest test
• Marketplace entry with the stale field
• Failed marketplace build log
• Successful control build log
• Failed regression test log

Focused repro test

import { readFile } from "node:fs/promises";
import { fileURLToPath } from "node:url";
import { Ajv2020 } from "ajv/dist/2020.js";
import { describe, expect, it } from "vitest";
import { z } from "zod";

const skillPath = fileURLToPath(
  new URL(
    "../../src/services/skills/builtin-skills/submit-a-plugin/SKILL.md",
    import.meta.url,
  ),
);
const schemaPath = fileURLToPath(
  new URL("../../../web/public/schemas/marketplace.schema.json", import.meta.url),
);

describe("submit-a-plugin marketplace entry example", () => {
  it("conforms to the published marketplace schema", async () => {
    const skill = await readFile(skillPath, "utf8");
    const marker = "Use this shape as a guide. Confirm every field against the current schema.";
    const markerIndex = skill.indexOf(marker);
    if (markerIndex < 0) throw new Error("entry example marker is missing");

    const match = /```json\n([\s\S]*?)\n```/u.exec(skill.slice(markerIndex));
    if (match?.[1] === undefined) throw new Error("entry JSON example is missing");
    const entry = JSON.parse(match[1]);

    const schema = z.record(z.string(), z.unknown()).parse(
      JSON.parse(await readFile(schemaPath, "utf8")),
    );
    const validate = new Ajv2020({ strict: false }).compile(schema);
    const manifest = {
      schemaVersion: 1,
      name: "example",
      displayName: "Example plugins",
      plugins: [entry],
    };

    expect(validate(manifest), JSON.stringify(validate.errors)).toBe(true);
  });
});

5. Root cause

The skill first entered the repository in commit def7bb83b.

That version correctly matched the first schema, which allowed engines.

Later that day, commit 1e56238f moved compatibility data to each plugin package.

The commit removed engines from the marketplace schema and runtime parser, but it missed the built-in skill.

The base skill prose still tells agents to copy engine ranges.

Its worked example repeats the stale field.

The published entry schema rejects all fields outside its property list.

The runtime parser explains that the plugin package owns compatibility data.

The existing skill tests check selected phrases only.

They do not parse the worked example or validate it against the schema.

6. Proposed fix (first principles)

1. Remove engines from the worked example.
2. Remove both sentences that tell agents to add or copy marketplace engine ranges.
3. State that the plugin package manifest owns compatibility ranges.
4. Add a test that extracts the example and validates a full manifest against the published schema.

The proposed patch made the new test pass. The existing skill tests also passed.

Test Files  2 passed (2)
Tests       5 passed (5)

Proposed patch · Passing test log

This source-only change does not alter the server and host daemon protocol.

7. Related issues

A GitHub issue search found no duplicate issue in get-bb/bb.

The report author cited marketplace PRs 123, 124, 125, and 127.

These PRs are evidence only. Issue #2508 has no linked open bb pull request for review.

8. Appendix

Important commands

gh issue view 2508 --comments
pnpm install --frozen-lockfile --prefer-offline
pnpm exec turbo run build
git clone --depth 1 https://github.com/get-bb/marketplace.git qa-marketplace
npm ci --ignore-scripts
npm run build
npm run check
pnpm exec vitest run test/skills/submit-a-plugin-schema.repro.test.ts
git fetch origin main
git log ad79bbb5ec90..origin/main -- affected-paths
git blame -L 203,267 ad79bbb5ec90 -- submit-a-plugin/SKILL.md

Raw evidence

• Issue text and issue JSON
• Turbo build log
• Marketplace install log
• Marketplace check log
• Marketplace PR evidence

Install caveat: The isolated /tmp worktree reached the shared inode limit during pnpm install.

The investigation then used the prepared dependency tree from the initial checkout. The required Turbo build passed with all 18 tasks.

The marketplace clone used its own clean npm ci dependency tree.

9. Verification

The verifier reproduced the marketplace schema error through step 4.

The verifier found that the first report did not create the focused test before it ran the test.

This revision added the exact install command and reran the steps from the bb repository root.

The focused test now ran and failed with the stated engines schema error.

The proposed fix passed both tests: two files and five tests passed.

This revision also fetched origin/main at cd1de34a0 and confirmed that it still has the defect.