{ "checker": { "claims": [ { "claim": "check lists every older Claude model ID to bump", "command": "node check --skip-tests --fail-on never", "evidence": "Model IDs to bump: 7 (claude-opus-5 \u00d75, claude-opus-4-6 \u00d71, anthropic.claude-opus-5-v1:0 \u00d71)\nStatic checks can't see what your framework sends. Next: npx github:qiwei66/model-bump probe -- \n", "expected": "Model ID count reported", "input": "", "kind": "claim", "outcome": "Real input produced the expected result: Model IDs to bump: 7", "returncode": 0, "source": "- Lists every older Claude model ID you'll need to bump.", "source_checked": true, "source_file": "README.md", "verdict": "yes" }, { "claim": "rules command lists the full rule set", "command": "node rules", "evidence": "content-by-position: Reads content[0] as text \u2014 Opus 5.5 responses can start with a thinking block, and then this is empty/undefined or throws\n [warn] content-by-position: Reads content[0].text \u2014 if this is a Claude response, block 0 can now be a thinking block\n [warn] content-by-position: Reads a tool_use block by position \u2014 on Opus 5.5 a thinking block can come first\n [warn] stream-first-block: Stream handler special-cases block index 0 \u2014 on Opus 5.5 block 0 can be a thinking block\n [error] computer-use-legacy: Legacy computer use tool \u2014 rejected on the Claude API and Google Cloud (Bedro", "expected": "content-by-position rule listed", "input": "rules", "kind": "claim", "outcome": "Real input produced the expected result: content-by-position", "returncode": 0, "source": "Run `npx github:qiwei66/model-bump rules` for the full list.", "source_checked": true, "source_file": "README.md", "verdict": "yes" } ], "cycle_checks": [ { "claim": "Scheduled or looping main behavior", "command": "", "evidence": "Entry points inspected (bin/model-bump.js, src/, README.md, package.json, .github/workflows/test.yml). No cron, scheduler, loop, webhook, --once or --now option exists; the tool runs once per invocation of check, rules, or probe.", "expected": "identify the product trigger", "input": "source inspection", "kind": "cycle", "outcome": "Entry points inspected (bin/model-bump.js, src/, README.md, package.json, .github/workflows/test.yml). No cron, scheduler, loop, webhook, --once or --now option exists; the tool runs once per invocation of check, rules, or probe.", "source": "repository entry points", "verdict": "yes" } ], "details": { "argv": [ "node", "", "check", "", "--skip-tests", "--fail-on", "never" ], "kind": "cli", "needs_browser": false, "needs_credentials": false, "returncode": 0, "scope": "product operation", "stderr": "", "stdout": "model-bump \u00b7 checking for Claude Opus 5.5 (claude-opus-5-5)\n You don't reference claude-opus-5-5 yet \u2014 this is what breaks when you bump.\n\n\u2716 error Forces a tool call \u2014 Opus 5.5 rejects tool_choice \"any\" / \"tool\" (400) [forced-tool-choice]\n broken-py/agent.py:19 tool_choice={\"type\": \"tool\", \"name\": \"get_weather\"},\n broken-py/agent.py:40 forced = llm.bind_tools([Weather], tool_choice=\"any\")\n broken-ts/index.ts:11 tool_choice: { type: \"any\" },\n broken-ts/index.ts:20 return generateText({ model: bedrock(\"anthropic.claude-opus-5-v1:0\"), toolChoice: \"required\", prompt: q });\n demo/app.py:13 tool_choice={\"type\": \"tool\", \"name\": \"category\"},\n fix: Use tool_choice \"auto\" + strict tool use, or structured outputs (output_config.format). Say in the prompt when the tool applies.\n docs: https://platform.claude.com/docs/en/models/opus-5-5/migration-guide#forced-tool-use\n\n[excerpt truncated]", "working": true }, "evidence_strength": "readme_checked", "example_checks": [ { "claim": "static scan of the fixtures reports the Opus 5.5 findings", "command": "node check --skip-tests --fail-on never", "deferred": false, "evidence": "edrock response_format can return plain text with HTTP 200 [dep:litellm]\n broken-py/requirements.txt\n fix: Check tool_calls / parse structured output defensively when drop_params is on. Fix for Bedrock tracked in BerriAI/litellm#42718.\n docs: https://github.com/BerriAI/litellm/issues/42717\n\nSummary: 15 errors \u00b7 8 warnings \u00b7 1 info in 3 files, 3 manifests (1 info hidden, use --verbose)\nModel IDs to bump: 7 (claude-opus-5 \u00d75, claude-opus-4-6 \u00d71, anthropic.claude-opus-5-v1:0 \u00d71)\nStatic checks can't see what your framework sends. Next: npx github:qiwei66/model-bump probe -- ", "expected": "summary of errors and warnings", "input": "fixtures directory", "kind": "example", "outcome": "command completed successfully", "returncode": 0, "source": "README.md ## check: static scan", "verdict": "yes" }, { "claim": "repo offline test suite passes (documented in package.json and CI)", "command": "npm test", "deferred": false, "evidence": "ns inside strings, comments and type declarations are skipped\n ---\n duration_ms: 0.42254\n type: 'test'\n ...\n# Subtest: capability-gated forced choice is downgraded to info\nok 26 - capability-gated forced choice is downgraded to info\n ---\n duration_ms: 0.087556\n type: 'test'\n ...\n# Subtest: content[0].text on a response is an error, on other names a warning\nok 27 - content[0].text on a response is an error, on other names a warning\n ---\n duration_ms: 0.117943\n type: 'test'\n ...\n1..27\n# tests 27\n# suites 0\n# pass 27\n# fail 0\n# cancelled 0\n# skipped 0\n# todo 0\n# duration_ms 628.190584", "expected": "all tests pass", "input": "none", "kind": "example", "outcome": "command completed successfully", "returncode": 0, "source": "package.json scripts.test", "verdict": "yes" }, { "claim": "probe wraps a real app command (README probe example)", "command": "node probe -- /tmp/mbvenv/bin/python ", "deferred": false, "evidence": "[model-bump] emulating Claude Opus 5.5 at http://127.0.0.1:35995 \u2192 running: /tmp/mbvenv/bin/python \n\nmodel-bump probe \u00b7 0 requests checked against Claude Opus 5.5\n No requests reached the probe.\n Does your client read ANTHROPIC_BASE_URL? Bedrock / Vertex / Foundry clients bypass it; point them at the Anthropic provider for the probe run.", "expected": "probe reports requests checked; app only defines functions so 0 requests", "input": "fixtures/demo/app.py run via probe", "kind": "example", "outcome": "command completed successfully", "returncode": 0, "source": "README.md ## probe: a fake Opus 5.5 for your real app", "verdict": "yes" } ], "status": "PASS" }, "deviations": [ "The pinned npx spec github:qiwei66/model-bump#b368b836... fails inside npm 10.9.9 with 'GitFetcher requires an Arborist constructor to pack a tarball'. Substitute: node bin/model-bump.js from the pinned checkout, the package's own declared bin, run from cwd /tmp.", "The unpinned npx github:qiwei66/model-bump check ran successfully before the pin was used; it resolves the default branch, so its commit was not verified against the pin.", "The fixture demo/app.py only defines functions and never calls them, so probe -- python app.py reported 0 requests. To show the fake API rejects and accepts requests, I ran probe --serve on port 8787 and POSTed a thinking-disabled request (got 400 with the Opus 5.5 error text) and a valid request (got 200 with a thinking block first), then stopped the server.", "No product cycle exists, so no cycle check was run." ], "fault_owner": null, "main_blocker": null, "operation": "Static Opus 5.5 breaking-change scan of code and dependencies (check), rule listing (rules), and running an app under the fake Opus 5.5 API (probe)", "operation_scope": "Local only, no API key. Node 22.23.4 and npm 10.9.9 on Ubuntu 24.04. Python 3.12.3 with a venv at /tmp/mbvenv holding anthropic==1.8.0 for the probe example. Source pinned to b368b836d61d97f26a88d05e2d04e54663268cb3, run via its bin/model-bump.js.", "outcome": "working", "plain_failure": null, "steps": [ "git init; git remote add origin https://github.com/qiwei66/model-bump; git fetch --depth 1 origin b368b836d61d97f26a88d05e2d04e54663268cb3; git reset --mixed FETCH_HEAD", "npx github:qiwei66/model-bump check . (README; ran unpinned from /tmp, produced findings and exit 1)", "npx github:qiwei66/model-bump rules (README; run via local bin instead, see deviations)", "npx github:qiwei66/model-bump probe --serve (README; run via local bin, POSTed requests with curl)", "npx github:qiwei66/model-bump probe -- python app.py (README; run via local bin with a venv python)" ], "stuck_step": null, "suggested_fix": "Fix the npm github commit-spec install failure in the npx path, or document the local bin fallback. The fixtures could call their functions so probe receives real requests." }