{ "checker": { "claims": [ { "claim": "Every CLI argument is forwarded to the real CLI", "command": "", "evidence": "Forwarding can only be shown with a real prompt (jev-claude -p ...), which needs a logged-in Claude Code session; here it exits 1 with 'Not logged in' (see probe.json credentials_error). The only runnable forwarded call, --version, is setup-only and is not counted as a claim result.", "expected": "", "input": "", "kind": "claim", "outcome": "Forwarding can only be shown with a real prompt (jev-claude -p ...), which needs a logged-in Claude Code session; here it exits 1 with 'Not logged in' (see probe.json credentials_error). The only runnable forwarded call, --version, is setup-only and is not counted as a claim result.", "source": "with a subscription. Every CLI argument is forwarded:", "source_checked": true, "source_file": "README.md", "verdict": "blocked" }, { "claim": "Both commands launch the real upstream CLI", "command": "", "evidence": "The launcher does start the real Claude Code binary (guest run prints 'Not logged in' from it), but a working launch needs a logged-in CLI and a JEV_API_KEY; the only key-free call is --version, which is setup-only.", "expected": "", "input": "", "kind": "claim", "outcome": "The launcher does start the real Claude Code binary (guest run prints 'Not logged in' from it), but a working launch needs a logged-in CLI and a JEV_API_KEY; the only key-free call is --version, which is setup-only.", "source": "Both commands launch the real upstream CLI. Jev only chooses the model for a fresh user turn.", "source_checked": true, "source_file": "README.md", "verdict": "blocked" }, { "claim": "Routing is fail-open: Jev failure never blocks the CLI.", "command": "", "evidence": "Needs a TypeSafe JEV_API_KEY (not available) and a logged-in Claude Code session (claude prints 'Not logged in'), so a routed turn cannot be run in this guest.", "expected": "", "input": "", "kind": "claim", "outcome": "Needs a TypeSafe JEV_API_KEY (not available) and a logged-in Claude Code session (claude prints 'Not logged in'), so a routed turn cannot be run in this guest.", "source": "Routing is fail-open: Jev failure never blocks the CLI.", "source_checked": true, "source_file": "README.md", "verdict": "blocked" }, { "claim": "Jev only chooses the model for a fresh user turn.", "command": "", "evidence": "Routing decisions need a JEV_API_KEY from TypeSafe and a logged-in Claude Code or Codex CLI; neither credential is available in this guest.", "expected": "", "input": "", "kind": "claim", "outcome": "Routing decisions need a JEV_API_KEY from TypeSafe and a logged-in Claude Code or Codex CLI; neither credential is available in this guest.", "source": "Both commands launch the real upstream CLI. Jev only chooses the model for a fresh user turn.", "source_checked": true, "source_file": "README.md", "verdict": "blocked" } ], "cycle_checks": [ { "claim": "Scheduled or looping main behavior", "command": "", "evidence": "Entry points inspected (bin/jev-claude.mjs, bin/jev-codex.mjs, bin/jev-explain.mjs, src/*.mjs). No cron, scheduler, --once/--now, loop or webhook exists. Routing runs once per fresh user turn inside the CLI session; the only setTimeout (src/router.mjs) is a per-request deadline. Nothing to trigger by hand as a product cycle.", "expected": "identify the product trigger", "input": "source inspection", "kind": "cycle", "outcome": "Entry points inspected (bin/jev-claude.mjs, bin/jev-codex.mjs, bin/jev-explain.mjs, src/*.mjs). No cron, scheduler, --once/--now, loop or webhook exists. Routing runs once per fresh user turn inside the CLI session; the only setTimeout (src/router.mjs) is a per-request deadline. Nothing to trigger by hand as a product cycle.", "source": "repository entry points", "verdict": "yes" } ], "details": { "checks": [ { "check": "install", "details": { "argv": [ "/opt/node20/bin/node", "bin/jev-claude.mjs", "--version" ], "kind": "cli", "needs_browser": false, "needs_credentials": false, "returncode": 0, "scope": "partial: install", "stderr": "[jev] no JEV_API_KEY found - starting Claude Code without routing\n[jev] set it in to enable routing", "stdout": "2.1.197 (Claude Code)", "working": false }, "optional": false, "status": "PASS" }, { "check": "credentials_error", "details": { "argv": [ "/opt/node20/bin/node", "bin/jev-claude.mjs", "-p", "what is 2+2?" ], "kind": "cli", "needs_browser": false, "needs_credentials": true, "returncode": 1, "scope": "partial: missing credentials", "stderr": "[jev] no JEV_API_KEY found - starting Claude Code without routing\n[jev] set it in to enable routing", "stdout": "Not logged in \u00b7 Please run /login", "working": false }, "optional": false, "status": "PASS" } ], "scope": "partial checks", "working": false }, "evidence_strength": "partly_checked", "example_checks": [ { "claim": "Development test suite runs offline and passes", "command": "/opt/node20/bin/node --test test/codex.test.mjs test/explain.test.mjs test/policy.test.mjs test/proxy.test.mjs test/settings.test.mjs", "deferred": false, "evidence": "t: removes the sentinel when there was no previous model\nok 63 - removes the sentinel when there was no previous model\n ---\n duration_ms: 4.161652\n ...\n# Subtest: leaves a real model the user chose during the session alone\nok 64 - leaves a real model the user chose during the session alone\n ---\n duration_ms: 3.717352\n ...\n# Subtest: a missing or unreadable settings file is not an error\nok 65 - a missing or unreadable settings file is not an error\n ---\n duration_ms: 2.528508\n ...\n1..65\n# tests 65\n# suites 0\n# pass 65\n# fail 0\n# cancelled 0\n# skipped 0\n# todo 0\n# duration_ms 1329.212723", "expected": "all tests pass", "input": "explicit list of the five *.test.mjs files", "kind": "example", "outcome": "command completed successfully", "returncode": 0, "source": "README.md Development section (node --test on test/*.test.mjs)", "verdict": "yes" }, { "claim": "npm test runs the documented test suite", "command": "", "deferred": false, "evidence": "npm test exits 1 on Node 20.20.2 (CI pins 20.12): package.json script 'node --test \"test/**/*.test.mjs\"' passes the unexpanded glob to node, which prints 'Could not find .../test/**/*.test.mjs'. Ran the same five test files explicitly instead (see previous example, 65 tests, 0 failures).", "expected": "documented command completes successfully", "input": "", "kind": "example", "outcome": "npm test exits 1 on Node 20.20.2 (CI pins 20.12): package.json script 'node --test \"test/**/*.test.mjs\"' passes the unexpanded glob to node, which prints 'Could not find .../test/**/*.test.mjs'. Ran the same five test files explicitly instead (see previous example, 65 tests, 0 failures).", "source": "README.md Development section (npm test)", "verdict": "blocked" } ], "status": "PASS" }, "deviations": [ "README has no installer command; installed Node 20.20.2 by downloading the official tarball into /opt/node20 (no curl|sh, no telemetry prompt)", "`npm test` (README Development) exits 1 on Node 20.20.2: package.json script `node --test \"test/**/*.test.mjs\"` passes the unexpanded glob, so node reports 'Could not find .../test/**/*.test.mjs'. CI pins Node 20.12, so the same failure is expected there. Ran the five *.test.mjs files explicitly instead: 65 pass, 0 fail.", "Installed Claude Code via npm (@anthropic-ai/claude-code 2.1.197) rather than the README-linked setup page, into /tmp/tools", "Codex CLI not installed (optional; README needs at least one of Claude Code or Codex)", "Did not obtain a JEV_API_KEY or log in to Claude Code; the routed request was never sent" ], "fault_owner": "prerequisite", "main_blocker": "No TypeSafe JEV_API_KEY and no Claude Code login in the guest, so routing cannot be exercised.", "operation": "Route a turn through jev-claude (README Quick start: `jev-claude -p \"fix the failing test\"`)", "operation_scope": "Needs a TypeSafe JEV_API_KEY (from docs.typesafe.ai) in ~/.jev-router.env or the environment, plus a logged-in Claude Code (`claude login`) or Codex CLI. Neither credential is available in this disposable guest. Installed Node 20.20.2 and Claude Code 2.1.197 and ran the real launcher; routing itself was not exercised.", "outcome": "needs_credentials", "plain_failure": "The router launches and forwards to Claude Code correctly, but a real routed request cannot be made here: it needs a TypeSafe key and a signed-in Claude Code account, which this test machine does not have. The npm test script also fails on Node 20 because of a quoting issue in its test command, although the tests themselves pass when run directly.", "steps": [ "Install Node.js 20.12+ (README requirement): tarball node-v20.20.2-linux-x64 from nodejs.org into /opt/node20 (the README has no installer of its own)", "git init; git remote add origin https://github.com/gargpratyush/jev-router; git fetch --depth 1 origin 38da6b84ea01241bfc41fbddc0928d0f40a703f0; git reset --mixed FETCH_HEAD", "npm install (README Development / Local repository)", "npm test (README Development) - fails, see deviations", "node --test test/codex.test.mjs test/explain.test.mjs test/policy.test.mjs test/proxy.test.mjs test/settings.test.mjs - 65 pass, 0 fail", "npm install -g --prefix /tmp/tools @anthropic-ai/claude-code (Claude Code prerequisite, linked from README)", "node bin/jev-claude.mjs --version - prints 2.1.197 (Claude Code); Claude is forwarded", "node bin/jev-claude.mjs -p \"what is 2+2?\" with no JEV_API_KEY - 'no JEV_API_KEY found - starting Claude Code without routing', then 'Not logged in \u00b7 Please run /login', exit 1", "node bin/jev-codex.mjs exec \"what is 2+2?\" with no key - codex not installed, exit 1" ], "stuck_step": "node bin/jev-claude.mjs -p \"what is 2+2?\" (README Quick start) - stops at 'Not logged in' because no JEV_API_KEY and no Claude login are present", "suggested_fix": "Provide a TypeSafe JEV_API_KEY (in ~/.jev-router.env) and run `claude login` in the guest, then rerun `jev-claude -p \"what is 2+2?\"` to exercise routing. Separately, change package.json's test script to pass the glob through a shell-safe form (e.g. `node --test test/*.test.mjs`, or list the files) so `npm test` works on Node 20.12+." }