gargpratyush/jev-router

Review signals: 1 connection review, 1 outside write, 1 sensitive read.

Reviewed as expected for a test run: resolver address lookups that send no data.

Repository processes attempted these non-loopback connections; proxies can hide the final hostname. 2 distinct locations. 15 attributed processes.

Repository processes attempted these writes outside documented output/cache paths. /sys/kernel/debug/tracing/trace_marker 10 attributed processes.

A repository process attempted to read this sensitive path. <guest-path> 15 attributed processes.

OSV scanned supported lockfiles; unpinned dependencies and unsupported formats are outside this check.

A short run cannot establish that code is safe.

Connections show IP addresses; proxies can hide the final hostname.

Tracing covers dispatched commands and their children, not the whole OS; guest root can interfere with tracing.

Full safety evidence →

[
  {
    "status": "PASS",
    "details": {
      "working": false,
      "argv": [
        "/opt/node20/bin/node",
        "bin/jev-claude.mjs",
        "--version"
      ],
      "kind": "cli",
      "returncode": 0,
      "stdout": "2.1.197 (Claude Code)",
      "stderr": "[jev] no JEV_API_KEY found - starting Claude Code without routing\n[jev] set it in <guest-path> to enable routing",
      "needs_credentials": false,
      "needs_browser": false,
      "scope": "partial: install"
    },
    "check": "install",
    "optional": false
  },
  {
    "status": "PASS",
    "details": {
      "working": false,
      "argv": [
        "/opt/node20/bin/node",
        "bin/jev-claude.mjs",
        "-p",
        "what is 2+2?"
      ],
      "kind": "cli",
      "returncode": 1,
      "stdout": "Not logged in \u00b7 Please run /login",
      "stderr": "[jev] no JEV_API_KEY found - starting Claude Code without routing\n[jev] set it in <guest-path> to enable routing",
      "needs_credentials": true,
      "needs_browser": false,
      "scope": "partial: missing credentials"
    },
    "check": "credentials_error",
    "optional": false
  }
]

• Every CLI argument is forwarded to the real CLI: Forwarding can only be shown with a real prompt (jev-claude -p ...), which needs a logged-in Claude Code session; here it exits 1 with 'Not logged in' (see probe.json credentials_error). The only runnable forwarded call, --version, is setup-only and is not counted as a claim result.

  Source: with a subscription. Every CLI argument is forwarded:

  Observed: Forwarding can only be shown with a real prompt (jev-claude -p ...), which needs a logged-in Claude Code session; here it exits 1 with 'Not logged in' (see probe.json credentials_error). The only runnable forwarded call, --version, is setup-only and is not counted as a claim result.

• Both commands launch the real upstream CLI: The launcher does start the real Claude Code binary (guest run prints 'Not logged in' from it), but a working launch needs a logged-in CLI and a JEV_API_KEY; the only key-free call is --version, which is setup-only.

  Source: Both commands launch the real upstream CLI. Jev only chooses the model for a fresh user turn.

  Observed: The launcher does start the real Claude Code binary (guest run prints 'Not logged in' from it), but a working launch needs a logged-in CLI and a JEV_API_KEY; the only key-free call is --version, which is setup-only.

• Routing is fail-open: Jev failure never blocks the CLI.: Needs a TypeSafe JEV_API_KEY (not available) and a logged-in Claude Code session (claude prints 'Not logged in'), so a routed turn cannot be run in this guest.

  Source: Routing is fail-open: Jev failure never blocks the CLI.

  Observed: Needs a TypeSafe JEV_API_KEY (not available) and a logged-in Claude Code session (claude prints 'Not logged in'), so a routed turn cannot be run in this guest.

• Jev only chooses the model for a fresh user turn.: Routing decisions need a JEV_API_KEY from TypeSafe and a logged-in Claude Code or Codex CLI; neither credential is available in this guest.

  Source: Both commands launch the real upstream CLI. Jev only chooses the model for a fresh user turn.

  Observed: Routing decisions need a JEV_API_KEY from TypeSafe and a logged-in Claude Code or Codex CLI; neither credential is available in this guest.

Examples

• Development test suite runs offline and passes: command completed successfully

  Source: README.md Development section (node --test on test/*.test.mjs)

  Input: explicit list of the five *.test.mjs files

  Expected: all tests pass

  Command: /opt/node20/bin/node --test test/codex.test.mjs test/explain.test.mjs test/policy.test.mjs test/proxy.test.mjs test/settings.test.mjs

  Observed: t: removes the sentinel when there was no previous model ok 63 - removes the sentinel when there was no previous model --- duration_ms: 4.161652 ... # Subtest: leaves a real model the user chose during the session alone ok 64 - leaves a real model the user chose during the session alone --- duration_ms: 3.717352 ... # Subtest: a missing or unreadable settings file is not an error ok 65 - a missing or unreadable settings file is not an error --- duration_ms: 2.528508 ... 1..65 # tests 65 # suites 0 # pass 65 # fail 0 # cancelled 0 # skipped 0 # todo 0 # duration_ms 1329.212723

• npm test runs the documented test suite: npm test exits 1 on Node 20.20.2 (CI pins 20.12): package.json script 'node --test "test/**/*.test.mjs"' passes the unexpanded glob to node, which prints 'Could not find .../test/**/*.test.mjs'. Ran the same five test files explicitly instead (see previous example, 65 tests, 0 failures).

  Source: README.md Development section (npm test)

  Expected: documented command completes successfully

  Observed: npm test exits 1 on Node 20.20.2 (CI pins 20.12): package.json script 'node --test "test/**/*.test.mjs"' passes the unexpanded glob to node, which prints 'Could not find .../test/**/*.test.mjs'. Ran the same five test files explicitly instead (see previous example, 65 tests, 0 failures).

Local or hosted: local

Route a turn through jev-claude (README Quick start: `jev-claude -p "fix the failing test"`)

cd <guest-path> && pwd && ls -la && id -u && cat /etc/os-release | head -3 && python3 --version && (git -C . status 2>&1 | head -2)
cd <guest-path> && cat README.md 2>/dev/null || find . -maxdepth 2 -iname 'readme*'
cd <guest-path> && cat package.json && ls -la bin src test .coldpass .github/* && (node --version; npm --version) 2>&1; which claude codex 2>&1; ls /inputs 2>&1 | head
cd <guest-path> && git init -q 2>&1 && git remote add origin https://github.com/gargpratyush/jev-router && timeout 120 git fetch --depth 1 origin 38da6b84ea01241bfc41fbddc0928d0f40a703f0 2>&1 | tail -5; git reset --mixed FETCH_HEAD 2>&1 | tail -2; git log --oneline -1 2>&1
cd /tmp && (sudo -n true && echo SUDO_OK) 2>&1; (curl -sS -m 30 -o /dev/null -w '%{http_code}\n' https://nodejs.org/dist/index.json 2>&1); (curl -sS -m 30 -o /dev/null -w '%{http_code}\n' https://registry.npmjs.org/ 2>&1); env | grep -i proxy | sed 's/=.*@/=***@/'
cd /tmp && set -o pipefail; curl -sS -m 120 -fL -o /tmp/node20.tar.xz https://nodejs.org/dist/latest-v20.x/SHASUMS256.txt >/dev/null 2>&1; N=$(curl -sS -m 60 https://nodejs.org/dist/latest-v20.x/SHASUMS256.txt | grep -o 'node-v20[0-9.]*-linux-x64.tar.xz' | head -1); echo "file=$N"; curl -sS -m 300 -fL -o /tmp/$N https://nodejs.org/dist/latest-v20.x/$N && sudo mkdir -p /opt/node20 && sudo tar -xJf /tmp/$N -C /opt/node20 --strip-components=1 && /opt/node20/bin/node --version && /opt/node20/bin/npm --version
export PATH=/opt/node20/bin:$PATH; cd <guest-path> && node --version && npm --version && npm install 2>&1 | tail -15; echo "rc=$?"; ls node_modules | head -30
export PATH=/opt/node20/bin:$PATH; cd <guest-path> && npm test 2>&1 | tail -12; echo "rc=${PIPESTATUS[0]}"
cd <guest-path> && cat .github/workflows/test.yml; echo ---; export PATH=/opt/node20/bin:$PATH; node --test test/*.test.mjs 2>&1 | tail -12; echo "rc=${PIPESTATUS[0]}"
cd <guest-path> && sed -n 1,60p bin/jev-claude.mjs | tr -d '\r'; echo ----; grep -n "API_KEY" src/*.mjs | tr -d '\r' | head
export PATH=/opt/node20/bin:$PATH; cd <guest-path> && env -u JEV_API_KEY -u TYPESAFE_API_KEY timeout 60 node bin/jev-codex.mjs exec "what is 2+2?" 2>&1 | head -20; echo "rc=${PIPESTATUS[0]}"; echo ---; env -u JEV_API_KEY -u TYPESAFE_API_KEY timeout 60 node bin/jev-claude.mjs -p "what is 2+2?" 2>&1 | head -20; echo "rc=${PIPESTATUS[0]}"
export PATH=/opt/node20/bin:$PATH; export NODE_USE_ENV_PROXY=1; cd /tmp && timeout 600 npm install -g --prefix /tmp/tools @anthropic-ai/claude-code 2>&1 | tail -5; echo "rc=${PIPESTATUS[0]}"; PATH=/tmp/tools/bin:$PATH claude --version 2>&1 | head -3
export PATH=/opt/node20/bin:/tmp/tools/bin:$PATH; cd <guest-path> && env -u JEV_API_KEY -u TYPESAFE_API_KEY timeout 120 node bin/jev-claude.mjs -p "what is 2+2?" 2>&1 | head -20; echo "rc=${PIPESTATUS[0]}"
export PATH=/opt/node20/bin:/tmp/tools/bin:$PATH; cd <guest-path> && env -u JEV_API_KEY -u TYPESAFE_API_KEY PATH=$PATH timeout 60 node bin/jev-claude.mjs --version 2>&1; echo "rc=$?"; echo ---; grep -rnE "setInterval|setTimeout|cron|--once|--now|webhook|schedul" bin src --include=*.mjs | tr -d '\r' | head; echo "grep-rc=$?"
cd <guest-path> && echo "HOME=$HOME" && mkdir -p .coldpass && E='{"PATH":"/tmp/tools/bin:/opt/node20/bin:/usr/local/bin:/usr/bin:/bin","HOME":"'$HOME'"}' && cat > .coldpass/probe.json <<EOF
{"checks":[
 {"check":"install","kind":"cli","argv":["/opt/node20/bin/node","bin/jev-claude.mjs","--version"],"cwd":"<guest-path>","env":$E,"expected_text":"2.1.197 (Claude Code)"},
 {"check":"credentials_error","kind":"cli","argv":["/opt/node20/bin/node","bin/jev-claude.mjs","-p","what is 2+2?"],"cwd":"<guest-path>","env":$E,"expected_returncode":1,"expected_text":"Not logged in"}
]}
EOF
cat > .coldpass/claims.json <<EOF
[
 {"kind":"claim","claim":"Every CLI argument is forwarded to the real CLI","source":"with a subscription. Every CLI argument is forwarded:","input":"--version","expected":"Claude Code version printed","probe_kind":"cli","argv":["/opt/node20/bin/node","bin/jev-claude.mjs","--version"],"expected_text":"2.1.197 (Claude Code)","cwd":"<guest-path>","env":$E},
 {"kind":"claim","claim":"Both commands launch the real upstream CLI","source":"Both commands launch the real upstream CLI. Jev only chooses the model for a fresh user turn.","input":"--version","expected":"real Claude Code version printed by the launcher","probe_kind":"cli","argv":["/opt/node20/bin/node","bin/jev-claude.mjs","--version"],"expected_text":"2.1.197 (Claude Code)","cwd":"<guest-path>","env":$E},
 {"claim":"Routing is fail-open: Jev failure never blocks the CLI.","source":"Routing is fail-open: Jev failure never blocks the CLI.","testable":false,"reason":"Needs a TypeSafe JEV_API_KEY (not available) and a logged-in Claude Code session (claude prints 'Not logged in'), so a routed turn cannot be run in this guest."},
 {"claim":"Jev only chooses the model for a fresh user turn.","source":"Both commands launch the real upstream CLI. Jev only chooses the model for a fresh user turn.","testable":false,"reason":"Routing decisions need a JEV_API_KEY from TypeSafe and a logged-in Claude Code or Codex CLI; neither credential is available in this guest."}
]
EOF
cat > .coldpass/examples.json <<EOF
[
 {"kind":"example","claim":"Development test suite runs offline and passes","source":"README.md Development section (node --test on test/*.test.mjs)","input":"explicit list of the five *.test.mjs files","expected":"all tests pass","argv":["/opt/node20/bin/node","--test","test/codex.test.mjs","test/explain.test.mjs","test/policy.test.mjs","test/proxy.test.mjs","test/settings.test.mjs"],"cwd":"<guest-path>","env":{"PATH":"/opt/node20/bin:/usr/local/bin:/usr/bin:/bin","HOME":"$HOME"},"expected_text":"# fail 0"},
 {"kind":"example","claim":"npm test runs the documented test suite","source":"README.md Development section (npm test)","reason":"npm test exits 1 on Node 20.20.2 (CI pins 20.12): package.json script 'node --test \"test/**/*.test.mjs\"' passes the unexpanded glob to node, which prints 'Could not find .../test/**/*.test.mjs'. Ran the same five test files explicitly instead (see previous example, 65 tests, 0 failures)."}
]
EOF
cat > .coldpass/cycles.json <<EOF
{"detected":false,"reason":"Entry points inspected (bin/jev-claude.mjs, bin/jev-codex.mjs, bin/jev-explain.mjs, src/*.mjs). No cron, scheduler, --once/--now, loop or webhook exists. Routing runs once per fresh user turn inside the CLI session; the only setTimeout (src/router.mjs) is a per-request deadline. Nothing to trigger by hand as a product cycle.","checks":[]}
EOF
for f in probe claims examples cycles; do python3 -I -m json.tool .coldpass/$f.json >/dev/null && echo "$f ok"; done; cat .coldpass/probe.json
cd <guest-path> && <env-var>=<guest-path> python3 -I <guest-path> --validate --replay 2>&1 | tail -60; echo "rc=${PIPESTATUS[0]}"; ls /inputs/coldpass-preflight.py 2>&1
cd <guest-path> && cat > .coldpass/claims.json <<'EOF'
[
 {"claim":"Every CLI argument is forwarded to the real CLI","source":"with a subscription. Every CLI argument is forwarded:","testable":false,"reason":"Forwarding can only be shown with a real prompt (jev-claude -p ...), which needs a logged-in Claude Code session; here it exits 1 with 'Not logged in' (see probe.json credentials_error). The only runnable forwarded call, --version, is setup-only and is not counted as a claim result."},
 {"claim":"Both commands launch the real upstream CLI","source":"Both commands launch the real upstream CLI. Jev only chooses the model for a fresh user turn.","testable":false,"reason":"The launcher does start the real Claude Code binary (guest run prints 'Not logged in' from it), but a working launch needs a logged-in CLI and a JEV_API_KEY; the only key-free call is --version, which is setup-only."},
 {"claim":"Routing is fail-open: Jev failure never blocks the CLI.","source":"Routing is fail-open: Jev failure never blocks the CLI.","testable":false,"reason":"Needs a TypeSafe JEV_API_KEY (not available) and a logged-in Claude Code session (claude prints 'Not logged in'), so a routed turn cannot be run in this guest."},
 {"claim":"Jev only chooses the model for a fresh user turn.","source":"Both commands launch the real upstream CLI. Jev only chooses the model for a fresh user turn.","testable":false,"reason":"Routing decisions need a JEV_API_KEY from TypeSafe and a logged-in Claude Code or Codex CLI; neither credential is available in this guest."}
]
EOF
python3 -I -m json.tool .coldpass/claims.json >/dev/null && echo ok; <env-var>=<guest-path> python3 -I <guest-path> --validate --replay > <guest-path> 2>&1; echo "rc=$?"; python3 -I -c "import json,sys; d=json.load(open('<guest-path>')) if False else None" ; head -c 1500 <guest-path>
cd <guest-path> && cat > .coldpass/report.json <<'EOF'
{"outcome":"needs_credentials","fault_owner":"prerequisite","operation":"Route a turn through jev-claude (README Quick start: `jev-claude -p \"fix the failing test\"`)","operation_scope":"Needs a TypeSafe JEV_API_KEY (from docs.typesafe.ai) in ~/.jev-router.env or the environment, plus a logged-in Claude Code (`claude login`) or Codex CLI. Neither credential is available in this disposable guest. Installed Node 20.20.2 and Claude Code 2.1.197 and ran the real launcher; routing itself was not exercised.","steps":["Install Node.js 20.12+ (README requirement): tarball node-v20.20.2-linux-x64 from nodejs.org into /opt/node20 (the README has no installer of its own)","git init; git remote add origin https://github.com/gargpratyush/jev-router; git fetch --depth 1 origin 38da6b84ea01241bfc41fbddc0928d0f40a703f0; git reset --mixed FETCH_HEAD","npm install (README Development / Local repository)","npm test (README Development) - fails, see deviations","node --test test/codex.test.mjs test/explain.test.mjs test/policy.test.mjs test/proxy.test.mjs test/settings.test.mjs - 65 pass, 0 fail","npm install -g --prefix /tmp/tools @anthropic-ai/claude-code (Claude Code prerequisite, linked from README)","node bin/jev-claude.mjs --version - prints 2.1.197 (Claude Code); Claude is forwarded","node bin/jev-claude.mjs -p \"what is 2+2?\" with no JEV_API_KEY - 'no JEV_API_KEY found - starting Claude Code without routing', then 'Not logged in · Please run /login', exit 1","node bin/jev-codex.mjs exec \"what is 2+2?\" with no key - codex not installed, exit 1"],"deviations":["README has no installer command; installed Node 20.20.2 by downloading the official tarball into /opt/node20 (no curl|sh, no telemetry prompt)","`npm test` (README Development) exits 1 on Node 20.20.2: package.json script `node --test \"test/**/*.test.mjs\"` passes the unexpanded glob, so node reports 'Could not find .../test/**/*.test.mjs'. CI pins Node 20.12, so the same failure is expected there. Ran the five *.test.mjs files explicitly instead: 65 pass, 0 fail.","Installed Claude Code via npm (@anthropic-ai/claude-code 2.1.197) rather than the README-linked setup page, into /tmp/tools","Codex CLI not installed (optional; README needs at least one of Claude Code or Codex)","Did not obtain a JEV_API_KEY or log in to Claude Code; the routed request was never sent"],"stuck_step":"node bin/jev-claude.mjs -p \"what is 2+2?\" (README Quick start) - stops at 'Not logged in' because no JEV_API_KEY and no Claude login are present","main_blocker":"No TypeSafe JEV_API_KEY and no Claude Code login in the guest, so routing cannot be exercised.","plain_failure":"The router launches and forwards to Claude Code correctly, but a real routed request cannot be made here: it needs a TypeSafe key and a signed-in Claude Code account, which this test machine does not have. The npm test script also fails on Node 20 because of a quoting issue in its test command, although the tests themselves pass when run directly.","suggested_fix":"Provide a TypeSafe JEV_API_KEY (in ~/.jev-router.env) and run `claude login` in the guest, then rerun `jev-claude -p \"what is 2+2?\"` to exercise routing. Separately, change package.json's test script to pass the glob through a shell-safe form (e.g. `node --test test/*.test.mjs`, or list the files) so `npm test` works on Node 20.12+."}
EOF
python3 -I -m json.tool .coldpass/report.json >/dev/null && echo report-ok; <env-var>=<guest-path> python3 -I <guest-path> --validate --replay 2>&1 | head -c 400; echo; echo "rc=${PIPESTATUS[0]}"; ps aux | grep -E "jev|claude" | grep -v grep | head -3