jkudish/jev-mcp

No problems traced to this repository code in these checks. 1 static matches retained as context.

1 hardcoded secret matches retained as context; full list in the evidence.

OSV scanned supported lockfiles; unpinned dependencies and unsupported formats are outside this check.

A short run cannot establish that code is safe.

Connections show IP addresses; proxies can hide the final hostname.

Tracing covers dispatched commands and their children, not the whole OS; guest root can interfere with tracing.

Full safety evidence →

[
  {
    "status": "PASS",
    "details": {
      "working": false,
      "kind": "mcp_http",
      "url": "http://127.0.0.1:8080/mcp",
      "hosting": "local",
      "transport": "streamable-http",
      "scope": "remote hosted MCP initialize + tools/list",
      "argv": [
        "node",
        "dist/index.js",
        "--http"
      ],
      "serverInfo": {
        "name": "jev-mcp",
        "version": "0.14.5"
      },
      "protocolVersion": "2025-06-18",
      "tools": [
        "jev_verify",
        "jev_screen",
        "jev_noul",
        "jev_find",
        "jev_classify",
        "jev_decide",
        "jev_rerank",
        "jev_compare",
        "jev_extract",
        "jev_audit",
        "jev_review",
        "jev_gate"
      ],
      "tool_schema_bytes": 24135,
      "tool_result_checked": false
    },
    "check": "list_tools",
    "optional": false
  },
  {
    "status": "PASS",
    "details": {
      "working": false,
      "argv": [
        "node",
        "dist/index.js"
      ],
      "kind": "mcp",
      "serverInfo": {
        "name": "jev-mcp",
        "version": "0.14.5"
      },
      "tools": [
        "jev_verify",
        "jev_screen",
        "jev_noul",
        "jev_find",
        "jev_classify",
        "jev_decide",
        "jev_rerank",
        "jev_compare",
        "jev_extract",
        "jev_audit",
        "jev_review",
        "jev_gate"
      ],
      "tool_schema_bytes": 24135,
      "tool_result_checked": false,
      "scope": "partial: list_tools"
    },
    "check": "list_tools",
    "optional": false
  },
  {
    "status": "PASS",
    "details": {
      "working": false,
      "argv": [
        "node",
        "dist/index.js"
      ],
      "kind": "mcp",
      "serverInfo": {
        "name": "jev-mcp",
        "version": "0.14.5"
      },
      "tools": [
        "jev_verify",
        "jev_screen",
        "jev_noul",
        "jev_find",
        "jev_classify",
        "jev_decide",
        "jev_rerank",
        "jev_compare",
        "jev_extract",
        "jev_audit",
        "jev_review",
        "jev_gate"
      ],
      "tool_schema_bytes": 24135,
      "tool_result": {
        "content": [
          {
            "type": "text",
            "text": "No TYPESAFE_API_KEY, OPENROUTER_API_KEY (sk-or-), Cloudflare token (CLOUDFLARE_API_TOKEN or JEV_CLOUDFLARE_API_TOKEN) + CLOUDFLARE_ACCOUNT_ID, or AI_GATEWAY_API_KEY found. Set one, or JEV_PROVIDER to choose explicitly."
          }
        ],
        "isError": true
      },
      "needs_credentials": true,
      "credentials_error": "MCP tool reported missing credentials: {\"content\": [{\"type\": \"text\", \"text\": \"No TYPESAFE_API_KEY, OPENROUTER_API_KEY (sk-or-), Cloudflare token (CLOUDFLARE_API_TOKEN or JEV_CLOUDFLARE_API_TOKEN) + CLOUDFLARE_ACCOUNT_ID, or AI_GATEWAY_API_KEY found. Set one, or JEV_PROVIDER to choose explicitly.\"}], \"isError\": true}"
    },
    "check": "credentials_error",
    "optional": false
  }
]

• Unit tests run without an API key: The only offline test command (npm test, 292 unit tests) is a test suite, not a product operation, so it cannot verify a product claim. It is run as a README example in examples.json instead.

  Source: # unit tests, no API key needed

  Observed: The only offline test command (npm test, 292 unit tests) is a test suite, not a product operation, so it cannot verify a product claim. It is run as a README example in examples.json instead.

• Live judgments need a provider API key: No TYPESAFE_API_KEY, OPENROUTER_API_KEY or Cloudflare/AI gateway credential is available in the guest, and credentials were not sought. The jev_noul call returns the missing-key error instead (see probe credentials_error).

  Source: Requires Node.js 22 or newer and an API key for a Jev provider

  Observed: No TYPESAFE_API_KEY, OPENROUTER_API_KEY or Cloudflare/AI gateway credential is available in the guest, and credentials were not sought. The jev_noul call returns the missing-key error instead (see probe credentials_error).

• HTTP server serves MCP at /mcp with a health check at /health: Only the /health start check was exercised (see probe). A tool call on /mcp needs a provider key, so the judgment result cannot be verified.

  Source: It listens on `PORT` (default `8080`) and serves MCP at `/mcp`, with a health check at `/health`.

  Observed: Only the /health start check was exercised (see probe). A tool call on /mcp needs a provider key, so the judgment result cannot be verified.

Examples

• Build the TypeScript server as documented: command completed successfully

  Source: README.md Development section

  Input: npm run build

  Expected: tsc compiles to dist/ without errors

  Command: npm run build

  Observed: > @jkudish/[email protected] build > tsc

• Run documented offline unit tests: command completed successfully

  Source: README.md Development section

  Input: npm test

  Expected: 292 unit tests pass with no API key

  Command: npm test

  Observed: hing echoed --- duration_ms: 51.602503 type: 'test' ... # Subtest: runRegex settles immediately with an error when the request is already aborted ok 291 - runRegex settles immediately with an error when the request is already aborted --- duration_ms: 0.950265 type: 'test' ... # Subtest: runRegex terminates its worker when the signal aborts mid-run ok 292 - runRegex terminates its worker when the signal aborts mid-run --- duration_ms: 25.51719 type: 'test' ... 1..292 # tests 292 # suites 0 # pass 292 # fail 0 # cancelled 0 # skipped 0 # todo 0 # duration_ms 139134.002911

Local or hosted: local

Start the Jev MCP server (stdio and stateless HTTP) and call a judgment tool (jev_noul) to get a calibrated probability

cd <guest-path> && ls -la && cat README.md 2>/dev/null | head -200; id -u; python3 --version; cat /etc/os-release | head -3
cd <guest-path> && which node npm git; node -v 2>&1; npm -v 2>&1; ls -la .coldpass; sed -n 200,400p README.md | grep -n -i -E "key|env|config|test|dry|offline|mock|cycle|cron|once|--" | head -80
cd <guest-path> && grep -n -E "^#|npm |node |TYPESAFE|JEV_MCP_MODEL" README.md | head -120; cat package.json
cd <guest-path> && (git rev-parse --is-inside-work-tree 2>/dev/null || git init -q) && git remote add origin https://github.com/jkudish/jev-mcp 2>&1; git fetch --depth 1 origin 6faa53961ade29edd0a9a2381ce74f84d64540b0 2>&1 | tail -3; git reset --mixed FETCH_HEAD 2>&1 | tail -2; git log --oneline -1; git status --short | head
sudo -n true && echo SUDO_OK; (curl -fsSL https://deb.nodesource.com/setup_22.x -o /tmp/ns22.sh && sudo -E bash /tmp/ns22.sh >/tmp/ns.log 2>&1 && sudo apt-get install -y nodejs >>/tmp/ns.log 2>&1); echo rc=$?; tail -3 /tmp/ns.log; node -v; npm -v
cd <guest-path> && export NODE_USE_ENV_PROXY=1 && npm install 2>&1 | tail -15; echo "rc=${PIPESTATUS[0]}"; npm run build 2>&1 | tail -10; echo "build rc=${PIPESTATUS[0]}"; ls dist 2>/dev/null | head
cd <guest-path> && export NODE_USE_ENV_PROXY=1 && npm test 2>&1 | tail -12; echo "test rc=${PIPESTATUS[0]}"; env | grep -E "TYPESAFE|OPENROUTER|JEV" | sed 's/=.*/=<set>/'; grep -rn "process.env" src/*.ts | head -30
mkdir -p /tmp/mcpdrive && cat > /tmp/mcpdrive/drive.mjs <<'EOF'
import { spawn } from "node:child_process";
const [,, tool, argsJson] = process.argv;
const srv = spawn("node", ["<guest-path>"], { cwd: "<guest-path>", stdio: ["pipe","pipe","inherit"] });
let buf = ""; const waiters = {};
srv.stdout.on("data", d => { buf += d; let i; while ((i = buf.indexOf("\n")) >= 0) { const line = buf.slice(0,i); buf = buf.slice(i+1); if (!line.trim()) continue; const m = JSON.parse(line); if (m.id !== undefined && waiters[m.id]) waiters[m.id](m); } });
const send = (o) => srv.stdin.write(JSON.stringify(o) + "\n");
const req = (id, method, params) => new Promise(r => { waiters[id] = r; send({ jsonrpc: "2.0", id, method, params }); });
(async () => {
  await req(1, "initialize", { protocolVersion: "2025-06-18", capabilities: {}, clientInfo: { name: "probe", version: "0" } });
  send({ jsonrpc: "2.0", method: "notifications/initialized" });
  if (!tool) { const l = await req(2, "tools/list", {}); console.log(l.result.tools.map(t => t.name).join(", ")); }
  else { const c = await req(3, "tools/call", { name: tool, arguments: JSON.parse(argsJson) }); console.log(JSON.stringify(c.result ?? c.error, null, 1)); }
  srv.kill(); process.exit(0);
})();
EOF
cd /tmp/mcpdrive && timeout 60 node drive.mjs 2>&1 | tail -5
cd /tmp/mcpdrive && timeout 60 node drive.mjs jev_noul '{"proposition":"The sky is blue on a clear day.","context":"Clear daytime sky observed from the ground."}' 2>&1 | tail -30; echo rc=$?
cd <guest-path> && sed -n 261,270p README.md
cd /tmp/mcpdrive && timeout 90 node drive.mjs jev_noul '{"propositions":["The sky is blue on a clear day."],"context":"Clear daytime sky observed from the ground."}' 2>&1 | tail -30
ls -la <guest-path> <guest-path> 2>&1 | head; ls /inputs 2>&1 | head; wc -l <guest-path> 2>/dev/null; grep -n -E "credentials_error|\"checks\"|'checks'|list_tools|stays_running|quote|expected_file|def " <guest-path> 2>/dev/null | head -80
sed -n 354,366p <guest-path>; sed -n 798,868p <guest-path>; sed -n 1000,1030p <guest-path> | head -5; grep -n "def cycles" -A30 <guest-path> | head -45
cd <guest-path> && (node dist/index.js --http > /tmp/mcpdrive/http.log 2>&1 &) ; sleep 4; curl -s -m 5 http://127.0.0.1:8080/health; echo; echo rc=$?; cat /tmp/mcpdrive/http.log | head -5; pkill -f "dist/index.js --http"; sleep 1; pgrep -fa "dist/index.js" || echo stopped
cd <guest-path> && nohup node dist/index.js --http > /tmp/mcpdrive/http.log 2>&1 & echo $! > /tmp/mcpdrive/http.pid; sleep 4; curl -s -m 5 http://127.0.0.1:8080/health; echo " curl_rc=$?"; head -5 /tmp/mcpdrive/http.log; kill $(cat /tmp/mcpdrive/http.pid); sleep 1; ps -p $(cat /tmp/mcpdrive/http.pid) >/dev/null && echo still_running || echo stopped
cd <guest-path> && mkdir -p .coldpass && grep -c -F "# unit tests, no API key needed" README.md && grep -c -F "Requires Node.js 22 or newer and an API key for a Jev provider" README.md && cat > .coldpass/probe.json <<'EOF'
{"checks":[
 {"check":"start","kind":"http","argv":["node","dist/index.js","--http"],"cwd":"<guest-path>","url":"http://127.0.0.1:8080/health"},
 {"check":"list_tools","kind":"mcp","argv":["node","dist/index.js"],"cwd":"<guest-path>"},
 {"check":"credentials_error","kind":"mcp","argv":["node","dist/index.js"],"cwd":"<guest-path>","tool":"jev_noul","arguments":{"propositions":["The sky is blue on a clear day."],"context":"Clear daytime sky observed from the ground."}}
]}
EOF
cat > .coldpass/claims.json <<'EOF'
[
 {"kind":"claim","claim":"Unit tests run without an API key","source":"# unit tests, no API key needed","input":"npm test","expected":"all unit tests pass","probe_kind":"cli","argv":["npm","test"],"expected_text":"# fail 0","cwd":"<guest-path>"},
 {"kind":"claim","claim":"Live judgments need a provider API key","source":"Requires Node.js 22 or newer and an API key for a Jev provider","testable":false,"reason":"No TYPESAFE_API_KEY, OPENROUTER_API_KEY or Cloudflare/AI gateway credential is available in the guest, and credentials were not sought. The jev_noul call returns the missing-key error instead (see probe credentials_error)."},
 {"kind":"claim","claim":"HTTP server serves MCP at /mcp with a health check at /health","source":"It listens on `PORT` (default `8080`) and serves MCP at `/mcp`, with a health check at `/health`.","testable":false,"reason":"Only the /health start check was exercised (see probe). A tool call on /mcp needs a provider key, so the judgment result cannot be verified."}
]
EOF
cat > .coldpass/examples.json <<'EOF'
[
 {"kind":"example","claim":"Build the TypeScript server as documented","source":"README.md Development section","input":"npm run build","expected":"tsc compiles to dist/ without errors","argv":["npm","run","build"],"cwd":"<guest-path>","expected_text":"> tsc"},
 {"kind":"example","claim":"Run documented offline unit tests","source":"README.md Development section","input":"npm test","expected":"292 unit tests pass with no API key","argv":["npm","test"],"cwd":"<guest-path>","expected_text":"# fail 0"}
]
EOF
cat > .coldpass/cycles.json <<'EOF'
{"detected":false,"reason":"Inspected README.md and package.json entry points (bin dist/index.js, src/index.ts, src/http.ts). The server is an on-demand stdio or stateless HTTP MCP server: it starts on a client request and handles tool calls; there is no cron, loop, --once/--now flag, scheduler or webhook trigger in the README or source. The only background behavior is the HTTP listener, which was verified by a start check instead."}
EOF
python3 -I -c "import json;[json.load(open('.coldpass/'+f)) for f in ['probe.json','claims.json','examples.json','cycles.json']];print('json ok')"
cd <guest-path> && export NODE_USE_ENV_PROXY=1 && <env-var>=<guest-path> timeout 900 python3 -I <guest-path> --validate --replay 2>&1 | tail -80; echo "rc=${PIPESTATUS[0]}"
cd <guest-path> && cat > .coldpass/claims.json <<'EOF'
[
 {"kind":"claim","claim":"Unit tests run without an API key","source":"# unit tests, no API key needed","testable":false,"reason":"The only offline test command (npm test, 292 unit tests) is a test suite, not a product operation, so it cannot verify a product claim. It is run as a README example in examples.json instead."},
 {"kind":"claim","claim":"Live judgments need a provider API key","source":"Requires Node.js 22 or newer and an API key for a Jev provider","testable":false,"reason":"No TYPESAFE_API_KEY, OPENROUTER_API_KEY or Cloudflare/AI gateway credential is available in the guest, and credentials were not sought. The jev_noul call returns the missing-key error instead (see probe credentials_error)."},
 {"kind":"claim","claim":"HTTP server serves MCP at /mcp with a health check at /health","source":"It listens on `PORT` (default `8080`) and serves MCP at `/mcp`, with a health check at `/health`.","testable":false,"reason":"Only the /health start check was exercised (see probe). A tool call on /mcp needs a provider key, so the judgment result cannot be verified."}
]
EOF
python3 -I -c "import json;json.load(open('.coldpass/claims.json'));print('ok')" && export NODE_USE_ENV_PROXY=1 && <env-var>=<guest-path> timeout 900 python3 -I <guest-path> --validate --replay 2>&1 | python3 -I -c "import json,sys; d=json.loads(sys.stdin.read()); print('valid:',d.get('valid')); print('problems:',d.get('problems')); print('replay:',d.get('replay',{}).get('status'))"
cd <guest-path> && (ss -ltnp 2>/dev/null | grep -E ":8080\b" || echo "port 8080 free") && mkdir -p .coldpass && cat > .coldpass/report.json <<'EOF'
{"outcome":"needs_credentials","fault_owner":"prerequisite","operation":"Start the Jev MCP server (stdio and stateless HTTP) and call a judgment tool (jev_noul) to get a calibrated probability","operation_scope":"Pinned source jkudish/jev-mcp@6faa539 (v0.14.5) built locally with Node 22.23.4. Server boots, lists all 12 tools, and the HTTP /health endpoint responds. Live judgments need a TypeSafe, OpenRouter, Cloudflare or AI gateway key, which is not available in this guest and was not sought. Unit tests (292) pass offline.","steps":["git init; git remote add origin https://github.com/jkudish/jev-mcp; git fetch --depth 1 origin 6faa53961ade29edd0a9a2381ce74f84d64540b0; git reset --mixed FETCH_HEAD","NodeSource setup_22.x + apt-get install nodejs (Node v22.23.4, npm 10.9.9)","npm install (README Development: npm install)","npm run build (README Development: npm run build)","npm test (README Development: unit tests, no API key needed): 292 pass, 0 fail","node dist/index.js (bin from package.json; README stdio server) - tools/list returns 12 tools","node dist/index.js (jev_noul with propositions) - returns 'No TYPESAFE_API_KEY, OPENROUTER_API_KEY ... found. Set one' (isError)","node dist/index.js --http - serves /health and /mcp on 127.0.0.1:8080; stopped afterwards"],"deviations":["Ubuntu 24.04 ships no Node.js; installed Node 22 from NodeSource (curl|bash of setup_22.x) as a prerequisite.","Ran the built bin 'node dist/index.js' from the pinned checkout instead of 'npx -y @jkudish/jev-mcp', because npx would fetch the published package rather than the pinned source.","The first jev_noul call used a wrong argument name ('proposition'); corrected to 'propositions' from the README. Not a product failure.","The first --http check killed my own shell via pkill -f; re-run with a PID file. Not a product failure.","Claim 'Unit tests run without an API key' marked testable:false: the checker does not count a unit suite as a product operation. Unit tests were run as an example instead.","No live API judgment was obtained: no provider key was available and none was sought."],"stuck_step":"Calling a judgment tool (e.g. jev_noul) requires a provider API key (TYPESAFE_API_KEY, OPENROUTER_API_KEY, Cloudflare token, or AI_GATEWAY_API_KEY).","main_blocker":"No provider API key in the guest, so every judgment tool returns a missing-key error.","plain_failure":null,"suggested_fix":"Run the probe again with a TypeSafe (or OpenRouter) key set in the server environment (TYPESAFE_API_KEY=ts_...) to observe a live judgment result and replay the live claims."}
EOF
python3 -I -c "import json;json.load(open('.coldpass/report.json'));print('report ok')"
ps -o pid,args -p 11372 2>/dev/null; kill 11372 2>/dev/null; sleep 2; ss -ltnp | grep -E ":8080\b" || echo "port 8080 free"
cd <guest-path> && export NODE_USE_ENV_PROXY=1 && <env-var>=<guest-path> timeout 900 python3 -I <guest-path> --validate --replay 2>&1 | python3 -I -c "import json,sys; d=json.loads(sys.stdin.read()); print('valid:',d.get('valid')); print('problems:',d.get('problems')); print('replay:',d.get('replay',{}).get('status')); print([ (c['check'],c['status']) for c in d['replay']['details']['checks']])"; ss -ltnp | grep -E ":8080\b" || echo "port 8080 free"
cd <guest-path> && cat > .coldpass/probe.json <<'EOF'
{"checks":[
 {"check":"list_tools","kind":"mcp_http","argv":["node","dist/index.js","--http"],"cwd":"<guest-path>","url":"http://127.0.0.1:8080/mcp"},
 {"check":"list_tools","kind":"mcp","argv":["node","dist/index.js"],"cwd":"<guest-path>"},
 {"check":"credentials_error","kind":"mcp","argv":["node","dist/index.js"],"cwd":"<guest-path>","tool":"jev_noul","arguments":{"propositions":["The sky is blue on a clear day."],"context":"Clear daytime sky observed from the ground."}}
]}
EOF
export NODE_USE_ENV_PROXY=1 && <env-var>=<guest-path> timeout 900 python3 -I <guest-path> --validate --replay 2>&1 | python3 -I -c "import json,sys; d=json.loads(sys.stdin.read()); print('valid:',d.get('valid')); print('problems:',str(d.get('problems'))[:600]); print('replay:',d.get('replay',{}).get('status')); print([ (c['check'],c['status'],c['details'].get('blocker','')[:200]) for c in d['replay']['details']['checks']])"; ss -ltnp | grep -E ":8080\b" || echo "port 8080 free"
cd <guest-path> && python3 -I -c "
import json
p='.coldpass/report.json'
r=json.load(open(p))
r['deviations'].append('Checker replay of an http /health start check followed the announced /mcp URL and got HTTP 405 on GET; the probe now uses kind mcp_http list_tools against http://127.0.0.1:8080/mcp (real MCP tools/list) instead.')
r['deviations'].append('A first --http instance was left running from an earlier attempt and held port 8080; it was stopped by PID before the final replay.')
json.dump(r,open(p,'w'),indent=1)
print('report ok')
" && ls .coldpass && git status --short | head -5