Haiku 5.5
Native agent · Not testedOperation / attempted scope pnpm start · local HTTP start + response check
Agent-reported scope: Install and build the Mobile Jev studio from the README, verify the offline test/lint/typecheck/format/build suite, and start the local studio UI (pnpm start, http://127.0.0.1:3040). The headline operation (driving a real Android phone through Mobilerun and TypeSafe) was not exercised.
Run details
- run
- droidrun--mobile-jev-claude-haiku-5-5-1792789649570607108
- date
- Oct 23, 2026
- time to working
- not recorded
- product hosting / execution
- Local HTTP server in the guest
- recorded operation scope
- product operation
- model
- claude-haiku-5-5
- agent / version
- claude-code / 2.1.300 (Claude Code)
- platform
- linux
- reported cost
- $0.023776
- Recorded source
- https://github.com/droidrun/mobile-jev
- Backend
- hyperv
- Tool transport
- mcp-assigned-vm
- Requested source pin (not an observed revision)
- {"commit": "395fc222beac4f059f9a0beb337d114a2b066e99", "url": "https://github.com/droidrun/mobile-jev"}
- Test version
- readme-bdc1b9b3c0d2673aef87e076d14084a8123ddcf09d520c1ee2e364a37c3f4bb9
- revision
- 395fc222beac4f059f9a0beb337d114a2b066e99
No Mobilerun or TypeSafe API keys and no ready Android device are available in this disposable guest.
Step: pnpm devices / live agent and demo commands: need MOBILERUN_API_KEY, TYPESAFE_API_KEY and an Android device
No Mobilerun or TypeSafe API keys and no ready Android device are available in this disposable guest.
Safety: Review signals: 29 known vulnerability.
Review signals: 29 known vulnerability.
OSV reports a known vulnerability in this dependency version. 5 distinct locations. 29 findings.
OSV scanned supported lockfiles; unpinned dependencies and unsupported formats are outside this check.
A short run cannot establish that code is safe.
Connections show IP addresses; proxies can hide the final hostname.
Tracing covers dispatched commands and their children, not the whole OS; guest root can interfere with tracing.
Answers recorded for this run
- Open the <email>:3040 and it shows a Run task control: Real input produced the expected result: Run task
Source: Open **http://127.0.0.1:3040**. Enter a goal and press **Run task**.
Input: GET http://127.0.0.1:3040/ after pnpm start
Expected: studio page with Run task control
Command: pnpm start
Observed: Run task<span class="shortcut">⌘ ↵</span></button></div></div><div class="suggestions"><span>A place to start</span><div><button><svg xmlns="http://www.w3.org/2000/svg" width="13" height="13" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-smartphone" aria-hidden="true"><rect width="14" height="20" x="5" y="2" rx="2" ry="2"></rect><path d="M12 18h.01"></path></svg>Find Android version<svg xmlns="http://www.w3.org/2000/svg" width="11" height="11" viewBox="0 0 24 24" fill="none" stroke="currentColor" strok
- pnpm check runs tests, lint, typecheck, formatting and production build: Probe stopped at a missing prerequisite: coldpass/workspace
> eslint .
> [email protected] typecheck <guest-path>
> pnpm --filter @jev-mobile/studio typecheck
> @jev-mobile/[email protected] typecheck <guest-path>
> next typegen && tsc --noEmit
Generating route types...
✓ Types generated successfully
> [email protected] format <guest-path>
> prettier . --check
Checking formatting...
ELIFECYCLE Command failed with exit code 1.
ELIFECYCLE Command failed with exit code 1.
[warn] .coldpass/observed.json
[warn] Code style issues found in the above file. Run Prettier with --write to fix.
Source: tests, lint, typecheck, formatting, production build
Input: pnpm check
Expected: formatting check and Next.js production build succeed
Command: pnpm check
Observed: coldpass/workspace > eslint . > [email protected] typecheck <guest-path> > pnpm --filter @jev-mobile/studio typecheck > @jev-mobile/[email protected] typecheck <guest-path> > next typegen && tsc --noEmit Generating route types... ✓ Types generated successfully > [email protected] format <guest-path> > prettier . --check Checking formatting... ELIFECYCLE Command failed with exit code 1. ELIFECYCLE Command failed with exit code 1. [warn] .coldpass/observed.json [warn] Code style issues found in the above file. Run Prettier with --write to fix.
Examples
- Development: run the offline test suite: command completed successfully
Source: README.md Development (pnpm check)
Input: pnpm test
Expected: 55 tests pass, 0 fail
Command: pnpm test
Observed: n wait (1.14292ms) ✔ transitional snapshots without a foreground app are polled before another model call (60.873548ms) ✔ execution is logged before a failing post-action observation (1.061345ms) ✔ stale decisions re-observe and re-predict; uncertain mutations never retry (1.388905ms) ✔ model transport reuses connections, preserves JSON, measures timing, and never retries failures (59.472379ms) ✔ metrics aggregate by endpoint, include failures, and exclude payloads and credentials (0.943217ms) ℹ tests 55 ℹ suites 0 ℹ pass 55 ℹ fail 0 ℹ cancelled 0 ℹ skipped 0 ℹ todo 0 ℹ duration_ms 3474.548684
- Run it: list devices before setting MOBILERUN_DEVICE_ID: Requires MOBILERUN_API_KEY and a ready Mobilerun Android device; neither is available in this guest. Ran instead: pnpm agent devices, which failed with 'A valid API key is required.' (exit 1).
Source: README.md Run it (pnpm devices)
Expected: documented command completes successfully
Observed: Requires MOBILERUN_API_KEY and a ready Mobilerun Android device; neither is available in this guest. Ran instead: pnpm agent devices, which failed with 'A valid API key is required.' (exit 1).
- Preview the next agent decision for a goal: Requires MOBILERUN_API_KEY, TYPESAFE_API_KEY and a ready device; no credentials are available in this guest, so no live model or device request was made.
Source: README.md Use the CLI (pnpm agent run)
Expected: documented command completes successfully
Observed: Requires MOBILERUN_API_KEY, TYPESAFE_API_KEY and a ready device; no credentials are available in this guest, so no live model or device request was made.
- Run the dark-theme demo: Requires live Mobilerun and TypeSafe credentials plus an Android device; not run. Deferred on purpose.
Source: README.md Included demo (pnpm demo dark-theme --reset)
Expected: documented command completes successfully
Observed: Requires live Mobilerun and TypeSafe credentials plus an Android device; not run. Deferred on purpose.
Local or hosted: local
Suggested next step (agent-reported): Re-run with MOBILERUN_API_KEY, TYPESAFE_API_KEY and MOBILERUN_DEVICE_ID set in .env.local (or exported): pnpm doctor, then pnpm agent run with --execute and pnpm demo dark-theme --reset. Separately, the repo's format check should exclude generated checker output (or the harness should write observed.json outside the repo) so pnpm check does not fail on it.
What the agent did
Install and build the Mobile Jev studio from the README, verify the offline test/lint/typecheck/format/build suite, and start the local studio UI (pnpm start, http://127.0.0.1:3040). The headline operation (driving a real Android phone through Mobilerun and TypeSafe) was not exercised.
cd <guest-path> && pwd && ls -la && id -u && cat /etc/os-release | head -3 && python3 --version && ls /inputs 2>/dev/null; ls <guest-path> 2>/dev/null
cd <guest-path> && find . -maxdepth 2 -not -path './.git*' | head -100 && cat README.md 2>/dev/null | head -200
cd <guest-path> && which node npm pnpm corepack curl; node --version 2>&1; cat .nvmrc; cat package.json; ls -la .coldpass; cat AGENTS.md | head -40; cat .env.example
cd <guest-path> && git init -q 2>&1; git remote add origin https://github.com/droidrun/mobile-jev 2>&1; timeout 300 git fetch --depth 1 origin 395fc222beac4f059f9a0beb337d114a2b066e99 2>&1 | tail -5; git reset --mixed FETCH_HEAD 2>&1 | tail -2; git log --oneline -1 2>&1
cd /tmp && curl -fsSL -o node24.tar.xz https://nodejs.org/dist/latest-v24.x/SHASUMS256.txt 2>&1 | tail -2; grep -o 'node-v24[^ ]*linux-x64.tar.xz' SHASUMS256.txt 2>/dev/null | head -1
cd /tmp && ls -la SHASUMS256.txt node24.tar.xz 2>&1; head -c 300 SHASUMS256.txt 2>/dev/null; echo; curl -sI https://nodejs.org/dist/ | head -3; env | grep -i proxy | sed 's/=.*@/=***@/'
cd /tmp && rm -f node24.tar.xz && curl -fsSL https://nodejs.org/dist/latest-v24.x/SHASUMS256.txt -o /tmp/SHASUMS256.txt; echo rc=$?; grep 'linux-x64.tar.xz' /tmp/SHASUMS256.txt
mkdir -p /tmp/node24 && cd /tmp/node24 && curl -fsSL -o node.tar.xz https://nodejs.org/dist/v24.22.1/node-v24.22.1-linux-x64.tar.xz && echo "6c172a9ab6718f7b801fb5a7bd486af2ee5e32758086a04b84fcce2aa77095e6 node.tar.xz" | sha256sum -c - && tar -xJf node.tar.xz --strip-components=1 && export PATH=/tmp/node24/bin:$PATH && node --version && npm --version && corepack --version && corepack enable 2>&1 | tail -2; export PATH=/tmp/node24/bin:$PATH; pnpm --version 2>&1
export PATH=/tmp/node24/bin:$PATH; cd <guest-path> && corepack use [email protected] >/dev/null 2>&1; git checkout -- package.json 2>/dev/null; git status --short | head; corepack enable 2>&1|tail -1; pnpm --version; ls -la /tmp/node24/bin | head -3
export PATH=/tmp/node24/bin:$PATH NODE_USE_ENV_PROXY=1; cd <guest-path> && timeout 900 pnpm install --frozen-lockfile 2>&1 | tail -25; echo rc=${PIPESTATUS[0]}
export PATH=/tmp/node24/bin:$PATH; cd <guest-path> && cp -n .env.example .env.local && timeout 900 pnpm check > /tmp/check.log 2>&1; echo rc=$?; tail -40 /tmp/check.log
export PATH=/tmp/node24/bin:$PATH; cd <guest-path> && grep -E '^# (tests|pass|fail)' /tmp/check.log; export PATH=/tmp/node24/bin:$PATH; timeout 60 pnpm agent --help 2>&1 | head -60; echo rc=$?; ls scripts/mobile-agent; cat scripts/run.mjs | head -40
export PATH=/tmp/node24/bin:$PATH; cd <guest-path> && timeout 600 pnpm test 2>&1 | grep -E '^# (tests|pass|fail|suites)'; echo; timeout 60 pnpm agent devices 2>&1 | tail -5; echo rc=$?; cat .env.local | sed 's/=.*/=<set-or-empty>/'
export PATH=/tmp/node24/bin:$PATH; cd <guest-path> && timeout 600 pnpm test > /tmp/test.log 2>&1; echo rc=$?; grep -E '^ℹ (tests|pass|fail)' /tmp/test.log; tail -3 /tmp/test.log; (cd <guest-path> && nohup pnpm start > /tmp/studio.log 2>&1 &) ; sleep 8; curl -s -o /tmp/root.html -w 'HTTP %{http_code}\n' http://127.0.0.1:3040/; grep -o '<title>[^<]*' /tmp/root.html | head -2; grep -o 'Run task' /tmp/root.html | head -1; tail -5 /tmp/studio.log
cd <guest-path> && find . -path ./node_modules -prune -o -path ./.next -prune -o -type f -print | head -50; grep -rn "pathname\|router\|'/runs\|\"/runs\|/api/" --include=*.ts --include=*.mjs server app 2>/dev/null | grep -v test | head -40
cd <guest-path> && cat "app/api/studio/[...path]/route.ts" | head -120; echo ----; curl -s -w '\nHTTP %{http_code}\n' http://127.0.0.1:3040/api/studio/runs; curl -s -w '\nHTTP %{http_code}\n' http://127.0.0.1:3040/api/studio/device | head -c 400
cd <guest-path> && grep -rn "setInterval\|cron\|--once\|--now\|webhook\|schedule" --include=*.mjs --include=*.ts --include=*.tsx scripts apps/jev-studio/server apps/jev-studio/app apps/jev-studio/launch.mjs 2>/dev/null | grep -v test | head; echo ---; grep -n "CI runs offline\|pnpm check #\|binds to localhost\|Open \*\*\|Run task\|offline\|CI requires no API keys" README.md; echo ---; pkill -f "next start" ; pkill -f "pnpm start"; sleep 2; curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:3040/ || true
cd <guest-path> && grep -rn "setInterval\|cron\|--once\|--now\|webhook\|schedule" --include=*.mjs --include=*.ts --include=*.tsx scripts apps/jev-studio/server apps/jev-studio/app apps/jev-studio/launch.mjs 2>/dev/null | grep -v test | head; echo ---; grep -n "CI runs offline\|pnpm check #\|Open \*\*\|Run task\|CI requires no API keys" README.md; echo ---; for p in $(pgrep -f 'next-server|next start|next/dist' ); do echo pid $p; done; (ss -ltnp 2>/dev/null || netstat -ltnp 2>/dev/null) | grep 3040
cd <guest-path> && for pid in $(ss -ltnp | grep ':3040 ' | grep -o 'pid=[0-9]*' | cut -d= -f2); do kill $pid; done; sleep 2; ss -ltnp | grep -c ':3040 ' ; pgrep -fa 'next-server' | grep -v pgrep | head -3; echo stopped
cd <guest-path> && cat > probe.json <<'EOF'
{"kind":"http","argv":["pnpm","start"],"cwd":"<guest-path>","url":"http://127.0.0.1:3040/","env":{"PATH":"/tmp/node24/bin:/usr/local/bin:/usr/bin:/bin"},"expected_text":"Run task"}
EOF
cat > claims.json <<'EOF'
[
{"kind":"claim","claim":"CI runs offline tests on Linux","source":"CI runs offline tests and the build on Linux.","input":"pnpm test (offline test suite, no API keys)","expected":"all tests pass","probe_kind":"cli","argv":["pnpm","test"],"expected_text":"pass 55","cwd":"<guest-path>","env":{"PATH":"/tmp/node24/bin:/usr/local/bin:/usr/bin:/bin"}},
{"kind":"claim","claim":"pnpm check runs tests, lint, typecheck, formatting and production build","source":"tests, lint, typecheck, formatting, production build","input":"pnpm check","expected":"formatting check and Next.js production build succeed","probe_kind":"cli","argv":["pnpm","check"],"expected_text":"Compiled successfully","cwd":"<guest-path>","env":{"PATH":"/tmp/node24/bin:/usr/local/bin:/usr/bin:/bin"}}
]
EOF
cat > examples.json <<'EOF'
[
{"kind":"example","claim":"Development: run the offline test suite","source":"README.md Development (pnpm check)","input":"pnpm test","expected":"55 tests pass, 0 fail","argv":["pnpm","test"],"cwd":"<guest-path>","env":{"PATH":"/tmp/node24/bin:/usr/local/bin:/usr/bin:/bin"},"expected_text":"pass 55"},
{"kind":"example","claim":"Run it: list devices before setting MOBILERUN_DEVICE_ID","source":"README.md Run it (pnpm devices)","reason":"Requires MOBILERUN_API_KEY and a ready Mobilerun Android device; neither is available in this guest. Ran instead: pnpm agent devices, which failed with 'A valid API key is required.' (exit 1)."},
{"kind":"example","claim":"Preview the next agent decision for a goal","source":"README.md Use the CLI (pnpm agent run)","reason":"Requires MOBILERUN_API_KEY, TYPESAFE_API_KEY and a ready device; no credentials are available in this guest, so no live model or device request was made."},
{"kind":"example","claim":"Run the dark-theme demo","source":"README.md Included demo (pnpm demo dark-theme --reset)","reason":"Requires live Mobilerun and TypeSafe credentials plus an Android device; not run. Deferred on purpose."}
]
EOF
cat > cycles.json <<'EOF'
{"detected":false,"reason":"Entry points inspected: scripts/run.mjs, scripts/mobile-agent/cli.mjs, scripts/demo.mjs, apps/jev-studio/app/api/studio/[...path]/route.ts. The only timers are a UI clock (use-studio.ts setInterval) and an SSE heartbeat (route.ts setInterval). No cron, scheduler, --once/--now, or webhook. The agent loop runs only when a user invokes pnpm agent run --execute, and it needs live credentials.","checks":[]}
EOF
for f in probe claims examples cycles; do python3 -I -m json.tool $f.json >/dev/null && echo "$f ok"; done
cd <guest-path> && export PATH=/tmp/node24/bin:$PATH && <env-var>=<guest-path> timeout 1500 python3 -I <guest-path> --validate --replay 2>&1 | tail -60; echo rc=${PIPESTATUS[0]}
export PATH=/tmp/node24/bin:$PATH; cd <guest-path> && node_modules/.bin/prettier --write .coldpass/probe.json .coldpass/claims.json .coldpass/examples.json .coldpass/cycles.json && node_modules/.bin/prettier --check .coldpass/*.json; cd .coldpass && cat > claims.json <<'EOF'
[
{"kind":"claim","claim":"Open the <email>:3040 and it shows a Run task control","source":"Open **http://127.0.0.1:3040**. Enter a goal and press **Run task**.","input":"GET http://127.0.0.1:3040/ after pnpm start","expected":"studio page with Run task control","probe_kind":"http","argv":["pnpm","start"],"url":"http://127.0.0.1:3040/","expected_text":"Run task","cwd":"<guest-path>","env":{"PATH":"/tmp/node24/bin:/usr/local/bin:/usr/bin:/bin"}},
{"kind":"claim","claim":"pnpm check runs tests, lint, typecheck, formatting and production build","source":"tests, lint, typecheck, formatting, production build","input":"pnpm check","expected":"formatting check and Next.js production build succeed","probe_kind":"cli","argv":["pnpm","check"],"expected_text":"Compiled successfully","cwd":"<guest-path>","env":{"PATH":"/tmp/node24/bin:/usr/local/bin:/usr/bin:/bin"}}
]
EOF
cd .. && node_modules/.bin/prettier --write .coldpass/claims.json >/dev/null && node_modules/.bin/prettier --check .coldpass/*.json && python3 -I -m json.tool .coldpass/claims.json >/dev/null && echo claims-json-ok
cd <guest-path> && ls -la .coldpass; head -c 400 .coldpass/observed.json; echo; grep -n "observed.json" <guest-path> | head
cd <guest-path> && rm -f .coldpass/observed.json && export PATH=/tmp/node24/bin:$PATH && <env-var>=<guest-path> timeout 1500 python3 -I <guest-path> --validate --replay > /tmp/validate1.json 2>&1; echo rc=$?; python3 -I -c "
import json,sys
d=json.load(open('/tmp/validate1.json')) if open('/tmp/validate1.json').read(1)=='{' else None
print('valid', d and d.get('valid')); print('problems', d and d.get('problems'))
" 2>&1 | head -30; ss -ltnp | grep 3040 || echo no-port-3040
cd <guest-path> && python3 -I -c "
import json
d=json.load(open('/tmp/validate1.json'))
print(d.get('replay',{}).get('status'))
det=d.get('replay',{}).get('details',{})
for c in det.get('claims',[]): print('CLAIM', c['claim'][:50], c.get('verdict'), c.get('outcome','')[:90])
for c in det.get('example_checks',[]): print('EX', c['claim'][:50], c.get('verdict'))
for c in det.get('cycle_checks',[]): print('CY', c.get('verdict'))
print('probe', det.get('status'), det.get('working'))
"
cd <guest-path> && python3 -I -c "
import json
d=json.load(open('<guest-path>'))
print(json.dumps(d,indent=0)[:200])
" ; python3 -I -c "
import json
d=json.load(open('<guest-path>'))
for k,v in d.items():
if isinstance(v,list):
for c in v: print(k, c.get('claim','')[:45], '|', c.get('verdict'), '|', str(c.get('outcome',''))[:70])
" 2>/dev/null | head -20
cd <guest-path> && cat > report.json <<'EOF'
{
"outcome": "needs_credentials",
"fault_owner": "prerequisite",
"operation": "Install and build the Mobile Jev studio from the README, verify the offline test/lint/typecheck/format/build suite, and start the local studio UI (pnpm start, http://127.0.0.1:3040). The headline operation (driving a real Android phone through Mobilerun and TypeSafe) was not exercised.",
"operation_scope": "Offline: Linux Ubuntu 24.04 guest, Node 24.22.1, pnpm 10.30.1, no API keys, no Android device. Verified: pnpm install --frozen-lockfile, pnpm check (55 offline tests pass, lint, typecheck, prettier, production build), pnpm start serving the studio page with 'Run task'. Not verified: any live Mobilerun device operation, any TypeSafe model call, pnpm demo.",
"steps": [
"git init; git remote add origin https://github.com/droidrun/mobile-jev; git fetch --depth 1 origin 395fc222beac4f059f9a0beb337d114a2b066e99; git reset --mixed FETCH_HEAD",
"Installed Node 24.22.1 (linux-x64 tarball from nodejs.org, sha256 verified against SHASUMS256.txt) into /tmp/node24",
"corepack enable (with /tmp/node24/bin on PATH); pnpm 10.30.1 from packageManager field",
"pnpm install --frozen-lockfile",
"cp .env.example .env.local (no keys filled in)",
"pnpm check (runs pnpm test, lint, typecheck, format, build) -> exit 0",
"pnpm test -> 55 pass, 0 fail",
"pnpm start -> studio on http://127.0.0.1:3040 returns the 'Run task' page (HTTP 200)",
"pnpm agent devices -> fails with 'A valid API key is required.' (no MOBILERUN_API_KEY): documented credential-error check",
"Stopped the studio server after probing; port 3040 confirmed free"
],
"deviations": [
"README does not give a Node installer; installed Node 24 from the official nodejs.org tarball because the guest had no node.",
"First corepack call outside the project fetched pnpm 12.15.0; re-ran inside the project so pnpm 10.30.1 from packageManager was used. A 'corepack use' attempt that touched package.json was reverted with git checkout; git status clean.",
"Live README commands (pnpm devices, pnpm doctor, pnpm agent run/observe/screenshot/profile, pnpm demo dark-theme --reset) were not run: they need MOBILERUN_API_KEY, TYPESAFE_API_KEY and a ready Mobilerun device, none supplied. Recorded as blocked examples.",
"The checker writes .coldpass/observed.json; Prettier (run by pnpm check, format step) then flags it, which made pnpm check fail during replay. Removed the stale checker-generated file before the final validation run. Also ran prettier --write on this run's .coldpass/*.json files so they pass the repo format check.",
"First claim used 'pnpm test' as the operation; the checker rejected it as a unit suite, so the claim was changed to the studio HTTP endpoint."
],
"stuck_step": "pnpm devices / live agent and demo commands: need MOBILERUN_API_KEY, TYPESAFE_API_KEY and an Android device",
"main_blocker": "No Mobilerun or TypeSafe API keys and no ready Android device are available in this disposable guest.",
"plain_failure": "The app builds, passes its offline tests, and its web studio starts and loads. It cannot control a phone or run a task, because that needs a Mobilerun account key, a TypeSafe key, and a connected Android device, which this test machine does not have.",
"suggested_fix": "Re-run with MOBILERUN_API_KEY, TYPESAFE_API_KEY and MOBILERUN_DEVICE_ID set in .env.local (or exported): pnpm doctor, then pnpm agent run with --execute and pnpm demo dark-theme --reset. Separately, the repo's format check should exclude generated checker output (or the harness should write observed.json outside the repo) so pnpm check does not fail on it."
}
EOF
cd <guest-path> && export PATH=/tmp/node24/bin:$PATH && node_modules/.bin/prettier --write .coldpass/report.json >/dev/null && python3 -I -m json.tool .coldpass/report.json >/dev/null && echo report-ok && rm -f .coldpass/observed.json && <env-var>=<guest-path> timeout 1500 python3 -I <guest-path> --validate --replay > /tmp/validate2.json 2>&1; echo rc=$?; head -c 300 /tmp/validate2.json; echo; grep -o '"valid": [a-z]*' /tmp/validate2.json; grep -o '"problems": \[[^]]*\]' /tmp/validate2.json | head -c 300; echo; ss -ltnp | grep -c ':3040 '; git status --short | headrun JSON · checker evidence · agent report + check · transcript excerpt unavailable · error excerpt unavailable