Haiku 5.5
Native agent · unverifiedOperation / attempted scope partial checks
Agent-reported scope: Follow the agent-bench README quickstart (python -m pip install ., agentbench doctor, agentbench demo) and run it
Run details
- run
- TayHCode--agent-bench-claude-haiku-5-5-1792656967238797283
- date
- Oct 22, 2026
- time to working
- not established
- product hosting / execution
- Not recorded
- recorded operation scope
- partial checks
- model
- claude-haiku-5-5
- agent / version
- claude-code / 2.1.300 (Claude Code)
- platform
- windows
- reported cost
- $0.030976
- Recorded source
- https://github.com/TayHCode/agent-bench
- Backend
- hyperv
- Tool transport
- mcp-assigned-vm
- Requested source pin (not an observed revision)
- {"commit": "ad01b757a0565c35fa590fe531964019363e70d5", "url": "https://github.com/TayHCode/agent-bench"}
- Test version
- readme-1924d8f529c6fc4500abb29916f5c9ac942027957e7b82d725d5610040cd3b4a
- revision
- ad01b757a0565c35fa590fe531964019363e70d5
Whose fault: prerequisite · diagnosed
Windows guest lacks nested virtualization, so WSL 2 cannot run and agentbench cannot start.
Recorded blocker / warning
Step: wsl --install (README quickstart: native Windows must use WSL). WSL 2 cannot start in this guest.
Windows guest lacks nested virtualization, so WSL 2 cannot run and agentbench cannot start.
Safety: No problems traced to this repository code in these checks.
No problems traced to this repository code in these checks.
OSV scanned supported lockfiles; unpinned dependencies and unsupported formats are outside this check.
A short run cannot establish that code is safe.
Connections show IP addresses; proxies can hide the final hostname.
Tracing covers dispatched commands and their children, not the whole OS; guest root can interfere with tracing.
windows behavior observation needs an OS-native file/network/process collector; only the static scan ran.
No Linux syscall trace was collected.
Partial install / setup / connection checks
[
{
"status": "PASS",
"details": {
"working": false,
"argv": [
"C:\\coldpass\\venv\\Scripts\\agentbench.exe",
"doctor"
],
"kind": "cli",
"returncode": 1,
"stdout": "agentbench doctor (Windows, Python 3.14.0)\n BLOCKER platform: native Windows is not supported: use WSL (Windows Subsystem for Linux)\n fix: wsl --install (then open the Ubuntu app and run the same agentbench commands there)\n ok python: Python 3.14.0\n BLOCKER git: git is not installed\n fix: agentbench needs WSL on Windows: run `wsl --install`, then work inside WSL\n ok disk: 38 GB free\n note claude: claude is not installed\n fix: npm install -g @anthropic-ai/claude-code\n note codex: codex is not installed\n fix: npm install -g @openai/codex\n note uv: uv is not installed (only needed for tasks whose checks use it, such as the built-in code tasks)\n[excerpt truncated]",
"stderr": "",
"needs_credentials": false,
"needs_browser": false,
"scope": "partial: install"
},
"check": "install",
"optional": false
}
]Answers recorded for this run
- Quickstart needs WSL on native Windows: Prerequisite missing: WSL 2 cannot start in this Windows guest ('WSL2 is unable to start since virtualization is not enabled on this machine'), so agentbench cannot run here. agentbench doctor on native Windows printed: 'BLOCKER platform: native Windows is not supported: use WSL'.
Source: on native Windows use WSL (`wsl --install`).
Observed: Prerequisite missing: WSL 2 cannot start in this Windows guest ('WSL2 is unable to start since virtualization is not enabled on this machine'), so agentbench cannot run here. agentbench doctor on native Windows printed: 'BLOCKER platform: native Windows is not supported: use WSL'.
- demo builds a toy repo and runs stand-in agents: agentbench demo refuses to run on native Windows (exit 2: 'error: native Windows is not supported: use WSL'). Not runnable until WSL 2 works in the guest.
Source: `demo` builds a toy repo, hides the test that proves a bug fixed
Observed: agentbench demo refuses to run on native Windows (exit 2: 'error: native Windows is not supported: use WSL'). Not runnable until WSL 2 works in the guest.
Examples
- Documented offline unit tests run the product's regression cases: Missing prerequisite: WSL 2 is required. Ran this documented command with the venv Python on native Windows: 209 tests, 25 failures and 85 errors, mostly from the native-Windows refusal ('native Windows is not supported: use WSL'). Not a passing example, so it is not replayed.
Source: README.md: python -m unittest discover -s tests -q
Expected: documented command completes successfully
Observed: Missing prerequisite: WSL 2 is required. Ran this documented command with the venv Python on native Windows: 209 tests, 25 failures and 85 errors, mostly from the native-Windows refusal ('native Windows is not supported: use WSL'). Not a passing example, so it is not replayed.
Local or hosted: local
Suggested next step (unverified): Run this check on a Linux or macOS guest, or on a Windows guest with nested virtualization enabled so WSL 2 can start. Then install Ubuntu with wsl --install and run the README quickstart inside WSL. Optionally, the product could document the Windows limitation at the top of the README.
What the agent did
Follow the agent-bench README quickstart (python -m pip install ., agentbench doctor, agentbench demo) and run it
cd C:\coldpass\workspace; Get-ChildItem -Force | Select-Object Name; Write-Output '---'; Get-Content README.md -TotalCount 200; Write-Output '---'; C:\coldpass\python\python.exe --version
cd C:\coldpass\workspace; git status 2>&1 | Select-Object -First 3; Write-Output '---'; Get-ChildItem C:\coldpass -Force | Select-Object Name; Write-Output '---'; Get-ChildItem C:\coldpass\checker -Force | Select-Object Name
Get-Command winget,choco,scoop -ErrorAction SilentlyContinue | Select-Object Name,Source; Get-ChildItem C:\coldpass\workspace\.coldpass -Force -ErrorAction SilentlyContinue | Select-Object Name; Get-Content C:\coldpass\workspace\pyproject.toml -TotalCount 40; Get-ChildItem C:\coldpass\workspace\tests -Force | Select-Object -First 5 Name
$ErrorActionPreference='Continue'; $d="$env:USERPROFILE\tools\mingit"; New-Item -ItemType Directory -Force -Path $d | Out-Null; $url='https://github.com/git-for-windows/git/releases/download/v2.47.1.windows.1/MinGit-2.47.1-busybox-64-bit.zip'; try { Invoke-WebRequest -Uri $url -OutFile "$d\mingit.zip" -UseBasicParsing -TimeoutSec 300; Expand-Archive -Force "$d\mingit.zip" -DestinationPath $d; & "$d\cmd\git.exe" --version } catch { "ERR: $($_.Exception.Message)" }
$g="$env:USERPROFILE\tools\mingit\cmd\git.exe"; cd C:\coldpass\workspace; & $g init 2>&1; & $g remote add origin https://github.com/TayHCode/agent-bench 2>&1; & $g fetch --depth 1 origin ad01b757a0565c35fa590fe531964019363e70d5 2>&1; & $g reset --mixed FETCH_HEAD 2>&1; & $g rev-parse HEAD 2>&1; & $g status --short 2>&1 | Select-Object -First 15
Get-ChildItem Env: | Where-Object { $_.Name -like 'GIT*' } | ForEach-Object { "$($_.Name)=$($_.Value)" }; $g="$env:USERPROFILE\tools\mingit\cmd\git.exe"; Get-ChildItem Env: | Where-Object { $_.Name -like 'GIT_CONFIG*' } | ForEach-Object { Remove-Item "Env:$($_.Name)" }; cd C:\coldpass\workspace; & $g init 2>&1 | Out-String; & $g remote add origin https://github.com/TayHCode/agent-bench 2>&1 | Out-String; & $g fetch --depth 1 origin ad01b757a0565c35fa590fe531964019363e70d5 2>&1 | Out-String; & $g reset --mixed FETCH_HEAD 2>&1 | Out-String; & $g rev-parse HEAD 2>&1 | Out-String
cd C:\coldpass\workspace; & C:\coldpass\python\python.exe -m venv C:\coldpass\venv 2>&1 | Out-String; C:\coldpass\venv\Scripts\python.exe --version; C:\coldpass\venv\Scripts\python.exe -m pip install . 2>&1 | Out-String | Select-Object -Last 40
cd C:\coldpass\workspace; & C:\coldpass\venv\Scripts\agentbench.exe doctor 2>&1 | Out-String; Write-Output "rc=$LASTEXITCODE"; & C:\coldpass\venv\Scripts\agentbench.exe demo 2>&1 | Out-String; Write-Output "rc=$LASTEXITCODE"
Get-Command wsl -ErrorAction SilentlyContinue | Select-Object Source; wsl --status 2>&1 | Out-String; wsl -l -v 2>&1 | Out-String; (Get-CimInstance Win32_OperatingSystem).Caption; [Security.Principal.WindowsIdentity]::GetCurrent().Name; ([Security.Principal.WindowsPrincipal][Security.Principal.WindowsIdentity]::GetCurrent()).IsInRole([Security.Principal.WindowsBuiltInRole]::Administrator); Get-ComputerInfo -Property HyperVRequirementVirtualizationFirmwareEnabled,HyperVisorPresent -ErrorAction SilentlyContinue | Out-String
$p = Start-Process -FilePath "$env:SystemRoot\System32\wsl.exe" -ArgumentList '--install','-d','Ubuntu','--no-launch' -PassThru -Wait -ErrorAction SilentlyContinue; if (-not $p) { "wsl.exe not present at System32; trying dism"; dism.exe /online /enable-feature /featurename:Microsoft-Windows-Subsystem-Linux /all /norestart 2>&1 | Out-String; dism.exe /online /enable-feature /featurename:VirtualMachinePlatform /all /norestart 2>&1 | Out-String } else { "exit=$($p.ExitCode)" }; Get-Command wsl -ErrorAction SilentlyContinue | Select-Object Source
Test-Path "$env:SystemRoot\System32\wsl.exe"; & "$env:SystemRoot\System32\wsl.exe" --status 2>&1 | Out-String; & "$env:SystemRoot\System32\wsl.exe" -l -v 2>&1 | Out-String; (Get-ItemProperty 'HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Component Based Servicing\RebootPending' -ErrorAction SilentlyContinue) -ne $null
dism.exe /online /enable-feature /featurename:Microsoft-Windows-Subsystem-Linux /all /norestart 2>&1 | Out-String; dism.exe /online /enable-feature /featurename:VirtualMachinePlatform /all /norestart 2>&1 | Out-String; Write-Output "rc=$LASTEXITCODE"
Start-Process -FilePath shutdown.exe -ArgumentList '/r','/t','5','/f' -WindowStyle Hidden; "restart scheduled"
Start-Sleep -Seconds 20; Write-Output (Get-CimInstance Win32_OperatingSystem).LastBootUpTime; Test-Path C:\coldpass\workspace\.git; Test-Path C:\coldpass\venv\Scripts\agentbench.exe
$w="$env:SystemRoot\System32\wsl.exe"; $p = Start-Process -FilePath $w -ArgumentList '--install','-d','Ubuntu','--no-launch' -PassThru -WindowStyle Hidden -RedirectStandardOutput C:\coldpass\wsl-install.out -RedirectStandardError C:\coldpass\wsl-install.err; if (-not $p.WaitForExit(900000)) { $p.Kill(); "timed out" } else { "exit=$($p.ExitCode)" }; Get-Content C:\coldpass\wsl-install.out -Raw -ErrorAction SilentlyContinue | ForEach-Object { $_ -replace "`0","" }; Get-Content C:\coldpass\wsl-install.err -Raw -ErrorAction SilentlyContinue | ForEach-Object { $_ -replace "`0","" }; & $w -l -v 2>&1 | Out-String | ForEach-Object { $_ -replace "`0","" }
$w="$env:SystemRoot\System32\wsl.exe"; & $w --status 2>&1 | Out-String | ForEach-Object { $_ -replace "`0","" }; & $w --list --online 2>&1 | Out-String | ForEach-Object { $_ -replace "`0","" }; Get-Content C:\coldpass\wsl-install.err -Raw -ErrorAction SilentlyContinue | ForEach-Object { $_ -replace "`0","" }
Get-ComputerInfo -Property HyperVRequirementVirtualizationFirmwareEnabled,HyperVRequirementVMMonitorModeExtensions,HyperVRequirementSecondLevelAddressTranslation,HyperVisorPresent | Out-String; $w="$env:SystemRoot\System32\wsl.exe"; $p = Start-Process -FilePath $w -ArgumentList '--install','-d','Ubuntu','--no-launch' -PassThru -WindowStyle Hidden -RedirectStandardOutput C:\coldpass\wsl2.out -RedirectStandardError C:\coldpass\wsl2.err; $done = $p.WaitForExit(900000); "done=$done exit=$($p.ExitCode)"; (Get-Content C:\coldpass\wsl2.out,C:\coldpass\wsl2.err -Raw -ErrorAction SilentlyContinue) -replace "`0","" | Select-Object -Last 30; & $w -l -v 2>&1 | Out-String | ForEach-Object { $_ -replace "`0","" }
dism.exe /online /get-featureinfo /featurename:VirtualMachinePlatform 2>&1 | Out-String; dism.exe /online /get-featureinfo /featurename:Microsoft-Windows-Subsystem-Linux 2>&1 | Out-String; systeminfo.exe 2>&1 | Select-String -Pattern 'Hyper-V','Virtualization','virtual' | ForEach-Object { $_.Line }
$c = Get-Content C:\coldpass\checker\check.py -Raw; Write-Output "len=$($c.Length)"; ($c -split "`n") | Select-String -Pattern 'checks','install','testable','needs_|"reason"|credentials_error|stays_running|def ' | Select-Object -First 90 | ForEach-Object { $_.Line }
cd C:\coldpass\workspace; Select-String -Path src\agentbench\*.py,README.md,CONTRIBUTING.md -Pattern 'cron|schedul|--once|--now|webhook|loop|while True' -SimpleMatch:$false | Select-Object -First 20 | ForEach-Object { "$($_.Path):$($_.LineNumber): $($_.Line.Trim())" }; Write-Output '--- test cmds'; Select-String -Path CONTRIBUTING.md,README.md -Pattern 'pytest|unittest|make ' | ForEach-Object { "$($_.LineNumber): $($_.Line.Trim())" } | Select-Object -First 15; Write-Output '--- ci'; Get-ChildItem .github\workflows | Select-Object Name
cd C:\coldpass\workspace; (Get-Content README.md)[262..272] -join "`n"; Write-Output '--- src'; Get-ChildItem src -Recurse -Filter *.py | Select-Object -First 30 | ForEach-Object { $_.FullName.Replace('C:\coldpass\workspace\','') }; Write-Output '--- tests'; & C:\coldpass\venv\Scripts\python.exe -m unittest discover -s tests -q 2>&1 | Out-String | Select-Object -Last 25
cd C:\coldpass\workspace; $out = & C:\coldpass\venv\Scripts\python.exe -m unittest discover -s tests -q 2>&1 | Out-String; $out.Length; $out.Substring([Math]::Max(0,$out.Length-1500))
$ws='C:\coldpass\workspace'; New-Item -ItemType Directory -Force -Path "$ws\.coldpass" | Out-Null; $enc = New-Object Text.UTF8Encoding $false
$probe = @'
{"checks":[{"check":"install","kind":"cli","argv":["C:\\coldpass\\venv\\Scripts\\agentbench.exe","doctor"],"cwd":"C:\\coldpass\\workspace","expected_returncode":1,"expected_text":"native Windows is not supported"}]}
'@
$claims = @'
[{"kind":"claim","claim":"Quickstart needs WSL on native Windows","source":"on native Windows use WSL (`wsl --install`).","testable":false,"reason":"Prerequisite missing: WSL 2 cannot start in this Windows guest ('WSL2 is unable to start since virtualization is not enabled on this machine'), so agentbench cannot run here. agentbench doctor on native Windows printed: 'BLOCKER platform: native Windows is not supported: use WSL'."},
{"kind":"claim","claim":"demo builds a toy repo and runs stand-in agents","source":"`demo` builds a toy repo, hides the test that proves a bug fixed","testable":false,"reason":"agentbench demo refuses to run on native Windows (exit 2: 'error: native Windows is not supported: use WSL'). Not runnable until WSL 2 works in the guest."}]
'@
$cycles = @'
{"detected":false,"reason":"Inspected src/ (no cron, scheduler, --once, --now, loop or webhook trigger found by search), README.md and CONTRIBUTING.md. agentbench runs only when a user invokes a command (doctor, demo, start, plan, matrix, run, report, task, verify); no product cycle exists to trigger.","checks":[]}
'@
$examples = @'
[{"kind":"example","claim":"Documented offline unit tests run the product's regression cases","source":"README.md: python -m unittest discover -s tests -q","reason":"Ran 'C:\coldpass\venv\Scripts\python.exe -m unittest discover -s tests -q' on native Windows: 209 tests, 25 failures and 85 errors, mostly from the native-Windows refusal ('native Windows is not supported: use WSL'). Not a passing example; WSL 2 is required for this suite to be meaningful."}]
'@
[IO.File]::WriteAllText("$ws\.coldpass\probe.json", $probe.Trim(), $enc)
[IO.File]::WriteAllText("$ws\.coldpass\claims.json", $claims.Trim(), $enc)
[IO.File]::WriteAllText("$ws\.coldpass\cycles.json", $cycles.Trim(), $enc)
[IO.File]::WriteAllText("$ws\.coldpass\examples.json", $examples.Trim(), $enc)
foreach ($f in 'probe','claims','cycles','examples') { $t = Get-Content "$ws\.coldpass\$f.json" -Raw; try { $null = $t | ConvertFrom-Json; "$f ok" } catch { "$f BAD: $($_.Exception.Message)" } }
$ws='C:\coldpass\workspace'; $enc = New-Object Text.UTF8Encoding $false; $p="$ws\.coldpass\examples.json"; $t = (Get-Content $p -Raw).Replace('C:\coldpass\venv','C:\\coldpass\\venv'); [IO.File]::WriteAllText($p, $t.Trim(), $enc); try { $null = (Get-Content $p -Raw) | ConvertFrom-Json; "examples ok" } catch { "BAD: $($_.Exception.Message)" }; $env:COLDPASS_WORKSPACE='C:\coldpass\workspace'; & C:\coldpass\python\python.exe -I C:\coldpass\checker\check.py --validate --replay 2>&1 | Out-String | Select-Object -Last 80
$ws='C:\coldpass\workspace'; $enc = New-Object Text.UTF8Encoding $false; $p="$ws\.coldpass\examples.json"; $examples = @'
[{"kind":"example","claim":"Documented offline unit tests run the product's regression cases","source":"README.md: python -m unittest discover -s tests -q","argv":["C:/coldpass/venv/Scripts/python.exe","-m","unittest","discover","-s","tests","-q"],"cwd":"C:/coldpass/workspace","reason":"Ran this documented command on native Windows: 209 tests, 25 failures and 85 errors, mostly from the native-Windows refusal ('native Windows is not supported: use WSL'). Not a passing example; WSL 2 is required for this suite to be meaningful."}]
'@; [IO.File]::WriteAllText($p, $examples.Trim(), $enc); try { $null = (Get-Content $p -Raw) | ConvertFrom-Json; "examples ok" } catch { "BAD: $($_.Exception.Message)" }; $env:COLDPASS_WORKSPACE='C:\coldpass\workspace'; & C:\coldpass\python\python.exe -I C:\coldpass\checker\check.py --validate --replay 2>&1 | Out-String | ForEach-Object { $_.Substring(0, [Math]::Min(1500, $_.Length)) }
$ws='C:\coldpass\workspace'; $enc = New-Object Text.UTF8Encoding $false; $p="$ws\.coldpass\examples.json"; $examples = @'
[{"kind":"example","claim":"Documented offline unit tests run the product's regression cases","source":"README.md: python -m unittest discover -s tests -q","reason":"Missing prerequisite: WSL 2 is required. Ran this documented command with the venv Python on native Windows: 209 tests, 25 failures and 85 errors, mostly from the native-Windows refusal ('native Windows is not supported: use WSL'). Not a passing example, so it is not replayed."}]
'@; [IO.File]::WriteAllText($p, $examples.Trim(), $enc); $env:COLDPASS_WORKSPACE='C:\coldpass\workspace'; & C:\coldpass\python\python.exe -I C:\coldpass\checker\check.py --validate --replay 2>&1 | Out-String | ForEach-Object { $_.Substring(0, [Math]::Min(1200, $_.Length)) }
$ws='C:\coldpass\workspace'; $enc = New-Object Text.UTF8Encoding $false; $report = @'
{"outcome":"blocked","fault_owner":"prerequisite","operation":"Follow the agent-bench README quickstart (python -m pip install ., agentbench doctor, agentbench demo) and run it","operation_scope":"Native Windows 11 guest (Hyper-V VM, no nested virtualization). README says it is tested on macOS, Linux and WSL and that native Windows must use WSL (wsl --install). Python 3.14.0 is present (3.11+ needed). Git was missing and was installed user-local as MinGit 2.47.1.","steps":["git init; git remote add origin https://github.com/TayHCode/agent-bench; git fetch --depth 1 origin ad01b757a0565c35fa590fe531964019363e70d5; git reset --mixed FETCH_HEAD (pinned commit checked out)","C:\\coldpass\\python\\python.exe -m venv C:\\coldpass\\venv; C:\\coldpass\\venv\\Scripts\\python.exe -m pip install . (installed agentbench-0.0.1)","C:\\coldpass\\venv\\Scripts\\agentbench.exe doctor (BLOCKER: native Windows is not supported: use WSL; BLOCKER: git is not installed, reported before MinGit was on PATH)","C:\\coldpass\\venv\\Scripts\\agentbench.exe demo (refused: error: native Windows is not supported: use WSL, exit 2)","wsl --install -d Ubuntu --no-launch (after enabling Microsoft-Windows-Subsystem-Linux and VirtualMachinePlatform via dism and rebooting): no distribution installed; wsl --status says WSL2 is unable to start since virtualization is not enabled on this machine"],"deviations":["Git was not installed; installed user-local MinGit 2.47.1 from https://github.com/git-for-windows/git/releases/download/v2.47.1.windows.1/MinGit-2.47.1-busybox-64-bit.zip into %USERPROFILE%\\tools\\mingit. Cleared the guest's GIT_CONFIG_* variables in each shell because they broke git init.","Product refuses native Windows, so the README's WSL path was attempted: dism enable of Microsoft-Windows-Subsystem-Linux and VirtualMachinePlatform (rc 3010), guest reboot, then wsl --install -d Ubuntu --no-launch. The distribution was not installed and WSL 2 cannot start in this guest, so the documented quickstart could not run.","No substitute run was made: running the product through PYTHONPATH or a patched platform check would mean bypassing the product's own platform guard, and editing product source is not allowed.","Documented unittest suite (python -m unittest discover -s tests -q) run with the venv Python: 209 tests, 25 failures, 85 errors, 1 skipped, mostly from the native-Windows refusal. Recorded as a reason-only example, not replayed.","The product's own 'agentbench doctor' was run as partial install evidence (exit 1, expected blocker text); no operation succeeded, so no working claim is recorded."],"stuck_step":"wsl --install (README quickstart: native Windows must use WSL). WSL 2 cannot start in this guest.","main_blocker":"Windows guest lacks nested virtualization, so WSL 2 cannot run and agentbench cannot start.","plain_failure":"agent-bench does not run on native Windows, and the Windows test machine cannot run the Linux environment (WSL) that the project requires. Nothing was verified beyond installation and the product's own refusal message.","suggested_fix":"Run this check on a Linux or macOS guest, or on a Windows guest with nested virtualization enabled so WSL 2 can start. Then install Ubuntu with wsl --install and run the README quickstart inside WSL. Optionally, the product could document the Windows limitation at the top of the README."}
'@; [IO.File]::WriteAllText("$ws\.coldpass\report.json", $report.Trim(), $enc); try { $null = (Get-Content "$ws\.coldpass\report.json" -Raw) | ConvertFrom-Json; "report ok" } catch { "BAD: $($_.Exception.Message)" }; $env:COLDPASS_WORKSPACE='C:\coldpass\workspace'; & C:\coldpass\python\python.exe -I C:\coldpass\checker\check.py --validate --replay 2>&1 | Out-String | ForEach-Object { $_.Substring(0, [Math]::Min(300, $_.Length)) }run JSON · checker evidence · agent report + check · transcript excerpt · error excerpt unavailable