coldpass

lukedawilson/agentic-dev-loop

GitHub ↗ · How this check works

Per-setup public labels: Linux / Luna: Works

Static status badge · SVG · read-only JSON. Embed with ![Coldpass status](https://coldpass.dev/badge/lukedawilson--agentic-dev-loop.svg).

agent-harness

Works

The recorded run passed on 1 setup.

Answers apply to each recorded setup, revision and operation. Earlier observations remain in Run details.

Tested on Linux
PlatformHaikuLuna
Linuxour test failed; no repo verdictyes

Linux / Haiku 5.5: our test failed; no repo verdict · Oct 27, 2026 · 78433a5708ae
Recorded operation: No working operation verified.

Linux / Luna: yes · Oct 27, 2026 · 78433a5708ae · codex / codex-cli 0.164.0
Recorded operation: ./run-demo.sh

Does what it says
1 of 1 claims checked out
  • The demo runs a scripted visualization replay without a pipeline.: Real input produced the expected result: Viz demo (success): http://127.0.0.1:8321

    Source: ./run-demo.sh # viz demo - scripted run replay, no pipeline needed

    Input: default success demo

    Expected: Viz demo (success): http://127.0.0.1:8321

    Command: ./run-demo.sh

    Observed: Viz demo (success): http://127.0.0.1:8321 Run finished - process exiting.

MCP
No
Local or hosted
local
Local: runs on your machine
Needs keys or browser
None seen
Maintained
Maintained
actively maintained: last commit 0 days ago; 195 commits and 0 merged PRs in 90 days; 0 open issues, 0 open PRs; latest release 2026-10-27

Last commit Oct 27, 2026

Setup time
10 min 34 s
10 min 34 s to a working result
Issue
none found in recorded check
Examples
1 of 1 ran as expected
  • Run scripted visualization demo replay: command completed successfully

    Source: README.md / Development environment

    Input: default success demo

    Expected: Viz demo (success): http://127.0.0.1:8321

    Command: ./run-demo.sh

    Observed: Viz demo (success): http://127.0.0.1:8321 Run finished - process exiting.

  • Shipped pipeline dry-run; attempted and blocked by a missing configured AIDLC unit: Ran the documented agentic-dev-loop agentic-dev-loop/pipeline.yaml --dry-run using an outside-workspace install. It exited 1 with FileNotFoundError: No unit directory named '7' under aidlc-docs/construction (available: plans, unit-1, unit-2, unit-3, unit-4, unit-5).

    Source: README.md / Usage

    Expected: documented command completes successfully

    Observed: Ran the documented agentic-dev-loop agentic-dev-loop/pipeline.yaml --dry-run using an outside-workspace install. It exited 1 with FileNotFoundError: No unit directory named '7' under aidlc-docs/construction (available: plans, unit-1, unit-2, unit-3, unit-4, unit-5).

Evidence
README behavior checked
Agent interfaces
CLI
CLI

Observed interfaces were exercised by the checker. Documentation alone does not confirm compatibility.

Tested with
Codex / Luna
  • Linux: Codex / Luna · agent codex-cli 0.164.0 · Checked operation passed · Oct 27, 2026 on 78433a5708ae

Each entry describes its recorded operation and source revision.

Safety
No unexpected network or file activity found in these checks

No problems traced to this repository code in these checks; a short run cannot prove safety.

Reviewed as expected for a test run: attempted writes that failed, so nothing was written.

Full event, path and process lists for each observed setup:

Repair history and test corrections
Run details
Earlier attempts (1)

Luna

Native agent · Checked operation passed

Checked operation ./run-demo.sh

Agent-reported scope: Run the documented scripted visualization demo

Run details
run
lukedawilson--agentic-dev-loop-gpt-6-luna-1793124029568735458
date
Oct 27, 2026
time to working
634.0s
product hosting / execution
Local command in the guest
recorded operation scope
product operation
model
gpt-6-luna
agent / version
codex / codex-cli 0.164.0
platform
linux
reported cost
not reported (not assumed free)
Recorded source
https://github.com/lukedawilson/agentic-dev-loop
Backend
hyperv
Tool transport
mcp-assigned-vm
Requested source pin (not an observed revision)
{"commit": "78433a5708ae0ed22d9d6fcc8422d3666fb52d96", "url": "https://github.com/lukedawilson/agentic-dev-loop"}
Test version
readme-38e6b13f3b13804effb95797e3466b1a1334da7a232d897cbc8965867f802605
revision
78433a5708ae0ed22d9d6fcc8422d3666fb52d96
Whose fault: repo · diagnosed

The shipped pipeline's dry-run requests nonexistent AIDLC unit 7.

Safety: No problems traced to this repository code in these checks.

No problems traced to this repository code in these checks.

Reviewed as expected for a test run: attempted writes that failed, so nothing was written.

OSV scanned supported lockfiles; unpinned dependencies and unsupported formats are outside this check.

A short run cannot establish that code is safe.

Connections show IP addresses; proxies can hide the final hostname.

Tracing covers dispatched commands and their children, not the whole OS; guest root can interfere with tracing.

Full safety evidence →

Answers recorded for this run
  • The demo runs a scripted visualization replay without a pipeline.: Real input produced the expected result: Viz demo (success): http://127.0.0.1:8321

    Source: ./run-demo.sh # viz demo - scripted run replay, no pipeline needed

    Input: default success demo

    Expected: Viz demo (success): http://127.0.0.1:8321

    Command: ./run-demo.sh

    Observed: Viz demo (success): http://127.0.0.1:8321 Run finished - process exiting.

Examples

  • Run scripted visualization demo replay: command completed successfully

    Source: README.md / Development environment

    Input: default success demo

    Expected: Viz demo (success): http://127.0.0.1:8321

    Command: ./run-demo.sh

    Observed: Viz demo (success): http://127.0.0.1:8321 Run finished - process exiting.

  • Shipped pipeline dry-run; attempted and blocked by a missing configured AIDLC unit: Ran the documented agentic-dev-loop agentic-dev-loop/pipeline.yaml --dry-run using an outside-workspace install. It exited 1 with FileNotFoundError: No unit directory named '7' under aidlc-docs/construction (available: plans, unit-1, unit-2, unit-3, unit-4, unit-5).

    Source: README.md / Usage

    Expected: documented command completes successfully

    Observed: Ran the documented agentic-dev-loop agentic-dev-loop/pipeline.yaml --dry-run using an outside-workspace install. It exited 1 with FileNotFoundError: No unit directory named '7' under aidlc-docs/construction (available: plans, unit-1, unit-2, unit-3, unit-4, unit-5).

Local or hosted: local

What the agent did

Run the documented scripted visualization demo

pwd; git status --short; python3 --version; rg --files -g 'README*' -g 'pyproject.toml' -g 'package.json' -g 'Makefile' -g 'AGENTS.md' -g '.github/workflows/**' | head -80
ls -la; find . -maxdepth 3 -type f | head -100
head -100 README.md; find .coldpass -maxdepth 2 -type f -print; cat pyproject.toml
grep -nE 'agentic-dev-loop |--version|quick|demo|pipeline.yaml' README.md | head -50; docker version --format '{{.Server.Version}}' 2>&1 || true; sed -n '100,190p' README.md
python3 -m venv .venv && .venv/bin/pip install -e .
cd <guest-path> && .venv/bin/agentic-dev-loop agentic-dev-loop/pipeline.yaml --dry-run
cd /tmp && python3 -m venv /tmp/adl-venv && /tmp/adl-venv/bin/pip install <guest-path>
cd <guest-path> && /tmp/adl-venv/bin/agentic-dev-loop agentic-dev-loop/pipeline.yaml --dry-run
sed -n '530,560p' README.md; cat run-demo.sh;
cd <guest-path> && ./run-demo.sh
sed -n '1,45p' viz_demo.py; sed -n '1,150p' <guest-path> | head -3; ls /inputs
python3 -c 'import os,urllib.request; u=os.environ["CDP_URL"]+"/json/version"; r=urllib.request.urlopen(u); print(r.status); print(r.read().decode())'
mkdir -p .coldpass; cat > .coldpass/probe.json <<'EOF'
{"kind":"cli","argv":["./run-demo.sh"],"cwd":"<guest-path>","expected_text":"Viz demo (success): http://127.0.0.1:8321"}
EOF
cat > .coldpass/claims.json <<'EOF'
[{"kind":"claim","claim":"The demo runs a scripted visualization replay without a pipeline.","source":"./run-demo.sh                         # viz demo - scripted run replay, no pipeline needed","input":"default success demo","expected":"Viz demo (success): http://127.0.0.1:8321","probe_kind":"cli","argv":["./run-demo.sh"],"expected_text":"Viz demo (success): http://127.0.0.1:8321","cwd":"<guest-path>"}]
EOF
cat > .coldpass/cycles.json <<'EOF'
{"detected":false,"reason":"README and entry points describe an agent loop triggered by invoking the CLI; no cron, scheduler, webhook, or autonomous cycle trigger was found. The manual CLI run is documented in README Usage." ,"checks":[]}
EOF
cat > .coldpass/examples.json <<'EOF'
[{"kind":"example","claim":"Run scripted visualization demo replay","source":"README.md / Development environment","input":"default success demo","expected":"Viz demo (success): http://127.0.0.1:8321","argv":["./run-demo.sh"],"cwd":"<guest-path>","expected_text":"Viz demo (success): http://127.0.0.1:8321"},{"kind":"example","claim":"Validate the shipped pipeline without LLM or Docker","source":"README.md / Usage","input":"shipped pipeline with --dry-run","expected":"resolved plan","argv":["/tmp/adl-venv/bin/agentic-dev-loop","agentic-dev-loop/pipeline.yaml","--dry-run"],"cwd":"<guest-path>","expected_returncode":1,"expected_text":"No unit directory named '7'"}]
EOF
cat > .coldpass/report.json <<'EOF'
{"outcome":"working","fault_owner":null,"operation":"Run the documented scripted visualization demo","operation_scope":"Python 3.12 editable installation and documented run-demo script","steps":["Read README and verified Python 3.12.3 meets the >=3.12 requirement","Installed the repository editable into .venv for its documented demo entry point","Ran ./run-demo.sh; it printed the success URL and exited 0","Queried supplied CDP endpoint /json/version; HTTP 200"],"deviations":["The shipped pipeline dry-run was attempted using an outside-workspace install, then failed because the pipeline requests unit 7 while only units 1–5 exist under aidlc-docs/construction."],"stuck_step":null,"main_blocker":null,"plain_failure":null,"suggested_fix":null}
EOF
<env-var>=<guest-path> python3 -I <guest-path> --validate --replay
cat > .coldpass/examples.json <<'EOF'
[{"kind":"example","claim":"Run scripted visualization demo replay","source":"README.md / Development environment","input":"default success demo","expected":"Viz demo (success): http://127.0.0.1:8321","argv":["./run-demo.sh"],"cwd":"<guest-path>","expected_text":"Viz demo (success): http://127.0.0.1:8321"},{"kind":"example","claim":"Shipped pipeline dry-run; attempted and blocked by a missing configured AIDLC unit","source":"README.md / Usage","reason":"Ran the documented agentic-dev-loop agentic-dev-loop/pipeline.yaml --dry-run using an outside-workspace install. It exited 1 with FileNotFoundError: No unit directory named '7' under aidlc-docs/construction (available: plans, unit-1, unit-2, unit-3, unit-4, unit-5)."}]
EOF
python3 - <<'PY'
import json
p='.coldpass/report.json'
d=json.load(open(p)); d['outcome']='blocked'; d['fault_owner']='repo'; d['main_blocker']="The shipped pipeline's dry-run requests nonexistent AIDLC unit 7."; d['plain_failure']="The documented pipeline example cannot load because construction unit 7 is missing."; d['suggested_fix']='Update the pipeline to a present unit or add unit-7 docs.'; open(p,'w').write(json.dumps(d,indent=2)+'\n')
PY
<env-var>=<guest-path> python3 -I <guest-path> --validate --replay

No product verdict. Run record.

No product verdict. Run record.

No product verdict. Run record.