{ "checker": { "claims": [ { "claim": "Copying an agent .toml file into the agent directory installs it", "command": "cp categories/01-core-development/backend-developer.toml ", "evidence": "name = \"backend-developer\"\ndescription = \"Use when a task needs scoped backend implementation or backend bug fixes after the owning path is known.\"\nmodel = \"gpt-5.6-sol\"\nmodel_reasoning_effort = \"high\"\nsandbox_mode = \"workspace-write\"\ndeveloper_instructions = \"\"\"\nOwn backend changes as production behavior with explicit data, auth, and failure-path integrity.\n\nWorking mode:\n1. Map entry point, domain logic boundary, and persistence side effects.\n2. Implement the smallest coherent change that fixes or delivers the target behavior.\n3. Validate behavior under normal and high-risk failure paths.\n\nF", "expected": "copied file in output contains name = \"backend-developer\"", "input": "categories/01-core-development/backend-developer.toml", "kind": "claim", "outcome": "Real input produced the expected result: name = \"backend-developer\"", "returncode": 0, "source": "Copy the `.toml` agent files you want into one of the directories above.", "source_checked": true, "source_file": "README.md", "verdict": "yes" } ], "cycle_checks": [ { "claim": "Scheduled or looping main behavior", "command": "", "evidence": "Entry points inspected: README.md, CONTRIBUTING.md, and all 175 .toml files. The repository is a static collection of Codex subagent definitions with no scheduler, cron, loop, webhook, --once or --now entry point; no CI workflow directory exists. Nothing runs on a schedule.", "expected": "identify the product trigger", "input": "source inspection", "kind": "cycle", "outcome": "Entry points inspected: README.md, CONTRIBUTING.md, and all 175 .toml files. The repository is a static collection of Codex subagent definitions with no scheduler, cron, loop, webhook, --once or --now entry point; no CI workflow directory exists. Nothing runs on a schedule.", "source": "repository entry points", "verdict": "yes" } ], "details": { "argv": [ "cp", "categories/01-core-development/backend-developer.toml", "" ], "expected_file": "", "file_checked": true, "file_excerpt": "name = \"backend-developer\"\ndescription = \"Use when a task needs scoped backend implementation or backend bug fixes after the owning path is known.\"\nmodel = \"gpt-5.6-sol\"\nmodel_reasoning_effort = \"high\"\nsandbox_mode = \"workspace-write\"\ndeveloper_instructions = \"\"\"\nOwn backend changes as production behavior with explicit data, auth, and failure-path integrity.\n\nWorking mode:\n1. Map entry point, domain logic boundary, and persistence side effects.\n2. Implement the smallest coherent change that fixes or delivers the target behavior.\n3. Validate behavior under normal and high-risk failure paths.\n\nFocus on:\n- request/event entry points and service boundary ownership\n- input validation and contract-safe output behavior\n- transaction boundaries and consistency guarantees\n- idempotency and retry behavior for side-effecting operations\n- authentication/authorization behavior in touched paths\n- logging, metrics, and operator-facing error visibility\n- backward compatibility for existing clients or downstream consumers\n\nImplementation checks:\n- avoid hidden side effects in shared helpers\n- keep domain logic centralized, not split across adapters/controllers\n- preserve existing behavior outside changed scope\n- make failure semantics explicit (timeouts, not found, conflict, transient failure)\n\nQuality checks:\n- validate one critical success path and one high-risk failure path\n- verify persistence and rollback behavior for changed write paths\n- ensure changed path still enforces auth/permission rules\n- call out environment dependencies not verifiable in local checks\n\nReturn:\n- files and backend path changed\n- behavior change summary\n- validation performed\n- residual risk and follow-up verification needed\n\nDo not broaden into unrelated refactors unless explicitly requested by the parent agent.\n\"\"\"\n", "kind": "cli", "needs_browser": false, "needs_credentials": false, "returncode": 0, "scope": "product operation", "stderr": "", "stdout": "", "working": true }, "evidence_strength": "readme_checked", "example_checks": [ { "claim": "Install a global Codex subagent with the README example command", "command": "cp categories/01-core-development/backend-developer.toml ", "deferred": false, "evidence": "", "expected": "file copied into the guest HOME agents directory (, since HOME=)", "input": "categories/01-core-development/backend-developer.toml", "kind": "example", "outcome": "command completed successfully with no output", "returncode": 0, "source": "README.md Installation example (mkdir -p ~/.codex/agents; cp categories/01-core-development/backend-developer.toml ~/.codex/agents/)", "verdict": "yes" } ], "status": "PASS" }, "deviations": [ "Git metadata was added with git init and fetch, as the task instructed; the README does not require it.", "The README example uses ~, which resolves to HOME= in this guest. The first replay example used an absolute path that does not exist, so it failed. I corrected the example to and re-ran; it passed.", "The probe and claim are plain file copies (cp). The task classes file tests as setup only, so they are not evidence of product behaviour. No real product operation exists to exercise.", "Codex was not installed. No Codex-native behaviour (agent discovery, delegation) was tested." ], "fault_owner": "repo", "main_blocker": null, "operation": "Install and use the Codex subagent definitions from awesome-codex-subagents (pinned 16f880a2a0486f880da75d2baffcd68af4cc4811)", "operation_scope": "Static repository of 175 Codex .toml subagent files. The only documented actions are copying .toml files into ~/.codex/agents/ or .codex/agents/ and then delegating to them inside Codex. Codex itself is external and not installed in this guest; the README gives no runnable install command for it. No server, CLI, or library is provided, and no tests or CI exist.", "outcome": "working", "plain_failure": "This repository contains no program to run. It is a set of agent instruction files. The only things the README asks you to do are copy those files into a folder, and Codex then reads them. I could copy and validate the files, but the agents themselves can only be exercised inside Codex, which is not part of this repository and was not set up here.", "steps": [ "git init; git remote add origin https://github.com/VoltAgent/awesome-codex-subagents; git fetch --depth 1 origin 16f880a2a0486f880da75d2baffcd68af4cc4811; git reset --mixed FETCH_HEAD (HEAD verified at pinned commit, clean tree)", "README Installation: mkdir -p ~/.codex/agents && cp categories/01-core-development/backend-developer.toml ~/.codex/agents/ (succeeded, with HOME=)", "CONTRIBUTING validation checklist: parsed all 175 categories/*/*.toml with python3 -I tomllib, checking name/description/developer_instructions (0 problems)", "COLDPASS checker --validate --replay: PASS, valid=true, no problems" ], "stuck_step": null, "suggested_fix": "If this should be testable, the repo would need a runnable check that Codex loads each agent (for example, a documented command that lists or invokes the installed agents), or an offline validator added to CI that the README points to. Otherwise, treat it as a content-only repo and mark it unsupported for runtime testing." }