{ "checker": { "claims": [ { "claim": "lint exits 0 when every citation checks out on disk", "command": " lint examples/demo_antemortem.md --repo .", "evidence": "PASS -- demo_antemortem.md validates clean (schema + classifications)\n", "expected": "PASS report for the demo artifact", "input": "examples/demo_antemortem.md", "kind": "claim", "outcome": "Real input produced the expected result: PASS -- demo_antemortem.md validates clean", "returncode": 0, "source": "`lint` exits `0` if every citation checks out on disk, `1` if any is fabricated or stale.", "source_checked": true, "source_file": "README.md", "verdict": "yes" }, { "claim": "strict-evidence lint reports missing evidence hashes through the MCP lint tool", "command": " tools/call {\"tool\": \"lint\", \"arguments\": {\"document\": \"examples/demo_antemortem.md\", \"repo\": \".\", \"strict_evidence\": true}}", "evidence": "\u2026classification t1: missing evidence_hash\\\",\\n \\\"classification t1: cited range too large (44 lines; max 40)\\\",\\n \\\"classification t2: missing evidence_hash\\\",\\n \\\"classification t3: missing evidence_hash\\\",\\n \\\"new_trap t_new_1: missing evidence_hash\\\",\\n \\\"new_trap t_new_1: cited range too large (46 lines; max 40)\\\"\\n ],\\n \\\"checked\\\": 4\\n}\"}], \"isError\": false}", "expected": "violation naming missing evidence_hash", "input": "{\"tool\": \"lint\", \"arguments\": {\"document\": \"examples/demo_antemortem.md\", \"repo\": \".\", \"strict_evidence\": true}}", "kind": "claim", "outcome": "Real input produced the expected result: classification t1: missing evidence_hash", "source": "`--strict-evidence` on `lint` requires the cited *text*, not just the line range, to be unchanged.", "source_checked": true, "source_file": "README.md", "verdict": "yes" }, { "claim": "metrics reports the fabrication rate for the demo artifact", "command": " metrics examples/demo_antemortem.json --repo .", "evidence": "Fabrication rate: 0.0% of cited\n- classification t1 (GHOST): VERIFIED [src/antemortem/cli.py:1-44]\n- classification t2 (REAL): VERIFIED [src/antemortem/citations.py:15-25]\n- classification t3 (GHOST): VERIFIED [src/antemortem/decision.py:30-35]\n- classification t4 (UNRESOLVED): UNRESOLVED (no citation)\n- new_trap t_new_1 (NEW): VERIFIED [src/antemortem/schema.py:177-222]\nStatus: PASS\n", "expected": "Fabrication rate of 0.0 percent of cited", "input": "examples/demo_antemortem.json", "kind": "claim", "outcome": "Real input produced the expected result: Fabrication rate: 0.0% of cited", "returncode": 0, "source": "antemortem metrics examples/demo_antemortem.json --repo .", "source_checked": true, "source_file": "README.md", "verdict": "yes" } ], "cycle_checks": [ { "claim": "Scheduled or looping main behavior", "command": "", "evidence": "Inspected README.md and src/antemortem entrypoints (cli.py, mcp/server.py, commands/). No cron, loop, webhook, watch, --once or --now scheduler was found; antemortem is a one-shot CLI and MCP tool server.", "expected": "identify the product trigger", "input": "source inspection", "kind": "cycle", "outcome": "Inspected README.md and src/antemortem entrypoints (cli.py, mcp/server.py, commands/). No cron, loop, webhook, watch, --once or --now scheduler was found; antemortem is a one-shot CLI and MCP tool server.", "source": "repository entry points", "verdict": "yes" } ], "details": { "argv": [ "" ], "kind": "mcp", "scope": "product operation", "serverInfo": { "name": "antemortem", "version": "1.30.0" }, "tool_result": { "content": [ { "text": "{\n \"ok\": false,\n \"violations\": [\n \"classification t1: missing evidence_hash\",\n \"classification t1: cited range too large (44 lines; max 40)\",\n \"classification t2: missing evidence_hash\",\n \"classification t3: missing evidence_hash\",\n \"new_trap t_new_1: missing evidence_hash\",\n \"new_trap t_new_1: cited range too large (46 lines; max 40)\"\n ],\n \"checked\": 4\n}", "type": "text" } ], "isError": false }, "tool_result_checked": true, "tool_schema_bytes": 6026, "tools": [ "scaffold", "run", "lint" ], "working": true }, "evidence_strength": "readme_checked", "example_checks": [ { "claim": "documented offline demo replays a recon and prints the decision", "command": " examples/demo_replay.py", "deferred": false, "evidence": "l line ranges in bounds\n - UNRESOLVED: citation correctly null (t4)\n\n---- Decision gate (four-level enum) ----\nSAFE_TO_PROCEED no REAL findings; lint and gate checks passed\nPROCEED_WITH_GUARDS REAL findings exist, every one has remediation <- THIS RUN\nNEEDS_MORE_EVIDENCE too many UNRESOLVED relative to total findings\nDO_NOT_PROCEED high-severity REAL without remediation, OR critic CONTRADICTED\n\n---- Install ----\npip install antemortem\nApache 2.0 - test count reproducible with pytest collection - Anthropic / OpenAI / Gemini / OpenAI-compatible\n\n[demo_replay] elapsed: 55.3s", "expected": "PROCEED_WITH_GUARDS decision", "input": "examples/demo_replay.py", "kind": "example", "outcome": "command completed successfully", "returncode": 0, "source": "README.md Try it in 30 seconds", "verdict": "yes" }, { "claim": "documented offline test suite passes", "command": " -m pytest -q", "deferred": false, "evidence": "................................. [ 13%]\n........................................................................ [ 26%]\n........................................................................ [ 39%]\n........................................................................ [ 52%]\n........................................................................ [ 66%]\n........................................................................ [ 79%]\n........................................................................ [ 92%]\n........................................ [100%]", "expected": "tests pass", "input": "tests directory", "kind": "example", "outcome": "command completed successfully", "returncode": 0, "source": "README.md offline test suite (python -m pytest -q)", "verdict": "yes" } ], "status": "PASS" }, "deviations": [ "MCP probe: the default lint call returns only ok/violations/checked with no quoted text, and the checker matches expected_text against escaped JSON, so I used strict_evidence=true, which returns specific violation strings. ok:false is the product's real result on this demo artifact (no evidence_hash), not a failure of the run.", "MCP scaffold was tried as a probe but rejected by the checker because output_directory must match an argv destination; MCP tool arguments cannot be marked, so I dropped it and removed the empty parent I had created.", "Checker path: /inputs/coldpass-preflight.py does not exist in the VM, so I used as the task's checker instruction allowed.", "Pin not applied: the workspace is not a git repository and was already populated, so I did not run the git fetch/reset steps." ], "fault_owner": null, "main_blocker": null, "operation": "Install antemortem 0.12.0 from the README and run its offline demo, lint, metrics, and MCP lint tool on the bundled demo recon artifact", "operation_scope": "Offline only: no provider API key or network calls. Python 3.12.3 in a venv at (README requires 3.11+). Source is the workspace copy; the pinned commit was not fetched because the workspace was already present and is not a git repo.", "outcome": "working", "plain_failure": null, "steps": [ "python3 -m venv ", " install -e \".[mcp]\" (installed antemortem-0.12.0 and antemortem-mcp)", "PYTHONIOENCODING=utf-8 python examples/demo_replay.py (run with venv python)", "antemortem lint examples/demo_antemortem.md --repo . -> PASS", "antemortem metrics examples/demo_antemortem.json --repo . -> Fabrication rate 0.0%", "antemortem-mcp stdio server, tools/call lint with strict_evidence=true -> ok:false with violations, returned as product output", "python -m pytest -q (offline test suite, all passed)" ], "stuck_step": null, "suggested_fix": "None required for the tested operations. Note that run and scaffold were not exercised: run needs a provider API key and scaffold needs an output directory the checker cannot mark." }