TayHCode/pstack-factory

Review signals: 1 hardcoded secret, 1 outside write.

Gitleaks found a possible hardcoded secret; value withheld. pstack/manifest.json

Repository processes attempted these writes outside documented output/cache paths. /etc 2 attributed processes.

OSV scanned supported lockfiles; unpinned dependencies and unsupported formats are outside this check.

A short run cannot establish that code is safe.

Connections show IP addresses; proxies can hide the final hostname.

Tracing covers dispatched commands and their children, not the whole OS; guest root can interfere with tracing.

Full safety evidence →

• Catalog lookup returns the architect skill entry: Real input produced the expected result: "piece": "skill/architect"

  Source: python3 pstack/catalog.py lookup architect

  Input: architect

  Expected: skill/architect entry

  Command: <guest-path> pstack/catalog.py lookup architect

  Observed: "piece": "skill/architect", "decision": "SKILL", "reason": "Choosing data types, ownership and module boundaries needs design judgment.", "effort": "M", "notes": "Both read the same guidance; replace Cursor Task/control calls through the overlay.", "paths": [ "pstack/upstream/skills/architect/SKILL.md" ], "enabled": true, "activation": "on-demand" }

• Fresh toy plan lists the rehearsal steps without writing: Real input produced the expected result: create disposable toy fixture

  Source: The first command prints the plan without writing.

  Input: --plan

  Expected: plan lists create disposable toy fixture

  Command: <guest-path> pstack/fresh_toy.py --out .scratch/claim-plan-out-checker-28be23b03ec5434b9dc404409c4d078b --plan

  Observed: create disposable toy fixture", "commit fixture locally", "run generic and product verification", "check sealed evidence" ], "external_effects": [], "owner_approval": false }

• Fresh toy rehearsal runs and passes end to end: Real input produced the expected result: PASS: tested source

  Source: python3 pstack/fresh_toy.py --out .scratch/fresh-toy-1

  Input: --out .scratch/claim-fresh-toy-run

  Expected: rehearsal prints PASS for tested source

  Command: <guest-path> pstack/fresh_toy.py --out .scratch/claim-fresh-toy-run-checker-9eca6de259974e9e85fdbe802c90b925

  Observed: PASS: tested source b5c8a50b0fdbcb5b3521be78b7e6f2b34b415bf1; current committed inputs and retained artifacts match { "version": 1, "outcome": "PASS", "source_policy": "command-boundaries", "steps": [ { "id": "behavior", "action": "drive", "expected": { "exit": 0, "stdout_contains": [ "FRESH_TOY PASS" ] }, "actual": { "exit": 0, "stdout_contains": { "FRESH_TOY PASS": true }, "stderr_contains": {} }, "outcome": "PASS", "artifacts_at_exit": { "behavior.o

Examples

• Documented dry-run of the fresh toy rehearsal prints its plan: command completed successfully

  Source: README.md: Fresh toy rehearsal (python3 pstack/fresh_toy.py --out .scratch/fresh-toy-1 --plan)

  Input: --plan

  Expected: prints the four planned steps

  Command: <guest-path> pstack/fresh_toy.py --out .scratch/example-plan-out-checker-dda75d9bea2b464f9c9b533ea6146772 --plan

  Observed: { "provenance": "SYNTHETIC_LOCAL_ONLY; no provider dispatch", "output": "<guest-path>", "steps": [ "create disposable toy fixture", "commit fixture locally", "run generic and product verification", "check sealed evidence" ], "external_effects": [], "owner_approval": false }

Local or hosted: local

Run pstack Factory's README-documented local workflow: install the package, install native skill links, look up a catalog entry, and run the fresh synthetic toy rehearsal (python3 pstack/fresh_toy.py --out ...), which builds a toy fixture, runs behavior checks, and validates sealed evidence.

cd <guest-path> && pwd && ls -la && git status 2>&1 | head -3; id -u; python3 --version; cat /etc/os-release | head -3; echo; cat README.md 2>/dev/null | head -200
ls <guest-path> /inputs 2>&1; ls -la <guest-path>
cd <guest-path> && which git && git --version; python3 -m venv --help >/dev/null 2>&1 && echo venv_ok || echo venv_missing; python3 -m pip --version 2>&1 | head -1; cat pyproject.toml; ls pstack loop tests | head -80
cd <guest-path> && git init -q && git remote add origin https://github.com/TayHCode/pstack-factory && timeout 120 git fetch --depth 1 origin 515d20342d52b0bf82e6828237595586d9a694db 2>&1 | tail -5; echo "rc=$?"; git reset --mixed FETCH_HEAD 2>&1 | tail -2; git log --oneline -1 2>&1; git status --short | head -5
cd <guest-path> && python3 -m venv <guest-path> 2>&1 | tail -3; ls <guest-path> 2>/dev/null | head -20
rm -rf <guest-path> && sudo -n apt-get update -qq 2>&1 | tail -2; sudo -n DEBIAN_FRONTEND=noninteractive apt-get install -y -qq python3-venv python3-pip 2>&1 | tail -3; cd <guest-path> && python3 -m venv <guest-path> && <guest-path> --version && <guest-path> -m pip --version
cd <guest-path> && export PATH=<guest-path> && python -m pip install . "pytest>=9,<10" 2>&1 | tail -4 && python pstack/install_native.py 2>&1 | tail -5; echo "rc=$?"
cd <guest-path> && export PATH=<guest-path> && python3 pstack/budget.py show 2>&1 | head -30; echo "--- lookup"; python3 pstack/catalog.py lookup architect 2>&1 | head -20; echo "--- integrity"; python3 pstack/check_integrity.py 2>&1 | tail -5; echo "rc=$?"; echo "--- installation"; python3 pstack/check_installation.py 2>&1 | tail -5; echo "rc=$?"
cd <guest-path> && export PATH=<guest-path> && timeout 900 flock /tmp/factory-pstack-tests.lock python3 -m pytest -q 2>&1 | tail -15; echo "rc=${PIPESTATUS[0]}"
mkdir -p <guest-path> && cd <guest-path> && export PATH=<guest-path> && nohup sh -c 'flock /tmp/factory-pstack-tests.lock python3 -m pytest -q -rf > <guest-path> 2>&1; echo EXIT=$? >> <guest-path>' >/dev/null 2>&1 &
echo started
cd <guest-path> && cat .coldpass/* 2>/dev/null | head -5; ls -la .coldpass; echo ---; grep -n -i "cli\|--out\|doctor\|evidence" README.md loop/README.md | head -30; echo; sed -n 1,80p loop/README.md
wc -l <guest-path>; grep -n "def \|argv\|kind\|cycles\|expected_file" <guest-path> | head -120
cd <guest-path> && sed -n 325,347p checker/check.py; echo ....; sed -n 657,668p checker/check.py; echo ....; grep -n "cycles\|examples.json\|claims.json\|probe.json\|report.json\|'--validate'\|--replay" checker/check.py | head -60; echo; tail -3 logs/pytest.log 2>/dev/null
cd <guest-path> && grep -n "UNIT_SUITE =\|PARTIAL_CHECKS =\|INTERPRETER =\|INLINE_CODE =" checker/check.py; sed -n 960,1010p checker/check.py; sed -n 747,770p checker/check.py
sleep 240; tail -c 1500 <guest-path>
sleep 600; tail -c 2000 <guest-path>; echo; pgrep -fa pytest | head -3
cd <guest-path> && <guest-path> pstack/fresh_toy.py --out .scratch/fresh-toy-1 --plan 2>&1 | head -60; echo "rc=${PIPESTATUS[0]}"; ls -la .scratch 2>&1
for i in $(seq 1 50); do grep -q EXIT= <guest-path> 2>/dev/null && break; sleep 20; done; tail -c 3000 <guest-path> | tail -40
for i in $(seq 1 85); do grep -q EXIT= <guest-path> 2>/dev/null && break; sleep 10; done; tail -c 1200 <guest-path>
for i in $(seq 1 85); do grep -q EXIT= <guest-path> 2>/dev/null && break; sleep 10; done; tail -c 2500 <guest-path>
cd <guest-path> && git log --oneline -3; git status --short | head; echo ---; grep -n "Error\|assert\|FAILED\|def test_source_commit" <guest-path> | head -30
cd <guest-path> && sed -n 120,145p tests/pstack/test_update_workflow.py; sed -n 47,52p <guest-path> | cut -c1-400
cd <guest-path> && ls -d .scratch/fresh-toy-1 2>&1; timeout 1500 <guest-path> pstack/fresh_toy.py --out .scratch/fresh-toy-1 > <guest-path> 2>&1; echo "rc=$?"; tail -c 3000 <guest-path>
cd <guest-path> && sed -n 930,958p checker/check.py; sed -n 1228,1250p checker/check.py
cd <guest-path> && grep -n "def claim_verdict" -A 22 checker/check.py | head -40; grep -n "def probe(" -A 12 checker/check.py | head -30
cd <guest-path> && grep -rn -i "cron\|schedule\|--once\|--now\|while True\|webhook\|sleep(" --include=*.py --include=*.md loop pstack factory 2>/dev/null | grep -v "upstream/" | head -20; echo ---; wc -c pstack/*.recipe.json pstack/recipe.json; echo ---; sed -n 1,12p loop/README.md | head -3; grep -n "factory_verify.py run\|fresh_toy" -r docs/*.md | head -5
cd <guest-path> && sed -n 1,20p factory/hooks.py; echo ...; sed -n 50,80p factory/hooks.py; echo ---; grep -rn -i "hook" README.md docs/SETUP.md | head -5
cd <guest-path> && rm -rf .scratch/fresh-toy-1 && mkdir -p .coldpass && PY=<guest-path> && cat > .coldpass/probe.json <<EOF
{"kind":"cli","argv":["$PY","pstack/fresh_toy.py","--out",".scratch/probe-fresh-toy"],"cwd":"<guest-path>","expected_text":"PASS: tested source"}
EOF
cat > .coldpass/claims.json <<EOF
[
{"kind":"claim","claim":"Catalog lookup returns the architect skill entry","source":"python3 pstack/catalog.py lookup architect","input":"architect","expected":"skill/architect entry","probe_kind":"cli","argv":["$PY","pstack/catalog.py","lookup","architect"],"expected_text":"\"piece\": \"skill/architect\"","cwd":"<guest-path>"},
{"kind":"claim","claim":"Fresh toy plan lists the rehearsal steps without writing","source":"The first command prints the plan without writing.","input":"--plan","expected":"plan lists create disposable toy fixture","probe_kind":"cli","argv":["$PY","pstack/fresh_toy.py","--out",".scratch/claim-plan-out","--plan"],"expected_text":"create disposable toy fixture","cwd":"<guest-path>"},
{"kind":"claim","claim":"Fresh toy rehearsal runs and passes end to end","source":"python3 pstack/fresh_toy.py --out .scratch/fresh-toy-1","input":"--out .scratch/claim-fresh-toy-run","expected":"rehearsal prints PASS for tested source","probe_kind":"cli","argv":["$PY","pstack/fresh_toy.py","--out",".scratch/claim-fresh-toy-run"],"expected_text":"PASS: tested source","cwd":"<guest-path>"}
]
EOF
cat > .coldpass/examples.json <<EOF
[{"kind":"example","claim":"Documented dry-run of the fresh toy rehearsal prints its plan","source":"README.md: Fresh toy rehearsal (python3 pstack/fresh_toy.py --out .scratch/fresh-toy-1 --plan)","input":"--plan","expected":"prints the four planned steps","argv":["$PY","pstack/fresh_toy.py","--out",".scratch/example-plan-out","--plan"],"cwd":"<guest-path>","expected_text":"create disposable toy fixture"}]
EOF
cat > .coldpass/cycles.json <<'EOF'
{"detected":false,"reason":"Entry points inspected: README documents only one-shot CLI commands (catalog lookup, budget show, fresh_toy.py, factory_verify.py run, check_evidence.py). No README-documented cron, schedule, loop, webhook, --once or --now trigger exists. factory/hooks.py has an event-triggered after-change runner with a while loop, but the README does not document it and it is not a scheduled product cycle, so no cycle was triggered."
,"checks":[]}
EOF
for f in probe claims examples cycles; do <guest-path> -m json.tool .coldpass/$f.json >/dev/null && echo "$f ok"; done
cd <guest-path> && COLDPASS_WORKSPACE=<guest-path> timeout 1500 python3 -I <guest-path> --validate --replay > <guest-path> 2>&1; echo "rc=$?"; head -c 6000 <guest-path>
cd <guest-path> && python3 -c "
import json,sys
d=json.load(open('<guest-path>'))
print('valid',d['valid'],'problems',d['problems'])
r=d.get('replay',{})
print('replay',r.get('status'))
for k in ('claims','example_checks','cycle_checks'):
  for c in r.get('details',{}).get(k,[]) or []: print(k, c.get('verdict'), c.get('outcome','')[:110])
" 2>&1 | head -30; tail -c 300 <guest-path>
cd <guest-path> && mkdir -p .coldpass && cat > <guest-path> <<'EOF'
import json
report = {
 "outcome": "working",
 "fault_owner": "repo",
 "operation": "Run pstack Factory's README-documented local workflow: install the package, install native skill links, look up a catalog entry, and run the fresh synthetic toy rehearsal (python3 pstack/fresh_toy.py --out ...), which builds a toy fixture, runs behavior checks, and validates sealed evidence.",
 "operation_scope": "Local only, no network or model calls. Ubuntu 24.04, Python 3.12.3 in a venv at <guest-path> (python3-venv installed via sudo apt-get; system Python had no pip). Git 2.43.0. pytest 9.1.1. Source pinned to commit 515d20342d52b0bf82e6828237595586d9a694db via git init/remote/fetch --depth 1/reset --mixed.",
 "steps": [
  "git init; git remote add origin https://github.com/TayHCode/pstack-factory; git fetch --depth 1 origin 515d20342d52b0bf82e6828237595586d9a694db; git reset --mixed FETCH_HEAD",
  "sudo -n apt-get install -y python3-venv python3-pip; python3 -m venv <guest-path>",
  "python -m pip install . \"pytest>=9,<10\"  (inside venv)",
  "python pstack/install_native.py  -> PASS: five repository-local native skill links installed",
  "python3 pstack/budget.py show",
  "python3 pstack/catalog.py lookup architect",
  "python3 pstack/check_integrity.py -> PASS: 169 upstream files match the manifest",
  "python3 pstack/check_installation.py -> PASS",
  "flock /tmp/factory-pstack-tests.lock python3 -m pytest -q  -> 1512 passed, 4 skipped, 501 subtests passed, 1 failed",
  "python3 pstack/fresh_toy.py --out .scratch/fresh-toy-1 --plan",
  "python3 pstack/fresh_toy.py --out .scratch/fresh-toy-1  -> PASS fresh-synthetic-toy 1/1; PASS: tested source"
 ],
 "deviations": [
  "System python3 had no pip and ensurepip was missing; installed python3-venv and python3-pip with sudo apt-get and used a venv instead of a bare system pip install.",
  "Used the absolute venv python (<guest-path>) for probe/claim argv instead of bare python3 so replay uses the same interpreter.",
  "pytest full suite took about 53 minutes and exceeded the 900s tool timeout on the first attempt; reran in background with a log file.",
  "Full pytest run: 1 failed test: tests/pstack/test_update_workflow.py::test_source_commit_precedes_real_runner_and_durable_evidence_is_pushed. The assertion (bundle recipe.json bytes == repo pstack/recipe.json bytes) failed at byte index 88 (a check argv differs). Not hand-edited; reported as a repo test failure. The README's own fresh toy rehearsal passed.",
  "The test suite created commits in its own temporary repositories; the workspace HEAD stayed at 515d203.",
  "Did not run the README's 'Verify a committed extraction' recipe (factory_verify.py run pstack/extraction.recipe.json) because it re-runs the full suite and needs a clean worktree; not run.",
  "Cycle: no scheduler, loop, webhook, --once or --now trigger is documented in the README; factory/hooks.py is an undocumented after-change event runner, so no cycle was triggered (detected:false)."
 ],
 "stuck_step": None,
 "main_blocker": None,
 "plain_failure": None,
 "suggested_fix": "Investigate the recipe.json byte mismatch in tests/pstack/test_update_workflow.py (test_source_commit_precedes_real_runner_and_durable_evidence_is_pushed); the documented install, catalog lookup and fresh toy rehearsal already pass without it."
}
with open('<guest-path>','w',encoding='utf-8') as f:
    json.dump(report,f,indent=2,ensure_ascii=False); f.write('\n')
print('written')
EOF
<guest-path> -I <guest-path> && <guest-path> -I -m json.tool .coldpass/report.json >/dev/null && echo json_ok; COLDPASS_WORKSPACE=<guest-path> timeout 1500 python3 -I <guest-path> --validate --replay > <guest-path> 2>&1; echo "rc=$?"; python3 -c "import json;d=json.load(open('<guest-path>'));print('valid',d['valid'],d['problems'],'replay',d['replay']['status'])"
cd <guest-path> && ls -la .coldpass; ls report.json 2>&1 | tail -1; pgrep -fa "pytest|fresh_toy|serve" | grep -v pgrep | head -3; echo done