OrionArchitekton/schemafit

GitHub ↗ · How this check works

Per-setup public labels: Linux / Haiku: Works · Linux / Luna: Works

Static status badge · SVG · read-only JSON. Embed with ![Coldpass status](https://coldpass.dev/badge/OrionArchitekton--schemafit.svg).

mcp-server

Works

README claims checked out on 2 recorded setups.

Answers apply to each recorded setup, revision and operation. Earlier observations remain in Run details.

Tested on Linux
PlatformHaikuLuna
Linuxyesyes

Linux / Haiku 5.5: yes · Oct 26, 2026 · 13fb6a43fd0e · claude-code / 2.1.310 (Claude Code)
Recorded operation: <guest-path> lint - --provider openai,anthropic,gemini,mistral,cohere

Linux / Luna: yes · Oct 26, 2026 · 13fb6a43fd0e · codex / codex-cli 0.163.0
Recorded operation: /tmp/schemafit-venv/bin/schemafit lint fixtures/drift-mock-bad.json --provider cohere --live-verify

Does what it says
1 of 1 claims checked out
  • Mock mode performs a deterministic network-free verification: Real input produced the expected result: cohere-drift

    Source: deterministic, network-free client modeled on the static rule pack

    Input: schemafit lint fixtures/drift-mock-bad.json --provider cohere --live-verify

    Expected: cohere-drift

    Command: /tmp/schemafit-venv/bin/schemafit lint fixtures/drift-mock-bad.json --provider cohere --live-verify

    Observed: cohere-drift) Rule-pack drift detected for cohere: live verification rejected the schema but static pack had no violations. Pack may lag provider docs (client=mock): modeled from the static rule pack (no network, no key) ref: https://docs.cohere.com/docs/structured-outputs [cohere] LIVE-VERIFY confirmed_by_provider=False client=mock (modeled from the static rule pack (no network, no key))

MCP
No
Local or hosted
local
Local: runs on your machine
Needs keys or browser
None seen
Maintained
Maintained
actively maintained: last commit 1 days ago; 1 commits and 1 merged PRs in 90 days; 0 open issues, 0 open PRs; latest release 2026-10-24

Last commit Oct 24, 2026

Setup time
1 min 10 s
1 min 10 s to a working result
Issue
none found in recorded check
Examples
1 of 1 ran as expected
  • Run the documented offline self-check: command completed successfully

    Source: README.md / Install

    Input: schemafit demo

    Expected: PROOF_OK

    Command: /tmp/schemafit-venv/bin/schemafit demo

    Observed: == schemafit demo == PROVIDER=anthropic INPUT=demo-bad VIOLATIONS=5 EXIT=1 VIOLATION_PATH=#/properties/age/maximum (keyword: maximum -> rejected by anthropic) --- after `schemafit repair --provider anthropic` --- VIOLATIONS=0 EXIT=0 auto_fixed=5 MULTI: openai=PASS anthropic=FAIL gemini=PASS mistral=PASS cohere=FAIL NOTE: gemini portability warnings=1 (version-sensitive, non-failing) PROOF_OK

Evidence
README behavior checked
Agent interfaces
CLI
CLI

Observed interfaces were exercised by the checker. Documentation alone does not confirm compatibility.

Tested with
Codex / Luna
  • Linux: Codex / Luna · agent codex-cli 0.163.0 · Checked operation passed · Oct 26, 2026 on 13fb6a43fd0e

Each entry describes its recorded operation and source revision.

Safety
No unexpected network or file activity found in these checks

No problems traced to this repository code in these checks; a short run cannot prove safety.

Reviewed as expected for a test run: attempted writes that failed, so nothing was written.

Full event, path and process lists for each observed setup:

Run details

Haiku 5.5

Native agent · Checked operation passed

Checked operation <guest-path> lint - --provider openai,anthropic,gemini,mistral,cohere

Agent-reported scope: Lint a JSON schema for provider-specific structured-output constraints (schemafit lint), plus the README demo, repair --out, and the CONTRIBUTING pytest/ruff checks

Run details
run
OrionArchitekton--schemafit-claude-haiku-5-5-1793018510130992510
date
Oct 26, 2026
time to working
106.1s
product hosting / execution
Local command in the guest
recorded operation scope
product operation
model
claude-haiku-5-5
agent / version
claude-code / 2.1.310 (Claude Code)
platform
linux
reported cost
$0.014556
Recorded source
https://github.com/OrionArchitekton/schemafit
Backend
hyperv
Tool transport
mcp-assigned-vm
Requested source pin (not an observed revision)
{"commit": "13fb6a43fd0e8890b9b594e035ac57f5b441e78d", "url": "https://github.com/OrionArchitekton/schemafit"}
Test version
readme-ca496ea10bb07418d1d8f1c256c2433a55547acc0fb62d8b6d86ae33c976fbfa
revision
13fb6a43fd0e8890b9b594e035ac57f5b441e78d
Whose fault: not assigned · resolved

Safety: No problems traced to this repository code in these checks.

No problems traced to this repository code in these checks.

Reviewed as expected for a test run: attempted writes that failed, so nothing was written.

OSV scanned supported lockfiles; unpinned dependencies and unsupported formats are outside this check.

A short run cannot establish that code is safe.

Connections show IP addresses; proxies can hide the final hostname.

Tracing covers dispatched commands and their children, not the whole OS; guest root can interfere with tracing.

Full safety evidence →

Answers recorded for this run
  • lint reports provider-specific constraint violations and exits 1 on errors: Real input produced the expected result: [anthropic] FAIL: 2 error(s), 0 warning(s)

    Source: Lint one schema against several providers (exit 1 if any error):

    Input: schema with pattern and minimum keywords piped on stdin

    Expected: anthropic FAIL with 2 errors, exit 1

    Command: <guest-path> lint - --provider openai,anthropic,gemini,mistral,cohere

    Observed: [anthropic] FAIL: 2 error(s), 0 warning(s) ERROR #/properties/qty/minimum (anthropic-no-minimum) Anthropic rejects the 'minimum' validation keyword (400 Bad Request). ref: https://github.com/vercel/ai/issues/13355 ERROR #/properties/sku/pattern (anthropic-no-pattern) Anthropic rejects the 'pattern' validation keyword (400 Bad Request). ref: https://github.com/vercel/ai/issues/13355 [gemini] PASS: 0 error(s), 0 warning(s) [mistral] PASS: 0 error(s), 0 warning(s) [cohere] FAIL: 1 error(s), 0 warning(s) ERROR #/properties/qty/minimum (cohere-

  • demo is an offline self-check that ends with PROOF_OK: Real input produced the expected result: PROOF_OK

    Source: offline self-check; ends with PROOF_OK

    Input: schemafit demo

    Expected: PROOF_OK printed

    Command: <guest-path> demo

    Observed: PROOF_OK

  • repair emits a best-effort provider-valid variant to --out: Real input produced the expected result: "additionalProperties": false

    Source: Emit a best-effort provider-valid variant (lossy transforms are flagged):

    Input: sample schema with pattern, repaired for anthropic

    Expected: fixed schema file without the pattern keyword

    Command: <guest-path> repair <guest-path> --provider anthropic --out <guest-path>

    Observed: "additionalProperties": false, "properties": { "sku": { "type": "string" } }, "required": [ "sku" ], "type": "object" }

Examples

  • documented dev test suite passes: command completed successfully

    Source: CONTRIBUTING.md

    Input: pytest -q run from the checkout

    Expected: all tests pass

    Command: <guest-path> -q -p no:cacheprovider

    Observed: ........................................................................ [ 79%] ................... [100%] 91 passed in 1.08s

  • documented lint step is clean: command completed successfully

    Source: CONTRIBUTING.md

    Input: ruff check . from the checkout

    Expected: no lint findings

    Command: <guest-path> check . --no-cache

    Observed: All checks passed!

  • README repair --out example writes a fixed schema file: command completed successfully

    Source: README.md

    Input: sample schema repaired for anthropic into a fresh --out file

    Expected: fixed.json written with additionalProperties false

    Command: <guest-path> repair <guest-path> --provider anthropic --out <guest-path>

    Observed: { "additionalProperties": false, "properties": { "sku": { "type": "string" } }, "required": [ "sku" ], "type": "object" }

Local or hosted: local

What the agent did

Lint a JSON schema for provider-specific structured-output constraints (schemafit lint), plus the README demo, repair --out, and the CONTRIBUTING pytest/ruff checks

cd <guest-path> && ls -la && cat README.md 2>/dev/null | head -c 6000; echo; python3 --version; id -u
ls -la /inputs 2>/dev/null; ls <guest-path> 2>/dev/null; echo "CDP: $CDP_URL"; curl -s -m 10 -w '\nHTTP %{http_code}\n' $CDP_URL/json/version
cd <guest-path> && tail -c +6000 README.md | head -c 7000
cd <guest-path> && python3 -m venv <guest-path> 2>&1 | tail -5; <guest-path> install -q --upgrade pip 2>&1 | tail -3; <guest-path> install <guest-path> 2>&1 | tail -5; ls <guest-path> | grep -i schemafit
cd <guest-path> && <guest-path> demo; echo "EXIT=$?"
cd <guest-path> && echo '{"type":"object","additionalProperties":false,"properties":{"sku":{"type":"string","pattern":"^[A-Z]{3}-[0-9]{4}$"},"qty":{"type":"integer","minimum":1}},"required":["sku","qty"]}' | <guest-path> lint - --provider openai,anthropic,gemini,mistral,cohere; echo "EXIT=$?"
cd <guest-path> && ls fixtures tests docs; grep -rniE "cron|schedule|--once|--now|webhook|loop" --include=*.py --include=*.md --include=*.yml . | grep -v CHANGELOG | head -20; echo ---; cd <guest-path> && echo '{"type":"object","additionalProperties":false,"properties":{"sku":{"type":"string","pattern":"^[A-Z]{3}-[0-9]{4}$"}},"required":["sku"]}' > <guest-path> && <guest-path> repair <guest-path> --provider anthropic; echo "EXIT=$?"
cd <guest-path> && cat CONTRIBUTING.md | head -c 1400; echo ---; cat AGENTS.md | head -c 1000; echo ---; mkdir -p <guest-path> && cd <guest-path> && <guest-path> repair <guest-path> --provider anthropic --out <guest-path>; echo "EXIT=$?"; cat <guest-path>; echo; ls -la <guest-path>
rm -f <guest-path>; <guest-path> install -q pytest ruff 2>&1 | tail -3; cd <guest-path> && <guest-path> -m pytest -q -p no:cacheprovider 2>&1 | tail -8; echo "EXIT=${PIPESTATUS[0]}"
cd <guest-path> && ls -la .coldpass && cat > .coldpass/probe.json <<'EOF'
{"kind":"cli","argv":["<guest-path>","lint","-","--provider","openai,anthropic,gemini,mistral,cohere"],"cwd":"<guest-path>","stdin":"{\"type\":\"object\",\"additionalProperties\":false,\"properties\":{\"sku\":{\"type\":\"string\",\"pattern\":\"^[A-Z]{3}-[0-9]{4}$\"},\"qty\":{\"type\":\"integer\",\"minimum\":1}},\"required\":[\"sku\",\"qty\"]}","expected_returncode":1,"expected_text":"[anthropic] FAIL: 2 error(s), 0 warning(s)"}
EOF
python3 -I -c "import json;json.load(open('<guest-path>'));print('ok')"
cd <guest-path> && cat > .coldpass/claims.json <<'EOF'
[
{"kind":"claim","claim":"lint reports provider-specific constraint violations and exits 1 on errors","source":"Lint one schema against several providers (exit 1 if any error):","input":"schema with pattern and minimum keywords piped on stdin","expected":"anthropic FAIL with 2 errors, exit 1","probe_kind":"cli","argv":["<guest-path>","lint","-","--provider","openai,anthropic,gemini,mistral,cohere"],"stdin":"{\"type\":\"object\",\"additionalProperties\":false,\"properties\":{\"sku\":{\"type\":\"string\",\"pattern\":\"^[A-Z]{3}-[0-9]{4}$\"},\"qty\":{\"type\":\"integer\",\"minimum\":1}},\"required\":[\"sku\",\"qty\"]}","expected_returncode":1,"expected_text":"[anthropic] FAIL: 2 error(s), 0 warning(s)","cwd":"<guest-path>"},
{"kind":"claim","claim":"demo is an offline self-check that ends with PROOF_OK","source":"offline self-check; ends with PROOF_OK","input":"schemafit demo","expected":"PROOF_OK printed","probe_kind":"cli","argv":["<guest-path>","demo"],"expected_text":"PROOF_OK","cwd":"<guest-path>"},
{"kind":"claim","claim":"repair emits a best-effort provider-valid variant to --out","source":"Emit a best-effort provider-valid variant (lossy transforms are flagged):","input":"sample schema with pattern, repaired for anthropic","expected":"fixed schema file without the pattern keyword","probe_kind":"cli","argv":["<guest-path>","repair","<guest-path>","--provider","anthropic","--out","<guest-path>"],"expected_file":"<guest-path>","expected_text":"\"additionalProperties\": false","cwd":"<guest-path>"}
]
EOF
python3 -I -c "import json;print(len(json.load(open('<guest-path>'))))"
cd <guest-path> && <guest-path> check . --no-cache 2>&1 | tail -5; echo "EXIT=${PIPESTATUS[0]}"
cd <guest-path> && cat > .coldpass/examples.json <<'EOF'
[
{"kind":"example","claim":"documented dev test suite passes","source":"CONTRIBUTING.md","input":"pytest -q run from the checkout","expected":"all tests pass","argv":["<guest-path>","-q","-p","no:cacheprovider"],"cwd":"<guest-path>","expected_text":"91 passed"},
{"kind":"example","claim":"documented lint step is clean","source":"CONTRIBUTING.md","input":"ruff check . from the checkout","expected":"no lint findings","argv":["<guest-path>","check",".","--no-cache"],"cwd":"<guest-path>","expected_text":"All checks passed!"},
{"kind":"example","claim":"README repair --out example writes a fixed schema file","source":"README.md","input":"sample schema repaired for anthropic into a fresh --out file","expected":"fixed.json written with additionalProperties false","argv":["<guest-path>","repair","<guest-path>","--provider","anthropic","--out","<guest-path>"],"expected_file":"<guest-path>","expected_text":"\"additionalProperties\": false","cwd":"<guest-path>"}
]
EOF
cat > .coldpass/cycles.json <<'EOF'
{"detected":false,"reason":"Inspected README.md and the package entry points (schemafit lint/repair/demo/providers CLI). The README and source contain no cron, scheduler, daemon, loop, webhook, or --once/--now trigger; schemafit is a one-shot CLI that lints or repairs a schema and exits. No product cycle exists to replay.","checks":[]}
EOF
python3 -I -c "import json;[json.load(open('<guest-path>'+f)) for f in ['examples.json','cycles.json']];print('ok')"; ls -la <guest-path>
cd <guest-path> && <env-var>=<guest-path> python3 -I <guest-path> --validate --replay; echo "EXIT=$?"
mkdir -p <guest-path> && cd <guest-path> && cat > .coldpass/report.json <<'EOF'
{"outcome":"working","fault_owner":null,"operation":"Lint a JSON schema for provider-specific structured-output constraints (schemafit lint), plus the README demo, repair --out, and the CONTRIBUTING pytest/ruff checks","operation_scope":"Pinned checkout 13fb6a43fd0e8890b9b594e035ac57f5b441e78d installed into a Python 3.12.3 venv at <guest-path> (README requires Python 3.11+). Offline, no API keys, no browser, no servers started.","steps":["python3 -m venv <guest-path>","<guest-path> install <guest-path>","<guest-path> demo  (ended with PROOF_OK, exit 0)","echo '<schema>' | <guest-path> lint - --provider openai,anthropic,gemini,mistral,cohere  (anthropic FAIL 2 errors, cohere FAIL 1 error, exit 1 as documented)","<guest-path> repair <guest-path> --provider anthropic --out <guest-path>  (wrote fixed schema, exit 0)","<guest-path> -q (91 passed, from CONTRIBUTING.md)","<guest-path> check . (All checks passed, from CONTRIBUTING.md)"],"deviations":["README's first install is 'pip install schemafit' from PyPI; I installed the pinned local checkout with pip into a venv so the tested code matches the pinned commit. Both report schemafit 0.5.1.","Used a venv instead of uvx (uv not installed); no system packages needed.","Installed pytest and ruff into the venv to run the documented CONTRIBUTING.md test and lint commands.","/inputs/coldpass-preflight.py does not exist in this guest; ran the checker at <guest-path> as instructed in the task, which validated and replayed cleanly.","No product scheduler/cycle exists; cycles.json has detected:false."],"stuck_step":null,"main_blocker":null,"plain_failure":null,"suggested_fix":null}
EOF
python3 -I -c "import json;print(json.load(open('<guest-path>'))['outcome'])"; ls -la <guest-path> 2>&1 | tail -1
cd <guest-path> && <env-var>=<guest-path> python3 -I <guest-path> --validate --replay | python3 -I -c "import json,sys;d=json.load(sys.stdin);print('valid=',d['valid'],'problems=',d['problems'],'replay=',d['replay']['status'])"; echo "EXIT=${PIPESTATUS[0]}"
Earlier attempts (1)

Luna

Native agent · Checked operation passed

Checked operation /tmp/schemafit-venv/bin/schemafit lint fixtures/drift-mock-bad.json --provider cohere --live-verify

Agent-reported scope: Run schemafit demo and lint a fixture with mock live verification

Run details
run
OrionArchitekton--schemafit-gpt-6-luna-1793019056556387304
date
Oct 26, 2026
time to working
69.5s
product hosting / execution
Local command in the guest
recorded operation scope
product operation
model
gpt-6-luna
agent / version
codex / codex-cli 0.163.0
platform
linux
reported cost
not reported (not assumed free)
Recorded source
https://github.com/OrionArchitekton/schemafit
Backend
hyperv
Tool transport
mcp-assigned-vm
Requested source pin (not an observed revision)
{"commit": "13fb6a43fd0e8890b9b594e035ac57f5b441e78d", "url": "https://github.com/OrionArchitekton/schemafit"}
Test version
readme-a17389bda4cc696f9601c0c4ff1b2c16e23bfa1d907883b715ccdd9e92ddec56
revision
13fb6a43fd0e8890b9b594e035ac57f5b441e78d
Whose fault: not assigned · resolved

Safety: No problems traced to this repository code in these checks.

No problems traced to this repository code in these checks.

Reviewed as expected for a test run: attempted writes that failed, so nothing was written.

OSV scanned supported lockfiles; unpinned dependencies and unsupported formats are outside this check.

A short run cannot establish that code is safe.

Connections show IP addresses; proxies can hide the final hostname.

Tracing covers dispatched commands and their children, not the whole OS; guest root can interfere with tracing.

Full safety evidence →

Answers recorded for this run
  • Mock mode performs a deterministic network-free verification: Real input produced the expected result: cohere-drift

    Source: deterministic, network-free client modeled on the static rule pack

    Input: schemafit lint fixtures/drift-mock-bad.json --provider cohere --live-verify

    Expected: cohere-drift

    Command: /tmp/schemafit-venv/bin/schemafit lint fixtures/drift-mock-bad.json --provider cohere --live-verify

    Observed: cohere-drift) Rule-pack drift detected for cohere: live verification rejected the schema but static pack had no violations. Pack may lag provider docs (client=mock): modeled from the static rule pack (no network, no key) ref: https://docs.cohere.com/docs/structured-outputs [cohere] LIVE-VERIFY confirmed_by_provider=False client=mock (modeled from the static rule pack (no network, no key))

Examples

  • Run the documented offline self-check: command completed successfully

    Source: README.md / Install

    Input: schemafit demo

    Expected: PROOF_OK

    Command: /tmp/schemafit-venv/bin/schemafit demo

    Observed: == schemafit demo == PROVIDER=anthropic INPUT=demo-bad VIOLATIONS=5 EXIT=1 VIOLATION_PATH=#/properties/age/maximum (keyword: maximum -> rejected by anthropic) --- after `schemafit repair --provider anthropic` --- VIOLATIONS=0 EXIT=0 auto_fixed=5 MULTI: openai=PASS anthropic=FAIL gemini=PASS mistral=PASS cohere=FAIL NOTE: gemini portability warnings=1 (version-sensitive, non-failing) PROOF_OK

Local or hosted: local

What the agent did

Run schemafit demo and lint a fixture with mock live verification

pwd; ls -la; sed -n '1,240p' README.md; python3 --version; id -u
rg --files schemafit; cat pyproject.toml; rg -n "schemafit lint|schemafit demo|drift-mock" README.md
python3 -m venv /tmp/schemafit-venv && /tmp/schemafit-venv/bin/python -m pip install -e . && /tmp/schemafit-venv/bin/schemafit demo
sed -n '1,160p' fixtures/drift-mock-bad.json; mkdir -p .coldpass; /tmp/schemafit-venv/bin/schemafit lint fixtures/drift-mock-bad.json --provider cohere --live-verify
cat > .coldpass/probe.json <<'EOF'
{"kind":"cli","argv":["/tmp/schemafit-venv/bin/schemafit","lint","fixtures/drift-mock-bad.json","--provider","cohere","--live-verify"],"cwd":"<guest-path>","expected_returncode":1,"expected_text":"cohere-drift"}
EOF
cat > .coldpass/claims.json <<'EOF'
[{"kind":"claim","claim":"Live-verify mock drift detection reports a cohere-drift finding","source":"Proofs include `schemafit lint fixtures/drift-mock-bad.json --provider cohere --live-verify`","input":"fixtures/drift-mock-bad.json with provider cohere in mock mode","expected":"cohere-drift finding","probe_kind":"cli","argv":["/tmp/schemafit-venv/bin/schemafit","lint","fixtures/drift-mock-bad.json","--provider","cohere","--live-verify"],"expected_text":"cohere-drift","expected_returncode":1,"cwd":"<guest-path>"}]
EOF
cat > .coldpass/examples.json <<'EOF'
[{"kind":"example","claim":"Run the documented offline self-check","source":"README.md / Install","input":"schemafit demo","expected":"PROOF_OK","argv":["/tmp/schemafit-venv/bin/schemafit","demo"],"cwd":"<guest-path>","expected_text":"PROOF_OK"}]
EOF
cat > .coldpass/cycles.json <<'EOF'
{"detected":false,"reason":"Inspected README usage, CI and shipped entry points: schemafit is a one-shot static linter/repair CLI; no product scheduler, loop, webhook, or cycle trigger is documented.","checks":[]}
EOF

<env-var>=<guest-path> python3 -I <guest-path> --validate --replay
python3 - <<'PY'
p='.coldpass/claims.json'
s=open(p).read().replace('"source":"Proofs include `schemafit lint fixtures/drift-mock-bad.json --provider cohere --live-verify`"','"source":"Proofs include `schemafit lint fixtures/drift-mock-bad.json --provider cohere --live-verify` (exit 1 + `cohere-drift`)."')
open(p,'w').write(s)
PY
<env-var>=<guest-path> python3 -I <guest-path> --validate --replay
sed -n '150,205p' README.md
cat > .coldpass/claims.json <<'EOF'
[{"kind":"claim","claim":"Mock mode performs a deterministic network-free verification","source":"deterministic, network-free client modeled on the static rule pack","input":"schemafit lint fixtures/drift-mock-bad.json --provider cohere --live-verify","expected":"cohere-drift","probe_kind":"cli","argv":["/tmp/schemafit-venv/bin/schemafit","lint","fixtures/drift-mock-bad.json","--provider","cohere","--live-verify"],"expected_text":"cohere-drift","expected_returncode":1,"cwd":"<guest-path>"}]
EOF

<env-var>=<guest-path> python3 -I <guest-path> --validate --replay
cat > .coldpass/report.json <<'EOF'
{"outcome":"working","fault_owner":null,"operation":"Run schemafit demo and lint a fixture with mock live verification","operation_scope":"Python 3.12.3 virtual environment; local offline CLI","steps":["python3 -m venv /tmp/schemafit-venv","/tmp/schemafit-venv/bin/python -m pip install -e .","/tmp/schemafit-venv/bin/schemafit demo","/tmp/schemafit-venv/bin/schemafit lint fixtures/drift-mock-bad.json --provider cohere --live-verify"],"deviations":[],"stuck_step":null,"main_blocker":null,"plain_failure":null,"suggested_fix":null}
EOF