coldpass

ashishsinha1602/langchain-schemagate

GitHub ↗ · How this check works

Per-setup public labels: Linux / Haiku: Works

Static status badge · SVG · read-only JSON. Embed with ![Coldpass status](https://coldpass.dev/badge/ashishsinha1602--langchain-schemagate.svg).

agent-harness

Works

The recorded run passed on 1 setup.

Answers apply to each recorded setup, revision and operation. Earlier observations remain in Run details.

Tested on Linux
PlatformHaikuLuna
Linuxyes

Linux / Haiku 5.5: yes · Oct 27, 2026 · 44330041b91f · claude-code / 2.1.310 (Claude Code)
Recorded operation: <guest-path> demo 'salary by employee' --principal okta:hr --role payroll

Does what it says
1 of 1 claims checked out
  • A table the caller may not read is never selected as a candidate, so it cannot reach the prompt: Real input produced the expected result: 10 of 42 objects selected

    Source: A table the caller can't read is never a candidate, so it can't

    Input: salary by employee

    Expected: 10 of 42 objects selected for a default caller without the payroll role

    Command: <guest-path> demo 'salary by employee'

    Observed: 10 of 42 objects selected main.hr_employee main.v_employee_headcount main.sales_order_status main.sales_invoice_draft main.core_country main.billing_credit_note main.core_party main.hr_department main.sales_order main.billing_invoice

MCP
No
Local or hosted
local
Local: runs on your machine
Needs keys or browser
None seen
Maintained
Maintained
actively maintained: last commit 0 days ago; 1 commits and 0 merged PRs in 90 days; 0 open issues, 0 open PRs; no releases

Last commit Oct 27, 2026

Setup time
3 min 6 s
3 min 6 s to a working result
Issue
none found in recorded check
Examples
1 of 1 ran as expected
  • repo test suite (CI command pytest -q) runs LangChain standard retriever tests and the access-control test: command completed successfully

    Source: .github/workflows/ci.yml

    Input: pytest -q

    Expected: all tests pass

    Command: <guest-path> -q

    Observed: te-packages/pytest_asyncio/plugin.py:300: PytestDeprecationWarning: The configuration option "asyncio_default_fixture_loop_scope" is unset. The event loop scope for asynchronous fixtures will default to the "fixture" caching scope. Future versions of pytest-asyncio will default the loop scope for asynchronous fixtures to "function" scope. Set the default fixture loop scope explicitly in order to avoid unexpected behavior in the future. Valid fixture loop scopes are: "function", "class", "module", "package", "session" warnings.warn(PytestDeprecationWarning(_DEFAULT_FIXTURE_LOOP_SCOPE_UNSET))

Evidence
README behavior checked
Agent interfaces
CLI
CLI

Observed interfaces were exercised by the checker. Documentation alone does not confirm compatibility.

Tested with
Claude Code / Haiku 5.5
  • Linux: Claude Code / Haiku 5.5 · agent 2.1.310 (Claude Code) · Checked operation passed · Oct 27, 2026 on 44330041b91f

Each entry describes its recorded operation and source revision.

Safety
No unexpected network or file activity found in these checks

No problems traced to this repository code in these checks; a short run cannot prove safety.

Reviewed as expected for a test run: attempted writes that failed, so nothing was written.

Full event, path and process lists for each observed setup:

Run details

Haiku 5.5

Native agent · Checked operation passed

Checked operation <guest-path> demo 'salary by employee' --principal okta:hr --role payroll

Agent-reported scope: Select the tables a question needs, filtered by the caller's roles, using the schemagate engine that langchain-schemagate wraps

Run details
run
ashishsinha1602--langchain-schemagate-claude-haiku-5-5-1793152078034794133
date
Oct 27, 2026
time to working
186.1s
product hosting / execution
Local command in the guest
recorded operation scope
product operation
model
claude-haiku-5-5
agent / version
claude-code / 2.1.310 (Claude Code)
platform
linux
reported cost
$0.009681
Recorded source
https://github.com/ashishsinha1602/langchain-schemagate
Backend
hyperv
Tool transport
mcp-assigned-vm
Requested source pin (not an observed revision)
{"commit": "44330041b91fb9e3825040a3763a857e9bfe169b", "url": "https://github.com/ashishsinha1602/langchain-schemagate"}
Test version
readme-c03644542c69939a9374958136a8b3431f7456269d3fb4b3c433785f59cfd637
revision
44330041b91fb9e3825040a3763a857e9bfe169b
Whose fault: not assigned · resolved

Safety: No problems traced to this repository code in these checks.

No problems traced to this repository code in these checks.

Reviewed as expected for a test run: attempted writes that failed, so nothing was written.

OSV scanned supported lockfiles; unpinned dependencies and unsupported formats are outside this check.

A short run cannot establish that code is safe.

Connections show IP addresses; proxies can hide the final hostname.

Tracing covers dispatched commands and their children, not the whole OS; guest root can interfere with tracing.

Full safety evidence →

Answers recorded for this run
  • A table the caller may not read is never selected as a candidate, so it cannot reach the prompt: Real input produced the expected result: 10 of 42 objects selected

    Source: A table the caller can't read is never a candidate, so it can't

    Input: salary by employee

    Expected: 10 of 42 objects selected for a default caller without the payroll role

    Command: <guest-path> demo 'salary by employee'

    Observed: 10 of 42 objects selected main.hr_employee main.v_employee_headcount main.sales_order_status main.sales_invoice_draft main.core_country main.billing_credit_note main.core_party main.hr_department main.sales_order main.billing_invoice

Examples

  • repo test suite (CI command pytest -q) runs LangChain standard retriever tests and the access-control test: command completed successfully

    Source: .github/workflows/ci.yml

    Input: pytest -q

    Expected: all tests pass

    Command: <guest-path> -q

    Observed: te-packages/pytest_asyncio/plugin.py:300: PytestDeprecationWarning: The configuration option "asyncio_default_fixture_loop_scope" is unset. The event loop scope for asynchronous fixtures will default to the "fixture" caching scope. Future versions of pytest-asyncio will default the loop scope for asynchronous fixtures to "function" scope. Set the default fixture loop scope explicitly in order to avoid unexpected behavior in the future. Valid fixture loop scopes are: "function", "class", "module", "package", "session" warnings.warn(PytestDeprecationWarning(_DEFAULT_FIXTURE_LOOP_SCOPE_UNSET))

Local or hosted: local

What the agent did

Select the tables a question needs, filtered by the caller's roles, using the schemagate engine that langchain-schemagate wraps

cd <guest-path> && pwd && ls -la && id -u && python3 --version && cat README.md | head -c 6000
cd <guest-path> && cat pyproject.toml && ls -R langchain_schemagate tests .github && cat .github/workflows/* 2>/dev/null | head -c 3000 && ls -la .coldpass && echo $CDP_URL
cd <guest-path> && python3 -m venv <guest-path> && <guest-path> install -q --upgrade pip 2>&1 | tail -3; cd <guest-path> && <guest-path> install -e ".[test]" 2>&1 | tail -5
cd <guest-path> && cat langchain_schemagate/__init__.py | head -c 4000; echo ----; cat tests/unit_tests/test_standard.py | head -c 1500; echo ----; <guest-path> -q 2>&1 | tail -15
cd <guest-path> && ls | grep -i -E "schema|gate" ; cat <guest-path> 2>/dev/null; cat <guest-path> 2>/dev/null; cd /tmp && <guest-path> -I -c "import schemagate.demo_schema as d; print(d.__file__)"; ls <guest-path>
cd /tmp && S=<guest-path>; $S --help 2>&1 | head -60; echo ----; grep -n -i "demo\|langchain" <guest-path> | head -30
cd /tmp && S=<guest-path>; $S demo "salary by employee" 2>&1 | head -40; echo "EXIT=$?"; echo ----; $S demo "salary by employee" --principal okta:hr --role payroll 2>&1 | head -40
cd <guest-path> && cat > probe.json <<'EOF'
{"kind":"cli","argv":["<guest-path>","demo","salary by employee","--principal","okta:hr","--role","payroll"],"cwd":"<guest-path>","expected_text":"main.hr_compensation"}
EOF
cat > claims.json <<'EOF'
[{"kind":"claim","claim":"A table the caller may not read is never selected as a candidate, so it cannot reach the prompt","source":"README.md","quote":"A table the caller can't read is never a candidate, so it can't","input":"salary by employee","expected":"10 of 42 objects selected for a default caller without the payroll role","probe_kind":"cli","argv":["<guest-path>","demo","salary by employee"],"expected_text":"10 of 42 objects selected","cwd":"<guest-path>"}]
EOF
cat > examples.json <<'EOF'
[{"kind":"example","claim":"repo test suite (CI command pytest -q) runs LangChain standard retriever tests and the access-control test","source":".github/workflows/ci.yml","input":"pytest -q","expected":"all tests pass","argv":["<guest-path>","-q"],"cwd":"<guest-path>","expected_text":"7 passed"}]
EOF
cat > cycles.json <<'EOF'
{"detected":false,"reason":"Entry points inspected: langchain_schemagate/__init__.py exports only SchemagateRetriever and prompt_fragment (a BaseRetriever library, no CLI, server, cron, scheduler, loop, webhook or --once/--now option). The schemagate dependency's CLI has no scheduler either. Nothing in this repo runs on a schedule, so no product cycle exists.","checks":[]}
EOF
for f in probe claims examples cycles; do <guest-path> -I -c "import json,sys;json.load(open('$f.json'));print('$f ok')"; done
cd <guest-path> && cat > claims.json <<'EOF'
[{"kind":"claim","claim":"A table the caller may not read is never selected as a candidate, so it cannot reach the prompt","source":"A table the caller can't read is never a candidate, so it can't","input":"salary by employee","expected":"10 of 42 objects selected for a default caller without the payroll role","probe_kind":"cli","argv":["<guest-path>","demo","salary by employee"],"expected_text":"10 of 42 objects selected","cwd":"<guest-path>"}]
EOF
cd <guest-path> && <env-var>=<guest-path> python3 -I <guest-path> --validate --replay 2>&1 | tail -60
cd <guest-path> && cat > report.json <<'EOF'
{"outcome":"working","fault_owner":null,"operation":"Select the tables a question needs, filtered by the caller's roles, using the schemagate engine that langchain-schemagate wraps","operation_scope":"Offline selection against schemagate's bundled 42-object demo schema, run via the documented schemagate demo CLI. The README's Postgres bootstrap (Catalog().bootstrap('postgresql://localhost/app')) and the LangChain retriever Python API were not run directly: no database is available in the guest, and the README gives Python code, not a CLI.","steps":["pip install -e \".[test]\"  (in a Python 3.12 venv at <guest-path>; CI command from .github/workflows/ci.yml)","pytest -q  (7 passed)","schemagate demo \"salary by employee\"  (analyst caller: 10 of 42 objects selected, hr_compensation absent)","schemagate demo \"salary by employee\" --principal okta:hr --role payroll  (9 of 42 selected, main.hr_compensation present)"],"deviations":["The README's Python quickstart (Catalog().bootstrap on localhost Postgres) was not run: no Postgres database in the guest.","The README has no CLI; the probe uses the schemagate CLI (documented in the schemagate dependency's packaged README, which the langchain README links to) as the real product entry point.","Python 3.12.3 was used (requires-python >=3.10; CI matrix 3.10 and 3.13 not tested)."],"stuck_step":null,"main_blocker":null,"plain_failure":null,"suggested_fix":"None for the tested path. To cover the README quickstart, run it against a Postgres instance and check that the retriever returns only tables the caller may read."}
EOF
<guest-path> -I -c "import json;json.load(open('report.json'));print('report ok')"; cd <guest-path> && <env-var>=<guest-path> python3 -I <guest-path> --validate --replay 2>&1 | head -c 300

No product verdict. Run record.