coldpass

jgstew/bigfix-remote-client-relevance

GitHub ↗ · How this check works

Per-setup public labels: Linux / Haiku: Works

Static status badge · SVG · read-only JSON. Embed with ![Coldpass status](https://coldpass.dev/badge/jgstew--bigfix-remote-client-relevance.svg).

agent-harness

Works

The recorded run passed on 1 setup.

Answers apply to each recorded setup, revision and operation. Earlier observations remain in Run details.

Tested on Linux
PlatformHaikuLuna
Linuxyes

Linux / Haiku 5.5: yes · Oct 27, 2026 · ed67fc5343c0 · claude-code / 2.1.310 (Claude Code)
Recorded operation: <guest-path> --container ubuntu:22.04 --qna-version 11.0 'name of operating system'

Does what it says
2 of 2 claims checked out
  • Evaluate a client relevance expression inside a container with a pinned qna version: Real input produced the expected result: Linux Ubuntu 22.04.5 LTS

    Source: bigfix-remote-client-relevance --container ubuntu:22.04 --qna-version 11.0 "name of operating system"

    Input: name of operating system in ubuntu:22.04 container, qna 11.0

    Expected: Linux Ubuntu 22.04.5 LTS

    Command: <guest-path> --container ubuntu:22.04 --qna-version 11.0 'name of operating system'

    Observed: Linux Ubuntu 22.04.5 LTS INFO bigfix_remote_client_relevance.orchestrate: evaluating 1 client-relevance expression(s) across 1 target/version pair(s), max_parallel=8

  • --schema prints the JSON Schema for a single result: Real input produced the expected result: ClientRelevanceResult

    Source: prints the JSON Schema for a single result and exits 0

    Input: --schema

    Expected: ClientRelevanceResult schema

    Command: <guest-path> --schema

    Observed: ClientRelevanceResult", "description": "One evaluation of one client-relevance expression on one target, for one qna version. Emitted by the CLI's --json and --jsonl output and by bigfix_remote_client_relevance.serialize.result_to_dict.", "type": "object", "properties": { "host": { "type": "string", "description": "\"local\", an SSH host, or \"container:<image>@<arch>\"." }, "transport": { "type": "string", "enum": [ "local", "ssh", "container", "fastquery" ] }, "platform": { "type": [ "string

MCP
Documented only
Local or hosted
both local and hosted
Both a local install and a hosted service
Needs keys or browser
None seen
Maintained
Maintained
actively maintained: last commit 0 days ago; 165 commits and 36 merged PRs in 90 days; 12 open issues, 1 open PRs; latest release 2026-10-27

Last commit Oct 27, 2026

Setup time
2 min 51 s
2 min 51 s to a working result
Issue
none found in recorded check
Examples
1 of 1 ran as expected
  • Print the JSON Schema for a single result without a target: command completed successfully

    Source: README.md ## Use / From a non-Python server

    Input: --schema

    Expected: schema title ClientRelevanceResult

    Command: <guest-path> --schema

    Observed: k time measured by the controller." }, "raw_qna_output": { "type": "string", "description": "Full qna stdout, for debugging and for agents to self-correct on. May be truncated with a trailing marker when the producer applied a cap." } }, "required": [ "host", "transport", "platform", "arch", "client_relevance", "ok", "answers", "answer_types", "error", "error_kind", "exit_code", "qna_version", "qna_fallback_from", "qna_path", "qna_time", "use_client_context", "elapsed_ms", "raw_qna_output" ] }

Evidence
README behavior checked
Agent interfaces
CLI, MCP (documented only)
CLI, MCP (documented only)

Observed interfaces were exercised by the checker. Documentation alone does not confirm compatibility.

Tested with
Claude Code / Haiku 5.5
  • Linux: Claude Code / Haiku 5.5 · agent 2.1.310 (Claude Code) · Checked operation passed · Oct 27, 2026 on ed67fc5343c0

Each entry describes its recorded operation and source revision.

Safety
Made outbound HTTPS connections: yes (hosts not recorded) by bigfix-remote-client-relevance

Review signals across current setups: connection review, sensitive read.

Repository processes attempted these non-loopback connections; proxies can hide the final hostname. 4 distinct locations. 5 attributed processes.

A repository process attempted to read this sensitive path. Kind of file: cloud login. 5 attributed processes.

Reviewed as expected for a test run: attempted writes that failed, so nothing was written.

Full event, path and process lists for each observed setup:

Repair history and test corrections
Run details

Haiku 5.5

Native agent · Checked operation passed

Checked operation <guest-path> --container ubuntu:22.04 --qna-version 11.0 'name of operating system'

Agent-reported scope: Evaluate a BigFix client-relevance expression ("name of operating system") inside an ubuntu:22.04 container with qna 11.0, via the README's documented --container command

Run details
run
jgstew--bigfix-remote-client-relevance-claude-haiku-5-5-1793139238808773790
date
Oct 27, 2026
time to working
171.5s
product hosting / execution
Local command in the guest
recorded operation scope
product operation
model
claude-haiku-5-5
agent / version
claude-code / 2.1.310 (Claude Code)
platform
linux
reported cost
$0.014063
Recorded source
https://github.com/jgstew/bigfix-remote-client-relevance
Backend
hyperv
Tool transport
mcp-assigned-vm
Requested source pin (not an observed revision)
{"commit": "ed67fc5343c0459060dbfbe14b14a0b37cdfde34", "url": "https://github.com/jgstew/bigfix-remote-client-relevance"}
Test version
readme-d5b140d3c0f8b95c2dd6fe6fc381a303cacb92679582c22317d8e78b572f0cf2
revision
ed67fc5343c0459060dbfbe14b14a0b37cdfde34
Current diagnosis: test · resolved

native Claude returned a different model

Repair record fix-07aa565184efb3ded7e8 · current status JSON. Original run outcomes remain unchanged.

Failed run: jgstew/bigfix-remote-client-relevance · Oct 27, 2026 · Our test failed; no repo verdict.

Test-repair rerun: jgstew/bigfix-remote-client-relevance · Oct 27, 2026 · Checked operation passed.

Double-checks failed run: jgstew/bigfix-remote-client-relevance · Oct 27, 2026.

Fix: fix-07aa565184efb3ded7e8.

Safety: Review signals: 1 connection review, 1 sensitive read.

Review signals: 1 connection review, 1 sensitive read.

Reviewed as expected for a test run: attempted writes that failed, so nothing was written.

Repository processes attempted these non-loopback connections; proxies can hide the final hostname. 4 distinct locations. 5 attributed processes.

A repository process attempted to read this sensitive path. Kind of file: cloud login. 5 attributed processes.

OSV scanned supported lockfiles; unpinned dependencies and unsupported formats are outside this check.

A short run cannot establish that code is safe.

Connections show IP addresses; proxies can hide the final hostname.

Tracing covers dispatched commands and their children, not the whole OS; guest root can interfere with tracing.

Full safety evidence →

Answers recorded for this run
  • Evaluate a client relevance expression inside a container with a pinned qna version: Real input produced the expected result: Linux Ubuntu 22.04.5 LTS

    Source: bigfix-remote-client-relevance --container ubuntu:22.04 --qna-version 11.0 "name of operating system"

    Input: name of operating system in ubuntu:22.04 container, qna 11.0

    Expected: Linux Ubuntu 22.04.5 LTS

    Command: <guest-path> --container ubuntu:22.04 --qna-version 11.0 'name of operating system'

    Observed: Linux Ubuntu 22.04.5 LTS INFO bigfix_remote_client_relevance.orchestrate: evaluating 1 client-relevance expression(s) across 1 target/version pair(s), max_parallel=8

  • --schema prints the JSON Schema for a single result: Real input produced the expected result: ClientRelevanceResult

    Source: prints the JSON Schema for a single result and exits 0

    Input: --schema

    Expected: ClientRelevanceResult schema

    Command: <guest-path> --schema

    Observed: ClientRelevanceResult", "description": "One evaluation of one client-relevance expression on one target, for one qna version. Emitted by the CLI's --json and --jsonl output and by bigfix_remote_client_relevance.serialize.result_to_dict.", "type": "object", "properties": { "host": { "type": "string", "description": "\"local\", an SSH host, or \"container:<image>@<arch>\"." }, "transport": { "type": "string", "enum": [ "local", "ssh", "container", "fastquery" ] }, "platform": { "type": [ "string

Examples

  • Print the JSON Schema for a single result without a target: command completed successfully

    Source: README.md ## Use / From a non-Python server

    Input: --schema

    Expected: schema title ClientRelevanceResult

    Command: <guest-path> --schema

    Observed: k time measured by the controller." }, "raw_qna_output": { "type": "string", "description": "Full qna stdout, for debugging and for agents to self-correct on. May be truncated with a trailing marker when the producer applied a cap." } }, "required": [ "host", "transport", "platform", "arch", "client_relevance", "ok", "answers", "answer_types", "error", "error_kind", "exit_code", "qna_version", "qna_fallback_from", "qna_path", "qna_time", "use_client_context", "elapsed_ms", "raw_qna_output" ] }

Local or hosted: both

What the agent did

Evaluate a BigFix client-relevance expression ("name of operating system") inside an ubuntu:22.04 container with qna 11.0, via the README's documented --container command

cd <guest-path> && pwd && ls -la && id -u && python3 --version && git status 2>&1 | head -3; cat README.md 2>/dev/null | head -200
cd <guest-path> && sed -n 200,420p README.md; cat .python-version; ls scripts; cat AGENTS.md | head -40; which uv docker podman ssh; uv --version 2>&1; echo CDP=$CDP_URL
cd <guest-path> && grep -n "^## \|^### \|online-evaluator\|--local" README.md | head -80; grep -n "^version\|requires-python" pyproject.toml; sudo -n true 2>&1 && echo SUDO_OK; ls ~/.bigfix 2>&1
cd <guest-path> && sed -n 488,520p README.md; sed -n 669,700p README.md; sed -n 462,488p README.md
cd <guest-path> && uv sync 2>&1 | tail -15; echo EXIT=$?; ls .venv/bin | grep -i bigfix; .venv/bin/bigfix-remote-client-relevance --version 2>&1; .venv/bin/bigfix-remote-client-relevance --schema | head -c 300
cd <guest-path> && COLUMNS=200 .venv/bin/bigfix-remote-client-relevance --help 2>&1 | head -80; grep -n "online" README.md | head; grep -n "online" -i docs/* 2>/dev/null | head -5
(sudo apt-get -o DPkg::Lock::Timeout=300 update -qq && sudo DEBIAN_FRONTEND=noninteractive apt-get -o DPkg::Lock::Timeout=300 install -y -qq docker.io) 2>&1 | tail -5; echo EXIT=$?; which docker; sudo systemctl start docker 2>&1 || sudo service docker start 2>&1; sleep 3; sudo docker info 2>&1 | head -5
docker info 2>&1 | grep -i "server version\|errors" ; docker pull ubuntu:22.04 2>&1 | tail -3; echo EXIT=$?
sudo usermod -aG docker ubuntu && sudo chmod 666 /var/run/docker.sock; sg docker -c "docker info 2>&1 | grep -i 'server version'"; sg docker -c "docker pull ubuntu:22.04 2>&1 | tail -3"; echo EXIT=$?
cd /tmp && timeout 900 <guest-path> --container ubuntu:22.04 --qna-version 11.0 "name of operating system" ; echo EXIT=$?
cd <guest-path> && grep -rniE "cron|schedule|--once|webhook|while True|sleep\(" src | head; grep -c 'bigfix-remote-client-relevance --container ubuntu:22.04 --qna-version 11.0 "name of operating system"' README.md; grep -c "prints the JSON Schema for a single result and exits 0" README.md; grep -c "Evaluate against the BigFix client on this machine" README.md
cd <guest-path> && mkdir -p .coldpass && BIN=<guest-path> && cat > .coldpass/probe.json <<EOF
{"kind":"cli","argv":["$BIN","--container","ubuntu:22.04","--qna-version","11.0","name of operating system"],"cwd":"<guest-path>","expected_text":"Linux Ubuntu 22.04.5 LTS"}
EOF
cat > .coldpass/claims.json <<EOF
[{"kind":"claim","claim":"Evaluate a client relevance expression inside a container with a pinned qna version","source":"bigfix-remote-client-relevance --container ubuntu:22.04 --qna-version 11.0 \"name of operating system\"","input":"name of operating system in ubuntu:22.04 container, qna 11.0","expected":"Linux Ubuntu 22.04.5 LTS","probe_kind":"cli","argv":["$BIN","--container","ubuntu:22.04","--qna-version","11.0","name of operating system"],"expected_text":"Linux Ubuntu 22.04.5 LTS","cwd":"<guest-path>"},
{"kind":"claim","claim":"--schema prints the JSON Schema for a single result","source":"prints the JSON Schema for a single result and exits 0","input":"--schema","expected":"ClientRelevanceResult schema","probe_kind":"cli","argv":["$BIN","--schema"],"expected_text":"ClientRelevanceResult","cwd":"<guest-path>"}]
EOF
cat > .coldpass/examples.json <<EOF
[{"kind":"example","claim":"Print the JSON Schema for a single result without a target","source":"README.md ## Use / From a non-Python server","input":"--schema","expected":"schema title ClientRelevanceResult","argv":["$BIN","--schema"],"cwd":"<guest-path>","env":{},"expected_text":"ClientRelevanceResult"}]
EOF
cat > .coldpass/cycles.json <<'EOF'
{"detected":false,"reason":"Entry points inspected (cli.py, orchestrate.py, transports/*). The package is a one-shot CLI and library with no cron, scheduler, --once/--now, webhook or daemon loop; the while-True/sleep hits in source are polling and retry helpers (ssh_discovery, container, fastquery, online_evaluator), not product cycles. Each run evaluates once and exits.","checks":[]}
EOF
python3 -m json.tool .coldpass/*.json >/dev/null && echo JSON_OK; <env-var>=<guest-path> python3 -I <guest-path> --validate --replay 2>&1 | tail -40; echo EXIT=$?
cd <guest-path> && cat > .coldpass/report.json <<'EOF'
{"outcome":"working","fault_owner":null,"operation":"Evaluate a BigFix client-relevance expression (\"name of operating system\") inside an ubuntu:22.04 container with qna 11.0, via the README's documented --container command","operation_scope":"Requires Docker (installed via apt docker.io) and network access to Docker Hub and support.bigfix.com for the qna 11.0.7.61 artifact. Local --local, SSH and Fast Query paths were not exercised (no local BigFix client, no SSH host, no BES REST connection).","steps":["uv --version (uv 0.13.5 present; Python 3.12.3 satisfies .python-version 3.12 and requires-python >=3.11)","uv sync (in <guest-path>, per AGENTS.md Commands)","sudo apt-get -o DPkg::Lock::Timeout=300 install -y docker.io","docker pull ubuntu:22.04","bigfix-remote-client-relevance --container ubuntu:22.04 --qna-version 11.0 \"name of operating system\" -> 'Linux Ubuntu 22.04.5 LTS'","bigfix-remote-client-relevance --schema -> ClientRelevanceResult JSON Schema"],"deviations":["Docker was not preinstalled; installed docker.io via apt-get (Docker 29.1.3). Docker socket was not accessible to the ubuntu user after install, so I added ubuntu to the docker group and ran chmod 666 /var/run/docker.sock in this disposable guest to allow non-root use.","Ran the CLI from the pinned checkout's .venv (uv sync) rather than uv tool install bigfix-remote-client-relevance, so the tested code matches the pinned commit (version 1.7.0).","Skipped --local, SSH, --inventory and --fastquery targets: no local qna, no SSH endpoint, no BES REST connection in this guest.","No product cycle exists (one-shot CLI); cycles.json has detected=false."],"stuck_step":null,"main_blocker":null,"plain_failure":null,"suggested_fix":"None for the tested path. Optional: document that --container needs Docker and a user with permission to reach the Docker socket."}
EOF
python3 -c "import json;json.load(open('.coldpass/report.json'));print('report ok')"; ls -la .coldpass

No product verdict. Run record.

No product verdict. Run record.