coldpass

collectiveai-team/botica

GitHub ↗ · How this check works

Per-setup public labels: Linux / Haiku: Works · Linux / Luna: Works

Static status badge · SVG · read-only JSON. Embed with ![Coldpass status](https://coldpass.dev/badge/collectiveai-team--botica.svg).

agent-harness

Works

The recorded run passed on 2 setups.

Answers apply to each recorded setup, revision and operation. Earlier observations remain in Run details.

Tested on Linux
PlatformHaikuLuna
Linuxyesyes

Linux / Haiku 5.5: yes · Oct 27, 2026 · 49f2459e260b · claude-code / 2.1.310 (Claude Code)
Recorded operation: <guest-path> lint

Linux / Luna: yes · Oct 27, 2026 · 49f2459e260b · codex / codex-cli 0.164.0
Recorded operation: uvx botica lint

Does what it says
2 of 2 claims checked out
  • botica lint exits 1 when a finding is an error: Real input produced the expected result: error [no-utils] src/utils.py is a catch-all module

    Source: `botica lint` exits with 1 when a finding is an error and with 2 when the run fails.

    Input: sample project with src/utils.py

    Expected: exit 1 and the no-utils error

    Command: <guest-path> lint

    Observed: error [no-utils] src/utils.py is a catch-all module — name modules for what they hold, not utils/helpers/aux/misc/common. src/utils.py:2:5: warning [log-no-print] Libraries log, they don't print: use the house get_logger (`botica refs show structured-logging`) instead of print() or sys.stdout/stderr.write in library code. `__main__.py`, `cli.py`, `cli/`, the tests and `if __name__ == "__main__":` blocks are exempt; another entry point goes in `extend-ignores`.

  • botica explain says what a rule asks and how it is configured: Real input produced the expected result: no-utils (BTSTRUC002): No catch-all modules

    Source: # what a rule asks, and how it is configured here

    Input: no-utils

    Expected: the rule description

    Command: <guest-path> explain no-utils

    Observed: no-utils (BTSTRUC002): No catch-all modules state: active, selected by `no-utils` severity: error status: stable (since 0.7.0) family: structure standard: CES-63 languages: go, javascript, python, rust, typescript kind: native (python, typescript, javascript, go, rust) aliases: none tags: naming options: none config: /tmp/sample_proj/botica.toml ignores: none ## Directive Don't create catch-all modules. A module named `utils`, `helpers`, `aux`, `misc` or `common` is an error, anywhere except under `tests/`. It applies to the files of Python (`.py`), TypeScript (

MCP
No
Local or hosted
local
Local: runs on your machine
Needs keys or browser
None seen
Maintained
Maintained
actively maintained: last commit 0 days ago; 633 commits and 72 merged PRs in 90 days; 1 open issues, 10 open PRs; 3 of 4 recent threads got a maintainer reply, median first reply 481.7h; latest release 2026-10-27

Last commit Oct 27, 2026

Setup time
1 min 21 s
1 min 21 s to a working result
Issue
none found in recorded check
Examples
2 of 2 ran as expected
  • list every rule with whether it is active: command completed successfully

    Source: README.md

    Input: rules --all in sample project

    Expected: rule list including no-utils

    Command: <guest-path> rules --all

    Observed: TTOOL002 ruff-selection inactive not selected BTTOOL003 pyrefly-strict inactive not selected BTTOOL004 no-type-cast inactive not selected BTTOOL005 ruff-heavy-imports-in-function inactive not selected BTTOOL006 ruff-imports-at-top inactive not selected BTWRIT001 plain-writing inactive not selected; judged outside lint BTWRIT002 minimal-comments inactive not selected; judged outside lint BTWRIT003 ubiquitous-language inactive not selected; judged outside lint BTWRIT004 agents-md-refs-exist inactive not selected

  • read the reference page of a topic: command completed successfully

    Source: README.md

    Input: refs show structured-logging

    Expected: topic text

    Command: <guest-path> refs show structured-logging

    Observed: es and render your event dictionary to JSON that is picked up by a log aggregation system like ELK or Graylog, structlog comes with batteries included - you just have to tell it to use its JSONRenderer. ## https://www.structlog.org/en/stable/api.html#structlog.dev.ConsoleRenderer the console renderer formats exceptions itself, so the processor that turns them into data goes only before `JSONRenderer` > Changed in version 21.2.0: ConsoleRenderer now handles the exc_info event dict key itself. Do not use the structlog.processors.format_exc_info processor together with ConsoleRenderer anymore!

Evidence
README behavior checked
Agent interfaces
CLI
CLI

Observed interfaces were exercised by the checker. Documentation alone does not confirm compatibility.

Tested with
Claude Code / Haiku 5.5
  • Linux: Claude Code / Haiku 5.5 · agent 2.1.310 (Claude Code) · Checked operation passed · Oct 27, 2026 on 49f2459e260b

Each entry describes its recorded operation and source revision.

Safety
No unexpected network or file activity found in these checks

No problems traced to this repository code in these checks; a short run cannot prove safety. 12 listed items (named-host connections, static matches, dependency advisories) are below and do not change this verdict.

Gitleaks found a possible hardcoded secret; value withheld. 2 distinct locations. 2 findings.

OSV reports a known vulnerability in this dependency version. 4 distinct locations. 4 findings.

Reviewed as expected for a test run: read only test files inside the test workspace or its temporary folders.

Full event, path and process lists for each observed setup:

Repair history and test corrections
Run details

Haiku 5.5

Native agent · Checked operation passed

Checked operation <guest-path> lint

Agent-reported scope: Install botica and run its linter (botica lint) plus explain on a project with a botica.toml

Run details
run
collectiveai-team--botica-claude-haiku-5-5-1793140482031955127
date
Oct 27, 2026
time to working
80.8s
product hosting / execution
Local command in the guest
recorded operation scope
product operation
model
claude-haiku-5-5
agent / version
claude-code / 2.1.310 (Claude Code)
platform
linux
reported cost
$0.013802
Recorded source
https://github.com/collectiveai-team/botica
Backend
hyperv
Tool transport
mcp-assigned-vm
Requested source pin (not an observed revision)
{"commit": "49f2459e260b08e62648f901de8aa910a1b54bad", "url": "https://github.com/collectiveai-team/botica"}
Test version
readme-d5b140d3c0f8b95c2dd6fe6fc381a303cacb92679582c22317d8e78b572f0cf2
revision
49f2459e260b08e62648f901de8aa910a1b54bad
Current diagnosis: test · resolved

native Claude returned a different model

Repair record fix-353a73773e075990097f · current status JSON. Original run outcomes remain unchanged.

Failed run: collectiveai-team/botica · Oct 27, 2026 · Our test failed; no repo verdict.

Test-repair rerun: collectiveai-team/botica · Oct 27, 2026 · Checked operation passed.

Double-checks failed run: collectiveai-team/botica · Oct 27, 2026.

Fix: fix-353a73773e075990097f.

Safety: Review signals: 2 hardcoded secret, 4 known vulnerability.

Review signals: 2 hardcoded secret, 4 known vulnerability.

Reviewed as expected for a test run: read only test files inside the test workspace or its temporary folders.

Gitleaks found a possible hardcoded secret; value withheld. 2 distinct locations. 2 findings.

OSV reports a known vulnerability in this dependency version. 4 distinct locations. 4 findings.

OSV scanned supported lockfiles; unpinned dependencies and unsupported formats are outside this check.

A short run cannot establish that code is safe.

Connections show IP addresses; proxies can hide the final hostname.

Tracing covers dispatched commands and their children, not the whole OS; guest root can interfere with tracing.

Full safety evidence →

Answers recorded for this run
  • botica lint exits 1 when a finding is an error: Real input produced the expected result: error [no-utils] src/utils.py is a catch-all module

    Source: `botica lint` exits with 1 when a finding is an error and with 2 when the run fails.

    Input: sample project with src/utils.py

    Expected: exit 1 and the no-utils error

    Command: <guest-path> lint

    Observed: error [no-utils] src/utils.py is a catch-all module — name modules for what they hold, not utils/helpers/aux/misc/common. src/utils.py:2:5: warning [log-no-print] Libraries log, they don't print: use the house get_logger (`botica refs show structured-logging`) instead of print() or sys.stdout/stderr.write in library code. `__main__.py`, `cli.py`, `cli/`, the tests and `if __name__ == "__main__":` blocks are exempt; another entry point goes in `extend-ignores`.

  • botica explain says what a rule asks and how it is configured: Real input produced the expected result: no-utils (BTSTRUC002): No catch-all modules

    Source: # what a rule asks, and how it is configured here

    Input: no-utils

    Expected: the rule description

    Command: <guest-path> explain no-utils

    Observed: no-utils (BTSTRUC002): No catch-all modules state: active, selected by `no-utils` severity: error status: stable (since 0.7.0) family: structure standard: CES-63 languages: go, javascript, python, rust, typescript kind: native (python, typescript, javascript, go, rust) aliases: none tags: naming options: none config: /tmp/sample_proj/botica.toml ignores: none ## Directive Don't create catch-all modules. A module named `utils`, `helpers`, `aux`, `misc` or `common` is an error, anywhere except under `tests/`. It applies to the files of Python (`.py`), TypeScript (

Examples

  • list every rule with whether it is active: command completed successfully

    Source: README.md

    Input: rules --all in sample project

    Expected: rule list including no-utils

    Command: <guest-path> rules --all

    Observed: TTOOL002 ruff-selection inactive not selected BTTOOL003 pyrefly-strict inactive not selected BTTOOL004 no-type-cast inactive not selected BTTOOL005 ruff-heavy-imports-in-function inactive not selected BTTOOL006 ruff-imports-at-top inactive not selected BTWRIT001 plain-writing inactive not selected; judged outside lint BTWRIT002 minimal-comments inactive not selected; judged outside lint BTWRIT003 ubiquitous-language inactive not selected; judged outside lint BTWRIT004 agents-md-refs-exist inactive not selected

  • read the reference page of a topic: command completed successfully

    Source: README.md

    Input: refs show structured-logging

    Expected: topic text

    Command: <guest-path> refs show structured-logging

    Observed: es and render your event dictionary to JSON that is picked up by a log aggregation system like ELK or Graylog, structlog comes with batteries included - you just have to tell it to use its JSONRenderer. ## https://www.structlog.org/en/stable/api.html#structlog.dev.ConsoleRenderer the console renderer formats exceptions itself, so the processor that turns them into data goes only before `JSONRenderer` > Changed in version 21.2.0: ConsoleRenderer now handles the exc_info event dict key itself. Do not use the structlog.processors.format_exc_info processor together with ConsoleRenderer anymore!

Local or hosted: local

What the agent did

Install botica and run its linter (botica lint) plus explain on a project with a botica.toml

cd <guest-path> && pwd && ls -la && git status 2>&1 | head -3; id -u; python3 --version; cat README.md 2>/dev/null | head -200
ls <guest-path>; ls /inputs 2>&1; echo "CDP=$CDP_URL $<env-var>"; uname -a; cat /etc/os-release | head -3
which uv uvx cargo node npm git curl 2>&1; cd <guest-path> && git init -q 2>&1 && git remote add origin https://github.com/collectiveai-team/botica 2>&1; timeout 120 git fetch --depth 1 origin 49f2459e260b08e62648f901de8aa910a1b54bad 2>&1 | tail -3; git reset --mixed FETCH_HEAD 2>&1 | tail -2; git log --oneline -1 2>&1
cd /tmp && uv --version && timeout 600 uv tool install botica 2>&1 | tail -5; which botica; botica --version 2>&1
export PATH="<guest-path>"; botica --version; cd <guest-path> && timeout 600 botica lint; echo "EXIT=$?"
export PATH="<guest-path>"; mkdir -p /tmp/sample_proj/src && cd /tmp/sample_proj && printf 'select = ["file-size-guard", "no-utils", "log-no-print"]\n\n[rules.file-size-guard]\nwarn = 500\nmax = 900\n' > botica.toml && printf 'def f():\n    print("hi")\n' > src/utils.py && botica lint; echo "EXIT=$?"; botica explain no-utils | head -20; echo; botica rules --all 2>&1 | head -5
export PATH="<guest-path>"; rm -rf /tmp/rules_demo && mkdir -p /tmp/rules_demo && cd /tmp/rules_demo && botica rules new structure/no-eval --kind ast-grep --language python --catalog .botica/rules/acme; echo "EXIT=$?"; find .botica | head -20; botica rules conform .botica/rules/acme; echo "CONF=$?"; cd <guest-path>; grep -rniE "cron|--once|--now|webhook|watch mode|schedule" --include=*.md docs website/src 2>/dev/null | grep -v node_modules | head -10; ls docs
export PATH="<guest-path>"; botica rules new --help; echo ---; botica --help | head -40; echo ---; grep -rn "rules new" <guest-path> --include=*.md* 2>/dev/null | head -5
cd <guest-path> && ls extend; sed -n 1,60p extend/catalogs.md
export PATH="<guest-path>"; rm -rf /tmp/rules_demo && mkdir -p /tmp/rules_demo && cd /tmp/rules_demo && mkdir -p .botica/rules/acme/structure && printf 'code = "ACME"\n' > .botica/rules/acme/catalog.toml && printf 'code = "STRUC"\ndescription = "The layout of the project"\nextends = "structure"\n' > .botica/rules/acme/structure/family.toml && botica rules new structure/no-blobs --kind ast-grep --language python --catalog .botica/rules/acme; echo "NEW=$?"; botica rules conform .botica/rules/acme; echo "CONF=$?"; find .botica -type f | sort
export PATH="<guest-path>"; cd <guest-path> && grep -rniE "cron|interval|watch|--once|--now|webhook|sleep\(|tokio::time" crates --include=*.rs | grep -v test | head -10; echo ---; cd /tmp/sample_proj && botica explain no-utils | grep -E "^(no-utils|state)"; botica refs show structured-logging | head -5; echo "REFS=$?"; cd /tmp/sample_proj && botica rules --all | grep -c "" 
cd <guest-path> && mkdir -p .coldpass && B=<guest-path> && cat > .coldpass/probe.json <<EOF
{"kind":"cli","argv":["$B","lint"],"cwd":"/tmp/sample_proj","expected_returncode":1,"expected_text":"error [no-utils] src/utils.py is a catch-all module"}
EOF
cat > .coldpass/claims.json <<EOF
[
{"kind":"claim","claim":"botica lint exits 1 when a finding is an error","source":"\`botica lint\` exits with 1 when a finding is an error and with 2 when the run fails.","input":"sample project with src/utils.py","expected":"exit 1 and the no-utils error","probe_kind":"cli","argv":["$B","lint"],"cwd":"/tmp/sample_proj","expected_returncode":1,"expected_text":"error [no-utils] src/utils.py is a catch-all module"},
{"kind":"claim","claim":"botica explain says what a rule asks and how it is configured","source":"# what a rule asks, and how it is configured here","input":"no-utils","expected":"the rule description","probe_kind":"cli","argv":["$B","explain","no-utils"],"cwd":"/tmp/sample_proj","expected_text":"no-utils (BTSTRUC002): No catch-all modules"}
]
EOF
cat > .coldpass/examples.json <<EOF
[
{"kind":"example","claim":"list every rule with whether it is active","source":"README.md","input":"rules --all in sample project","expected":"rule list including no-utils","argv":["$B","rules","--all"],"cwd":"/tmp/sample_proj","expected_text":"no-utils"},
{"kind":"example","claim":"read the reference page of a topic","source":"README.md","input":"refs show structured-logging","expected":"topic text","argv":["$B","refs","show","structured-logging"],"cwd":"/tmp/sample_proj","expected_text":"Structured logging with structlog"}
]
EOF
cat > .coldpass/cycles.json <<EOF
{"detected":false,"reason":"Inspected README.md and crates/ entry points: botica lint, rules, explain, refs, recipe and rules new are one-shot commands; the only long-running entry point is botica server (LSP over stdio), started by an editor, not a schedule. No cron, --once, --now or webhook in the product.","checks":[]}
EOF
for f in probe claims examples cycles; do python3 -I -m json.tool .coldpass/$f.json >/dev/null && echo "$f ok"; done; cat .coldpass/claims.json | head -3
cd <guest-path> && cat > .coldpass/report.json <<'EOF'
{"outcome":"working","fault_owner":null,"operation":"Install botica and run its linter (botica lint) plus explain on a project with a botica.toml","operation_scope":"Linux x86_64 Ubuntu 24.04, uv 0.13.5, botica 0.17.0 from PyPI; no browser, network or credentials needed for lint","steps":["git init; git remote add origin https://github.com/collectiveai-team/botica; git fetch --depth 1 origin 49f2459e260b08e62648f901de8aa910a1b54bad; git reset --mixed FETCH_HEAD","uv tool install botica","botica lint (in repo: exit 0 with file-size-guard warnings)","botica.toml with select = [file-size-guard, no-utils, log-no-print]; botica lint on src/utils.py -> exit 1 with error [no-utils]","botica explain no-utils","botica rules --all","botica refs show structured-logging"],"deviations":["uv tool install put botica in <guest-path>, which is not on PATH; used the absolute path <guest-path> in all evidence","The /inputs/coldpass-preflight.py path given in the task does not exist; ran <guest-path> as instructed","botica rules new + rules conform on a scaffolded rule fails with 'fixtures/expected.json lists nothing' because the generated fixtures are TODO templates; this is the documented scaffold state, not a product failure, and was not used as evidence"],"stuck_step":null,"main_blocker":null,"plain_failure":null,"suggested_fix":"none required"}
EOF
python3 -I -m json.tool .coldpass/report.json >/dev/null && echo report ok; <env-var>=<guest-path> python3 -I <guest-path> --validate --replay 2>&1 | tail -40; echo "CHECK=$?"
Earlier attempts (1)

Luna

Native agent · Checked operation passed

Checked operation uvx botica lint

Agent-reported scope: Run the documented Botica lint command over the checkout

Run details
run
collectiveai-team--botica-gpt-6-luna-1793130484790422263
date
Oct 27, 2026
time to working
33.0s
product hosting / execution
Local command in the guest
recorded operation scope
product operation
model
gpt-6-luna
agent / version
codex / codex-cli 0.164.0
platform
linux
reported cost
not reported (not assumed free)
Recorded source
https://github.com/collectiveai-team/botica
Backend
hyperv
Tool transport
mcp-assigned-vm
Requested source pin (not an observed revision)
{"commit": "49f2459e260b08e62648f901de8aa910a1b54bad", "url": "https://github.com/collectiveai-team/botica"}
Test version
readme-38e6b13f3b13804effb95797e3466b1a1334da7a232d897cbc8965867f802605
revision
49f2459e260b08e62648f901de8aa910a1b54bad
Whose fault: not assigned · resolved

Safety: Review signals: 2 hardcoded secret, 4 known vulnerability.

Review signals: 2 hardcoded secret, 4 known vulnerability.

Reviewed as expected for a test run: read only test files inside the test workspace or its temporary folders.

Gitleaks found a possible hardcoded secret; value withheld. 2 distinct locations. 2 findings.

OSV reports a known vulnerability in this dependency version. 4 distinct locations. 4 findings.

OSV scanned supported lockfiles; unpinned dependencies and unsupported formats are outside this check.

A short run cannot establish that code is safe.

Connections show IP addresses; proxies can hide the final hostname.

Tracing covers dispatched commands and their children, not the whole OS; guest root can interfere with tracing.

Full safety evidence →

Answers recorded for this run
  • botica lint runs rules over project files and reports findings: Real input produced the expected result: warning [file-size-guard]

    Source: Botica is a linter and a set of recipes for the repositories of a team.

    Input: repository checkout

    Expected: lint findings are printed

    Command: uvx botica lint

    Observed: warning [file-size-guard] crates/botica-astgrep/src/chains.rs has 685 lines (warns at 400, fails at 700) crates/botica-catalog/src/recipes_params.rs:1:1: warning [file-size-guard] crates/botica-catalog/src/recipes_params.rs has 510 lines (warns at 400, fails at 700) crates/botica-catalog/src/recipes_steps.rs:1:1: warning [file-size-guard] crates/botica-catalog/src/recipes_steps.rs has 466 lines (warns at 400, fails at 700) crates/botica-catalog/tests/catalog.rs:1:1: warning [file-size-guard] crates/botica-catalog/tests/catalog.rs has 411 lines (warns at 400, fails at 700) crates/botica-catalog

Examples

  • Run lint over the current directory: command completed successfully

    Source: README.md / Use

    Input: current repository

    Expected: findings printed

    Command: uvx botica lint

    Observed: has 481 lines (warns at 400, fails at 700) crates/botica-lsp/src/server.rs:1:1: warning [file-size-guard] crates/botica-lsp/src/server.rs has 463 lines (warns at 400, fails at 700) crates/botica-lsp/tests/common/mod.rs:1:1: warning [file-size-guard] crates/botica-lsp/tests/common/mod.rs has 407 lines (warns at 400, fails at 700) scripts/check_rule_change.py:1:1: warning [file-size-guard] scripts/check_rule_change.py has 409 lines (warns at 400, fails at 700) tests/test_check_rule_change.py:1:1: warning [file-size-guard] tests/test_check_rule_change.py has 518 lines (warns at 400, fails at 700)

Local or hosted: local

What the agent did

Run the documented Botica lint command over the checkout

pwd && ls -la && cat README.md && python3 --version && id -u && git status --short --branch
cat docs/develop.md; command -v uv || true; command -v cargo || true; command -v rustc || true; command -v npx || true; rg -n "cycle|schedule|webhook|--once|--now|cron" crates README.md
/
uv --version && cargo --version && rustc --version && uvx botica lint
mkdir -p .coldpass; python3 - <<'PY'
import json
p={"kind":"cli","argv":["uvx","botica","lint"],"cwd":"<guest-path>","expected_text":"warning [file-size-guard]"}
c=[{"kind":"claim","claim":"botica lint runs rules over project files and reports findings","source":"`botica lint` runs rules over the files of a project: the built-in catalog, and the rules","input":"repository checkout","expected":"lint findings are printed","probe_kind":"cli","argv":["uvx","botica","lint"],"expected_text":"warning [file-size-guard]","cwd":"<guest-path>"}]
open('.coldpass/probe.json','w').write(json.dumps(p,indent=2)+'\n')
open('.coldpass/claims.json','w').write(json.dumps(c,indent=2)+'\n')
open('.coldpass/examples.json','w').write(json.dumps([{"kind":"example","claim":"Run lint over the current directory","source":"README.md / Use","input":"current repository","expected":"findings printed","argv":["uvx","botica","lint"],"cwd":"<guest-path>","expected_text":"warning [file-size-guard]"}],indent=2)+'\n')
open('.coldpass/cycles.json','w').write(json.dumps({"detected":False,"reason":"README and entry points describe command-driven lint and recipe operations; no scheduler, recurring loop, or webhook cycle is documented.","checks":[]},indent=2)+'\n')
PY
<env-var>=<guest-path> python3 -I <guest-path> --validate --replay
python3 - <<'PY'
import json
p='.coldpass/claims.json'; a=json.load(open(p)); a[0]['source']='`botica lint` runs rules over the files of a project:'; open(p,'w').write(json.dumps(a,indent=2)+'\n')
PY
<env-var>=<guest-path> python3 -I <guest-path> --validate --replay
python3 - <<'PY'
import json
p='.coldpass/claims.json'; a=json.load(open(p)); a[0]['source']='Botica is a linter and a set of recipes for the repositories of a team.'; open(p,'w').write(json.dumps(a,indent=2)+'\n')
PY
<env-var>=<guest-path> python3 -I <guest-path> --validate --replay
python3 - <<'PY'
import json
p={"outcome":"working","fault_owner":None,"operation":"Run the documented Botica lint command over the checkout","operation_scope":"uvx installs the published CLI in its isolated environment; Python 3.12.3 and uv 0.13.5 were available.","steps":["Read README Install and Use instructions.","Ran uvx botica lint from the repository root; it returned 0 and printed rule findings."],"deviations":[],"stuck_step":None,"main_blocker":None,"plain_failure":None,"suggested_fix":None}
open('.coldpass/report.json','w').write(json.dumps(p,indent=2)+'\n')
PY

No product verdict. Run record.