Coolhand-Labs/coolhand-python

Review signals: 1 connection review, 8 known vulnerability. 8 static matches retained as context.

Reviewed as expected for a test run: attempted writes that failed, so nothing was written.

Repository processes attempted these non-loopback connections; proxies can hide the final hostname. 35.241.4.177:443 10 attributed processes.

OSV reports a known vulnerability in this dependency version. 5 distinct locations. 8 findings.

8 hardcoded secret matches retained as context; full list in the evidence.

OSV scanned supported lockfiles; unpinned dependencies and unsupported formats are outside this check.

A short run cannot establish that code is safe.

Connections show IP addresses; proxies can hide the final hostname.

Tracing covers dispatched commands and their children, not the whole OS; guest root can interfere with tracing.

Full safety evidence →

• Streamed responses are automatically logged to Coolhand: Real input produced the expected result: PASS streaming_capture: 5 sync streams captured with token metadata

  Source: Complete streamed response automatically logged to Coolhand!

  Input: sync SSE streams from anthropic and openai shapes via httpx

  Expected: streams captured with message id and token counts

  Command: <guest-path> examples/streaming_capture.py

  Observed: PASS streaming_capture: 5 sync streams captured with token metadata

• Importing coolhand auto-initializes monitoring: Real input produced the expected result: Monitoring active: True

  Source: Auto-initializes and starts monitoring

  Input: import coolhand via auto_monitor_example.py

  Expected: monitoring reports active

  Command: <guest-path> examples/auto_monitor_example.py

  Observed: Monitoring active: True Cleaning up... Data flushed and delivery attempted ======================================================= Auto-monitor example completed! Key takeaways: 1. Just 'import coolhand' can be enough to get started 2. Auto-initialization detects AI/ML environments 3. Zero configuration needed in many cases 4. Pass silent=True/False and base_url to Coolhand() for environment-specific setups 5. Environment variables control auto-initialization No matching field provided. Feedback may not be linked to an LLM request. Consider providing one of: llm_request_log_id, llm_provider

Examples

• Offline streaming capture check passes without keys: command completed successfully

  Source: examples/README.md Streaming capture check

  Input: streaming_capture.py

  Expected: PASS lines and exit 0

  Command: <guest-path> examples/streaming_capture.py

  Observed: PASS anthropic via iter_bytes: id and token counts captured PASS anthropic via iter_lines: id and token counts captured PASS openai via iter_lines: id and token counts captured PASS openai via iter_bytes: id and token counts captured PASS openai stopping at [DONE]: id and token counts captured PASS streaming_capture: 5 sync streams captured with token metadata

• Zero-config monitoring example runs end to end: command completed successfully

  Source: examples/README.md table row auto_monitor_example.py

  Input: auto_monitor_example.py

  Expected: example completes and reports delivery attempted

  Command: <guest-path> examples/auto_monitor_example.py

  Observed: id, original_output, or client_unique_id WARNING:coolhand.feedback_service:Failed to submit feedback: HTTP 401 - Unauthorized INFO:coolhand.client:Shutting down Coolhand... INFO:coolhand.client:Coolhand shutdown complete INFO:coolhand.client:Shutting down Coolhand... INFO:coolhand.client:Coolhand shutdown complete INFO:coolhand.client:Shutting down Coolhand... INFO:coolhand.client:Coolhand shutdown complete INFO:coolhand.client:Shutting down Coolhand... INFO:coolhand.client:Coolhand shutdown complete INFO:coolhand.client:Shutting down Coolhand... INFO:coolhand.client:Coolhand shutdown complete

• Repo test suite passes offline: command completed successfully

  Source: Makefile test target

  Input: make test

  Expected: pytest summary with passed tests

  Command: make test

  Observed: omDramatiqThread::test_pydantic_ai_multiple_tasks <guest-path> DeprecationWarning: The model 'claude-3-5-haiku-20241022' is deprecated and will reach end-of-life on February 19th, 2026. Please migrate to a newer model. Visit https://docs.anthropic.com/en/docs/resources/model-deprecations for more information. return cast(BetaMessage, await send(False)) -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html ================= 1413 passed, 3 skipped, 3 warnings in 45.97s =================

Cycles

• Queued interactions are delivered by a background worker loop and drained on shutdown: Triggered one cycle by hand (run-once: auto_monitor_example.py enqueues interactions then calls shutdown/flush; normal trigger: queue-driven, no timer: worker blocks on the dispatch queue and drains on shutdown): Setting up demonstration environment... Coolhand Python SDK - Auto-Monitor Example ======================================================= Auto-Initialization Example ================================================== ✓ Coolhand was automatically initialized! Session ID: session_1793148889232 Monitoring enabled: True Config source: Environment variables and auto-detection Has API key: False Zero-Configuration Usage ------------------------------ Using Coolhand with zero configuration... ✓ Submitted positive feedback ✓ Submitted feedback with explanation Making simulated OpenAI API request... ✓ Request completed and automatically logged Making simulated Anthropic API request... ✓ Request completed and automatically logged ✓ All requests automatically monitored and logged Different Setup Modes ------------------------- 1. Notebook Setup (interactive, verbose): ✓ silent: False 2. Production Setup (silent): ✓ silent: True 3. Self-Hosted Setup (custom base_url): ✓ base_url: https://coolhandlabs.com Environment Detection -------------------- Auto-initialization activates when it detects: ✓ AI API keys found: OPENAI_API_KEY ○ No AI/ML packages detected ○ Not in interactive session To control auto-initialization: - Set COOLHAND_AUTO_INIT=false to disable - Set COOLHAND_ENABLED=true to force enable - Set COOLHAND_API_KEY to enable with your API key Monitoring Context Manager ------------------------- Current monitoring status: True Temporarily stopping and restarting monitoring: Monitoring stopped — HTTP requests will pass through unlogged Monitoring restored: True Collecting User Feedback ----------------------- ✓ Collected feedback on auto-monitoring experience Final Status ------------ Session ID: session_1793148889232 Total interactions logged: 2 Monitoring active: True Cleaning up... Data flushed and delivery attempted ======================================================= Auto-monitor example completed! Key takeaways: 1. Just 'import coolhand' can be enough to get started 2. Auto-initialization detects AI/ML environments 3. Zero configuration needed in many cases 4. Pass silent=True/False and base_url to Coolhand() for environment-specific setups 5. Environment variables control auto-initialization No matching field provided. Feedback may not be linked to an LLM request. Consider providing one of: llm_request_log_id, llm_provider_unique_id, original_output, or client_unique_id Failed to submit feedback: HTTP 401 - Unauthorized No matching field provided. Feedback may not be linked to an LLM request. Consider providing one of: llm_request_log_id, llm_provider_unique_id, original_output, or client_unique_id Failed to submit feedback: HTTP 401 - Unauthorized INFO:coolhand:HTTP monitoring started INFO:coolhand:Coolhand initialized (session: session_1793148890106) INFO:coolhand:HTTP monitoring started INFO:coolhand:Coolhand initialized (session: session_1793148890110) INFO:coolhand:HTTP monitoring started INFO:coolhand:Coolhand initialized (session: session_1793148890114) INFO:coolhand.httpx_interceptor:Global HTTP monitoring disabled INFO:coolhand:HTTP monitoring stopped INFO:coolhand.httpx_interceptor:Global HTTP monitoring enabled INFO:coolhand:HTTP monitoring started WARNING:coolhand.feedback_service:No matching field provided. Feedback may not be linked to an LLM request. Consider providing one of: llm_request_log_id, llm_provider_unique_id, original_output, or client_unique_id WARNING:coolhand.feedback_service:Failed to submit feedback: HTTP 401 - Unauthorized WARNING:coolhand.feedback_service:No matching field provided. Feedback may not be linked to an LLM request. Consider providing one of: llm_request_log_id, llm_provider_unique_id, original_output, or client_unique_id WARNING:coolhand.feedback_service:Failed to submit feedback: HTTP 401 - Unauthorized INFO:coolhand.client:Shutting down Coolhand... INFO:coolhand.client:Coolhand shutdown complete INFO:coolhand.client:Shutting down Coolhand... INFO:coolhand.client:Coolhand shutdown complete INFO:coolhand.client:Shutting down Coolhand... INFO:coolhand.client:Coolhand shutdown complete INFO:coolhand.client:Shutting down Coolhand... INFO:coolhand.client:Coolhand shutdown complete INFO:coolhand.client:Shutting down Coolhand... INFO:coolhand.client:Coolhand shutdown complete

  Source: src/coolhand/client.py

  Input: auto_monitor_example.py with no COOLHAND_API_KEY

  Expected: example reports data flushed and delivery attempted

  Command: <guest-path> examples/auto_monitor_example.py

  Observed: Setting up demonstration environment... Coolhand Python SDK - Auto-Monitor Example ======================================================= Auto-Initialization Example ================================================== ✓ Coolhand was automatically initialized! Session ID: session_1793148889232 Monitoring enabled: True Config source: Environment variables and auto-detection Has API key: False Zero-Configuration Usage ------------------------------ Using Coolhand with zero configuration... ✓ Submitted positive feedback ✓ Submitted feedback with explanation Making simulated OpenAI API request... ✓ Request completed and automatically logged Making simulated Anthropic API request... ✓ Request completed and automatically logged ✓ All requests automatically monitored and logged Different Setup Modes ------------------------- 1. Notebook Setup (interactive, verbose): ✓ silent: False 2. Production Setup (silent): ✓ silent: True 3. Self-Hosted Setup (custom base_url): ✓ base_url: https://coolhandlabs.com Environment Detection -------------------- Auto-initialization activates when it detects: ✓ AI API keys found: OPENAI_API_KEY ○ No AI/ML packages detected ○ Not in interactive session To control auto-initialization: - Set COOLHAND_AUTO_INIT=false to disable - Set COOLHAND_ENABLED=true to force enable - Set COOLHAND_API_KEY to enable with your API key Monitoring Context Manager ------------------------- Current monitoring status: True Temporarily stopping and restarting monitoring: Monitoring stopped — HTTP requests will pass through unlogged Monitoring restored: True Collecting User Feedback ----------------------- ✓ Collected feedback on auto-monitoring experience Final Status ------------ Session ID: session_1793148889232 Total interactions logged: 2 Monitoring active: True Cleaning up... Data flushed and delivery attempted ======================================================= Auto-monitor example completed! Key takeaways: 1. Just 'import coolhand' can be enough to get started 2. Auto-initialization detects AI/ML environments 3. Zero configuration needed in many cases 4. Pass silent=True/False and base_url to Coolhand() for environment-specific setups 5. Environment variables control auto-initialization No matching field provided. Feedback may not be linked to an LLM request. Consider providing one of: llm_request_log_id, llm_provider_unique_id, original_output, or client_unique_id Failed to submit feedback: HTTP 401 - Unauthorized No matching field provided. Feedback may not be linked to an LLM request. Consider providing one of: llm_request_log_id, llm_provider_unique_id, original_output, or client_unique_id Failed to submit feedback: HTTP 401 - Unauthorized INFO:coolhand:HTTP monitoring started INFO:coolhand:Coolhand initialized (session: session_1793148890106) INFO:coolhand:HTTP monitoring started INFO:coolhand:Coolhand initialized (session: session_1793148890110) INFO:coolhand:HTTP monitoring started INFO:coolhand:Coolhand initialized (session: session_1793148890114) INFO:coolhand.httpx_interceptor:Global HTTP monitoring disabled INFO:coolhand:HTTP monitoring stopped INFO:coolhand.httpx_interceptor:Global HTTP monitoring enabled INFO:coolhand:HTTP monitoring started WARNING:coolhand.feedback_service:No matching field provided. Feedback may not be linked to an LLM request. Consider providing one of: llm_request_log_id, llm_provider_unique_id, original_output, or client_unique_id WARNING:coolhand.feedback_service:Failed to submit feedback: HTTP 401 - Unauthorized WARNING:coolhand.feedback_service:No matching field provided. Feedback may not be linked to an LLM request. Consider providing one of: llm_request_log_id, llm_provider_unique_id, original_output, or client_unique_id WARNING:coolhand.feedback_service:Failed to submit feedback: HTTP 401 - Unauthorized INFO:coolhand.client:Shutting down Coolhand... INFO:coolhand.client:Coolhand shutdown complete INFO:coolhand.client:Shutting down Coolhand... INFO:coolhand.client:Coolhand shutdown complete INFO:coolhand.client:Shutting down Coolhand... INFO:coolhand.client:Coolhand shutdown complete INFO:coolhand.client:Shutting down Coolhand... INFO:coolhand.client:Coolhand shutdown complete INFO:coolhand.client:Shutting down Coolhand... INFO:coolhand.client:Coolhand shutdown complete

Local or hosted: both

Run the coolhand Python SDK's offline monitoring examples and repo test suite: HTTP stream capture (examples/streaming_capture.py) and zero-config auto-monitoring (examples/auto_monitor_example.py)

cd <guest-path> && pwd && ls -la && git status 2>&1 | head -3; id -u; python3 --version; echo "CDP_URL=$CDP_URL"; test -f README.md && wc -c README.md
cd <guest-path> && cat README.md | head -c 7900
cd <guest-path> && tail -c +7900 README.md | head -c 7500; echo; echo ---; cat .python-version; grep -n "requires-python\|scripts\|\[project" pyproject.toml; echo ---; cat Makefile | head -80
cd <guest-path> && python3 -m venv --help >/dev/null 2>&1 && echo venv_ok || echo venv_missing; python3 -m pip --version 2>&1; which uv; curl -sS -m 10 -o /dev/null -w "%{http_code}\n" https://coolhandlabs.com/ 2>&1; curl -sS -m 10 -w "\n%{http_code}\n" $CDP_URL/json/version 2>&1 | head -c 600
mkdir -p <guest-path> && python3 -m venv <guest-path> && <guest-path> -m pip install -q --upgrade pip 2>&1 | tail -3; <guest-path> --version
cd <guest-path> && <guest-path> -m pip install -q <guest-path> 2>&1 | tail -5; echo "rc=$?"; <guest-path> -m pip show coolhand 2>&1 | head -3
cd <guest-path> && find examples scripts -type f | head -50; echo ---; cat examples/README.md | head -c 4000
cd <guest-path> && head -c 3500 docs/help-center.md; echo; echo ---; grep -n "scripts" -A4 pyproject.toml | head; echo ---; cat scripts/live-examples.sh | head -30
cd <guest-path> && timeout 120 <guest-path> examples/streaming_capture.py; echo "rc=$?"
cd <guest-path> && timeout 120 <guest-path> examples/auto_monitor_example.py 2>&1 | tail -30; echo "rc=${PIPESTATUS[0]}"
cd <guest-path> && sed -n 1,68p pyproject.toml | grep -v "^$" | head -70
cd <guest-path> && timeout 900 make install 2>&1 | tail -15; echo "rc=${PIPESTATUS[0]}"; ls -d .venv 2>/dev/null && .venv/bin/python --version
cd <guest-path> && timeout 300 uv run python examples/streaming_capture.py; echo "rc=$?"
cd <guest-path> && timeout 900 make test 2>&1 | tail -8; echo "rc=${PIPESTATUS[0]}"
cd <guest-path> && cat README.md | head -c 3500; echo ---; cat requirements.txt; cat .env.example
cd <guest-path> && grep -rn "Thread(\|while True\|schedule\|heartbeat\|--once\|--now\|webhook" src/coolhand/*.py | grep -v "^\s*#" | head -30; ls src/coolhand; cat .coldpass/* 2>/dev/null | head -5; ls -la .coldpass
cd <guest-path> && sed -n 606,680p src/coolhand/client.py
cd <guest-path> && PY=<guest-path> && cat > probe.json <<EOF
{"kind":"cli","argv":["$PY","examples/streaming_capture.py"],"cwd":"<guest-path>","expected_text":"PASS streaming_capture: 5 sync streams captured with token metadata"}
EOF
cat > claims.json <<EOF
[
{"kind":"claim","claim":"Streamed responses are automatically logged to Coolhand","source":"Complete streamed response automatically logged to Coolhand!","input":"sync SSE streams from anthropic and openai shapes via httpx","expected":"streams captured with message id and token counts","probe_kind":"cli","argv":["$PY","examples/streaming_capture.py"],"expected_text":"PASS streaming_capture: 5 sync streams captured with token metadata","cwd":"<guest-path>"},
{"kind":"claim","claim":"Importing coolhand auto-initializes monitoring","source":"Auto-initializes and starts monitoring","input":"import coolhand via auto_monitor_example.py","expected":"monitoring reports active","probe_kind":"cli","argv":["$PY","examples/auto_monitor_example.py"],"expected_text":"Monitoring active: True","cwd":"<guest-path>"}
]
EOF
python3 -I -c 'import json;[json.load(open(f)) for f in ["probe.json","claims.json"]];print("json ok")'
cd <guest-path> && PY=<guest-path> && cat > examples.json <<EOF
[
{"kind":"example","claim":"Offline streaming capture check passes without keys","source":"examples/README.md Streaming capture check","input":"streaming_capture.py","expected":"PASS lines and exit 0","argv":["$PY","examples/streaming_capture.py"],"cwd":"<guest-path>","expected_text":"PASS streaming_capture: 5 sync streams captured with token metadata"},
{"kind":"example","claim":"Zero-config monitoring example runs end to end","source":"examples/README.md table row auto_monitor_example.py","input":"auto_monitor_example.py","expected":"example completes and reports delivery attempted","argv":["$PY","examples/auto_monitor_example.py"],"cwd":"<guest-path>","expected_text":"Auto-monitor example completed!"},
{"kind":"example","claim":"Repo test suite passes offline","source":"Makefile test target","input":"make test","expected":"pytest summary with passed tests","argv":["make","test"],"cwd":"<guest-path>","expected_text":"passed"}
]
EOF
cat > cycles.json <<EOF
{"detected":true,"checks":[{"claim":"Queued interactions are delivered by a background worker loop and drained on shutdown","source":"src/coolhand/client.py","quote":"item = self._dispatch_queue.get()","schedule":"queue-driven, no timer: worker blocks on the dispatch queue and drains on shutdown","method":"run-once: auto_monitor_example.py enqueues interactions then calls shutdown/flush","input":"auto_monitor_example.py with no COOLHAND_API_KEY","expected":"example reports data flushed and delivery attempted","kind":"cli","argv":["$PY","examples/auto_monitor_example.py"],"cwd":"<guest-path>","expected_text":"Data flushed and delivery attempted"}]}
EOF
python3 -I -c 'import json;[json.load(open(f)) for f in ["examples.json","cycles.json"]];print("json ok")'
cd <guest-path> && ls -la <guest-path> 2>&1 | head; <env-var>=<guest-path> timeout 900 python3 -I <guest-path> --validate --replay 2>&1 | tail -c 6000; echo "rc=${PIPESTATUS[0]}"
cd <guest-path> && <env-var>=<guest-path> timeout 900 python3 -I <guest-path> --validate --replay > <guest-path> 2>&1; echo "rc=$?"; grep -o '"valid": [a-z]*' <guest-path> | sort | uniq -c; grep -o '"verdict": "[a-z]*"' <guest-path> | sort | uniq -c; grep -o '"claim_checks[^[]*' <guest-path> | head -c 300; echo; grep -o '"error[^,]*' <guest-path> | head -5
cd <guest-path> && cat > report.json <<'EOF'
{"outcome":"working","fault_owner":null,"operation":"Run the coolhand Python SDK's offline monitoring examples and repo test suite: HTTP stream capture (examples/streaming_capture.py) and zero-config auto-monitoring (examples/auto_monitor_example.py)","operation_scope":"Local source snapshot at pinned tree, Python 3.12.3, Linux guest, no COOLHAND_API_KEY and no provider keys. Covers SDK interception of httpx streams and auto-init, not live delivery to Coolhand servers or provider calls.","steps":["python3 -m venv <guest-path> && pip install <guest-path> (coolhand 0.11.1 installed from source)","make install (uv sync --all-extras, creates .venv on Python 3.12.3)","uv run python examples/streaming_capture.py (PASS x5, exit 0)","examples/auto_monitor_example.py (completed, exit 0)","make test (1413 passed, 3 skipped)","COOLHAND_WORKSPACE=... check.py --validate --replay (valid=true)"],"deviations":["README says 'pip install coolhand'; installed the snapshot source instead to test the pinned code.","The library has no console script or CLI; the documented runnable entry points are the examples/ scripts, which were used as the probe entry points.","First run of examples/streaming_capture.py failed with ModuleNotFoundError: httpx (httpx is only in the 'test' extra); fixed by running the README-documented make install.","HelpCenterDocs and Coolhand API calls were not exercised: no product CLI and no COOLHAND_API_KEY. Feedback during auto_monitor_example returns HTTP 401 without a key, which is expected for an unkeyed run.","Git pin not applied: the workspace has no git metadata and I did not run the git init/fetch steps, so the snapshot was used as delivered.","CDP endpoint: GET http://<internal-ip>:9226/json/version returned HTTP 200 with Chrome/155 JSON; no browser was needed by this SDK."],"stuck_step":null,"main_blocker":null,"plain_failure":null,"suggested_fix":null}
EOF
python3 -I -c 'import json;d=json.load(open("report.json"));print(d["outcome"], d["fault_owner"])'; ls -la <guest-path>
cd <guest-path> && sed -i 's/COOLHAND_WORKSPACE=\.\.\. check\.py/<env-var>=<guest-path> python3 -I <guest-path>' report.json && python3 -I -c 'import json;d=json.load(open("report.json"));print([s for s in d["steps"] if "check" in s])'