{ "checker": { "claims": [ { "claim": "demo runs the guard in front of a built-in paper broker with simulated prices", "command": "node src/cli.js demo tools/call {\"tool\": \"get_stock_latest_quote\", \"arguments\": {\"symbol\": \"SPY\"}}", "evidence": "\u2026simulated\\\"\\n}\"}]}", "expected": "simulated quote", "input": "{\"tool\": \"get_stock_latest_quote\", \"arguments\": {\"symbol\": \"SPY\"}}", "kind": "claim", "outcome": "Real input produced the expected result: simulated", "source": "`tas-guard demo` runs the guard in front of a built-in paper broker with simulated prices.", "source_checked": true, "source_file": "README.md", "verdict": "yes" }, { "claim": "1 SPY at $580 is allowed", "command": "node src/cli.js demo tools/call {\"tool\": \"place_stock_order\", \"arguments\": {\"symbol\": \"SPY\", \"side\": \"buy\", \"qty\": \"1\", \"type\": \"limit\", \"limit_price\": \"580\"}}", "evidence": "\u2026accepted\\\"\\n }\\n}\"}]}", "expected": "order accepted", "input": "{\"tool\": \"place_stock_order\", \"arguments\": {\"symbol\": \"SPY\", \"side\": \"buy\", \"qty\": \"1\", \"type\": \"limit\", \"limit_price\": \"580\"}}", "kind": "claim", "outcome": "Real input produced the expected result: accepted", "source": "1 SPY at $580 (allowed)", "source_checked": true, "source_file": "README.md", "verdict": "yes" }, { "claim": "tas_guard_status shows the limits the guard is working under", "command": "node src/cli.js demo tools/call {\"tool\": \"tas_guard_status\", \"arguments\": {}}", "evidence": "\u2026\\\"max_order_value\\\": 2000,\\n \\\"max_daily_value\\\": 5000,\\n \\\"max_orders_per_day\\\": 20,\\n \\\"max_orders_per_minute\\\": 3,\\n \\\"approve_over\\\": 1000,\\n \\\"symbols_allow\\\": [\\n \\\"SPY\\\",\\n \\\"QQQ\\\",\\n \\\"AAPL\\\",\\n \\\"MSFT\\\",\\n \\\"NVDA\\\"\\n ],\\n \\\"symbols_block\\\": [],\\n \\\"live_requires_approval\\\": false\\n },\\n \\\"today\\\": {\\n \\\"day\\\": \\\"2026-10-28\\\",\\n \\\"value_used\\\": 1160,\\n \\\"value_left\\\": 3840,\\n \\\"orders\\\": 2,\\n \\\"orders_left\\\": 18\\n },\\n \\\"pending_approvals\\\": []\\n}\"}]}", "expected": "status reports max_order_value 2000", "input": "{\"tool\": \"tas_guard_status\", \"arguments\": {}}", "kind": "claim", "outcome": "Real input produced the expected result: \\\"max_order_value\\\": 2000", "source": "Ask the agent to call `tas_guard_status` to see the limits it's working under.", "source_checked": true, "source_file": "README.md", "verdict": "yes" } ], "cycle_checks": [ { "claim": "Scheduled or looping main behavior", "command": "", "evidence": "Inspected README.md (Quick start, Try it, Phone approvals and kill switch) and src/cli.js, src/telegram.js, src/proxy.js. The only recurring loop is Telegram long-polling (src/telegram.js poll) that talks to an external Telegram Bot API; it cannot be triggered locally without external notifications, which are not allowed. No cron, --once, --now or webhook trigger exists in the product, so no manual cycle was run.", "expected": "identify the product trigger", "input": "source inspection", "kind": "cycle", "outcome": "Inspected README.md (Quick start, Try it, Phone approvals and kill switch) and src/cli.js, src/telegram.js, src/proxy.js. The only recurring loop is Telegram long-polling (src/telegram.js poll) that talks to an external Telegram Bot API; it cannot be triggered locally without external notifications, which are not allowed. No cron, --once, --now or webhook trigger exists in the product, so no manual cycle was run.", "source": "repository entry points", "verdict": "yes" } ], "details": { "argv": [ "node", "src/cli.js", "demo" ], "kind": "mcp", "scope": "product operation", "serverInfo": { "name": "tas-guard-demo-broker", "version": "1" }, "tool_result": { "content": [ { "text": "{\n \"demo\": true,\n \"symbol\": \"SPY\",\n \"bid\": 579.71,\n \"ask\": 580.29,\n \"note\": \"simulated\"\n}", "type": "text" } ] }, "tool_result_checked": true, "tool_schema_bytes": 2221, "tools": [ "get_account_info", "get_stock_latest_quote", "get_all_positions", "get_orders", "place_stock_order", "cancel_order_by_id", "cancel_all_orders", "close_all_positions", "tas_guard_status" ], "working": true }, "evidence_strength": "readme_checked", "example_checks": [ { "claim": "Solana mode demo runs seven agent transactions through the guard", "command": "node src/cli.js sol-demo", "deferred": false, "evidence": " so directly. Blocked by policy.\n\n5. Agent: Approve unlimited USDC spending for a contract\n \ud83d\udeab Blocked: Unlimited token delegation to 52oh\u20263Qx6. Blocked by policy.\n\n6. Agent: Send 150 USDC ($150)\n \ud83d\udeab Blocked: Value $150.00 (0.000005 SOL + 150 USDC (devnet)) is over the per-transaction limit of $100.00.\n\n7. Agent: Pay a 0.14 SOL priority fee\n \ud83d\udeab Blocked: Fee of 0.140005 SOL is over max_fee_sol (0.01 SOL). Priority fee: 100000000 micro-lamports \u00d7 1400000 units.\n\nToday: 2 sent, $45.00 of $300 used. Audit log: 8 entries, hash chain intact.\nUSDC still in the wallet: 500. The drainer got nothing.", "expected": "guard blocks drainer and over-limit transactions and reports hash chain intact", "input": "sol-demo", "kind": "example", "outcome": "command completed successfully", "returncode": 0, "source": "README.md Solana mode / Try it: `npx tas-guard sol-demo`", "verdict": "yes" }, { "claim": "Repository offline test suite (README Development section)", "command": "npm test", "deferred": false, "evidence": " log is hash-chained, redacts secrets and detects tampering\nok 76 - audit log is hash-chained, redacts secrets and detects tampering\n ---\n duration_ms: 10.221668\n type: 'test'\n ...\n# Subtest: approvals are single-use and expire\nok 77 - approvals are single-use and expire\n ---\n duration_ms: 16.924288\n type: 'test'\n ...\n# Subtest: config files with a Windows UTF-8 BOM are read\nok 78 - config files with a Windows UTF-8 BOM are read\n ---\n duration_ms: 5.288989\n type: 'test'\n ...\n1..78\n# tests 78\n# suites 0\n# pass 78\n# fail 0\n# cancelled 0\n# skipped 0\n# todo 0\n# duration_ms 26672.122397", "expected": "78 tests pass", "input": "npm test", "kind": "example", "outcome": "command completed successfully", "returncode": 0, "source": "README.md Development", "verdict": "yes" } ], "status": "PASS" }, "deviations": [ "`npx -y tas-guard demo` was not used: the repo is the pinned source, so the local bin target src/cli.js was run with node instead. `npm link` to install the tas-guard bin failed with EACCES on the global npm prefix (not writable); no sudo install was attempted.", "Replacing the README's sequence of 1 SPY allowed / 10 SPY blocked claims: the checker treats any MCP result with isError:true as a failure (check.py line ~1273), and the guard reports blocks with isError:true by design. The blocked-order behavior was observed manually (see steps) but is NOT validated by claims.json; the status claim (tas_guard_status) was used instead.", "First validation attempt failed for harness reasons, not product reasons: the checker applies each probe/claim env to os.environ globally, so all replays shared one TAS_GUARD_HOME state directory, and the guard's max_orders_per_minute=3 limit tripped when the probe and order claims ran within one minute. Fixed by using one fresh state dir (/tmp/tg-state-run2 for claims, /tmp/tg-state-run3 for the probe), making the probe a read-only quote, and waiting 75s before the final run. The policy was not changed.", "The demo's stdio server exits with code 1 when stdin closes (`upstream exited (null)`), which is the demo's normal end-of-session behavior; the checker accepted this.", "Daily state persists in /tmp/tg-state-run2 (3 orders, 1740 of 5000 spent). Repeat replays of the allowed-order claim add 580 each until the daily cap; the per-minute cap of 3 applies to order calls within a rolling minute.", "cycles: no product cycle detected. The only loop is Telegram long-polling against an external service, which cannot be exercised locally without external notifications." ], "fault_owner": null, "main_blocker": null, "operation": "Run TAS Guard demo (tas-guard demo) as an MCP stdio server in front of a built-in paper broker, and call its tools: read a simulated quote, place an allowed 1 SPY limit order, and read the guard status", "operation_scope": "Node.js >=18 (engines), tested on Node v22.23.4 / npm 10.9.9; zero npm dependencies; no broker account, no real money; local stdio only; CDP endpoint not used (no browser operation)", "outcome": "working", "plain_failure": null, "steps": [ "Read README.md (Try it in 10 seconds; Quick start; Development)", "node --version / npm --version (v22.23.4 / 10.9.9 present; README requires node >=18 per package.json engines)", "npm test (README Development: 78 tests) -> 78 pass, 0 fail", "node src/cli.js demo (stands in for `npx -y tas-guard demo`; see deviations) driven over MCP stdio", "tools/call get_stock_latest_quote {symbol: SPY} -> simulated bid/ask 579.71/580.29", "tools/call place_stock_order {SPY, buy, qty 1, limit 580} -> status accepted", "tools/call place_stock_order {SPY, buy, qty 10, limit 580} -> blocked: 'Order value 5,800 is over the per-order limit of 2,000' (isError true, observed manually)", "tools/call tas_guard_status {} -> max_order_value 2000, max_daily_value 5000, max_orders_per_minute 3", "node src/cli.js sol-demo -> exit 0, 'hash chain intact' (README Solana mode example)", "= python3 -I --validate --replay -> valid true, replay PASS" ], "stuck_step": null, "suggested_fix": "None needed for the product. For the checker: treat an MCP isError result that carries an expected blocked message as a valid verdict, and scope env/state per claim so policy counters do not leak between replays." }