{
  "checker": {
    "claims": [
      {
        "claim": "Ten MCP tools are provided read-only by default",
        "command": "/tmp/toadstone-venv/bin/python -m src.mcp tools/call {\"tool\": \"check_reference\", \"arguments\": {\"check\": \"missing_canonical\"}}",
        "evidence": "{\"content\": [{\"type\": \"text\", \"text\": \"{\\n  \\\"error\\\": \\\"Could not reach Toadstone at http://127.0.0.1:5057: ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=5057): Max retries exceeded with url: /api/checks (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x71114197edb0>: Failed to establish a new connection: [Errno 111] Connection refused'))\\\",\\n  \\\"status\\\": null,\\n  \\\"path\\\": \\\"/api/checks\\\",\\n  \\\"hint\\\": \\\"Toadstone is not reachable; check that it is running.\\\"\\n}\"}], \"isError\": true}",
        "expected": "read-only server lists check_reference and refuses writes",
        "input": "{\"tool\": \"check_reference\", \"arguments\": {\"check\": \"missing_canonical\"}}",
        "kind": "claim",
        "outcome": "Real input did not produce the expected result: missing_canonical",
        "source": "Ten tools, read-only by default, **unmetered \u2014",
        "source_checked": true,
        "source_file": "README.md",
        "verdict": "no"
      }
    ],
    "cycle_checks": [
      {
        "claim": "crawl cycle coverage fixture",
        "command": "/tmp/toadstone-venv/bin/python tests/coverage_site.py",
        "evidence": "crawling the fixture site at http://127.0.0.1:8971 ...\nStarting sitemap discovery for http://127.0.0.1:8971/\nInitializing JavaScript rendering...\nStarting Playwright browser...\nDiscovering sitemaps for http://127.0.0.1:8971...\nParsing sitemap: http://127.0.0.1:8971/sitemap.xml\nFound 13 URLs in sitemap\nParsing sitemap: http://127.0.0.1:8971/sitemap_index.xml\nParsing sitemap: http://127.0.0.1:8971/sitemaps.xml\nParsing sitemap: http://127.0.0.1:8971/sitemap/sitemap.xml\nParsing sitemap: http://127.0.0.1:8971/sitemap-index.xml\nFound sitemap index with 4 nested sitemaps\nParsing sitemap: http://127.0.0.1:8971/sitemap.xml\nFound 13 URLs in sitemap\nParsing sitemap: http://127.0.0.1:8971/sitemap-empty.xml\nParsing sitemap: http://127.0.0.1:8971/sitemap-nested.xml\nFound sitemap index with 1 nested sitemaps\nParsing sitemap: http://127.0.0.1:8971/sitemap.xml\nFound 13 URLs in sitemap\nParsing sitemap: https://127.0.0.1:8971/sitemap-https.xml\nFailed to initialize JavaScript rendering: BrowserType.launch: Executable doesn't exist at <guest-path>\n\u2554\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2557\n\u2551 Looks like Playwright was just installed or updated.       \u2551\n\u2551 Please run the following command to download new browsers: \u2551\n\u2551                                                            \u2551\n\u2551     playwright install                                     \u2551\n\u2551                                                            \u2551\n\u2551 <3 Playwright Team                                         \u2551\n\u255a\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u255d\nJavaScript rendering resources cleaned up\nUpdating linked_from data for all URLs...\nUpdated linked_from data for 0 URLs\nRunning duplication detection...\nDuplication detection complete. Total issues: 0\nError parsing sitemap https://127.0.0.1:8971/sitemap-https.xml: HTTPSConnectionPool(host='127.0.0.1', port=8971): Max retries exceeded with url: /sitemap-https.xml (Caused by SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1000)')))\nSitemap processing: 33 added, 6 filtered\nSitemap discovery completed. Total discovered URLs: 10\nSitemap audit: 12 URLs across 5 file(s), 11 issue(s)\nTechnical checks: 0 issue(s)\nHeader checks: 0 issue(s)\nLink graph: 0 pages, 0 issue(s)\nCMS: 2 item(s), 10 issue(s)\nRules: 1 rule(s), 0 issue(s)\nJavaScript rendering resources cleaned up\nCrawl completed. Discovered: 10, Crawled: 0\n\nWARNING: the crawl was still running after 60s. Everything below is partial.\nlog analysis: 4 issue(s) from 21 lines\nagent analysts (stubbed verdict): 0 issue(s)\ngoogle: not enough crawled pages to judge\nlink metrics: no crawled pages\n\npages crawled : 0\nchecks fired  : 22\n\n  bot_spoofing\n  cms_author_unexpected\n  cms_field_placeholder\n  cms_not_crawled\n  cms_published_not_in_sitemap\n  cms_seo_description_too_long\n  cms_seo_title_too_long\n  cms_stray_whitespace\n  crawl_budget_wasted\n  log_format_unrecognised\n  requested_but_not_crawled\n  sitemap_duplicate_urls\n  sitemap_file_error\n  sitemap_inconsistent_trailing_slash\n  sitemap_lastmod_coverage_low\n  sitemap_lastmod_future\n  sitemap_lastmod_malformed\n  sitemap_lastmod_missing\n  sitemap_mixed_hosts\n  sitemap_nested_index\n  sitemap_url_blocked_by_robots\n  sitemap_url_not_crawled\n\nFAIL: expected at least 160 distinct checks, got 22\nException in thread Thread-3 (_crawl_worker):\nTraceback (most recent call last):\n  File \"/usr/lib/python3.12/threading.py\", line 1073, in _bootstrap_inner\n    self.run()\n  File \"/usr/lib/python3.12/threading.py\", line 1010, in run\n    self._target(*self._args, **self._kwargs)\n  File \"<guest-path>\", line 435, in _crawl_worker\n    asyncio.run(self._crawl_async_with_js())\n  File \"/tmp/toadstone-venv/lib/python3.12/site-packages/nest_asyncio.py\", line 31, in run\n    return loop.run_until_complete(task)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/tmp/toadstone-venv/lib/python3.12/site-packages/nest_asyncio.py\", line 99, in run_until_complete\n    return f.result()\n           ^^^^^^^^^^\n  File \"/usr/lib/python3.12/asyncio/futures.py\", line 203, in result\n    raise self._exception.with_traceback(self._exception_tb)\n  File \"/usr/lib/python3.12/asyncio/tasks.py\", line 314, in __step_run_and_handle_result\n    result = coro.send(None)\n             ^^^^^^^^^^^^^^^\n  File \"<guest-path>\", line 562, in _crawl_async_with_js\n    await self.js_renderer.initialize()\n  File \"<guest-path>\", line 49, in initialize\n    self.browser = await self.playwright.chromium.launch(headless=headless, args=args)\n                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/tmp/toadstone-venv/lib/python3.12/site-packages/playwright/async_api/_generated.py\", line 4177, in launch\n    await self._impl_obj.launch(\n  File \"/tmp/toadstone-venv/lib/python3.12/site-packages/playwright/_impl/_browser_type.py\", line 101, in launch\n    await self._channel.send(\n  File \"/tmp/toadstone-venv/lib/python3.12/site-packages/playwright/_impl/_connection.py\", line 78, in send\n    return await self._connection.wrap_api_call(\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/tmp/toadstone-venv/lib/python3.12/site-packages/playwright/_impl/_connection.py\", line 636, in wrap_api_call\n    raise rewrite_error(error, f\"{parsed_st['apiName']}: {error}\") from None\nplaywright._impl._errors.Error: BrowserType.launch: Executable doesn't exist at <guest-path>\n\u2554\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2557\n\u2551 Looks like Playwright was just installed or updated.       \u2551\n\u2551 Please run the following command to download new browsers: \u2551\n\u2551                                                            \u2551\n\u2551     playwright install                                     \u2551\n\u2551                                                            \u2551\n\u2551 <3 Playwright Team                                         \u2551\n\u255a\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u255d; product invocation failed: {\"argv\": [\"/tmp/toadstone-venv/bin/python\", \"tests/coverage_site.py\"], \"returncode\": 1, \"stdout\": \"crawling the fixture site at http://127.0.0.1:8971 ...\\nStarting sitemap discovery for http://127.0.0.1:8971/\\nInitializing JavaScript rendering...\\nStarting Playwright browser...\\nDiscovering sitemaps for http://127.0.0.1:8971...\\nParsing sitemap: http://127.0.0.1:8971/sitemap.xml\\nFound 13 URLs in sitemap\\nParsing sitemap: http://127.0.0.1:8971/sitemap_index.xml\\nParsing sitemap: http://127.0.0.1:8971/sitemaps.xml\\nParsing sitemap: http://127.0.0.1:8971/sitemap/sitemap.xml\\nParsing sitemap: http://127.0.0.1:8971/sitemap-index.xml\\nFound sitemap index with 4 nested sitemaps\\nParsing sitemap: http://127.0.0.1:8971/sitemap.xml\\nFound 13 URLs in sitemap\\nParsing sitemap: http://127.0.0.1:8971/sitemap-empty.xml\\nParsing sitemap: http://127.0.0.1:8971/sitemap-nested.xml\\nFound sitemap index with 1 nested sitemaps\\nParsing sitemap: http://127.0.0.1:8971/sitemap.xml\\nFound 13 URLs in sitemap\\nParsing sitemap: https://127.0.0.1:8971/sitemap-https.xml\\nFailed to initialize JavaScript rendering: BrowserType.launch: Executable doesn't exist at <guest-path>\\n\\u2554\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2557\\n\\u2551 Looks like Playwright was just installed or updated.       \\u2551\\n\\u2551 Please run the following command to download new browsers: \\u2551\\n\\u2551                                                            \\u2551\\n\\u2551     playwright install                                     \\u2551\\n\\u2551                                                            \\u2551\\n\\u2551 <3 Playwright Team                                         \\u2551\\n\\u255a\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u255d\\nJavaScript rendering resources cleaned up\\nUpdating linked_from data for all URLs...\\nUpdated linked_from data for 0 URLs\\nRunning duplication detection...\\nDuplication detection complete. Total issues: 0\\nError parsing sitemap https://127.0.0.1:8971/sitemap-https.xml: HTTPSConnectionPool(host='127.0.0.1', port=8971): Max retries exceeded with url: /sitemap-https.xml (Caused by SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1000)')))\\nSitemap processing: 33 added, 6 filtered\\nSitemap discovery completed. Total discovered URLs: 10\\nSitemap audit: 12 URLs across 5 file(s), 11 issue(s)\\nTechnical checks: 0 issue(s)\\nHeader checks: 0 issue(s)\\nLink graph: 0 pages, 0 issue(s)\\nCMS: 2 item(s), 10 issue(s)\\nRules: 1 rule(s), 0 issue(s)\\nJavaScript rendering resources cleaned up\\nCrawl completed. Discovered: 10, Crawled: 0\\n\\nWARNING: the crawl was still running after 60s. Everything below is partial.\\nlog analysis: 4 issue(s) from 21 lines\\nagent analysts (stubbed verdict): 0 issue(s)\\ngoogle: not enough crawled pages to judge\\nlink metrics: no crawled pages\\n\\npages crawled : 0\\nchecks fired  : 22\\n\\n  bot_spoofing\\n  cms_author_unexpected\\n  cms_field_placeholder\\n  cms_not_crawled\\n  cms_published_not_in_sitemap\\n  cms_seo_description_too_long\\n  cms_seo_title_too_long\\n  cms_stray_whitespace\\n  crawl_budget_wasted\\n  log_format_unrecognised\\n  requested_but_not_crawled\\n  sitemap_duplicate_urls\\n  sitemap_file_error\\n  sitemap_inconsistent_trailing_slash\\n  sitemap_lastmod_coverage_low\\n  sitemap_lastmod_future\\n  sitemap_lastmod_malformed\\n  sitemap_lastmod_missing\\n  sitemap_mixed_hosts\\n  sitemap_nested_index\\n  sitemap_url_blocked_by_robots\\n  sitemap_url_not_crawled\\n\\nFAIL: expected at least 160 distinct checks, got 22\\n\", \"stderr\": \"Exception in thread Thread-3 (_crawl_worker):\\nTraceback (most recent call last):\\n  File \\\"/usr/lib/python3.12/threading.py\\\", line 1073, in _bootstrap_inner\\n    self.run()\\n  File \\\"/usr/lib/python3.12/threading.py\\\", line 1010, in run\\n    self._target(*self._args, **self._kwargs)\\n  File \\\"<guest-path>\\\", line 435, in _crawl_worker\\n    asyncio.run(self._crawl_async_with_js())\\n  File \\\"/tmp/toadstone-venv/lib/python3.12/site-packages/nest_asyncio.py\\\", line 31, in run\\n    return loop.run_until_complete(task)\\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\\n  File \\\"/tmp/toadstone-venv/lib/python3.12/site-packages/nest_asyncio.py\\\", line 99, in run_until_complete\\n    return f.result()\\n           ^^^^^^^^^^\\n  File \\\"/usr/lib/python3.12/asyncio/futures.py\\\", line 203, in result\\n    raise self._exception.with_traceback(self._exception_tb)\\n  File \\\"/usr/lib/python3.12/asyncio/tasks.py\\\", line 314, in __step_run_and_handle_result\\n    result = coro.send(None)\\n             ^^^^^^^^^^^^^^^\\n  File \\\"<guest-path>\\\", line 562, in _crawl_async_with_js\\n    await self.js_renderer.initialize()\\n  File \\\"<guest-path>\\\", line 49, in initialize\\n    self.browser = await self.playwright.chromium.launch(headless=headless, args=args)\\n                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\\n  File \\\"/tmp/toadstone-venv/lib/python3.12/site-packages/playwright/async_api/_generated.py\\\", line 4177, in launch\\n    await self._impl_obj.launch(\\n  File \\\"/tmp/toadstone-venv/lib/python3.12/site-packages/playwright/_impl/_browser_type.py\\\", line 101, in launch\\n    await self._channel.send(\\n  File \\\"/tmp/toadstone-venv/lib/python3.12/site-packages/playwright/_impl/_connection.py\\\", line 78, in send\\n    return await self._connection.wrap_api_call(\\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\\n  File \\\"/tmp/toadstone-venv/lib/python3.12/site-packages/playwright/_impl/_connection.py\\\", line 636, in wrap_api_call\\n    raise rewrite_error(error, f\\\"{parsed_st['apiName']}: {error}\\\") from None\\nplaywright._impl._errors.Error: BrowserType.launch: Executable doesn't exist at <guest-path>\\n\\u2554\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2557\\n\\u2551 Looks like Playwright was just installed or updated.       \\u2551\\n\\u2551 Please run the following command to download new browsers: \\u2551\\n\\u2551                                                            \\u2551\\n\\u2551     playwright install                                     \\u2551\\n\\u2551                                                            \\u2551\\n\\u2551 <3 Playwright Team                                         \\u2551\\n\\u255a\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u255d\\n\", \"needs_credentials\": false, \"needs_browser\": false}",
        "expected": "distinct checks recorded in TOADSTONE_SLUG_LOG",
        "input": "bundled coverage fixture",
        "kind": "cycle",
        "method": "run-once",
        "outcome": "Triggered one cycle by hand (run-once; normal trigger: run once when invoked): crawling the fixture site at http://127.0.0.1:8971 ...\nStarting sitemap discovery for http://127.0.0.1:8971/\nInitializing JavaScript rendering...\nStarting Playwright browser...\nDiscovering sitemaps for http://127.0.0.1:8971...\nParsing sitemap: http://127.0.0.1:8971/sitemap.xml\nFound 13 URLs in sitemap\nParsing sitemap: http://127.0.0.1:8971/sitemap_index.xml\nParsing sitemap: http://127.0.0.1:8971/sitemaps.xml\nParsing sitemap: http://127.0.0.1:8971/sitemap/sitemap.xml\nParsing sitemap: http://127.0.0.1:8971/sitemap-index.xml\nFound sitemap index with 4 nested sitemaps\nParsing sitemap: http://127.0.0.1:8971/sitemap.xml\nFound 13 URLs in sitemap\nParsing sitemap: http://127.0.0.1:8971/sitemap-empty.xml\nParsing sitemap: http://127.0.0.1:8971/sitemap-nested.xml\nFound sitemap index with 1 nested sitemaps\nParsing sitemap: http://127.0.0.1:8971/sitemap.xml\nFound 13 URLs in sitemap\nParsing sitemap: https://127.0.0.1:8971/sitemap-https.xml\nFailed to initialize JavaScript rendering: BrowserType.launch: Executable doesn't exist at <guest-path>\n\u2554\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2557\n\u2551 Looks like Playwright was just installed or updated.       \u2551\n\u2551 Please run the following command to download new browsers: \u2551\n\u2551                                                            \u2551\n\u2551     playwright install                                     \u2551\n\u2551                                                            \u2551\n\u2551 <3 Playwright Team                                         \u2551\n\u255a\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u255d\nJavaScript rendering resources cleaned up\nUpdating linked_from data for all URLs...\nUpdated linked_from data for 0 URLs\nRunning duplication detection...\nDuplication detection complete. Total issues: 0\nError parsing sitemap https://127.0.0.1:8971/sitemap-https.xml: HTTPSConnectionPool(host='127.0.0.1', port=8971): Max retries exceeded with url: /sitemap-https.xml (Caused by SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1000)')))\nSitemap processing: 33 added, 6 filtered\nSitemap discovery completed. Total discovered URLs: 10\nSitemap audit: 12 URLs across 5 file(s), 11 issue(s)\nTechnical checks: 0 issue(s)\nHeader checks: 0 issue(s)\nLink graph: 0 pages, 0 issue(s)\nCMS: 2 item(s), 10 issue(s)\nRules: 1 rule(s), 0 issue(s)\nJavaScript rendering resources cleaned up\nCrawl completed. Discovered: 10, Crawled: 0\n\nWARNING: the crawl was still running after 60s. Everything below is partial.\nlog analysis: 4 issue(s) from 21 lines\nagent analysts (stubbed verdict): 0 issue(s)\ngoogle: not enough crawled pages to judge\nlink metrics: no crawled pages\n\npages crawled : 0\nchecks fired  : 22\n\n  bot_spoofing\n  cms_author_unexpected\n  cms_field_placeholder\n  cms_not_crawled\n  cms_published_not_in_sitemap\n  cms_seo_description_too_long\n  cms_seo_title_too_long\n  cms_stray_whitespace\n  crawl_budget_wasted\n  log_format_unrecognised\n  requested_but_not_crawled\n  sitemap_duplicate_urls\n  sitemap_file_error\n  sitemap_inconsistent_trailing_slash\n  sitemap_lastmod_coverage_low\n  sitemap_lastmod_future\n  sitemap_lastmod_malformed\n  sitemap_lastmod_missing\n  sitemap_mixed_hosts\n  sitemap_nested_index\n  sitemap_url_blocked_by_robots\n  sitemap_url_not_crawled\n\nFAIL: expected at least 160 distinct checks, got 22\nException in thread Thread-3 (_crawl_worker):\nTraceback (most recent call last):\n  File \"/usr/lib/python3.12/threading.py\", line 1073, in _bootstrap_inner\n    self.run()\n  File \"/usr/lib/python3.12/threading.py\", line 1010, in run\n    self._target(*self._args, **self._kwargs)\n  File \"<guest-path>\", line 435, in _crawl_worker\n    asyncio.run(self._crawl_async_with_js())\n  File \"/tmp/toadstone-venv/lib/python3.12/site-packages/nest_asyncio.py\", line 31, in run\n    return loop.run_until_complete(task)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/tmp/toadstone-venv/lib/python3.12/site-packages/nest_asyncio.py\", line 99, in run_until_complete\n    return f.result()\n           ^^^^^^^^^^\n  File \"/usr/lib/python3.12/asyncio/futures.py\", line 203, in result\n    raise self._exception.with_traceback(self._exception_tb)\n  File \"/usr/lib/python3.12/asyncio/tasks.py\", line 314, in __step_run_and_handle_result\n    result = coro.send(None)\n             ^^^^^^^^^^^^^^^\n  File \"<guest-path>\", line 562, in _crawl_async_with_js\n    await self.js_renderer.initialize()\n  File \"<guest-path>\", line 49, in initialize\n    self.browser = await self.playwright.chromium.launch(headless=headless, args=args)\n                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/tmp/toadstone-venv/lib/python3.12/site-packages/playwright/async_api/_generated.py\", line 4177, in launch\n    await self._impl_obj.launch(\n  File \"/tmp/toadstone-venv/lib/python3.12/site-packages/playwright/_impl/_browser_type.py\", line 101, in launch\n    await self._channel.send(\n  File \"/tmp/toadstone-venv/lib/python3.12/site-packages/playwright/_impl/_connection.py\", line 78, in send\n    return await self._connection.wrap_api_call(\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/tmp/toadstone-venv/lib/python3.12/site-packages/playwright/_impl/_connection.py\", line 636, in wrap_api_call\n    raise rewrite_error(error, f\"{parsed_st['apiName']}: {error}\") from None\nplaywright._impl._errors.Error: BrowserType.launch: Executable doesn't exist at <guest-path>\n\u2554\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2557\n\u2551 Looks like Playwright was just installed or updated.       \u2551\n\u2551 Please run the following command to download new browsers: \u2551\n\u2551                                                            \u2551\n\u2551     playwright install                                     \u2551\n\u2551                                                            \u2551\n\u2551 <3 Playwright Team                                         \u2551\n\u255a\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u255d; product invocation failed: {\"argv\": [\"/tmp/toadstone-venv/bin/python\", \"tests/coverage_site.py\"], \"returncode\": 1, \"stdout\": \"crawling the fixture site at http://127.0.0.1:8971 ...\\nStarting sitemap discovery for http://127.0.0.1:8971/\\nInitializing JavaScript rendering...\\nStarting Playwright browser...\\nDiscovering sitemaps for http://127.0.0.1:8971...\\nParsing sitemap: http://127.0.0.1:8971/sitemap.xml\\nFound 13 URLs in sitemap\\nParsing sitemap: http://127.0.0.1:8971/sitemap_index.xml\\nParsing sitemap: http://127.0.0.1:8971/sitemaps.xml\\nParsing sitemap: http://127.0.0.1:8971/sitemap/sitemap.xml\\nParsing sitemap: http://127.0.0.1:8971/sitemap-index.xml\\nFound sitemap index with 4 nested sitemaps\\nParsing sitemap: http://127.0.0.1:8971/sitemap.xml\\nFound 13 URLs in sitemap\\nParsing sitemap: http://127.0.0.1:8971/sitemap-empty.xml\\nParsing sitemap: http://127.0.0.1:8971/sitemap-nested.xml\\nFound sitemap index with 1 nested sitemaps\\nParsing sitemap: http://127.0.0.1:8971/sitemap.xml\\nFound 13 URLs in sitemap\\nParsing sitemap: https://127.0.0.1:8971/sitemap-https.xml\\nFailed to initialize JavaScript rendering: BrowserType.launch: Executable doesn't exist at <guest-path>\\n\\u2554\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2557\\n\\u2551 Looks like Playwright was just installed or updated.       \\u2551\\n\\u2551 Please run the following command to download new browsers: \\u2551\\n\\u2551                                                            \\u2551\\n\\u2551     playwright install                                     \\u2551\\n\\u2551                                                            \\u2551\\n\\u2551 <3 Playwright Team                                         \\u2551\\n\\u255a\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u255d\\nJavaScript rendering resources cleaned up\\nUpdating linked_from data for all URLs...\\nUpdated linked_from data for 0 URLs\\nRunning duplication detection...\\nDuplication detection complete. Total issues: 0\\nError parsing sitemap https://127.0.0.1:8971/sitemap-https.xml: HTTPSConnectionPool(host='127.0.0.1', port=8971): Max retries exceeded with url: /sitemap-https.xml (Caused by SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1000)')))\\nSitemap processing: 33 added, 6 filtered\\nSitemap discovery completed. Total discovered URLs: 10\\nSitemap audit: 12 URLs across 5 file(s), 11 issue(s)\\nTechnical checks: 0 issue(s)\\nHeader checks: 0 issue(s)\\nLink graph: 0 pages, 0 issue(s)\\nCMS: 2 item(s), 10 issue(s)\\nRules: 1 rule(s), 0 issue(s)\\nJavaScript rendering resources cleaned up\\nCrawl completed. Discovered: 10, Crawled: 0\\n\\nWARNING: the crawl was still running after 60s. Everything below is partial.\\nlog analysis: 4 issue(s) from 21 lines\\nagent analysts (stubbed verdict): 0 issue(s)\\ngoogle: not enough crawled pages to judge\\nlink metrics: no crawled pages\\n\\npages crawled : 0\\nchecks fired  : 22\\n\\n  bot_spoofing\\n  cms_author_unexpected\\n  cms_field_placeholder\\n  cms_not_crawled\\n  cms_published_not_in_sitemap\\n  cms_seo_description_too_long\\n  cms_seo_title_too_long\\n  cms_stray_whitespace\\n  crawl_budget_wasted\\n  log_format_unrecognised\\n  requested_but_not_crawled\\n  sitemap_duplicate_urls\\n  sitemap_file_error\\n  sitemap_inconsistent_trailing_slash\\n  sitemap_lastmod_coverage_low\\n  sitemap_lastmod_future\\n  sitemap_lastmod_malformed\\n  sitemap_lastmod_missing\\n  sitemap_mixed_hosts\\n  sitemap_nested_index\\n  sitemap_url_blocked_by_robots\\n  sitemap_url_not_crawled\\n\\nFAIL: expected at least 160 distinct checks, got 22\\n\", \"stderr\": \"Exception in thread Thread-3 (_crawl_worker):\\nTraceback (most recent call last):\\n  File \\\"/usr/lib/python3.12/threading.py\\\", line 1073, in _bootstrap_inner\\n    self.run()\\n  File \\\"/usr/lib/python3.12/threading.py\\\", line 1010, in run\\n    self._target(*self._args, **self._kwargs)\\n  File \\\"<guest-path>\\\", line 435, in _crawl_worker\\n    asyncio.run(self._crawl_async_with_js())\\n  File \\\"/tmp/toadstone-venv/lib/python3.12/site-packages/nest_asyncio.py\\\", line 31, in run\\n    return loop.run_until_complete(task)\\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\\n  File \\\"/tmp/toadstone-venv/lib/python3.12/site-packages/nest_asyncio.py\\\", line 99, in run_until_complete\\n    return f.result()\\n           ^^^^^^^^^^\\n  File \\\"/usr/lib/python3.12/asyncio/futures.py\\\", line 203, in result\\n    raise self._exception.with_traceback(self._exception_tb)\\n  File \\\"/usr/lib/python3.12/asyncio/tasks.py\\\", line 314, in __step_run_and_handle_result\\n    result = coro.send(None)\\n             ^^^^^^^^^^^^^^^\\n  File \\\"<guest-path>\\\", line 562, in _crawl_async_with_js\\n    await self.js_renderer.initialize()\\n  File \\\"<guest-path>\\\", line 49, in initialize\\n    self.browser = await self.playwright.chromium.launch(headless=headless, args=args)\\n                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\\n  File \\\"/tmp/toadstone-venv/lib/python3.12/site-packages/playwright/async_api/_generated.py\\\", line 4177, in launch\\n    await self._impl_obj.launch(\\n  File \\\"/tmp/toadstone-venv/lib/python3.12/site-packages/playwright/_impl/_browser_type.py\\\", line 101, in launch\\n    await self._channel.send(\\n  File \\\"/tmp/toadstone-venv/lib/python3.12/site-packages/playwright/_impl/_connection.py\\\", line 78, in send\\n    return await self._connection.wrap_api_call(\\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\\n  File \\\"/tmp/toadstone-venv/lib/python3.12/site-packages/playwright/_impl/_connection.py\\\", line 636, in wrap_api_call\\n    raise rewrite_error(error, f\\\"{parsed_st['apiName']}: {error}\\\") from None\\nplaywright._impl._errors.Error: BrowserType.launch: Executable doesn't exist at <guest-path>\\n\\u2554\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2557\\n\\u2551 Looks like Playwright was just installed or updated.       \\u2551\\n\\u2551 Please run the following command to download new browsers: \\u2551\\n\\u2551                                                            \\u2551\\n\\u2551     playwright install                                     \\u2551\\n\\u2551                                                            \\u2551\\n\\u2551 <3 Playwright Team                                         \\u2551\\n\\u255a\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u2550\\u255d\\n\", \"needs_credentials\": false, \"needs_browser\": false}",
        "returncode": 1,
        "schedule": "run once when invoked",
        "source": "tests/coverage_site.py",
        "verdict": "no"
      }
    ],
    "details": {
      "argv": [
        "/tmp/toadstone-venv/bin/python",
        "-m",
        "src.mcp"
      ],
      "blocker": "MCP tool result did not confirm the expected product output",
      "kind": "mcp",
      "serverInfo": {
        "name": "toadstone",
        "version": "1.0.0"
      },
      "tool_result": {
        "content": [
          {
            "text": "{\n  \"error\": \"Could not reach Toadstone at http://127.0.0.1:5057: ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=5057): Max retries exceeded with url: /api/checks (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x71fec9b86810>: Failed to establish a new connection: [Errno 111] Connection refused'))\",\n  \"status\": null,\n  \"path\": \"/api/checks\",\n  \"hint\": \"Toadstone is not reachable; check that it is running.\"\n}",
            "type": "text"
          }
        ],
        "isError": true
      },
      "tool_schema_bytes": 6572,
      "tools": [
        "list_crawls",
        "crawl_status",
        "start_crawl",
        "list_issues",
        "issue_detail",
        "page_detail",
        "regression_diff",
        "export_dataset",
        "check_reference",
        "accept_issue"
      ],
      "working": false
    },
    "evidence_strength": "partly_checked",
    "example_checks": [
      {
        "claim": "Documented Python quick start can install requirements",
        "command": "/tmp/toadstone-venv/bin/pip install -r requirements.txt",
        "deferred": false,
        "evidence": "venv/lib/python3.12/site-packages (from cryptography->-r requirements.txt (line 15)) (2.1.1)\nRequirement already satisfied: pycparser in /tmp/toadstone-venv/lib/python3.12/site-packages (from cffi>=2.0.0->cryptography->-r requirements.txt (line 15)) (3.12)\nRequirement already satisfied: MarkupSafe>=2.0 in /tmp/toadstone-venv/lib/python3.12/site-packages (from Jinja2>=3.1.2->flask==2.3.3->-r requirements.txt (line 4)) (3.0.4)\nRequirement already satisfied: typing-extensions in /tmp/toadstone-venv/lib/python3.12/site-packages (from pyee<15,>=13->playwright->-r requirements.txt (line 7)) (4.16.0)",
        "expected": "requirements installed in supported Python 3.12 environment",
        "input": "pip install -r requirements.txt",
        "kind": "example",
        "outcome": "command completed successfully",
        "returncode": 0,
        "source": "README.md / Python, no container",
        "verdict": "yes"
      }
    ],
    "status": "FAIL"
  },
  "deviations": [
    "Attempt to install Chromium was rejected by environment policy; only shared CDP endpoint may be used. Actual failure: Playwright says executable does not exist at <guest-path>"
  ],
  "fault_owner": "prerequisite",
  "main_blocker": "The checker-required crawl cannot initialize its local Playwright browser; no local browser installation is allowed.",
  "operation": "MCP check_reference and list_issues against a crawl of the bundled fixture",
  "operation_scope": "Python 3.12.3 with requirements installed; Chromium required by documented crawl",
  "outcome": "blocked",
  "plain_failure": "Without Chromium the fixture crawl records zero crawled pages, so it cannot provide the nonempty issues needed to verify remediation and list_issues.",
  "steps": [
    "git fetch pinned commit 4841d0bb9faba944661217542f13710582c66bbc",
    "python3 -m venv /tmp/toadstone-venv",
    "/tmp/toadstone-venv/bin/pip install -r requirements.txt",
    "/tmp/toadstone-venv/bin/python main.py --local --port 5057",
    "Attempted README coverage fixture python tests/coverage_site.py and failed because Playwright Chromium executable is missing"
  ],
  "stuck_step": "crawl fixture with the documented coverage site",
  "suggested_fix": "Allow the crawler to connect to the supplied shared CDP endpoint, or provide the supported Chromium runtime in the guest."
}