{ "checker": { "claims": [ { "claim": "Explain one concept from the graph", "command": " explain graphify/build.py::build_from_json", "evidence": "Node: build_from_json()\n ID: graphify_build_build_from_json\n Source: graphify/build.py L1153\n Type: code\n Community: 11\n Degree: 294\n\nConnections (294):\n <-- test_languages.py [imports] [EXTRACTED] tests/test_languages.py:L1593\n <-- test_extract.py [imports] [EXTRACTED] tests/test_extract.py:L9\n <-- _rebuild_code() [calls] [EXTRACTED] graphify/watch.py:L2355\n --> networkx [calls] [EXTRACTED] graphify/build.py:L1319\n --> re [calls] [EXTRACTED] graphify/build.py:L1249\n <-- cli.py [imports] [EXTRACTED] graphify/cli.py:L2437\n <-- dispatch_command() [calls] [EXTRACTED]", "expected": "Node explanation with source location and connections", "input": "graphify/build.py::build_from_json", "kind": "claim", "outcome": "Real input produced the expected result: Node: build_from_json()", "returncode": 0, "source": "Ask a question, trace the path between two things, or explain one concept.", "source_checked": true, "source_file": "README.md", "verdict": "yes" }, { "claim": "Query returns a scoped subgraph for a plain-language question", "command": " query 'what connects extract to the graph?' --budget 800", "evidence": "NODE extract() [src=graphify/extract.py loc=L9219 community=12]\nNODE Graph [src= loc= community=85]\nNODE graph() [src=tests/test_cache_stale_import_target.py loc=L89 community=35]\nNODE test_cpp_cross_file_member_call_connects_with_relative_paths() [src=tests/test_cpp_objc_cross_file_calls.py loc=L45 community=306]\nNODE extract.py [src=graphify/extract.py loc=L1 community=2]\nNODE test_languages.py [src=tests/test_languages.py loc=L1 community=3]\nNODE test_extract.py [src=tests/test_extract.py loc=L1 community=4]\nNODE _rebuild_code() [src=graphify/watch.py loc=L1677 community=19]\nNODE _make_id()", "expected": "Subgraph listing the extract() node", "input": "what connects extract to the graph?", "kind": "claim", "outcome": "Real input produced the expected result: NODE extract()", "returncode": 0, "source": "returns a scoped subgraph for a plain-language question", "source_checked": true, "source_file": "README.md", "verdict": "yes" } ], "cycle_checks": [ { "claim": "Watch loop rebuilds the graph on code changes; one rebuild run by hand", "command": " update --no-cluster", "evidence": "Re-extracting code files in (no LLM needed)...\n[graphify watch] 13 file(s) not classified (no supported extension or shebang), skipped: .dockerignore, .gitattributes, .gitignore, .gitkeep, CODEOWNERS, Dockerfile (+7 more)\n AST extraction: 28/28 uncached files (100%) [2 workers]\n[graphify watch] Rebuilt (no clustering): 22253 nodes, 47412 edges\n[graphify watch] graph.json updated in \nCode graph updated. For doc/paper/image changes run /graphify --update in your AI assistant.\nTip: set GEMINI_API_KEY or GOOGLE_API_KEY to use Gemini for semantic extraction.\n warning: 7 .sql file(s) contributed nothing to the graph because a dependency is missing: tree_sitter_sql not installed. Install it with: pip install \"graphifyy[sql]\" (#1745)\n warning: 1 .dm file(s) contributed nothing to the graph because a dependency is missing: tree-sitter-dm not installed. Install it with: pip install \"graphifyy[dm]\" (#1745)\n warning: 1 .erl file(s) contributed nothing to the graph because a dependency is missing: tree-sitter-language-pack not installed. Install it with: pip install \"graphifyy[erlang]\" (#1745)\n warning: 1 .lisp file(s) contributed nothing to the graph because a dependency is missing: tree-sitter-commonlisp not installed. Install it with: pip install \"graphifyy[commonlisp]\" (#1745)\n warning: 1 .r file(s) contributed nothing to the graph because a dependency is missing: tree-sitter-language-pack not installed. Install it with: pip install \"graphifyy[r]\" (#1745)\n warning: 1 .resource file(s) contributed nothing to the graph because a dependency is missing: robotframework not installed. Install it with: pip install \"graphifyy[robot]\" (#1745)\n warning: 1 .robot file(s) contributed nothing to the graph because a dependency is missing: robotframework not installed. Install it with: pip install \"graphifyy[robot]\" (#1745)\n warning: 1 .sol file(s) contributed nothing to the graph because a dependency is missing: tree-sitter-solidity not installed. Install it with: pip install \"graphifyy[solidity]\" (#1745)\n warning: 1 .vb file(s) contributed nothing to the graph because a dependency is missing: tree-sitter-vb-dotnet not installed. Install it with: pip install \"graphifyy[vbnet]\" (#1745)\n warning: 1 file(s) had syntax errors and may be partially extracted: tests/fixtures/sample.luau (first error at line 8, 4 symbol(s) extracted)\n warning: 26 code file(s) yielded no symbols (0.03 MB) \u2014 a data file, or source this extractor models no symbols for: graphify/exporters/__init__.py, graphify/manifest.py, pyproject.toml, tests/fixtures/cpp_samedir/plain.h, tests/fixtures/crate_a/Cargo.toml (+21 more)", "expected": "one rebuild pass writes graph.json", "input": " --no-cluster", "kind": "cycle", "method": "run-once: graphify update on the same tree with no source edits (watch loop not started)", "outcome": "Triggered one cycle by hand (run-once: graphify update on the same tree with no source edits (watch loop not started); normal trigger: watch loop (debounced rebuild when files change)): Re-extracting code files in (no LLM needed)...\n[graphify watch] 13 file(s) not classified (no supported extension or shebang), skipped: .dockerignore, .gitattributes, .gitignore, .gitkeep, CODEOWNERS, Dockerfile (+7 more)\n AST extraction: 28/28 uncached files (100%) [2 workers]\n[graphify watch] Rebuilt (no clustering): 22253 nodes, 47412 edges\n[graphify watch] graph.json updated in \nCode graph updated. For doc/paper/image changes run /graphify --update in your AI assistant.\nTip: set GEMINI_API_KEY or GOOGLE_API_KEY to use Gemini for semantic extraction.\n warning: 7 .sql file(s) contributed nothing to the graph because a dependency is missing: tree_sitter_sql not installed. Install it with: pip install \"graphifyy[sql]\" (#1745)\n warning: 1 .dm file(s) contributed nothing to the graph because a dependency is missing: tree-sitter-dm not installed. Install it with: pip install \"graphifyy[dm]\" (#1745)\n warning: 1 .erl file(s) contributed nothing to the graph because a dependency is missing: tree-sitter-language-pack not installed. Install it with: pip install \"graphifyy[erlang]\" (#1745)\n warning: 1 .lisp file(s) contributed nothing to the graph because a dependency is missing: tree-sitter-commonlisp not installed. Install it with: pip install \"graphifyy[commonlisp]\" (#1745)\n warning: 1 .r file(s) contributed nothing to the graph because a dependency is missing: tree-sitter-language-pack not installed. Install it with: pip install \"graphifyy[r]\" (#1745)\n warning: 1 .resource file(s) contributed nothing to the graph because a dependency is missing: robotframework not installed. Install it with: pip install \"graphifyy[robot]\" (#1745)\n warning: 1 .robot file(s) contributed nothing to the graph because a dependency is missing: robotframework not installed. Install it with: pip install \"graphifyy[robot]\" (#1745)\n warning: 1 .sol file(s) contributed nothing to the graph because a dependency is missing: tree-sitter-solidity not installed. Install it with: pip install \"graphifyy[solidity]\" (#1745)\n warning: 1 .vb file(s) contributed nothing to the graph because a dependency is missing: tree-sitter-vb-dotnet not installed. Install it with: pip install \"graphifyy[vbnet]\" (#1745)\n warning: 1 file(s) had syntax errors and may be partially extracted: tests/fixtures/sample.luau (first error at line 8, 4 symbol(s) extracted)\n warning: 26 code file(s) yielded no symbols (0.03 MB) \u2014 a data file, or source this extractor models no symbols for: graphify/exporters/__init__.py, graphify/manifest.py, pyproject.toml, tests/fixtures/cpp_samedir/plain.h, tests/fixtures/crate_a/Cargo.toml (+21 more)", "returncode": 0, "schedule": "watch loop (debounced rebuild when files change)", "source": "graphify/watch.py", "verdict": "yes" } ], "details": { "argv": [ "", "explain", "graphify/build.py::build_from_json" ], "kind": "cli", "needs_browser": false, "needs_credentials": false, "returncode": 0, "scope": "product operation", "stderr": "", "stdout": "Node: build_from_json()\n ID: graphify_build_build_from_json\n Source: graphify/build.py L1153\n Type: code\n Community: 11\n Degree: 294\n\nConnections (294):\n <-- test_languages.py [imports] [EXTRACTED] tests/test_languages.py:L1593\n <-- test_extract.py [imports] [EXTRACTED] tests/test_extract.py:L9\n <-- _rebuild_code() [calls] [EXTRACTED] graphify/watch.py:L2355\n --> networkx [calls] [EXTRACTED] graphify/build.py:L1319\n[excerpt truncated]", "working": true }, "evidence_strength": "readme_checked", "example_checks": [ { "claim": "Offline code-only extraction builds the graph without an LLM", "command": " extract . --code-only", "deferred": false, "evidence": "ile(s) contributed nothing to the graph because a dependency is missing: robotframework not installed. Install it with: pip install \"graphifyy[robot]\" (#1745)\n warning: 1 .sol file(s) contributed nothing to the graph because a dependency is missing: tree-sitter-solidity not installed. Install it with: pip install \"graphifyy[solidity]\" (#1745)\n warning: 1 .vb file(s) contributed nothing to the graph because a dependency is missing: tree-sitter-vb-dotnet not installed. Install it with: pip install \"graphifyy[vbnet]\" (#1745)\n[graphify] Pruned 8 node(s) from 8 deleted or excluded source file(s).", "expected": "graph.json written with no LLM", "input": ". --code-only", "kind": "example", "outcome": "command completed successfully", "returncode": 0, "source": "README.md Privacy: graphify extract runs fully offline with --code-only", "verdict": "yes" }, { "claim": "Path between two nodes (README common commands; undirected because no directed path exists)", "command": " path graphify/build.py::build_from_json graphify/cli.py::dispatch_command --undirected", "deferred": false, "evidence": "Shortest path (1 hops):\n build_from_json() <--calls [EXTRACTED]-- dispatch_command()", "expected": "Shortest path of 1 hop", "input": "build_from_json to dispatch_command, undirected", "kind": "example", "outcome": "command completed successfully", "returncode": 0, "source": "README.md Common commands: graphify path", "verdict": "yes" }, { "claim": "Repository test suite (CONTRIBUTING.md) run with the documented command", "command": "", "deferred": false, "evidence": "Ran uv run --frozen pytest tests/ -q: 8240 passed, 213 skipped, 18 failed. 15 test_skillgen failures need git history blobs (not in depth-1 pinned fetch); 3 test_ext_tagged_id_clash failures need the optional sql extra (passes with --extra sql). Not replayed as an example.", "expected": "documented command completes successfully", "input": "", "kind": "example", "outcome": "Ran uv run --frozen pytest tests/ -q: 8240 passed, 213 skipped, 18 failed. 15 test_skillgen failures need git history blobs (not in depth-1 pinned fetch); 3 test_ext_tagged_id_clash failures need the optional sql extra (passes with --extra sql). Not replayed as an example.", "source": "CONTRIBUTING.md: uv run pytest tests/ -q", "verdict": "blocked" } ], "status": "PASS" }, "deviations": [ "uv was not preinstalled; installed it with the README's curl | sh installer.", "The workspace was not a git repository, so the source was pinned with a depth-1 fetch of the commit and git reset --mixed FETCH_HEAD.", "uv run without --frozen rewrote uv.lock (revision 3 to 5). Restored it with git checkout -- uv.lock and used uv run --frozen afterwards.", "Test suite: 18 failures. 15 are in tests/test_skillgen.py, which needs a historical git blob that the depth-1 fetch does not contain. 3 are in tests/test_ext_tagged_id_clash.py, which need the optional sql extra; they pass with --extra sql. Not fixed in the repo; recorded as prerequisites.", "graphify update without --code-only reported about 22k nodes, while extract --code-only reported 18698 nodes. This inconsistency was observed but not investigated." ], "fault_owner": null, "main_blocker": null, "operation": "Build an offline code knowledge graph of the repository with graphify extract --code-only, then query it with graphify explain and graphify path", "operation_scope": "Ubuntu guest, Python 3.12.3 (README requires 3.10+), uv 0.13.5 installed with the README curl installer, graphifyy installed via uv tool install. Offline code-only path with GRAPHIFY_BACKEND=none and GRAPHIFY_QUERY_LOG_DISABLE=1. No API keys, no browser, no credentials. Source pinned at c588b72d936927790a87ad009955078ed40562a3.", "outcome": "working", "plain_failure": null, "steps": [ "curl -LsSf https://astral.sh/uv/install.sh | sh (uv was missing)", "uv tool install graphifyy", "git init; git remote add origin https://github.com/Graphify-Labs/graphify; git fetch --depth 1 origin c588b72d936927790a87ad009955078ed40562a3; git reset --mixed FETCH_HEAD", "graphify extract . --code-only (GRAPHIFY_BACKEND=none)", "graphify cluster-only ", "graphify explain graphify/build.py::build_from_json", "graphify path --undirected between two code nodes", "graphify update --no-cluster (watch-style rebuild cycle)", "uv run --frozen pytest tests/ -q (documented in CONTRIBUTING.md)", "= python3 -I --validate --replay" ], "stuck_step": null, "suggested_fix": "For a fully green test run, use a full clone instead of depth 1 so the skillgen baseline blobs exist, and install the optional sql extra. Use --frozen (or pin uv) so uv run does not rewrite uv.lock. Investigate the node-count difference between update and extract --code-only." }