diff --git a/.changeset/tall-owls-check.md b/.changeset/tall-owls-check.md new file mode 100644 index 0000000000..226e89eea8 --- /dev/null +++ b/.changeset/tall-owls-check.md @@ -0,0 +1,6 @@ +--- +"@codegraphy-dev/core": major +"@codegraphy-dev/plugin-api": minor +--- + +Make `codegraphy search` discover exact live source and cached AST Symbols with deterministic natural-phrase File fallback results, add compact task-personalized `codegraphy map` File maps with typed Relationships, add `codegraphy query` overviews with prioritized declarations and Relationships, add safe workspace `settings get/set/unset` commands, reject corrupt persisted settings instead of silently replacing them with defaults, return structured data from `plugins list`, return compact partial results from Graph Scope mutations, keep successful non-verbose Indexing output on stdout only, query complete cached relationship types by default for exact targets, and resolve calls through explicit reexport Relationships to implementation Symbols. diff --git a/CONTEXT.md b/CONTEXT.md index 8e563f725a..bdad3d722a 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -18,7 +18,7 @@ CodeGraphy turns a folder into an interactive Relationship Graph so people and a | **Plugin Node** | A Node contributed by a plugin for a concept that Core does not own. | | **Relationship** | A meaningful connection between two Nodes. | | **Edge** | A semantic Relationship record with a source, target, and Edge Type. An interface decides how to render it. | -| **Edge Type** | The semantic category of an Edge, such as import, call, reference, inherit, contains, or nests. | +| **Edge Type** | The semantic category of an Edge, such as import, reexport, call, reference, inherit, contains, or nests. | | **Edge Direction** | The source-to-target direction of a Relationship. The source initiates the import, call, reference, containment, or other relation. | | **Dependency** | A Relationship whose Edge Type means one Node needs another to build, run, or resolve. Do not use dependency as a synonym for every Relationship. | | **Downstream** | Following Edge Direction away from a Node. The Edge Type explains what the direction means. | @@ -34,7 +34,7 @@ A **Graph Scope Capability Declaration** tells CodeGraphy which Node Types and E CodeGraphy narrows graph data in one order: ```text -Relationship Graph -> Scoped Graph -> Filtered Graph -> Searched Graph -> Visible Graph +Relationship Graph -> Scoped Graph -> Filtered Graph -> Graph View Search -> Searched Graph -> Visible Graph ``` | Stage | Meaning | @@ -43,13 +43,16 @@ Relationship Graph -> Scoped Graph -> Filtered Graph -> Searched Graph -> Visibl | **Scoped Graph** | The Relationship Graph after Graph Scope removes disabled types. | | **Filter** | Persisted include and exclude rules for recurring workspace noise. | | **Filtered Graph** | The Scoped Graph after Filter rules. | -| **Search** | A temporary text query that narrows the current graph without changing Filter settings. | -| **Searched Graph** | The Filtered Graph after Search. | +| **Graph View Search** | A temporary text query that narrows the current graph without changing Filter settings. | +| **Searched Graph** | The Filtered Graph after Graph View Search. | +| **CLI Search** | A bounded discovery query over exact live source, cached AST Symbols, and indexed Nodes. Sparse natural multi-term phrases add deterministic all-term File candidates. It reports source and cache provenance and does not change settings. | +| **Task Map** | A bounded task-personalized File map combining independent live terms, selected declarations, and cached typed Relationships with source-area diversity. | +| **Target Query** | A bounded overview of one exact File or Symbol Node, including prioritized declarations and incoming/outgoing Relationships. | | **Visible Graph** | The graph shown on screen after Graph Scope, Filter, Search, Show Orphans, and other view projection rules. | | **Orphan Node** | A Node with no remaining Edges after graph narrowing. | | **Show Orphans** | A final Graph View setting that keeps or removes Orphan Nodes. | -Graph Scope runs before Filter, Filter runs before Search, and sorting or pagination runs after those stages. Core Graph Query uses the same order. Show Orphans is a Graph View presentation setting rather than an Indexing or Graph Query input. +Graph Scope runs before Filter, Filter runs before Graph View Search, and sorting or pagination runs after those stages. Graph Query inventories use that shaped graph. CLI Search, Target Query, Path, and exact targeted Relationship selectors instead read the complete cached Node and Edge Types unless an invocation explicitly projects that dimension with `--node-type` or `--edge-type`; path Filters still apply. This keeps Graph View preferences from hiding indexed call or reexport evidence. Show Orphans is a Graph View presentation setting rather than an Indexing or Graph Query input. ## Selection, Focus, and Collapse @@ -92,7 +95,7 @@ Interaction rules: | **Refresh Graph** | Restart layout physics without processing source data. | | **Re-index Workspace** | Run Indexing, save the Graph Cache, and refresh the graph. | -Indexing runs File Discovery, Tree-sitter Analysis, Plugin Analysis, and Graph Projection. The Graph Cache stores unscoped analysis facts so Graph Scope can hide data without deleting it. Active Filters and Git ignored state exclude files from fresh analysis and the file budget; facts cached while those files were eligible remain reusable but stay out of the current graph. Expensive facts such as Symbol or plugin-owned tiers can load when their scope needs them and remain cached for reuse. +Indexing runs File Discovery, Tree-sitter Analysis, Plugin Analysis, and Graph Projection. JavaScript-family reexports are explicit Relationships; renamed exports are Alias Symbol Nodes, so calls can resolve through barrels to implementation Symbols across full and incremental Indexing. The Graph Cache stores unscoped analysis facts so Graph Scope can hide data without deleting it. Active Filters and Git ignored state exclude files from fresh analysis and the file budget; facts cached while those files were eligible remain reusable but stay out of the current graph. Expensive facts such as Symbol or plugin-owned tiers can load when their scope needs them and remain cached for reuse. The Graph View can use a whole-view loading state before its first graph payload. Graph Cache Sync, Live Update, plugin changes, and Re-index keep the current graph visible after that first render and use graph-local progress. @@ -105,12 +108,14 @@ The Graph View can use a whole-view loading state before its first graph payload | **tldraw Interface** | `@codegraphy-dev/tldraw` owns its launcher, tldraw document lifecycle, native shapes, controls, and adapters over Core and renderer physics. | | **Graph Renderer** | `@codegraphy-dev/graph-renderer` owns WebGPU drawing and deterministic WebAssembly physics. It does not own product settings, persistence, or plugins. | | **CodeGraphy CLI** | The terminal interface installed by `@codegraphy-dev/core`. It targets the current directory unless `-C, --workspace ` selects another workspace. | -| **Graph Query CLI** | `nodes`, `search`, `edges`, `dependencies`, `dependents`, and `path`, all with bounded JSON output. | -| **CodeGraphy Agent Skill** | Instructions that teach shell-capable agents when to index, which Graph Query command to choose, and when to inspect source. | +| **CodeGraphy Exploration CLI** | `search` combines exact evidence with deterministic all-term fallback ranking for natural phrases; `map` builds a compact task-personalized File map; `query` inspects one exact File or Symbol with prioritized declarations and Relationships. All return bounded JSON with provenance. | +| **CodeGraphy Settings CLI** | `settings`, `settings get`, `settings set`, and `settings unset` read or safely mutate supported workspace settings without silently repairing corrupt persisted input. | +| **Graph Query CLI** | `nodes`, `edges`, `dependencies`, `dependents`, and `path`, all with bounded JSON output over the shaped Relationship Graph. | +| **CodeGraphy Agent Skill** | Generalized instructions that explain the Relationship Graph, lifecycle, query surfaces, machine-readable output, freshness, shaping, and limits so a shell-capable agent can choose its own navigation strategy. | | **Core Plugin API** | `@codegraphy-dev/plugin-api` contracts for headless Core analysis and semantic graph extensions. | | **Extension Plugin API** | `@codegraphy-dev/extension-plugin-api` contracts for VS Code Extension and Graph View extensions. | -The CLI never searches parent directories for a workspace. Indexing and Graph Query are separate operations, so a query does not perform Indexing. Agents run Indexing when their task may have changed cached knowledge, then choose the narrowest query that answers the question. +The CLI never searches parent directories for a workspace. Indexing and exploration are separate operations, so `search` and `query` never perform Indexing. CLI Search reads source text live from eligible indexed File Nodes and labels cached Symbol provenance as fresh or stale. When Indexing reports a file-budget cap, agents can inspect or raise `maxFiles`, adjust durable filters, and explicitly re-index before querying. Persisted known settings fields are validated strictly; malformed input is reported and never replaced by defaults during mutation. The Agent Skill describes available evidence and its tradeoffs without prescribing a command sequence, call budget, search wording, or stopping rule. The agent chooses how CodeGraphy fits the task. Clean implementation benchmarks compare raw agents with CodeGraphy-plus-skill agents on the same fresh structural feature-change fixture, using three repetitions per condition and cumulative model tokens per correct task as the primary acceptance metric. A CLI candidate must also be selected by treatment agents before aggregate differences can be attributed to it. Personalized Task Map passed a direct adopted A/B after the required raw comparison; see ADR 0011, ADR 0012, and ADR 0013. ## Plugins diff --git a/README.md b/README.md index ec70167c88..d18c34167d 100644 --- a/README.md +++ b/README.md @@ -76,8 +76,11 @@ The terminal CLI supports Node.js 20 through 22. Node 22 LTS is recommended. ```bash npm install -g @codegraphy-dev/core +codegraphy settings get maxFiles +codegraphy settings set maxFiles 2500 codegraphy index codegraphy search SettingsPanel +codegraphy query packages/extension/src/webview/app/shell/view.tsx codegraphy dependencies packages/extension/src/webview/app/shell/view.tsx ``` @@ -123,11 +126,14 @@ All `codegraphy ...` commands are published by `@codegraphy-dev/core`. Data comm | Command | Result | |---|---| -| `codegraphy status` | Reports fresh, stale, missing, or unusable Graph Cache state. | +| `codegraphy status` | Reports fresh, stale, or missing Graph Cache state. | | `codegraphy doctor` | Checks runtime, settings, Graph Cache schema, integrity, foreign keys, counts, and plugin state. | -| `codegraphy index` | Makes the selected workspace Graph Cache current. | +| `codegraphy index` | Makes the Graph Cache current and reports actionable file-budget truncation. | +| `codegraphy settings [get|set|unset]` | Safely reads or changes validated workspace settings such as `maxFiles`. | | `codegraphy nodes` | Lists bounded Nodes from saved Graph Scope. | -| `codegraphy search ` | Searches Nodes. | +| `codegraphy search ` | Finds exact evidence and uses deterministic all-term File ranking for sparse natural multi-term phrases. | +| `codegraphy map ` | Builds a compact task-personalized File map with declarations and typed connecting Relationships. | +| `codegraphy query ` | Inspects one exact File or Symbol with prioritized declarations and incoming/outgoing Relationships. | | `codegraphy edges` | Lists bounded Edges. | | `codegraphy dependencies ` | Lists outgoing Relationships for a file or exact Symbol Node. | | `codegraphy dependents ` | Lists incoming Relationships for a file or exact Symbol Node. | @@ -136,11 +142,11 @@ All `codegraphy ...` commands are published by `@codegraphy-dev/core`. Data comm | `codegraphy filter` | Reads or changes persisted filter patterns. | | `codegraphy plugins` | Registers, links, lists, enables, or disables plugins. | -Run `codegraphy --help` for exact arguments. Query, settings, Indexing, and diagnostic commands keep machine-readable JSON on stdout. Verbose diagnostics go to stderr. +Target Query, Path, and exact targeted Relationship selectors use complete cached Node and Edge Types by default rather than saved Graph View Scope; explicit `--node-type` or `--edge-type` arguments still constrain that dimension. JavaScript-family `reexport` Relationships let calls through barrels terminate at implementation Symbols. Run `codegraphy --help` for exact arguments. Query, settings, Indexing, and diagnostic commands keep machine-readable JSON on stdout. Verbose diagnostics go to stderr. ### Agent Skill -The [CodeGraphy Agent Skill](./skills/codegraphy/SKILL.md) teaches shell-capable agents to keep the index current and choose a bounded Graph Query before reading source. Install it from a clone of this repo: +The [CodeGraphy Agent Skill](./skills/codegraphy/SKILL.md) explains the Relationship Graph, lifecycle, query surfaces, JSON output, freshness, shaping, and limits so shell-capable agents can choose their own navigation strategy. Clean coding benchmarks compare fresh raw agents against CodeGraphy-plus-skill agents on the same feature change and use cumulative model tokens per correct task as the primary acceptance metric; see [ADR 0011](./docs/adr/0011-agents-choose-their-codegraphy-strategy.md). Install the skill from a clone of this repo: ```bash npx skills@latest add ./skills/codegraphy diff --git a/docs/adr/0003-cli-and-agent-skill-replace-mcp.md b/docs/adr/0003-cli-and-agent-skill-replace-mcp.md index a00e58b9db..b5a6f41207 100644 --- a/docs/adr/0003-cli-and-agent-skill-replace-mcp.md +++ b/docs/adr/0003-cli-and-agent-skill-replace-mcp.md @@ -1,6 +1,6 @@ # CLI and Agent Skill replace MCP -**Status:** Accepted +**Status:** Accepted; exploration vocabulary amended by ADR 0007 CodeGraphy's agent interface will use the Core-owned `codegraphy` CLI and a distributable Agent Skill. Delete the `@codegraphy-dev/mcp` package and MCP-specific product surface. @@ -14,7 +14,7 @@ CodeGraphy's agent interface will use the Core-owned `codegraphy` CLI and a dist Add a small positional JSON Graph Query surface to the Core CLI. Expose the extension's persisted Graph Scope, Filter, and plugin controls through the same workspace settings. Make `codegraphy index` reuse and patch compatible persisted cache data. Publish a general `codegraphy` Agent Skill and delete MCP. -The public Graph Query vocabulary is `nodes`, `search`, `edges`, `dependencies`, `dependents`, and `path`. Symbols remain Node Types, and Relationships remain Edges. `nodes`, `search`, `edges`, `dependencies`, and `dependents` use a default limit of 100 and accept `--limit` plus `--offset` for bounded continuation. `path` uses fixed depth and path-count bounds instead of pagination. Commands target the current directory unless the global `-C, --workspace ` option selects another CodeGraphy Workspace. +The original public Graph Query vocabulary was `nodes`, `search`, `edges`, `dependencies`, `dependents`, and `path`. ADR 0007 later deepens `search`, adds executable Target Query, and leaves `nodes`, `edges`, `dependencies`, `dependents`, and `path` as graph navigation commands. Symbols remain Node Types, and Relationships remain Edges. `nodes`, `search`, `edges`, `dependencies`, and `dependents` use a default limit of 100 and accept `--limit` plus `--offset` for bounded continuation. `path` uses fixed depth and path-count bounds instead of pagination. Commands target the current directory unless the global `-C, --workspace ` option selects another CodeGraphy Workspace. Every query accepts repeatable, comma-separated `--filter`, `--node-type`, and `--edge-type` options. These are one-off projections layered over persisted workspace settings for that invocation. A child `--node-type` enables the parent Node Types needed to evaluate Graph Scope. Node Type matching follows the Graph Scope hierarchy and symbol meaning: a parent includes matching descendants, and overlapping types can match the same symbol. Each result keeps its most specific stored `nodeType`. These options do not write `.codegraphy/settings.json`; durable changes still use `codegraphy filter` and `codegraphy scope`. diff --git a/docs/adr/0006-batch-queries-share-one-graph-snapshot.md b/docs/adr/0006-batch-queries-share-one-graph-snapshot.md new file mode 100644 index 0000000000..a20a40743d --- /dev/null +++ b/docs/adr/0006-batch-queries-share-one-graph-snapshot.md @@ -0,0 +1,41 @@ +# Batch queries share one Graph Cache snapshot + +**Status:** Superseded by ADR 0007 + +## Context + +Coding agents frequently need several independent Graph Query results at once, such as outgoing and incoming Relationships for a known File Node. Running each existing CLI command separately repeats Node process startup, workspace status checks, settings reads, and Graph Cache hydration. + +A two-query benchmark on the TypeScript example workspace measured separate `dependencies` and `dependents` processes at 293.5 ms median and 299.9 ms p95. A batch against one prepared graph snapshot measured 150.9 ms median and 155.9 ms p95. Compact output grew from 683 bytes to 741 bytes, so batching improves latency and round trips rather than response size. + +## Decision + +Add `codegraphy batch` as a transport over the six existing Graph Query commands. It does not add a Graph Query report or change the public Graph Query vocabulary accepted by ADR 0003. + +`batch` reads one bounded JSON document from standard input: + +```json +{ + "queries": [ + { "id": "uses", "argv": ["dependencies", "src/app.ts"] }, + { "id": "used-by", "argv": ["dependents", "src/app.ts"] } + ] +} +``` + +Each `argv` uses the existing parser for `nodes`, `search`, `edges`, `dependencies`, `dependents`, or `path`. Existing defaults, bounds, one-off Graph Scope and Filter projections, and selector safety therefore stay on one forward path. + +The complete input is validated before Graph Query execution. Query IDs are unique strings. A batch contains 1 through 100 queries and at most 1 MiB of UTF-8 JSON. The public Core batch request enforces the same query-count bound; the byte bound belongs to the stdin transport. The command resolves one CodeGraphy Workspace, reads one Graph Cache snapshot, then applies each query's projection against that snapshot. + +Success results preserve input order and include their caller-provided IDs. Failure is all-or-nothing: the CLI returns a failed outer envelope, discards sibling results, and identifies the failed query ID, command, code, and message in `error.details`. + +Batch remains read-only. It never performs Indexing, mutates settings, executes non-query commands, refers to an earlier result, or searches parent directories for a workspace. Agents continue to run explicit Indexing when cached knowledge may be outdated. + +## Consequences + +- Agents can group independent, already-known Graph Queries to reduce process startup and Graph Cache reads. +- Dependent exploration still uses separate calls: a batch is not a pipeline or query language. +- Single queries keep their existing commands and result shapes. +- Batch input costs some additional bytes and its output can be slightly larger than separate envelopes. Agents should use it for latency and round-trip reduction, not assume token savings. +- Core exposes a batch workspace request that prepares query source data once and projects it independently for every item. +- A future compound `inspect` operation remains a separate semantic decision. Batch does not justify adding hidden traversal or inference behavior. diff --git a/docs/adr/0007-search-and-target-query-replace-batch.md b/docs/adr/0007-search-and-target-query-replace-batch.md new file mode 100644 index 0000000000..e78b8d9f3a --- /dev/null +++ b/docs/adr/0007-search-and-target-query-replace-batch.md @@ -0,0 +1,38 @@ +# Search and Target Query replace Batch + +**Status:** Accepted + +## Context + +ADR 0006 accepted `batch` from a process-only benchmark: two predetermined Graph Query commands shared one Graph Cache snapshot and reduced median process latency from 293.5 ms to 150.9 ms. That measurement did not test an agent completing an adaptive coding task. + +A controlled follow-up ran 18 fresh agents across an easy source-location task and a harder settings-diagnosis task. Each of three navigation conditions ran three times against identical prepared snapshots. All answers scored 10/10. Median adaptive CodeGraphy results were worse than ordinary navigation: 49.3 seconds, 12 calls, and 48,319 tokens versus 28.2 seconds, 8 calls, and 20,433 tokens on the easy task; 188.4 seconds, 44 calls, and 429,216 tokens versus 127.3 seconds, 37 calls, and 208,120 tokens on the hard task. + +Batch reduced CodeGraphy round trips but still did not beat ordinary-navigation elapsed time. Trace review showed the larger problem: `search` only narrowed graph Node metadata, exact source phrases returned nothing, the apparent `query` help group was not an executable command, and agents manually composed many low-level traversals before reading source. + +The Graph Cache already stores AST Symbol facts with File ownership. Current source text remains authoritative and need not be duplicated in SQLite merely to support discovery. + +## Decision + +Remove the public `batch` CLI command and exported Core Batch request. Keep one adaptive forward path. + +Make `search ` a workspace discovery operation. It merges, ranks, and globally paginates: + +- live source-line matches from eligible indexed File Nodes; +- cached AST Symbol matches with their File provenance; and +- indexed non-Symbol Node matches. + +Literal matching is case-insensitive. `*` is a line-local or name-local wildcard. Symbol matches include cached declaration metadata and `filePath`; text matches include one-based line and column plus a bounded excerpt. Search reads source text live and compares it with indexed content hashes so its result states whether cached Symbol facts are fresh or stale. Persisted and one-off path Filters apply. Graph Scope does not hide discovery facts. + +Make `query ` executable. It resolves one exact workspace-relative File path or Symbol Node ID and returns a fixed-bounded overview from one prepared Graph Cache snapshot: the target, declared AST Symbols, outgoing Relationships, and incoming Relationships. It never infers a natural-language answer, performs Indexing, or accepts a pipeline. Agents use `search` to discover an exact selector, `query` for its first semantic overview, and `dependencies`, `dependents`, or `path` only for continuation. + +Root help and the CodeGraphy Agent Skill teach that sequence and explain every navigation command. They do not require a preparatory `status` call. A missing-cache result triggers explicit Indexing; source inspection remains the final evidence step. + +## Consequences + +- Exact source phrases and wildcard patterns can locate production and test evidence without an ordinary text-search fallback. +- AST Symbols are discoverable from cached facts even when Graph Scope hides Symbol Nodes from graph presentation. +- `query` replaces the confusing pseudo-group with one deep target-overview interface. +- Live text and cached graph facts can differ; the result exposes provenance and freshness instead of silently reindexing. +- Source scanning adds bounded filesystem work to `search`. It reads only eligible indexed File Nodes, skips unreadable, binary, and files larger than 1 MiB, and does not affect graph-only commands. +- The accepted success gate is agent task completion—correctness, elapsed time, tool calls, and total model tokens—not isolated process latency. diff --git a/docs/adr/0008-agent-search-and-settings-workflow.md b/docs/adr/0008-agent-search-and-settings-workflow.md new file mode 100644 index 0000000000..735a5372ee --- /dev/null +++ b/docs/adr/0008-agent-search-and-settings-workflow.md @@ -0,0 +1,79 @@ +# Deterministic phrase search and safe agent settings + +**Status:** Accepted + +## Context + +A controlled hard-task run took 151.1 seconds although the prepared Graph Cache was fresh and covered all 903 eligible files. The run used 26 model turns, 34 tools, and 307,302 processed tokens. It exhausted its four CodeGraphy calls on Search, never used Target Query, followed unrelated `FilterOptions` and `includePatterns` results, and then reread known source through many sequential reads and greps. The CLI processes themselves accounted for only about 4–5 seconds. + +The specific retrieval failure was `search "filter command"`. Literal phrase matching returned one parse-test line instead of `src/cli/filter/command.ts`. + +BM25 was considered as a natural-language fallback. The Robertson BM25 literature defines a term-frequency and inverse-document-frequency rank with document-length normalization. Lucene's `BM25Similarity` uses the common defaults `k1 = 1.2` and `b = 0.75`; SQLite FTS5 exposes the same constants and supports column weighting. + +A 902-document prototype compared literal matching, deterministic all-term ranking, and BM25 over path-boosted source documents. Both alternatives ranked `src/cli/filter/command.ts` first for `filter command`; BM25 handled some looser multi-term queries better, while deterministic ranking preserved stronger path precision. + +A forced six-agent A/B test then required the same first command, `codegraphy search "filter command"`, so every treatment exercised the fallback. BM25 returned the target File first, but its broader ten-File candidate list did not improve task completion: + +| Hard-task median | Literal | BM25 | BM25 delta | +|---|---:|---:|---:| +| Elapsed | 118.7 s | 135.6 s | 14.2% slower | +| Tool calls | 27 | 32 | 18.5% more | +| Total tokens | 176,147 | 183,241 | 4.0% more | +| Tool output | 115,837 B | 87,835 B | 24.2% fewer | + +The direct retrieval improved, but the larger ranked candidate set encouraged more continuation and fallback exploration. An additional unforced A/B did not reliably invoke a natural phrase and was therefore treated as stochastic evidence, not a BM25 comparison. + +The same traces raised a second hypothesis: embedding bounded live source in Target Query might remove a source-read turn. Two three-pair follow-ups rejected it. In v6, source-context Query was 3.4% slower than the prior CodeGraphy interface and used 28.1% more tokens. Stronger no-reread guidance in v7 reduced median elapsed time and calls, but tokens remained 61.2% higher and agents still duplicated or serially followed the context. Source embedding shifted work into larger accumulated model contexts rather than reliably removing it. + +Agents also could not safely raise `maxFiles` or change another top-level workspace setting through one structured interface. Existing persisted-settings readers silently replaced malformed JSON and malformed known fields with defaults, so a mutation could overwrite corruption. + +The selected graph-focused Query, deterministic phrase fallback, and bounded-skill workflow were then compared with ordinary navigation in three fresh paired v10 runs. Every answer contained the complete diagnosis and test surface. Final medians improved all four optimization metrics: + +| Hard-task median | Ordinary navigation | Selected CodeGraphy | Improvement | +|---|---:|---:|---:| +| Elapsed | 131.7 s | 115.0 s | 12.7% faster | +| Tool calls | 32 | 26 | 18.8% fewer | +| Total tokens | 188,319 | 106,418 | 43.5% fewer | +| Tool output | 155,243 B | 100,455 B | 35.3% fewer | + +The treatment won elapsed time in two of three pairs; the sample remains intentionally small. Separate v8/v9 comparisons against the prior CodeGraphy interface confirmed lower elapsed time and tokens but showed call-count variance, so no universal superiority claim is made. + +## Decision + +Do not ship BM25 as the default or add a public ranking flag. + +Keep exact case-insensitive literal, wildcard, AST Symbol, Node, and source-line matches as the primary Search path. When a whitespace-separated natural phrase has fewer than five direct matches: + +- tokenize camelCase, paths, and source text; +- remove a small fixed stop-word set; +- require every query term; +- prefer documents whose path contains every term; +- return at most three deterministic File candidates before lower-ranked direct source evidence. + +Keep Target Query focused on graph evidence. It returns prioritized declarations and incoming/outgoing Relationships; callable and type declarations rank ahead of local constants. Do not embed source text. Once target and test Files are known, the Agent Skill directs one parallel source-read turn instead of serial reads or repeated graph searches. + +Add one workspace-settings interface: + +- `settings` reads all effective settings; +- `settings get ` reads one effective value; +- `settings set ` validates and persists one supported top-level setting; +- `settings unset ` restores its default. + +Settings mutations preserve unknown top-level fields, validate the complete existing file before writing, report `indexRequired`, and never replace malformed persisted input with defaults. Persisted known fields use strict validation; the standalone normalization function remains tolerant for programmatic inputs. + +Indexing remains explicit and separate from settings and querying. Its structured `discovery` result reports `indexedFiles`, `totalFound`, and `limitReached`; a capped result includes the exact `settings set maxFiles` recovery command. An agent raises the file budget or adjusts durable filters only after Indexing reports truncation, then runs `index` and resumes Search or Target Query. + +## Consequences + +- Natural `filter command` discovery returns one high-confidence File candidate instead of a broad lexical list. +- Exact identifier behavior remains deterministic and unchanged. +- Query output stays graph-focused and bounded; exact source remains a deliberate read after all relevant Files are known. +- Configuration corruption becomes visible to Filter, Settings, Doctor, and other persisted-settings callers, and corrupt bytes are preserved. +- BM25 remains a rejected experiment for this corpus and task distribution. It may be reconsidered for substantially larger corpora only with end-to-end agent evidence, not retrieval scores alone. +- SQLite FTS5 is not added. Search already reads current source text, and the in-memory deterministic fallback adds negligible work without introducing another persisted source index or tokenizer contract. + +## References + +- Stephen E. Robertson et al., “Okapi at TREC-3” / BM25 foundations: https://www.staff.city.ac.uk/~sbrp622/papers/foundations_bm25_review.pdf +- Apache Lucene `BM25Similarity`: https://lucene.apache.org/core/9_12_1/core/org/apache/lucene/search/similarities/BM25Similarity.html +- SQLite FTS5 `bm25()` ranking: https://www.sqlite.org/fts5.html#the_bm25_function diff --git a/docs/adr/0009-targeted-queries-use-complete-reexport-relationships.md b/docs/adr/0009-targeted-queries-use-complete-reexport-relationships.md new file mode 100644 index 0000000000..77c675084b --- /dev/null +++ b/docs/adr/0009-targeted-queries-use-complete-reexport-relationships.md @@ -0,0 +1,74 @@ +# Targeted queries use complete reexport relationships + +**Status:** Accepted + +## Context + +The saved Graph View Scope enables File Nodes and structural import-like Edges by default while hiding detailed Symbol and call facts. That presentation preference leaked into CLI traversal: an exact Symbol `query` could expose cached calls, but `dependencies`, `dependents`, and `path` could return no result for the same selectors. General call reports also projected unresolved JavaScript-family calls to Files when an imported binding passed through a barrel, even though the Graph Cache contained the implementation Symbol. + +Graphify was evaluated directly on the 912-file Core fixture. Code-only extraction took 5.53 seconds and breadth-first queries took 0.39–0.46 seconds, but phrase traversals returned broad truncated neighborhoods. Exact `runFilterCommand` traversal and `explain` were materially better: they exposed the direct extracted call to `readCodeGraphyWorkspaceSettingsOrInitial` with direction and provenance. This supports exact-anchor relationship deepening rather than broad natural-language traversal. + +A bounded connected-search prototype expanded one exact lexical anchor through typed outgoing relationships to depth two, retained at most one incoming Edge, and suppressed expansion through p99-degree hubs. Three controlled variants were mixed: + +- the first treatment slightly improved median elapsed time and tokens but increased calls and output; +- refinement plus guidance reduced calls and output but was 1.3% slower and used 2.2% more tokens; +- the final exact-anchor treatment was 2.8% faster, but median calls increased from 23 to 31, tokens from 98,117 to 126,766, and output from 93,765 to 105,773 bytes. + +Agents still read source after receiving the neighborhoods. Generic connection augmentation therefore did not earn its added response surface. Leiden communities, personalized PageRank, and god-node ranking remain research candidates rather than default query behavior. + +The narrower hypothesis was to make existing exact operations truthful. JavaScript-family export statements previously emitted ordinary `import` Relationships. Workspace enrichment could use transient binding metadata to resolve a call through a barrel during a full index, but the normalized Graph Cache intentionally does not persist analyzer metadata. A later incremental CLI process therefore fell back to the barrel File. + +A three-pair hard-task comparison tested complete targeted scope plus direct call resolution through reexports against the prior graph-focused CLI. The first implementation produced these medians: + +| Hard-task median | Prior graph | Complete reexport graph | Delta | +|---|---:|---:|---:| +| Elapsed | 120.2 s | 107.9 s | 10.3% faster | +| Tool calls | 21 | 24 | 14.3% more | +| Total tokens | 125,606 | 121,129 | 3.6% fewer | +| Tool output | 110,361 B | 101,930 B | 7.6% fewer | + +Every run used four CodeGraphy calls. The treatment improved elapsed time, tokens, and output while increasing tool calls. A final paired run after making reexports explicit is recorded below. + +The final selected interface and stable skill were then rerun in three fresh pairs: + +| Hard-task median | Prior graph | Complete reexport graph | Delta | +|---|---:|---:|---:| +| Elapsed | 103.7 s | 106.9 s | 3.1% slower | +| Tool calls | 27 | 23 | 14.8% fewer | +| Total tokens | 128,784 | 173,530 | 34.7% more | +| Tool output | 107,563 B | 103,294 B | 4.0% fewer | +| Blinded correctness | 10/10 | 9/10 | one point lower | + +The treatment won elapsed time in two of three matched pairs but lost on the independent cell medians. A separate final round likewise reduced output by 4.7% while elapsed, calls, and tokens varied in the other direction. Across the two final rounds, smaller tool output was the only stable aggregate improvement. The explicit relationship model is retained for structural correctness, incremental parity, 14.8% fewer calls in the selected round, and consistently smaller output—not as a claim that it universally speeds this task. + +A follow-up skill sentence that explicitly preferred callable Symbol targets made all treatment agents choose `runFilterCommand` directly and reduced that treatment's cross-round median calls from 24 to 21. Against its paired prior-graph condition, however, it was 19.7% slower with 2.9% more tokens and 4.1% more output, so the sentence was removed. + +The samples are intentionally small and retain the variance seen in earlier agent experiments. Exact reexport relationships are an evidence-backed graph correction with measured tradeoffs, not universal agent superiority. + +## Decision + +Use the complete cached Node and Edge Types for CLI Search, Target Query, Path, and exact targeted Relationship selectors. Saved Graph View Scope continues to shape inventory and presentation. An explicit `--node-type` or `--edge-type` projection constrains only that dimension for the current invocation. + +When File selectors expand to their Symbols for Path, try exact File endpoints first and stop after finding that exact route. Do not emit redundant longer Symbol-expanded routes after an exact File path succeeds. + +Add `reexport` to the Core Relationship vocabulary and Plugin API. JavaScript-family export statements emit `reexport` rather than overloading `import`. Renamed exports create Alias Symbol Nodes, with the reexport originating at the alias. Reexport Edges remain hidden in the default Graph View Scope but are available to complete targeted queries and explicit projections. + +Resolve call, event, inherit, and reference targets through named, aliased, and export-star reexport chains. Keep lexical import Relationships pointed at the barrel so dependency invalidation remains correct. Because the Graph Cache persists the reexport Edge Type, target Symbol, and Alias Symbol identity, a new CLI process can recover the same implementation target after incremental Indexing without persisting analyzer metadata or adding a compatibility path. + +Do not add generic connected neighborhoods to Search or Target Query. Do not add a public traversal command, Leiden dependency, PageRank ranker, or god-node report without a controlled task where the added observation improves the selected metric without an unacceptable regression in the others. + +## Consequences + +- Exact Symbol dependencies and paths no longer depend on Graph View visibility settings. +- Calls through barrels terminate at implementation Symbols while lexical imports still identify the barrel. +- `reexport` is independently filterable and queryable as an Edge Type. +- Renamed exports are first-class Alias Symbols instead of metadata that disappears at the Graph Cache boundary. +- Full and incremental CLI processes produce the same tested call target through named reexports. +- Broad connection expansion remains rejected for the current agent workflow despite useful isolated rankings. + +## References + +- Graphify source and traversal implementation: https://github.com/Graphify-Labs/graphify +- ADR 0002, SQLite Graph Cache boundaries +- ADR 0007, Search and Target Query +- ADR 0008, deterministic phrase search and safe agent settings diff --git a/docs/adr/0010-agent-navigation-is-token-first-and-task-selective.md b/docs/adr/0010-agent-navigation-is-token-first-and-task-selective.md new file mode 100644 index 0000000000..b1202f5271 --- /dev/null +++ b/docs/adr/0010-agent-navigation-is-token-first-and-task-selective.md @@ -0,0 +1,96 @@ +# Agent navigation is token-first and task-selective + +**Status:** Superseded by ADR 0011 + +## Context + +Earlier CodeGraphy experiments optimized elapsed time, tool calls, model tokens, and tool output together. The measurements repeatedly showed that process time was not the dominant cost: extra model turns amplified cached context, and a graph workflow could look fast while consuming substantially more model tokens than ordinary navigation. Model tokens per correct task are therefore the primary product metric. Elapsed time, calls, output bytes, and indexing cost remain diagnostics. + +Five fresh low-thinking research agents inspected Graphify, LARGER, Aider RepoMap, RepoGraph, LocAgent, and GraphLocator from primary papers and source where available: + +- Graphify chooses a few lexical seeds, bounds BFS/DFS, suppresses expansion through high-degree hubs, and serializes compact Node/Edge lines under an approximate 2,000-token budget. Exact-symbol traversal and `explain` were more useful than broad phrase traversal on the CodeGraphy fixture. +- LARGER's paper describes adding at most ten confidence-filtered graph neighbors to lexical observations without another agent action. No official implementation was published, so its omitted path-confidence and community-scoring details cannot be copied exactly. +- Aider builds a def/reference File graph, runs personalized PageRank biased by active files and prompt-mentioned identifiers, and binary-searches a rendered repository map into a token budget. +- RepoGraph exposes one shallow symbol action. LocAgent combines lexical seeds with typed one-hop traversal. GraphLocator supports edge-centered search but adds LLM filtering and much larger turn budgets. + +The shared useful idea is bounded task-selective evidence, not graph traversal by default. + +## Clean-install evaluation + +The evaluation moved from answer-only localization prompts to two real historical CodeGraphy regressions at their original failing commits: + +1. globally enabling a shared Plugin ID updated only one conflicting installed package record; +2. filtered files consumed `maxFiles`, starving eligible files and polluting a fresh Graph Cache. + +Each run started from an isolated copy containing the original failing regression test. Repository Markdown, Agent Skills, context files, prompt templates, extensions, sessions, shared agent homes, and visible answers were absent. The raw arm's restricted `PATH` could not resolve `codegraphy`. CLI arms received a freshly built external CLI and a separately prepared Graph Cache. Every run used `openai-codex/gpt-5.6-sol` with low thinking, could edit production code, and was accepted only when the untouched historical regression test passed. Indexing cost was measured outside task time. + +Simply placing the CLI on `PATH` did not produce a treatment: none of six skill-free agents invoked the unknown command. A tiny command contract caused agents to use it, but results split by task. Plugin activation improved from 108,404 to 44,159 median tokens, while discovery regressed from 134,679 to 207,836. Discoverability and command-selection policy are therefore part of the agent interface, but they must not contain task answers. + +A clean three-arm skill screen then compared raw navigation, a 74-word two-call rule, and the previous full skill: + +| Median total tokens | Raw | Short bounded skill | Previous full skill | +|---|---:|---:|---:| +| Plugin activation | 199,805 | 135,099 | 85,480 | +| Discovery budget | 313,846 | 178,759 | 225,249 | +| Correct runs | 5/6 | 6/6 | 6/6 | + +The short rule controlled exploration but did not teach Target Query selection. The full skill localized the plugin task well but allowed repeated broad phrase searches on the discovery task. A compact policy combined both lessons: + +- use CodeGraphy only when source ownership or a Relationship is unknown; +- skip it when an exact source or failing test already localizes a small task; +- search with an exact identifier plus one task-domain word, or at most three short task literals; +- never submit a sentence or guessed API name; +- normally allow one Search and one exact Target Query, with one additional Relationship continuation only when unresolved; +- read returned paths directly instead of relocalizing them through repository-wide search. + +The compact policy screening medians were 70,258 tokens for Plugin activation and 170,295 for discovery, with all six runs correct. The exact shipped 3,407-byte skill was then paired against raw navigation in a fresh six-run comparison: + +| Median | Raw | CodeGraphy + shipped skill | Delta | +|---|---:|---:|---:| +| Plugin activation tokens | 173,833 | 70,556 | 59.4% fewer | +| Discovery budget tokens | 258,625 | 247,174 | 4.4% fewer | +| Pooled tokens | 199,642 | 113,360 | 43.2% fewer | +| Plugin activation calls | 16 | 18 | 2 more | +| Discovery budget calls | 28 | 22 | 6 fewer | +| Correct runs | 6/6 | 6/6 | equal | + +The sample is small and task variance remains high. It establishes a selected token-first policy for these tasks, not universal superiority. + +## Rejected additions + +A LARGER-inspired prototype attached up to two live per-term source lines to every natural-phrase File result. It reduced Plugin activation from the shipped skill's 70,556-token median to 59,356, but increased discovery from 247,174 to 577,260 and raised calls to 30–37. Superficially term-rich but causally irrelevant files looked authoritative. The prototype was reverted completely. + +Seeded weighted Leiden was evaluated on the clean fixture's File projection: 2,729 files, 5,853 weighted links, 549 communities, modularity 0.864, and 45.2 ms runtime. Community-density reranking improved none of six real localization phrases. It moved the discovery target from rank 3 to 10 and the Plugin target from rank 1 to 2. Communities remain useful architecture metadata candidates, but are not a default localization prior or output field. + +Generic connected neighborhoods remain rejected by ADR 0009. Agents read source after receiving them, so their added output increased end-to-end tokens. Target source embedding, BM25 fallback, mandatory Symbol preference, and an underspecified minimal skill remain rejected by ADR 0008 and ADR 0009. + +## Decision + +Use cumulative model `totalTokens` per correct task as the primary acceptance metric for agent-interface experiments. Report input, cache-read, output, and reasoning tokens separately, plus correctness, tool classes, result bytes, calls, elapsed time, and clean indexing cost. + +Ship the compact task-selective Agent Skill. The normal navigation budget is two CodeGraphy calls, with a third only for one unresolved Relationship continuation. Teach agents when not to use CodeGraphy and require direct reads after useful paths are known. + +Evaluate CLI behavior, Agent Skill behavior, and algorithm changes as separate arms. A raw baseline has no CodeGraphy executable or skill. A CLI-discoverability arm may state that the command exists but must not provide task-specific guidance. Skill arms differ only by frozen generalized instruction text. + +Do not add search evidence, graph neighborhoods, PageRank, Leiden communities, god-node output, or new traversal commands because they are plausible in isolation. Add one only after a clean held-out coding task shows lower end-to-end model tokens at non-inferior correctness. + +## Consequences + +- CodeGraphy is a selective accelerator, not a mandatory first step. +- Skill length is not minimized independently; enough command-selection guidance can reduce total trajectory tokens despite a larger initial prompt. +- Search phrase construction is part of retrieval quality. Common identifiers need one domain term; full natural-language sentences are discouraged. +- Runtime and tool-call improvements cannot compensate for token regression. +- Clean-install benchmarks must keep answers, repository instructions, caches, homes, and skill text isolated by condition. +- Leiden clustering has been tested and rejected as a default retrieval prior, while remaining available for a future explicit subsystem use case. + +## References + +- Graphify source: https://github.com/Graphify-Labs/graphify +- LARGER paper: https://arxiv.org/abs/2605.16352 +- Aider RepoMap: https://github.com/Aider-AI/aider/blob/main/aider/repomap.py +- RepoGraph: https://github.com/ozyyshr/RepoGraph +- LocAgent: https://github.com/gersteinlab/LocAgent +- GraphLocator: https://github.com/oceaneLIU/GraphLocator +- ADR 0007, Search and Target Query +- ADR 0008, agent Search and settings workflow +- ADR 0009, complete reexport Relationships and rejected connected neighborhoods diff --git a/docs/adr/0011-agents-choose-their-codegraphy-strategy.md b/docs/adr/0011-agents-choose-their-codegraphy-strategy.md new file mode 100644 index 0000000000..743ebc2c2e --- /dev/null +++ b/docs/adr/0011-agents-choose-their-codegraphy-strategy.md @@ -0,0 +1,50 @@ +# Agents choose their CodeGraphy strategy + +**Status:** Accepted + +## Context + +ADR 0010 selected a compact Agent Skill after two historical regression benchmarks. That skill improved model-token use, but it also prescribed a command sequence, fixed call budget, search wording, stopping rules, and when CodeGraphy was permitted. Those constraints measured compliance with one navigation policy rather than whether an agent understood CodeGraphy well enough to reason about the task. + +An unknown executable on `PATH` is not a meaningful CodeGraphy treatment because agents do not know that it exists or what its graph represents. The deployable agent interface is the CLI together with its generalized Agent Skill. + +Future CLI experiments also need a stable implementation workload. Diagnosis-only questions do not exercise the complete loop of localization, source reasoning, editing, test selection, and verification. Changing benchmark tasks between candidates makes token differences uninterpretable. + +## Decision + +The CodeGraphy Agent Skill describes the product rather than directing an agent workflow. It covers: + +- Relationship Graph Nodes, directed typed Edges, and static-analysis limits; +- Indexing, Graph Cache state, Plugins, Filters, Graph Scope, settings, status, and doctor; +- Search, Target Query, inventories, dependencies, dependents, and Path; +- live source evidence versus cached Symbols and Relationships; +- exact target identity, wildcard and phrase behavior, projections, pagination, bounds, and completeness; +- JSON stdout, structured stderr, exit status, verbose diagnostics, and recovery details; +- coverage, freshness, hub, static-analysis, and interpretation limitations. + +The skill does not prescribe a command sequence, maximum call count, preferred search phrase, stopping rule, or conditions under which the agent is allowed to use CodeGraphy. The agent selects commands and combines graph evidence with ordinary repository tools according to its own reasoning. + +Agent-utility experiments use one frozen moderate feature-change task until a deliberate benchmark-version change. Each condition runs three times in a fresh isolated workspace and agent home: + +- **Raw:** no CodeGraphy executable, Graph Cache, skill, repository Agent Markdown, prior session, or answer-bearing context. +- **CodeGraphy:** the candidate CLI plus the exact shipped generalized skill and a freshly prepared per-run Graph Cache. Indexing cost is measured separately. + +Both conditions receive the same neutral task prompt, source tree, dependency state, tool access, model, and thinking level. A run is correct only when the unchanged hidden validation passes and the requested behavior is implemented. Cumulative model tokens per correct run are primary. Mean and median tokens, token classes, calls, tool output, elapsed time, fallback navigation, files read, and indexing cost are reported. + +Each CLI or skill candidate changes one independently attributable behavior and reruns the same three-versus-three comparison. Candidates are retained only with non-inferior correctness and an end-to-end token improvement that is not explained solely by one outlier. Rejected implementation and generated fixtures are removed before the next candidate. + +## Consequences + +- CodeGraphy is evaluated as the installable CLI-plus-skill product, not as an undiscoverable executable. +- Agents can exploit commands and combinations not anticipated by the skill author. +- Skill improvements deepen accurate understanding rather than steering agents toward benchmark-specific trajectories. +- Feature implementation, hidden tests, and fresh workspaces make the benchmark closer to real coding work than answer-only localization. +- Three repetitions provide a fast iteration signal, not a universal performance claim; promising candidates require broader held-out confirmation. +- ADR 0010 remains the historical record of the token-first experiments, but its prescribed navigation policy is superseded. + +## References + +- ADR 0007, Search and Target Query +- ADR 0008, retrieval and settings experiments +- ADR 0009, exact reexport Relationships +- ADR 0010, initial token-first clean-install experiments diff --git a/docs/adr/0012-agent-benchmarks-require-structural-work-and-tool-adoption.md b/docs/adr/0012-agent-benchmarks-require-structural-work-and-tool-adoption.md new file mode 100644 index 0000000000..24f0836049 --- /dev/null +++ b/docs/adr/0012-agent-benchmarks-require-structural-work-and-tool-adoption.md @@ -0,0 +1,100 @@ +# Agent benchmarks require structural work and tool adoption + +**Status:** Accepted + +## Context + +ADR 0011 established fresh three-versus-three implementation benchmarks and a generalized, non-prescriptive Agent Skill. The first candidate task added `path --max-depth`. All six agents implemented it correctly, but only one of three CodeGraphy agents made one CLI help call. The task named the exact command and reduced to local parser/help work, so its token difference could not measure graph navigation. + +A connection-aware offline graph-document feature was broader, but its first prompt named the owning tldraw package. After removing that ownership leak, agents still localized the task primarily through lexical evidence. Corrected telemetry later found that one treatment agent made two CodeGraphy calls through `pnpm codegraphy`; the original counter recognized only a bare executable. Treatment still regressed against raw agents despite equal 3/3 correctness: + +| Mean | Raw | CodeGraphy + skill | Delta | +|---|---:|---:|---:| +| Model tokens | 369,924 | 404,126 | 9.2% more | +| Tool calls | 22.0 | 28.7 | 30.3% more | +| Tool output | 123,569 B | 140,578 B | 13.8% more | +| Elapsed | 126.0 s | 135.1 s | 7.2% slower | +| CodeGraphy calls | 0 | 0.7 | one of three adopted | + +The task was therefore rejected as a stable graph benchmark. It primarily measured the overhead and model variance of loading a skill around work that ordinary lexical search already solved directly. + +The replacement structural task starts from the parent of historical commit `e83057517`. Agents add cleanup ownership for headless Plugin runtimes across registry removal, one-shot Indexing success/failure, persistent engine rebuilds, and idempotent disposal. The hidden grader replaces two test modules with the original feature tests and runs Core typecheck. The untouched parent fails four behavioral assertions. Repository Markdown and source history are absent; dependencies come from the historical frozen lockfile. + +The neutral generalized-skill baseline produced equal 2/3 correctness: + +| Mean | Raw | CodeGraphy + neutral skill | Delta | +|---|---:|---:|---:| +| Tokens, all runs | 772,809 | 473,583 | 38.7% fewer | +| Tokens, correct runs | 820,903 | 475,038 | 42.1% fewer | +| Tool calls | 30.7 | 29.3 | 4.3% fewer | +| Tool output | 228,083 B | 242,281 B | 6.2% more | +| Elapsed | 171.7 s | 137.2 s | 20.1% faster | +| Correctness | 2/3 | 2/3 | equal | + +Only the failed treatment run invoked CodeGraphy, however. The successful token difference cannot be attributed to graph queries and remains a high-variance product-level signal rather than CLI evidence. + +## Tested candidates + +### Coverage-balanced Triage + +A `triage ` prototype independently matched task terms, weighted lexical rarity and File graph degree, preferred production source, and round-robin balanced source areas. It returned at most eight Files by default with matched terms and no excerpts, inferred answers, or generic neighborhoods. + +On the lexical feature task, all three agents read the skill but none selected Triage. Correctness remained 3/3 while treatment regressed 21.9% in tokens, 41.0% in calls, 13.7% in output, and 6.1% in elapsed time. The command had no adoption and was reverted. + +### Evidence-first generalized skill + +A reordered skill described questions answered by live text, exact Target Query, dependencies, dependents, Path, and inventories before lifecycle details. It explicitly said there was no required command order or call count. On the structural task, no agent invoked CodeGraphy. Treatment correctness was 3/3 versus raw 2/3, but mean tokens per correct run increased from 242,665 to 806,164, calls increased 56.1%, and elapsed increased 55.6%. The wording was reverted. + +### Combined Triage and evidence-first skill + +A detached candidate combined both rejected ideas to test their interaction. Every treatment agent selected Triage exactly once, proving discoverability and adoption. It still failed the correctness and token gate: + +| Mean | Raw | Triage + evidence skill | Delta | +|---|---:|---:|---:| +| Tokens, all runs | 619,031 | 559,269 | 9.7% fewer | +| Tokens, correct runs | 619,031 | 696,005 | 12.4% more | +| Tool calls | 26.3 | 34.7 | 31.6% more | +| Tool output | 259,628 B | 203,540 B | 21.6% fewer | +| Elapsed | 137.0 s | 153.6 s | 12.1% slower | +| Correctness | 3/3 | 2/3 | one fewer | +| CodeGraphy calls | 0 | 1.0 | adopted | + +Triage reduced observation bytes, but successful trajectories consumed more model tokens and one implementation was incomplete. The detached worktree and prototype were removed. + +### Incoming Impact radius + +An `impact ` prototype traversed incoming Relationships from one exact File or Symbol to depth one through four, projected results to Files, and reported shortest distance, typed reasons, pagination, visited Nodes, and completeness. Runtime call/import reasons ranked ahead of type-only consumers. On the structural fixture, a 1.4 KB result around `plugins/registry.ts` placed `engine.ts`, `engineSetup.ts`, and `workspace.ts` in the first page. + +All three treatment agents read the updated skill and none selected Impact. Corrected telemetry found three ordinary CodeGraphy Search calls in one successful treatment run. Raw correctness was 3/3 versus treatment 2/3. Mean correct-run tokens increased from 318,247 to 832,347, calls increased 60.3%, output increased 28.2%, and elapsed increased 65.7%. These trajectory regressions cannot be attributed to unused Impact output, but non-adoption fails the public-interface gate. The implementation, tests, documentation, and changeset update were reverted. + +### Compact capability reference + +A subtraction candidate reduced the neutral capability reference from 6,082 to 4,051 bytes without adding workflow policy. It increased voluntary CodeGraphy adoption to two of three treatment runs, with four Search calls in each adopted run. Both adopted runs omitted the required bulk-disposal API and failed; the only successful treatment did not invoke CodeGraphy. Treatment correctness fell to 1/3 from raw 3/3, and correct-run tokens increased 40.4%. The shorter skill and its release assertion were reverted. + +## Decision + +Retain the neutral generalized Agent Skill from ADR 0011. It explains the graph model, lifecycle, command semantics, output, freshness, shaping, and limitations without prescribing a workflow. Do not retain Triage or evidence-first comparative framing. + +Retain the headless Plugin-runtime disposal task as a harder confirmation benchmark, and select an independently viable relationship task before screening new candidates. Report both all-run and correct-run token statistics. A CLI candidate must be invoked by treatment agents before a result can be attributed to that candidate. Non-adoption is a failed interface experiment even when aggregate model variance favors the treatment. + +Correctness remains a gate. Lower output bytes, elapsed time, or all-run mean tokens do not justify a candidate whose correct-run tokens regress or whose correctness drops. + +Three repetitions remain a fast screening instrument rather than confirmation. Raw means varied materially between rounds, so candidates that pass screening require broader paired tasks before a universal product claim. + +## Consequences + +- Exact, local parser features are too shallow for the ongoing graph benchmark. +- Lexical cross-package tasks are also insufficient when ordinary search exposes ownership directly. +- New commands must earn both discoverability and successful-trajectory value. +- Skill wording is itself an intervention and can increase exhaustive ordinary exploration even without CLI use. +- Tool output reduction is diagnostic; cumulative model tokens per correct implementation remain primary. +- Watcher, neighbor, conflict, inference, and hub experiments remain candidates under the same adoption and correctness gate; the tested incoming Impact command is rejected. +- ADR 0013 records the rejected fuzzy Symbol fallback. +- ADR 0013 records the selected relationship fixture and accepted personalized Task Map. + +## References + +- ADR 0009, rejected connected neighborhoods +- ADR 0010, initial token-first clean-install experiments +- ADR 0011, generalized skill and feature-change benchmark protocol +- Historical feature commit `e83057517`, Core Plugin runtime disposal diff --git a/docs/adr/0013-personalized-task-maps-reduce-agent-navigation.md b/docs/adr/0013-personalized-task-maps-reduce-agent-navigation.md new file mode 100644 index 0000000000..c06666fb35 --- /dev/null +++ b/docs/adr/0013-personalized-task-maps-reduce-agent-navigation.md @@ -0,0 +1,124 @@ +# Personalized task maps reduce agent navigation + +**Status:** Accepted + +## Context + +ADR 0012 required structural coding work, hidden behavioral grading, correct-run token accounting, and actual candidate adoption. Repeated use of the Plugin-runtime disposal fixture then saturated that one task. A second historical fixture was needed to distinguish a generally useful graph action from benchmark-specific steering. + +The first replacement candidate, complete-graph cache persistence from parent `daa5b9dff`, was rejected as a benchmark. The untouched parent failed six hidden behaviors and the historical patch passed all sixteen focused tests, but no low-thinking agent completed both the Core save path and every Extension persistence path. A tightened one-versus-one viability pilot also produced 0/2 correctness. The task was too broad for rapid screening. + +The selected relationship fixture starts from parent `e93ac48a`. It contains two independent background Plugin failures: scheduled extension-host graph work and Webview Plugin asset loading. Hidden tests require both rejected Promises to be observed, contextual errors to retain the original failure, and asset-load completion to run after rejection. The grader initially required historical log wording; that implementation-coupled assertion was replaced by behavioral checks. The untouched parent remains red, the historical patch and both independent pilot solutions pass, and all repository Markdown, VCS history, sessions, answers, and benchmark infrastructure remain outside agent workspaces. + +The corrected benchmark invocation counter recognizes direct `codegraphy`, package-manager launchers such as `pnpm codegraphy` and `pnpm exec codegraphy`, and direct Node launchers. Earlier reports understated existing CLI adoption when agents used package-manager scripts. + +With the existing CLI and neutral shipped skill unchanged, the relationship fixture established 3/3 correctness in both arms: + +| Mean | Raw | Existing CLI + skill | Delta | +|---|---:|---:|---:| +| Model tokens | 278,262 | 238,049 | 14.5% fewer | +| Median tokens | 270,199 | 147,307 | 45.5% fewer | +| Tool calls | 19.0 | 25.0 | 31.6% more | +| Tool output | 170,084 B | 92,201 B | 45.8% fewer | +| Elapsed | 108.8 s | 89.8 s | 17.5% faster | +| CodeGraphy adoption | 0/3 | 3/3 | 2.7 calls/run | + +This established that agents voluntarily select the existing CLI on a task that remains objectively solvable by raw navigation. + +## Candidate + +`codegraphy map ` accepts task or issue text and returns a bounded personalized File map. It: + +- extracts independently useful non-stopword terms with light identifier stemming; +- scores term presence with lexical rarity rather than term frequency; +- combines live eligible source with cached File, Symbol, and typed Relationship facts; +- uses personalized PageRank to retain connected Files; +- balances source areas so one dense subsystem does not consume the complete result; +- returns eight Files by default, rejects CLI bounds above twenty, and includes at most three selected declarations per File and twelve typed Relationships among returned Files; +- reports pagination, truncation completeness, live-source freshness, and Graph Cache freshness; +- returns no source excerpts, inferred answers, hidden workflow policy, or model-specific token estimates. + +On the selected fixture, one 5.4 KB response places both required production Files in its first six results. On the harder disposal fixture, source-area balancing exposes the Core Indexing runtime and Plugin Registry rather than filling the page with sibling engine modules. + +## Benchmark results + +### Required raw comparison + +The Map candidate retained 3/3 correctness in both arms. Candidate treatment reduced mean tokens and output, but Map itself was selected in only one of three runs, so the aggregate result was not sufficient attribution: + +| Mean | Raw | Map CLI + skill | Delta | +|---|---:|---:|---:| +| Model tokens | 281,602 | 205,919 | 26.9% fewer | +| Median tokens | 198,193 | 209,454 | 5.7% more | +| Tool calls | 19.3 | 27.0 | 39.7% more | +| Tool output | 126,647 B | 84,756 B | 33.1% fewer | +| Elapsed | 80.7 s | 89.5 s | 10.9% slower | +| Map adoption | 0/3 | 1/3 | partial | + +### Direct candidate A/B + +A fresh direct comparison then held CLI access, skill installation, pre-Indexing, model, prompt, hidden grader, and workspace isolation constant. The control used pre-Map commit `93ffde349`; the treatment used the Map candidate. Both arms were 3/3 correct, and treatment agents selected Map in two of three runs: + +| Mean | Existing CLI + skill | Map CLI + skill | Delta | +|---|---:|---:|---:| +| Model tokens | 257,334 | 167,940 | **34.7% fewer** | +| Median tokens | 243,105 | 159,426 | **34.4% fewer** | +| Tool calls | 32.0 | 21.0 | **34.4% fewer** | +| CodeGraphy calls | 5.0 | 3.3 | **33.3% fewer** | +| Tool output | 115,915 B | 71,745 B | **38.1% fewer** | +| Elapsed | 139.9 s | 97.2 s | **30.5% faster** | +| Correctness | 3/3 | 3/3 | equal | + +The Map-only treatment trajectory completed correctly with one CodeGraphy call, twenty total calls, and 220,943 model tokens. Another trajectory used Map after two Search calls and completed with 123,451 tokens. + +### Harder-task confirmation + +A fresh disposal-fixture confirmation retained equal 2/3 correctness. Correct-run mean tokens were 246,332 for CLI plus skill versus 685,081 raw, a 64.0% reduction, but no treatment agent selected Map. This confirms existing CLI value on that round, not Map value. It also shows that Map is task-selective rather than a required replacement for Search. + +### Compact payload subtraction + +A follow-up candidate reduced default output from eight Files, three declarations per File, and twelve Relationships to six, one, and eight. The isolated payload fell from about 5.4 KB to 2.4 KB while retaining both relationship-task targets and the harder-task Indexing runtime and Plugin Registry anchors. Its required raw comparison remained 3/3 correct and reduced mean treatment tokens by 32.0%, but that comparison could not isolate payload shape from general CLI value. + +A direct accepted-Map-versus-compact-Map A/B had 3/3 correctness and Map adoption in all six runs. Compact Map regressed mean tokens 73.2%, median tokens 53.2%, calls 10.0%, output 84.5%, and elapsed time 45.9%. Agents compensated for omitted context with substantially more fallback Search. The compact bounds were reverted; the accepted eight/three/twelve defaults remain. + +### Fuzzy Symbol fallback + +A Search candidate returned up to three exact Symbol identities labeled `match: "fuzzy"` only when an identifier-like pattern had no exact, wildcard, phrase, source, Node, or Symbol evidence. Isolated smoke repaired a one-character scheduler typo and the guessed `loadPluginAssets` identifier observed in an earlier trajectory. + +In its required relationship-task comparison, treatment correctness was 3/3 versus raw 2/3 and correct-run mean tokens were 1.7% lower. However, no treatment Search activated the fallback. Agents selected exact and phrase searches instead, while calls, output, and elapsed time regressed. The candidate failed attribution and was fully reverted. + +## Remaining queue disposition + +The remaining candidate queue does not contain another clean, benchmarkable forward path under current architecture: + +- **One-hop neighbors / edge explanation:** exact `query` already returns bounded incoming and outgoing typed Relationships, while `dependencies` and `dependents` expose either direction. Another neighbor command would duplicate those contracts and recreate the rejected generic-neighborhood path. +- **Watcher-assisted impact:** the CLI is intentionally one-shot. A watcher requires a resident lifecycle, cache locking, debounce and recrawl semantics, and query-time mutation. That conflicts with ADR 0003's no-server boundary and the read-only query contract. Explicit incremental `index` remains the coherent refresh operation. +- **Hub diagnostics:** hub suppression and community priors did not improve held-out localization, and a standalone god-node inventory has no demonstrated adoption in coding work. Task Map already uses degree-normalized transitions and source-area diversity without presenting centrality as risk. +- **Conflict / co-change risk:** Core has no VCS-history abstraction, rename policy, time window, or persistence model. Clean benchmarks intentionally remove VCS history. Adding Git subprocess behavior solely for a benchmark would violate fixture isolation and make historical coupling look like static truth. +- **Bounded inference:** call, import, inheritance, and reference Relationships have different transitivity. Generic two-hop inference would create semantically false Edges unless each language Plugin defines and tests explicit rules and confidence provenance. + +These are blockers or duplicate interfaces, not invitations for flags, shims, or weakened tests. Reconsider them only with a new architectural requirement or a viable historical task that naturally exercises the missing evidence. + +## Decision + +Retain `codegraphy map ` as a public bounded query. Its selected direct A/B passed adoption, correctness, and correct-run token gates while also reducing calls, output, and elapsed time. + +Keep Search, exact Target Query, dependencies, dependents, and Path unchanged. Map is another evidence surface, not a prescribed first action or universal navigation policy. The Agent Skill describes its output semantics but does not specify command order, call counts, task wording, or permission conditions. + +Treat all measurements as small-sample evidence. Three-versus-three runs screen candidates; they do not establish universal superiority. Future confirmation should use the relationship fixture plus at least one additional independently viable structural task. + +## Consequences + +- Candidate attribution can use an additional direct CLI-control A/B after the required raw comparison when general CLI behavior already differs from raw navigation. +- Viability pilots may reject benchmark fixtures before a full candidate run, but product iterations still require three runs per compared condition. +- Hidden graders must assert behavior rather than historical wording or decomposition. +- Invocation telemetry must recognize equivalent executable launch forms. +- Complete-graph persistence remains an unsuitable rapid screen despite being a valid historical feature. +- Incoming Impact, Triage, connected Search augmentation, BM25 fallback, phrase-line evidence, fuzzy Symbol fallback, compact skill subtraction, compact Map payloads, and default Leiden ranking remain rejected. + +## References + +- ADR 0011, generalized skill and fresh implementation benchmarks +- ADR 0012, structural work and adoption gate +- Aider repository map implementation and personalized PageRank +- RepoGraph and LocAgent single-action graph localization interfaces diff --git a/packages/core/README.md b/packages/core/README.md index bb6c39fb57..c4dbb22bc3 100644 --- a/packages/core/README.md +++ b/packages/core/README.md @@ -8,14 +8,19 @@ The published CLI currently supports Node.js 20 through 22; Node 22 LTS is recom The VS Code extension bundles this package for extension runtime behavior. Users install `@codegraphy-dev/core` globally only when they want terminal workflows such as Indexing, diagnostics, graph queries, Graph Scope and filter configuration, plugin registration, or workspace plugin enablement. -All `codegraphy ...` terminal commands live in this package. `codegraphy index` incrementally makes a workspace Graph Cache current and persists the complete Relationship Graph independently of Graph Scope, so later scope changes only affect projection and queries. Active Filters and Git ignored state gate fresh analysis without deleting reusable facts cached while those files were eligible. Query commands use positional inputs and bounded defaults; Symbol Nodes are exposed through `nodes` and `search`, and `edges` is the single Relationship Graph primitive. Every query also accepts repeatable, comma-separated `--filter`, `--node-type`, and `--edge-type` options for one invocation without changing workspace settings. Node Type projections follow Graph Scope hierarchy and symbol meaning while each result keeps its specific stored Node Type. Commands use the current directory unless the global `-C, --workspace ` option selects another workspace. +All `codegraphy ...` terminal commands live in this package. `codegraphy index` incrementally makes a workspace Graph Cache current, reports structured file-budget truncation with an exact recovery command, and persists the complete Relationship Graph independently of Graph Scope. Active Filters and Git ignored state gate fresh analysis without deleting reusable facts cached while those files were eligible. `settings get/set/unset` safely reads or mutates validated workspace settings and refuses to overwrite corrupt JSON. `search` merges exact live source, cached AST Symbol, and indexed Node evidence, then uses deterministic all-term File ranking when a natural multi-term phrase has few literal matches. `map` combines independent task terms, selected declarations, and personalized graph ranking into a bounded File map with typed connecting Relationships. `query` inspects one exact File path or Symbol Node ID and returns prioritized declarations plus incoming and outgoing Relationships. Exact targeted queries use the complete cached graph by default, independently of saved Graph View Scope, while explicit `--node-type` or `--edge-type` projections constrain that dimension. JavaScript-family reexports are indexed explicitly, allowing call Relationships to resolve through barrels to implementation Symbols. The narrower `nodes`, `edges`, `dependencies`, `dependents`, and `path` commands continue or enumerate the graph. Graph navigation accepts repeatable, comma-separated `--filter`, `--node-type`, and `--edge-type` options for one invocation without changing workspace settings. Commands use the current directory unless the global `-C, --workspace ` option selects another workspace. ```bash +codegraphy settings get maxFiles +codegraphy settings set maxFiles 2500 codegraphy index codegraphy status codegraphy doctor codegraphy nodes codegraphy search SettingsPanel +codegraphy search 'Indexing *workspace*' +codegraphy map 'settings corruption during filtering' +codegraphy query src/cli/index/command.ts codegraphy edges codegraphy dependencies src/app.ts codegraphy dependents src/config.ts @@ -31,9 +36,9 @@ Run `codegraphy --help` for the full workflow and `codegraphy --help` ## Current Entry Points - CodeGraphy Workspace paths: resolve `.codegraphy/settings.json` and `.codegraphy/graph.sqlite` for any folder path. -- Workspace Settings: read, normalize, write, and fingerprint workspace-local settings, including Graph Scope, filters, and ordered plugin entries without erasing extension-owned presentation settings. +- Workspace Settings: strictly validate persisted known fields, safely read or mutate them through the CLI, and preserve unknown extension fields without replacing corrupt input with defaults. - File Discovery: discover analyzable files and directories, apply active custom/plugin/Git filters before the eligible-file budget, and report cache-retention paths without VS Code APIs. -- Built-in language analysis: parse supported languages and produce file, symbol, import, call, inherit, reference, and type-import relationships. +- Built-in language analysis: parse supported languages and produce file, symbol, import, reexport, call, inherit, reference, and type-import relationships. - File Analysis: run cache-aware per-file plugin analysis and project file relationships without VS Code APIs. - Core Indexing: index an explicit CodeGraphy Workspace path, run headless plugins, build the Relationship Graph, and write the workspace Graph Cache. - Workspace Analysis: orchestrate discovery, pre-analysis hooks, file analysis, cache updates, and graph rebuilds through headless dependencies. @@ -44,7 +49,10 @@ Run `codegraphy --help` for the full workflow and `codegraphy --help` - Graph Cache status: report whether a workspace-local Graph Cache exists without using VS Code APIs. - Workspace status: report fresh, stale, or missing Graph Cache state with inspectable stale reasons. - Graph Cache storage: load, save, clear, and inspect normalized File, Node, Symbol, and Edge rows in the SQLite-backed Graph Cache at `/.codegraphy/graph.sqlite`. -- Graph Query: search scoped Nodes, list scoped Edges, trace dependencies and dependents, and find bounded paths over Relationship Graph data plus persisted analysis metadata. +- Workspace Search: merge bounded exact evidence with deterministic all-term File fallback candidates for natural multi-term phrases. +- Task Map: rank task-relevant Files from live terms and cached Relationships, with selected declarations and typed connections. +- Target Query: inspect one exact File or Symbol with prioritized declarations and bounded Relationships. +- Graph Query: list scoped Nodes and Edges, then use complete cached types by default for exact targeted relationships and bounded paths unless an invocation explicitly projects Node or Edge Types. The core package exposes `indexCodeGraphyWorkspace` for explicit path-based Indexing. VS Code and CLI adapters call this package instead of owning independent indexing behavior. diff --git a/packages/core/src/analysis/fileAnalysis/enrichment.ts b/packages/core/src/analysis/fileAnalysis/enrichment.ts index af3e9bb32d..a9c9d3c4fe 100644 --- a/packages/core/src/analysis/fileAnalysis/enrichment.ts +++ b/packages/core/src/analysis/fileAnalysis/enrichment.ts @@ -8,6 +8,10 @@ export function enrichWorkspaceFileAnalysis( fileAnalysis: ReadonlyMap, ): Map { const symbolsByFilePath = createSymbolsByFilePath(fileAnalysis); + const relationsByFilePath = new Map([...fileAnalysis.values()].map(analysis => [ + analysis.filePath, + analysis.relations ?? [], + ])); return new Map( Array.from(fileAnalysis.entries()).map(([filePath, analysis]) => [ @@ -15,7 +19,7 @@ export function enrichWorkspaceFileAnalysis( { ...analysis, relations: (analysis.relations ?? []).map((relation) => - enrichRelationTargetSymbol(relation, symbolsByFilePath), + enrichRelationTargetSymbol(relation, symbolsByFilePath, relationsByFilePath), ), }, ]), diff --git a/packages/core/src/analysis/fileAnalysis/reexportTarget.ts b/packages/core/src/analysis/fileAnalysis/reexportTarget.ts new file mode 100644 index 0000000000..ddc6ac2576 --- /dev/null +++ b/packages/core/src/analysis/fileAnalysis/reexportTarget.ts @@ -0,0 +1,118 @@ +import type { + IAnalysisRelation, + IAnalysisSymbol, +} from '@codegraphy-dev/plugin-api'; +import { readRelationSymbolName, resolveTargetSymbolId } from './targetSymbolName'; + +export interface ResolvedTargetSymbol { + filePath: string; + symbolId: string; +} + +function readMetadataString(relation: IAnalysisRelation, key: string): string | undefined { + const value = relation.metadata?.[key]; + return typeof value === 'string' && value.length > 0 ? value : undefined; +} + +function isMetadataTrue(relation: IAnalysisRelation, key: string): boolean { + return relation.metadata?.[key] === true; +} + +function persistedReexportTargetName( + reexport: IAnalysisRelation, + symbolsByFilePath: ReadonlyMap, +): string | undefined { + if (reexport.kind !== 'reexport' || !reexport.toFilePath || !reexport.toSymbolId) { + return undefined; + } + return symbolsByFilePath + .get(reexport.toFilePath) + ?.find(symbol => symbol.id === reexport.toSymbolId) + ?.name; +} + +function forwardedReexportRelation( + relation: IAnalysisRelation, + reexport: IAnalysisRelation, + symbolName: string, + symbolsByFilePath: ReadonlyMap, +): IAnalysisRelation | undefined { + if (!reexport.toFilePath) return undefined; + const reexportAll = isMetadataTrue(reexport, 'reexportAll'); + const exportedName = readMetadataString(reexport, 'exportedName'); + const persistedTargetName = persistedReexportTargetName(reexport, symbolsByFilePath); + const namedReexport = isMetadataTrue(reexport, 'reexport') && exportedName === symbolName; + const persistedFileReexport = reexport.kind === 'reexport' + && !reexport.toSymbolId + && !isMetadataTrue(reexport, 'reexport'); + if (!reexportAll && !namedReexport && !persistedFileReexport + && persistedTargetName !== symbolName) return undefined; + const importedName = reexportAll || persistedFileReexport + ? symbolName + : readMetadataString(reexport, 'importedName') ?? persistedTargetName ?? symbolName; + return { + ...relation, + toFilePath: reexport.toFilePath, + resolvedPath: reexport.resolvedPath ?? reexport.toFilePath, + metadata: { + ...relation.metadata, + importedName, + }, + }; +} + +function followsReexportedTargets(relation: IAnalysisRelation): boolean { + return ['call', 'event', 'inherit', 'reference'].includes(relation.kind); +} + +export function resolveRelationTargetSymbols( + relation: IAnalysisRelation, + symbolsByFilePath: ReadonlyMap, + relationsByFilePath: ReadonlyMap, + visitedFiles: ReadonlySet = new Set(), +): ResolvedTargetSymbol[] { + if (!relation.toFilePath || visitedFiles.has(relation.toFilePath)) return []; + const targetSymbols = symbolsByFilePath.get(relation.toFilePath) ?? []; + const persistedTarget = relation.toSymbolId + ? targetSymbols.find(symbol => symbol.id === relation.toSymbolId) + : undefined; + const directSymbolId = persistedTarget?.id ?? (targetSymbols.length > 0 + ? resolveTargetSymbolId(relation, targetSymbols) + : undefined); + const directSymbol = targetSymbols.find(symbol => symbol.id === directSymbolId); + if (directSymbol && directSymbol.kind !== 'alias') { + return [{ filePath: relation.toFilePath, symbolId: directSymbol.id }]; + } + if (directSymbol) { + const nextVisited = new Set([...visitedFiles, relation.toFilePath]); + return (relationsByFilePath.get(relation.toFilePath) ?? []) + .filter(aliasRelation => aliasRelation.fromSymbolId === directSymbol.id) + .flatMap(aliasRelation => resolveRelationTargetSymbols( + aliasRelation, + symbolsByFilePath, + relationsByFilePath, + nextVisited, + )); + } + if (!followsReexportedTargets(relation)) return []; + + const symbolName = readRelationSymbolName(relation); + if (!symbolName) return []; + const nextVisited = new Set([...visitedFiles, relation.toFilePath]); + return (relationsByFilePath.get(relation.toFilePath) ?? []).flatMap((reexport) => { + const forwarded = forwardedReexportRelation( + relation, + reexport, + symbolName, + symbolsByFilePath, + ); + return forwarded + ? resolveRelationTargetSymbols( + forwarded, + symbolsByFilePath, + relationsByFilePath, + nextVisited, + ) + : []; + }); +} diff --git a/packages/core/src/analysis/fileAnalysis/targetSymbol.ts b/packages/core/src/analysis/fileAnalysis/targetSymbol.ts index 57e59a2291..45a71b2aea 100644 --- a/packages/core/src/analysis/fileAnalysis/targetSymbol.ts +++ b/packages/core/src/analysis/fileAnalysis/targetSymbol.ts @@ -2,26 +2,27 @@ import type { IAnalysisRelation, IAnalysisSymbol, } from '@codegraphy-dev/plugin-api'; -import { resolveTargetSymbolId } from './targetSymbolName'; +import { resolveRelationTargetSymbols } from './reexportTarget'; export function enrichRelationTargetSymbol( relation: IAnalysisRelation, symbolsByFilePath: ReadonlyMap, + relationsByFilePath: ReadonlyMap = new Map(), ): IAnalysisRelation { - if (relation.toSymbolId || !relation.toFilePath) { - return relation; - } + if (relation.toSymbolId || !relation.toFilePath) return relation; - const targetSymbols = symbolsByFilePath.get(relation.toFilePath); - if (!targetSymbols?.length) { - return relation; - } - - const resolvedSymbolId = resolveTargetSymbolId(relation, targetSymbols); - return resolvedSymbolId - ? { - ...relation, - toSymbolId: resolvedSymbolId, - } - : relation; + const targets = resolveRelationTargetSymbols( + relation, + symbolsByFilePath, + relationsByFilePath, + ); + const uniqueTargets = new Map(targets.map(target => [target.symbolId, target])); + if (uniqueTargets.size !== 1) return relation; + const target = [...uniqueTargets.values()][0]; + return { + ...relation, + toFilePath: target.filePath, + resolvedPath: target.filePath, + toSymbolId: target.symbolId, + }; } diff --git a/packages/core/src/analysis/fileAnalysis/targetSymbolName.ts b/packages/core/src/analysis/fileAnalysis/targetSymbolName.ts index c799fa48be..17546aeff7 100644 --- a/packages/core/src/analysis/fileAnalysis/targetSymbolName.ts +++ b/packages/core/src/analysis/fileAnalysis/targetSymbolName.ts @@ -8,14 +8,13 @@ export function resolveTargetSymbolId( targetSymbols: readonly IAnalysisSymbol[], ): string | undefined { const symbolName = readRelationSymbolName(relation); - const namedSymbolId = symbolName - ? resolveUniqueSymbolName(targetSymbols, symbolName) - : undefined; - - return namedSymbolId ?? resolveUniqueTargetSymbol(targetSymbols); + if (symbolName) return resolveUniqueSymbolName(targetSymbols, symbolName); + return hasRelationSymbolNameHint(relation) + ? undefined + : resolveUniqueTargetSymbol(targetSymbols); } -function readRelationSymbolName( +export function readRelationSymbolName( relation: IAnalysisRelation, ): string | undefined { const memberName = readRelationMetadataString(relation, 'memberName'); @@ -24,6 +23,14 @@ function readRelationSymbolName( return memberName ?? readNamedImport(importedName) ?? relation.specifier; } +function hasRelationSymbolNameHint(relation: IAnalysisRelation): boolean { + return Boolean( + readRelationMetadataString(relation, 'memberName') + || readRelationMetadataString(relation, 'importedName') + || relation.specifier, + ); +} + function readNamedImport(importedName: string | undefined): string | undefined { return importedName && importedName !== '*' && importedName !== 'default' ? importedName diff --git a/packages/core/src/cli/command.ts b/packages/core/src/cli/command.ts index 39ecd6d80a..d0cfc1f7e5 100644 --- a/packages/core/src/cli/command.ts +++ b/packages/core/src/cli/command.ts @@ -6,6 +6,7 @@ import { runPluginsCommand } from './plugins/command'; import { runQueryCommand } from './query/command'; import { runScopeCommand } from './scope/command'; import { runStatusCommand } from './status/command'; +import { runSettingsCommand } from './settings/command'; import type { CliCommand } from './parse'; import { createDiagnosticEvent, formatDiagnosticEventLine } from '../diagnostics/events'; @@ -83,6 +84,9 @@ export async function runCliCommand( case 'scope': result = runScopeCommand(command); break; + case 'settings': + result = runSettingsCommand(command); + break; case 'status': result = runStatusCommand(command.workspacePath, undefined, { verbose: command.verbose, diff --git a/packages/core/src/cli/doctor/command.ts b/packages/core/src/cli/doctor/command.ts index d85d8657c5..43bbfa5f3a 100644 --- a/packages/core/src/cli/doctor/command.ts +++ b/packages/core/src/cli/doctor/command.ts @@ -4,67 +4,18 @@ import { readCodeGraphyInstalledPluginCache } from '../../plugins/installedCache import { resolveCodeGraphyWorkspacePath } from '../../workspace/requestPaths'; import { CODEGRAPHY_MARKDOWN_PLUGIN_ID, + createInitialCodeGraphyWorkspaceSettings, readCodeGraphyWorkspaceSettingsOrInitial, + WorkspaceSettingsError, } from '../../workspace/settings'; import { getWorkspaceSettingsPath } from '../../workspace/paths'; import { readCodeGraphyWorkspaceStatus } from '../../workspace/status'; import type { CommandExecutionResult } from '../command'; import type { CliCommand } from '../parseTypes'; -import { hasSupportedRawPluginIdentity } from '../../workspace/settingsPlugins'; import { inspectWorkspaceAnalysisDatabase } from '../../graphCache/database/storage'; import { readCodeGraphyWorkspaceMeta } from '../../workspace/meta'; import { createDoctorCacheCheck } from './cacheCheck/model'; -function isRecord(value: unknown): value is Record { - return Boolean(value) && typeof value === 'object' && !Array.isArray(value); -} - -function isStringArray(value: unknown): boolean { - return Array.isArray(value) && value.every(entry => typeof entry === 'string'); -} - -function isBooleanRecord(value: unknown): boolean { - return isRecord(value) && Object.values(value).every(entry => typeof entry === 'boolean'); -} - -function validateKnownSettings(value: Record): string | undefined { - const stringArrays = ['include', 'filterPatterns', 'disabledCustomFilterPatterns'] as const; - for (const key of stringArrays) { - if (key in value && !isStringArray(value[key])) return `${key} must be an array of strings`; - } - const booleans = ['respectGitignore'] as const; - for (const key of booleans) { - if (key in value && typeof value[key] !== 'boolean') return `${key} must be a boolean`; - } - if ('maxFiles' in value && (typeof value.maxFiles !== 'number' || !Number.isFinite(value.maxFiles))) { - return 'maxFiles must be a finite number'; - } - if ('nodeVisibility' in value && !isBooleanRecord(value.nodeVisibility)) { - return 'nodeVisibility must be an object of boolean values'; - } - if ('edgeVisibility' in value && !isBooleanRecord(value.edgeVisibility)) { - return 'edgeVisibility must be an object of boolean values'; - } - if ('pluginData' in value && !isRecord(value.pluginData)) return 'pluginData must be an object'; - if ('plugins' in value) { - if (!Array.isArray(value.plugins)) return 'plugins must be an array'; - for (const entry of value.plugins) { - if (!isRecord(entry)) return 'plugins entries must be objects'; - if ('id' in entry && typeof entry.id !== 'string') return 'plugin id must be a string'; - if ('package' in entry && typeof entry.package !== 'string') return 'plugin package must be a string'; - if ('enabled' in entry && typeof entry.enabled !== 'boolean') return 'plugin enabled must be a boolean'; - if ('options' in entry && !isRecord(entry.options)) return 'plugin options must be an object'; - if ('disabledFilterPatterns' in entry && !isStringArray(entry.disabledFilterPatterns)) { - return 'plugin disabledFilterPatterns must be an array of strings'; - } - if (!hasSupportedRawPluginIdentity(entry)) { - return 'plugin entry must have a nonblank id with enabled, or a nonblank package'; - } - } - } - return undefined; -} - function readSettingsCheck(workspaceRoot: string): Record { const settingsPath = getWorkspaceSettingsPath(workspaceRoot); if (!fs.existsSync(settingsPath)) { @@ -75,16 +26,15 @@ function readSettingsCheck(workspaceRoot: string): Record { }; } try { - const value: unknown = JSON.parse(fs.readFileSync(settingsPath, 'utf-8')); - if (!isRecord(value)) throw new Error('expected a JSON object'); - const validationError = validateKnownSettings(value); - if (validationError) throw new Error(validationError); + readCodeGraphyWorkspaceSettingsOrInitial(workspaceRoot); return { ok: true, path: settingsPath }; } catch (error) { return { ok: false, path: settingsPath, - message: error instanceof Error ? error.message : String(error), + message: error instanceof WorkspaceSettingsError + ? error.reason + : error instanceof Error ? error.message : String(error), action: 'Repair `.codegraphy/settings.json`, then rerun `codegraphy doctor`.', }; } @@ -95,10 +45,12 @@ export function runDoctorCommand(command: CliCommand): CommandExecutionResult { const runtimeMajor = Number(process.versions.node.split('.')[0]); const runtimeOk = runtimeMajor >= 20 && runtimeMajor < 23; const settingsCheck = readSettingsCheck(workspaceRoot); - const status = readCodeGraphyWorkspaceStatus(workspaceRoot); + const settings = settingsCheck.ok + ? readCodeGraphyWorkspaceSettingsOrInitial(workspaceRoot) + : createInitialCodeGraphyWorkspaceSettings(); + const status = readCodeGraphyWorkspaceStatus(workspaceRoot, { settings }); const meta = readCodeGraphyWorkspaceMeta(workspaceRoot); const cacheInspection = inspectWorkspaceAnalysisDatabase(workspaceRoot); - const settings = readCodeGraphyWorkspaceSettingsOrInitial(workspaceRoot); const activity = createPluginActivityState({ settings, installedPlugins: readCodeGraphyInstalledPluginCache().plugins, diff --git a/packages/core/src/cli/help/command.ts b/packages/core/src/cli/help/command.ts index 703a55a1c2..cca6492850 100644 --- a/packages/core/src/cli/help/command.ts +++ b/packages/core/src/cli/help/command.ts @@ -8,8 +8,8 @@ const ROOT_HELP = [ 'Workflow:', ' 1. Configure graph contributors with Plugins when needed.', ' 2. Index the complete workspace graph into its Graph Cache.', - ' 3. Shape returned results with Filters and Graph Scope.', - ' 4. Query the resulting graph.', + ' 3. Discover source and AST Symbols with search; inspect an exact result with query.', + ' 4. Continue through the shaped graph with dependencies, dependents, or path.', '', 'Workspace commands:', ' codegraphy index Create or update the Graph Cache', @@ -17,6 +17,10 @@ const ROOT_HELP = [ ' codegraphy doctor Diagnose runtime, settings, cache, and Plugins', '', 'Settings commands:', + ' codegraphy settings Read all effective workspace settings', + ' codegraphy settings get Read one effective workspace setting', + ' codegraphy settings set Validate and persist one workspace setting', + ' codegraphy settings unset Restore one setting to its default', ' codegraphy filter Read or change persisted Filters', ' codegraphy filter Add or remove one Filter pattern', ' codegraphy scope Read or change persisted Graph Scope', @@ -24,14 +28,18 @@ const ROOT_HELP = [ ' codegraphy scope edge Change one Edge Type', ' codegraphy plugins Register Plugins and change global or workspace activity', '', - 'Query commands:', + 'Explore:', + ' codegraphy search Find live source text and cached AST Symbols', + ' codegraphy map Build a compact task-personalized File map', + ' codegraphy query Inspect one exact File or Symbol Node', + '', + 'Navigate the Relationship Graph:', ' codegraphy nodes List Nodes in the shaped graph', - ' codegraphy search Find Nodes by text', ' codegraphy edges List Relationships in the shaped graph', ' codegraphy dependencies List outgoing Relationships', ' codegraphy dependents List incoming Relationships', ' codegraphy path Find a bounded path between Nodes', - ' All query commands accept one-off --filter, --node-type, and --edge-type options.', + ' Graph navigation commands accept one-off --filter, --node-type, and --edge-type options.', '', 'Global options:', ' -C, --workspace Use another workspace (default: current directory)', @@ -64,7 +72,7 @@ const COMMAND_HELP: Record = { 'Enabled Plugins contribute facts during Indexing.', '', 'Effects: Writes .codegraphy/graph.sqlite and workspace index metadata.', - 'Output: JSON indexing summary.', + 'Output: JSON indexing summary plus discovery.limitReached and an actionable maxFiles command when capped.', 'Example: codegraphy index', ].join('\n'), status: [ @@ -86,6 +94,37 @@ const COMMAND_HELP: Record = { 'Failure envelope: Includes the completed checks in error.details.', 'Example: codegraphy doctor', ].join('\n'), + settings: [ + 'Usage: codegraphy settings [get | set | unset ]', + '', + 'Read or safely update workspace settings such as maxFiles, include, or respectGitignore.', + 'Values passed to set are JSON: numbers and booleans are unquoted; arrays and objects need shell quotes.', + 'Mutations validate the complete persisted file and never replace malformed settings with defaults.', + 'Supported keys: maxFiles, include, respectGitignore, filterPatterns, disabledCustomFilterPatterns,', + ' nodeVisibility, edgeVisibility, plugins, interfaces, pluginData.', + '', + 'Effects: Read-only without set/unset. Set and unset write .codegraphy/settings.json.', + 'Output: JSON effective values; mutations report previous/value and whether Indexing is required.', + 'Examples:', + ' codegraphy settings get maxFiles', + ' codegraphy settings set maxFiles 2500', + " codegraphy settings set include '[\"packages/**/*.ts\"]'", + ' codegraphy settings unset maxFiles', + ].join('\n'), + query: [ + 'Usage: codegraphy query ', + '', + 'Inspect one exact File path or Symbol Node ID returned by `codegraphy search`.', + 'Returns the target, prioritized declared AST Symbols, and incoming and outgoing Relationships.', + 'Use dependencies, dependents, or path only when this bounded overview needs continuation.', + '', + 'Arguments:', + ' Workspace-relative File path or exact Symbol Node ID', + '', + 'Effects: Read-only. Requires an existing Graph Cache and never performs Indexing.', + 'Output: JSON target overview with fixed per-section bounds and page totals.', + 'Example: codegraphy query packages/core/src/cli/command.ts', + ].join('\n'), nodes: [ 'Usage: codegraphy nodes [--limit ] [--offset ]', '', @@ -100,20 +139,40 @@ const COMMAND_HELP: Record = { 'Example: codegraphy nodes', ].join('\n'), search: [ - 'Usage: codegraphy search [--limit ] [--offset ] ', + 'Usage: codegraphy search [--limit ] [--offset ] ', '', - 'Find graph Nodes whose searchable fields contain text.', + 'Discover matching live source lines, cached AST Symbols, and indexed File or concept Nodes.', + 'Literal matching is case-insensitive. Add a `*` wildcard to span characters on one line or name.', + 'When a natural multi-term phrase has few literal matches, all-term path/source ranking finds relevant Files.', + 'Every Symbol match includes the File it came from. Text matches include line, column, and excerpt.', '', 'Arguments:', - ' Text to match in the shaped graph', + ' Literal source/name/path text, optionally containing `*` wildcards', '', - 'Effects: Read-only.', - 'Output: Bounded JSON Node results and page metadata.', + 'Effects: Read-only. Does not perform Indexing; source text is read live from indexed files.', + 'Output: One ranked, bounded JSON match list plus live/cached freshness provenance.', 'Options:', - ' --limit Maximum results to return (default: 100)', + ' --limit Maximum combined matches to return (default: 20)', ' --offset Zero-based result offset (default: 0)', + 'Example: codegraphy search runIndexCommand', + "Example: codegraphy search 'Indexing *workspace*'", + ].join('\n'), + map: [ + 'Usage: codegraphy map [--limit ] [--offset ] ', + '', + 'Build a personalized File map from independent task terms and typed Relationships.', + 'Files are ranked from live lexical evidence and the cached graph; selected declarations and connecting Edges explain inclusion.', + '', + 'Arguments:', + ' Task or issue text', + '', + 'Effects: Read-only. Does not perform Indexing; source text is read live from indexed files.', + 'Output: Bounded JSON Files, matched terms, declarations, typed Relationships, freshness, and completeness.', + 'Options:', + ' --limit Maximum Files to return (default: 8; range: 1-20)', + ' --offset Zero-based File offset (default: 0)', ...QUERY_PROJECTION_OPTIONS, - 'Example: codegraphy search SettingsPanel', + "Example: codegraphy map 'plugin cleanup after workspace failure'", ].join('\n'), edges: [ 'Usage: codegraphy edges [--limit ] [--offset ]', diff --git a/packages/core/src/cli/index/command.ts b/packages/core/src/cli/index/command.ts index 6d10099c0d..ed94718489 100644 --- a/packages/core/src/cli/index/command.ts +++ b/packages/core/src/cli/index/command.ts @@ -35,7 +35,7 @@ export async function runIndexCommand( options: IndexCommandOptions = {}, ): Promise { const resolvedWorkspaceRoot = resolveCodeGraphyWorkspacePath(workspacePath, dependencies.cwd()); - dependencies.writeStatus(`Indexing ${resolvedWorkspaceRoot}...`); + if (options.verbose) dependencies.writeStatus(`Indexing ${resolvedWorkspaceRoot}...`); const diagnostics: DiagnosticEventSink | undefined = options.verbose ? { emit(event: DiagnosticEvent): void { diff --git a/packages/core/src/cli/parse.ts b/packages/core/src/cli/parse.ts index afcd6956fc..5114b15e6a 100644 --- a/packages/core/src/cli/parse.ts +++ b/packages/core/src/cli/parse.ts @@ -3,9 +3,15 @@ import { isHelpCommandName } from './parseHelp'; import { isPluginCommand, parsePluginsCommand } from './parsePlugins'; import { isGraphQueryReport, parseQueryCommand } from './parseQuery'; import { parseWorkspaceCommand } from './parseWorkspace'; +import { parseSettingsCommand } from './parseSettings'; import type { CliCommand } from './parseTypes'; -export type { CliCommand, CliCommandName, PluginsCommandAction } from './parseTypes'; +export type { + CliCommand, + CliCommandName, + PluginsCommandAction, + WorkspaceSettingsCommandAction, +} from './parseTypes'; interface GlobalFlags { argv: string[]; @@ -70,7 +76,9 @@ function isKnownCommandName(name: string): boolean { || name === 'index' || name === 'plugins' || name === 'scope' + || name === 'settings' || name === 'status' + || name === 'query' || isGraphQueryReport(name); } @@ -118,6 +126,9 @@ export function parseCliCommand(argv: string[]): CliCommand { case 'scope': command = parseScopeCommand(rest); break; + case 'settings': + command = parseSettingsCommand(rest); + break; case 'filter': command = parseFilterCommand(rest); break; diff --git a/packages/core/src/cli/parsePlugins.ts b/packages/core/src/cli/parsePlugins.ts index fe7da3162d..e3fa407e02 100644 --- a/packages/core/src/cli/parsePlugins.ts +++ b/packages/core/src/cli/parsePlugins.ts @@ -8,9 +8,18 @@ export function isPluginCommand(value: string | undefined): boolean { export function parsePluginsCommand(argv: string[]): CliCommand { const [action, ...rawOperands] = argv; - const separatedOperands = rawOperands[0] === '--' ? rawOperands.slice(1) : rawOperands; - const globalFlags = separatedOperands.filter(operand => operand === '--global' || operand === '-g'); - const operands = separatedOperands.filter(operand => operand !== '--global' && operand !== '-g'); + const globalFlags: string[] = []; + const operands: string[] = []; + let optionsEnded = false; + for (const operand of rawOperands) { + if (!optionsEnded && operand === '--') { + optionsEnded = true; + } else if (!optionsEnded && (operand === '--global' || operand === '-g')) { + globalFlags.push(operand); + } else { + operands.push(operand); + } + } const [packageName, extra] = operands; if (!action || action === 'help') { diff --git a/packages/core/src/cli/parseQuery.ts b/packages/core/src/cli/parseQuery.ts index 21990bf68b..5e5596bc4f 100644 --- a/packages/core/src/cli/parseQuery.ts +++ b/packages/core/src/cli/parseQuery.ts @@ -5,12 +5,17 @@ const QUERY_COMMANDS = new Set([ 'dependencies', 'dependents', 'edges', + 'map', 'nodes', 'path', + 'query', 'search', ]); const DEFAULT_LIMIT = 100; +const DEFAULT_SEARCH_LIMIT = 20; +const DEFAULT_TASK_MAP_LIMIT = 8; +const MAX_TASK_MAP_LIMIT = 20; const DEFAULT_MAX_DEPTH = 6; const DEFAULT_MAX_PATHS = 5; @@ -22,6 +27,24 @@ interface ParsedQueryArguments { projection?: WorkspaceGraphQueryProjection; } +interface QueryBuilderInput { + command: string; + operands: string[]; + page: { limit: number; offset?: number }; + projection?: WorkspaceGraphQueryProjection; +} + +type ParsedOption = + | { type: 'limit' | 'offset'; value: number } + | { type: 'projection'; key: keyof WorkspaceGraphQueryProjection; values: string[] } + | { type: 'error'; message: string }; + +const PROJECTION_OPTION_KEYS: Record = { + '--filter': 'filterPatterns', + '--node-type': 'nodeTypes', + '--edge-type': 'edgeTypes', +}; + export function isGraphQueryReport(value: string | undefined): boolean { return value !== undefined && QUERY_COMMANDS.has(value); } @@ -36,69 +59,106 @@ function parseInteger(value: string | undefined, minimum: number): number | unde return Number.isSafeInteger(parsed) && parsed >= minimum ? parsed : undefined; } +function parsePaginationOption( + command: string, + argument: '--limit' | '--offset', + value: string | undefined, + allowPagination: boolean, +): ParsedOption { + if (!allowPagination) return { type: 'error', message: `Unknown option for ${command}: ${argument}` }; + const minimum = argument === '--limit' ? 1 : 0; + const parsed = parseInteger(value, minimum); + if (parsed !== undefined) return { type: argument === '--limit' ? 'limit' : 'offset', value: parsed }; + const requirement = argument === '--limit' ? 'a positive integer' : 'a non-negative integer'; + return { type: 'error', message: `${argument} requires ${requirement}` }; +} + +function parseProjectionOption( + argument: string, + value: string | undefined, + key: keyof WorkspaceGraphQueryProjection, +): ParsedOption { + const values = !value || value.startsWith('-') + ? [] + : value.split(',').map(item => item.trim()).filter(Boolean); + return values.length > 0 + ? { type: 'projection', key, values } + : { type: 'error', message: `${argument} requires a comma-separated list` }; +} + +function parseOption( + command: string, + argument: string, + value: string | undefined, + allowPagination: boolean, +): ParsedOption | undefined { + if (argument === '--limit' || argument === '--offset') { + return parsePaginationOption(command, argument, value, allowPagination); + } + const projectionKey = PROJECTION_OPTION_KEYS[argument]; + return projectionKey ? parseProjectionOption(argument, value, projectionKey) : undefined; +} + +function applyOption( + parsed: ParsedOption, + state: { limit: number; offset?: number; projection: WorkspaceGraphQueryProjection }, +): void { + if (parsed.type === 'limit') state.limit = parsed.value; + if (parsed.type === 'offset') state.offset = parsed.value; + if (parsed.type === 'projection') { + state.projection[parsed.key] = [...new Set([...(state.projection[parsed.key] ?? []), ...parsed.values])]; + } +} + +function completeParsedArguments( + operands: string[], + state: { limit: number; offset?: number; projection: WorkspaceGraphQueryProjection }, +): ParsedQueryArguments { + return { + operands, + limit: state.limit, + ...(state.offset !== undefined ? { offset: state.offset } : {}), + ...(Object.keys(state.projection).length > 0 ? { projection: state.projection } : {}), + }; +} + function parseArguments( command: string, argv: string[], allowPagination: boolean, + defaultLimit = DEFAULT_LIMIT, ): ParsedQueryArguments { const operands: string[] = []; - let limit = DEFAULT_LIMIT; - let offset: number | undefined; + const state: { limit: number; offset?: number; projection: WorkspaceGraphQueryProjection } = { + limit: defaultLimit, + projection: {}, + }; let optionsEnded = false; - const projection: NonNullable = {}; for (let index = 0; index < argv.length; index += 1) { const argument = argv[index]; - if (!optionsEnded && argument === '--') { - optionsEnded = true; + if (optionsEnded) { + operands.push(argument); continue; } - if (!optionsEnded && (argument === '--limit' || argument === '--offset')) { - if (!allowPagination) { - return { operands, limit, parseError: `Unknown option for ${command}: ${argument}` }; - } - const minimum = argument === '--limit' ? 1 : 0; - const value = parseInteger(argv[index + 1], minimum); - if (value === undefined) { - const requirement = argument === '--limit' ? 'a positive integer' : 'a non-negative integer'; - return { operands, limit, parseError: `${argument} requires ${requirement}` }; - } - if (argument === '--limit') limit = value; - else offset = value; - index += 1; + if (argument === '--') { + optionsEnded = true; continue; } - if (!optionsEnded && ( - argument === '--filter' - || argument === '--node-type' - || argument === '--edge-type' - )) { - const optionValue = argv[index + 1]; - const values = !optionValue || optionValue.startsWith('-') - ? [] - : optionValue.split(',').map(value => value.trim()).filter(Boolean); - if (values.length === 0) { - return { operands, limit, parseError: `${argument} requires a comma-separated list` }; - } - const key = argument === '--filter' - ? 'filterPatterns' - : argument === '--node-type' ? 'nodeTypes' : 'edgeTypes'; - projection[key] = [...new Set([...(projection[key] ?? []), ...values])]; + const option = parseOption(command, argument, argv[index + 1], allowPagination); + if (option?.type === 'error') return { operands, limit: state.limit, parseError: option.message }; + if (option) { + applyOption(option, state); index += 1; continue; } - if (!optionsEnded && argument.startsWith('-')) { - return { operands, limit, parseError: `Unknown option for ${command}: ${argument}` }; + if (argument.startsWith('-')) { + return { operands, limit: state.limit, parseError: `Unknown option for ${command}: ${argument}` }; } operands.push(argument); } - return { - operands, - limit, - ...(offset !== undefined ? { offset } : {}), - ...(Object.keys(projection).length > 0 ? { projection } : {}), - }; + return completeParsedArguments(operands, state); } function requireOperands(command: string, args: string[], count: number, usage: string): CliCommand | undefined { @@ -111,7 +171,7 @@ function query( invokedCommand: string, report: GraphQueryReport, arguments_: Record, - projection?: ParsedQueryArguments['projection'], + projection?: WorkspaceGraphQueryProjection, ): CliCommand { return { name: 'query', @@ -122,52 +182,87 @@ function query( }; } -export function parseQueryCommand(argv: string[]): CliCommand { - const [command = '', ...rawArgs] = argv; - if (!isGraphQueryReport(command)) { - return parseError(command, `Unknown query command: ${command}`); +function buildList(input: QueryBuilderInput): CliCommand { + const invalid = requireOperands(input.command, input.operands, 0, ''); + return invalid ?? query(input.command, input.command as 'nodes' | 'edges', input.page, input.projection); +} + +function buildSearch(input: QueryBuilderInput): CliCommand { + const invalid = requireOperands(input.command, input.operands, 1, ''); + if (invalid) return invalid; + if (!input.operands[0].replace(/\*/g, '').trim()) { + return parseError(input.command, 'search pattern must contain a literal character'); } + return query(input.command, 'search', { pattern: input.operands[0], ...input.page }, input.projection); +} - const acceptsPagination = command !== 'path'; - const parsed = parseArguments(command, rawArgs, acceptsPagination); - if (parsed.parseError) return parseError(command, parsed.parseError); - const { operands, limit, offset, projection } = parsed; - const page = { limit, ...(offset !== undefined ? { offset } : {}) }; - - switch (command) { - case 'nodes': - case 'edges': { - const invalid = requireOperands(command, operands, 0, ''); - return invalid ?? query(command, command, page, projection); - } - case 'search': { - const invalid = requireOperands(command, operands, 1, ''); - return invalid ?? query(command, 'nodes', { search: operands[0], ...page }, projection); - } - case 'dependencies': { - const invalid = requireOperands(command, operands, 1, ''); - return invalid ?? query(command, 'edges', { - from: operands[0], expandFileSelectors: true, projectFileEndpoints: true, ...page, - }, projection); - } - case 'dependents': { - const invalid = requireOperands(command, operands, 1, ''); - return invalid ?? query(command, 'edges', { - to: operands[0], expandFileSelectors: true, projectFileEndpoints: true, ...page, - }, projection); - } - case 'path': { - const invalid = requireOperands(command, operands, 2, ' '); - return invalid ?? query(command, 'paths', { - from: operands[0], - to: operands[1], - maxDepth: DEFAULT_MAX_DEPTH, - maxPaths: DEFAULT_MAX_PATHS, - expandFileSelectors: true, - projectFileEndpoints: true, - }, projection); - } +function buildTaskMap(input: QueryBuilderInput): CliCommand { + const invalid = requireOperands(input.command, input.operands, 1, ''); + if (invalid) return invalid; + if (input.page.limit > MAX_TASK_MAP_LIMIT) { + return parseError(input.command, `map --limit must be between 1 and ${MAX_TASK_MAP_LIMIT}`); } + return query(input.command, 'task-map', { query: input.operands[0], ...input.page }, input.projection); +} + +function buildOverview(input: QueryBuilderInput): CliCommand { + const invalid = requireOperands(input.command, input.operands, 1, ''); + return invalid ?? query(input.command, 'overview', { target: input.operands[0] }, input.projection); +} + +function buildConnection(input: QueryBuilderInput, endpoint: 'from' | 'to'): CliCommand { + const invalid = requireOperands(input.command, input.operands, 1, ''); + return invalid ?? query(input.command, 'edges', { + [endpoint]: input.operands[0], + expandFileSelectors: true, + projectFileEndpoints: true, + ...input.page, + }, input.projection); +} - return parseError(command, `Unknown query command: ${command}`); +function buildPath(input: QueryBuilderInput): CliCommand { + const invalid = requireOperands(input.command, input.operands, 2, ' '); + return invalid ?? query(input.command, 'paths', { + from: input.operands[0], + to: input.operands[1], + maxDepth: DEFAULT_MAX_DEPTH, + maxPaths: DEFAULT_MAX_PATHS, + expandFileSelectors: true, + projectFileEndpoints: true, + }, input.projection); +} + +const QUERY_BUILDERS: Record CliCommand> = { + nodes: buildList, + edges: buildList, + search: buildSearch, + map: buildTaskMap, + query: buildOverview, + dependencies: input => buildConnection(input, 'from'), + dependents: input => buildConnection(input, 'to'), + path: buildPath, +}; + +export function parseQueryCommand(argv: string[]): CliCommand { + const [command = '', ...rawArgs] = argv; + const builder = QUERY_BUILDERS[command]; + if (!builder) return parseError(command, `Unknown query command: ${command}`); + const parsed = parseArguments( + command, + rawArgs, + command !== 'path' && command !== 'query', + command === 'search' + ? DEFAULT_SEARCH_LIMIT + : command === 'map' ? DEFAULT_TASK_MAP_LIMIT : DEFAULT_LIMIT, + ); + if (parsed.parseError) return parseError(command, parsed.parseError); + return builder({ + command, + operands: parsed.operands, + page: { + limit: parsed.limit, + ...(parsed.offset !== undefined ? { offset: parsed.offset } : {}), + }, + ...(parsed.projection ? { projection: parsed.projection } : {}), + }); } diff --git a/packages/core/src/cli/parseSettings.ts b/packages/core/src/cli/parseSettings.ts new file mode 100644 index 0000000000..0feea00ed9 --- /dev/null +++ b/packages/core/src/cli/parseSettings.ts @@ -0,0 +1,55 @@ +import type { CliCommand } from './parseTypes'; + +const WORKSPACE_SETTING_KEYS = new Set([ + 'maxFiles', + 'include', + 'respectGitignore', + 'filterPatterns', + 'disabledCustomFilterPatterns', + 'nodeVisibility', + 'edgeVisibility', + 'plugins', + 'interfaces', + 'pluginData', +]); + +function parseError(message: string): CliCommand { + return { name: 'settings', parseError: message }; +} + +function readOperands(argv: string[]): string[] { + const terminator = argv.indexOf('--'); + return terminator < 0 ? argv : [...argv.slice(0, terminator), ...argv.slice(terminator + 1)]; +} + +function validateKey(key: string | undefined): CliCommand | undefined { + if (!key) return parseError('settings command requires a workspace setting key'); + return WORKSPACE_SETTING_KEYS.has(key) + ? undefined + : parseError(`Unknown workspace setting: ${key}`); +} + +export function parseSettingsCommand(argv: string[]): CliCommand { + const [action, key, value, extra] = readOperands(argv); + if (!action) return { name: 'settings' }; + if (!['get', 'set', 'unset'].includes(action)) return parseError(`Unknown settings action: ${action}`); + const invalidKey = validateKey(key); + if (invalidKey) return invalidKey; + if (action === 'get' || action === 'unset') { + return extra || value + ? parseError(`Unexpected argument for settings ${action}: ${value ?? extra}`) + : { name: 'settings', settingsAction: action, settingsKey: key }; + } + if (value === undefined) return parseError('settings set requires a JSON value'); + if (extra) return parseError(`Unexpected argument for settings set: ${extra}`); + try { + return { + name: 'settings', + settingsAction: 'set', + settingsKey: key, + settingsValue: JSON.parse(value) as unknown, + }; + } catch { + return parseError(`settings set value must be valid JSON: ${value}`); + } +} diff --git a/packages/core/src/cli/parseTypes.ts b/packages/core/src/cli/parseTypes.ts index b3423c719e..742d01b82a 100644 --- a/packages/core/src/cli/parseTypes.ts +++ b/packages/core/src/cli/parseTypes.ts @@ -11,9 +11,11 @@ export type CliCommandName = | 'plugins' | 'query' | 'scope' + | 'settings' | 'status' | 'version'; export type PluginsCommandAction = 'disable' | 'enable' | 'help' | 'inherit' | 'link' | 'list' | 'register'; +export type WorkspaceSettingsCommandAction = 'get' | 'set' | 'unset'; export interface CliCommand { name: CliCommandName; @@ -27,6 +29,9 @@ export interface CliCommand { parseError?: string; projection?: WorkspaceGraphQueryProjection; report?: GraphQueryReport; + settingsAction?: WorkspaceSettingsCommandAction; + settingsKey?: string; + settingsValue?: unknown; verbose?: boolean; workspacePath?: string; } diff --git a/packages/core/src/cli/plugins/list.ts b/packages/core/src/cli/plugins/list.ts index b78709eb3d..ed59e4abf8 100644 --- a/packages/core/src/cli/plugins/list.ts +++ b/packages/core/src/cli/plugins/list.ts @@ -1,5 +1,6 @@ import { readCodeGraphyWorkspaceSettingsOrInitial } from '../../workspace/settings'; import { createPluginActivityState } from '../../plugins/activityState/model'; +import type { CodeGraphyInstalledPluginRecord } from '../../plugins/installedCache'; import type { CommandExecutionResult } from '../command'; import type { CliCommand } from '../parse'; import type { PluginsCommandDependencies } from './dependencies'; @@ -9,56 +10,72 @@ import { } from './installed'; import { resolveWorkspaceRoot } from './workspace'; +type PluginListState = 'active' | 'disabled' | 'enabled-unavailable'; + +function readPluginState( + pluginId: string, + activity: ReturnType, +): PluginListState { + if (activity.activePluginIds.has(pluginId)) return 'active'; + if (activity.enabledPluginIds.has(pluginId)) return 'enabled-unavailable'; + return 'disabled'; +} + +function createRegisteredPluginOutput( + plugin: CodeGraphyInstalledPluginRecord, + workspaceActivation: Map, + activity: ReturnType, +): Record { + return { + id: plugin.id, + package: plugin.package, + host: plugin.host, + globallyEnabled: plugin.globallyEnabled, + workspaceActivation: workspaceActivation.get(plugin.id) ?? 'inherit', + state: readPluginState(plugin.id, activity), + }; +} + export function runListCommand( command: CliCommand, dependencies: PluginsCommandDependencies, ): CommandExecutionResult { const workspaceRoot = resolveWorkspaceRoot(command.workspacePath, dependencies); + const settings = readCodeGraphyWorkspaceSettingsOrInitial(workspaceRoot); const registeredPlugins = listRegisteredPluginsWithBundledMarkdown( dependencies.readInstalledPluginCache({ homeDir: dependencies.homeDir, }), ); const activity = createPluginActivityState({ - settings: readCodeGraphyWorkspaceSettingsOrInitial(workspaceRoot), + settings, installedPlugins: registeredPlugins, }); - const activePluginIds = [...activity.activePluginIds]; - const enabledPluginIds = [...activity.enabledPluginIds]; - const enabledPluginIdSet = new Set(enabledPluginIds); - const unavailablePluginIds = enabledPluginIds.filter(pluginId => !activity.activePluginIds.has(pluginId)); - const disabledPlugins = registeredPlugins.filter(plugin => - !enabledPluginIdSet.has(getRegisteredPluginId(plugin)) + const workspaceActivation = new Map( + settings.plugins.map(plugin => [plugin.id, plugin.activation] as const), ); - const disabledPluginIds = [...new Set(disabledPlugins.map(getRegisteredPluginId))]; - - const lines = [ - `CodeGraphy plugins for ${workspaceRoot}`, - '', - 'Enabled in workspace:', - ...( - activePluginIds.length > 0 - ? activePluginIds.map((pluginId, index) => `${index + 1}. ${pluginId}`) - : ['none'] - ), - '', - 'Enabled but unavailable:', - ...( - unavailablePluginIds.length > 0 - ? unavailablePluginIds.map(pluginId => `- ${pluginId}`) - : ['none'] - ), - '', - 'Registered but disabled:', - ...( - disabledPluginIds.length > 0 - ? disabledPluginIds.map(pluginId => `- ${pluginId}`) - : ['none'] - ), - ]; + const registeredPluginIds = new Set(registeredPlugins.map(getRegisteredPluginId)); + const unavailablePlugins = [...activity.enabledPluginIds] + .filter(pluginId => !registeredPluginIds.has(pluginId)) + .map(pluginId => ({ + id: pluginId, + workspaceActivation: workspaceActivation.get(pluginId) ?? 'inherit', + state: readPluginState(pluginId, activity), + })); return { exitCode: 0, - output: lines.join('\n'), + output: JSON.stringify({ + workspaceRoot, + plugins: [ + ...registeredPlugins.map(plugin => createRegisteredPluginOutput( + plugin, + workspaceActivation, + activity, + )), + ...unavailablePlugins, + ], + warnings: activity.warnings, + }), }; } diff --git a/packages/core/src/cli/query/command.ts b/packages/core/src/cli/query/command.ts index a71f71bb9e..1d7ee6618b 100644 --- a/packages/core/src/cli/query/command.ts +++ b/packages/core/src/cli/query/command.ts @@ -23,7 +23,7 @@ const DEFAULT_DEPENDENCIES: QueryCommandDependencies = { query: requestWorkspaceGraphQuery, }; -const PATH_ARGUMENTS = ['from', 'to', 'filePath', 'relatedFrom', 'relatedTo'] as const; +const PATH_ARGUMENTS = ['from', 'to', 'target', 'filePath', 'relatedFrom', 'relatedTo'] as const; function canonicalizeExistingPathPrefix(inputPath: string): string { let existingPath = inputPath; @@ -62,7 +62,7 @@ function normalizeWorkspaceSelector(selector: string, workspaceRoot: string): st return normalized; } -function normalizeQueryArguments( +export function normalizeQueryArguments( queryArguments: Record, workspaceRoot: string, ): Record { diff --git a/packages/core/src/cli/result/parser.ts b/packages/core/src/cli/result/parser.ts index 461ce776b1..3258ea38be 100644 --- a/packages/core/src/cli/result/parser.ts +++ b/packages/core/src/cli/result/parser.ts @@ -38,9 +38,11 @@ export function readCliError(output: unknown): CliError { ? nested.message : typeof parsed.message === 'string' ? parsed.message : 'Command failed.'; const action = typeof nested?.action === 'string' ? nested.action : parsed.action; + const details = nested?.details ?? parsed.details; return { code, message, ...(typeof action === 'string' ? { action } : {}), + ...(details !== undefined ? { details } : {}), }; } diff --git a/packages/core/src/cli/run.ts b/packages/core/src/cli/run.ts index 10d01f7f48..bdc6a6859e 100644 --- a/packages/core/src/cli/run.ts +++ b/packages/core/src/cli/run.ts @@ -2,6 +2,7 @@ import { runCliCommand, type CommandExecutionResult } from './command'; import { parseCliCommand } from './parse'; import type { CliCommand } from './parseTypes'; import { formatCliResult } from './result/serializer'; +import { WorkspaceSettingsError } from '../workspace/settings'; export interface RunCliDependencies { runCommand(command: CliCommand): Promise; @@ -27,10 +28,17 @@ export async function runCli( } catch (error) { result = { exitCode: 1, - output: JSON.stringify({ - error: 'command_failed', - message: error instanceof Error ? error.message : String(error), - }), + output: JSON.stringify(error instanceof WorkspaceSettingsError + ? { + error: 'invalid_workspace_settings', + message: error.reason, + action: 'Repair `.codegraphy/settings.json`, then retry.', + details: { path: error.settingsPath }, + } + : { + error: 'command_failed', + message: error instanceof Error ? error.message : String(error), + }), }; } diff --git a/packages/core/src/cli/scope/command.ts b/packages/core/src/cli/scope/command.ts index c5b7974d8d..1850abe00c 100644 --- a/packages/core/src/cli/scope/command.ts +++ b/packages/core/src/cli/scope/command.ts @@ -69,9 +69,15 @@ function createNodeVisibilityUpdates(type: string, enabled: boolean): Record; + nodeTypes: ReadonlySet; +} + function createScopeOutput( settings: ReturnType, workspaceRoot: string, + selection?: ScopeOutputSelection, ): string { const status = readCodeGraphyWorkspaceStatus(workspaceRoot); const snapshot = status.hasGraphCache @@ -114,16 +120,21 @@ function createScopeOutput( )) ); return JSON.stringify({ - nodes: nodeTypes.map(type => ({ - type, - enabled: nodeVisibility[type] ?? definitions.get(type)?.defaultVisible ?? false, - available: availableNodeTypes.has(type), - })), - edges: edgeTypes.map(type => ({ - type, - enabled: edgeVisibility[type] ?? true, - available: observedEdgeTypes.has(type), - })), + complete: selection === undefined, + nodes: nodeTypes + .filter(type => selection === undefined || selection.nodeTypes.has(type)) + .map(type => ({ + type, + enabled: nodeVisibility[type] ?? definitions.get(type)?.defaultVisible ?? false, + available: availableNodeTypes.has(type), + })), + edges: edgeTypes + .filter(type => selection === undefined || selection.edgeTypes.has(type)) + .map(type => ({ + type, + enabled: edgeVisibility[type] ?? true, + available: observedEdgeTypes.has(type), + })), indexRequired, ...(indexRequired ? { action: 'Run `codegraphy index` to hydrate the required symbol analysis.' } @@ -140,6 +151,7 @@ export function runScopeCommand( const kind = command.arguments?.kind; const type = command.arguments?.type; const enabled = command.arguments?.enabled; + let selection: ScopeOutputSelection | undefined; if ((kind === 'node' || kind === 'edge') && typeof type === 'string' && typeof enabled === 'boolean') { const updates = kind === 'node' @@ -151,7 +163,11 @@ export function runScopeCommand( updates, ); settings = readCodeGraphyWorkspaceSettingsOrInitial(workspaceRoot); + selection = { + nodeTypes: new Set(kind === 'node' ? Object.keys(updates) : []), + edgeTypes: new Set(kind === 'edge' ? Object.keys(updates) : []), + }; } - return { exitCode: 0, output: createScopeOutput(settings, workspaceRoot) }; + return { exitCode: 0, output: createScopeOutput(settings, workspaceRoot, selection) }; } diff --git a/packages/core/src/cli/settings/command.ts b/packages/core/src/cli/settings/command.ts new file mode 100644 index 0000000000..11687be10e --- /dev/null +++ b/packages/core/src/cli/settings/command.ts @@ -0,0 +1,56 @@ +import { getWorkspaceSettingsPath } from '../../workspace/paths'; +import { resolveCodeGraphyWorkspacePath } from '../../workspace/requestPaths'; +import { + patchCodeGraphyWorkspaceSettings, + readCodeGraphyWorkspaceSettingsOrInitial, + removeCodeGraphyWorkspaceSetting, +} from '../../workspace/settings'; +import { readCodeGraphyWorkspaceStatus } from '../../workspace/status'; +import type { CommandExecutionResult } from '../command'; +import type { CliCommand } from '../parseTypes'; + +function settingValue(settings: object, key: string): unknown { + return Reflect.get(settings, key) as unknown; +} + +export function runSettingsCommand(command: CliCommand): CommandExecutionResult { + const workspaceRoot = resolveCodeGraphyWorkspacePath(command.workspacePath, process.cwd()); + const settingsPath = getWorkspaceSettingsPath(workspaceRoot); + const current = readCodeGraphyWorkspaceSettingsOrInitial(workspaceRoot); + const key = command.settingsKey; + + if (!command.settingsAction) { + return { + exitCode: 0, + output: JSON.stringify({ workspaceRoot, settingsPath, settings: current }), + }; + } + if (!key) throw new Error('Workspace setting key is required'); + const previous = settingValue(current, key); + if (command.settingsAction === 'get') { + return { + exitCode: 0, + output: JSON.stringify({ workspaceRoot, settingsPath, key, value: previous }), + }; + } + + if (command.settingsAction === 'set') { + patchCodeGraphyWorkspaceSettings(workspaceRoot, { [key]: command.settingsValue }); + } else { + removeCodeGraphyWorkspaceSetting(workspaceRoot, key); + } + const updated = readCodeGraphyWorkspaceSettingsOrInitial(workspaceRoot); + const status = readCodeGraphyWorkspaceStatus(workspaceRoot); + return { + exitCode: 0, + output: JSON.stringify({ + workspaceRoot, + settingsPath, + key, + previous, + value: settingValue(updated, key), + indexRequired: status.state !== 'fresh', + ...(status.state === 'fresh' ? {} : { action: 'Run `codegraphy index` before querying cached AST or Relationships.' }), + }), + }; +} diff --git a/packages/core/src/graphQuery/data.ts b/packages/core/src/graphQuery/data.ts index 2f67c048d2..7e42157ff9 100644 --- a/packages/core/src/graphQuery/data.ts +++ b/packages/core/src/graphQuery/data.ts @@ -1,8 +1,17 @@ import type { IAnalysisRelation, IAnalysisSymbol } from '@codegraphy-dev/plugin-api'; import type { IGraphData } from '../graph/contracts'; +export interface GraphQuerySourceText { + files: readonly { filePath: string; content: string }[]; + filesScanned: number; + filesSkipped: number; + hasChangedFiles?: boolean; +} + export interface GraphQueryData { graphData: IGraphData; symbols?: readonly IAnalysisSymbol[]; relations?: readonly IAnalysisRelation[]; + sourceText?: GraphQuerySourceText; + cacheState?: 'fresh' | 'stale'; } diff --git a/packages/core/src/graphQuery/execute.ts b/packages/core/src/graphQuery/execute.ts index fb20192266..92654fa320 100644 --- a/packages/core/src/graphQuery/execute.ts +++ b/packages/core/src/graphQuery/execute.ts @@ -2,15 +2,21 @@ import type { GraphQueryData } from './data'; import type { GraphQueryConfig, GraphQueryConnectionConfig, + GraphQueryOverviewConfig, GraphQueryPathConfig, GraphQueryRequest, GraphQueryResult, + GraphQuerySearchConfig, GraphQuerySymbolsConfig, + GraphQueryTaskMapConfig, } from './model'; +import { inspectGraphTarget } from './overview'; import { findGraphPaths } from './paths'; import { listGraphEdges, listGraphNodes } from './reports'; import { listGraphRelationships } from './relationships'; +import { searchGraph } from './search'; import { listGraphSymbols } from './symbols'; +import { mapGraphTask } from './taskMap'; import { deriveScopedGraphQueryData } from './visible'; type GraphQueryHandler = ( @@ -24,6 +30,9 @@ type GraphQueryHandlers = { relationships: GraphQueryHandler; symbols: GraphQueryHandler; paths: GraphQueryHandler; + search: GraphQueryHandler; + 'task-map': GraphQueryHandler; + overview: GraphQueryHandler; }; const GRAPH_QUERY_HANDLERS: GraphQueryHandlers = { @@ -32,6 +41,12 @@ const GRAPH_QUERY_HANDLERS: GraphQueryHandlers = { relationships: (data, args) => listGraphRelationships(data, args), symbols: (data, args) => listGraphSymbols(data, args), paths: (data, args) => findGraphPaths(deriveScopedGraphQueryData(data.graphData, args), args), + search: (data, args) => searchGraph(data, args), + 'task-map': (data, args) => mapGraphTask({ + ...data, + graphData: deriveScopedGraphQueryData(data.graphData, args), + }, args), + overview: (data, args) => inspectGraphTarget(data, args), }; export function executeGraphQuery( diff --git a/packages/core/src/graphQuery/index.ts b/packages/core/src/graphQuery/index.ts index a4fde321aa..d35f638359 100644 --- a/packages/core/src/graphQuery/index.ts +++ b/packages/core/src/graphQuery/index.ts @@ -7,6 +7,8 @@ export type { GraphQueryFilterOperator, GraphQueryNodeReport, GraphQueryNodeReportItem, + GraphQueryOverviewConfig, + GraphQueryOverviewReport, GraphQueryPage, GraphQueryPathConfig, GraphQueryPathReport, @@ -18,15 +20,24 @@ export type { GraphQueryRelationshipReportItem, GraphQueryRelationshipSymbol, GraphQueryScope, + GraphQuerySearchConfig, + GraphQuerySearchMatch, + GraphQuerySearchReport, GraphQuerySort, GraphQueryResult, GraphQuerySymbolReport, GraphQuerySymbolReportItem, GraphQuerySymbolsConfig, + GraphQueryTaskMapConfig, + GraphQueryTaskMapFile, + GraphQueryTaskMapReport, } from './model'; export type { GraphQueryData } from './data'; export { executeGraphQuery } from './execute'; +export { inspectGraphTarget } from './overview'; export { findGraphPaths } from './paths'; export { listGraphEdges, listGraphNodes } from './reports'; export { listGraphRelationships } from './relationships'; +export { searchGraph } from './search'; export { listGraphSymbols } from './symbols'; +export { mapGraphTask } from './taskMap'; diff --git a/packages/core/src/graphQuery/model.ts b/packages/core/src/graphQuery/model.ts index f5ad066c8c..c9668e5290 100644 --- a/packages/core/src/graphQuery/model.ts +++ b/packages/core/src/graphQuery/model.ts @@ -61,6 +61,18 @@ export interface GraphQueryPathConfig extends GraphQueryConfig { projectFileEndpoints?: boolean; } +export interface GraphQuerySearchConfig extends GraphQueryConfig { + pattern: string; +} + +export interface GraphQueryTaskMapConfig extends GraphQueryConfig { + query: string; +} + +export interface GraphQueryOverviewConfig { + target: string; +} + export interface GraphQueryPage { offset: number; limit: number; @@ -153,23 +165,107 @@ export interface GraphQueryPathReport { }; } +export type GraphQuerySearchMatch = + | { type: 'node'; node: GraphQueryNodeReportItem } + | { type: 'symbol'; symbol: GraphQuerySymbolReportItem } + | { + type: 'text'; + filePath: string; + line: number; + column: number; + excerpt: string; + }; + +export interface GraphQuerySearchReport { + pattern: string; + matches: GraphQuerySearchMatch[]; + page: GraphQueryPage; + sources: { + text: { + freshness: 'live'; + filesScanned: number; + filesSkipped: number; + }; + symbols: { + freshness: 'cached'; + cacheState: 'fresh' | 'stale'; + }; + }; +} + +export interface GraphQueryTaskMapFile { + path: string; + nodeType: 'file'; + matchedTerms: string[]; + symbols: { id?: string; name: string; kind?: string }[]; +} + +export interface GraphQueryTaskMapReport { + query: string; + terms: string[]; + files: GraphQueryTaskMapFile[]; + relationships: GraphQueryEdgeReportItem[]; + page: GraphQueryPage; + limits: { + relationships: number; + complete: boolean; + }; + sources: { + text: { + freshness: 'live'; + filesScanned: number; + filesSkipped: number; + }; + graph: { + freshness: 'cached'; + cacheState: 'fresh' | 'stale'; + }; + }; +} + +export interface GraphQueryOverviewReport { + target: GraphQueryNodeReportItem; + declaredSymbols: GraphQuerySymbolReport; + outgoing: GraphQueryEdgeReport; + incoming: GraphQueryEdgeReport; + limits: { + declaredSymbols: number; + relationshipsPerDirection: number; + }; +} + +export interface GraphQueryTargetNotFoundReport { + error: 'query_target_not_found'; + message: string; +} + export type GraphQueryReport = | 'nodes' | 'edges' | 'relationships' | 'symbols' - | 'paths'; + | 'paths' + | 'search' + | 'task-map' + | 'overview'; export type GraphQueryRequest = | { report: 'nodes'; arguments?: GraphQueryConfig } | { report: 'edges'; arguments?: GraphQueryConnectionConfig } | { report: 'relationships'; arguments?: GraphQueryConnectionConfig } | { report: 'symbols'; arguments?: GraphQuerySymbolsConfig } - | { report: 'paths'; arguments: GraphQueryPathConfig }; + | { report: 'paths'; arguments: GraphQueryPathConfig } + | { report: 'search'; arguments: GraphQuerySearchConfig } + | { report: 'task-map'; arguments: GraphQueryTaskMapConfig } + | { report: 'overview'; arguments: GraphQueryOverviewConfig }; export type GraphQueryResult = | GraphQueryNodeReport | GraphQueryEdgeReport | GraphQueryRelationshipReport | GraphQuerySymbolReport - | GraphQueryPathReport; + | GraphQueryPathReport + | GraphQuerySearchReport + | GraphQueryTaskMapReport + | GraphQueryOverviewReport + | GraphQueryTargetNotFoundReport; diff --git a/packages/core/src/graphQuery/overview.ts b/packages/core/src/graphQuery/overview.ts new file mode 100644 index 0000000000..dbb7797d7d --- /dev/null +++ b/packages/core/src/graphQuery/overview.ts @@ -0,0 +1,125 @@ +import type { IAnalysisSymbol } from '@codegraphy-dev/plugin-api'; +import type { NodeType } from '../graph/contracts'; +import { getNodeType } from '../visibleGraph/model'; +import type { GraphQueryData } from './data'; +import type { + GraphQueryNodeReportItem, + GraphQueryOverviewConfig, + GraphQueryOverviewReport, + GraphQuerySymbolReport, + GraphQueryTargetNotFoundReport, +} from './model'; +import { toNodeReportItem } from './nodeReport'; +import { listGraphEdges } from './reports'; +import { listGraphSymbols } from './symbols'; +import { toSymbolReportBase } from './symbols/metadata'; + +const DECLARED_SYMBOL_LIMIT = 25; +const RELATIONSHIP_LIMIT = 25; + +function symbolTarget(symbol: IAnalysisSymbol): GraphQueryNodeReportItem { + return { + path: symbol.id, + nodeType: `symbol:${symbol.kind ?? 'unknown'}` as NodeType, + symbol: { + ...toSymbolReportBase(symbol), + id: symbol.id, + kind: symbol.kind ?? 'unknown', + filePath: symbol.filePath, + }, + }; +} + +function resolveTarget(data: GraphQueryData, selector: string): GraphQueryNodeReportItem | undefined { + const node = data.graphData.nodes.find(candidate => candidate.id === selector); + if (node) return toNodeReportItem(node); + const symbol = data.symbols?.find(candidate => candidate.id === selector); + return symbol ? symbolTarget(symbol) : undefined; +} + +function declarationKindRank(kind: string | undefined): number { + const ranks: Record = { + function: 0, + method: 0, + class: 1, + interface: 1, + type: 1, + enum: 1, + constant: 3, + variable: 3, + }; + return kind ? ranks[kind] ?? 2 : 2; +} + +function listOverviewSymbols(data: GraphQueryData, filePath: string): GraphQuerySymbolReport { + const complete = listGraphSymbols(data, { + filePath, + limit: Math.max(1, data.symbols?.length ?? 0), + }).symbols.sort((left, right) => ( + declarationKindRank(left.kind) - declarationKindRank(right.kind) + || left.name.localeCompare(right.name) + || (left.id ?? '').localeCompare(right.id ?? '') + )); + const symbols = complete.slice(0, DECLARED_SYMBOL_LIMIT); + return { + symbols, + page: { + offset: 0, + limit: DECLARED_SYMBOL_LIMIT, + returned: symbols.length, + total: complete.length, + nextOffset: complete.length > DECLARED_SYMBOL_LIMIT ? DECLARED_SYMBOL_LIMIT : null, + }, + }; +} + +export function inspectGraphTarget( + data: GraphQueryData, + config: GraphQueryOverviewConfig, +): GraphQueryOverviewReport | GraphQueryTargetNotFoundReport { + const target = resolveTarget(data, config.target); + if (!target) { + return { + error: 'query_target_not_found', + message: `No indexed Node or Symbol has the exact id: ${config.target}`, + }; + } + + const relationshipData = { + ...data, + graphData: { + nodes: data.graphData.nodes, + edges: data.graphData.edges.filter(edge => edge.kind !== 'contains'), + }, + }; + const completeScope = { + nodes: Object.fromEntries(data.graphData.nodes.map(node => [getNodeType(node), true])), + edges: Object.fromEntries(data.graphData.edges.map(edge => [edge.kind, true])), + }; + const filePath = target.symbol?.filePath ?? target.path; + + return { + target, + declaredSymbols: target.symbol + ? listGraphSymbols(data, { filePath: '__symbol-target__', limit: DECLARED_SYMBOL_LIMIT }) + : listOverviewSymbols(data, filePath), + outgoing: listGraphEdges(relationshipData.graphData, { + from: target.path, + scope: completeScope, + expandFileSelectors: !target.symbol, + projectFileEndpoints: !target.symbol, + limit: RELATIONSHIP_LIMIT, + }), + incoming: listGraphEdges(relationshipData.graphData, { + to: target.path, + scope: completeScope, + expandFileSelectors: !target.symbol, + projectFileEndpoints: !target.symbol, + limit: RELATIONSHIP_LIMIT, + }), + limits: { + declaredSymbols: DECLARED_SYMBOL_LIMIT, + relationshipsPerDirection: RELATIONSHIP_LIMIT, + }, + }; +} diff --git a/packages/core/src/graphQuery/paths.ts b/packages/core/src/graphQuery/paths.ts index 11d54685e7..af8bae2def 100644 --- a/packages/core/src/graphQuery/paths.ts +++ b/packages/core/src/graphQuery/paths.ts @@ -21,14 +21,29 @@ interface PathCollection { truncated: boolean; } +function exactSelectorFirst(ids: readonly string[], selector: string): string[] { + return [...ids].sort((left, right) => ( + Number(right === selector) - Number(left === selector) + || left.localeCompare(right) + )); +} + function collectExpandedPaths( graphData: IGraphData, config: GraphQueryPathConfig, maxDepth: number, maxPaths: number, ): PathCollection { - const fromIds = resolveSelectorNodeIds(graphData, config.from, config.expandFileSelectors === true); - const toIds = resolveSelectorNodeIds(graphData, config.to, config.expandFileSelectors === true); + const fromIds = exactSelectorFirst( + resolveSelectorNodeIds(graphData, config.from, config.expandFileSelectors === true), + config.from, + ); + const toIds = exactSelectorFirst( + resolveSelectorNodeIds(graphData, config.to, config.expandFileSelectors === true), + config.to, + ); + const fromIsFile = isFileSelector(graphData, config.from); + const toIsFile = isFileSelector(graphData, config.to); const paths: string[][] = []; let truncated = false; @@ -42,6 +57,11 @@ function collectExpandedPaths( ); paths.push(...result.paths); truncated ||= result.truncated; + const reachedExactFileEndpoint = result.paths.length > 0 && ( + (fromIsFile && from === config.from) + || (toIsFile && to === config.to) + ); + if (reachedExactFileEndpoint) return { paths, truncated }; if (paths.length >= maxPaths) return { paths: paths.slice(0, maxPaths), truncated: true }; } } diff --git a/packages/core/src/graphQuery/search.ts b/packages/core/src/graphQuery/search.ts new file mode 100644 index 0000000000..d89c5b1760 --- /dev/null +++ b/packages/core/src/graphQuery/search.ts @@ -0,0 +1,187 @@ +import type { IAnalysisSymbol } from '@codegraphy-dev/plugin-api'; +import type { IGraphNode } from '../graph/contracts'; +import type { GraphQueryData } from './data'; +import type { + GraphQuerySearchConfig, + GraphQuerySearchMatch, + GraphQuerySearchReport, + GraphQuerySymbolReportItem, +} from './model'; +import { toNodeReportItem } from './nodeReport'; +import { paginate } from './pagination'; +import { toSymbolReportBase } from './symbols/metadata'; +import { rankSearchDocuments } from './search/ranking'; + +const MAX_EXCERPT_LENGTH = 240; +const SOURCE_FILE_EXTENSION = /\.(?:[cm]?[jt]sx?|py|go|rs|java|kt|kts|swift|dart|cs|c|cc|cpp|cxx|h|hh|hpp|hxx|m|mm|php|rb|lua|scala|sc|hs|lhs|pas|pp)$/iu; + +interface RankedMatch { + match: GraphQuerySearchMatch; + rank: number; + sortKey: string; +} + +interface PatternMatcher { + exact(value: string): boolean; + find(value: string): number; + matches(value: string): boolean; +} + +function escapeRegularExpression(value: string): string { + return value.replace(/[.+?^${}()|[\]\\]/g, '\\$&'); +} + +function createPatternMatcher(pattern: string): PatternMatcher { + const normalizedPattern = pattern.toLocaleLowerCase(); + if (!pattern.includes('*')) { + return { + exact: value => value.toLocaleLowerCase() === normalizedPattern, + find: value => value.toLocaleLowerCase().indexOf(normalizedPattern), + matches: value => value.toLocaleLowerCase().includes(normalizedPattern), + }; + } + + const expression = pattern.split('*').map(escapeRegularExpression).join('.*'); + const regex = new RegExp(expression, 'iu'); + return { + exact: value => new RegExp(`^(?:${expression})$`, 'iu').test(value), + find: value => regex.exec(value)?.index ?? -1, + matches: value => regex.test(value), + }; +} + +function symbolReportItem(symbol: IAnalysisSymbol): GraphQuerySymbolReportItem { + return { + ...toSymbolReportBase(symbol), + filePath: symbol.filePath, + }; +} + +function nodeMatch(node: IGraphNode, matcher: PatternMatcher): RankedMatch | undefined { + if (node.symbol || (!matcher.matches(node.id) && !matcher.matches(node.label))) { + return undefined; + } + const exact = matcher.exact(node.id) || matcher.exact(node.label); + return { + match: { type: 'node', node: toNodeReportItem(node) }, + rank: exact ? 1 : 3, + sortKey: `${node.id}\u0000${node.label}`, + }; +} + +function symbolMatch(symbol: IAnalysisSymbol, matcher: PatternMatcher): RankedMatch | undefined { + if (!matcher.matches(symbol.name)) { + return undefined; + } + return { + match: { type: 'symbol', symbol: symbolReportItem(symbol) }, + rank: matcher.exact(symbol.name) ? 0 : 2, + sortKey: `${symbol.filePath}\u0000${symbol.name}\u0000${symbol.kind ?? ''}\u0000${symbol.id}`, + }; +} + +function readSearchTerms(pattern: string): string[] { + return pattern.toLocaleLowerCase().split(/[^\p{L}\p{N}_]+/u).flatMap((term) => { + if (term.length < 3) return []; + return term.endsWith('ing') && term.length > 5 ? [term, term.slice(0, -3)] : [term]; + }); +} + +function sourceMatchRank(filePath: string, pattern: string): number { + const normalizedPath = filePath.toLocaleLowerCase(); + const pathTerms = normalizedPath.split(/[^\p{L}\p{N}_]+/u); + const searchTerms = readSearchTerms(pattern); + if (searchTerms.some(term => pathTerms.includes(term))) return 3; + if (searchTerms.some(term => normalizedPath.includes(term))) return 4; + return SOURCE_FILE_EXTENSION.test(filePath) ? 5 : 6; +} + +function createExcerpt(lineText: string, columnIndex: number): string { + if (lineText.length <= MAX_EXCERPT_LENGTH) return lineText; + const start = Math.max(0, Math.min(columnIndex - 80, lineText.length - MAX_EXCERPT_LENGTH)); + const prefix = start > 0 ? '…' : ''; + const contentLength = MAX_EXCERPT_LENGTH - prefix.length - 1; + const end = Math.min(lineText.length, start + contentLength); + const suffix = end < lineText.length ? '…' : ''; + return `${prefix}${lineText.slice(start, end)}${suffix}`; +} + +function sourceMatches(data: GraphQueryData, pattern: string, matcher: PatternMatcher): RankedMatch[] { + return (data.sourceText?.files ?? []).flatMap(({ filePath, content }) => ( + content.split(/\r?\n/u).flatMap((lineText, lineIndex) => { + const columnIndex = matcher.find(lineText); + if (columnIndex < 0) return []; + return [{ + match: { + type: 'text' as const, + filePath, + line: lineIndex + 1, + column: columnIndex + 1, + excerpt: createExcerpt(lineText, columnIndex), + }, + rank: sourceMatchRank(filePath, pattern), + sortKey: `${filePath}\u0000${String(lineIndex).padStart(10, '0')}\u0000${String(columnIndex).padStart(10, '0')}`, + }]; + }) + )); +} + +function phraseFallbackMatches( + data: GraphQueryData, + pattern: string, + existingMatches: readonly RankedMatch[], +): RankedMatch[] { + if (pattern.includes('*') || existingMatches.length >= 5) return []; + const nodesByPath = new Map(data.graphData.nodes + .filter(node => !node.symbol) + .map(node => [node.id, node])); + const existingNodePaths = new Set(existingMatches.flatMap(item => ( + item.match.type === 'node' ? [item.match.node.path] : [] + ))); + return rankSearchDocuments(pattern, (data.sourceText?.files ?? []).map(file => ({ + id: file.filePath, + path: file.filePath, + text: file.content, + }))).slice(0, 3).flatMap((result, index) => { + const node = nodesByPath.get(result.id); + return node && !existingNodePaths.has(result.id) ? [{ + match: { type: 'node' as const, node: toNodeReportItem(node) }, + rank: 4, + sortKey: String(index).padStart(10, '0'), + }] : []; + }); +} + +export function searchGraph( + data: GraphQueryData, + config: GraphQuerySearchConfig, +): GraphQuerySearchReport { + const matcher = createPatternMatcher(config.pattern); + const directMatches = [ + ...(data.symbols ?? []).map(symbol => symbolMatch(symbol, matcher)).filter(match => match !== undefined), + ...data.graphData.nodes.map(node => nodeMatch(node, matcher)).filter(match => match !== undefined), + ...sourceMatches(data, config.pattern, matcher), + ]; + const rankedMatches = [ + ...directMatches, + ...phraseFallbackMatches(data, config.pattern, directMatches), + ].sort((left, right) => left.rank - right.rank || left.sortKey.localeCompare(right.sortKey)); + const page = paginate(rankedMatches.map(item => item.match), config); + + return { + pattern: config.pattern, + matches: page.items, + page: page.page, + sources: { + text: { + freshness: 'live', + filesScanned: data.sourceText?.filesScanned ?? 0, + filesSkipped: data.sourceText?.filesSkipped ?? 0, + }, + symbols: { + freshness: 'cached', + cacheState: data.cacheState ?? 'fresh', + }, + }, + }; +} diff --git a/packages/core/src/graphQuery/search/ranking.ts b/packages/core/src/graphQuery/search/ranking.ts new file mode 100644 index 0000000000..b9f4eeaf2d --- /dev/null +++ b/packages/core/src/graphQuery/search/ranking.ts @@ -0,0 +1,60 @@ +const PATH_TERM_WEIGHT = 100; +const PATH_FRAGMENT_WEIGHT = 20; + +const STOP_WORDS = new Set([ + 'a', 'an', 'and', 'are', 'as', 'at', 'be', 'but', 'by', 'for', 'from', + 'how', 'in', 'into', 'is', 'it', 'of', 'on', 'or', 'that', 'the', 'this', + 'to', 'with', +]); + +export interface SearchDocument { + id: string; + path: string; + text: string; +} + +export interface RankedSearchDocument { + id: string; + score: number; +} + +function tokenize(value: string): string[] { + const separated = value.replace(/([\p{Ll}\d])(\p{Lu})/gu, '$1 $2'); + return separated.match(/[\p{L}\d]+/gu)?.map(term => term.toLocaleLowerCase()) + .filter(term => term.length > 1 && !STOP_WORDS.has(term)) ?? []; +} + +function scoreDocument( + document: SearchDocument, + queryTerms: readonly string[], +): RankedSearchDocument | undefined { + const path = document.path.toLocaleLowerCase(); + const pathTerms = tokenize(document.path); + const textTerms = tokenize(document.text); + const availableTerms = new Set([...pathTerms, ...textTerms]); + if (!queryTerms.every(term => availableTerms.has(term))) return undefined; + const score = queryTerms.reduce((total, term) => ( + total + + pathTerms.filter(pathTerm => pathTerm === term).length * PATH_TERM_WEIGHT + + (path.includes(term) ? PATH_FRAGMENT_WEIGHT : 0) + + Math.min(10, textTerms.filter(textTerm => textTerm === term).length) + ), 0); + return { id: document.id, score }; +} + +export function rankSearchDocuments( + pattern: string, + documents: readonly SearchDocument[], +): RankedSearchDocument[] { + if (!/\s/u.test(pattern)) return []; + const queryTerms = [...new Set(tokenize(pattern))]; + if (queryTerms.length < 2) return []; + const ranked = documents.flatMap(document => scoreDocument(document, queryTerms) ?? []); + const pathMatches = ranked.filter(result => { + const document = documents.find(candidate => candidate.id === result.id); + const pathTerms = new Set(tokenize(document?.path ?? '')); + return queryTerms.every(term => pathTerms.has(term)); + }); + return (pathMatches.length > 0 ? pathMatches : ranked) + .sort((left, right) => right.score - left.score || left.id.localeCompare(right.id)); +} diff --git a/packages/core/src/graphQuery/taskMap.ts b/packages/core/src/graphQuery/taskMap.ts new file mode 100644 index 0000000000..fd096000cc --- /dev/null +++ b/packages/core/src/graphQuery/taskMap.ts @@ -0,0 +1,127 @@ +import type { GraphQueryData } from './data'; +import type { + GraphQueryTaskMapConfig, + GraphQueryTaskMapFile, + GraphQueryTaskMapReport, +} from './model'; +import { paginate } from './pagination'; +import { + createTaskMapDocuments, + rankTaskMapDocument, + selectTaskMapTerms, + taskMapTermFrequencies, +} from './taskMap/lexical'; +import { rankTaskMapGraph } from './taskMap/pagerank'; +import { + createTaskMapFileLinks, + indexTaskMapSymbols, + selectTaskMapRelationships, +} from './taskMap/projection'; +import { balanceTaskMapSourceAreas } from './taskMap/sourceAreas'; + +const DEFAULT_FILES = 8; +const MAX_FILES = 20; +const MAX_RELATIONSHIPS = 12; + +interface RankedTaskFile { + file: GraphQueryTaskMapFile; + lexicalScore: number; + graphScore: number; + score: number; +} + +function requestedFileLimit(config: GraphQueryTaskMapConfig): number { + if (!Number.isSafeInteger(config.limit) || (config.limit ?? 0) <= 0) return DEFAULT_FILES; + return Math.min(config.limit ?? DEFAULT_FILES, MAX_FILES); +} + +function connectedTaskMapFiles( + lexical: ReadonlyMap, + links: ReadonlyMap>, +): Set { + const connected = new Set(); + for (const [path, rank] of lexical) { + if (rank.score <= 0) continue; + connected.add(path); + for (const neighbor of links.get(path)?.keys() ?? []) connected.add(neighbor); + } + return connected; +} + +export function mapGraphTask( + data: GraphQueryData, + config: GraphQueryTaskMapConfig, +): GraphQueryTaskMapReport { + const query = typeof config.query === 'string' ? config.query : ''; + const documents = createTaskMapDocuments(data); + const terms = selectTaskMapTerms(query, documents); + const frequencies = taskMapTermFrequencies(terms, documents); + const lexical = new Map(documents.map(document => [ + document.node.id, + rankTaskMapDocument(document, terms, frequencies, documents.length), + ])); + const filePaths = new Set(documents.map(document => document.node.id)); + const links = createTaskMapFileLinks(data, filePaths); + const graphRanks = rankTaskMapGraph( + links, + new Map([...lexical].map(([path, rank]) => [path, rank.score])), + ); + const connected = connectedTaskMapFiles(lexical, links); + const maxLexicalScore = [...lexical.values()] + .reduce((maximum, rank) => Math.max(maximum, rank.score), 1); + const maxGraphScore = [...graphRanks.values()] + .reduce((maximum, rank) => Math.max(maximum, rank), 1 / Math.max(documents.length, 1)); + const symbols = indexTaskMapSymbols(data); + const ranked: RankedTaskFile[] = documents.flatMap((document) => { + const lexicalRanked = lexical.get(document.node.id) ?? { matchedTerms: [], score: 0 }; + if (lexicalRanked.score <= 0 && !connected.has(document.node.id)) return []; + const graphScore = graphRanks.get(document.node.id) ?? 0; + return [{ + file: { + path: document.node.id, + nodeType: 'file' as const, + matchedTerms: lexicalRanked.matchedTerms, + symbols: symbols.get(document.node.id) ?? [], + }, + lexicalScore: lexicalRanked.score, + graphScore, + score: lexicalRanked.score / maxLexicalScore * 0.85 + graphScore / maxGraphScore * 0.15, + }]; + }).sort((left, right) => ( + Number(right.lexicalScore > 0) - Number(left.lexicalScore > 0) + || right.score - left.score + || right.lexicalScore - left.lexicalScore + || left.file.path.localeCompare(right.file.path) + )); + const effectiveConfig = { ...config, limit: requestedFileLimit(config) }; + const balanced = balanceTaskMapSourceAreas(ranked); + const page = paginate(balanced.map(item => item.file), effectiveConfig); + const relationships = selectTaskMapRelationships( + data, + new Set(page.items.map(file => file.path)), + MAX_RELATIONSHIPS, + ); + + return { + query, + terms, + files: page.items, + relationships: relationships.relationships, + page: page.page, + limits: { + relationships: MAX_RELATIONSHIPS, + complete: page.page.offset === 0 && page.page.nextOffset === null && relationships.complete, + }, + sources: { + text: { + freshness: 'live', + filesScanned: data.sourceText?.filesScanned ?? 0, + filesSkipped: data.sourceText?.filesSkipped ?? 0, + }, + graph: { + freshness: 'cached', + cacheState: data.cacheState ?? 'fresh', + }, + }, + }; +} diff --git a/packages/core/src/graphQuery/taskMap/lexical.ts b/packages/core/src/graphQuery/taskMap/lexical.ts new file mode 100644 index 0000000000..92fcaed753 --- /dev/null +++ b/packages/core/src/graphQuery/taskMap/lexical.ts @@ -0,0 +1,123 @@ +import type { IGraphNode } from '../../graph/contracts'; +import type { GraphQueryData } from '../data'; + +const MAX_QUERY_TERMS = 16; +const STOP_WORDS = new Set([ + 'a', 'an', 'and', 'are', 'as', 'at', 'be', 'by', 'during', 'for', 'from', 'in', + 'into', 'is', 'it', 'of', 'on', 'or', 'that', 'the', 'this', 'to', 'with', +]); + +export interface TaskMapDocument { + node: IGraphNode; + pathText: string; + sourceText: string; +} + +export interface TaskMapLexicalRank { + matchedTerms: string[]; + score: number; +} + +function tokenize(value: string): string[] { + const separated = value.replace(/([\p{Ll}\d])(\p{Lu})/gu, '$1 $2'); + return separated.match(/[\p{L}\d]+/gu)?.map(term => term.toLocaleLowerCase()) + .filter(term => term.length > 2 && !STOP_WORDS.has(term)) ?? []; +} + +function termVariants(term: string): string[] { + const roots = new Set([term]); + if (term.endsWith('ing') && term.length > 5) { + const root = term.slice(0, -3); + roots.add(root); + if (root.at(-1) === root.at(-2)) roots.add(root.slice(0, -1)); + } + if (term.endsWith('ed') && term.length > 4) { + roots.add(term.slice(0, -2)); + roots.add(term.slice(0, -1)); + } + if (term.endsWith('s') && term.length > 4) roots.add(term.slice(0, -1)); + + const variants = new Set(roots); + for (const root of roots) { + variants.add(`${root}s`); + if (root.endsWith('e')) { + variants.add(`${root}d`); + variants.add(`${root.slice(0, -1)}ing`); + } else { + variants.add(`${root}ed`); + variants.add(`${root}ing`); + if (/[^aeiou]$/u.test(root)) { + variants.add(`${root}${root.at(-1)}ed`); + variants.add(`${root}${root.at(-1)}ing`); + } + } + } + return [...variants]; +} + +function normalizeSearchText(value: string): string { + const separated = value.replace(/([\p{Ll}\d])(\p{Lu})/gu, '$1 $2').toLocaleLowerCase(); + return ` ${separated.replace(/[^\p{L}\d]+/gu, ' ')} `; +} + +function includesTerm(value: string, queryTerm: string): boolean { + return termVariants(queryTerm).some(term => value.includes(` ${term} `)); +} + +export function createTaskMapDocuments(data: GraphQueryData): TaskMapDocument[] { + const files = new Map(data.graphData.nodes + .filter(node => node.nodeType === 'file' && !node.symbol) + .map(node => [node.id, node])); + return (data.sourceText?.files ?? []).flatMap(({ filePath, content }) => { + const node = files.get(filePath); + return node ? [{ + node, + pathText: normalizeSearchText(filePath), + sourceText: normalizeSearchText(content), + }] : []; + }); +} + +export function selectTaskMapTerms(query: string, documents: readonly TaskMapDocument[]): string[] { + const candidates = [...new Set(tokenize(query))]; + const frequency = new Map(candidates.map(term => [ + term, + documents.filter(document => includesTerm(document.pathText, term) || includesTerm(document.sourceText, term)).length, + ])); + return candidates + .filter(term => (frequency.get(term) ?? 0) > 0) + .map((term, index) => ({ term, index, frequency: frequency.get(term) ?? 0 })) + .sort((left, right) => left.frequency - right.frequency || left.index - right.index) + .slice(0, MAX_QUERY_TERMS) + .sort((left, right) => left.index - right.index) + .map(item => item.term); +} + +export function taskMapTermFrequencies( + terms: readonly string[], + documents: readonly TaskMapDocument[], +): Map { + return new Map(terms.map(term => [ + term, + documents.filter(document => includesTerm(document.pathText, term) || includesTerm(document.sourceText, term)).length, + ])); +} + +export function rankTaskMapDocument( + document: TaskMapDocument, + queryTerms: readonly string[], + frequencies: ReadonlyMap, + documentCount: number, +): TaskMapLexicalRank { + const matchedTerms = queryTerms.filter(term => ( + includesTerm(document.pathText, term) || includesTerm(document.sourceText, term) + )); + const score = matchedTerms.reduce((total, term) => { + const inverseFrequency = Math.log((documentCount + 1) / ((frequencies.get(term) ?? 0) + 1)) + 1; + const pathMatch = includesTerm(document.pathText, term); + const textMatch = includesTerm(document.sourceText, term); + return total + inverseFrequency * (pathMatch ? 4 : textMatch ? 1 : 0); + }, 0); + const isTest = /(?:^|\/)(?:__tests__|tests?)(?:\/|\.)|\.(?:spec|test)\.[^/]+$/iu.test(document.node.id); + return { matchedTerms, score: isTest ? score * 0.15 : score }; +} diff --git a/packages/core/src/graphQuery/taskMap/pagerank.ts b/packages/core/src/graphQuery/taskMap/pagerank.ts new file mode 100644 index 0000000000..e55ceac747 --- /dev/null +++ b/packages/core/src/graphQuery/taskMap/pagerank.ts @@ -0,0 +1,36 @@ +const ITERATIONS = 20; +const DAMPING = 0.85; + +export function rankTaskMapGraph( + links: ReadonlyMap>, + personalization: ReadonlyMap, +): Map { + const paths = [...links.keys()]; + const totalPersonalization = [...personalization.values()].reduce((total, value) => total + value, 0); + const normalized = new Map(paths.map(path => [ + path, + totalPersonalization > 0 ? (personalization.get(path) ?? 0) / totalPersonalization : 1 / Math.max(paths.length, 1), + ])); + let ranks = new Map(normalized); + + for (let iteration = 0; iteration < ITERATIONS; iteration += 1) { + const danglingMass = paths + .filter(path => (links.get(path)?.size ?? 0) === 0) + .reduce((total, path) => total + (ranks.get(path) ?? 0), 0); + const next = new Map(paths.map(path => [ + path, + ((1 - DAMPING) + DAMPING * danglingMass) * (normalized.get(path) ?? 0), + ])); + for (const path of paths) { + const neighbors = links.get(path) ?? new Map(); + const totalWeight = [...neighbors.values()].reduce((total, weight) => total + weight, 0); + if (totalWeight === 0) continue; + for (const [neighbor, weight] of neighbors) { + next.set(neighbor, (next.get(neighbor) ?? 0) + DAMPING * (ranks.get(path) ?? 0) * weight / totalWeight); + } + } + ranks = next; + } + + return ranks; +} diff --git a/packages/core/src/graphQuery/taskMap/projection.ts b/packages/core/src/graphQuery/taskMap/projection.ts new file mode 100644 index 0000000000..4b9f1daf67 --- /dev/null +++ b/packages/core/src/graphQuery/taskMap/projection.ts @@ -0,0 +1,80 @@ +import type { GraphEdgeKind, IGraphNode } from '../../graph/contracts'; +import type { GraphQueryData } from '../data'; +import type { GraphQueryTaskMapFile, GraphQueryTaskMapReport } from '../model'; + +const MAX_SYMBOLS_PER_FILE = 3; + +function filePathForNode(node: IGraphNode | undefined): string | undefined { + if (!node) return undefined; + if (node.nodeType === 'file' && !node.symbol) return node.id; + return node.symbol?.filePath; +} + +function edgeWeight(kind: GraphEdgeKind): number { + if (kind === 'call' || kind === 'event' || kind === 'inherit' || kind === 'reference') return 3; + if (kind === 'type-import') return 1; + return 2; +} + +export function createTaskMapFileLinks( + data: GraphQueryData, + filePaths: ReadonlySet, +): Map> { + const nodes = new Map(data.graphData.nodes.map(node => [node.id, node])); + const links = new Map([...filePaths].map(filePath => [filePath, new Map()])); + for (const edge of data.graphData.edges) { + if (edge.kind === 'contains') continue; + const from = filePathForNode(nodes.get(edge.from)); + const to = filePathForNode(nodes.get(edge.to)); + if (!from || !to || from === to || !filePaths.has(from) || !filePaths.has(to)) continue; + const weight = edgeWeight(edge.kind); + const fromLinks = links.get(from); + const toLinks = links.get(to); + fromLinks?.set(to, (fromLinks.get(to) ?? 0) + weight); + toLinks?.set(from, (toLinks.get(from) ?? 0) + weight); + } + return links; +} + +export function indexTaskMapSymbols(data: GraphQueryData): Map { + const symbols = new Map(); + for (const symbol of data.symbols ?? []) { + const fileSymbols = symbols.get(symbol.filePath) ?? []; + fileSymbols.push({ + ...(symbol.id ? { id: symbol.id } : {}), + name: symbol.name, + ...(symbol.kind ? { kind: symbol.kind } : {}), + }); + symbols.set(symbol.filePath, fileSymbols); + } + for (const [filePath, fileSymbols] of symbols) { + symbols.set(filePath, fileSymbols + .sort((left, right) => left.name.localeCompare(right.name) || (left.id ?? '').localeCompare(right.id ?? '')) + .slice(0, MAX_SYMBOLS_PER_FILE)); + } + return symbols; +} + +export function selectTaskMapRelationships( + data: GraphQueryData, + selectedPaths: ReadonlySet, + limit: number, +): { relationships: GraphQueryTaskMapReport['relationships']; complete: boolean } { + const nodes = new Map(data.graphData.nodes.map(node => [node.id, node])); + const grouped = new Map>(); + for (const edge of data.graphData.edges) { + if (edge.kind === 'contains') continue; + const from = filePathForNode(nodes.get(edge.from)); + const to = filePathForNode(nodes.get(edge.to)); + if (!from || !to || from === to || !selectedPaths.has(from) || !selectedPaths.has(to)) continue; + const key = `${from}\u0000${to}`; + const kinds = grouped.get(key) ?? new Set(); + kinds.add(edge.kind); + grouped.set(key, kinds); + } + const all = [...grouped].map(([key, kinds]) => { + const [from = '', to = ''] = key.split('\u0000'); + return { from, to, edgeTypes: [...kinds].sort() }; + }).sort((left, right) => left.from.localeCompare(right.from) || left.to.localeCompare(right.to)); + return { relationships: all.slice(0, limit), complete: all.length <= limit }; +} diff --git a/packages/core/src/graphQuery/taskMap/sourceAreas.ts b/packages/core/src/graphQuery/taskMap/sourceAreas.ts new file mode 100644 index 0000000000..3af11e39d3 --- /dev/null +++ b/packages/core/src/graphQuery/taskMap/sourceAreas.ts @@ -0,0 +1,44 @@ +export interface TaskMapSourceAreaItem { + file: { path: string }; + lexicalScore: number; + score: number; +} + +function rankingGroup(filePath: string): string { + const segments = filePath.split('/'); + const sourceIndex = segments.findIndex(segment => segment === 'src' || segment === 'tests'); + if (sourceIndex < 0) return segments.slice(0, Math.min(segments.length, 4)).join('/'); + if (sourceIndex + 1 >= segments.length - 1) { + return segments.slice(0, sourceIndex + 1).join('/'); + } + const sourceArea = segments[sourceIndex + 1]; + const areaDepth = sourceArea === 'extension' || sourceArea === 'webview' ? 2 : 1; + return segments.slice(0, sourceIndex + 1 + areaDepth).join('/'); +} + +export function balanceTaskMapSourceAreas(ranked: readonly T[]): T[] { + const groups = new Map(); + for (const item of ranked) { + const group = rankingGroup(item.file.path); + const items = groups.get(group) ?? []; + items.push(item); + groups.set(group, items); + } + const ordered = [...groups.entries()].sort((left, right) => { + const leftRank = left[1][0]; + const rightRank = right[1][0]; + if (!leftRank || !rightRank) return left[0].localeCompare(right[0]); + return Number(rightRank.lexicalScore > 0) - Number(leftRank.lexicalScore > 0) + || rightRank.score - leftRank.score + || rightRank.lexicalScore - leftRank.lexicalScore + || left[0].localeCompare(right[0]); + }); + const balanced: T[] = []; + for (let index = 0; balanced.length < ranked.length; index += 1) { + for (const [, items] of ordered) { + const item = items[index]; + if (item) balanced.push(item); + } + } + return balanced; +} diff --git a/packages/core/src/graphScope/defaults.ts b/packages/core/src/graphScope/defaults.ts index bb7fb52110..5f348c448c 100644 --- a/packages/core/src/graphScope/defaults.ts +++ b/packages/core/src/graphScope/defaults.ts @@ -17,6 +17,7 @@ export const CORE_GRAPH_EDGE_TYPES = [ 'load', 'nests', 'overrides', + 'reexport', 'reference', 'type', 'type-import', @@ -35,6 +36,7 @@ export const CORE_GRAPH_EDGE_DEFAULT_VISIBILITY: Record = { inherit: false, load: false, overrides: false, + reexport: false, reference: false, type: false, 'type-import': false, diff --git a/packages/core/src/index.ts b/packages/core/src/index.ts index 166f3aa0b5..f11c70cf1a 100644 --- a/packages/core/src/index.ts +++ b/packages/core/src/index.ts @@ -443,6 +443,8 @@ export type { GraphQueryFilterOperator, GraphQueryNodeReport, GraphQueryNodeReportItem, + GraphQueryOverviewConfig, + GraphQueryOverviewReport, GraphQueryPage, GraphQueryPathConfig, GraphQueryPathReport, @@ -455,16 +457,25 @@ export type { GraphQueryRelationshipSymbol, GraphQueryResult, GraphQueryScope, + GraphQuerySearchConfig, + GraphQuerySearchMatch, + GraphQuerySearchReport, GraphQuerySort, GraphQuerySymbolReport, GraphQuerySymbolReportItem, GraphQuerySymbolsConfig, + GraphQueryTaskMapConfig, + GraphQueryTaskMapFile, + GraphQueryTaskMapReport, } from './graphQuery'; export { executeGraphQuery, findGraphPaths, + inspectGraphTarget, listGraphEdges, listGraphNodes, listGraphRelationships, listGraphSymbols, + mapGraphTask, + searchGraph, } from './graphQuery'; diff --git a/packages/core/src/plugins/routing/router/results/keys.ts b/packages/core/src/plugins/routing/router/results/keys.ts index dcdf3b7951..fa99778f4e 100644 --- a/packages/core/src/plugins/routing/router/results/keys.ts +++ b/packages/core/src/plugins/routing/router/results/keys.ts @@ -15,6 +15,22 @@ function getBaseRelationKeyParts( ]; } +function getBindingRelationKeyParts( + relation: NonNullable[number], +): string[] { + const metadata = relation.metadata; + const parts = [ + metadata?.bindingKind, + metadata?.importedName, + metadata?.localName, + metadata?.memberName, + metadata?.exportedName, + metadata?.reexport, + metadata?.reexportAll, + ].map(value => value === undefined || value === null ? '' : String(value)); + return parts.some(Boolean) ? ['binding', ...parts] : []; +} + function getResolvedRelationKeyParts( relation: NonNullable[number], ): string[] { @@ -27,7 +43,10 @@ function getResolvedRelationKeyParts( } export function getRelationKey(relation: NonNullable[number]): string { - const key = getBaseRelationKeyParts(relation); + const key = [ + ...getBaseRelationKeyParts(relation), + ...getBindingRelationKeyParts(relation), + ]; if (relation.kind === 'call' || relation.kind === 'reference' || relation.kind === 'event') { key.push(...getResolvedRelationKeyParts(relation)); diff --git a/packages/core/src/treeSitter/runtime/analyzeJavaScript/file.ts b/packages/core/src/treeSitter/runtime/analyzeJavaScript/file.ts index 61488e6268..9d75874aab 100644 --- a/packages/core/src/treeSitter/runtime/analyzeJavaScript/file.ts +++ b/packages/core/src/treeSitter/runtime/analyzeJavaScript/file.ts @@ -72,7 +72,13 @@ const JAVASCRIPT_NODE_VISITORS: Record = { }, enum_declaration: handleTypeDeclarationNode, export_statement: (node, context) => { - handleJavaScriptExportStatement(node, context.filePath, context.relations); + handleJavaScriptExportStatement( + node, + context.filePath, + context.relations, + context.symbols, + context.symbolsEnabled, + ); }, function_declaration: (node, context) => context.symbolsEnabled diff --git a/packages/core/src/treeSitter/runtime/analyzeJavaScript/imports.ts b/packages/core/src/treeSitter/runtime/analyzeJavaScript/imports.ts index bbabc737e9..40bd607dde 100644 --- a/packages/core/src/treeSitter/runtime/analyzeJavaScript/imports.ts +++ b/packages/core/src/treeSitter/runtime/analyzeJavaScript/imports.ts @@ -1,10 +1,10 @@ import type Parser from 'tree-sitter'; -import type { IAnalysisRelation } from '@codegraphy-dev/plugin-api'; +import type { IAnalysisRelation, IAnalysisSymbol } from '@codegraphy-dev/plugin-api'; import { TREE_SITTER_SOURCE_IDS } from '../languages'; import { resolveTreeSitterImportPath } from '../resolve'; import type { ImportedBinding, SymbolWalkState, TreeWalkAction } from '../analyze/model'; import { getStringSpecifier } from '../analyze/nodes'; -import { addRelation } from '../analyze/results'; +import { addRelation, createSymbol } from '../analyze/results'; import { hasValueImport } from './importKinds'; import { addTypeImportRelations, addValueImportRelations } from './importRelations'; import { hasDirectTypeKeyword, hasTypeSpecifierImport } from './typeImports/markers'; @@ -49,23 +49,61 @@ export function handleJavaScriptImportStatement( return { skipChildren: true }; } -export function handleJavaScriptExportStatement( - node: Parser.SyntaxNode, - filePath: string, +function addReexportRelation( relations: IAnalysisRelation[], + filePath: string, + specifier: string, + resolvedPath: string | null, + metadata: NonNullable, + fromSymbolId?: string, ): void { - const specifier = getStringSpecifier(node.namedChildren.find((child) => child.type === 'string')); - if (!specifier) { - return; - } - - const resolvedPath = resolveTreeSitterImportPath(filePath, specifier); addRelation(relations, { - kind: 'import', + kind: 'reexport', sourceId: TREE_SITTER_SOURCE_IDS.import, fromFilePath: filePath, + ...(fromSymbolId ? { fromSymbolId } : {}), specifier, resolvedPath, toFilePath: resolvedPath, + metadata, }); } + +function readExportSpecifierName( + exportSpecifier: Parser.SyntaxNode, + field: 'name' | 'alias', +): string | undefined { + return exportSpecifier.childForFieldName(field)?.text; +} + +export function handleJavaScriptExportStatement( + node: Parser.SyntaxNode, + filePath: string, + relations: IAnalysisRelation[], + symbols: IAnalysisSymbol[] = [], + symbolsEnabled = true, +): void { + const specifier = getStringSpecifier(node.namedChildren.find((child) => child.type === 'string')); + if (!specifier) return; + const resolvedPath = resolveTreeSitterImportPath(filePath, specifier); + const exportClause = node.namedChildren.find((child) => child.type === 'export_clause'); + if (!exportClause) { + addReexportRelation(relations, filePath, specifier, resolvedPath, { reexportAll: true }); + return; + } + + for (const exported of exportClause.namedChildren.filter(child => child.type === 'export_specifier')) { + const importedName = readExportSpecifierName(exported, 'name'); + if (!importedName) continue; + const exportedName = readExportSpecifierName(exported, 'alias') ?? importedName; + const alias = symbolsEnabled && exportedName !== importedName + ? createSymbol(filePath, 'alias', exportedName, exported) + : undefined; + if (alias) symbols.push(alias); + addReexportRelation(relations, filePath, specifier, resolvedPath, { + reexport: true, + importedName, + exportedName, + }, alias?.id); + } +} diff --git a/packages/core/src/treeSitter/runtime/capabilities.ts b/packages/core/src/treeSitter/runtime/capabilities.ts index 455660a70b..d66900f12c 100644 --- a/packages/core/src/treeSitter/runtime/capabilities.ts +++ b/packages/core/src/treeSitter/runtime/capabilities.ts @@ -18,6 +18,7 @@ interface TreeSitterCapabilityContext { const DEFAULT_TREE_SITTER_EDGE_TYPE_CAPABILITIES = [ 'import', + 'reexport', 'reference', 'call', 'type-import', @@ -32,7 +33,7 @@ const TREE_SITTER_EDGE_TYPE_CAPABILITIES_BY_LANGUAGE = { go: ['import', 'reference', 'call', 'inherit', 'contains'], haskell: ['import', 'reference', 'call', 'contains'], java: ['import', 'reference', 'call', 'inherit'], - javascript: ['import', 'call', 'inherit'], + javascript: ['import', 'reexport', 'call', 'inherit'], kotlin: ['import', 'reference', 'call', 'inherit'], lua: ['import', 'reference', 'call'], objectiveC: ['import', 'reference', 'call', 'inherit'], @@ -43,8 +44,8 @@ const TREE_SITTER_EDGE_TYPE_CAPABILITIES_BY_LANGUAGE = { rust: ['import', 'reference', 'call'], scala: ['import', 'reference', 'call', 'inherit'], swift: ['import', 'reference', 'call', 'inherit'], - tsx: ['import', 'type-import', 'call', 'inherit', 'contains'], - typescript: ['import', 'type-import', 'call', 'inherit', 'contains'], + tsx: ['import', 'reexport', 'type-import', 'call', 'inherit', 'contains'], + typescript: ['import', 'reexport', 'type-import', 'call', 'inherit', 'contains'], } as const satisfies Record; const TREE_SITTER_NODE_TYPE_CAPABILITIES_BY_LANGUAGE = { diff --git a/packages/core/src/workspace/queryGraph.ts b/packages/core/src/workspace/queryGraph.ts index 79e53b1888..b233552f50 100644 --- a/packages/core/src/workspace/queryGraph.ts +++ b/packages/core/src/workspace/queryGraph.ts @@ -1,3 +1,6 @@ +import * as fs from 'node:fs'; +import * as path from 'node:path'; +import { createWorkspaceFileContentHash } from '../analysis/cache'; import { readWorkspaceAnalysisDatabaseSnapshot } from '../graphCache/database/storage'; import { filterInactivePluginSnapshotFacts } from '../plugins/activityState/analysisFacts'; import { createPluginActivityState } from '../plugins/activityState/model'; @@ -6,10 +9,66 @@ import { CODEGRAPHY_MARKDOWN_PLUGIN_ID, readCodeGraphyWorkspaceSettings } from ' import { normalizeWorkspaceQueryFacts } from './queryFacts'; import { matchesAnyPattern } from '../discovery/pathMatching'; import type { IGraphData } from '../graph/contracts'; +import { getNodeType } from '../visibleGraph/model'; +import type { GraphQuerySourceText } from '../graphQuery/data'; import { resolveProjectedGraphNodeTypes } from './graphScopeProjection/model'; import { resolveSavedGraphScope } from './graphScopeSettings'; import type { WorkspaceGraphQueryProjection } from './requestTypes'; +const MAX_QUERY_SOURCE_FILE_BYTES = 1024 * 1024; + +function isInsideWorkspace(workspaceRoot: string, absolutePath: string): boolean { + const relativePath = path.relative(workspaceRoot, absolutePath); + return relativePath !== '..' + && !relativePath.startsWith(`..${path.sep}`) + && !path.isAbsolute(relativePath); +} + +interface QuerySourceFileResult { + file?: GraphQuerySourceText['files'][number]; + changed: boolean; +} + +function readQuerySourceFile( + workspaceRoot: string, + filePath: string, + indexedContentHash: string | undefined, +): QuerySourceFileResult { + const absolutePath = path.resolve(workspaceRoot, filePath); + if (!isInsideWorkspace(workspaceRoot, absolutePath)) return { changed: false }; + + try { + if (fs.statSync(absolutePath).size > MAX_QUERY_SOURCE_FILE_BYTES) return { changed: false }; + const content = fs.readFileSync(absolutePath, 'utf8'); + if (content.includes('\0')) return { changed: false }; + return { + file: { filePath, content }, + changed: indexedContentHash !== undefined + && indexedContentHash !== createWorkspaceFileContentHash(content), + }; + } catch { + return { changed: false }; + } +} + +export function readWorkspaceQuerySourceText( + workspaceRoot: string, + graphData: IGraphData, + indexedContentHashes: ReadonlyMap = new Map(), +): GraphQuerySourceText { + const results = graphData.nodes + .filter(node => getNodeType(node) === 'file') + .map(node => readQuerySourceFile(workspaceRoot, node.id, indexedContentHashes.get(node.id))); + const files = results.flatMap(result => result.file ? [result.file] : []); + + return { + files, + filesScanned: files.length, + filesSkipped: results.length - files.length, + hasChangedFiles: results.some(result => result.changed), + }; +} + function applyPathFilters(graphData: IGraphData, patterns: readonly string[]): IGraphData { if (patterns.length === 0) return graphData; const nodes = graphData.nodes.filter((node) => { @@ -23,10 +82,9 @@ function applyPathFilters(graphData: IGraphData, patterns: readonly string[]): I }; } -export function readWorkspaceQueryGraph( +export function readWorkspaceQuerySource( workspaceRoot: string, installedPluginCache: CodeGraphyInstalledPluginCache, - projection: WorkspaceGraphQueryProjection = {}, ) { const settings = readCodeGraphyWorkspaceSettings(workspaceRoot); const snapshot = readWorkspaceAnalysisDatabaseSnapshot(workspaceRoot); @@ -40,20 +98,39 @@ export function readWorkspaceQueryGraph( nodes: snapshot.files.flatMap(file => file.analysis.nodeTypes ?? []), edges: snapshot.files.flatMap(file => file.analysis.edgeTypes ?? []), }; - const disabledFilterPatterns = new Set(settings.disabledCustomFilterPatterns); + + return { + declarations, + graphData: snapshot.graph, + indexedContentHashes: new Map(snapshot.files.flatMap(file => ( + file.contentHash ? [[file.filePath, file.contentHash] as const] : [] + ))), + settings, + snapshotFacts: normalizeWorkspaceQueryFacts( + filterInactivePluginSnapshotFacts(snapshot, activePluginIds), + workspaceRoot, + ), + }; +} + +export function projectWorkspaceQueryGraph( + source: ReturnType, + projection: WorkspaceGraphQueryProjection = {}, +) { + const disabledFilterPatterns = new Set(source.settings.disabledCustomFilterPatterns); const graphData = applyPathFilters( - snapshot.graph, + source.graphData, [ - ...settings.filterPatterns.filter(pattern => !disabledFilterPatterns.has(pattern)), + ...source.settings.filterPatterns.filter(pattern => !disabledFilterPatterns.has(pattern)), ...(projection.filterPatterns ?? []), ], ); - const savedScope = resolveSavedGraphScope(settings, graphData, declarations); + const savedScope = resolveSavedGraphScope(source.settings, graphData, source.declarations); const scope = { nodes: projection.nodeTypes ? Object.fromEntries([ ...Object.keys(savedScope.nodes).map(type => [type, false] as const), - ...resolveProjectedGraphNodeTypes(projection.nodeTypes, declarations.nodes) + ...resolveProjectedGraphNodeTypes(projection.nodeTypes, source.declarations.nodes) .map(type => [type, true] as const), ]) : savedScope.nodes, @@ -67,12 +144,20 @@ export function readWorkspaceQueryGraph( return { graphData, - nodeTypes: declarations.nodes, + nodeTypes: source.declarations.nodes, scope, - settings, - snapshotFacts: normalizeWorkspaceQueryFacts( - filterInactivePluginSnapshotFacts(snapshot, activePluginIds), - workspaceRoot, - ), + settings: source.settings, + snapshotFacts: source.snapshotFacts, }; } + +export function readWorkspaceQueryGraph( + workspaceRoot: string, + installedPluginCache: CodeGraphyInstalledPluginCache, + projection: WorkspaceGraphQueryProjection = {}, +) { + return projectWorkspaceQueryGraph( + readWorkspaceQuerySource(workspaceRoot, installedPluginCache), + projection, + ); +} diff --git a/packages/core/src/workspace/requestIndexing.ts b/packages/core/src/workspace/requestIndexing.ts index 4d7154ae0c..531e2946f3 100644 --- a/packages/core/src/workspace/requestIndexing.ts +++ b/packages/core/src/workspace/requestIndexing.ts @@ -58,6 +58,14 @@ export async function requestCodeGraphyIndexWorkspace( files: result.files.length, nodes: result.graph.nodes.length, edges: result.graph.edges.length, + discovery: { + indexedFiles: result.files.length, + totalFound: result.totalFound, + limitReached: result.limitReached, + ...(result.limitReached ? { + action: `Run \`codegraphy settings set maxFiles ${result.totalFound}\`, then rerun \`codegraphy index\`.`, + } : {}), + }, indexing: result.indexing, }; } diff --git a/packages/core/src/workspace/requestQuery.ts b/packages/core/src/workspace/requestQuery.ts index 3a0367d41c..2529eb221a 100644 --- a/packages/core/src/workspace/requestQuery.ts +++ b/packages/core/src/workspace/requestQuery.ts @@ -6,15 +6,27 @@ import { executeGraphQuery, type GraphQueryRequest, } from '../graphQuery'; +import { getNodeType } from '../visibleGraph/model'; import { emitGraphQueryCacheMissing, emitGraphQueryCompleted, emitGraphQueryStarted } from './queryDiagnostics'; -import { readWorkspaceQueryGraph } from './queryGraph'; +import { + projectWorkspaceQueryGraph, + readWorkspaceQuerySource, + readWorkspaceQuerySourceText, +} from './queryGraph'; import { resolveCodeGraphyWorkspacePath } from './requestPaths'; -import type { WorkspaceGraphQueryInput, WorkspaceGraphQueryResult } from './requestTypes'; +import type { + WorkspaceGraphQueryInput, + WorkspaceGraphQueryResult, +} from './requestTypes'; import { readCodeGraphyWorkspaceStatus } from './status'; export interface WorkspaceGraphQueryDependencies { cwd(): string; readInstalledPluginCache(): CodeGraphyInstalledPluginCache; + readQuerySource?( + workspaceRoot: string, + installedPluginCache: CodeGraphyInstalledPluginCache, + ): ReturnType; } const DEFAULT_DEPENDENCIES: WorkspaceGraphQueryDependencies = { @@ -29,46 +41,61 @@ function createGraphQueryOperationId(): string { return `query-${graphQueryOperationCounter}`; } -export async function requestWorkspaceGraphQuery( - input: WorkspaceGraphQueryInput, - dependencies: WorkspaceGraphQueryDependencies = DEFAULT_DEPENDENCIES, -): Promise { +function createCacheMissingResult(workspaceRoot: string): WorkspaceGraphQueryResult { + return { + error: 'graph_cache_not_found', + message: 'This CodeGraphy Workspace has not been indexed. Run `codegraphy index`, then retry.', + workspaceRoot, + }; +} + +function hasTargetSelector(arguments_: Record): boolean { + return ['from', 'to', 'target', 'filePath', 'relatedFrom', 'relatedTo'] + .some(key => typeof arguments_[key] === 'string'); +} + +function shouldApplyWorkspaceGraphScope(input: WorkspaceGraphQueryInput): boolean { + if (input.report === 'relationships') return true; + if (input.report === 'search' || input.report === 'task-map' || input.report === 'overview' || input.report === 'paths') return false; + return !hasTargetSelector(input.arguments); +} + +function executeWorkspaceGraphQuery( + input: Omit, + workspaceRoot: string, + status: ReturnType, + source: ReturnType, +): WorkspaceGraphQueryResult { const startedAt = performance.now(); - const workspaceRoot = resolveCodeGraphyWorkspacePath(input.workspacePath, dependencies.cwd()); const operationId = createGraphQueryOperationId(); emitGraphQueryStarted({ diagnostics: input.diagnostics, operationId, report: input.report, workspaceRoot }); - const status = readCodeGraphyWorkspaceStatus(workspaceRoot); - if (!status.hasGraphCache) { - emitGraphQueryCacheMissing({ - diagnostics: input.diagnostics, - operationId, - report: input.report, - status, - workspaceRoot, - }); - return { - error: 'graph_cache_not_found', - message: 'This CodeGraphy Workspace has not been indexed. Run `codegraphy index`, then retry.', - workspaceRoot, - }; - } - - const { graphData, nodeTypes, scope, snapshotFacts } = readWorkspaceQueryGraph( - workspaceRoot, - dependencies.readInstalledPluginCache(), + const { graphData, nodeTypes, scope, snapshotFacts } = projectWorkspaceQueryGraph( + source, input.projection, ); + const sourceText = input.report === 'search' || input.report === 'task-map' + ? readWorkspaceQuerySourceText(workspaceRoot, graphData, source.indexedContentHashes) + : undefined; + const completeScope = { + nodes: Object.fromEntries(graphData.nodes.map(node => [getNodeType(node), true])), + edges: Object.fromEntries(graphData.edges.map(edge => [edge.kind, true])), + }; + const queryScope = shouldApplyWorkspaceGraphScope(input) + ? { nodes: scope.nodes, edges: scope.edges } + : { + nodes: input.projection?.nodeTypes ? scope.nodes : completeScope.nodes, + edges: input.projection?.edgeTypes ? scope.edges : completeScope.edges, + }; const queryResult = executeGraphQuery({ graphData, symbols: snapshotFacts.symbols, relations: snapshotFacts.relations, + ...(sourceText ? { sourceText } : {}), + cacheState: status.state === 'stale' || sourceText?.hasChangedFiles ? 'stale' : 'fresh', }, { report: input.report, arguments: { - scope: { - nodes: scope.nodes, - edges: scope.edges, - }, + scope: queryScope, ...(input.projection?.nodeTypes ? { nodeTypeDefinitions: nodeTypes, @@ -97,3 +124,33 @@ export async function requestWorkspaceGraphQuery( }, }; } + +export async function requestWorkspaceGraphQuery( + input: WorkspaceGraphQueryInput, + dependencies: WorkspaceGraphQueryDependencies = DEFAULT_DEPENDENCIES, +): Promise { + const workspaceRoot = resolveCodeGraphyWorkspacePath(input.workspacePath, dependencies.cwd()); + const status = readCodeGraphyWorkspaceStatus(workspaceRoot); + if (!status.hasGraphCache) { + const operationId = createGraphQueryOperationId(); + emitGraphQueryStarted({ diagnostics: input.diagnostics, operationId, report: input.report, workspaceRoot }); + emitGraphQueryCacheMissing({ + diagnostics: input.diagnostics, + operationId, + report: input.report, + status, + workspaceRoot, + }); + return createCacheMissingResult(workspaceRoot); + } + + return executeWorkspaceGraphQuery( + input, + workspaceRoot, + status, + (dependencies.readQuerySource ?? readWorkspaceQuerySource)( + workspaceRoot, + dependencies.readInstalledPluginCache(), + ), + ); +} diff --git a/packages/core/src/workspace/requestTypes.ts b/packages/core/src/workspace/requestTypes.ts index e767e7bcc8..1847e186ef 100644 --- a/packages/core/src/workspace/requestTypes.ts +++ b/packages/core/src/workspace/requestTypes.ts @@ -5,7 +5,10 @@ export type GraphQueryReport = | 'edges' | 'relationships' | 'symbols' - | 'paths'; + | 'paths' + | 'search' + | 'task-map' + | 'overview'; export interface WorkspacePathInput { diagnostics?: DiagnosticEventSink; @@ -28,6 +31,12 @@ export interface IndexWorkspaceResult { workspaceRoot: string; graphCache: string; message: string; + discovery: { + indexedFiles: number; + totalFound: number; + limitReached: boolean; + action?: string; + }; indexing: { mode: 'full' | 'incremental'; analyzedFiles: number; diff --git a/packages/core/src/workspace/settings.ts b/packages/core/src/workspace/settings.ts index e64e2a5426..9845298091 100644 --- a/packages/core/src/workspace/settings.ts +++ b/packages/core/src/workspace/settings.ts @@ -14,8 +14,11 @@ export { ensureCodeGraphyWorkspaceSettings, readCodeGraphyWorkspaceSettings, readCodeGraphyWorkspaceSettingsOrInitial, + readRawWorkspaceSettingsOrInitial, patchCodeGraphyWorkspaceSettings, patchCodeGraphyWorkspaceSettingRecord, + removeCodeGraphyWorkspaceSetting, writeCodeGraphyWorkspacePluginData, writeCodeGraphyWorkspaceSettings, + WorkspaceSettingsError, } from './settingsStorage'; diff --git a/packages/core/src/workspace/settingsStorage.ts b/packages/core/src/workspace/settingsStorage.ts index ce34799431..8ed6a58f56 100644 --- a/packages/core/src/workspace/settingsStorage.ts +++ b/packages/core/src/workspace/settingsStorage.ts @@ -1,23 +1,47 @@ import * as fs from 'node:fs'; import * as path from 'node:path'; import { getWorkspaceSettingsPath } from './paths'; -import { - createDefaultCodeGraphyWorkspaceSettings, - createInitialCodeGraphyWorkspaceSettings, -} from './settingsDefaults'; +import { createInitialCodeGraphyWorkspaceSettings } from './settingsDefaults'; import { normalizeCodeGraphyWorkspaceSettings } from './settingsNormalize'; import { unknownRecordSchema } from '../values'; import type { CodeGraphyWorkspaceSettings } from './settingsContracts'; import { hasSupportedRawPluginIdentity } from './settingsPlugins'; +import { validateWorkspaceSettingsRecord } from './settingsValidation'; import { CODEGRAPHY_MARKDOWN_PLUGIN_ID, CODEGRAPHY_MARKDOWN_PLUGIN_PACKAGE_NAME, } from './settingsDefaults'; +export class WorkspaceSettingsError extends Error { + constructor( + readonly settingsPath: string, + readonly reason: string, + ) { + super(`Invalid CodeGraphy workspace settings at ${settingsPath}: ${reason}`); + } +} + function writeRawWorkspaceSettings(workspaceRoot: string, settings: Record): void { const settingsPath = getWorkspaceSettingsPath(workspaceRoot); + let validated: Record; + try { + validated = validateWorkspaceSettingsRecord(settings); + } catch (error) { + const reason = error instanceof Error ? error.message : String(error); + throw new WorkspaceSettingsError(settingsPath, reason); + } fs.mkdirSync(path.dirname(settingsPath), { recursive: true }); - fs.writeFileSync(settingsPath, `${JSON.stringify(settings, null, 2)}\n`); + fs.writeFileSync(settingsPath, `${JSON.stringify(validated, null, 2)}\n`); +} + +function readRawWorkspaceSettings(workspaceRoot: string): Record { + const settingsPath = getWorkspaceSettingsPath(workspaceRoot); + try { + return validateWorkspaceSettingsRecord(JSON.parse(fs.readFileSync(settingsPath, 'utf-8'))); + } catch (error) { + const message = error instanceof Error ? error.message : String(error); + throw new WorkspaceSettingsError(settingsPath, message); + } } function readPluginIdentity(value: unknown): string | undefined { @@ -77,13 +101,9 @@ function mergeRawPluginEntries(rawValue: unknown, plugins: CodeGraphyWorkspaceSe export function readCodeGraphyWorkspaceSettings( workspaceRoot: string, ): CodeGraphyWorkspaceSettings { - try { - return normalizeCodeGraphyWorkspaceSettings( - JSON.parse(fs.readFileSync(getWorkspaceSettingsPath(workspaceRoot), 'utf-8')), - ); - } catch { - return createDefaultCodeGraphyWorkspaceSettings(); - } + return fs.existsSync(getWorkspaceSettingsPath(workspaceRoot)) + ? normalizeCodeGraphyWorkspaceSettings(readRawWorkspaceSettings(workspaceRoot)) + : normalizeCodeGraphyWorkspaceSettings({}); } export function readCodeGraphyWorkspaceSettingsOrInitial( @@ -110,15 +130,10 @@ export function writeCodeGraphyWorkspaceSettings( }); } -function readRawWorkspaceSettingsOrInitial(workspaceRoot: string): Record { - try { - const parsed = unknownRecordSchema.safeParse( - JSON.parse(fs.readFileSync(getWorkspaceSettingsPath(workspaceRoot), 'utf-8')), - ); - return parsed.success ? { ...parsed.data } : { ...createInitialCodeGraphyWorkspaceSettings() }; - } catch { - return { ...createInitialCodeGraphyWorkspaceSettings() }; - } +export function readRawWorkspaceSettingsOrInitial(workspaceRoot: string): Record { + return fs.existsSync(getWorkspaceSettingsPath(workspaceRoot)) + ? readRawWorkspaceSettings(workspaceRoot) + : { ...createInitialCodeGraphyWorkspaceSettings() }; } export function patchCodeGraphyWorkspaceSettings( @@ -131,6 +146,15 @@ export function patchCodeGraphyWorkspaceSettings( }); } +export function removeCodeGraphyWorkspaceSetting( + workspaceRoot: string, + key: string, +): void { + const settings = readRawWorkspaceSettingsOrInitial(workspaceRoot); + delete settings[key]; + writeRawWorkspaceSettings(workspaceRoot, settings); +} + export function patchCodeGraphyWorkspaceSettingRecord( workspaceRoot: string, key: string, diff --git a/packages/core/src/workspace/settingsValidation.ts b/packages/core/src/workspace/settingsValidation.ts new file mode 100644 index 0000000000..bcfa98552d --- /dev/null +++ b/packages/core/src/workspace/settingsValidation.ts @@ -0,0 +1,78 @@ +import { hasSupportedRawPluginIdentity } from './settingsPlugins'; + +function isRecord(value: unknown): value is Record { + return Boolean(value) && typeof value === 'object' && !Array.isArray(value); +} + +function isStringArray(value: unknown): boolean { + return Array.isArray(value) && value.every(entry => typeof entry === 'string'); +} + +function isBooleanRecord(value: unknown): boolean { + return isRecord(value) && Object.values(value).every(entry => typeof entry === 'boolean'); +} + +function validatePluginSettings(value: unknown): string | undefined { + if (!Array.isArray(value)) return 'plugins must be an array'; + for (const entry of value) { + if (!isRecord(entry)) return 'plugins entries must be objects'; + if ('id' in entry && typeof entry.id !== 'string') return 'plugin id must be a string'; + if ('package' in entry && typeof entry.package !== 'string') return 'plugin package must be a string'; + if ('activation' in entry && !['inherit', 'enabled', 'disabled'].includes(String(entry.activation))) { + return 'plugin activation must be inherit, enabled, or disabled'; + } + if ('enabled' in entry && typeof entry.enabled !== 'boolean') return 'plugin enabled must be a boolean'; + if ('options' in entry && !isRecord(entry.options)) return 'plugin options must be an object'; + if ('disabledFilterPatterns' in entry && !isStringArray(entry.disabledFilterPatterns)) { + return 'plugin disabledFilterPatterns must be an array of strings'; + } + if (!hasSupportedRawPluginIdentity(entry)) { + return 'plugin entry must have a nonblank id with activation, or a nonblank package'; + } + } + return undefined; +} + +function validateInterfaceSettings(value: unknown): string | undefined { + if (!Array.isArray(value)) return 'interfaces must be an array'; + for (const entry of value) { + if (!isRecord(entry) || typeof entry.id !== 'string' || !entry.id.trim() || !('data' in entry)) { + return 'interface entries must have a nonblank id and data'; + } + } + return undefined; +} + +export function validateWorkspaceSettingsRecord(value: unknown): Record { + if (!isRecord(value)) throw new Error('workspace settings must be a JSON object'); + const stringArrays = ['include', 'filterPatterns', 'disabledCustomFilterPatterns'] as const; + for (const key of stringArrays) { + if (key in value && !isStringArray(value[key])) throw new Error(`${key} must be an array of strings`); + } + if ('respectGitignore' in value && typeof value.respectGitignore !== 'boolean') { + throw new Error('respectGitignore must be a boolean'); + } + if ('maxFiles' in value && ( + typeof value.maxFiles !== 'number' + || !Number.isSafeInteger(value.maxFiles) + || value.maxFiles < 1 + )) { + throw new Error('maxFiles must be a positive integer'); + } + if ('nodeVisibility' in value && !isBooleanRecord(value.nodeVisibility)) { + throw new Error('nodeVisibility must be an object of boolean values'); + } + if ('edgeVisibility' in value && !isBooleanRecord(value.edgeVisibility)) { + throw new Error('edgeVisibility must be an object of boolean values'); + } + if ('pluginData' in value && !isRecord(value.pluginData)) throw new Error('pluginData must be an object'); + if ('plugins' in value) { + const error = validatePluginSettings(value.plugins); + if (error) throw new Error(error); + } + if ('interfaces' in value) { + const error = validateInterfaceSettings(value.interfaces); + if (error) throw new Error(error); + } + return { ...value }; +} diff --git a/packages/core/tests/analysis/fileAnalysis/enrichment.test.ts b/packages/core/tests/analysis/fileAnalysis/enrichment.test.ts index 9996edb879..86d11647e8 100644 --- a/packages/core/tests/analysis/fileAnalysis/enrichment.test.ts +++ b/packages/core/tests/analysis/fileAnalysis/enrichment.test.ts @@ -29,10 +29,7 @@ function analysis( filePath: string, values: Partial, ): IFileAnalysisResult { - return { - filePath, - ...values, - }; + return { filePath, ...values }; } describe('pipeline/fileAnalysis/enrichment', () => { diff --git a/packages/core/tests/analysis/fileAnalysis/reexportTarget.test.ts b/packages/core/tests/analysis/fileAnalysis/reexportTarget.test.ts new file mode 100644 index 0000000000..d9810242ac --- /dev/null +++ b/packages/core/tests/analysis/fileAnalysis/reexportTarget.test.ts @@ -0,0 +1,206 @@ +import { describe, expect, it } from 'vitest'; +import type { + IAnalysisRelation, + IAnalysisSymbol, + IFileAnalysisResult, +} from '@codegraphy-dev/plugin-api'; +import { enrichWorkspaceFileAnalysis } from '../../../src/analysis/fileAnalysis/enrichment'; + +function symbol(filePath: string, name: string): IAnalysisSymbol { + return { + filePath, + id: `${filePath}:${name}`, + kind: 'function', + name, + }; +} + +function relation(overrides: Partial): IAnalysisRelation { + return { + fromFilePath: '/workspace/src/source.ts', + kind: 'import', + sourceId: 'test-source', + toFilePath: '/workspace/src/target.ts', + ...overrides, + }; +} + +function analysis( + filePath: string, + values: Partial, +): IFileAnalysisResult { + return { filePath, ...values }; +} + +function caller(importedName: string): IFileAnalysisResult { + return analysis('/workspace/src/caller.ts', { + relations: [relation({ + kind: 'call', + toFilePath: '/workspace/src/barrel.ts', + metadata: { importedName }, + })], + }); +} + +function expectCallerTarget( + result: ReadonlyMap, + filePath: string, + symbolId: string, +): void { + expect(result.get('src/caller.ts')?.relations?.[0]).toEqual( + expect.objectContaining({ toFilePath: filePath, toSymbolId: symbolId }), + ); +} + +describe('pipeline/fileAnalysis/reexportTarget', () => { + it('resolves named calls through a star re-export without redirecting lexical imports', () => { + const source = caller('readSetting'); + source.relations?.push(relation({ + kind: 'import', + toFilePath: '/workspace/src/barrel.ts', + metadata: { importedName: 'readSetting' }, + })); + const result = enrichWorkspaceFileAnalysis(new Map([ + ['src/caller.ts', source], + ['src/barrel.ts', analysis('/workspace/src/barrel.ts', { + relations: [relation({ + fromFilePath: '/workspace/src/barrel.ts', + toFilePath: '/workspace/src/storage.ts', + metadata: { reexportAll: true }, + })], + })], + ['src/storage.ts', analysis('/workspace/src/storage.ts', { + symbols: [ + symbol('/workspace/src/storage.ts', 'readSetting'), + symbol('/workspace/src/storage.ts', 'writeSetting'), + ], + })], + ])); + + expect(result.get('src/caller.ts')?.relations).toEqual([ + expect.objectContaining({ + toFilePath: '/workspace/src/storage.ts', + toSymbolId: '/workspace/src/storage.ts:readSetting', + }), + expect.objectContaining({ toFilePath: '/workspace/src/barrel.ts' }), + ]); + }); + + it('reuses persisted re-export Symbol edges without analyzer metadata', () => { + const result = enrichWorkspaceFileAnalysis(new Map([ + ['src/caller.ts', caller('readSetting')], + ['src/barrel.ts', analysis('/workspace/src/barrel.ts', { + relations: [relation({ + kind: 'reexport', + fromFilePath: '/workspace/src/barrel.ts', + toFilePath: '/workspace/src/storage.ts', + toSymbolId: '/workspace/src/storage.ts:readSetting', + })], + })], + ['src/storage.ts', analysis('/workspace/src/storage.ts', { + symbols: [symbol('/workspace/src/storage.ts', 'readSetting')], + })], + ])); + + expectCallerTarget( + result, + '/workspace/src/storage.ts', + '/workspace/src/storage.ts:readSetting', + ); + }); + + it('reuses persisted file-level re-export edges as export-star traversal', () => { + const result = enrichWorkspaceFileAnalysis(new Map([ + ['src/caller.ts', caller('readSetting')], + ['src/barrel.ts', analysis('/workspace/src/barrel.ts', { + relations: [relation({ + kind: 'reexport', + fromFilePath: '/workspace/src/barrel.ts', + toFilePath: '/workspace/src/storage.ts', + })], + })], + ['src/storage.ts', analysis('/workspace/src/storage.ts', { + symbols: [ + symbol('/workspace/src/storage.ts', 'readSetting'), + symbol('/workspace/src/storage.ts', 'writeSetting'), + ], + })], + ])); + + expectCallerTarget( + result, + '/workspace/src/storage.ts', + '/workspace/src/storage.ts:readSetting', + ); + }); + + it('follows a persisted re-export alias to its implementation Symbol', () => { + const alias = { + ...symbol('/workspace/src/barrel.ts', 'publicRead'), + id: '/workspace/src/barrel.ts:publicRead:alias', + kind: 'alias', + }; + const result = enrichWorkspaceFileAnalysis(new Map([ + ['src/caller.ts', caller('publicRead')], + ['src/barrel.ts', analysis('/workspace/src/barrel.ts', { + symbols: [alias], + relations: [relation({ + kind: 'reexport', + fromFilePath: '/workspace/src/barrel.ts', + fromSymbolId: alias.id, + toFilePath: '/workspace/src/storage.ts', + toSymbolId: '/workspace/src/storage.ts:internalRead', + })], + })], + ['src/storage.ts', analysis('/workspace/src/storage.ts', { + symbols: [symbol('/workspace/src/storage.ts', 'internalRead')], + })], + ])); + + expectCallerTarget( + result, + '/workspace/src/storage.ts', + '/workspace/src/storage.ts:internalRead', + ); + }); + + it('resolves an aliased named re-export without following unrelated exports', () => { + const result = enrichWorkspaceFileAnalysis(new Map([ + ['src/caller.ts', caller('publicRead')], + ['src/barrel.ts', analysis('/workspace/src/barrel.ts', { + relations: [ + relation({ + fromFilePath: '/workspace/src/barrel.ts', + toFilePath: '/workspace/src/storage.ts', + metadata: { + reexport: true, + exportedName: 'publicRead', + importedName: 'internalRead', + }, + }), + relation({ + fromFilePath: '/workspace/src/barrel.ts', + toFilePath: '/workspace/src/other.ts', + metadata: { + reexport: true, + exportedName: 'other', + importedName: 'other', + }, + }), + ], + })], + ['src/storage.ts', analysis('/workspace/src/storage.ts', { + symbols: [symbol('/workspace/src/storage.ts', 'internalRead')], + })], + ['src/other.ts', analysis('/workspace/src/other.ts', { + symbols: [symbol('/workspace/src/other.ts', 'other')], + })], + ])); + + expectCallerTarget( + result, + '/workspace/src/storage.ts', + '/workspace/src/storage.ts:internalRead', + ); + }); +}); diff --git a/packages/core/tests/analysis/fileAnalysis/targetSymbolName.test.ts b/packages/core/tests/analysis/fileAnalysis/targetSymbolName.test.ts index 385534fc8e..cacc54b357 100644 --- a/packages/core/tests/analysis/fileAnalysis/targetSymbolName.test.ts +++ b/packages/core/tests/analysis/fileAnalysis/targetSymbolName.test.ts @@ -32,6 +32,13 @@ describe('pipeline/fileAnalysis/targetSymbolName', () => { )).toBe('/workspace/src/target.ts:target'); }); + it('does not substitute the sole target when a requested name is missing', () => { + expect(resolveTargetSymbolId( + relation({ metadata: { importedName: 'missing' } }), + [symbol('/workspace/src/target.ts', 'other')], + )).toBeUndefined(); + }); + it('does not pick an ambiguous named target symbol', () => { expect(resolveTargetSymbolId( relation({ metadata: { importedName: 'target' } }), diff --git a/packages/core/tests/cli/graphControls.test.ts b/packages/core/tests/cli/graphControls.test.ts index 9adf0c7114..7be198c77e 100644 --- a/packages/core/tests/cli/graphControls.test.ts +++ b/packages/core/tests/cli/graphControls.test.ts @@ -31,6 +31,26 @@ describe('cli graph controls', () => { '-C', workspace, 'scope', 'edge', 'call', 'off', ], { stdout, stderr })).resolves.toBe(0); + const nodeResult = JSON.parse(stdout.mock.calls[0][0]); + expect(nodeResult.data).toMatchObject({ + complete: false, + nodes: expect.arrayContaining([ + { type: 'symbol', enabled: true, available: false }, + { type: 'symbol:callable', enabled: true, available: false }, + { type: 'symbol:function', enabled: true, available: false }, + { type: 'symbol:method', enabled: true, available: false }, + ]), + edges: [], + }); + expect(nodeResult.data.nodes).toHaveLength(4); + + const edgeResult = JSON.parse(stdout.mock.calls[1][0]); + expect(edgeResult.data).toMatchObject({ + complete: false, + nodes: [], + edges: [{ type: 'call', enabled: false, available: false }], + }); + expect(JSON.parse(await fs.readFile(path.join(workspace, '.codegraphy/settings.json'), 'utf-8'))).toMatchObject({ extensionPanelPlacement: 'right', nodeVisibility: { @@ -50,6 +70,23 @@ describe('cli graph controls', () => { expect(stderr).not.toHaveBeenCalled(); }); + it('refuses to hide or overwrite malformed persisted settings', async () => { + const workspace = await createWorkspace(); + const settingsPath = path.join(workspace, '.codegraphy/settings.json'); + await fs.writeFile(settingsPath, '{ malformed'); + const before = await fs.readFile(settingsPath, 'utf8'); + const stderr = vi.fn(); + + await expect(runCli(['-C', workspace, 'filter'], { stderr })).resolves.toBe(1); + await expect(runCli(['-C', workspace, 'filter', 'add', '**/generated/**'], { stderr })).resolves.toBe(1); + + expect(stderr).toHaveBeenCalledTimes(2); + expect(JSON.parse(stderr.mock.calls[0][0])).toMatchObject({ + error: { code: 'invalid_workspace_settings', details: { path: settingsPath } }, + }); + await expect(fs.readFile(settingsPath, 'utf8')).resolves.toBe(before); + }); + it('lists discoverable scope and mutates filter patterns idempotently', async () => { const workspace = await createWorkspace(); const outputs: string[] = []; @@ -64,6 +101,7 @@ describe('cli graph controls', () => { expect(settings.filterPatterns).toEqual(['**/generated/**']); expect(JSON.parse(outputs.at(-1) ?? '')).toMatchObject({ data: { + complete: true, nodes: expect.arrayContaining([ { type: 'file', enabled: true, available: true }, { type: 'symbol:function', enabled: false, available: false }, diff --git a/packages/core/tests/cli/help/command.test.ts b/packages/core/tests/cli/help/command.test.ts index 767afb5909..974fb2ee7a 100644 --- a/packages/core/tests/cli/help/command.test.ts +++ b/packages/core/tests/cli/help/command.test.ts @@ -7,7 +7,11 @@ describe('cli/help/command', () => { expect(result.exitCode).toBe(0); expect(result.output).toContain('codegraphy doctor'); - expect(result.output).toContain('codegraphy search '); + expect(result.output).not.toContain('codegraphy batch'); + expect(result.output).toContain('codegraphy settings'); + expect(result.output).toContain('codegraphy search '); + expect(result.output).toContain('codegraphy map '); + expect(result.output).toContain('codegraphy query '); expect(result.output).toContain('codegraphy dependencies '); expect(result.output).toContain('codegraphy path '); expect(result.output).toContain('codegraphy scope node '); @@ -27,9 +31,10 @@ describe('cli/help/command', () => { expect(output).toContain('1. Configure graph contributors with Plugins when needed.'); expect(output).toContain('2. Index the complete workspace graph into its Graph Cache.'); - expect(output).toContain('3. Shape returned results with Filters and Graph Scope.'); - expect(output).toContain('4. Query the resulting graph.'); + expect(output).toContain('3. Discover source and AST Symbols with search; inspect an exact result with query.'); + expect(output).toContain('4. Continue through the shaped graph with dependencies, dependents, or path.'); expect(output).toContain('codegraphy index Create or update the Graph Cache'); + expect(output).toContain('codegraphy settings set '); expect(output).toContain('codegraphy filter Read or change persisted Filters'); expect(output).toContain('codegraphy dependencies List outgoing Relationships'); expect(output).toContain('Exit status: 0 success, 1 operational failure, 2 invalid invocation.'); @@ -38,8 +43,21 @@ describe('cli/help/command', () => { it('reports local pagination options for bounded list queries', () => { expect(createHelpResult(['status']).output).toContain('Usage: codegraphy status'); + expect(createHelpResult(['query']).output).toContain('Usage: codegraphy query '); + expect(createHelpResult(['query']).output).toContain('declared AST Symbols'); + expect(createHelpResult(['query']).output).toContain('incoming and outgoing Relationships'); + expect(createHelpResult(['settings']).output).toContain('settings set maxFiles 2500'); + expect(createHelpResult(['settings']).output).toContain('Supported keys: maxFiles, include, respectGitignore'); + expect(createHelpResult(['settings']).output).toContain('never replace malformed settings with defaults'); expect(createHelpResult(['nodes']).output).toContain('Usage: codegraphy nodes'); expect(createHelpResult(['search']).output).toContain('Usage: codegraphy search'); + expect(createHelpResult(['search']).output).toContain('live source lines'); + expect(createHelpResult(['search']).output).toContain('cached AST Symbols'); + expect(createHelpResult(['search']).output).toContain('`*` wildcard'); + expect(createHelpResult(['search']).output).toContain('all-term path/source ranking'); + expect(createHelpResult(['map']).output).toContain('Usage: codegraphy map'); + expect(createHelpResult(['map']).output).toContain('personalized File map'); + expect(createHelpResult(['map']).output).toContain('typed Relationships'); expect(createHelpResult(['dependencies']).output).toContain('Usage: codegraphy dependencies'); expect(createHelpResult(['dependencies']).output).toContain('--limit '); expect(createHelpResult(['dependencies']).output).toContain('--offset '); @@ -47,7 +65,7 @@ describe('cli/help/command', () => { }); it('documents one-off query projections without implying settings changes', () => { - for (const command of ['nodes', 'search', 'edges', 'dependencies', 'dependents', 'path']) { + for (const command of ['nodes', 'edges', 'dependencies', 'dependents', 'path']) { const output = createHelpResult([command]).output; expect(output).toContain('--filter '); expect(output).toContain('--node-type '); @@ -66,7 +84,8 @@ describe('cli/help/command', () => { const indexHelp = createHelpResult(['index']).output; expect(indexHelp).toContain('Create or update the workspace Graph Cache.'); expect(indexHelp).toContain('Effects: Writes .codegraphy/graph.sqlite'); - expect(indexHelp).toContain('Output: JSON indexing summary.'); + expect(indexHelp).toContain('discovery.limitReached'); + expect(indexHelp).toContain('actionable maxFiles command'); expect(indexHelp).toContain('Example: codegraphy index'); const filterHelp = createHelpResult(['filter']).output; diff --git a/packages/core/tests/cli/index/command.test.ts b/packages/core/tests/cli/index/command.test.ts index 55332b13cf..1f6470d335 100644 --- a/packages/core/tests/cli/index/command.test.ts +++ b/packages/core/tests/cli/index/command.test.ts @@ -2,7 +2,7 @@ import { describe, expect, it, vi } from 'vitest'; import { runIndexCommand } from '../../../src/cli/index/command'; describe('index/command', () => { - it('indexes the current workspace by default and prints wait feedback', async () => { + it('keeps non-verbose stderr clean while indexing the current workspace', async () => { let finishIndex: (() => void) | undefined; const writeStatus = vi.fn(); const indexing = runIndexCommand(undefined, { @@ -17,6 +17,7 @@ describe('index/command', () => { workspaceRoot: '/workspace/project', graphCache: '.codegraphy/graph.sqlite', message: 'CodeGraphy indexing completed. CLI queries can now read the Graph Cache.', + discovery: { indexedFiles: 2, totalFound: 2, limitReached: false }, indexing: { mode: 'full', analyzedFiles: 2, deletedFiles: 0, reusedFiles: 0 }, }; }, @@ -25,9 +26,7 @@ describe('index/command', () => { await Promise.resolve(); - expect(writeStatus).toHaveBeenCalledWith( - 'Indexing /workspace/project...', - ); + expect(writeStatus).not.toHaveBeenCalled(); finishIndex?.(); await expect(indexing).resolves.toMatchObject({ exitCode: 0 }); @@ -40,6 +39,7 @@ describe('index/command', () => { workspaceRoot: workspacePath ?? '/workspace/project', graphCache: '.codegraphy/graph.sqlite', message: 'indexed', + discovery: { indexedFiles: 0, totalFound: 0, limitReached: false }, indexing: { mode: 'full', analyzedFiles: 0, deletedFiles: 0, reusedFiles: 0 }, }), writeStatus: vi.fn(), @@ -52,6 +52,7 @@ describe('index/command', () => { it('passes verbose diagnostics to the workspace indexing request', async () => { const diagnosticAreas: string[] = []; + const writeStatus = vi.fn(); await runIndexCommand('/workspace/other', { cwd: () => '/workspace/project', @@ -65,13 +66,15 @@ describe('index/command', () => { workspaceRoot: '/workspace/other', graphCache: '.codegraphy/graph.sqlite', message: 'indexed', + discovery: { indexedFiles: 2, totalFound: 2, limitReached: false }, indexing: { mode: 'full', analyzedFiles: 2, deletedFiles: 0, reusedFiles: 0 }, }; }, writeDiagnostic: line => diagnosticAreas.push(line), - writeStatus: vi.fn(), + writeStatus, }, { verbose: true }); + expect(writeStatus).toHaveBeenCalledWith('Indexing /workspace/other...'); expect(diagnosticAreas).toEqual([ '[CodeGraphy] Indexing complete: 2 files, operation=index-1', ]); diff --git a/packages/core/tests/cli/parse.test.ts b/packages/core/tests/cli/parse.test.ts index 45f68cd93c..e892c176cb 100644 --- a/packages/core/tests/cli/parse.test.ts +++ b/packages/core/tests/cli/parse.test.ts @@ -10,6 +10,9 @@ describe('cli/parse', () => { expect(parseCliCommand(['--version'])).toEqual({ name: 'version' }); expect(parseCliCommand(['-V'])).toEqual({ name: 'version' }); expect(parseCliCommand(['--', 'index'])).toEqual({ name: 'index' }); + expect(parseCliCommand(['batch'])).toMatchObject({ + parseError: 'Unknown command: batch', + }); }); it('uses cwd by default and one global workspace override for every workspace command', () => { @@ -52,8 +55,40 @@ describe('cli/parse', () => { arguments: { action: 'add', pattern: '-draft/**' }, }); expect(parseCliCommand(['search', '--', '-C'])).toMatchObject({ - invokedCommand: 'search', - arguments: { search: '-C', limit: 100 }, + report: 'search', + arguments: { pattern: '-C', limit: 20 }, + }); + }); + + it('parses structured workspace settings reads and mutations', () => { + expect(parseCliCommand(['settings'])).toEqual({ name: 'settings' }); + expect(parseCliCommand(['settings', 'get', 'maxFiles'])).toEqual({ + name: 'settings', + settingsAction: 'get', + settingsKey: 'maxFiles', + }); + expect(parseCliCommand(['settings', 'set', 'maxFiles', '2500'])).toEqual({ + name: 'settings', + settingsAction: 'set', + settingsKey: 'maxFiles', + settingsValue: 2500, + }); + expect(parseCliCommand(['settings', 'set', 'include', '["packages/**/*.ts"]'])).toEqual({ + name: 'settings', + settingsAction: 'set', + settingsKey: 'include', + settingsValue: ['packages/**/*.ts'], + }); + expect(parseCliCommand(['settings', 'unset', 'maxFiles'])).toEqual({ + name: 'settings', + settingsAction: 'unset', + settingsKey: 'maxFiles', + }); + expect(parseCliCommand(['settings', 'set', 'maxFiles', 'many'])).toMatchObject({ + parseError: 'settings set value must be valid JSON: many', + }); + expect(parseCliCommand(['settings', 'set', 'unknown', '1'])).toMatchObject({ + parseError: 'Unknown workspace setting: unknown', }); }); @@ -62,6 +97,10 @@ describe('cli/parse', () => { name: 'help', helpPath: ['dependencies'], }); + expect(parseCliCommand(['settings', '--help'])).toEqual({ + name: 'help', + helpPath: ['settings'], + }); expect(parseCliCommand(['scope', '--help'])).toEqual({ name: 'help', helpPath: ['scope'], @@ -70,6 +109,14 @@ describe('cli/parse', () => { name: 'help', helpPath: ['plugins', 'enable'], }); + expect(parseCliCommand(['query', '--help'])).toEqual({ + name: 'help', + helpPath: ['query'], + }); + expect(parseCliCommand(['help', 'query'])).toEqual({ + name: 'help', + helpPath: ['query'], + }); }); it('rejects per-command workspaces, query flags, retired commands, and malformed globals', () => { @@ -79,7 +126,7 @@ describe('cli/parse', () => { expect(parseCliCommand(['edges', '--from', 'src/app.ts'])).toMatchObject({ parseError: 'Unknown option for edges: --from', }); - for (const command of ['setup', 'relationships', 'symbols', 'paths', 'query']) { + for (const command of ['setup', 'relationships', 'symbols', 'paths', 'batch']) { expect(parseCliCommand([command])).toMatchObject({ parseError: `Unknown command: ${command}`, }); diff --git a/packages/core/tests/cli/parsePlugins.test.ts b/packages/core/tests/cli/parsePlugins.test.ts index eaf361cb44..7998169327 100644 --- a/packages/core/tests/cli/parsePlugins.test.ts +++ b/packages/core/tests/cli/parsePlugins.test.ts @@ -37,6 +37,23 @@ describe('cli/parsePlugins', () => { action: 'inherit', packageName: 'codegraphy.particles', }); + expect(parsePluginsCommand(['enable', '--', '--global'])).toEqual({ + name: 'plugins', + action: 'enable', + packageName: '--global', + }); + expect(parsePluginsCommand(['enable', '--global', '--', 'codegraphy.vue'])).toEqual({ + name: 'plugins', + action: 'enable', + packageName: 'codegraphy.vue', + pluginScope: 'global', + }); + expect(parsePluginsCommand(['enable', '--global', '--', '--global'])).toEqual({ + name: 'plugins', + action: 'enable', + packageName: '--global', + pluginScope: 'global', + }); }); it('routes an empty group to plugin help', () => { diff --git a/packages/core/tests/cli/parseQuery.test.ts b/packages/core/tests/cli/parseQuery.test.ts index 8eec1b6223..632284cf97 100644 --- a/packages/core/tests/cli/parseQuery.test.ts +++ b/packages/core/tests/cli/parseQuery.test.ts @@ -18,9 +18,20 @@ describe('cli/parseQuery', () => { it('maps intent commands to graph reports with positional operands', () => { expect(parseQueryCommand(['search', 'render settings'])).toEqual({ name: 'query', - invokedCommand: 'search', - report: 'nodes', - arguments: { search: 'render settings', limit: 100 }, + report: 'search', + arguments: { pattern: 'render settings', limit: 20 }, + }); + expect(parseQueryCommand(['map', 'render settings'])).toEqual({ + name: 'query', + invokedCommand: 'map', + report: 'task-map', + arguments: { query: 'render settings', limit: 8 }, + }); + expect(parseQueryCommand(['query', 'src/cli/command.ts'])).toEqual({ + name: 'query', + invokedCommand: 'query', + report: 'overview', + arguments: { target: 'src/cli/command.ts' }, }); expect(parseQueryCommand(['dependencies', 'src/app.ts'])).toEqual({ name: 'query', @@ -51,12 +62,12 @@ describe('cli/parseQuery', () => { it('parses local pagination options and the end-of-options delimiter', () => { expect(parseQueryCommand(['search', '--limit', '5', '--offset', '10', 'render'])).toMatchObject({ - invokedCommand: 'search', - arguments: { search: 'render', limit: 5, offset: 10 }, + report: 'search', + arguments: { pattern: 'render', limit: 5, offset: 10 }, }); expect(parseQueryCommand(['search', '--', '-generated'])).toMatchObject({ - invokedCommand: 'search', - arguments: { search: '-generated', limit: 100 }, + report: 'search', + arguments: { pattern: '-generated', limit: 20 }, }); }); @@ -91,6 +102,10 @@ describe('cli/parseQuery', () => { expect(parseQueryCommand(['nodes', '--limit', '0'])).toMatchObject({ parseError: '--limit requires a positive integer', }); + expect(parseQueryCommand(['map', 'task', '--limit', '21'])).toMatchObject({ + invokedCommand: 'map', + parseError: 'map --limit must be between 1 and 20', + }); expect(parseQueryCommand(['nodes', '--offset', '-1'])).toMatchObject({ parseError: '--offset requires a non-negative integer', }); @@ -108,7 +123,19 @@ describe('cli/parseQuery', () => { }); expect(parseQueryCommand(['search'])).toMatchObject({ invokedCommand: 'search', - parseError: 'search requires ', + parseError: 'search requires ', + }); + expect(parseQueryCommand(['search', '*'])).toMatchObject({ + invokedCommand: 'search', + parseError: 'search pattern must contain a literal character', + }); + expect(parseQueryCommand(['map'])).toMatchObject({ + invokedCommand: 'map', + parseError: 'map requires ', + }); + expect(parseQueryCommand(['query'])).toMatchObject({ + invokedCommand: 'query', + parseError: 'query requires ', }); expect(parseQueryCommand(['dependencies', 'a.ts', 'b.ts'])).toMatchObject({ parseError: 'Unexpected argument for dependencies: b.ts', diff --git a/packages/core/tests/cli/parseSettings.test.ts b/packages/core/tests/cli/parseSettings.test.ts new file mode 100644 index 0000000000..e3edf67348 --- /dev/null +++ b/packages/core/tests/cli/parseSettings.test.ts @@ -0,0 +1,26 @@ +import { describe, expect, it } from 'vitest'; +import { parseSettingsCommand } from '../../src/cli/parseSettings'; + +describe('cli/parseSettings', () => { + it('parses reads and JSON mutations', () => { + expect(parseSettingsCommand([])).toEqual({ name: 'settings' }); + expect(parseSettingsCommand(['get', 'maxFiles'])).toMatchObject({ + name: 'settings', settingsAction: 'get', settingsKey: 'maxFiles', + }); + expect(parseSettingsCommand(['set', 'include', '["src/**/*.ts"]'])).toMatchObject({ + name: 'settings', settingsAction: 'set', settingsKey: 'include', settingsValue: ['src/**/*.ts'], + }); + expect(parseSettingsCommand(['unset', 'maxFiles'])).toMatchObject({ + name: 'settings', settingsAction: 'unset', settingsKey: 'maxFiles', + }); + }); + + it('rejects unknown keys and malformed JSON values', () => { + expect(parseSettingsCommand(['set', 'unknown', '1'])).toMatchObject({ + parseError: 'Unknown workspace setting: unknown', + }); + expect(parseSettingsCommand(['set', 'maxFiles', 'many'])).toMatchObject({ + parseError: 'settings set value must be valid JSON: many', + }); + }); +}); diff --git a/packages/core/tests/cli/plugins/commandWorkspaceState.test.ts b/packages/core/tests/cli/plugins/commandWorkspaceState.test.ts index eb7ffab16b..b5a9772960 100644 --- a/packages/core/tests/cli/plugins/commandWorkspaceState.test.ts +++ b/packages/core/tests/cli/plugins/commandWorkspaceState.test.ts @@ -145,6 +145,52 @@ describe('plugins/command workspace state', () => { ]); }); + it('returns structured plugin activity for automation', async () => { + const homeDir = await fs.mkdtemp(path.join(os.tmpdir(), 'codegraphy-user-home-')); + const workspaceRoot = await fs.mkdtemp(path.join(os.tmpdir(), 'codegraphy-workspace-plugin-list-')); + const vue = createPluginRecord( + '@codegraphy-dev/plugin-vue', + '/global/@codegraphy-dev/plugin-vue', + 'codegraphy.vue', + ); + writeCodeGraphyInstalledPluginCache({ version: 3, plugins: [vue] }, { homeDir }); + await runPluginsCommand({ + name: 'plugins', + action: 'enable', + packageName: 'codegraphy.vue', + workspacePath: workspaceRoot, + }, { homeDir }); + + const result = await runPluginsCommand({ + name: 'plugins', + action: 'list', + workspacePath: workspaceRoot, + }, { homeDir }); + + expect(JSON.parse(result.output)).toEqual({ + workspaceRoot, + plugins: [ + { + id: CODEGRAPHY_MARKDOWN_PLUGIN_ID, + package: CODEGRAPHY_MARKDOWN_PLUGIN_PACKAGE_NAME, + host: 'core', + globallyEnabled: true, + workspaceActivation: 'inherit', + state: 'active', + }, + { + id: 'codegraphy.vue', + package: '@codegraphy-dev/plugin-vue', + host: 'core', + globallyEnabled: false, + workspaceActivation: 'enabled', + state: 'active', + }, + ], + warnings: [], + }); + }); + it('lists disabled bundled Markdown without requiring it in the user installed plugin cache', async () => { const homeDir = await fs.mkdtemp(path.join(os.tmpdir(), 'codegraphy-user-home-')); const workspaceRoot = await fs.mkdtemp(path.join(os.tmpdir(), 'codegraphy-workspace-markdown-')); @@ -162,9 +208,14 @@ describe('plugins/command workspace state', () => { workspacePath: workspaceRoot, }, { homeDir }); - expect(result.output).toContain('Enabled in workspace:\nnone'); - expect(result.output).toContain('Registered but disabled:'); - expect(result.output).toContain(`- ${CODEGRAPHY_MARKDOWN_PLUGIN_ID}`); + expect(JSON.parse(result.output)).toMatchObject({ + plugins: [{ + id: CODEGRAPHY_MARKDOWN_PLUGIN_ID, + workspaceActivation: 'disabled', + state: 'disabled', + }], + warnings: [], + }); }); it('lists enabled workspace plugins separately from registered disabled plugins', async () => { @@ -201,12 +252,14 @@ describe('plugins/command workspace state', () => { workspacePath: workspaceRoot, }, { homeDir }); - expect(result.output).toContain(`CodeGraphy plugins for ${workspaceRoot}`); - expect(result.output).toContain('Enabled in workspace:'); - expect(result.output).toContain(`1. ${CODEGRAPHY_MARKDOWN_PLUGIN_ID}`); - expect(result.output).toContain('2. codegraphy.vue'); - expect(result.output).toContain('Registered but disabled:'); - expect(result.output).not.toContain(`- ${CODEGRAPHY_MARKDOWN_PLUGIN_ID}`); + expect(JSON.parse(result.output)).toMatchObject({ + workspaceRoot, + plugins: [ + { id: CODEGRAPHY_MARKDOWN_PLUGIN_ID, state: 'active' }, + { id: 'codegraphy.vue', state: 'active' }, + ], + warnings: [], + }); }); it('lists enabled conflicting descriptors as unavailable instead of disabled', async () => { @@ -231,8 +284,21 @@ describe('plugins/command workspace state', () => { workspacePath: workspaceRoot, }, { homeDir }); - expect(result.output).toContain('Enabled but unavailable:'); - expect(result.output).toContain('- acme.conflict'); - expect(result.output).not.toContain('Registered but disabled:\n- acme.conflict'); + const output = JSON.parse(result.output); + expect(output.plugins).toEqual(expect.arrayContaining([ + expect.objectContaining({ + id: 'acme.conflict', + package: '@acme/plugin-one', + state: 'enabled-unavailable', + }), + expect.objectContaining({ + id: 'acme.conflict', + package: '@acme/plugin-two', + state: 'enabled-unavailable', + }), + ])); + expect(output.warnings).toEqual([ + expect.stringContaining("CodeGraphy plugin 'acme.conflict' is enabled but multiple installed packages claim it"), + ]); }); }); diff --git a/packages/core/tests/cli/query/command.test.ts b/packages/core/tests/cli/query/command.test.ts index 65dc3b0cc9..a2f4a30cf6 100644 --- a/packages/core/tests/cli/query/command.test.ts +++ b/packages/core/tests/cli/query/command.test.ts @@ -30,6 +30,30 @@ describe('cli/query/command', () => { }); }); + it('normalizes an absolute query target to its workspace-relative Node id', async () => { + const workspaceRoot = await fs.mkdtemp(path.join(os.tmpdir(), 'codegraphy-query-target-')); + const filePath = path.join(workspaceRoot, 'src', 'command.ts'); + await fs.mkdir(path.dirname(filePath)); + await fs.writeFile(filePath, 'export const command = true;\n'); + let receivedInput: unknown; + + await runQueryCommand({ + name: 'query', + invokedCommand: 'query', + report: 'overview', + workspacePath: workspaceRoot, + arguments: { target: filePath }, + }, { + cwd: () => workspaceRoot, + query: async (input) => { + receivedInput = input; + return {}; + }, + }); + + expect(receivedInput).toMatchObject({ arguments: { target: 'src/command.ts' } }); + }); + it('accepts an in-workspace path whose segment starts with two dots', async () => { const workspaceRoot = await fs.mkdtemp(path.join(os.tmpdir(), 'codegraphy-query-segment-')); const selector = path.join(workspaceRoot, '..cache', 'entry.ts'); diff --git a/packages/core/tests/cli/result/parser.test.ts b/packages/core/tests/cli/result/parser.test.ts index 608784ddc4..aa9652d368 100644 --- a/packages/core/tests/cli/result/parser.test.ts +++ b/packages/core/tests/cli/result/parser.test.ts @@ -21,11 +21,17 @@ describe('cli/result/parser', () => { action: 'Retry.', }); expect(readCliError({ - error: { code: 'bad', message: 'Nested failure.', action: 'Repair settings.' }, + error: { + code: 'bad', + message: 'Nested failure.', + action: 'Repair settings.', + details: { id: 'query-1' }, + }, })).toEqual({ code: 'bad', message: 'Nested failure.', action: 'Repair settings.', + details: { id: 'query-1' }, }); }); }); diff --git a/packages/core/tests/cli/settings/command.test.ts b/packages/core/tests/cli/settings/command.test.ts new file mode 100644 index 0000000000..eb863d560b --- /dev/null +++ b/packages/core/tests/cli/settings/command.test.ts @@ -0,0 +1,102 @@ +import * as fs from 'node:fs/promises'; +import * as os from 'node:os'; +import * as path from 'node:path'; +import { describe, expect, it, vi } from 'vitest'; +import { runCli } from '../../../src/cli/run'; + +async function createWorkspace(settings: unknown): Promise { + const workspace = await fs.mkdtemp(path.join(os.tmpdir(), 'codegraphy-cli-settings-')); + await fs.mkdir(path.join(workspace, '.codegraphy'), { recursive: true }); + await fs.writeFile( + path.join(workspace, '.codegraphy/settings.json'), + typeof settings === 'string' ? settings : `${JSON.stringify(settings, null, 2)}\n`, + ); + return workspace; +} + +describe('cli/settings command', () => { + it('reads all effective settings or one requested value', async () => { + const workspace = await createWorkspace({ version: 1, maxFiles: 1200, futureSetting: 'preserved' }); + const outputs: string[] = []; + const stdout = (output: string): void => { outputs.push(output); }; + + await expect(runCli(['-C', workspace, 'settings'], { stdout })).resolves.toBe(0); + await expect(runCli(['-C', workspace, 'settings', 'get', 'maxFiles'], { stdout })).resolves.toBe(0); + + expect(JSON.parse(outputs[0])).toMatchObject({ + command: 'settings', + data: { + workspaceRoot: workspace, + settings: { maxFiles: 1200, respectGitignore: true }, + }, + }); + expect(JSON.parse(outputs[1])).toMatchObject({ + command: 'settings', + data: { workspaceRoot: workspace, key: 'maxFiles', value: 1200 }, + }); + }); + + it('sets and unsets a validated setting without losing unknown fields', async () => { + const workspace = await createWorkspace({ version: 1, maxFiles: 1200, futureSetting: 'preserved' }); + const outputs: string[] = []; + const stdout = (output: string): void => { outputs.push(output); }; + + await expect(runCli(['-C', workspace, 'settings', 'set', 'maxFiles', '2500'], { stdout })).resolves.toBe(0); + expect(JSON.parse(outputs[0])).toMatchObject({ + data: { + key: 'maxFiles', + previous: 1200, + value: 2500, + indexRequired: true, + }, + }); + expect(JSON.parse(await fs.readFile(path.join(workspace, '.codegraphy/settings.json'), 'utf8'))).toMatchObject({ + maxFiles: 2500, + futureSetting: 'preserved', + }); + + await expect(runCli(['-C', workspace, 'settings', 'unset', 'maxFiles'], { stdout })).resolves.toBe(0); + expect(JSON.parse(outputs[1])).toMatchObject({ + data: { key: 'maxFiles', previous: 2500, value: 1000, indexRequired: true }, + }); + const raw = JSON.parse(await fs.readFile(path.join(workspace, '.codegraphy/settings.json'), 'utf8')); + expect(raw).not.toHaveProperty('maxFiles'); + expect(raw.futureSetting).toBe('preserved'); + }); + + it('rejects invalid values and leaves the persisted bytes unchanged', async () => { + const workspace = await createWorkspace({ version: 1, maxFiles: 1200 }); + const settingsPath = path.join(workspace, '.codegraphy/settings.json'); + const before = await fs.readFile(settingsPath, 'utf8'); + const stderr = vi.fn(); + + await expect(runCli(['-C', workspace, 'settings', 'set', 'maxFiles', '-1'], { stderr })).resolves.toBe(1); + + expect(JSON.parse(stderr.mock.calls[0][0])).toMatchObject({ + error: { + code: 'invalid_workspace_settings', + message: 'maxFiles must be a positive integer', + details: { path: settingsPath }, + }, + }); + await expect(fs.readFile(settingsPath, 'utf8')).resolves.toBe(before); + }); + + it('does not overwrite malformed persisted settings', async () => { + const workspace = await createWorkspace('{ malformed'); + const settingsPath = path.join(workspace, '.codegraphy/settings.json'); + const before = await fs.readFile(settingsPath, 'utf8'); + const stderr = vi.fn(); + + await expect(runCli(['-C', workspace, 'settings', 'set', 'maxFiles', '2500'], { stderr })).resolves.toBe(1); + + expect(JSON.parse(stderr.mock.calls[0][0])).toMatchObject({ + error: { + code: 'invalid_workspace_settings', + message: expect.any(String), + details: { path: settingsPath }, + }, + }); + await expect(fs.readFile(settingsPath, 'utf8')).resolves.toBe(before); + }); +}); diff --git a/packages/core/tests/graphQuery/overview.test.ts b/packages/core/tests/graphQuery/overview.test.ts new file mode 100644 index 0000000000..c3169975c1 --- /dev/null +++ b/packages/core/tests/graphQuery/overview.test.ts @@ -0,0 +1,154 @@ +import { describe, expect, it } from 'vitest'; +import type { IAnalysisRelation, IAnalysisSymbol } from '@codegraphy-dev/plugin-api'; +import type { IGraphData } from '../../src/graph/contracts'; +import { inspectGraphTarget } from '../../src/graphQuery/overview'; + +const graphData: IGraphData = { + nodes: [ + { id: 'src/command.ts', label: 'command.ts', nodeType: 'file' }, + { id: 'src/settings.ts', label: 'settings.ts', nodeType: 'file' }, + { id: 'tests/command.test.ts', label: 'command.test.ts', nodeType: 'file' }, + { + id: 'src/command.ts#runCommand:function', + label: 'runCommand', + nodeType: 'symbol:function', + symbol: { + id: 'src/command.ts#runCommand:function', + filePath: 'src/command.ts', + name: 'runCommand', + kind: 'function', + }, + }, + ], + edges: [ + { id: 'command-settings', from: 'src/command.ts', to: 'src/settings.ts', kind: 'import', sources: [] }, + { id: 'test-command', from: 'tests/command.test.ts', to: 'src/command.ts', kind: 'import', sources: [] }, + { + id: 'command-symbol', + from: 'src/command.ts', + to: 'src/command.ts#runCommand:function', + kind: 'contains', + sources: [], + }, + ], +}; + +const symbols: IAnalysisSymbol[] = [{ + id: 'src/command.ts#runCommand:function', + filePath: 'src/command.ts', + name: 'runCommand', + kind: 'function', +}]; + +const relations: IAnalysisRelation[] = [ + { + kind: 'import', + sourceId: 'core:typescript:import', + fromFilePath: 'src/command.ts', + toFilePath: 'src/settings.ts', + }, + { + kind: 'import', + sourceId: 'core:typescript:import', + fromFilePath: 'tests/command.test.ts', + toFilePath: 'src/command.ts', + }, +]; + +describe('core/graphQuery target overview', () => { + it('returns declarations plus incoming and outgoing Relationships for one exact target', () => { + const result = inspectGraphTarget({ graphData, symbols, relations }, { target: 'src/command.ts' }); + + expect(result).toMatchObject({ + target: { path: 'src/command.ts', nodeType: 'file' }, + declaredSymbols: { + symbols: [{ + id: 'src/command.ts#runCommand:function', + filePath: 'src/command.ts', + name: 'runCommand', + kind: 'function', + }], + page: { returned: 1, total: 1, nextOffset: null }, + }, + outgoing: { + edges: [{ from: 'src/command.ts', to: 'src/settings.ts', edgeTypes: ['import'] }], + page: { returned: 1, total: 1, nextOffset: null }, + }, + incoming: { + edges: [{ from: 'tests/command.test.ts', to: 'src/command.ts', edgeTypes: ['import'] }], + page: { returned: 1, total: 1, nextOffset: null }, + }, + limits: { declaredSymbols: 25, relationshipsPerDirection: 25 }, + }); + }); + + it('prioritizes callable and type declarations over local constants', () => { + const crowdedSymbols: IAnalysisSymbol[] = [ + ...Array.from({ length: 30 }, (_, index) => ({ + id: `src/settings.ts#aValue${index}:constant`, + filePath: 'src/settings.ts', + name: `aValue${String(index).padStart(2, '0')}`, + kind: 'constant', + })), + { + id: 'src/settings.ts#readSettings:function', + filePath: 'src/settings.ts', + name: 'readSettings', + kind: 'function', + }, + ]; + + const result = inspectGraphTarget( + { graphData, symbols: crowdedSymbols, relations }, + { target: 'src/settings.ts' }, + ); + + expect('declaredSymbols' in result ? result.declaredSymbols.symbols[0] : undefined).toMatchObject({ + name: 'readSettings', + kind: 'function', + }); + expect('declaredSymbols' in result ? result.declaredSymbols.page : undefined).toMatchObject({ + returned: 25, + total: 31, + nextOffset: 25, + }); + }); + + it('inspects an exact cached Symbol with its containing File provenance', () => { + const result = inspectGraphTarget( + { + graphData, + symbols: [...symbols, { + id: 'src/settings.ts#readSettings:function', + filePath: 'src/settings.ts', + name: 'readSettings', + kind: 'function', + range: { startLine: 2, endLine: 4 }, + }], + relations, + }, + { target: 'src/settings.ts#readSettings:function' }, + ); + + expect(result).toMatchObject({ + target: { + path: 'src/settings.ts#readSettings:function', + nodeType: 'symbol:function', + symbol: { + id: 'src/settings.ts#readSettings:function', + name: 'readSettings', + kind: 'function', + filePath: 'src/settings.ts', + }, + }, + declaredSymbols: { symbols: [] }, + }); + }); + + it('returns a typed error when the exact target is absent', () => { + expect(inspectGraphTarget({ graphData, symbols, relations }, { target: 'src/missing.ts' })).toEqual({ + error: 'query_target_not_found', + message: 'No indexed Node or Symbol has the exact id: src/missing.ts', + }); + }); +}); diff --git a/packages/core/tests/graphQuery/paths.test.ts b/packages/core/tests/graphQuery/paths.test.ts index 7abf6432a7..f6802662ed 100644 --- a/packages/core/tests/graphQuery/paths.test.ts +++ b/packages/core/tests/graphQuery/paths.test.ts @@ -137,6 +137,35 @@ describe('core/graphQuery paths report', () => { }); }); + it('prefers an exact File endpoint before expanding its Symbols', () => { + const mixedGraph: IGraphData = { + nodes: [ + { id: 'a.ts', label: 'a.ts', nodeType: 'file' }, + { id: 'a.ts#run:function', label: 'run', nodeType: 'symbol:function', symbol: { + id: 'a.ts#run:function', filePath: 'a.ts', name: 'run', kind: 'function', + } }, + { id: 'z.ts', label: 'z.ts', nodeType: 'file' }, + { id: 'z.ts#target:function', label: 'target', nodeType: 'symbol:function', symbol: { + id: 'z.ts#target:function', filePath: 'z.ts', name: 'target', kind: 'function', + } }, + ], + edges: [ + { id: 'run-z', from: 'a.ts#run:function', to: 'z.ts', kind: 'call', sources: [] }, + { id: 'z-target', from: 'z.ts', to: 'z.ts#target:function', kind: 'contains', sources: [] }, + ], + }; + + expect(findGraphPaths(mixedGraph, { + from: 'a.ts#run:function', + to: 'z.ts', + expandFileSelectors: true, + projectFileEndpoints: true, + })).toMatchObject({ + paths: [['a.ts#run:function', 'z.ts']], + complete: true, + }); + }); + it('keeps collecting raw symbol routes until projected file paths are unique', () => { const symbolNode = (id: string, filePath: string) => ({ id, diff --git a/packages/core/tests/graphQuery/search.test.ts b/packages/core/tests/graphQuery/search.test.ts new file mode 100644 index 0000000000..64fa38f9b7 --- /dev/null +++ b/packages/core/tests/graphQuery/search.test.ts @@ -0,0 +1,197 @@ +import { describe, expect, it } from 'vitest'; +import type { IAnalysisSymbol } from '@codegraphy-dev/plugin-api'; +import type { IGraphData } from '../../src/graph/contracts'; +import { searchGraph } from '../../src/graphQuery/search'; + +const graphData: IGraphData = { + nodes: [ + { id: 'src/index.ts', label: 'index.ts', nodeType: 'file' }, + { id: 'tests/index.test.ts', label: 'index.test.ts', nodeType: 'file' }, + { + id: 'src/index.ts#runIndexCommand:function', + label: 'runIndexCommand', + nodeType: 'symbol:function', + symbol: { + id: 'src/index.ts#runIndexCommand:function', + filePath: 'src/index.ts', + name: 'runIndexCommand', + kind: 'function', + }, + }, + ], + edges: [], +}; + +const symbols: IAnalysisSymbol[] = [{ + id: 'src/index.ts#runIndexCommand:function', + filePath: 'src/index.ts', + name: 'runIndexCommand', + kind: 'function', + signature: 'async function runIndexCommand()', + metadata: { language: 'typescript', source: 'core:treesitter' }, +}]; + +function search(pattern: string) { + return searchGraph({ + graphData, + symbols, + relations: [], + sourceText: { + files: [ + { filePath: 'src/index.ts', content: 'writeStatus(`Indexing ${workspace}...`);\n' }, + { filePath: 'tests/index.test.ts', content: "it('keeps non-verbose stderr clean', () => {});\n" }, + ], + filesScanned: 2, + filesSkipped: 0, + }, + cacheState: 'fresh', + }, { pattern, limit: 20 }); +} + +describe('core/graphQuery search', () => { + it('merges cached AST Symbols with live source matches and file provenance', () => { + expect(search('runIndexCommand')).toEqual({ + pattern: 'runIndexCommand', + matches: [ + { + type: 'symbol', + symbol: { + id: 'src/index.ts#runIndexCommand:function', + filePath: 'src/index.ts', + name: 'runIndexCommand', + kind: 'function', + signature: 'async function runIndexCommand()', + language: 'typescript', + source: 'core:treesitter', + }, + }, + ], + page: { offset: 0, limit: 20, returned: 1, total: 1, nextOffset: null }, + sources: { + text: { freshness: 'live', filesScanned: 2, filesSkipped: 0 }, + symbols: { freshness: 'cached', cacheState: 'fresh' }, + }, + }); + }); + + it('finds indexed Nodes by exact path or partial label', () => { + expect(search('src/index.ts').matches[0]).toEqual({ + type: 'node', + node: { path: 'src/index.ts', nodeType: 'file' }, + }); + expect(search('index').matches).toEqual(expect.arrayContaining([{ + type: 'node', + node: { path: 'src/index.ts', nodeType: 'file' }, + }])); + }); + + it('finds exact source text with one-based locations and bounded excerpts', () => { + expect(search('Indexing ').matches).toEqual([ + { + type: 'text', + filePath: 'src/index.ts', + line: 1, + column: 14, + excerpt: 'writeStatus(`Indexing ${workspace}...`);', + }, + ]); + }); + + it('supports case-insensitive line-local star wildcards', () => { + expect(search('*stderr clean').matches).toEqual([ + { + type: 'text', + filePath: 'tests/index.test.ts', + line: 1, + column: 1, + excerpt: "it('keeps non-verbose stderr clean', () => {});", + }, + ]); + }); + + it('centers long excerpts on the matching source text', () => { + const longLine = `${'before '.repeat(30)}needle${' after'.repeat(30)}`; + const result = searchGraph({ + graphData, + sourceText: { + files: [{ filePath: 'src/long.ts', content: `${longLine}\n` }], + filesScanned: 1, + filesSkipped: 0, + }, + }, { pattern: 'needle', limit: 20 }); + const match = result.matches[0]; + + expect(match).toMatchObject({ type: 'text', filePath: 'src/long.ts' }); + expect(match && 'excerpt' in match ? match.excerpt : '').toContain('needle'); + expect(match && 'excerpt' in match ? match.excerpt.length : 0).toBeLessThanOrEqual(240); + }); + + it('falls back to all-term File ranking for natural multi-term searches', () => { + const result = searchGraph({ + graphData: { + nodes: [ + { id: 'src/cli/filter/command.ts', label: 'command.ts', nodeType: 'file' }, + { id: 'src/cli/doctor/command.ts', label: 'command.ts', nodeType: 'file' }, + { id: 'tests/cli/parse.test.ts', label: 'parse.test.ts', nodeType: 'file' }, + ], + edges: [], + }, + sourceText: { + files: [ + { + filePath: 'src/cli/filter/command.ts', + content: 'export function runFilterCommand() { return readWorkspaceSettings(); }\n', + }, + { + filePath: 'src/cli/doctor/command.ts', + content: 'export function runDoctorCommand() { return validateSettings(); }\n', + }, + { + filePath: 'tests/cli/parse.test.ts', + content: "it('parses compact scope and filter commands', () => {});\n", + }, + ], + filesScanned: 3, + filesSkipped: 0, + }, + }, { pattern: 'filter command', limit: 20 }); + + expect(result.matches[0]).toEqual({ + type: 'node', + node: { path: 'src/cli/filter/command.ts', nodeType: 'file' }, + }); + expect(result.matches).toContainEqual(expect.objectContaining({ + type: 'text', + filePath: 'tests/cli/parse.test.ts', + })); + }); + + it('keeps exact identifier Symbols ahead of natural phrase fallback candidates', () => { + expect(search('runIndexCommand').matches[0]).toMatchObject({ + type: 'symbol', + symbol: { name: 'runIndexCommand' }, + }); + }); + + it('ranks source paths related to the phrase ahead of documentation history', () => { + const result = searchGraph({ + graphData, + symbols, + relations: [], + sourceText: { + files: [ + { filePath: 'CHANGELOG.md', content: 'Changed Indexing behavior.\n' }, + { filePath: 'src/cli/index/command.ts', content: 'writeStatus(`Indexing ${workspace}...`);\n' }, + ], + filesScanned: 2, + filesSkipped: 0, + }, + }, { pattern: 'Indexing ', limit: 1 }); + + expect(result.matches).toEqual([expect.objectContaining({ + type: 'text', + filePath: 'src/cli/index/command.ts', + })]); + expect(result.page).toMatchObject({ returned: 1, total: 2, nextOffset: 1 }); + }); +}); diff --git a/packages/core/tests/graphQuery/search/ranking.test.ts b/packages/core/tests/graphQuery/search/ranking.test.ts new file mode 100644 index 0000000000..a52ed018a3 --- /dev/null +++ b/packages/core/tests/graphQuery/search/ranking.test.ts @@ -0,0 +1,36 @@ +import { describe, expect, it } from 'vitest'; +import { rankSearchDocuments } from '../../../src/graphQuery/search/ranking'; + +describe('core/graphQuery/search natural phrase ranking', () => { + it('tokenizes camelCase identifiers and prioritizes all-term path matches', () => { + const ranked = rankSearchDocuments('filter command', [ + { + id: 'parse-test', + path: 'tests/cli/parse.test.ts', + text: "it('parses compact scope and filter commands', () => {});", + }, + { + id: 'filter-command', + path: 'src/cli/filter/command.ts', + text: 'export function runFilterCommand() {}', + }, + ]); + + expect(ranked.map(result => result.id)).toEqual(['filter-command']); + }); + + it('returns no fallback ranking for a single exact-search term', () => { + expect(rankSearchDocuments('runFilterCommand', [{ + id: 'filter-command', + path: 'src/cli/filter/command.ts', + text: 'export function runFilterCommand() {}', + }])).toEqual([]); + }); + + it('ignores common natural-language stop words', () => { + expect(rankSearchDocuments('the filter command', [ + { id: 'filter', path: 'src/filter/command.ts', text: '' }, + { id: 'other', path: 'src/other.ts', text: 'the command' }, + ])[0]?.id).toBe('filter'); + }); +}); diff --git a/packages/core/tests/graphQuery/taskMap.test.ts b/packages/core/tests/graphQuery/taskMap.test.ts new file mode 100644 index 0000000000..36c53f6d32 --- /dev/null +++ b/packages/core/tests/graphQuery/taskMap.test.ts @@ -0,0 +1,139 @@ +import { describe, expect, it } from 'vitest'; +import type { GraphQueryData } from '../../src/graphQuery/data'; +import { mapGraphTask } from '../../src/graphQuery/taskMap'; + +const data: GraphQueryData = { + graphData: { + nodes: [ + { id: 'src/registry.ts', label: 'registry.ts', nodeType: 'file' }, + { id: 'src/engine.ts', label: 'engine.ts', nodeType: 'file' }, + { id: 'src/workspace.ts', label: 'workspace.ts', nodeType: 'file' }, + { id: 'src/unrelated.ts', label: 'unrelated.ts', nodeType: 'file' }, + { id: 'tests/registry.test.ts', label: 'registry.test.ts', nodeType: 'file' }, + { + id: 'src/registry.ts#unloadPlugin:function', + label: 'unloadPlugin', + nodeType: 'symbol:function', + symbol: { id: 'src/registry.ts#unloadPlugin:function', filePath: 'src/registry.ts', name: 'unloadPlugin', kind: 'function' }, + }, + { + id: 'src/engine.ts#rebuild:function', + label: 'rebuild', + nodeType: 'symbol:function', + symbol: { id: 'src/engine.ts#rebuild:function', filePath: 'src/engine.ts', name: 'rebuild', kind: 'function' }, + }, + ], + edges: [ + { id: 'engine-registry', from: 'src/engine.ts#rebuild:function', to: 'src/registry.ts#unloadPlugin:function', kind: 'call', sources: [] }, + { id: 'workspace-engine', from: 'src/workspace.ts', to: 'src/engine.ts', kind: 'import', sources: [] }, + { id: 'test-registry', from: 'tests/registry.test.ts', to: 'src/registry.ts', kind: 'import', sources: [] }, + ], + }, + symbols: [ + { id: 'src/registry.ts#unloadPlugin:function', filePath: 'src/registry.ts', name: 'unloadPlugin', kind: 'function' }, + { id: 'src/engine.ts#rebuild:function', filePath: 'src/engine.ts', name: 'rebuild', kind: 'function' }, + ], + sourceText: { + files: [ + { filePath: 'src/registry.ts', content: 'export async function unloadPlugin() { /* plugin runtime cleanup */ }' }, + { filePath: 'src/engine.ts', content: 'export function rebuild() {}' }, + { filePath: 'src/workspace.ts', content: 'export function loadWorkspaceLifecycle() {}' }, + { filePath: 'src/unrelated.ts', content: 'export const unrelated = true;' }, + { filePath: 'tests/registry.test.ts', content: 'it("cleans plugin runtime after failure", () => {});' }, + ], + filesScanned: 5, + filesSkipped: 0, + }, + cacheState: 'fresh', +}; + +describe('core/graphQuery task map', () => { + it('combines task terms, declarations, and typed graph links in a small File map', () => { + expect(mapGraphTask(data, { + query: 'plugin runtime cleanup during workspace loading failure', + limit: 4, + })).toEqual({ + query: 'plugin runtime cleanup during workspace loading failure', + terms: ['plugin', 'runtime', 'cleanup', 'workspace', 'loading', 'failure'], + files: [ + { + path: 'src/workspace.ts', + nodeType: 'file', + matchedTerms: ['workspace', 'loading'], + symbols: [], + }, + { + path: 'tests/registry.test.ts', + nodeType: 'file', + matchedTerms: ['plugin', 'runtime', 'failure'], + symbols: [], + }, + { + path: 'src/registry.ts', + nodeType: 'file', + matchedTerms: ['plugin', 'runtime', 'cleanup'], + symbols: [{ id: 'src/registry.ts#unloadPlugin:function', name: 'unloadPlugin', kind: 'function' }], + }, + { + path: 'src/engine.ts', + nodeType: 'file', + matchedTerms: [], + symbols: [{ id: 'src/engine.ts#rebuild:function', name: 'rebuild', kind: 'function' }], + }, + ], + relationships: [ + { from: 'src/engine.ts', to: 'src/registry.ts', edgeTypes: ['call'] }, + { from: 'src/workspace.ts', to: 'src/engine.ts', edgeTypes: ['import'] }, + { from: 'tests/registry.test.ts', to: 'src/registry.ts', edgeTypes: ['import'] }, + ], + page: { offset: 0, limit: 4, returned: 4, total: 4, nextOffset: null }, + limits: { relationships: 12, complete: true }, + sources: { + text: { freshness: 'live', filesScanned: 5, filesSkipped: 0 }, + graph: { freshness: 'cached', cacheState: 'fresh' }, + }, + }); + }); + + it('reports truncation for every partial page and clamps public File bounds', () => { + const first = mapGraphTask(data, { query: 'plugin runtime failure', limit: 1 }); + const final = mapGraphTask(data, { query: 'plugin runtime failure', limit: 1, offset: 2 }); + const oversized = mapGraphTask(data, { query: 'plugin runtime failure', limit: 100 }); + + expect(first.page).toMatchObject({ limit: 1, returned: 1, total: 3, nextOffset: 1 }); + expect(first.limits.complete).toBe(false); + expect(first.relationships).toEqual([]); + expect(final.page).toMatchObject({ offset: 2, returned: 1, nextOffset: null }); + expect(final.limits.complete).toBe(false); + expect(oversized.page.limit).toBe(20); + }); + + it('returns an empty bounded map rather than throwing for malformed library input', () => { + expect(mapGraphTask(data, { query: undefined as unknown as string })).toMatchObject({ + query: '', + terms: [], + files: [], + relationships: [], + page: { limit: 8, returned: 0, total: 0 }, + limits: { complete: true }, + }); + }); + + it('matches common query inflections symmetrically without substring matches', () => { + const inflectedData: GraphQueryData = { + ...data, + sourceText: { + ...data.sourceText!, + files: data.sourceText!.files.map(file => file.filePath === 'src/unrelated.ts' + ? { ...file, content: 'export function runPluginAfterTaskFailed() {}' } + : file), + }, + }; + const report = mapGraphTask(inflectedData, { query: 'running plugin fail', limit: 8 }); + const unrelated = report.files.find(file => file.path === 'src/unrelated.ts'); + const registry = report.files.find(file => file.path === 'src/registry.ts'); + + expect(unrelated?.matchedTerms).toEqual(['running', 'plugin', 'fail']); + expect(registry?.matchedTerms).toEqual(['plugin']); + }); +}); diff --git a/packages/core/tests/graphQuery/taskMap/lexical.test.ts b/packages/core/tests/graphQuery/taskMap/lexical.test.ts new file mode 100644 index 0000000000..a4b4efc31e --- /dev/null +++ b/packages/core/tests/graphQuery/taskMap/lexical.test.ts @@ -0,0 +1,42 @@ +import { describe, expect, it } from 'vitest'; +import type { GraphQueryData } from '../../../src/graphQuery/data'; +import { + createTaskMapDocuments, + rankTaskMapDocument, + selectTaskMapTerms, +} from '../../../src/graphQuery/taskMap/lexical'; + +const data: GraphQueryData = { + graphData: { + nodes: [ + { id: 'src/task.ts', label: 'task.ts', nodeType: 'file' }, + { id: 'src/unload.ts', label: 'unload.ts', nodeType: 'file' }, + ], + edges: [], + }, + sourceText: { + files: [ + { filePath: 'src/task.ts', content: 'export function runPluginAfterTaskFailed() {}' }, + { filePath: 'src/unload.ts', content: 'export function unloadPlugin() {}' }, + ], + filesScanned: 2, + filesSkipped: 0, + }, +}; + +describe('core/graphQuery task map lexical ranking', () => { + it('matches common inflections symmetrically without substring matches', () => { + const documents = createTaskMapDocuments(data); + const terms = selectTaskMapTerms('running plugin fail loading', documents); + const frequencies = new Map(terms.map(term => [ + term, + documents.filter(document => rankTaskMapDocument(document, [term], new Map([[term, 1]]), 2).score > 0).length, + ])); + + expect(terms).toEqual(['running', 'plugin', 'fail']); + expect(rankTaskMapDocument(documents[0]!, terms, frequencies, 2).matchedTerms) + .toEqual(['running', 'plugin', 'fail']); + expect(rankTaskMapDocument(documents[1]!, ['loading'], new Map([['loading', 1]]), 2).matchedTerms) + .toEqual([]); + }); +}); diff --git a/packages/core/tests/graphQuery/taskMap/pagerank.test.ts b/packages/core/tests/graphQuery/taskMap/pagerank.test.ts new file mode 100644 index 0000000000..898cd56d82 --- /dev/null +++ b/packages/core/tests/graphQuery/taskMap/pagerank.test.ts @@ -0,0 +1,19 @@ +import { describe, expect, it } from 'vitest'; +import { rankTaskMapGraph } from '../../../src/graphQuery/taskMap/pagerank'; + +describe('core/graphQuery task map PageRank', () => { + it('preserves rank mass when personalized Files have no graph neighbors', () => { + const ranks = rankTaskMapGraph(new Map([ + ['isolated.ts', new Map()], + ['left.ts', new Map([['right.ts', 1]])], + ['right.ts', new Map([['left.ts', 1]])], + ]), new Map([ + ['isolated.ts', 1], + ['left.ts', 0], + ['right.ts', 0], + ])); + + expect([...ranks.values()].reduce((total, rank) => total + rank, 0)).toBeCloseTo(1, 10); + expect(ranks.get('isolated.ts')).toBeCloseTo(1, 10); + }); +}); diff --git a/packages/core/tests/graphQuery/taskMap/projection.test.ts b/packages/core/tests/graphQuery/taskMap/projection.test.ts new file mode 100644 index 0000000000..32e970a983 --- /dev/null +++ b/packages/core/tests/graphQuery/taskMap/projection.test.ts @@ -0,0 +1,38 @@ +import { describe, expect, it } from 'vitest'; +import type { GraphQueryData } from '../../../src/graphQuery/data'; +import { + indexTaskMapSymbols, + selectTaskMapRelationships, +} from '../../../src/graphQuery/taskMap/projection'; + +const filePaths = Array.from({ length: 14 }, (_, index) => `src/file-${index}.ts`); +const data: GraphQueryData = { + graphData: { + nodes: filePaths.map(path => ({ id: path, label: path, nodeType: 'file' as const })), + edges: filePaths.slice(1).map((path, index) => ({ + id: `edge-${index}`, + from: filePaths[0]!, + to: path, + kind: 'import' as const, + sources: [], + })), + }, + symbols: Array.from({ length: 5 }, (_, index) => ({ + id: `src/file-0.ts#symbol${index}:function`, + filePath: 'src/file-0.ts', + name: `symbol${index}`, + kind: 'function', + })), +}; + +describe('core/graphQuery task map projection', () => { + it('bounds declarations and typed relationships with truthful completeness', () => { + const symbols = indexTaskMapSymbols(data); + const relationships = selectTaskMapRelationships(data, new Set(filePaths), 12); + + expect(symbols.get('src/file-0.ts')).toHaveLength(3); + expect(relationships.relationships).toHaveLength(12); + expect(relationships.relationships.every(item => item.edgeTypes[0] === 'import')).toBe(true); + expect(relationships.complete).toBe(false); + }); +}); diff --git a/packages/core/tests/graphQuery/taskMap/sourceAreas.test.ts b/packages/core/tests/graphQuery/taskMap/sourceAreas.test.ts new file mode 100644 index 0000000000..0be92a57d6 --- /dev/null +++ b/packages/core/tests/graphQuery/taskMap/sourceAreas.test.ts @@ -0,0 +1,26 @@ +import { describe, expect, it } from 'vitest'; +import { balanceTaskMapSourceAreas } from '../../../src/graphQuery/taskMap/sourceAreas'; + +function item(path: string, score: number) { + return { file: { path }, lexicalScore: score, score }; +} + +describe('core/graphQuery task map source areas', () => { + it('keeps root source Files in one area while separating deeper Extension and Webview areas', () => { + const balanced = balanceTaskMapSourceAreas([ + item('src/a.ts', 10), + item('src/b.ts', 9), + item('packages/app/src/webview/app/target.ts', 8), + item('packages/app/src/webview/app/second.ts', 7), + item('packages/app/src/webview/components/view.ts', 6), + ]); + + expect(balanced.map(entry => entry.file.path)).toEqual([ + 'src/a.ts', + 'packages/app/src/webview/app/target.ts', + 'packages/app/src/webview/components/view.ts', + 'src/b.ts', + 'packages/app/src/webview/app/second.ts', + ]); + }); +}); diff --git a/packages/core/tests/graphScope/defaults.test.ts b/packages/core/tests/graphScope/defaults.test.ts index 8ce3ccab2e..440b53feeb 100644 --- a/packages/core/tests/graphScope/defaults.test.ts +++ b/packages/core/tests/graphScope/defaults.test.ts @@ -19,6 +19,7 @@ describe('graphScope/defaults', () => { expect(CORE_GRAPH_EDGE_DEFAULT_VISIBILITY.import).toBe(true); expect(CORE_GRAPH_EDGE_DEFAULT_VISIBILITY.using).toBe(true); expect(CORE_GRAPH_EDGE_DEFAULT_VISIBILITY.call).toBe(false); + expect(CORE_GRAPH_EDGE_DEFAULT_VISIBILITY.reexport).toBe(false); expect(CORE_GRAPH_EDGE_DEFAULT_VISIBILITY.reference).toBe(false); }); }); diff --git a/packages/core/tests/plugins/routingAnalysis.test.ts b/packages/core/tests/plugins/routingAnalysis.test.ts index 2736a8c3a2..aafb67f63e 100644 --- a/packages/core/tests/plugins/routingAnalysis.test.ts +++ b/packages/core/tests/plugins/routingAnalysis.test.ts @@ -122,6 +122,36 @@ describe('plugins/routing', () => { consoleError.mockRestore(); }); + it('retains distinct named relations to the same module', async () => { + const result = await analyzeFileResult( + 'src/app.ts', + 'content', + '/workspace', + new Map(), + new Map(), + async () => ({ + filePath: 'src/app.ts', + relations: [ + relation({ + kind: 'call', + specifier: './settings', + metadata: { importedName: 'readSettings' }, + }), + relation({ + kind: 'call', + specifier: './settings', + metadata: { importedName: 'writeSettings' }, + }), + ], + }), + ); + + expect(result?.relations?.map(item => item.metadata?.importedName)).toEqual([ + 'readSettings', + 'writeSettings', + ]); + }); + it('limits plugin analysis to selected plugin ids when requested', async () => { const selected = plugin('selected', ['.ts'], vi.fn(async () => ({ filePath: 'src/app.ts', diff --git a/packages/core/tests/plugins/routingRelationKeys.test.ts b/packages/core/tests/plugins/routingRelationKeys.test.ts index 3172db9745..ae8638a470 100644 --- a/packages/core/tests/plugins/routingRelationKeys.test.ts +++ b/packages/core/tests/plugins/routingRelationKeys.test.ts @@ -17,6 +17,29 @@ describe('plugins/routing', () => { }))).toBe('import|import-source|src/a.ts|node:a|symbol:a|./b|static|value'); }); + it('distinguishes named bindings and re-exports from the same module', () => { + const first = getRelationKey(relation({ + kind: 'import', + specifier: './storage', + metadata: { + reexport: true, + importedName: 'internalRead', + exportedName: 'publicRead', + }, + })); + const second = getRelationKey(relation({ + kind: 'import', + specifier: './storage', + metadata: { + reexport: true, + importedName: 'internalWrite', + exportedName: 'publicWrite', + }, + })); + + expect(first).not.toBe(second); + }); + it('adds node and symbol destinations for non-resolved relation kinds', () => { expect(getRelationKey(relation({ kind: 'import', diff --git a/packages/core/tests/treeSitter/analyze.test.ts b/packages/core/tests/treeSitter/analyze.test.ts index 23bb5515e0..ff1cb4ed71 100644 --- a/packages/core/tests/treeSitter/analyze.test.ts +++ b/packages/core/tests/treeSitter/analyze.test.ts @@ -66,7 +66,7 @@ describe('pipeline/plugins/treesitter/runtime/analyze', () => { const appSource = [ "import { boot } from './lib';", "import { readFileSync } from 'node:fs';", - "export { helper } from './helper';", + "export { helper as publicHelper } from './helper';", "const lazy = import('./helper');", "const legacy = require('./lib');", 'function run() {', @@ -101,6 +101,11 @@ describe('pipeline/plugins/treesitter/runtime/analyze', () => { kind: 'method', filePath: appPath, }), + expect.objectContaining({ + name: 'publicHelper', + kind: 'alias', + filePath: appPath, + }), ]), ); expect(result?.relations).toEqual( @@ -122,12 +127,17 @@ describe('pipeline/plugins/treesitter/runtime/analyze', () => { sourceId: 'core:treesitter:import', }), expect.objectContaining({ - kind: 'import', + kind: 'reexport', specifier: './helper', resolvedPath: path.join(workspaceRoot, 'src/helper.ts'), fromFilePath: appPath, toFilePath: path.join(workspaceRoot, 'src/helper.ts'), sourceId: 'core:treesitter:import', + metadata: { + reexport: true, + importedName: 'helper', + exportedName: 'publicHelper', + }, }), expect.objectContaining({ kind: 'import', diff --git a/packages/core/tests/treeSitter/capabilities.test.ts b/packages/core/tests/treeSitter/capabilities.test.ts index f19acc222b..83dcea4999 100644 --- a/packages/core/tests/treeSitter/capabilities.test.ts +++ b/packages/core/tests/treeSitter/capabilities.test.ts @@ -27,6 +27,7 @@ describe('pipeline/plugins/treesitter/runtime/capabilities', () => { it('advertises TypeScript graph scope edges supported by the current analyzer', () => { expect(listTreeSitterEdgeTypeCapabilities(['src/commented.ts'])).toEqual([ 'import', + 'reexport', 'type-import', 'call', 'inherit', @@ -271,6 +272,7 @@ describe('pipeline/plugins/treesitter/runtime/capabilities', () => { nodeTypes: [], edgeTypes: [ 'import', + 'reexport', 'reference', 'call', 'type-import', diff --git a/packages/core/tests/treeSitter/core.test.ts b/packages/core/tests/treeSitter/core.test.ts index 93ccfdea3c..2a67ce8e99 100644 --- a/packages/core/tests/treeSitter/core.test.ts +++ b/packages/core/tests/treeSitter/core.test.ts @@ -30,6 +30,7 @@ describe('core tree-sitter baseline analysis', () => { it('reports core Tree-sitter edge capabilities without plugin metadata', () => { expect(listCoreTreeSitterEdgeTypeCapabilities()).toEqual([ 'import', + 'reexport', 'reference', 'call', 'type-import', @@ -47,6 +48,7 @@ describe('core tree-sitter baseline analysis', () => { 'import', 'call', 'inherit', + 'reexport', 'type-import', 'contains', ]); @@ -73,6 +75,7 @@ describe('core tree-sitter baseline analysis', () => { 'import', 'call', 'inherit', + 'reexport', 'type-import', 'contains', ], diff --git a/packages/core/tests/treeSitter/javascript/imports.test.ts b/packages/core/tests/treeSitter/javascript/imports.test.ts index 0df9271128..1f64649c60 100644 --- a/packages/core/tests/treeSitter/javascript/imports.test.ts +++ b/packages/core/tests/treeSitter/javascript/imports.test.ts @@ -11,6 +11,7 @@ const { addImportRelation, addTypeImportRelation, addRelation, + createSymbol, } = vi.hoisted(() => ({ resolveTreeSitterImportPath: vi.fn(), collectImportBindings: vi.fn(), @@ -18,6 +19,7 @@ const { addImportRelation: vi.fn(), addTypeImportRelation: vi.fn(), addRelation: vi.fn(), + createSymbol: vi.fn(), })); vi.mock('../../../src/treeSitter/runtime/resolve', () => ({ @@ -36,6 +38,7 @@ vi.mock('../../../src/treeSitter/runtime/analyze/results', () => ({ addImportRelation, addTypeImportRelation, addRelation, + createSymbol, })); describe('extension/pipeline/treesitter/javascriptImports', () => { @@ -47,6 +50,8 @@ describe('extension/pipeline/treesitter/javascriptImports', () => { addImportRelation.mockReset(); addTypeImportRelation.mockReset(); addRelation.mockReset(); + createSymbol.mockReset(); + createSymbol.mockReturnValue({ id: 'publicRead-alias' }); collectImportBindings.mockReturnValue([]); }); @@ -338,7 +343,7 @@ describe('extension/pipeline/treesitter/javascriptImports', () => { expect(addTypeImportRelation).not.toHaveBeenCalled(); }); - it('adds import relations for export specifiers', () => { + it('adds re-export relations for export specifiers', () => { resolveTreeSitterImportPath.mockReturnValue('/workspace/src/lib.ts'); getStringSpecifier.mockReturnValueOnce('./lib').mockReturnValueOnce(null); const stringNode = { type: 'string' }; @@ -359,12 +364,62 @@ describe('extension/pipeline/treesitter/javascriptImports', () => { expect(getStringSpecifier).toHaveBeenNthCalledWith(2, undefined); expect(addRelation).toHaveBeenCalledTimes(1); expect(addRelation).toHaveBeenCalledWith(relations, { - kind: 'import', + kind: 'reexport', sourceId: 'core:treesitter:import', fromFilePath: '/workspace/src/app.ts', specifier: './lib', resolvedPath: '/workspace/src/lib.ts', toFilePath: '/workspace/src/lib.ts', + metadata: { reexportAll: true }, + }); + }); + + it('records imported and public names for named re-exports', () => { + resolveTreeSitterImportPath.mockReturnValue('/workspace/src/lib.ts'); + getStringSpecifier.mockReturnValue('./lib'); + const name = { type: 'identifier', text: 'internalRead' }; + const alias = { type: 'identifier', text: 'publicRead' }; + const exportSpecifier = { + type: 'export_specifier', + namedChildren: [name, alias], + childForFieldName: (field: string) => field === 'name' ? name : field === 'alias' ? alias : null, + }; + const relations: never[] = []; + const symbols: never[] = []; + + handleJavaScriptExportStatement( + { + namedChildren: [ + { type: 'export_clause', namedChildren: [exportSpecifier] }, + { type: 'string' }, + ], + } as never, + '/workspace/src/app.ts', + relations, + symbols, + true, + ); + + expect(createSymbol).toHaveBeenCalledWith( + '/workspace/src/app.ts', + 'alias', + 'publicRead', + exportSpecifier, + ); + expect(symbols).toEqual([{ id: 'publicRead-alias' }]); + expect(addRelation).toHaveBeenCalledWith(relations, { + kind: 'reexport', + sourceId: 'core:treesitter:import', + fromFilePath: '/workspace/src/app.ts', + specifier: './lib', + resolvedPath: '/workspace/src/lib.ts', + toFilePath: '/workspace/src/lib.ts', + fromSymbolId: 'publicRead-alias', + metadata: { + reexport: true, + importedName: 'internalRead', + exportedName: 'publicRead', + }, }); }); }); diff --git a/packages/core/tests/workspace/coreBackedCommands.test.ts b/packages/core/tests/workspace/coreBackedCommands.test.ts index 20666152a4..28b79ae8e5 100644 --- a/packages/core/tests/workspace/coreBackedCommands.test.ts +++ b/packages/core/tests/workspace/coreBackedCommands.test.ts @@ -64,6 +64,11 @@ describe('core-backed CodeGraphy Workspace commands', () => { expect(indexResult).toMatchObject({ workspaceRoot, graphCache: '.codegraphy/graph.sqlite', + discovery: { + indexedFiles: 2, + totalFound: 2, + limitReached: false, + }, indexing: { mode: 'full', analyzedFiles: 2, @@ -84,6 +89,25 @@ describe('core-backed CodeGraphy Workspace commands', () => { ]); }); + it('returns an actionable file-budget result when discovery is truncated', async () => { + const workspaceRoot = await fs.mkdtemp(path.join(os.tmpdir(), 'codegraphy-cli-budget-')); + await fs.writeFile(path.join(workspaceRoot, 'one.ts'), 'export const one = 1;\n'); + await fs.writeFile(path.join(workspaceRoot, 'two.ts'), 'export const two = 2;\n'); + writeCodeGraphyWorkspaceSettings(workspaceRoot, { + ...readCodeGraphyWorkspaceSettings(workspaceRoot), + maxFiles: 1, + }); + + const result = await requestCodeGraphyIndexWorkspace({ workspacePath: workspaceRoot }); + + expect(result.discovery).toEqual({ + indexedFiles: 1, + totalFound: 2, + limitReached: true, + action: 'Run `codegraphy settings set maxFiles 2`, then rerun `codegraphy index`.', + }); + }); + it('reports a fresh cache immediately after indexing with an unavailable configured plugin', async () => { const workspaceRoot = await fs.mkdtemp(path.join(os.tmpdir(), 'codegraphy-cli-missing-plugin-')); await fs.writeFile(path.join(workspaceRoot, 'Home.md'), '# Home\n', 'utf-8'); diff --git a/packages/core/tests/workspace/queryGraph.test.ts b/packages/core/tests/workspace/queryGraph.test.ts new file mode 100644 index 0000000000..6a7e759e96 --- /dev/null +++ b/packages/core/tests/workspace/queryGraph.test.ts @@ -0,0 +1,51 @@ +import * as fs from 'node:fs/promises'; +import * as os from 'node:os'; +import * as path from 'node:path'; +import { describe, expect, it } from 'vitest'; +import { requestCodeGraphyIndexWorkspace } from '../../src/workspace/requestIndexing'; +import { + projectWorkspaceQueryGraph, + readWorkspaceQuerySource, + readWorkspaceQuerySourceText, +} from '../../src/workspace/queryGraph'; + +describe('workspace/queryGraph', () => { + it('projects independent Filters without mutating the shared query source', async () => { + const workspaceRoot = await fs.mkdtemp(path.join(os.tmpdir(), 'codegraphy-query-source-')); + await fs.writeFile(path.join(workspaceRoot, 'entry.txt'), 'entry\n'); + await fs.writeFile(path.join(workspaceRoot, 'model.txt'), 'model\n'); + await requestCodeGraphyIndexWorkspace({ workspacePath: workspaceRoot }); + const source = readWorkspaceQuerySource(workspaceRoot, { version: 3, plugins: [] }); + + const filtered = projectWorkspaceQueryGraph(source, { filterPatterns: ['model.txt'] }); + const complete = projectWorkspaceQueryGraph(source); + + expect(filtered.graphData.nodes.map(node => node.id)).toEqual(['entry.txt']); + expect(complete.graphData.nodes.map(node => node.id)).toEqual(['entry.txt', 'model.txt']); + expect(source.graphData.nodes.map(node => node.id)).toEqual(['entry.txt', 'model.txt']); + }); + + it('skips binary, oversized, unreadable, and outside-workspace source files', async () => { + const workspaceRoot = await fs.mkdtemp(path.join(os.tmpdir(), 'codegraphy-query-text-')); + await fs.writeFile(path.join(workspaceRoot, 'binary.dat'), 'before\0after'); + await fs.writeFile(path.join(workspaceRoot, 'large.ts'), Buffer.alloc(1024 * 1024 + 1)); + + const result = readWorkspaceQuerySourceText(workspaceRoot, { + nodes: [ + { id: 'binary.dat', label: 'binary.dat', nodeType: 'file' }, + { id: 'large.ts', label: 'large.ts', nodeType: 'file' }, + { id: 'missing.ts', label: 'missing.ts', nodeType: 'file' }, + { id: '../outside.ts', label: 'outside.ts', nodeType: 'file' }, + { id: 'folder', label: 'folder', nodeType: 'folder' }, + ], + edges: [], + }); + + expect(result).toEqual({ + files: [], + filesScanned: 0, + filesSkipped: 4, + hasChangedFiles: false, + }); + }); +}); diff --git a/packages/core/tests/workspace/requestQuery.test.ts b/packages/core/tests/workspace/requestQuery.test.ts new file mode 100644 index 0000000000..1de84df86a --- /dev/null +++ b/packages/core/tests/workspace/requestQuery.test.ts @@ -0,0 +1,195 @@ +import * as fs from 'node:fs/promises'; +import * as os from 'node:os'; +import * as path from 'node:path'; +import { describe, expect, it } from 'vitest'; +import { requestCodeGraphyIndexWorkspace } from '../../src/workspace/requestIndexing'; +import { requestWorkspaceGraphQuery } from '../../src/workspace/requestQuery'; + +describe('workspace/requestQuery', () => { + it('searches current source text and cached AST Symbols through one bounded report', async () => { + const workspaceRoot = await fs.mkdtemp(path.join(os.tmpdir(), 'codegraphy-query-search-')); + await fs.writeFile(path.join(workspaceRoot, 'entry.ts'), [ + 'export function runIndexCommand(): void {', + ' process.stderr.write(`Indexing ${workspaceRoot}...`);', + '}', + '', + ].join('\n')); + await requestCodeGraphyIndexWorkspace({ workspacePath: workspaceRoot }); + + const symbolResult = await requestWorkspaceGraphQuery({ + workspacePath: workspaceRoot, + report: 'search', + arguments: { pattern: 'runIndexCommand', limit: 20 }, + }); + const textResult = await requestWorkspaceGraphQuery({ + workspacePath: workspaceRoot, + report: 'search', + arguments: { pattern: 'Indexing ', limit: 20 }, + }); + + expect(symbolResult).toMatchObject({ + sources: { symbols: { freshness: 'cached', cacheState: 'fresh' } }, + }); + expect(symbolResult.matches).toEqual(expect.arrayContaining([{ + type: 'symbol', + symbol: expect.objectContaining({ name: 'runIndexCommand', kind: 'function', filePath: 'entry.ts' }), + }])); + expect(textResult).toMatchObject({ + matches: [{ + type: 'text', + filePath: 'entry.ts', + line: 2, + excerpt: ' process.stderr.write(`Indexing ${workspaceRoot}...`);', + }], + sources: { text: { freshness: 'live', filesScanned: 1, filesSkipped: 0 } }, + }); + }); + + it('builds a task-personalized File map from live terms and cached Relationships', async () => { + const workspaceRoot = await fs.mkdtemp(path.join(os.tmpdir(), 'codegraphy-query-task-map-')); + await fs.writeFile(path.join(workspaceRoot, 'setting.ts'), 'export function readSetting(): string { return "value"; }\n'); + await fs.writeFile(path.join(workspaceRoot, 'command.ts'), [ + "import { readSetting } from './setting';", + 'export function runCommand(): string { return readSetting(); }', + '', + ].join('\n')); + await requestCodeGraphyIndexWorkspace({ workspacePath: workspaceRoot }); + + const result = await requestWorkspaceGraphQuery({ + workspacePath: workspaceRoot, + report: 'task-map', + arguments: { query: 'setting command', limit: 4 }, + }); + + expect(result).toMatchObject({ + terms: ['setting', 'command'], + files: [ + { path: 'command.ts', matchedTerms: ['setting', 'command'] }, + { path: 'setting.ts', matchedTerms: ['setting'] }, + ], + relationships: [{ from: 'command.ts', to: 'setting.ts', edgeTypes: expect.arrayContaining(['import']) }], + limits: { complete: true }, + sources: { text: { freshness: 'live' }, graph: { freshness: 'cached', cacheState: 'fresh' } }, + }); + + const projected = await requestWorkspaceGraphQuery({ + workspacePath: workspaceRoot, + report: 'task-map', + arguments: { query: 'setting command', limit: 4 }, + projection: { edgeTypes: ['type-import'] }, + }); + expect(projected.files).toHaveLength(2); + expect(projected.relationships).toEqual([]); + }); + + it('uses complete graph scope for an exact targeted Relationship query', async () => { + const workspaceRoot = await fs.mkdtemp(path.join(os.tmpdir(), 'codegraphy-query-target-scope-')); + await fs.writeFile(path.join(workspaceRoot, 'dependency.ts'), [ + 'export function readSetting(): string {', + " return 'value';", + '}', + '', + ].join('\n')); + await fs.writeFile(path.join(workspaceRoot, 'entry.ts'), [ + "import { readSetting } from './dependency';", + 'export function runCommand(): string {', + ' return readSetting();', + '}', + '', + ].join('\n')); + await requestCodeGraphyIndexWorkspace({ workspacePath: workspaceRoot }); + + for (const projection of [undefined, { edgeTypes: ['call'] }]) { + const result = await requestWorkspaceGraphQuery({ + workspacePath: workspaceRoot, + report: 'edges', + arguments: { + from: 'entry.ts#runCommand:function', + edgeType: 'call', + limit: 20, + }, + ...(projection ? { projection } : {}), + }); + + expect(result.edges).toEqual([{ + from: 'entry.ts#runCommand:function', + to: 'dependency.ts#readSetting:function', + edgeTypes: ['call'], + }]); + } + }); + + it('resolves targeted calls through named re-export barrels', async () => { + const workspaceRoot = await fs.mkdtemp(path.join(os.tmpdir(), 'codegraphy-query-reexport-')); + await fs.writeFile(path.join(workspaceRoot, 'storage.ts'), [ + 'export function readSetting(): string {', + " return 'value';", + '}', + '', + ].join('\n')); + await fs.writeFile( + path.join(workspaceRoot, 'settings.ts'), + "export { readSetting } from './storage';\n", + ); + await fs.writeFile(path.join(workspaceRoot, 'entry.ts'), [ + "import { readSetting } from './settings';", + 'export function runCommand(): string {', + ' return readSetting();', + '}', + '', + ].join('\n')); + await requestCodeGraphyIndexWorkspace({ workspacePath: workspaceRoot }); + + const result = await requestWorkspaceGraphQuery({ + workspacePath: workspaceRoot, + report: 'edges', + arguments: { + from: 'entry.ts#runCommand:function', + edgeType: 'call', + limit: 20, + }, + projection: { edgeTypes: ['call'] }, + }); + + expect(result.edges).toEqual([{ + from: 'entry.ts#runCommand:function', + to: 'storage.ts#readSetting:function', + edgeTypes: ['call'], + }]); + + const reexports = await requestWorkspaceGraphQuery({ + workspacePath: workspaceRoot, + report: 'edges', + arguments: { + from: 'settings.ts', + edgeType: 'reexport', + limit: 20, + }, + projection: { edgeTypes: ['reexport'] }, + }); + expect(reexports.edges).toEqual([{ + from: 'settings.ts', + to: 'storage.ts#readSetting:function', + edgeTypes: ['reexport'], + }]); + }); + + it('reads live text after Indexing while marking cached Symbols stale', async () => { + const workspaceRoot = await fs.mkdtemp(path.join(os.tmpdir(), 'codegraphy-query-live-text-')); + const entryPath = path.join(workspaceRoot, 'entry.ts'); + await fs.writeFile(entryPath, 'export const original = 1;\n'); + await requestCodeGraphyIndexWorkspace({ workspacePath: workspaceRoot }); + await fs.writeFile(entryPath, 'export const changedAfterIndex = 2;\n'); + + const result = await requestWorkspaceGraphQuery({ + workspacePath: workspaceRoot, + report: 'search', + arguments: { pattern: 'changedAfterIndex', limit: 20 }, + }); + + expect(result).toMatchObject({ + matches: [{ type: 'text', filePath: 'entry.ts', line: 1 }], + sources: { symbols: { freshness: 'cached', cacheState: 'stale' } }, + }); + }); +}); diff --git a/packages/core/tests/workspace/settings.test.ts b/packages/core/tests/workspace/settings.test.ts index baaa246d76..a0b73fea24 100644 --- a/packages/core/tests/workspace/settings.test.ts +++ b/packages/core/tests/workspace/settings.test.ts @@ -70,7 +70,7 @@ describe('CodeGraphy Workspace settings', () => { await expect(fs.access(getWorkspaceSettingsPath(workspaceRoot))).rejects.toThrow(); }); - it('normalizes workspace plugin entries from settings.json', async () => { + it('rejects malformed workspace plugin entries from settings.json', async () => { const workspaceRoot = await createWorkspace(); await fs.mkdir(path.dirname(getWorkspaceSettingsPath(workspaceRoot)), { recursive: true }); await fs.writeFile( @@ -92,15 +92,9 @@ describe('CodeGraphy Workspace settings', () => { 'utf-8', ); - expect(readCodeGraphyWorkspaceSettings(workspaceRoot)).toMatchObject({ - maxFiles: 50, - plugins: [{ - id: 'codegraphy.vue', - activation: 'disabled', - disabledFilterPatterns: ['**/__pycache__/**'], - options: { includeTests: true }, - }], - }); + expect(() => readCodeGraphyWorkspaceSettings(workspaceRoot)).toThrow( + 'plugin disabledFilterPatterns must be an array of strings', + ); }); it('keeps only the last settings entry for each plugin id', () => { @@ -190,8 +184,7 @@ describe('CodeGraphy Workspace settings', () => { enabled: true, futureMarkdownSetting: 'keep', }, - { id: 'future.plugin', futureOnlyShape: true }, - 3, + { id: 'future.plugin', activation: 'disabled', futureOnlyShape: true }, ], })); @@ -204,8 +197,7 @@ describe('CodeGraphy Workspace settings', () => { package: '@codegraphy-dev/plugin-markdown', futureMarkdownSetting: 'keep', }), - { id: 'future.plugin', futureOnlyShape: true }, - 3, + { id: 'future.plugin', activation: 'disabled', futureOnlyShape: true }, ])); }); @@ -247,18 +239,14 @@ describe('CodeGraphy Workspace settings', () => { await fs.writeFile(getWorkspaceSettingsPath(workspaceRoot), JSON.stringify({ plugins: [ { id: CODEGRAPHY_MARKDOWN_PLUGIN_ID, activation: 'enabled' }, - { id: 'future.plugin', futureOnlyShape: true }, - 3, + { id: 'future.plugin', activation: 'disabled', futureOnlyShape: true }, ], })); const settings = readCodeGraphyWorkspaceSettings(workspaceRoot); writeCodeGraphyWorkspaceSettings(workspaceRoot, { ...settings, plugins: [] }); - expect(JSON.parse(await fs.readFile(getWorkspaceSettingsPath(workspaceRoot), 'utf-8')).plugins).toEqual([ - { id: 'future.plugin', futureOnlyShape: true }, - 3, - ]); + expect(JSON.parse(await fs.readFile(getWorkspaceSettingsPath(workspaceRoot), 'utf-8')).plugins).toEqual([]); }); it('returns defaults for non-object settings values', () => { diff --git a/packages/core/tests/workspace/settingsValidation.test.ts b/packages/core/tests/workspace/settingsValidation.test.ts new file mode 100644 index 0000000000..7fdad2805c --- /dev/null +++ b/packages/core/tests/workspace/settingsValidation.test.ts @@ -0,0 +1,30 @@ +import { describe, expect, it } from 'vitest'; +import { validateWorkspaceSettingsRecord } from '../../src/workspace/settingsValidation'; + +describe('workspace/settingsValidation', () => { + it('accepts supported settings while preserving unknown fields', () => { + expect(validateWorkspaceSettingsRecord({ + maxFiles: 2500, + include: ['src/**'], + respectGitignore: true, + futureSetting: { mode: 'fast' }, + })).toEqual({ + maxFiles: 2500, + include: ['src/**'], + respectGitignore: true, + futureSetting: { mode: 'fast' }, + }); + }); + + it('rejects malformed known settings', () => { + expect(() => validateWorkspaceSettingsRecord({ maxFiles: 0 })).toThrow( + 'maxFiles must be a positive integer', + ); + expect(() => validateWorkspaceSettingsRecord({ filterPatterns: 'generated' })).toThrow( + 'filterPatterns must be an array of strings', + ); + expect(() => validateWorkspaceSettingsRecord({ plugins: [{ id: 'plugin.without.activation' }] })).toThrow( + 'plugin entry must have a nonblank id with activation, or a nonblank package', + ); + }); +}); diff --git a/packages/plugin-api/README.md b/packages/plugin-api/README.md index 383353a6c2..77d4e7368f 100644 --- a/packages/plugin-api/README.md +++ b/packages/plugin-api/README.md @@ -41,6 +41,7 @@ This package is type-only. Use `import type`. Core plugins can: - analyze workspace files; +- use Core Relationship kinds including `import`, `reexport`, `call`, `reference`, and `inherit`; - add semantic Nodes, Symbols, and Relationships; - add semantic Node Type and Edge Type definitions; - declare Graph Scope capabilities and default filters; diff --git a/packages/plugin-api/src/graph.ts b/packages/plugin-api/src/graph.ts index 4ef003cfee..3ec198269a 100644 --- a/packages/plugin-api/src/graph.ts +++ b/packages/plugin-api/src/graph.ts @@ -18,6 +18,7 @@ export type CoreEdgeKind = | 'inherit' | 'implements' | 'event' + | 'reexport' | 'reference' | 'load' | 'contains' diff --git a/packages/plugin-api/tests/pluginContracts.test.ts b/packages/plugin-api/tests/pluginContracts.test.ts index daca1ad982..8e79e5604e 100644 --- a/packages/plugin-api/tests/pluginContracts.test.ts +++ b/packages/plugin-api/tests/pluginContracts.test.ts @@ -2,6 +2,7 @@ import { describe, expectTypeOf, it } from 'vitest'; import type { CodeGraphyAccessKey, + CoreEdgeKind, IAccessProvider, IPlugin, IPluginGraphScopeCapabilityContext, @@ -12,6 +13,10 @@ import type { } from '../src'; describe('plugin API contracts', () => { + it('includes re-exports in the Core relationship vocabulary', () => { + expectTypeOf<'reexport'>().toMatchTypeOf(); + }); + it('lets plugins declare graph scope capabilities separately from emitted graph output', () => { const plugin = { id: 'acme.routes', diff --git a/skills/codegraphy/SKILL.md b/skills/codegraphy/SKILL.md index 5a90035535..4725d0358d 100644 --- a/skills/codegraphy/SKILL.md +++ b/skills/codegraphy/SKILL.md @@ -1,16 +1,63 @@ --- name: codegraphy -description: Use the CodeGraphy CLI to index, shape, and query a workspace graph. +description: Understand and operate the CodeGraphy CLI for workspace source, symbol, and relationship exploration. --- # CodeGraphy -CodeGraphy turns a workspace into a queryable graph. +CodeGraphy represents a workspace as a **Relationship Graph**. Nodes identify Files, Folders, Packages, AST Symbols, and plugin-defined concepts. Directed Edges identify relationships such as imports, reexports, calls, references, inheritance, and containment. Edge direction and type explain why two Nodes are connected. -## Workflow +The graph is navigation evidence. Static relationships can identify ownership and possible change surfaces, but they do not prove runtime behavior. Source, tests, generated behavior, dynamic dispatch, and unsupported language semantics can add facts that are absent from the graph. -1. Run `codegraphy index`. Configure graph contributors with `codegraphy plugins` before indexing when needed. -2. Shape returned results with `codegraphy filter` and `codegraphy scope`. These do not reindex or remove stored graph facts. -3. Query with `nodes`, `search`, `edges`, `dependencies`, `dependents`, or `path`. +## Graph lifecycle and freshness -`index` stores the complete graph in `.codegraphy/graph.sqlite`. Query commands accept one-off `--filter`, `--node-type`, and `--edge-type` options without changing settings. Use `status` for cache freshness and `doctor` for runtime, settings, cache, and plugin diagnostics. Run `codegraphy --help` or `codegraphy --help` for the exact contract and examples. Commands use the current directory by default, accept `-C ` for another workspace, and return a JSON envelope unless they are help or version commands. +`codegraphy index` discovers eligible workspace files, runs built-in and enabled Plugin analysis, and creates or incrementally updates `.codegraphy/graph.sqlite`. The Graph Cache stores complete indexed facts before Graph Scope and path Filters shape query results. + +`codegraphy filter` changes persisted path exclusions without rebuilding cached analysis. `codegraphy scope` changes which Node Types and Edge Types appear in the shaped graph. `codegraphy settings` exposes workspace discovery, indexing, filter, scope, Plugin, and interface settings; settings mutations report whether another Index is required. `codegraphy plugins` controls installed Plugin registration and activation. + +Query with any read-only graph command after a Graph Cache exists. A `graph_cache_not_found` result means that no indexed snapshot is available. `codegraphy status` reports supported missing, stale, and fresh cache conditions. `codegraphy doctor` checks runtime, settings, cache, and Plugin health and includes recovery information for unhealthy checks. + +Most Symbols and Relationships are cached. Search also reads current source text from eligible indexed File Nodes. A single response can therefore contain `freshness: "live"` source matches and cached Symbol matches whose `cacheState` is `fresh` or `stale`. Indexing is what makes changed AST and Relationship facts current. + +## Query surfaces + +| Command | Information returned | +|---|---| +| `search ` | One ranked result set merging live source locations, cached AST Symbols, and indexed Nodes. | +| `map ` | A compact task-personalized File map with matched terms, selected declarations, and typed connecting Relationships. | +| `query ` | A bounded overview of one exact File or Symbol Node, with prioritized declarations and incoming and outgoing Relationships. | +| `nodes` | A paginated Node inventory from the shaped graph. | +| `edges` | A paginated Relationship inventory from the shaped graph. | +| `dependencies ` | Outgoing Relationships from a File path or exact Node ID. | +| `dependents ` | Incoming Relationships to a File path or exact Node ID. | +| `path ` | A bounded Relationship route between two File paths or exact Node IDs. | + +Inventory reports reflect persisted Graph Scope. Search, exact Target Query, Path, and targeted relationship selectors use complete cached Node and Edge Types unless an invocation explicitly projects a dimension with `--node-type` or `--edge-type`. Path Filters still apply. + +`--filter`, `--node-type`, and `--edge-type` are one-off query projections and do not modify `.codegraphy/settings.json`. Persisted Filter and Scope changes affect later commands without deleting the complete cached facts. + +## Search and target identity + +Search literal matching is case-insensitive. `*` is a line-local wildcard over source, names, and paths. A whitespace-containing phrase with sparse literal matches can also produce deterministic File candidates whose paths or source contain all query terms. Search is lexical rather than a semantic-answer engine. + +Source matches include File path, line, column, excerpt, and live freshness. Symbol matches include an exact Symbol ID and source location. Node matches include exact Node identity. An exact File path or Symbol ID from these results can address Target Query and relationship commands; a display label is not necessarily a Node ID. + +Search, inventories, overviews, and relationship reports are bounded. Pagination uses `page` metadata recording offset, limit, returned count, total count, and `nextOffset` when another page exists. Target Query reports independent declaration and relationship bounds. Path reports include traversal limits and a `complete` boolean; `complete: false` means the configured search bound was reached before the entire search space was exhausted. + +## Machine-readable contract + +Normal command results are JSON envelopes. Successful data is written to stdout. Operational and invalid-invocation failures use structured error envelopes on stderr and nonzero exit statuses. `--verbose` adds lifecycle diagnostics to stderr without changing the data envelope. + +Common error codes distinguish invalid arguments, missing or stale workspace state, an exact target that is absent, malformed settings, and operational failures. Error `details` and `actions` carry command-specific recovery context when available. + +## Interpretation limits + +- Graph coverage depends on eligible files, the indexing file budget, enabled Plugins, supported languages, and analyzer capabilities. +- Persisted Filters and Graph Scope can intentionally hide facts from broad inventories. +- Live text and cached structural facts can have different freshness in the same result. +- Imports, calls, references, and inferred or plugin-defined Edges have different semantics; an Edge Type should be interpreted rather than treated as generic proximity. +- Incoming Relationships suggest consumers or possible impact, while outgoing Relationships suggest dependencies; neither alone defines the complete edit set. +- Hubs, barrels, generated files, tests, and shared utilities can have many legitimate Relationships and can dominate broad graph results. +- Bounded or paginated output is not evidence that omitted results do not exist. + +Current command syntax, options, and examples are available from `codegraphy --help` and `codegraphy --help`. diff --git a/skills/codegraphy/agents/openai.yaml b/skills/codegraphy/agents/openai.yaml index c452d1e161..50445a5e78 100644 --- a/skills/codegraphy/agents/openai.yaml +++ b/skills/codegraphy/agents/openai.yaml @@ -1,4 +1,4 @@ interface: display_name: "CodeGraphy" - short_description: "Configure and query indexed code graphs" - default_prompt: "Use $codegraphy to index this workspace and answer a graph structure, dependency, or impact question." + short_description: "Discover source, AST symbols, and code relationships" + default_prompt: "Use $codegraphy when its prepared Relationship Graph can reduce source navigation for this task." diff --git a/tests/release/codegraphySkill.test.mjs b/tests/release/codegraphySkill.test.mjs index a07e006c5c..9c5eb62d30 100644 --- a/tests/release/codegraphySkill.test.mjs +++ b/tests/release/codegraphySkill.test.mjs @@ -14,7 +14,7 @@ test('the generalized CodeGraphy skill teaches the lifecycle and delegates synta const lifecyclePositions = lifecycle.map(term => skill.indexOf(term)); assert.ok(lifecyclePositions.every(position => position >= 0)); assert.deepEqual(lifecyclePositions, [...lifecyclePositions].sort((left, right) => left - right)); - for (const command of ['nodes', 'search', 'edges', 'dependencies', 'dependents', 'path']) { + for (const command of ['nodes', 'search', 'map', 'query', 'edges', 'dependencies', 'dependents', 'path']) { assert.match(skill, new RegExp(`\\b${command}\\b`)); } assert.match(skill, /codegraphy --help/); @@ -22,6 +22,27 @@ test('the generalized CodeGraphy skill teaches the lifecycle and delegates synta assert.doesNotMatch(skill, /MCP|graph\.lbug/); }); +test('the CodeGraphy skill explains the tool without prescribing an agent workflow', () => { + const skill = readFileSync(skillPath, 'utf8'); + + for (const concept of [ + 'Relationship Graph', + 'Graph Cache', + 'live source', + 'cached', + 'Graph Scope', + 'pagination', + 'stdout', + 'stale', + ]) { + assert.match(skill, new RegExp(concept, 'i')); + } + assert.doesNotMatch(skill, /Token-bounded navigation policy/i); + assert.doesNotMatch(skill, /(?:at most|no more than|normally make)\s+(?:one|two|three|\d+)\s+CodeGraphy calls?/i); + assert.doesNotMatch(skill, /(?:second and final|a third is allowed|commands launched concurrently count)/i); + assert.doesNotMatch(skill, /(?:never submit|one domain word|task literals|after one empty or refined search)/i); +}); + test('the old MCP package and skill are absent from the release source', () => { assert.equal(existsSync(path.join(repoRoot, 'packages', 'mcp', 'package.json')), false); assert.equal(existsSync(path.join(repoRoot, 'skills', 'codegraphy-mcp', 'SKILL.md')), false);