Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
6e9e6f3
chore: clarify plan location in template
lmeyerov Dec 29, 2025
209b2a3
feat: add collections validation and gfql support
lmeyerov Dec 29, 2025
673eed8
fix: satisfy mypy in collections validation
lmeyerov Dec 29, 2025
e1d0526
feat: add collections helper constructors
lmeyerov Dec 30, 2025
858ff91
feat: wrap collection set expr to gfql chain
lmeyerov Dec 30, 2025
6b5370c
Refine collections types and helpers
lmeyerov Jan 2, 2026
04dd228
Fix collections typing for mypy
lmeyerov Jan 5, 2026
322d152
Validate collections settings inputs
lmeyerov Jan 7, 2026
b170076
Simplify collections typing
lmeyerov Jan 7, 2026
9064aee
Reuse gfql chain normalization in collections helpers
lmeyerov Jan 7, 2026
a7b2a2d
Refine collections gfql normalization for mypy
lmeyerov Jan 8, 2026
8389026
Normalize collections GFQL via Chain and reject Let
lmeyerov Jan 8, 2026
1144977
Avoid Chain.from_json in collections normalization
lmeyerov Jan 8, 2026
4e889b2
Allow Let in collections normalization
lmeyerov Jan 8, 2026
59ba9a3
Simplify collections gfql wrapping
lmeyerov Jan 8, 2026
2988311
Slim collections validation helpers
lmeyerov Jan 8, 2026
2f426d0
Simplify collections input parsing
lmeyerov Jan 8, 2026
924f234
fix: canonicalize collections validation and encoding
lmeyerov Jan 13, 2026
41a33c4
refactor: simplify collections normalization
lmeyerov Jan 13, 2026
0de6944
chore: move collections notes to development
lmeyerov Jan 16, 2026
9fd5533
Merge branch 'master' into feat/collections-support
mj3cheun Feb 6, 2026
23d1f00
fix: address PR #874 review comments for collections validation
lmeyerov Feb 8, 2026
00ec4f8
docs: update CHANGELOG with collections review fixes
lmeyerov Feb 8, 2026
d2bdf02
refactor: inline _coerce_collection_list into _parse_collections_input
lmeyerov Feb 8, 2026
47d87e8
fix: add type casts for mypy after inlining _coerce_collection_list
lmeyerov Feb 8, 2026
2b5fa78
fix: auto-generate collection IDs in autofix mode
lmeyerov Feb 8, 2026
1ab2e82
style: use kebab-case for auto-generated collection IDs
lmeyerov Feb 8, 2026
3d74b1e
refactor: move GFQL normalization to compute/ast.py
lmeyerov Feb 9, 2026
1f0c387
feat: validate intersection DAG (cycles, self-refs, intersections-of-…
lmeyerov Feb 10, 2026
75839b2
fix: catch AssertionError from AST from_json in validation
lmeyerov Feb 11, 2026
2f12a79
refactor(ai): migrate prompts into skills and port docs to .agents (#…
lmeyerov Mar 3, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 45 additions & 0 deletions .agents/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# .agents

Assistant-facing support material for this repository.

## Purpose

- Keep reusable agent docs and helper scripts in one place.
- Keep task-oriented skill definitions in `.agents/skills/`.

## Layout

```text
.agents/
├── README.md
├── assets/
│ ├── find_dynamic_imports.sh
│ ├── generate_comment_inventory.sh
│ └── pysa_extract_callers.py
└── docs/
└── gfql/
├── README.md
├── calls_checklist.md
├── conformance.md
├── oracle.md
└── predicates_checklist.md
```

## Related Paths

- Skills: `.agents/skills/`
- Skill index: `.agents/skills/SKILLS.md`
- Plans (gitignored): `plans/`

## Quick Commands

```bash
# Generate dynamic import inventory for refactors
./.agents/assets/find_dynamic_imports.sh master plans/<task>/dynamic_imports.md

# Generate comment inventory for cleanup passes
./.agents/assets/generate_comment_inventory.sh master plans/<task>/comment_inventory.md

# Extract callers from pysa call graph
python3 .agents/assets/pysa_extract_callers.py pysa_results/call-graph.json PlotterBase.PlotterBase.bind
```
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
#!/bin/bash
# Find Dynamic Imports for HOISTIMPORTS Protocol
#
# Usage: ./ai/assets/find_dynamic_imports.sh [base_branch] [output_file]
# Usage: ./.agents/assets/find_dynamic_imports.sh [base_branch] [output_file]
#
# This script automates Phase 1 of the HOISTIMPORTS protocol by extracting all
# dynamic imports (imports inside functions/methods/conditionals) added in a PR
# and formatting them with context for categorization.
#
# Example:
# ./ai/assets/find_dynamic_imports.sh master plans/my-feature/dynamic_imports.md
# ./.agents/assets/find_dynamic_imports.sh master plans/my-feature/dynamic_imports.md

set -euo pipefail

Expand Down
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
#!/bin/bash
# Generate Comment Inventory for DECOMMENT Protocol
#
# Usage: ./ai/assets/generate_comment_inventory.sh [base_branch] [output_file]
# Usage: ./.agents/assets/generate_comment_inventory.sh [base_branch] [output_file]
#
# This script automates Phase 1 of the DECOMMENT protocol by extracting all
# comments added in a PR and formatting them with context for categorization.
#
# Example:
# ./ai/assets/generate_comment_inventory.sh master plans/my-feature/comment_inventory.md
# ./.agents/assets/generate_comment_inventory.sh master plans/my-feature/comment_inventory.md

set -euo pipefail

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,10 @@
"""Extract caller information from Pysa call-graph.json

Usage:
python3 ai/assets/pysa_extract_callers.py path/to/call-graph.json method1 method2 ...
python3 .agents/assets/pysa_extract_callers.py path/to/call-graph.json method1 method2 ...

Example:
python3 ai/assets/pysa_extract_callers.py \\
python3 .agents/assets/pysa_extract_callers.py \\
pysa_results/call-graph.json \\
PlotterBase.PlotterBase.bind \\
PlotterBase.PlotterBase.nodes
Expand Down
2 changes: 1 addition & 1 deletion ai/docs/gfql/README.md → .agents/docs/gfql/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Guide for AI assistants working with GFQL (Graph Frame Query Language) in PyGrap
- [`calls_checklist.md`](./calls_checklist.md) — Required steps for exposing or updating GFQL `call()` functions.
- [`predicates_checklist.md`](./predicates_checklist.md) — End-to-end checklist for predicate implementations.
- [`conformance.md`](./conformance.md) — Cypher TCK conformance harness and CI wiring.
- [`../prompts/GFQL_LLM_GUIDE_MAINTENANCE.md`](../prompts/GFQL_LLM_GUIDE_MAINTENANCE.md) — Guidance for keeping AI assistants aligned with GFQL changes.
- [`../../skills/gfql-llm-guide-maintenance/SKILL.md`](../../skills/gfql-llm-guide-maintenance/SKILL.md) — Guidance for keeping AI assistants aligned with GFQL changes.

### Essential GFQL Operations
```python
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ When exposing or updating GFQL `call()` functions, the change is **multi-system*
runtime executors, JSON schema validators, static typing, documentation, and tests
all need to stay in sync. Use this playbook (paired with
[`predicates_checklist.md`](./predicates_checklist.md) and the guidance in
[`../prompts/GFQL_LLM_GUIDE_MAINTENANCE.md`](../../prompts/GFQL_LLM_GUIDE_MAINTENANCE.md))
[`../../skills/gfql-llm-guide-maintenance/SKILL.md`](../../skills/gfql-llm-guide-maintenance/SKILL.md))
to avoid regressions.

> **Scope**: Items below are required for *every* new or modified `call()` entry.
Expand Down Expand Up @@ -90,7 +90,7 @@ to avoid regressions.
## 📎 Optional (Promote When User-Facing)

- **Marketing / blog teaser** when the call is a marquee feature.
- **LLM prompt updates** (`ai/prompts/gfql/*`) beyond baseline instructions.
- **LLM prompt updates** (`.agents/skills/gfql-llm-guide-maintenance/SKILL.md`) beyond baseline instructions.
- **Customer templates** or dataset refreshes referencing the new capability.

Keep this file synchronized with future GFQL evolutions—open a PR to amend when the onboarding surface changes.
File renamed without changes.
File renamed without changes.
11 changes: 11 additions & 0 deletions .agents/skills/SKILLS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Skills Catalog

Each skill is a folder containing `SKILL.md` with YAML frontmatter.

- `conventional-commits`
- `decomment`
- `gfql-llm-guide-maintenance`
- `hoist-imports`
- `lint-types-check`
- `plan`
- `pyre-analysis`
Original file line number Diff line number Diff line change
@@ -1,6 +1,11 @@
---
name: conventional-commits
description: "Create atomic PyGraphistry commits with conventional commit messages and patch-based staging. Use when preparing commit sequences, selecting staged hunks, or writing commit messages."
---

# Conventional Commits Guide for PyGraphistry

**⚠️ BEFORE STARTING: Reread [ai/prompts/PLAN.md](PLAN.md). Reuse existing `plans/*/plan.md` if one exists, else start a new one.**
**⚠️ BEFORE STARTING: Reread [the planning skill](../plan/SKILL.md). Reuse existing `plans/*/plan.md` if one exists, else start a new one.**

PyGraphistry-specific commit conventions only. See PLAN.md for execution protocol.

Expand Down
11 changes: 8 additions & 3 deletions ai/prompts/DECOMMENT.md → .agents/skills/decomment/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
name: decomment
description: "Remove low-value comments from pull requests while preserving necessary documentation. Use before review or merge when cleaning up explanatory comments in changed code."
---

# Decomment Protocol

**Purpose:** Remove unnecessary comments from PRs while preserving valuable documentation.
Expand All @@ -15,7 +20,7 @@ This protocol guides the systematic removal of redundant comments from code chan

## Prerequisites

1. **Plan File**: Ensure you have a current plan file (see `ai/prompts/PLAN.md`)
1. **Plan File**: Ensure you have a current plan file (see `../plan/SKILL.md`)
- If no plan exists for this work, create one following PLAN.md template
- If plan exists but is stale, refresh it with current state
2. **Clean Git State**: All changes committed to feature branch
Expand All @@ -39,7 +44,7 @@ This protocol guides the systematic removal of redundant comments from code chan
**Commands:**
```bash
# Automated inventory generation (RECOMMENDED)
./ai/assets/generate_comment_inventory.sh master plans/[task]/comment_inventory.md
./.agents/assets/generate_comment_inventory.sh master plans/[task]/comment_inventory.md

# Manual alternatives:
# Get base branch
Expand Down Expand Up @@ -217,7 +222,7 @@ Phase 4: Verification Results
ls plans/[current-task]/plan.md

# If no plan exists:
# 1. Create plan following ai/prompts/PLAN.md template
# 1. Create plan following ../plan/SKILL.md template
# 2. Document decomment work as Phase N.A: "Remove redundant comments"

# If plan exists but stale:
Expand Down
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
name: gfql-llm-guide-maintenance
description: "Maintain GFQL LLM guidance docs when call surfaces, predicates, algorithms, or common failure patterns change. Use for release updates and guide conformance checks."
---

# LLM_GUIDE.md Maintenance Process

**Purpose**: Update `LLM_GUIDE.md` when GFQL features change or user patterns emerge.
Expand All @@ -13,7 +18,7 @@
5. **LLMs generating invalid JSON** → [LLM Failures](#llm-failures)
6. **Multiple changes** → [Multi-Change Releases](#multi-change-releases)

**See also**: the end-to-end checklist in `ai/docs/gfql/calls_checklist.md`
**See also**: the end-to-end checklist in `.agents/docs/gfql/calls_checklist.md`
whenever a `call()` surface changes.

## Update Triggers
Expand Down Expand Up @@ -97,7 +102,7 @@ grep "new_fn" graphistry/tests/compute/ -rn
```

**Update**: `## Graph Algorithms` (add example) + `## Call Functions` (list)
- Cross-check `ai/docs/gfql/calls_checklist.md` for required code/tests/doc updates
- Cross-check `.agents/docs/gfql/calls_checklist.md` for required code/tests/doc updates

**Format**:
```markdown
Expand Down
Original file line number Diff line number Diff line change
@@ -1,10 +1,15 @@
---
name: hoist-imports
description: "Refactor dynamic imports to top-level imports with an inventory, categorization, and verification workflow. Use during cleanup or review-prep when import placement needs standardization."
---

# Hoist Imports Protocol

**Purpose:** Move dynamic imports to top-level unless there's a documented reason not to.

## Quick Reference (60-second version)

1. **Identify**: Run `./ai/assets/find_dynamic_imports.sh master plans/[task]/dynamic_imports.md`
1. **Identify**: Run `./.agents/assets/find_dynamic_imports.sh master plans/[task]/dynamic_imports.md`
2. **Categorize**: Review each as HOIST (move to top) or KEEP (document reason)
3. **Hoist**: Move imports to top-level following PEP 8 section ordering
4. **Verify**: Run tests, confirm no circular imports or heavy load issues
Expand All @@ -27,7 +32,7 @@ This protocol guides the systematic refactoring of dynamic imports (imports insi

## Prerequisites

1. **Plan File**: Ensure you have a current plan file (see `ai/prompts/PLAN.md`)
1. **Plan File**: Ensure you have a current plan file (see `../plan/SKILL.md`)
2. **Clean Git State**: All changes committed to feature branch
3. **PR Context**: Know which branch the PR will land into (usually `master` or `main`)

Expand All @@ -50,7 +55,7 @@ This protocol guides the systematic refactoring of dynamic imports (imports insi
**Commands:**
```bash
# Automated inventory generation (RECOMMENDED)
./ai/assets/find_dynamic_imports.sh master plans/[task]/dynamic_imports.md
./.agents/assets/find_dynamic_imports.sh master plans/[task]/dynamic_imports.md

# Manual alternatives (if automation script unavailable):
# Get base branch
Expand Down Expand Up @@ -324,7 +329,7 @@ Add this phase to your plan:

**Tool Calls:**
```bash
./ai/assets/find_dynamic_imports.sh master plans/[task]/dynamic_imports.md
./.agents/assets/find_dynamic_imports.sh master plans/[task]/dynamic_imports.md
# [edit commands]
git commit -m "refactor: Hoist dynamic imports to top-level"
git push
Expand Down
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
name: lint-types-check
description: "Run iterative lint and typecheck remediation for PyGraphistry. Use when fixing flake8/mypy issues and driving a branch to clean validation results."
---

# Lint and Type Check Template for PyGraphistry

<!-- FILE TRACKING HEADER - FILL IN WHEN FILE CREATED -->
Expand Down
13 changes: 9 additions & 4 deletions ai/prompts/PLAN.md → .agents/skills/plan/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
name: plan
description: "Create and maintain a resumable phased execution plan for complex tasks. Use when work spans multiple steps or sessions and requires strict status tracking."
---

# Task Plan Template

<!-- ═══════════════════════════════════════════════════════════════════════════
Expand Down Expand Up @@ -125,9 +130,9 @@ git log --oneline -n 10
- Source: `graphistry/`
- Tests: `graphistry/tests/` (mirrors source structure: `graphistry/foo/bar.py` → `graphistry/tests/foo/test_bar.py`)
- Docs: `docs/`
- Plans: `plans/` (gitignored - safe for auxiliary files, temp secrets, working data)
- AI prompts: `ai/prompts/`
- AI docs: `ai/docs/`
- Plans: `plans/` (gitignored - safe for auxiliary files, temp secrets, working data; Codex: avoid `~/.codex/plans`; if used, copy here then delete)
- AI skills: `.agents/skills/`
- AI docs: `.agents/docs/`

### Security & Working Files

Expand All @@ -143,7 +148,7 @@ echo "export GRAPHISTRY_SERVER='https://hub.graphistry.com'" >> plans/[task]/.en
source plans/[task]/.env
```

**For commits:** See [ai/prompts/CONVENTIONAL_COMMITS.md](CONVENTIONAL_COMMITS.md)
**For commits:** See [the conventional commits skill](../conventional-commits/SKILL.md)

## Phase Protocol

Expand Down
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
name: pyre-analysis
description: "Run Pyre/Pysa analysis for call graphs and caller extraction in PyGraphistry. Use for type-aware refactors, indirect dependency tracing, and call-chain debugging."
---

# Pyre/Pysa Analysis for PyGraphistry

**Recommendation**: Use AST (< 1s) for direct patterns, then Pysa (~60s) for call chains
Expand All @@ -15,14 +20,14 @@ docker run --rm -v $(pwd):/workspace -w /workspace python:3.12-slim \
ls -lh pysa_results/call-graph.json # Should be ~16MB, 37626 functions
```

**3. Extract callers** (see `ai/assets/pysa_extract_callers.py`):
**3. Extract callers** (see `.agents/assets/pysa_extract_callers.py`):
```bash
# Single method
python3 ai/assets/pysa_extract_callers.py pysa_results/call-graph.json \
python3 .agents/assets/pysa_extract_callers.py pysa_results/call-graph.json \
PlotterBase.PlotterBase.bind

# Multiple methods
python3 ai/assets/pysa_extract_callers.py pysa_results/call-graph.json \
python3 .agents/assets/pysa_extract_callers.py pysa_results/call-graph.json \
PlotterBase.PlotterBase.bind \
PlotterBase.PlotterBase.nodes \
PlotterBase.PlotterBase.edges
Expand Down
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -105,3 +105,7 @@ AI_PROGRESS/
plans/
tmp/
test_env/

# Claude Code local setup
.claude/settings.local.json
.claude/skills
17 changes: 17 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,23 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm
## [Development]
<!-- Do Not Erase This Section - Used for tracking unreleased changes -->

### Added
- **Collections**: New `g.collections(...)` API for defining subsets via GFQL expressions with priority-based visual encodings. Includes helper constructors `graphistry.collection_set(...)` and `graphistry.collection_intersection(...)`, support for `showCollections`, `collectionsGlobalNodeColor`, and `collectionsGlobalEdgeColor` URL params, and automatic JSON encoding. Accepts GFQL AST, Chain objects, or wire-protocol dicts (#874).
- **Docs / Collections**: Added collections usage guide in visualization/layout/settings, tutorial notebook (`demos/more_examples/graphistry_features/collections.ipynb`), and cross-references in 10-minute guides, cheatsheet, and GFQL docs (#875).

### Changed
- **Collections**: Autofix validation now drops invalid collections (e.g., invalid GFQL ops) and non-string collection color fields instead of string-coercing them; warnings still emit when `warn=True`.
- **Collections**: `collections(...)` now always canonicalizes to URL-encoded JSON (string inputs are parsed + re-encoded); the `encode` parameter was removed to avoid ambiguous behavior.
- **Collections**: Set collections now require an `id` field (server requires it for subgraph storage); missing IDs are warned and dropped in autofix mode rather than auto-generated.
- **Collections**: Intersection collections now cross-validate that referenced set IDs exist; dangling references are warned and dropped in autofix mode.
- **Collections**: GFQL parsing consolidated to use `_wrap_gfql_expr` from `collections.py` as the canonical implementation with precise exception handling.

### Tests
- **Collections**: Added `test_collections.py` covering encoding, GFQL Chain/AST normalization, wire-protocol acceptance, validation modes, and helper constructors.

### Docs
- **AI tooling**: Migrated legacy assistant prompts from `ai/prompts/` into structured skills under `.agents/skills/*/SKILL.md` (with frontmatter), and moved assistant docs/assets into `.agents/` with updated internal references (#930).

## [0.50.6 - 2026-01-27]

### Fixed
Expand Down
12 changes: 11 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
@@ -1 +1,11 @@
See [ai/README.md](ai/README.md) for AI assistant development guidance.
@AGENTS.md

## Claude Code Setup

If no repo skills appear in Claude Code (check with `/skills`), configure the local symlink:

```bash
ln -s ../.agents/skills .claude/skills
```

If `.claude/skills` already exists and is wrong, replace it with the symlink above.
Loading