Skip to content

DOC Upgrade to jupyterbook v2 and add proper landing page#1458

Open
romanlutz wants to merge 22 commits intoAzure:mainfrom
romanlutz:upgrade-jupyter-book
Open

DOC Upgrade to jupyterbook v2 and add proper landing page#1458
romanlutz wants to merge 22 commits intoAzure:mainfrom
romanlutz:upgrade-jupyter-book

Conversation

@romanlutz
Copy link
Contributor

@romanlutz romanlutz commented Mar 12, 2026

Related to #362

Summary

This PR upgrades our documentation framework from Jupyter Book 1.0.4 (Sphinx-based, maintenance-only) to Jupyter Book 2.1.2 (MyST engine, actively developed) and introduces a custom landing page with a
hero banner and key capabilities overview.

Why

  • JB
    1.x is in maintenance mode — no new features, only critical fixes
  • JB
    2.x offers faster builds (~18s vs ~60-100s), modern web features, native mermaid support, and a unified myst.yml config
  • Our landing page was a plain markdown page — now it has a hero banner, feature highlights, and installation cards matching modern documentation sites

What Changed

Documentation Framework

  • Upgraded jupyter-book from ==1.0.4 to >=2.0.0
  • Migrated _config.yml + _toc.yml → unified myst.yml (auto-migrated by JB 2 CLI)
  • Removed conf.py and api.rst (Sphinx-specific, no longer needed)
  • Mermaid diagrams now render natively (removed sphinxcontrib-mermaid dependency)

API Reference

  • JB 2 drops all Sphinx extensions, so autodoc/autosummary/napoleon no longer work
  • Built a replacement using griffe (ISC license) for static code analysis with Google-style docstring parsing
  • build_scripts/pydoc2json.py generates structured JSON from Python source
  • build_scripts/gen_api_md.py converts JSON → MyST markdown API pages
  • This is a workaround until MyST adds native API doc support (tracking issue)
  • Generated doc/api/ and doc/_api/ are gitignored — fully regenerated during make docs-build

Landing Page

  • Custom hero section with banner.png as full-width background
  • "Key Capabilities" section highlighting: Automated Red Teaming, Scenario Framework, CoPyRIT, Any Target, Built-in Memory, Flexible Scoring
  • Installation guide with 4 clickable cards (Docker, uv, DevContainers, Local Dev)
  • Top-level horizontal navigation bar (Cookbooks, Setup, Contributing, User Guide, API Reference, Blog)
  • Custom CSS for dark mode logo fix and hero banner styling

Build Pipeline

  • make docs-build now runs: API JSON generation → API markdown generation → JB 2 build → RSS feed
  • make docs-api for standalone API reference regeneration
  • Updated .github/workflows/docs.yml to use make docs-build
  • Updated .readthedocs.yaml for JB 2 (added Node.js requirement, custom build commands)
  • New build_scripts/validate_docs.py pre-commit hook — validates myst.yml TOC references and detects orphaned files (<1s vs full build)

Dependencies

  • Added: griffe>=2.0.0, jupyter-book>=2.0.0
  • Removed: jupyter-book==1.0.4, sphinxcontrib-mermaid>=1.0.0
  • Kept: jupytext (still needed for .py/.ipynb sync)

How to Test

Build the docs

make docs-build

Or serve locally

cd doc && jupyter-book start --port 8888

Screenshots

image image

romanlutz and others added 15 commits March 12, 2026 06:44
- Upgrade jupyter-book from 1.0.4 (Sphinx-based) to 2.1.2 (MyST engine)
- Replace Sphinx autodoc/autosummary/napoleon with griffe-based API doc generation
- Add scripts/pydoc2json.py and scripts/gen_api_md.py for API doc generation
- Add doc/api/ with generated API reference pages for all pyrit submodules
- Add doc/myst.yml unified config replacing _config.yml and _toc.yml
- Custom landing page with hero banner, key capabilities, installation cards
- Top-level navigation bar
- Update Makefile, GitHub Actions, ReadTheDocs, pre-commit hooks
- Remove old Sphinx-dependent build scripts and config files
- Add griffe to dev dependencies, remove sphinxcontrib-mermaid

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…down)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
These pre-commit hooks are still needed for .ipynb files (strip kernelspec
metadata and sanitize user-specific paths).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ok.py)

build_scripts/validate_docs.py checks myst.yml TOC references exist and
detects orphaned doc files. Runs in <1s vs full build taking 18s+.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…s to hyphens

JB2 URL slugging: 0_dataset → dataset, 1a_install_uv → a-install-uv,
0_prompt_targets → prompt-targets. Updated all site.nav URLs to match.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…xists)

API docs are now auto-generated from source by griffe, which reads __all__
directly. The generation script itself ensures completeness.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
romanlutz and others added 7 commits March 12, 2026 19:32
- Replace backslash path separators with forward slashes in doc/myst.yml
  for cross-platform compatibility (Windows + WSL/Linux)
- Replace snake emojis with sun emojis on landing page cards (conda -> uv)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- pydoc2json.py: pass include_submodules=True recursively so nested
  submodules (e.g. pyrit.executor.attack) are included in pyrit_all.json
- gen_api_md.py: split aggregate JSON into per-module files before
  generating markdown, so all API pages referenced by myst.yml exist
- generate_rss.py: update path from _build/html to _build/site for JB2

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
JB2 no longer produces standalone HTML blog pages. Rewrite
generate_rss.py to parse blog source markdown files directly
from doc/blog/ and output RSS to doc/_build/site/blog/rss.xml.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants