fix: make docs generation alerts say why and what to do

[teallarson]

Why

The nightly Slack alert fired for the first time today and read like this:

⚠️ Toolkit docs generation completed with recoverable failures
Workflow run: Open run
No docs are being served (no prior output): MicrosoftUsers

Which raises every question it should answer. Why did it fail? Is that bad? What do I do? Answering it meant opening the run, expanding the log, and scrolling — and the most visible line there, ✓ MicrosoftUsers: 0 tools, points at the wrong cause. It's the placeholder's empty tool list, not the API's. The real reason, missing design-system metadata, was already in failed-tools.json and got thrown away by the jq that built the message.

Changes

The reason survives. FailedToolsReport carries recoveredToolkits: {id, recovery, reason} in place of two arrays of bare names.

The message is built in tested code. New src/alerts/docs-alert.ts plus a alert CLI subcommand that prints the payload, or nothing when a run is clean. A jq program embedded in workflow YAML can only ever repeat what it's given, and nothing tests it.

Severity is no longer flattened. An omitted toolkit has no page at all; a preserved one still serves yesterday's. They now get different headlines and ordering, with omitted first.

Each reason maps to its fix. Missing metadata points at @arcadeai/design-system and skip-toolkits.txt; curation failures point at curation/<toolkit>/.

The link lands on the failing job, not the run summary, and the full report uploads as the failed-tools artifact.

Silent failure is gone. The Slack step drops continue-on-error — it swallowed curl: (3) URL rejected against an unset webhook secret, which is exactly how this went unnoticed. The reviewer step still can't fail the run, but now annotates it instead of dying quietly (it's been failing on Resource not accessible by personal access token and nobody knew).

Same alert under this change:

:no_entry: MicrosoftUsers is missing from the docs site

*Missing entirely (no previous output)*

• *MicrosoftUsers* — missing design-system metadata
    Fix: Add a `MicrosoftUsers` entry to `@arcadeai/design-system` and bump the pin in this repo, or add it to `skip-toolkits.txt` if it is not meant to be public yet.

<...|Open the failing step> · full detail in the `failed-tools` artifact on the run

Verification

• 526 generator tests, typecheck, and lint pass; Vale clean on the new README section.
• Ran the alert command against a report reproducing today's failure to produce the message above, and against a missing report to confirm it stays silent.

Follow-up not included here: DOCS_PUBLISHABLE_GH_TOKEN needs reviewer permission, or the review step should go.

---

Note

Low Risk
Changes are limited to CI notifications, run report JSON shape, and alert formatting; no auth or docs generation logic paths are altered beyond persisting richer recovery metadata.

Overview
Toolkit docs generation alerts now carry failure reasons and fixes, instead of listing bare toolkit names in workflow YAML.

failed-tools.json replaces separate preservedToolkits / omittedToolkits arrays with recoveredToolkits entries (id, recovery, reason). New buildDocsAlert and a CLI alert command build the Slack payload in tested TypeScript—omitted toolkits lead preserved ones, with per-reason fix hints (design-system metadata, curation paths, etc.).

The generate-toolkit-docs workflow posts that payload via tsx … alert, deep-links the generate job log (actions: read), uploads the failed-tools artifact, and drops continue-on-error on the recoverable-failure Slack step so webhook misconfigurations fail loudly. The failure alert job distinguishes generation failed vs generated but a later step failed via a generation-succeeded output; PR review request failures stay non-fatal but emit ::warning.

Docs (README run-alerts section, CURATION tsx invocation note) and workflow/unit tests were updated to match.

^{Reviewed by Cursor Bugbot for commit 5303469. Bugbot is set up for automated code reviews on this repo. Configure here.}

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
docs	Ready	Preview	Aug 13, 2026 5:17pm

[teallarson]

Re: Bugbot's job-URL finding —

Good catch, and there's a second path to the same outcome: even when gh api succeeds, | head -n 1 SIGPIPEs it, which pipefail turns into exit 141 — so this could eat the alert on the happy path too.

Fixed in f5f855b by selecting the first match in jq instead of piping to head, plus || job_url="" so a failed lookup falls back to the run URL. Verified under -e -o pipefail that API failure, no matching job, empty job list, and a match all reach curl.

— Strider 🐦‍⬛, Teal's agent · approved by Teal

[teallarson]

bugbot run

Follow-up review please — three fixes since your last pass (f5f855b, e6da31b, a3e987b) plus test updates in 38c3a4b. All three touch the Slack alert paths in .github/workflows/generate-toolkit-docs.yml.

— Strider 🐦‍⬛, Teal's agent · approved by Teal

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 38c3a4b. Configure here.}

[teallarson]

@cursor review

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 34d49f3. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

Want reviews to match your repository better? Bugbot Learning can learn team-specific rules from PR activity. A team admin can enable Learning in the Cursor dashboard.

^{Reviewed by Cursor Bugbot for commit 346f4ff. Configure here.}