Add an SDK Tier Audit skill and agentic workflow#1503
Open
jeffhandley wants to merge 7 commits intomodelcontextprotocol:mainfrom
Open
Add an SDK Tier Audit skill and agentic workflow#1503jeffhandley wants to merge 7 commits intomodelcontextprotocol:mainfrom
jeffhandley wants to merge 7 commits intomodelcontextprotocol:mainfrom
Conversation
Add a weekly agentic workflow that performs SEP-1730 tier audits of the C# MCP SDK using the mcp-sdk-tier-audit skill from the conformance repository. The workflow: - Runs weekly on Thursdays (fuzzy ~6:30am EST) and on manual dispatch - Supports two scopes: Conformance + Repo Health (default) and Repo Health - Allows overriding the C# SDK and conformance repo/branch targets - Files issues with tier results, auto-closing previous audit issues - Uses a dedicated AUDIT_PAT pool for Copilot engine authentication - Skips scheduled runs on forks; allows manual dispatch anywhere The audit logic is extracted into a reusable skill at .github/skills/sdk-tier-audit/SKILL.md with cross-platform instructions (macOS/Linux and Windows) so it can also be invoked locally. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add 'output' input: 'Create Issue' (default) or 'Action Summary'
When 'Action Summary', the report is written to the step summary
and uploaded as an artifact but no issue is created.
- Condense repo/branch inputs into owner/repo:branch format:
csharp_sdk: 'modelcontextprotocol/csharp-sdk:main'
conformance: 'modelcontextprotocol/conformance:main'
Parsed with bash parameter expansion (${var%%:*} / ${var#*:})
- Add post-steps to upload /tmp/audit-report.md as an artifact
(90-day retention, ignore if missing)
- Executive summary now uses bullet points instead of a paragraph
- Audit report always written to $GITHUB_STEP_SUMMARY regardless
of output mode, so the summary page always shows results
- Agent writes a single /tmp/audit-report.md combining executive
summary + assessment + remediation, used for both issue body
and action summary
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add tools.github.min-integrity: approved for content guard policy - Add pull-requests: read permission (required by default toolsets) - Add FORCE_JAVASCRIPT_ACTIONS_TO_NODE24=true env var to opt into Node 24 early and suppress the Node 20 deprecation warning - Add max: 1 to create-issue safe output for explicit limit Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Two issues fixed: 1. Tier logic: The prompt was telling the agent to 'Apply tier logic' itself (Step 2.4). Now the prompt explicitly says: 'Do not apply your own tier logic or scoring — use only the conformance skill's thresholds, rules, and templates.' All tier determination is delegated to the conformance repo's mcp-sdk-tier-audit skill. 2. Action summary: The report was not appearing on the workflow summary page because the agent wasn't writing to $GITHUB_STEP_SUMMARY reliably. Now Step 3 is restructured with explicit requirements: - MUST write /tmp/audit-report.md (artifact) - MUST cat it to $GITHUB_STEP_SUMMARY (action summary) - Both required BEFORE Step 4 (publish) - 'Action Summary' mode now explicitly calls noop - Issue body must be identical to the action summary content Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Introduce an SDK Tier Audit skill and agentic workflow to automate weekly runs of the audit, producing an issue in this repo to capture the audit report.
The workflow offers execution modes for:
Other notes:
Example results: