From 9faa59e8a2468db95229faa17b2c53c852b16144 Mon Sep 17 00:00:00 2001 From: Jeff Handley Date: Sat, 4 Apr 2026 20:38:20 -0700 Subject: [PATCH 1/7] Move release readme out of workflows folder --- .github/{workflows/release.md => release-readme.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename .github/{workflows/release.md => release-readme.md} (100%) diff --git a/.github/workflows/release.md b/.github/release-readme.md similarity index 100% rename from .github/workflows/release.md rename to .github/release-readme.md From e3664afa3d47ec4f5faf71b128b1e4c9becef6ef Mon Sep 17 00:00:00 2001 From: Jeff Handley Date: Sat, 4 Apr 2026 13:32:16 -0700 Subject: [PATCH 2/7] Initialize Agentic Workflows --- .gitattributes | 2 + .github/agents/agentic-workflows.agent.md | 178 ++++++++++++++++++++++ .vscode/settings.json | 5 + 3 files changed, 185 insertions(+) create mode 100644 .github/agents/agentic-workflows.agent.md create mode 100644 .vscode/settings.json diff --git a/.gitattributes b/.gitattributes index 956f7739c..43ed78791 100644 --- a/.gitattributes +++ b/.gitattributes @@ -62,3 +62,5 @@ *.fsproj text *.dbproj text *.sln text eol=crlf + +.github/workflows/*.lock.yml linguist-generated=true merge=ours \ No newline at end of file diff --git a/.github/agents/agentic-workflows.agent.md b/.github/agents/agentic-workflows.agent.md new file mode 100644 index 000000000..4d5ecfe8e --- /dev/null +++ b/.github/agents/agentic-workflows.agent.md @@ -0,0 +1,178 @@ +--- +description: GitHub Agentic Workflows (gh-aw) - Create, debug, and upgrade AI-powered workflows with intelligent prompt routing +disable-model-invocation: true +--- + +# GitHub Agentic Workflows Agent + +This agent helps you work with **GitHub Agentic Workflows (gh-aw)**, a CLI extension for creating AI-powered workflows in natural language using markdown files. + +## What This Agent Does + +This is a **dispatcher agent** that routes your request to the appropriate specialized prompt based on your task: + +- **Creating new workflows**: Routes to `create` prompt +- **Updating existing workflows**: Routes to `update` prompt +- **Debugging workflows**: Routes to `debug` prompt +- **Upgrading workflows**: Routes to `upgrade-agentic-workflows` prompt +- **Creating report-generating workflows**: Routes to `report` prompt — consult this whenever the workflow posts status updates, audits, analyses, or any structured output as issues, discussions, or comments +- **Creating shared components**: Routes to `create-shared-agentic-workflow` prompt +- **Fixing Dependabot PRs**: Routes to `dependabot` prompt — use this when Dependabot opens PRs that modify generated manifest files (`.github/workflows/package.json`, `.github/workflows/requirements.txt`, `.github/workflows/go.mod`). Never merge those PRs directly; instead update the source `.md` files and rerun `gh aw compile --dependabot` to bundle all fixes +- **Analyzing test coverage**: Routes to `test-coverage` prompt — consult this whenever the workflow reads, analyzes, or reports on test coverage data from PRs or CI runs + +Workflows may optionally include: + +- **Project tracking / monitoring** (GitHub Projects updates, status reporting) +- **Orchestration / coordination** (one workflow assigning agents or dispatching and coordinating other workflows) + +## Files This Applies To + +- Workflow files: `.github/workflows/*.md` and `.github/workflows/**/*.md` +- Workflow lock files: `.github/workflows/*.lock.yml` +- Shared components: `.github/workflows/shared/*.md` +- Configuration: https://github.com/github/gh-aw/blob/v0.66.1/.github/aw/github-agentic-workflows.md + +## Problems This Solves + +- **Workflow Creation**: Design secure, validated agentic workflows with proper triggers, tools, and permissions +- **Workflow Debugging**: Analyze logs, identify missing tools, investigate failures, and fix configuration issues +- **Version Upgrades**: Migrate workflows to new gh-aw versions, apply codemods, fix breaking changes +- **Component Design**: Create reusable shared workflow components that wrap MCP servers + +## How to Use + +When you interact with this agent, it will: + +1. **Understand your intent** - Determine what kind of task you're trying to accomplish +2. **Route to the right prompt** - Load the specialized prompt file for your task +3. **Execute the task** - Follow the detailed instructions in the loaded prompt + +## Available Prompts + +### Create New Workflow +**Load when**: User wants to create a new workflow from scratch, add automation, or design a workflow that doesn't exist yet + +**Prompt file**: https://github.com/github/gh-aw/blob/v0.66.1/.github/aw/create-agentic-workflow.md + +**Use cases**: +- "Create a workflow that triages issues" +- "I need a workflow to label pull requests" +- "Design a weekly research automation" + +### Update Existing Workflow +**Load when**: User wants to modify, improve, or refactor an existing workflow + +**Prompt file**: https://github.com/github/gh-aw/blob/v0.66.1/.github/aw/update-agentic-workflow.md + +**Use cases**: +- "Add web-fetch tool to the issue-classifier workflow" +- "Update the PR reviewer to use discussions instead of issues" +- "Improve the prompt for the weekly-research workflow" + +### Debug Workflow +**Load when**: User needs to investigate, audit, debug, or understand a workflow, troubleshoot issues, analyze logs, or fix errors + +**Prompt file**: https://github.com/github/gh-aw/blob/v0.66.1/.github/aw/debug-agentic-workflow.md + +**Use cases**: +- "Why is this workflow failing?" +- "Analyze the logs for workflow X" +- "Investigate missing tool calls in run #12345" + +### Upgrade Agentic Workflows +**Load when**: User wants to upgrade workflows to a new gh-aw version or fix deprecations + +**Prompt file**: https://github.com/github/gh-aw/blob/v0.66.1/.github/aw/upgrade-agentic-workflows.md + +**Use cases**: +- "Upgrade all workflows to the latest version" +- "Fix deprecated fields in workflows" +- "Apply breaking changes from the new release" + +### Create a Report-Generating Workflow +**Load when**: The workflow being created or updated produces reports — recurring status updates, audit summaries, analyses, or any structured output posted as a GitHub issue, discussion, or comment + +**Prompt file**: https://github.com/github/gh-aw/blob/v0.66.1/.github/aw/report.md + +**Use cases**: +- "Create a weekly CI health report" +- "Post a daily security audit to Discussions" +- "Add a status update comment to open PRs" + +### Create Shared Agentic Workflow +**Load when**: User wants to create a reusable workflow component or wrap an MCP server + +**Prompt file**: https://github.com/github/gh-aw/blob/v0.66.1/.github/aw/create-shared-agentic-workflow.md + +**Use cases**: +- "Create a shared component for Notion integration" +- "Wrap the Slack MCP server as a reusable component" +- "Design a shared workflow for database queries" + +### Fix Dependabot PRs +**Load when**: User needs to close or fix open Dependabot PRs that update dependencies in generated manifest files (`.github/workflows/package.json`, `.github/workflows/requirements.txt`, `.github/workflows/go.mod`) + +**Prompt file**: https://github.com/github/gh-aw/blob/v0.66.1/.github/aw/dependabot.md + +**Use cases**: +- "Fix the open Dependabot PRs for npm dependencies" +- "Bundle and close the Dependabot PRs for workflow dependencies" +- "Update @playwright/test to fix the Dependabot PR" + +### Analyze Test Coverage +**Load when**: The workflow reads, analyzes, or reports test coverage — whether triggered by a PR, a schedule, or a slash command. Always consult this prompt before designing the coverage data strategy. + +**Prompt file**: https://github.com/github/gh-aw/blob/v0.66.1/.github/aw/test-coverage.md + +**Use cases**: +- "Create a workflow that comments coverage on PRs" +- "Analyze coverage trends over time" +- "Add a coverage gate that blocks PRs below a threshold" + +## Instructions + +When a user interacts with you: + +1. **Identify the task type** from the user's request +2. **Load the appropriate prompt** from the GitHub repository URLs listed above +3. **Follow the loaded prompt's instructions** exactly +4. **If uncertain**, ask clarifying questions to determine the right prompt + +## Quick Reference + +```bash +# Initialize repository for agentic workflows +gh aw init + +# Generate the lock file for a workflow +gh aw compile [workflow-name] + +# Debug workflow runs +gh aw logs [workflow-name] +gh aw audit + +# Upgrade workflows +gh aw fix --write +gh aw compile --validate +``` + +## Key Features of gh-aw + +- **Natural Language Workflows**: Write workflows in markdown with YAML frontmatter +- **AI Engine Support**: Copilot, Claude, Codex, or custom engines +- **MCP Server Integration**: Connect to Model Context Protocol servers for tools +- **Safe Outputs**: Structured communication between AI and GitHub API +- **Strict Mode**: Security-first validation and sandboxing +- **Shared Components**: Reusable workflow building blocks +- **Repo Memory**: Persistent git-backed storage for agents +- **Sandboxed Execution**: All workflows run in the Agent Workflow Firewall (AWF) sandbox, enabling full `bash` and `edit` tools by default + +## Important Notes + +- Always reference the instructions file at https://github.com/github/gh-aw/blob/v0.66.1/.github/aw/github-agentic-workflows.md for complete documentation +- Use the MCP tool `agentic-workflows` when running in GitHub Copilot Cloud +- Workflows must be compiled to `.lock.yml` files before running in GitHub Actions +- **Bash tools are enabled by default** - Don't restrict bash commands unnecessarily since workflows are sandboxed by the AWF +- Follow security best practices: minimal permissions, explicit network access, no template injection +- **Network configuration**: Use ecosystem identifiers (`node`, `python`, `go`, etc.) or explicit FQDNs in `network.allowed`. Bare shorthands like `npm` or `pypi` are **not** valid. See https://github.com/github/gh-aw/blob/v0.66.1/.github/aw/network.md for the full list of valid ecosystem identifiers and domain patterns. +- **Single-file output**: When creating a workflow, produce exactly **one** workflow `.md` file. Do not create separate documentation files (architecture docs, runbooks, usage guides, etc.). If documentation is needed, add a brief `## Usage` section inside the workflow file itself. diff --git a/.vscode/settings.json b/.vscode/settings.json new file mode 100644 index 000000000..dbd4bd793 --- /dev/null +++ b/.vscode/settings.json @@ -0,0 +1,5 @@ +{ + "github.copilot.enable": { + "markdown": true + } +} \ No newline at end of file From e74f402e80a6ba451f84995b1b9a22127452e547 Mon Sep 17 00:00:00 2001 From: Jeff Handley Date: Sat, 4 Apr 2026 13:39:03 -0700 Subject: [PATCH 3/7] Install the select-copilot-pat action and integrate into agent --- .github/actions/select-copilot-pat/README.md | 147 ++++++++++++++++++ .github/actions/select-copilot-pat/action.yml | 62 ++++++++ .github/agents/agentic-workflows.agent.md | 83 +++++++++- 3 files changed, 291 insertions(+), 1 deletion(-) create mode 100644 .github/actions/select-copilot-pat/README.md create mode 100644 .github/actions/select-copilot-pat/action.yml diff --git a/.github/actions/select-copilot-pat/README.md b/.github/actions/select-copilot-pat/README.md new file mode 100644 index 000000000..08408208c --- /dev/null +++ b/.github/actions/select-copilot-pat/README.md @@ -0,0 +1,147 @@ +# Select Copilot PAT + +Selects a random Copilot PAT from a numbered pool of secrets. This addresses limitations that arise from having a single PAT shared across all agentic workflows, such as rate-limiting. + +**This is a stop-gap workaround.** As soon as organization/enterprise billing is offered for agentic workflows, this approach will be removed from our workflows. + +## Repository Onboarding + +To use Agentic Workflows in a repository: + +1. Follow the instructions for [Configuring Your Repository | Agentic Authoring | GitHub Agentic Workflows][configure-repo]. +2. Copy this `select-copilot-pat` folder into the repository under `.github/actions/select-copilot-pat`, including both the `README.md` and `action.yml`. +3. Merge those additions into the repository and then follow the instructions for the PAT Creation and Usage below. + +> **Optional:** If you plan to manage secrets or workflows from the command line (e.g., `gh aw secrets set`), [install the `gh aw` CLI extension][cli-setup]: +> +> ```sh +> gh extension install github/gh-aw +> ``` + +## PAT Management + +Team members provide PATs into the pools for the repository by adding them as repository secrets with secret names matching the pattern of `_<0-9>`, such as `COPILOT_PAT_0`. + +[Use this link to prefill the PAT creation form with the required settings][create-pat]: + +1. **Resource owner** is your **user account**, not an organization. +2. **Copilot Requests (Read)** must be granted. +3. **Repository access** set to **Public repositories** only — or, if the consuming workflow's agent needs to read private repositories, add those repos explicitly. +4. Any additional **read-only** permissions the consuming workflow's agent requires (e.g., reading issues or metadata from other repositories) should also be included in the PAT scope. Consult the workflow's documentation for its specific requirements. + +The **Token Name** _does not_ need to match the secret name and is only visible to the owner of the PAT. It's recommended to use a token name indicating the PAT is used for agentic workflows in this repository. The **Description** is also only used for your own reference. + +Team members providing PATs for workflows should set recurring reminders to regenerate and update their PATs in the repository secrets before they expire. + +PATs are added to repositories through the **Settings > Secrets and variables > Actions** UI, saved as **Repository secrets** and matching the `_<0-9>` naming convention. This can also be done using the GitHub CLI. + +```sh +gh aw secrets set "_<0-9>" --value "" --repo +``` + +## Workflow Output Attribution + +Team members' PATs are _only_ used for the Copilot requests from within the agentic portion of the workflow. All outputs from the workflow use the `github-actions[bot]` account token. Issues, PRs, comments, and all other content generated by the workflow will be attributed to `github-actions[bot]`--not the team member's account or token. + +## Usage + +Add the following frontmatter at the top-level of an agentic workflow. These elements are not supported through [imports][imports], so they must be copied into all workflows. + +Up to 10 `SECRET_#` environment variables can be passed to the action, numbered 0-9. Different workflows can use different pools of PATs if desired. Change the `secrets.COPILOT_PAT_0` through `secrets.COPILOT_PAT_9` secret names in both the `select-copilot-pat` step `env` values and in the `case` expression under the `engine: env` configuration. + +```yml +on: + # Add the pre-activation step of selecting a random PAT from the supplied secrets + steps: + - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 + name: Checkout the select-copilot-pat action folder + with: + persist-credentials: false + sparse-checkout: .github/actions/select-copilot-pat + sparse-checkout-cone-mode: true + fetch-depth: 1 + + - id: select-copilot-pat + name: Select Copilot token from pool + uses: ./.github/actions/select-copilot-pat + env: + # If the secret names are changed here, they must also be changed + # in the `engine: env` case expression + SECRET_0: ${{ secrets.COPILOT_PAT_0 }} + SECRET_1: ${{ secrets.COPILOT_PAT_1 }} + SECRET_2: ${{ secrets.COPILOT_PAT_2 }} + SECRET_3: ${{ secrets.COPILOT_PAT_3 }} + SECRET_4: ${{ secrets.COPILOT_PAT_4 }} + SECRET_5: ${{ secrets.COPILOT_PAT_5 }} + SECRET_6: ${{ secrets.COPILOT_PAT_6 }} + SECRET_7: ${{ secrets.COPILOT_PAT_7 }} + SECRET_8: ${{ secrets.COPILOT_PAT_8 }} + SECRET_9: ${{ secrets.COPILOT_PAT_9 }} + +# Add the pre-activation output of the randomly selected PAT +jobs: + pre-activation: + outputs: + copilot_pat_number: ${{ steps.select-copilot-pat.outputs.copilot_pat_number }} + +# Override the COPILOT_GITHUB_TOKEN expression used in the activation job +# Consume the PAT number from the pre-activation step and select the corresponding secret +engine: + id: copilot + env: + # We cannot use line breaks in this expression as it leads to a syntax error in the compiled workflow + # If none of the `COPILOT_PAT_#` secrets were selected, then the default COPILOT_GITHUB_TOKEN is used + COPILOT_GITHUB_TOKEN: ${{ case(needs.pre_activation.outputs.copilot_pat_number == '0', secrets.COPILOT_PAT_0, needs.pre_activation.outputs.copilot_pat_number == '1', secrets.COPILOT_PAT_1, needs.pre_activation.outputs.copilot_pat_number == '2', secrets.COPILOT_PAT_2, needs.pre_activation.outputs.copilot_pat_number == '3', secrets.COPILOT_PAT_3, needs.pre_activation.outputs.copilot_pat_number == '4', secrets.COPILOT_PAT_4, needs.pre_activation.outputs.copilot_pat_number == '5', secrets.COPILOT_PAT_5, needs.pre_activation.outputs.copilot_pat_number == '6', secrets.COPILOT_PAT_6, needs.pre_activation.outputs.copilot_pat_number == '7', secrets.COPILOT_PAT_7, needs.pre_activation.outputs.copilot_pat_number == '8', secrets.COPILOT_PAT_8, needs.pre_activation.outputs.copilot_pat_number == '9', secrets.COPILOT_PAT_9, secrets.COPILOT_GITHUB_TOKEN) }} +``` + +## Design / Security + +There are several details of this implementation that keep our workflows and repositories safe. + +1. **Secrets adhere to existing trust boundaries.** The pool of PAT secrets is + provided to the `select-copilot-pat` action within the `pre_activation` + job, which is a deterministic and trusted portion of the workflow. No + untrusted context or input is within scope during this job. The action step + runs within that job, and the secrets do not get passed across contexts. The + `select-copilot-pat` action only references the secret values to determine + which values are non-empty, filtering the secret numbers to those with + values. +1. **The `select-copilot-pat` action does not require any permissions.** It + merely selects a random number from the pool of non-empty secrets and + returns the _number_ (**not the secret**). The consuming workflow uses the + returned secret number to provide the corresponding PAT to the agent job. +1. **The implementation uses existing extensibility hooks in Agentic + Workflows.** Everything is supported by `gh aw compile` in this approach, + and no hand-editing of the compiled output is required. The `pre_activation` + job is designed for this type of extensibility, and the + [secret override][secret-override] capability was added to support using a + secret with a name different from the default `COPILOT_GITHUB_TOKEN`. + +Each of the references below contributed to the design and implementation to ensure a secure and reliable design. + +## References + +- [Agentic Workflows CLI Extension][cli-setup] +- [Agentic Authoring][configure-repo] +- [Authentication][authentication] +- [Agentic Workflow Imports][imports] +- [Custom Steps][steps] +- [Custom Jobs][jobs] +- [Job Outputs][job-outputs] +- [Engine Configuration][engine] +- [Engine Environment Variables][engine-vars] +- [Case Function in Workflow Expressions][case-expression] +- [Update agentic engine token handling to use user-provided secrets (github/gh-aw#18017)][secret-override] + +[cli-setup]: https://github.github.com/gh-aw/setup/cli/ +[configure-repo]: https://github.github.com/gh-aw/guides/agentic-authoring/#configuring-your-repository +[authentication]: https://github.github.com/gh-aw/reference/auth/ +[create-pat]: https://github.com/settings/personal-access-tokens/new?name=Agentic%20Workflows%20PAT&description=GitHub+Agentic+Workflows+-+Copilot+engine+authentication.+Must+be+configured+with+Copilot+Requests+permission+and+user+account+as+resource+owner.&user_copilot_requests=read +[imports]: https://github.github.com/gh-aw/reference/imports/ +[steps]: https://github.github.com/gh-aw/reference/frontmatter/#custom-steps-steps +[jobs]: https://github.github.com/gh-aw/reference/frontmatter/#custom-jobs-jobs +[job-outputs]: https://github.github.com/gh-aw/reference/frontmatter/#job-outputs +[engine]: https://github.github.com/gh-aw/reference/frontmatter/#ai-engine-engine +[engine-vars]: https://github.github.com/gh-aw/reference/engines/#engine-environment-variables +[case-expression]: https://docs.github.com/en/actions/reference/workflows-and-actions/expressions#case +[secret-override]: https://github.com/github/gh-aw/pull/18017 diff --git a/.github/actions/select-copilot-pat/action.yml b/.github/actions/select-copilot-pat/action.yml new file mode 100644 index 000000000..d3d38a7e5 --- /dev/null +++ b/.github/actions/select-copilot-pat/action.yml @@ -0,0 +1,62 @@ +--- +name: "Select Copilot PAT from Pool" +description: >- + Selects a random Copilot PAT from a numbered pool of secrets. Secrets are + passed as environment variables SECRET_0 through SECRET_9 by the calling + workflow step. + +inputs: + random-seed: + description: >- + A seed number to use for the random PAT selection, for deterministic + selection if needed. + required: false + default: "" + +outputs: + copilot_pat_number: + description: >- + The 0-9 secret number selected from the pool of specified secrets + value: ${{ steps.select-pat-number.outputs.copilot_pat_number }} + +runs: + using: composite + steps: + - id: select-pat-number + shell: bash + env: + RANDOM_SEED: ${{ inputs.random-seed }} + run: | + # Collect numbers with non-empty secrets from SECRET_0..SECRET_9. + PAT_NUMBERS=() + for i in $(seq 0 9); do + var="SECRET_${i}" + val="${!var}" + if [ -n "$val" ]; then + PAT_NUMBERS+=(${i}) + fi + done + + # If none of the secrets in the pool have values, emit a warning + # and do not set an output value. The consumer can then fall back + # to using COPILOT_GITHUB_TOKEN. + if [ ${#PAT_NUMBERS[@]} -eq 0 ]; then + warning_message="::warning::None of the specified secrets had values " + warning_message+="(checked SECRET_0 through SECRET_9)" + echo "$warning_message" + exit 0 + fi + + # Select a random index using the seed if specified + if [ -n "$RANDOM_SEED" ]; then + RANDOM=$RANDOM_SEED + fi + + PAT_INDEX=$(( RANDOM % ${#PAT_NUMBERS[@]} )) + PAT_NUMBER="${PAT_NUMBERS[$PAT_INDEX]}" + selection_message="Selected token ${PAT_NUMBER}" + selection_details="(index: ${PAT_INDEX}; pool size: ${#PAT_NUMBERS[@]})" + echo "${selection_message} ${selection_details}" + + # Set the PAT number as the output + echo "copilot_pat_number=${PAT_NUMBER}" >> "$GITHUB_OUTPUT" diff --git a/.github/agents/agentic-workflows.agent.md b/.github/agents/agentic-workflows.agent.md index 4d5ecfe8e..3fa7b27a8 100644 --- a/.github/agents/agentic-workflows.agent.md +++ b/.github/agents/agentic-workflows.agent.md @@ -13,7 +13,7 @@ This is a **dispatcher agent** that routes your request to the appropriate speci - **Creating new workflows**: Routes to `create` prompt - **Updating existing workflows**: Routes to `update` prompt -- **Debugging workflows**: Routes to `debug` prompt +- **Debugging workflows**: Routes to `debug` prompt - **Upgrading workflows**: Routes to `upgrade-agentic-workflows` prompt - **Creating report-generating workflows**: Routes to `report` prompt — consult this whenever the workflow posts status updates, audits, analyses, or any structured output as issues, discussions, or comments - **Creating shared components**: Routes to `create-shared-agentic-workflow` prompt @@ -176,3 +176,84 @@ gh aw compile --validate - Follow security best practices: minimal permissions, explicit network access, no template injection - **Network configuration**: Use ecosystem identifiers (`node`, `python`, `go`, etc.) or explicit FQDNs in `network.allowed`. Bare shorthands like `npm` or `pypi` are **not** valid. See https://github.com/github/gh-aw/blob/v0.66.1/.github/aw/network.md for the full list of valid ecosystem identifiers and domain patterns. - **Single-file output**: When creating a workflow, produce exactly **one** workflow `.md` file. Do not create separate documentation files (architecture docs, runbooks, usage guides, etc.). If documentation is needed, add a brief `## Usage` section inside the workflow file itself. + +## Repository-Specific Requirements: Copilot PAT Pool + +**⚠️ MANDATORY**: Every agentic workflow in this repository **must** include the `select-copilot-pat` integration. This selects a random Copilot PAT from a numbered pool of secrets to avoid rate-limiting from a single shared PAT. This is a stop-gap workaround until organization-level billing is offered for agentic workflows. + +These elements are **not supported through imports**, so they must be copied into the frontmatter of every agentic workflow. See `.github/actions/select-copilot-pat/README.md` for full documentation and `.github/workflows/copilot-echo.md` for a working example. + +When **creating** or **updating** an agentic workflow, always include the following three frontmatter blocks. These go between the `---` markers alongside the workflow's other frontmatter (permissions, safe-outputs, etc.). + +### Comment block (placed before the steps, inside the `on:` block) + +Always include this comment block before the steps to explain the PAT pool override: + +```yaml +# ############################################################### +# Override the COPILOT_GITHUB_TOKEN secret usage for the workflow +# with a randomly-selected token from a pool of secrets. +# +# As soon as organization-level billing is offered for Agentic +# Workflows, this stop-gap approach will be removed. +# +# See: /.github/actions/select-copilot-pat/README.md +# ############################################################### +``` + +### Block 1: Pre-activation steps (nested under `on:`) + +Add this under the `on:` key, at the same level as the trigger configuration, immediately after the comment block above: + +```yaml + # Add the pre-activation step of selecting a random PAT from the supplied secrets + steps: + - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 + name: Checkout the select-copilot-pat action folder + with: + persist-credentials: false + sparse-checkout: .github/actions/select-copilot-pat + sparse-checkout-cone-mode: true + fetch-depth: 1 + + - id: select-copilot-pat + name: Select Copilot token from pool + uses: ./.github/actions/select-copilot-pat + env: + SECRET_0: ${{ secrets.COPILOT_PAT_0 }} + SECRET_1: ${{ secrets.COPILOT_PAT_1 }} + SECRET_2: ${{ secrets.COPILOT_PAT_2 }} + SECRET_3: ${{ secrets.COPILOT_PAT_3 }} + SECRET_4: ${{ secrets.COPILOT_PAT_4 }} + SECRET_5: ${{ secrets.COPILOT_PAT_5 }} + SECRET_6: ${{ secrets.COPILOT_PAT_6 }} + SECRET_7: ${{ secrets.COPILOT_PAT_7 }} + SECRET_8: ${{ secrets.COPILOT_PAT_8 }} + SECRET_9: ${{ secrets.COPILOT_PAT_9 }} +``` + +### Block 2: Pre-activation job outputs (top-level `jobs:`) + +```yaml +jobs: + pre-activation: + outputs: + copilot_pat_number: ${{ steps.select-copilot-pat.outputs.copilot_pat_number }} +``` + +### Block 3: Engine configuration with PAT override (top-level `engine:`) + +```yaml +engine: + id: copilot + env: + # We cannot use line breaks in this expression as it leads to a syntax error in the compiled workflow + # If none of the `COPILOT_PAT_#` secrets were selected, then the default COPILOT_GITHUB_TOKEN is used + COPILOT_GITHUB_TOKEN: ${{ case(needs.pre_activation.outputs.copilot_pat_number == '0', secrets.COPILOT_PAT_0, needs.pre_activation.outputs.copilot_pat_number == '1', secrets.COPILOT_PAT_1, needs.pre_activation.outputs.copilot_pat_number == '2', secrets.COPILOT_PAT_2, needs.pre_activation.outputs.copilot_pat_number == '3', secrets.COPILOT_PAT_3, needs.pre_activation.outputs.copilot_pat_number == '4', secrets.COPILOT_PAT_4, needs.pre_activation.outputs.copilot_pat_number == '5', secrets.COPILOT_PAT_5, needs.pre_activation.outputs.copilot_pat_number == '6', secrets.COPILOT_PAT_6, needs.pre_activation.outputs.copilot_pat_number == '7', secrets.COPILOT_PAT_7, needs.pre_activation.outputs.copilot_pat_number == '8', secrets.COPILOT_PAT_8, needs.pre_activation.outputs.copilot_pat_number == '9', secrets.COPILOT_PAT_9, secrets.COPILOT_GITHUB_TOKEN) }} +``` + +**Important notes about the engine block:** + +- The `COPILOT_GITHUB_TOKEN` `case()` expression **must** remain on a single line — line breaks cause syntax errors in the compiled workflow. +- If no `COPILOT_PAT_#` secrets are configured, the expression falls back to the default `COPILOT_GITHUB_TOKEN` secret. +- Do **not** specify `engine: copilot` as a simple string — use the object form shown above so the `env:` override can be included. From 8adae1ceb25d05c5374b3c9227f6f09a5b13e96d Mon Sep 17 00:00:00 2001 From: Jeff Handley Date: Sat, 4 Apr 2026 17:14:34 -0700 Subject: [PATCH 4/7] Add SDK Tier Audit agentic workflow and skill Add a weekly agentic workflow that performs SEP-1730 tier audits of the C# MCP SDK using the mcp-sdk-tier-audit skill from the conformance repository. The workflow: - Runs weekly on Thursdays (fuzzy ~6:30am EST) and on manual dispatch - Supports two scopes: Conformance + Repo Health (default) and Repo Health - Allows overriding the C# SDK and conformance repo/branch targets - Files issues with tier results, auto-closing previous audit issues - Uses a dedicated AUDIT_PAT pool for Copilot engine authentication - Skips scheduled runs on forks; allows manual dispatch anywhere The audit logic is extracted into a reusable skill at .github/skills/sdk-tier-audit/SKILL.md with cross-platform instructions (macOS/Linux and Windows) so it can also be invoked locally. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/aw/actions-lock.json | 34 + .github/skills/sdk-tier-audit/SKILL.md | 204 ++++ .github/workflows/sdk-tier-audit.lock.yml | 1342 +++++++++++++++++++++ .github/workflows/sdk-tier-audit.md | 443 +++++++ 4 files changed, 2023 insertions(+) create mode 100644 .github/aw/actions-lock.json create mode 100644 .github/skills/sdk-tier-audit/SKILL.md create mode 100644 .github/workflows/sdk-tier-audit.lock.yml create mode 100644 .github/workflows/sdk-tier-audit.md diff --git a/.github/aw/actions-lock.json b/.github/aw/actions-lock.json new file mode 100644 index 000000000..138b2808a --- /dev/null +++ b/.github/aw/actions-lock.json @@ -0,0 +1,34 @@ +{ + "entries": { + "actions/download-artifact@v8.0.1": { + "repo": "actions/download-artifact", + "version": "v8.0.1", + "sha": "3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c" + }, + "actions/github-script@v8": { + "repo": "actions/github-script", + "version": "v8", + "sha": "ed597411d8f924073f98dfc5c65a23a2325f34cd" + }, + "actions/setup-dotnet@v5.2.0": { + "repo": "actions/setup-dotnet", + "version": "v5.2.0", + "sha": "c2fa09f4bde5ebb9d1777cf28262a3eb3db3ced7" + }, + "actions/setup-node@v6.3.0": { + "repo": "actions/setup-node", + "version": "v6.3.0", + "sha": "53b83947a5a98c8d113130e565377fae1a50d02f" + }, + "actions/upload-artifact@v7": { + "repo": "actions/upload-artifact", + "version": "v7", + "sha": "bbbca2ddaa5d8feaa63e36b76fdaad77386f024f" + }, + "github/gh-aw-actions/setup@v0.66.1": { + "repo": "github/gh-aw-actions/setup", + "version": "v0.66.1", + "sha": "73ae9ce231580f337133352d321d42b6bf54b6a9" + } + } +} diff --git a/.github/skills/sdk-tier-audit/SKILL.md b/.github/skills/sdk-tier-audit/SKILL.md new file mode 100644 index 000000000..07c84b4fc --- /dev/null +++ b/.github/skills/sdk-tier-audit/SKILL.md @@ -0,0 +1,204 @@ +--- +name: sdk-tier-audit +description: >- + Perform a tier audit of the C# MCP SDK against SEP-1730 (the SDK Tiering System). + Clones the conformance and C# SDK repositories, builds and runs conformance tests, + runs the tier-check CLI, evaluates documentation and policies, and produces + assessment and remediation reports. Delegates all audit logic to the + mcp-sdk-tier-audit skill in the conformance repository. +compatibility: Requires git, .NET SDK 10.0+, Node.js 20+, and npm. Needs read access to the C# SDK and conformance GitHub repositories. +argument-hint: '[scope] [--csharp-sdk-repo ] [--csharp-sdk-branch ] [--conformance-repo ] [--conformance-branch ]' +--- + +# SDK Tier Audit + +Perform a tier audit of the C# MCP SDK against [SEP-1730](https://github.com/modelcontextprotocol/modelcontextprotocol/issues/1730) (the SDK Tiering System). This skill delegates all audit logic to the `mcp-sdk-tier-audit` skill from the conformance repository — it handles setup, then follows the conformance skill's instructions. + +If any step fails, stop and report the error to the user. Do not proceed to the next step. + +## Arguments + +Parse optional arguments from the user's input: + +- **scope** — `Conformance + Repo Health` (default) or `Repo Health` +- **--csharp-sdk-repo** — C# SDK repository as `owner/repo` (default: `modelcontextprotocol/csharp-sdk`) +- **--csharp-sdk-branch** — C# SDK branch (default: `main`) +- **--conformance-repo** — Conformance repository as `owner/repo` (default: `modelcontextprotocol/conformance`) +- **--conformance-branch** — Conformance repo branch (default: `main`) + +If the user provides just a scope keyword (e.g., `/sdk-tier-audit Repo Health`), use that as the scope. All other arguments use the defaults if not specified. + +## Prerequisites + +The following tools must be available: + +- **git** — to clone repositories +- **.NET SDK** (10.0+) — to build and run the C# SDK conformance server/client +- **Node.js** (20+) and **npm** — to build and run the conformance CLI + +## Step 1: Clone and Build + +### Clone the C# SDK + +**macOS / Linux:** + +```bash +git clone --depth 1 -b https://github.com/.git /tmp/csharp-sdk +``` + +**Windows (PowerShell):** + +```powershell +git clone --depth 1 -b https://github.com/.git $env:TEMP\csharp-sdk +``` + +### Clone the conformance repository + +**macOS / Linux:** + +```bash +git clone --depth 1 -b https://github.com/.git /tmp/conformance +``` + +**Windows (PowerShell):** + +```powershell +git clone --depth 1 -b https://github.com/.git $env:TEMP\conformance +``` + +### Build the conformance CLI + +**macOS / Linux:** + +```bash +cd /tmp/conformance && npm ci && npm run build +``` + +**Windows (PowerShell):** + +```powershell +cd $env:TEMP\conformance +npm ci +if ($LASTEXITCODE -ne 0) { throw "npm ci failed" } +npm run build +if ($LASTEXITCODE -ne 0) { throw "npm run build failed" } +``` + +Use `/tmp` paths on macOS/Linux and `$env:TEMP` paths on Windows throughout the remaining steps. + +## Step 2: Start Conformance Server (if scope includes conformance) + +Skip this step if the scope is "Repo Health". + +### Build the C# SDK + +```bash +cd /tmp/csharp-sdk && dotnet build +``` + +### Start the conformance server + +The server must remain running throughout the audit. Use `nohup` (macOS/Linux) or `Start-Process` (Windows) to prevent the process from dying when the shell session changes. + +**macOS / Linux:** + +```bash +cd /tmp/csharp-sdk +nohup dotnet run --no-build --project tests/ModelContextProtocol.ConformanceServer --framework net9.0 -- --urls http://localhost:3003 > /tmp/conformance-server.log 2>&1 & +# Wait for the server to be ready (macOS lacks `timeout`, so use a loop) +for i in $(seq 1 60); do + curl -sf http://localhost:3003/health > /dev/null 2>&1 && break + sleep 1 +done +curl -sf http://localhost:3003/health > /dev/null || { echo "Server failed to start — check /tmp/conformance-server.log"; exit 1; } +echo "Conformance server ready" +``` + +**Windows (PowerShell):** + +```powershell +cd $env:TEMP\csharp-sdk +Start-Process -NoNewWindow dotnet -ArgumentList "run","--project","tests/ModelContextProtocol.ConformanceServer","--framework","net9.0","--","--urls","http://localhost:3003" +# Wait for the server to be ready +$timeout = 60; $elapsed = 0 +while ($elapsed -lt $timeout) { + try { Invoke-WebRequest -Uri http://localhost:3003/health -UseBasicParsing -ErrorAction Stop | Out-Null; break } + catch { Start-Sleep 1; $elapsed++ } +} +if ($elapsed -ge $timeout) { throw "Conformance server did not start within $timeout seconds" } +``` + +## Step 3: Run the Audit + +Read the **"Any Other AI Coding Agent"** section from the conformance skill's README: + +- macOS/Linux: `/tmp/conformance/.claude/skills/mcp-sdk-tier-audit/README.md` +- Windows: `$env:TEMP\conformance\.claude\skills\mcp-sdk-tier-audit\README.md` + +Follow those instructions exactly, using the reference files in the `references/` directory alongside it. + +The instructions describe a 5-step process: + +1. **Run the tier-check CLI** to get the deterministic scorecard +2. **Evaluate documentation coverage** using the prompt in `references/docs-coverage-prompt.md` +3. **Evaluate policies** using the prompt in `references/policy-evaluation-prompt.md` +4. **Apply tier logic** using the thresholds in `references/tier-requirements.md` +5. **Generate report** using the template in `references/report-template.md` + +### CLI parameters + +Derive the `owner/repo` from the C# SDK clone's git remote: + +```bash +cd /tmp/csharp-sdk && git remote get-url origin | sed 's#.*github.com[:/]##; s#\.git$##' +``` + +Derive the branch from the local checkout: + +```bash +cd /tmp/csharp-sdk && git rev-parse --abbrev-ref HEAD +``` + +**If scope is "Conformance + Repo Health"**, run with both server and client conformance: + +```bash +cd /tmp/conformance +node dist/index.js tier-check \ + --repo \ + --branch \ + --conformance-server-url http://localhost:3003 \ + --client-cmd 'dotnet run --no-build --project /tmp/csharp-sdk/tests/ModelContextProtocol.ConformanceClient --framework net9.0 -- $MCP_CONFORMANCE_SCENARIO' \ + --output json +``` + +**If scope is "Repo Health"**, run without conformance: + +```bash +cd /tmp/conformance +node dist/index.js tier-check \ + --repo \ + --branch \ + --skip-conformance \ + --output json +``` + +### Documentation and policy evaluation + +After running the CLI, perform the documentation coverage and policy evaluations by reading and following the prompts in the reference files. The SDK checkout at `/tmp/csharp-sdk` (or `$env:TEMP\csharp-sdk` on Windows) is the local path for these evaluations. + +### Report generation + +Write the assessment and remediation reports to the conformance repo's `results/` directory following the template in `references/report-template.md`: + +- `results/-csharp-sdk-assessment.md` +- `results/-csharp-sdk-remediation.md` + +## Step 4: Present Results + +After the audit completes, present the user with: + +1. **Executive summary** — The tier classification and primary reasons (3-5 sentences) +2. **Report file locations** — Paths to the assessment and remediation files +3. **Key gaps** — Top items needed for tier advancement + +If the audit failed at any step, explain what happened and which step failed. diff --git a/.github/workflows/sdk-tier-audit.lock.yml b/.github/workflows/sdk-tier-audit.lock.yml new file mode 100644 index 000000000..37976ecd1 --- /dev/null +++ b/.github/workflows/sdk-tier-audit.lock.yml @@ -0,0 +1,1342 @@ +# ___ _ _ +# / _ \ | | (_) +# | |_| | __ _ ___ _ __ | |_ _ ___ +# | _ |/ _` |/ _ \ '_ \| __| |/ __| +# | | | | (_| | __/ | | | |_| | (__ +# \_| |_/\__, |\___|_| |_|\__|_|\___| +# __/ | +# _ _ |___/ +# | | | | / _| | +# | | | | ___ _ __ _ __| |_| | _____ ____ +# | |/\| |/ _ \ '__| |/ /| _| |/ _ \ \ /\ / / ___| +# \ /\ / (_) | | | | ( | | | | (_) \ V V /\__ \ +# \/ \/ \___/|_| |_|\_\|_| |_|\___/ \_/\_/ |___/ +# +# This file was automatically generated by gh-aw (v0.66.1). DO NOT EDIT. +# +# To update this file, edit the corresponding .md file and run: +# gh aw compile +# Not all edits will cause changes to this file. +# +# For more information: https://github.github.com/gh-aw/introduction/overview/ +# +# SDK Tier Audit +# +# gh-aw-metadata: {"schema_version":"v3","frontmatter_hash":"70bf24be23625c84fbdbec90dff9f6066a02da072946c9525fedc95b49a7de92","compiler_version":"v0.66.1","strict":true,"agent_id":"copilot"} + +name: "SDK Tier Audit" +"on": + schedule: + - cron: "10 11 * * 4" + # Friendly format: weekly on thursday around 6:30am utc-5 (scattered) + # steps: # Steps injected into pre-activation job + # - name: Checkout the select-copilot-pat action folder + # uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd + # with: + # fetch-depth: 1 + # persist-credentials: false + # sparse-checkout: .github/actions/select-copilot-pat + # sparse-checkout-cone-mode: true + # - env: + # SECRET_0: ${{ secrets.AUDIT_PAT_0 }} + # SECRET_1: ${{ secrets.AUDIT_PAT_1 }} + # SECRET_2: ${{ secrets.AUDIT_PAT_2 }} + # SECRET_3: ${{ secrets.AUDIT_PAT_3 }} + # SECRET_4: ${{ secrets.AUDIT_PAT_4 }} + # SECRET_5: ${{ secrets.AUDIT_PAT_5 }} + # SECRET_6: ${{ secrets.AUDIT_PAT_6 }} + # SECRET_7: ${{ secrets.AUDIT_PAT_7 }} + # SECRET_8: ${{ secrets.AUDIT_PAT_8 }} + # SECRET_9: ${{ secrets.AUDIT_PAT_9 }} + # id: select-copilot-pat + # name: Select Copilot token from pool + # uses: ./.github/actions/select-copilot-pat + workflow_dispatch: + inputs: + aw_context: + default: "" + description: Agent caller context (used internally by Agentic Workflows). + required: false + type: string + conformance: + default: modelcontextprotocol/conformance:main + description: Conformance repo (owner/repo:branch) + required: true + type: string + csharp_sdk: + default: modelcontextprotocol/csharp-sdk:main + description: "C# SDK (owner/repo:branch)" + required: true + type: string + output: + default: Create Issue + description: Where to publish results + options: + - Create Issue + - Action Summary + required: true + type: choice + scope: + default: Conformance + Repo Health + description: Audit scope + options: + - Conformance + Repo Health + - Repo Health + required: true + type: choice + +permissions: {} + +concurrency: + cancel-in-progress: true + group: tier-audit-${{ github.event.inputs.scope || 'Conformance + Repo Health' }} + +run-name: "SDK Tier Audit" + +env: + FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true" + +jobs: + activation: + needs: pre_activation + if: > + needs.pre_activation.outputs.activated == 'true' && (github.repository_owner == 'modelcontextprotocol' || + github.event_name == 'workflow_dispatch') + runs-on: ubuntu-slim + permissions: + contents: read + outputs: + comment_id: "" + comment_repo: "" + lockdown_check_failed: ${{ steps.generate_aw_info.outputs.lockdown_check_failed == 'true' }} + model: ${{ steps.generate_aw_info.outputs.model }} + secret_verification_result: ${{ steps.validate-secret.outputs.verification_result }} + steps: + - name: Setup Scripts + uses: github/gh-aw-actions/setup@73ae9ce231580f337133352d321d42b6bf54b6a9 # v0.66.1 + with: + destination: ${{ runner.temp }}/gh-aw/actions + - name: Generate agentic run info + id: generate_aw_info + env: + GH_AW_INFO_ENGINE_ID: "copilot" + GH_AW_INFO_ENGINE_NAME: "GitHub Copilot CLI" + GH_AW_INFO_MODEL: ${{ vars.GH_AW_MODEL_AGENT_COPILOT || 'auto' }} + GH_AW_INFO_VERSION: "latest" + GH_AW_INFO_AGENT_VERSION: "latest" + GH_AW_INFO_CLI_VERSION: "v0.66.1" + GH_AW_INFO_WORKFLOW_NAME: "SDK Tier Audit" + GH_AW_INFO_EXPERIMENTAL: "false" + GH_AW_INFO_SUPPORTS_TOOLS_ALLOWLIST: "true" + GH_AW_INFO_STAGED: "false" + GH_AW_INFO_ALLOWED_DOMAINS: '["defaults","node","dotnet","github"]' + GH_AW_INFO_FIREWALL_ENABLED: "true" + GH_AW_INFO_AWF_VERSION: "v0.25.13" + GH_AW_INFO_AWMG_VERSION: "" + GH_AW_INFO_FIREWALL_TYPE: "squid" + GH_AW_COMPILED_STRICT: "true" + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + with: + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/generate_aw_info.cjs'); + await main(core, context); + - name: Validate COPILOT_GITHUB_TOKEN secret + id: validate-secret + run: ${RUNNER_TEMP}/gh-aw/actions/validate_multi_secret.sh COPILOT_GITHUB_TOKEN 'GitHub Copilot CLI' https://github.github.com/gh-aw/reference/engines/#github-copilot-default + env: + COPILOT_GITHUB_TOKEN: ${{ case(needs.pre_activation.outputs.copilot_pat_number == '0', secrets.AUDIT_PAT_0, needs.pre_activation.outputs.copilot_pat_number == '1', secrets.AUDIT_PAT_1, needs.pre_activation.outputs.copilot_pat_number == '2', secrets.AUDIT_PAT_2, needs.pre_activation.outputs.copilot_pat_number == '3', secrets.AUDIT_PAT_3, needs.pre_activation.outputs.copilot_pat_number == '4', secrets.AUDIT_PAT_4, needs.pre_activation.outputs.copilot_pat_number == '5', secrets.AUDIT_PAT_5, needs.pre_activation.outputs.copilot_pat_number == '6', secrets.AUDIT_PAT_6, needs.pre_activation.outputs.copilot_pat_number == '7', secrets.AUDIT_PAT_7, needs.pre_activation.outputs.copilot_pat_number == '8', secrets.AUDIT_PAT_8, needs.pre_activation.outputs.copilot_pat_number == '9', secrets.AUDIT_PAT_9, secrets.COPILOT_GITHUB_TOKEN) }} + - name: Checkout .github and .agents folders + uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 + with: + persist-credentials: false + sparse-checkout: | + .github + .agents + sparse-checkout-cone-mode: true + fetch-depth: 1 + - name: Check workflow lock file + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_WORKFLOW_FILE: "sdk-tier-audit.lock.yml" + GH_AW_CONTEXT_WORKFLOW_REF: "${{ github.workflow_ref }}" + with: + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/check_workflow_timestamp_api.cjs'); + await main(); + - name: Check compile-agentic version + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_COMPILED_VERSION: "v0.66.1" + with: + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/check_version_updates.cjs'); + await main(); + - name: Create prompt with built-in context + env: + GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt + GH_AW_SAFE_OUTPUTS: ${{ runner.temp }}/gh-aw/safeoutputs/outputs.jsonl + GH_AW_GITHUB_ACTOR: ${{ github.actor }} + GH_AW_GITHUB_EVENT_COMMENT_ID: ${{ github.event.comment.id }} + GH_AW_GITHUB_EVENT_DISCUSSION_NUMBER: ${{ github.event.discussion.number }} + GH_AW_GITHUB_EVENT_ISSUE_NUMBER: ${{ github.event.issue.number }} + GH_AW_GITHUB_EVENT_PULL_REQUEST_NUMBER: ${{ github.event.pull_request.number }} + GH_AW_GITHUB_REPOSITORY: ${{ github.repository }} + GH_AW_GITHUB_RUN_ID: ${{ github.run_id }} + GH_AW_GITHUB_WORKSPACE: ${{ github.workspace }} + # poutine:ignore untrusted_checkout_exec + run: | + bash ${RUNNER_TEMP}/gh-aw/actions/create_prompt_first.sh + { + cat << 'GH_AW_PROMPT_cffe1da634462c6b_EOF' + + GH_AW_PROMPT_cffe1da634462c6b_EOF + cat "${RUNNER_TEMP}/gh-aw/prompts/xpia.md" + cat "${RUNNER_TEMP}/gh-aw/prompts/temp_folder_prompt.md" + cat "${RUNNER_TEMP}/gh-aw/prompts/markdown.md" + cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_prompt.md" + cat << 'GH_AW_PROMPT_cffe1da634462c6b_EOF' + + Tools: create_issue, missing_tool, missing_data, noop + + + The following GitHub context information is available for this workflow: + {{#if __GH_AW_GITHUB_ACTOR__ }} + - **actor**: __GH_AW_GITHUB_ACTOR__ + {{/if}} + {{#if __GH_AW_GITHUB_REPOSITORY__ }} + - **repository**: __GH_AW_GITHUB_REPOSITORY__ + {{/if}} + {{#if __GH_AW_GITHUB_WORKSPACE__ }} + - **workspace**: __GH_AW_GITHUB_WORKSPACE__ + {{/if}} + {{#if __GH_AW_GITHUB_EVENT_ISSUE_NUMBER__ }} + - **issue-number**: #__GH_AW_GITHUB_EVENT_ISSUE_NUMBER__ + {{/if}} + {{#if __GH_AW_GITHUB_EVENT_DISCUSSION_NUMBER__ }} + - **discussion-number**: #__GH_AW_GITHUB_EVENT_DISCUSSION_NUMBER__ + {{/if}} + {{#if __GH_AW_GITHUB_EVENT_PULL_REQUEST_NUMBER__ }} + - **pull-request-number**: #__GH_AW_GITHUB_EVENT_PULL_REQUEST_NUMBER__ + {{/if}} + {{#if __GH_AW_GITHUB_EVENT_COMMENT_ID__ }} + - **comment-id**: __GH_AW_GITHUB_EVENT_COMMENT_ID__ + {{/if}} + {{#if __GH_AW_GITHUB_RUN_ID__ }} + - **workflow-run-id**: __GH_AW_GITHUB_RUN_ID__ + {{/if}} + + + GH_AW_PROMPT_cffe1da634462c6b_EOF + cat "${RUNNER_TEMP}/gh-aw/prompts/github_mcp_tools_with_safeoutputs_prompt.md" + cat << 'GH_AW_PROMPT_cffe1da634462c6b_EOF' + + {{#runtime-import .github/workflows/sdk-tier-audit.md}} + GH_AW_PROMPT_cffe1da634462c6b_EOF + } > "$GH_AW_PROMPT" + - name: Interpolate variables and render templates + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt + with: + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/interpolate_prompt.cjs'); + await main(); + - name: Substitute placeholders + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt + GH_AW_GITHUB_ACTOR: ${{ github.actor }} + GH_AW_GITHUB_EVENT_COMMENT_ID: ${{ github.event.comment.id }} + GH_AW_GITHUB_EVENT_DISCUSSION_NUMBER: ${{ github.event.discussion.number }} + GH_AW_GITHUB_EVENT_ISSUE_NUMBER: ${{ github.event.issue.number }} + GH_AW_GITHUB_EVENT_PULL_REQUEST_NUMBER: ${{ github.event.pull_request.number }} + GH_AW_GITHUB_REPOSITORY: ${{ github.repository }} + GH_AW_GITHUB_RUN_ID: ${{ github.run_id }} + GH_AW_GITHUB_WORKSPACE: ${{ github.workspace }} + GH_AW_NEEDS_PRE_ACTIVATION_OUTPUTS_ACTIVATED: ${{ needs.pre_activation.outputs.activated }} + with: + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + + const substitutePlaceholders = require('${{ runner.temp }}/gh-aw/actions/substitute_placeholders.cjs'); + + // Call the substitution function + return await substitutePlaceholders({ + file: process.env.GH_AW_PROMPT, + substitutions: { + GH_AW_GITHUB_ACTOR: process.env.GH_AW_GITHUB_ACTOR, + GH_AW_GITHUB_EVENT_COMMENT_ID: process.env.GH_AW_GITHUB_EVENT_COMMENT_ID, + GH_AW_GITHUB_EVENT_DISCUSSION_NUMBER: process.env.GH_AW_GITHUB_EVENT_DISCUSSION_NUMBER, + GH_AW_GITHUB_EVENT_ISSUE_NUMBER: process.env.GH_AW_GITHUB_EVENT_ISSUE_NUMBER, + GH_AW_GITHUB_EVENT_PULL_REQUEST_NUMBER: process.env.GH_AW_GITHUB_EVENT_PULL_REQUEST_NUMBER, + GH_AW_GITHUB_REPOSITORY: process.env.GH_AW_GITHUB_REPOSITORY, + GH_AW_GITHUB_RUN_ID: process.env.GH_AW_GITHUB_RUN_ID, + GH_AW_GITHUB_WORKSPACE: process.env.GH_AW_GITHUB_WORKSPACE, + GH_AW_NEEDS_PRE_ACTIVATION_OUTPUTS_ACTIVATED: process.env.GH_AW_NEEDS_PRE_ACTIVATION_OUTPUTS_ACTIVATED + } + }); + - name: Validate prompt placeholders + env: + GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt + # poutine:ignore untrusted_checkout_exec + run: bash ${RUNNER_TEMP}/gh-aw/actions/validate_prompt_placeholders.sh + - name: Print prompt + env: + GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt + # poutine:ignore untrusted_checkout_exec + run: bash ${RUNNER_TEMP}/gh-aw/actions/print_prompt_summary.sh + - name: Upload activation artifact + if: success() + uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7 + with: + name: activation + path: | + /tmp/gh-aw/aw_info.json + /tmp/gh-aw/aw-prompts/prompt.txt + retention-days: 1 + + agent: + needs: + - activation + - tier-check + runs-on: ubuntu-latest + permissions: + contents: read + issues: read + pull-requests: read + concurrency: + group: "gh-aw-copilot-${{ github.workflow }}" + env: + DEFAULT_BRANCH: ${{ github.event.repository.default_branch }} + GH_AW_ASSETS_ALLOWED_EXTS: "" + GH_AW_ASSETS_BRANCH: "" + GH_AW_ASSETS_MAX_SIZE_KB: 0 + GH_AW_MCP_LOG_DIR: /tmp/gh-aw/mcp-logs/safeoutputs + GH_AW_WORKFLOW_ID_SANITIZED: sdktieraudit + outputs: + checkout_pr_success: ${{ steps.checkout-pr.outputs.checkout_pr_success || 'true' }} + effective_tokens: ${{ steps.parse-mcp-gateway.outputs.effective_tokens }} + has_patch: ${{ steps.collect_output.outputs.has_patch }} + inference_access_error: ${{ steps.detect-inference-error.outputs.inference_access_error || 'false' }} + model: ${{ needs.activation.outputs.model }} + output: ${{ steps.collect_output.outputs.output }} + output_types: ${{ steps.collect_output.outputs.output_types }} + steps: + - name: Setup Scripts + uses: github/gh-aw-actions/setup@73ae9ce231580f337133352d321d42b6bf54b6a9 # v0.66.1 + with: + destination: ${{ runner.temp }}/gh-aw/actions + - name: Set runtime paths + id: set-runtime-paths + run: | + echo "GH_AW_SAFE_OUTPUTS=${RUNNER_TEMP}/gh-aw/safeoutputs/outputs.jsonl" >> "$GITHUB_OUTPUT" + echo "GH_AW_SAFE_OUTPUTS_CONFIG_PATH=${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" >> "$GITHUB_OUTPUT" + echo "GH_AW_SAFE_OUTPUTS_TOOLS_PATH=${RUNNER_TEMP}/gh-aw/safeoutputs/tools.json" >> "$GITHUB_OUTPUT" + - name: Checkout repository + uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 + with: + persist-credentials: false + - name: Setup .NET + uses: actions/setup-dotnet@c2fa09f4bde5ebb9d1777cf28262a3eb3db3ced7 # v5.2.0 + with: + dotnet-version: '10.0' + - name: Setup Node.js + uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0 + with: + node-version: '22' + package-manager-cache: false + - name: Create gh-aw temp directory + run: bash ${RUNNER_TEMP}/gh-aw/actions/create_gh_aw_tmp_dir.sh + - name: Configure gh CLI for GitHub Enterprise + run: bash ${RUNNER_TEMP}/gh-aw/actions/configure_gh_for_ghe.sh + env: + GH_TOKEN: ${{ github.token }} + - env: + AUDIT_CONF_BRANCH: ${{ needs.tier-check.outputs.conformance_branch }} + AUDIT_CONF_REPO: ${{ needs.tier-check.outputs.conformance_repo }} + AUDIT_OUTPUT: ${{ needs.tier-check.outputs.output }} + AUDIT_SCOPE: ${{ needs.tier-check.outputs.scope }} + AUDIT_SDK_BRANCH: ${{ needs.tier-check.outputs.csharp_sdk_branch }} + AUDIT_SDK_REPO: ${{ needs.tier-check.outputs.csharp_sdk_repo }} + TIER_CHECK_JSON: ${{ needs.tier-check.outputs.tier_check_json }} + name: Write tier-check outputs to files + run: |- + echo "$TIER_CHECK_JSON" > /tmp/tier-check-scorecard.json + mkdir -p /tmp/audit-params + echo "$AUDIT_SCOPE" > /tmp/audit-params/scope + echo "$AUDIT_OUTPUT" > /tmp/audit-params/output + echo "$AUDIT_SDK_REPO" > /tmp/audit-params/csharp-sdk-repo + echo "$AUDIT_SDK_BRANCH" > /tmp/audit-params/csharp-sdk-branch + echo "$AUDIT_CONF_REPO" > /tmp/audit-params/conformance-repo + echo "$AUDIT_CONF_BRANCH" > /tmp/audit-params/conformance-branch + + - name: Configure Git credentials + env: + REPO_NAME: ${{ github.repository }} + SERVER_URL: ${{ github.server_url }} + run: | + git config --global user.email "github-actions[bot]@users.noreply.github.com" + git config --global user.name "github-actions[bot]" + git config --global am.keepcr true + # Re-authenticate git with GitHub token + SERVER_URL_STRIPPED="${SERVER_URL#https://}" + git remote set-url origin "https://x-access-token:${{ github.token }}@${SERVER_URL_STRIPPED}/${REPO_NAME}.git" + echo "Git configured with standard GitHub Actions identity" + - name: Checkout PR branch + id: checkout-pr + if: | + github.event.pull_request || github.event.issue.pull_request + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_TOKEN: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN || secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + with: + github-token: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN || secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/checkout_pr_branch.cjs'); + await main(); + - name: Install GitHub Copilot CLI + run: ${RUNNER_TEMP}/gh-aw/actions/install_copilot_cli.sh latest + env: + GH_HOST: github.com + - name: Install AWF binary + run: bash ${RUNNER_TEMP}/gh-aw/actions/install_awf_binary.sh v0.25.13 + - name: Parse integrity filter lists + id: parse-guard-vars + env: + GH_AW_BLOCKED_USERS_VAR: ${{ vars.GH_AW_GITHUB_BLOCKED_USERS || '' }} + GH_AW_TRUSTED_USERS_VAR: ${{ vars.GH_AW_GITHUB_TRUSTED_USERS || '' }} + GH_AW_APPROVAL_LABELS_VAR: ${{ vars.GH_AW_GITHUB_APPROVAL_LABELS || '' }} + run: bash ${RUNNER_TEMP}/gh-aw/actions/parse_guard_list.sh + - name: Download container images + run: bash ${RUNNER_TEMP}/gh-aw/actions/download_docker_images.sh ghcr.io/github/gh-aw-firewall/agent:0.25.13 ghcr.io/github/gh-aw-firewall/api-proxy:0.25.13 ghcr.io/github/gh-aw-firewall/squid:0.25.13 ghcr.io/github/gh-aw-mcpg:v0.2.12 ghcr.io/github/github-mcp-server:v0.32.0 node:lts-alpine + - name: Write Safe Outputs Config + run: | + mkdir -p ${RUNNER_TEMP}/gh-aw/safeoutputs + mkdir -p /tmp/gh-aw/safeoutputs + mkdir -p /tmp/gh-aw/mcp-logs/safeoutputs + cat > ${RUNNER_TEMP}/gh-aw/safeoutputs/config.json << 'GH_AW_SAFE_OUTPUTS_CONFIG_c6fd50f1e250c752_EOF' + {"create_issue":{"close_older_issues":true,"labels":["automation"],"max":1,"title_prefix":"[C# SDK Tier Audit] "},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"}} + GH_AW_SAFE_OUTPUTS_CONFIG_c6fd50f1e250c752_EOF + - name: Write Safe Outputs Tools + run: | + cat > ${RUNNER_TEMP}/gh-aw/safeoutputs/tools_meta.json << 'GH_AW_SAFE_OUTPUTS_TOOLS_META_5ec9df3bfed4f693_EOF' + { + "description_suffixes": { + "create_issue": " CONSTRAINTS: Maximum 1 issue(s) can be created. Title will be prefixed with \"[C# SDK Tier Audit] \". Labels [\"automation\"] will be automatically added." + }, + "repo_params": {}, + "dynamic_tools": [] + } + GH_AW_SAFE_OUTPUTS_TOOLS_META_5ec9df3bfed4f693_EOF + cat > ${RUNNER_TEMP}/gh-aw/safeoutputs/validation.json << 'GH_AW_SAFE_OUTPUTS_VALIDATION_2ba4f188e5e9c93d_EOF' + { + "create_issue": { + "defaultMax": 1, + "fields": { + "body": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 65000 + }, + "labels": { + "type": "array", + "itemType": "string", + "itemSanitize": true, + "itemMaxLength": 128 + }, + "parent": { + "issueOrPRNumber": true + }, + "repo": { + "type": "string", + "maxLength": 256 + }, + "temporary_id": { + "type": "string" + }, + "title": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 128 + } + } + }, + "missing_data": { + "defaultMax": 20, + "fields": { + "alternatives": { + "type": "string", + "sanitize": true, + "maxLength": 256 + }, + "context": { + "type": "string", + "sanitize": true, + "maxLength": 256 + }, + "data_type": { + "type": "string", + "sanitize": true, + "maxLength": 128 + }, + "reason": { + "type": "string", + "sanitize": true, + "maxLength": 256 + } + } + }, + "missing_tool": { + "defaultMax": 20, + "fields": { + "alternatives": { + "type": "string", + "sanitize": true, + "maxLength": 512 + }, + "reason": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 256 + }, + "tool": { + "type": "string", + "sanitize": true, + "maxLength": 128 + } + } + }, + "noop": { + "defaultMax": 1, + "fields": { + "message": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 65000 + } + } + } + } + GH_AW_SAFE_OUTPUTS_VALIDATION_2ba4f188e5e9c93d_EOF + node ${RUNNER_TEMP}/gh-aw/actions/generate_safe_outputs_tools.cjs + - name: Generate Safe Outputs MCP Server Config + id: safe-outputs-config + run: | + # Generate a secure random API key (360 bits of entropy, 40+ chars) + # Mask immediately to prevent timing vulnerabilities + API_KEY=$(openssl rand -base64 45 | tr -d '/+=') + echo "::add-mask::${API_KEY}" + + PORT=3001 + + # Set outputs for next steps + { + echo "safe_outputs_api_key=${API_KEY}" + echo "safe_outputs_port=${PORT}" + } >> "$GITHUB_OUTPUT" + + echo "Safe Outputs MCP server will run on port ${PORT}" + + - name: Start Safe Outputs MCP HTTP Server + id: safe-outputs-start + env: + DEBUG: '*' + GH_AW_SAFE_OUTPUTS: ${{ steps.set-runtime-paths.outputs.GH_AW_SAFE_OUTPUTS }} + GH_AW_SAFE_OUTPUTS_PORT: ${{ steps.safe-outputs-config.outputs.safe_outputs_port }} + GH_AW_SAFE_OUTPUTS_API_KEY: ${{ steps.safe-outputs-config.outputs.safe_outputs_api_key }} + GH_AW_SAFE_OUTPUTS_TOOLS_PATH: ${{ runner.temp }}/gh-aw/safeoutputs/tools.json + GH_AW_SAFE_OUTPUTS_CONFIG_PATH: ${{ runner.temp }}/gh-aw/safeoutputs/config.json + GH_AW_MCP_LOG_DIR: /tmp/gh-aw/mcp-logs/safeoutputs + run: | + # Environment variables are set above to prevent template injection + export DEBUG + export GH_AW_SAFE_OUTPUTS + export GH_AW_SAFE_OUTPUTS_PORT + export GH_AW_SAFE_OUTPUTS_API_KEY + export GH_AW_SAFE_OUTPUTS_TOOLS_PATH + export GH_AW_SAFE_OUTPUTS_CONFIG_PATH + export GH_AW_MCP_LOG_DIR + + bash ${RUNNER_TEMP}/gh-aw/actions/start_safe_outputs_server.sh + + - name: Start MCP Gateway + id: start-mcp-gateway + env: + GH_AW_SAFE_OUTPUTS: ${{ steps.set-runtime-paths.outputs.GH_AW_SAFE_OUTPUTS }} + GH_AW_SAFE_OUTPUTS_API_KEY: ${{ steps.safe-outputs-start.outputs.api_key }} + GH_AW_SAFE_OUTPUTS_PORT: ${{ steps.safe-outputs-start.outputs.port }} + GITHUB_MCP_SERVER_TOKEN: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN || secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + run: | + set -eo pipefail + mkdir -p /tmp/gh-aw/mcp-config + + # Export gateway environment variables for MCP config and gateway script + export MCP_GATEWAY_PORT="80" + export MCP_GATEWAY_DOMAIN="host.docker.internal" + MCP_GATEWAY_API_KEY=$(openssl rand -base64 45 | tr -d '/+=') + echo "::add-mask::${MCP_GATEWAY_API_KEY}" + export MCP_GATEWAY_API_KEY + export MCP_GATEWAY_PAYLOAD_DIR="/tmp/gh-aw/mcp-payloads" + mkdir -p "${MCP_GATEWAY_PAYLOAD_DIR}" + export MCP_GATEWAY_PAYLOAD_SIZE_THRESHOLD="524288" + export DEBUG="*" + + export GH_AW_ENGINE="copilot" + export MCP_GATEWAY_DOCKER_COMMAND='docker run -i --rm --network host -v /var/run/docker.sock:/var/run/docker.sock -e MCP_GATEWAY_PORT -e MCP_GATEWAY_DOMAIN -e MCP_GATEWAY_API_KEY -e MCP_GATEWAY_PAYLOAD_DIR -e MCP_GATEWAY_PAYLOAD_SIZE_THRESHOLD -e DEBUG -e MCP_GATEWAY_LOG_DIR -e GH_AW_MCP_LOG_DIR -e GH_AW_SAFE_OUTPUTS -e GH_AW_SAFE_OUTPUTS_CONFIG_PATH -e GH_AW_SAFE_OUTPUTS_TOOLS_PATH -e GH_AW_ASSETS_BRANCH -e GH_AW_ASSETS_MAX_SIZE_KB -e GH_AW_ASSETS_ALLOWED_EXTS -e DEFAULT_BRANCH -e GITHUB_MCP_SERVER_TOKEN -e GITHUB_MCP_GUARD_MIN_INTEGRITY -e GITHUB_MCP_GUARD_REPOS -e GITHUB_REPOSITORY -e GITHUB_SERVER_URL -e GITHUB_SHA -e GITHUB_WORKSPACE -e GITHUB_TOKEN -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e GITHUB_JOB -e GITHUB_ACTION -e GITHUB_EVENT_NAME -e GITHUB_EVENT_PATH -e GITHUB_ACTOR -e GITHUB_ACTOR_ID -e GITHUB_TRIGGERING_ACTOR -e GITHUB_WORKFLOW -e GITHUB_WORKFLOW_REF -e GITHUB_WORKFLOW_SHA -e GITHUB_REF -e GITHUB_REF_NAME -e GITHUB_REF_TYPE -e GITHUB_HEAD_REF -e GITHUB_BASE_REF -e GH_AW_SAFE_OUTPUTS_PORT -e GH_AW_SAFE_OUTPUTS_API_KEY -v /tmp/gh-aw/mcp-payloads:/tmp/gh-aw/mcp-payloads:rw -v /opt:/opt:ro -v /tmp:/tmp:rw -v '"${GITHUB_WORKSPACE}"':'"${GITHUB_WORKSPACE}"':rw ghcr.io/github/gh-aw-mcpg:v0.2.12' + + mkdir -p /home/runner/.copilot + cat << GH_AW_MCP_CONFIG_567e7781e5d7a545_EOF | bash ${RUNNER_TEMP}/gh-aw/actions/start_mcp_gateway.sh + { + "mcpServers": { + "github": { + "type": "stdio", + "container": "ghcr.io/github/github-mcp-server:v0.32.0", + "env": { + "GITHUB_HOST": "\${GITHUB_SERVER_URL}", + "GITHUB_PERSONAL_ACCESS_TOKEN": "\${GITHUB_MCP_SERVER_TOKEN}", + "GITHUB_READ_ONLY": "1", + "GITHUB_TOOLSETS": "context,repos,issues,pull_requests" + }, + "guard-policies": { + "allow-only": { + "approval-labels": ${{ steps.parse-guard-vars.outputs.approval_labels }}, + "blocked-users": ${{ steps.parse-guard-vars.outputs.blocked_users }}, + "min-integrity": "approved", + "repos": "all", + "trusted-users": ${{ steps.parse-guard-vars.outputs.trusted_users }} + } + } + }, + "safeoutputs": { + "type": "http", + "url": "http://host.docker.internal:$GH_AW_SAFE_OUTPUTS_PORT", + "headers": { + "Authorization": "\${GH_AW_SAFE_OUTPUTS_API_KEY}" + }, + "guard-policies": { + "write-sink": { + "accept": [ + "*" + ] + } + } + } + }, + "gateway": { + "port": $MCP_GATEWAY_PORT, + "domain": "${MCP_GATEWAY_DOMAIN}", + "apiKey": "${MCP_GATEWAY_API_KEY}", + "payloadDir": "${MCP_GATEWAY_PAYLOAD_DIR}" + } + } + GH_AW_MCP_CONFIG_567e7781e5d7a545_EOF + - name: Download activation artifact + uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 + with: + name: activation + path: /tmp/gh-aw + - name: Clean git credentials + continue-on-error: true + run: bash ${RUNNER_TEMP}/gh-aw/actions/clean_git_credentials.sh + - name: Execute GitHub Copilot CLI + id: agentic_execution + # Copilot CLI tool arguments (sorted): + timeout-minutes: 120 + run: | + set -o pipefail + touch /tmp/gh-aw/agent-step-summary.md + # shellcheck disable=SC1003 + sudo -E awf --container-workdir "${GITHUB_WORKSPACE}" --mount "${RUNNER_TEMP}/gh-aw:${RUNNER_TEMP}/gh-aw:ro" --mount "${RUNNER_TEMP}/gh-aw:/host${RUNNER_TEMP}/gh-aw:ro" --env-all --exclude-env COPILOT_GITHUB_TOKEN --exclude-env GITHUB_MCP_SERVER_TOKEN --exclude-env MCP_GATEWAY_API_KEY --allow-domains '*.githubusercontent.com,*.vsblob.vsassets.io,api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.npms.io,api.nuget.org,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,azuresearch-usnc.nuget.org,azuresearch-ussc.nuget.org,builds.dotnet.microsoft.com,bun.sh,cdn.jsdelivr.net,ci.dot.net,codeload.github.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,dc.services.visualstudio.com,deb.nodesource.com,deno.land,dist.nuget.org,docs.github.com,dot.net,dotnet.microsoft.com,dotnetcli.blob.core.windows.net,esm.sh,get.pnpm.io,github-cloud.githubusercontent.com,github-cloud.s3.amazonaws.com,github.blog,github.com,github.githubassets.com,googleapis.deno.dev,googlechromelabs.github.io,host.docker.internal,json-schema.org,json.schemastore.org,jsr.io,keyserver.ubuntu.com,lfs.github.com,nodejs.org,npm.pkg.github.com,npmjs.com,npmjs.org,nuget.org,nuget.pkg.github.com,nugetregistryv2prod.blob.core.windows.net,objects.githubusercontent.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,oneocsp.microsoft.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,pkgs.dev.azure.com,ppa.launchpad.net,raw.githubusercontent.com,registry.bower.io,registry.npmjs.com,registry.npmjs.org,registry.yarnpkg.com,repo.yarnpkg.com,s.symcb.com,s.symcd.com,security.ubuntu.com,skimdb.npmjs.com,storage.googleapis.com,telemetry.enterprise.githubcopilot.com,telemetry.vercel.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com,www.microsoft.com,www.npmjs.com,www.npmjs.org,yarnpkg.com' --log-level info --proxy-logs-dir /tmp/gh-aw/sandbox/firewall/logs --audit-dir /tmp/gh-aw/sandbox/firewall/audit --enable-host-access --image-tag 0.25.13 --skip-pull --enable-api-proxy \ + -- /bin/bash -c '/usr/local/bin/copilot --add-dir /tmp/gh-aw/ --log-level all --log-dir /tmp/gh-aw/sandbox/agent/logs/ --disable-builtin-mcps --allow-all-tools --allow-all-paths --add-dir "${GITHUB_WORKSPACE}" --prompt "$(cat /tmp/gh-aw/aw-prompts/prompt.txt)"' 2>&1 | tee -a /tmp/gh-aw/agent-stdio.log + env: + COPILOT_AGENT_RUNNER_TYPE: STANDALONE + COPILOT_GITHUB_TOKEN: ${{ case(needs.pre_activation.outputs.copilot_pat_number == '0', secrets.AUDIT_PAT_0, needs.pre_activation.outputs.copilot_pat_number == '1', secrets.AUDIT_PAT_1, needs.pre_activation.outputs.copilot_pat_number == '2', secrets.AUDIT_PAT_2, needs.pre_activation.outputs.copilot_pat_number == '3', secrets.AUDIT_PAT_3, needs.pre_activation.outputs.copilot_pat_number == '4', secrets.AUDIT_PAT_4, needs.pre_activation.outputs.copilot_pat_number == '5', secrets.AUDIT_PAT_5, needs.pre_activation.outputs.copilot_pat_number == '6', secrets.AUDIT_PAT_6, needs.pre_activation.outputs.copilot_pat_number == '7', secrets.AUDIT_PAT_7, needs.pre_activation.outputs.copilot_pat_number == '8', secrets.AUDIT_PAT_8, needs.pre_activation.outputs.copilot_pat_number == '9', secrets.AUDIT_PAT_9, secrets.COPILOT_GITHUB_TOKEN) }} + COPILOT_MODEL: ${{ vars.GH_AW_MODEL_AGENT_COPILOT || '' }} + GH_AW_MCP_CONFIG: /home/runner/.copilot/mcp-config.json + GH_AW_PHASE: agent + GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt + GH_AW_SAFE_OUTPUTS: ${{ steps.set-runtime-paths.outputs.GH_AW_SAFE_OUTPUTS }} + GH_AW_VERSION: v0.66.1 + GITHUB_API_URL: ${{ github.api_url }} + GITHUB_AW: true + GITHUB_HEAD_REF: ${{ github.head_ref }} + GITHUB_MCP_SERVER_TOKEN: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN || secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + GITHUB_REF_NAME: ${{ github.ref_name }} + GITHUB_SERVER_URL: ${{ github.server_url }} + GITHUB_STEP_SUMMARY: /tmp/gh-aw/agent-step-summary.md + GITHUB_WORKSPACE: ${{ github.workspace }} + GIT_AUTHOR_EMAIL: github-actions[bot]@users.noreply.github.com + GIT_AUTHOR_NAME: github-actions[bot] + GIT_COMMITTER_EMAIL: github-actions[bot]@users.noreply.github.com + GIT_COMMITTER_NAME: github-actions[bot] + XDG_CONFIG_HOME: /home/runner + - name: Detect inference access error + id: detect-inference-error + if: always() + continue-on-error: true + run: bash ${RUNNER_TEMP}/gh-aw/actions/detect_inference_access_error.sh + - name: Configure Git credentials + env: + REPO_NAME: ${{ github.repository }} + SERVER_URL: ${{ github.server_url }} + run: | + git config --global user.email "github-actions[bot]@users.noreply.github.com" + git config --global user.name "github-actions[bot]" + git config --global am.keepcr true + # Re-authenticate git with GitHub token + SERVER_URL_STRIPPED="${SERVER_URL#https://}" + git remote set-url origin "https://x-access-token:${{ github.token }}@${SERVER_URL_STRIPPED}/${REPO_NAME}.git" + echo "Git configured with standard GitHub Actions identity" + - name: Copy Copilot session state files to logs + if: always() + continue-on-error: true + run: bash ${RUNNER_TEMP}/gh-aw/actions/copy_copilot_session_state.sh + - name: Stop MCP Gateway + if: always() + continue-on-error: true + env: + MCP_GATEWAY_PORT: ${{ steps.start-mcp-gateway.outputs.gateway-port }} + MCP_GATEWAY_API_KEY: ${{ steps.start-mcp-gateway.outputs.gateway-api-key }} + GATEWAY_PID: ${{ steps.start-mcp-gateway.outputs.gateway-pid }} + run: | + bash ${RUNNER_TEMP}/gh-aw/actions/stop_mcp_gateway.sh "$GATEWAY_PID" + - name: Redact secrets in logs + if: always() + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + with: + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/redact_secrets.cjs'); + await main(); + env: + GH_AW_SECRET_NAMES: 'AUDIT_PAT_0,AUDIT_PAT_1,AUDIT_PAT_2,AUDIT_PAT_3,AUDIT_PAT_4,AUDIT_PAT_5,AUDIT_PAT_6,AUDIT_PAT_7,AUDIT_PAT_8,AUDIT_PAT_9,COPILOT_GITHUB_TOKEN,GH_AW_GITHUB_MCP_SERVER_TOKEN,GH_AW_GITHUB_TOKEN,GITHUB_TOKEN' + SECRET_AUDIT_PAT_0: ${{ secrets.AUDIT_PAT_0 }} + SECRET_AUDIT_PAT_1: ${{ secrets.AUDIT_PAT_1 }} + SECRET_AUDIT_PAT_2: ${{ secrets.AUDIT_PAT_2 }} + SECRET_AUDIT_PAT_3: ${{ secrets.AUDIT_PAT_3 }} + SECRET_AUDIT_PAT_4: ${{ secrets.AUDIT_PAT_4 }} + SECRET_AUDIT_PAT_5: ${{ secrets.AUDIT_PAT_5 }} + SECRET_AUDIT_PAT_6: ${{ secrets.AUDIT_PAT_6 }} + SECRET_AUDIT_PAT_7: ${{ secrets.AUDIT_PAT_7 }} + SECRET_AUDIT_PAT_8: ${{ secrets.AUDIT_PAT_8 }} + SECRET_AUDIT_PAT_9: ${{ secrets.AUDIT_PAT_9 }} + SECRET_COPILOT_GITHUB_TOKEN: ${{ secrets.COPILOT_GITHUB_TOKEN }} + SECRET_GH_AW_GITHUB_MCP_SERVER_TOKEN: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN }} + SECRET_GH_AW_GITHUB_TOKEN: ${{ secrets.GH_AW_GITHUB_TOKEN }} + SECRET_GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + - name: Append agent step summary + if: always() + run: bash ${RUNNER_TEMP}/gh-aw/actions/append_agent_step_summary.sh + - name: Copy Safe Outputs + if: always() + env: + GH_AW_SAFE_OUTPUTS: ${{ steps.set-runtime-paths.outputs.GH_AW_SAFE_OUTPUTS }} + run: | + mkdir -p /tmp/gh-aw + cp "$GH_AW_SAFE_OUTPUTS" /tmp/gh-aw/safeoutputs.jsonl 2>/dev/null || true + - name: Ingest agent output + id: collect_output + if: always() + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_SAFE_OUTPUTS: ${{ steps.set-runtime-paths.outputs.GH_AW_SAFE_OUTPUTS }} + GH_AW_ALLOWED_DOMAINS: "*.githubusercontent.com,*.vsblob.vsassets.io,api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.npms.io,api.nuget.org,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,azuresearch-usnc.nuget.org,azuresearch-ussc.nuget.org,builds.dotnet.microsoft.com,bun.sh,cdn.jsdelivr.net,ci.dot.net,codeload.github.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,dc.services.visualstudio.com,deb.nodesource.com,deno.land,dist.nuget.org,docs.github.com,dot.net,dotnet.microsoft.com,dotnetcli.blob.core.windows.net,esm.sh,get.pnpm.io,github-cloud.githubusercontent.com,github-cloud.s3.amazonaws.com,github.blog,github.com,github.githubassets.com,googleapis.deno.dev,googlechromelabs.github.io,host.docker.internal,json-schema.org,json.schemastore.org,jsr.io,keyserver.ubuntu.com,lfs.github.com,nodejs.org,npm.pkg.github.com,npmjs.com,npmjs.org,nuget.org,nuget.pkg.github.com,nugetregistryv2prod.blob.core.windows.net,objects.githubusercontent.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,oneocsp.microsoft.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,pkgs.dev.azure.com,ppa.launchpad.net,raw.githubusercontent.com,registry.bower.io,registry.npmjs.com,registry.npmjs.org,registry.yarnpkg.com,repo.yarnpkg.com,s.symcb.com,s.symcd.com,security.ubuntu.com,skimdb.npmjs.com,storage.googleapis.com,telemetry.enterprise.githubcopilot.com,telemetry.vercel.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com,www.microsoft.com,www.npmjs.com,www.npmjs.org,yarnpkg.com" + GITHUB_SERVER_URL: ${{ github.server_url }} + GITHUB_API_URL: ${{ github.api_url }} + with: + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/collect_ndjson_output.cjs'); + await main(); + - name: Parse agent logs for step summary + if: always() + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_AGENT_OUTPUT: /tmp/gh-aw/sandbox/agent/logs/ + with: + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/parse_copilot_log.cjs'); + await main(); + - name: Parse MCP Gateway logs for step summary + if: always() + id: parse-mcp-gateway + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + with: + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/parse_mcp_gateway_log.cjs'); + await main(); + - name: Print firewall logs + if: always() + continue-on-error: true + env: + AWF_LOGS_DIR: /tmp/gh-aw/sandbox/firewall/logs + run: | + # Fix permissions on firewall logs so they can be uploaded as artifacts + # AWF runs with sudo, creating files owned by root + sudo chmod -R a+r /tmp/gh-aw/sandbox/firewall/logs 2>/dev/null || true + # Only run awf logs summary if awf command exists (it may not be installed if workflow failed before install step) + if command -v awf &> /dev/null; then + awf logs summary | tee -a "$GITHUB_STEP_SUMMARY" + else + echo 'AWF binary not installed, skipping firewall log summary' + fi + - name: Parse token usage for step summary + if: always() + continue-on-error: true + run: bash ${RUNNER_TEMP}/gh-aw/actions/parse_token_usage.sh + - name: Write agent output placeholder if missing + if: always() + run: | + if [ ! -f /tmp/gh-aw/agent_output.json ]; then + echo '{"items":[]}' > /tmp/gh-aw/agent_output.json + fi + - if: always() + name: Upload audit report + uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4 + with: + if-no-files-found: ignore + name: audit-report + path: /tmp/audit-report.md + retention-days: 90 + + - name: Upload agent artifacts + if: always() + continue-on-error: true + uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7 + with: + name: agent + path: | + /tmp/gh-aw/aw-prompts/prompt.txt + /tmp/gh-aw/sandbox/agent/logs/ + /tmp/gh-aw/redacted-urls.log + /tmp/gh-aw/mcp-logs/ + /tmp/gh-aw/proxy-logs/ + !/tmp/gh-aw/proxy-logs/proxy-tls/ + /tmp/gh-aw/agent_usage.json + /tmp/gh-aw/agent-stdio.log + /tmp/gh-aw/agent/ + /tmp/gh-aw/safeoutputs.jsonl + /tmp/gh-aw/agent_output.json + /tmp/gh-aw/aw-*.patch + /tmp/gh-aw/aw-*.bundle + if-no-files-found: ignore + - name: Upload firewall audit logs + if: always() + continue-on-error: true + uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7 + with: + name: firewall-audit-logs + path: | + /tmp/gh-aw/sandbox/firewall/logs/ + /tmp/gh-aw/sandbox/firewall/audit/ + if-no-files-found: ignore + + conclusion: + needs: + - activation + - agent + - detection + - safe_outputs + - tier-check + if: always() && (needs.agent.result != 'skipped' || needs.activation.outputs.lockdown_check_failed == 'true') + runs-on: ubuntu-slim + permissions: + contents: read + issues: write + concurrency: + group: "gh-aw-conclusion-sdk-tier-audit" + cancel-in-progress: false + outputs: + noop_message: ${{ steps.noop.outputs.noop_message }} + tools_reported: ${{ steps.missing_tool.outputs.tools_reported }} + total_count: ${{ steps.missing_tool.outputs.total_count }} + steps: + - name: Setup Scripts + uses: github/gh-aw-actions/setup@73ae9ce231580f337133352d321d42b6bf54b6a9 # v0.66.1 + with: + destination: ${{ runner.temp }}/gh-aw/actions + - name: Download agent output artifact + id: download-agent-output + continue-on-error: true + uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 + with: + name: agent + path: /tmp/gh-aw/ + - name: Setup agent output environment variable + id: setup-agent-output-env + if: steps.download-agent-output.outcome == 'success' + run: | + mkdir -p /tmp/gh-aw/ + find "/tmp/gh-aw/" -type f -print + echo "GH_AW_AGENT_OUTPUT=/tmp/gh-aw/agent_output.json" >> "$GITHUB_OUTPUT" + - name: Process No-Op Messages + id: noop + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_AGENT_OUTPUT: ${{ steps.setup-agent-output-env.outputs.GH_AW_AGENT_OUTPUT }} + GH_AW_NOOP_MAX: "1" + GH_AW_WORKFLOW_NAME: "SDK Tier Audit" + GH_AW_RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} + GH_AW_AGENT_CONCLUSION: ${{ needs.agent.result }} + GH_AW_NOOP_REPORT_AS_ISSUE: "true" + with: + github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/handle_noop_message.cjs'); + await main(); + - name: Record Missing Tool + id: missing_tool + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_AGENT_OUTPUT: ${{ steps.setup-agent-output-env.outputs.GH_AW_AGENT_OUTPUT }} + GH_AW_MISSING_TOOL_CREATE_ISSUE: "true" + GH_AW_WORKFLOW_NAME: "SDK Tier Audit" + with: + github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/missing_tool.cjs'); + await main(); + - name: Handle Agent Failure + id: handle_agent_failure + if: always() + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_AGENT_OUTPUT: ${{ steps.setup-agent-output-env.outputs.GH_AW_AGENT_OUTPUT }} + GH_AW_WORKFLOW_NAME: "SDK Tier Audit" + GH_AW_RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} + GH_AW_AGENT_CONCLUSION: ${{ needs.agent.result }} + GH_AW_WORKFLOW_ID: "sdk-tier-audit" + GH_AW_ENGINE_ID: "copilot" + GH_AW_SECRET_VERIFICATION_RESULT: ${{ needs.activation.outputs.secret_verification_result }} + GH_AW_CHECKOUT_PR_SUCCESS: ${{ needs.agent.outputs.checkout_pr_success }} + GH_AW_INFERENCE_ACCESS_ERROR: ${{ needs.agent.outputs.inference_access_error }} + GH_AW_LOCKDOWN_CHECK_FAILED: ${{ needs.activation.outputs.lockdown_check_failed }} + GH_AW_GROUP_REPORTS: "false" + GH_AW_FAILURE_REPORT_AS_ISSUE: "true" + GH_AW_TIMEOUT_MINUTES: "120" + with: + github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/handle_agent_failure.cjs'); + await main(); + + detection: + needs: agent + if: > + always() && needs.agent.result != 'skipped' && (needs.agent.outputs.output_types != '' || needs.agent.outputs.has_patch == 'true') + runs-on: ubuntu-latest + permissions: + contents: read + outputs: + detection_conclusion: ${{ steps.detection_conclusion.outputs.conclusion }} + detection_success: ${{ steps.detection_conclusion.outputs.success }} + steps: + - name: Setup Scripts + uses: github/gh-aw-actions/setup@73ae9ce231580f337133352d321d42b6bf54b6a9 # v0.66.1 + with: + destination: ${{ runner.temp }}/gh-aw/actions + - name: Download agent output artifact + id: download-agent-output + continue-on-error: true + uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 + with: + name: agent + path: /tmp/gh-aw/ + - name: Setup agent output environment variable + id: setup-agent-output-env + if: steps.download-agent-output.outcome == 'success' + run: | + mkdir -p /tmp/gh-aw/ + find "/tmp/gh-aw/" -type f -print + echo "GH_AW_AGENT_OUTPUT=/tmp/gh-aw/agent_output.json" >> "$GITHUB_OUTPUT" + - name: Checkout repository for patch context + if: needs.agent.outputs.has_patch == 'true' + uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 + with: + persist-credentials: false + # --- Threat Detection --- + - name: Download container images + run: bash ${RUNNER_TEMP}/gh-aw/actions/download_docker_images.sh ghcr.io/github/gh-aw-firewall/agent:0.25.13 ghcr.io/github/gh-aw-firewall/api-proxy:0.25.13 ghcr.io/github/gh-aw-firewall/squid:0.25.13 + - name: Check if detection needed + id: detection_guard + if: always() + env: + OUTPUT_TYPES: ${{ needs.agent.outputs.output_types }} + HAS_PATCH: ${{ needs.agent.outputs.has_patch }} + run: | + if [[ -n "$OUTPUT_TYPES" || "$HAS_PATCH" == "true" ]]; then + echo "run_detection=true" >> "$GITHUB_OUTPUT" + echo "Detection will run: output_types=$OUTPUT_TYPES, has_patch=$HAS_PATCH" + else + echo "run_detection=false" >> "$GITHUB_OUTPUT" + echo "Detection skipped: no agent outputs or patches to analyze" + fi + - name: Clear MCP configuration for detection + if: always() && steps.detection_guard.outputs.run_detection == 'true' + run: | + rm -f /tmp/gh-aw/mcp-config/mcp-servers.json + rm -f /home/runner/.copilot/mcp-config.json + rm -f "$GITHUB_WORKSPACE/.gemini/settings.json" + - name: Prepare threat detection files + if: always() && steps.detection_guard.outputs.run_detection == 'true' + run: | + mkdir -p /tmp/gh-aw/threat-detection/aw-prompts + cp /tmp/gh-aw/aw-prompts/prompt.txt /tmp/gh-aw/threat-detection/aw-prompts/prompt.txt 2>/dev/null || true + cp /tmp/gh-aw/agent_output.json /tmp/gh-aw/threat-detection/agent_output.json 2>/dev/null || true + for f in /tmp/gh-aw/aw-*.patch; do + [ -f "$f" ] && cp "$f" /tmp/gh-aw/threat-detection/ 2>/dev/null || true + done + for f in /tmp/gh-aw/aw-*.bundle; do + [ -f "$f" ] && cp "$f" /tmp/gh-aw/threat-detection/ 2>/dev/null || true + done + echo "Prepared threat detection files:" + ls -la /tmp/gh-aw/threat-detection/ 2>/dev/null || true + - name: Setup threat detection + if: always() && steps.detection_guard.outputs.run_detection == 'true' + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + WORKFLOW_NAME: "SDK Tier Audit" + WORKFLOW_DESCRIPTION: "SDK Tier Audit" + HAS_PATCH: ${{ needs.agent.outputs.has_patch }} + with: + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/setup_threat_detection.cjs'); + await main(); + - name: Ensure threat-detection directory and log + if: always() && steps.detection_guard.outputs.run_detection == 'true' + run: | + mkdir -p /tmp/gh-aw/threat-detection + touch /tmp/gh-aw/threat-detection/detection.log + - name: Install GitHub Copilot CLI + run: ${RUNNER_TEMP}/gh-aw/actions/install_copilot_cli.sh latest + env: + GH_HOST: github.com + - name: Install AWF binary + run: bash ${RUNNER_TEMP}/gh-aw/actions/install_awf_binary.sh v0.25.13 + - name: Execute GitHub Copilot CLI + if: always() && steps.detection_guard.outputs.run_detection == 'true' + id: detection_agentic_execution + # Copilot CLI tool arguments (sorted): + timeout-minutes: 20 + run: | + set -o pipefail + touch /tmp/gh-aw/agent-step-summary.md + # shellcheck disable=SC1003 + sudo -E awf --container-workdir "${GITHUB_WORKSPACE}" --mount "${RUNNER_TEMP}/gh-aw:${RUNNER_TEMP}/gh-aw:ro" --mount "${RUNNER_TEMP}/gh-aw:/host${RUNNER_TEMP}/gh-aw:ro" --env-all --exclude-env COPILOT_GITHUB_TOKEN --allow-domains api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,github.com,host.docker.internal,telemetry.enterprise.githubcopilot.com --log-level info --proxy-logs-dir /tmp/gh-aw/sandbox/firewall/logs --audit-dir /tmp/gh-aw/sandbox/firewall/audit --enable-host-access --image-tag 0.25.13 --skip-pull --enable-api-proxy \ + -- /bin/bash -c '/usr/local/bin/copilot --add-dir /tmp/gh-aw/ --log-level all --log-dir /tmp/gh-aw/sandbox/agent/logs/ --disable-builtin-mcps --allow-all-tools --add-dir "${GITHUB_WORKSPACE}" --prompt "$(cat /tmp/gh-aw/aw-prompts/prompt.txt)"' 2>&1 | tee -a /tmp/gh-aw/threat-detection/detection.log + env: + COPILOT_AGENT_RUNNER_TYPE: STANDALONE + COPILOT_GITHUB_TOKEN: ${{ case(needs.pre_activation.outputs.copilot_pat_number == '0', secrets.AUDIT_PAT_0, needs.pre_activation.outputs.copilot_pat_number == '1', secrets.AUDIT_PAT_1, needs.pre_activation.outputs.copilot_pat_number == '2', secrets.AUDIT_PAT_2, needs.pre_activation.outputs.copilot_pat_number == '3', secrets.AUDIT_PAT_3, needs.pre_activation.outputs.copilot_pat_number == '4', secrets.AUDIT_PAT_4, needs.pre_activation.outputs.copilot_pat_number == '5', secrets.AUDIT_PAT_5, needs.pre_activation.outputs.copilot_pat_number == '6', secrets.AUDIT_PAT_6, needs.pre_activation.outputs.copilot_pat_number == '7', secrets.AUDIT_PAT_7, needs.pre_activation.outputs.copilot_pat_number == '8', secrets.AUDIT_PAT_8, needs.pre_activation.outputs.copilot_pat_number == '9', secrets.AUDIT_PAT_9, secrets.COPILOT_GITHUB_TOKEN) }} + COPILOT_MODEL: ${{ vars.GH_AW_MODEL_DETECTION_COPILOT || '' }} + GH_AW_PHASE: detection + GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt + GH_AW_VERSION: v0.66.1 + GITHUB_API_URL: ${{ github.api_url }} + GITHUB_AW: true + GITHUB_HEAD_REF: ${{ github.head_ref }} + GITHUB_REF_NAME: ${{ github.ref_name }} + GITHUB_SERVER_URL: ${{ github.server_url }} + GITHUB_STEP_SUMMARY: /tmp/gh-aw/agent-step-summary.md + GITHUB_WORKSPACE: ${{ github.workspace }} + GIT_AUTHOR_EMAIL: github-actions[bot]@users.noreply.github.com + GIT_AUTHOR_NAME: github-actions[bot] + GIT_COMMITTER_EMAIL: github-actions[bot]@users.noreply.github.com + GIT_COMMITTER_NAME: github-actions[bot] + XDG_CONFIG_HOME: /home/runner + - name: Upload threat detection log + if: always() && steps.detection_guard.outputs.run_detection == 'true' + uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7 + with: + name: detection + path: /tmp/gh-aw/threat-detection/detection.log + if-no-files-found: ignore + - name: Parse and conclude threat detection + id: detection_conclusion + if: always() + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + RUN_DETECTION: ${{ steps.detection_guard.outputs.run_detection }} + with: + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/parse_threat_detection_results.cjs'); + await main(); + + pre_activation: + if: github.repository_owner == 'modelcontextprotocol' || github.event_name == 'workflow_dispatch' + runs-on: ubuntu-slim + outputs: + activated: ${{ steps.check_membership.outputs.is_team_member == 'true' }} + copilot_pat_number: ${{ steps.select-copilot-pat.outputs.copilot_pat_number }} + matched_command: '' + select-copilot-pat_result: ${{ steps.select-copilot-pat.outcome }} + steps: + - name: Setup Scripts + uses: github/gh-aw-actions/setup@73ae9ce231580f337133352d321d42b6bf54b6a9 # v0.66.1 + with: + destination: ${{ runner.temp }}/gh-aw/actions + - name: Check team membership for workflow + id: check_membership + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_REQUIRED_ROLES: "admin,maintainer,write" + with: + github-token: ${{ secrets.GITHUB_TOKEN }} + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/check_membership.cjs'); + await main(); + - name: Checkout the select-copilot-pat action folder + uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 + with: + fetch-depth: 1 + persist-credentials: false + sparse-checkout: .github/actions/select-copilot-pat + sparse-checkout-cone-mode: true + - name: Select Copilot token from pool + id: select-copilot-pat + uses: ./.github/actions/select-copilot-pat + env: + SECRET_0: ${{ secrets.AUDIT_PAT_0 }} + SECRET_1: ${{ secrets.AUDIT_PAT_1 }} + SECRET_2: ${{ secrets.AUDIT_PAT_2 }} + SECRET_3: ${{ secrets.AUDIT_PAT_3 }} + SECRET_4: ${{ secrets.AUDIT_PAT_4 }} + SECRET_5: ${{ secrets.AUDIT_PAT_5 }} + SECRET_6: ${{ secrets.AUDIT_PAT_6 }} + SECRET_7: ${{ secrets.AUDIT_PAT_7 }} + SECRET_8: ${{ secrets.AUDIT_PAT_8 }} + SECRET_9: ${{ secrets.AUDIT_PAT_9 }} + + safe_outputs: + needs: + - agent + - detection + if: (!cancelled()) && needs.agent.result != 'skipped' && needs.detection.result == 'success' + runs-on: ubuntu-slim + permissions: + contents: read + issues: write + timeout-minutes: 15 + env: + GH_AW_CALLER_WORKFLOW_ID: "${{ github.repository }}/sdk-tier-audit" + GH_AW_EFFECTIVE_TOKENS: ${{ needs.agent.outputs.effective_tokens }} + GH_AW_ENGINE_ID: "copilot" + GH_AW_ENGINE_MODEL: ${{ needs.agent.outputs.model }} + GH_AW_WORKFLOW_ID: "sdk-tier-audit" + GH_AW_WORKFLOW_NAME: "SDK Tier Audit" + outputs: + code_push_failure_count: ${{ steps.process_safe_outputs.outputs.code_push_failure_count }} + code_push_failure_errors: ${{ steps.process_safe_outputs.outputs.code_push_failure_errors }} + create_discussion_error_count: ${{ steps.process_safe_outputs.outputs.create_discussion_error_count }} + create_discussion_errors: ${{ steps.process_safe_outputs.outputs.create_discussion_errors }} + created_issue_number: ${{ steps.process_safe_outputs.outputs.created_issue_number }} + created_issue_url: ${{ steps.process_safe_outputs.outputs.created_issue_url }} + process_safe_outputs_processed_count: ${{ steps.process_safe_outputs.outputs.processed_count }} + process_safe_outputs_temporary_id_map: ${{ steps.process_safe_outputs.outputs.temporary_id_map }} + steps: + - name: Setup Scripts + uses: github/gh-aw-actions/setup@73ae9ce231580f337133352d321d42b6bf54b6a9 # v0.66.1 + with: + destination: ${{ runner.temp }}/gh-aw/actions + - name: Download agent output artifact + id: download-agent-output + continue-on-error: true + uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 + with: + name: agent + path: /tmp/gh-aw/ + - name: Setup agent output environment variable + id: setup-agent-output-env + if: steps.download-agent-output.outcome == 'success' + run: | + mkdir -p /tmp/gh-aw/ + find "/tmp/gh-aw/" -type f -print + echo "GH_AW_AGENT_OUTPUT=/tmp/gh-aw/agent_output.json" >> "$GITHUB_OUTPUT" + - name: Configure GH_HOST for enterprise compatibility + id: ghes-host-config + shell: bash + run: | + # Derive GH_HOST from GITHUB_SERVER_URL so the gh CLI targets the correct + # GitHub instance (GHES/GHEC). On github.com this is a harmless no-op. + GH_HOST="${GITHUB_SERVER_URL#https://}" + GH_HOST="${GH_HOST#http://}" + echo "GH_HOST=${GH_HOST}" >> "$GITHUB_ENV" + - name: Process Safe Outputs + id: process_safe_outputs + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_AGENT_OUTPUT: ${{ steps.setup-agent-output-env.outputs.GH_AW_AGENT_OUTPUT }} + GH_AW_ALLOWED_DOMAINS: "*.githubusercontent.com,*.vsblob.vsassets.io,api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.npms.io,api.nuget.org,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,azuresearch-usnc.nuget.org,azuresearch-ussc.nuget.org,builds.dotnet.microsoft.com,bun.sh,cdn.jsdelivr.net,ci.dot.net,codeload.github.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,dc.services.visualstudio.com,deb.nodesource.com,deno.land,dist.nuget.org,docs.github.com,dot.net,dotnet.microsoft.com,dotnetcli.blob.core.windows.net,esm.sh,get.pnpm.io,github-cloud.githubusercontent.com,github-cloud.s3.amazonaws.com,github.blog,github.com,github.githubassets.com,googleapis.deno.dev,googlechromelabs.github.io,host.docker.internal,json-schema.org,json.schemastore.org,jsr.io,keyserver.ubuntu.com,lfs.github.com,nodejs.org,npm.pkg.github.com,npmjs.com,npmjs.org,nuget.org,nuget.pkg.github.com,nugetregistryv2prod.blob.core.windows.net,objects.githubusercontent.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,oneocsp.microsoft.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,pkgs.dev.azure.com,ppa.launchpad.net,raw.githubusercontent.com,registry.bower.io,registry.npmjs.com,registry.npmjs.org,registry.yarnpkg.com,repo.yarnpkg.com,s.symcb.com,s.symcd.com,security.ubuntu.com,skimdb.npmjs.com,storage.googleapis.com,telemetry.enterprise.githubcopilot.com,telemetry.vercel.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com,www.microsoft.com,www.npmjs.com,www.npmjs.org,yarnpkg.com" + GITHUB_SERVER_URL: ${{ github.server_url }} + GITHUB_API_URL: ${{ github.api_url }} + GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"create_issue\":{\"close_older_issues\":true,\"labels\":[\"automation\"],\"max\":1,\"title_prefix\":\"[C# SDK Tier Audit] \"},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"}}" + with: + github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/safe_output_handler_manager.cjs'); + await main(); + - name: Upload Safe Output Items + if: always() + uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7 + with: + name: safe-output-items + path: /tmp/gh-aw/safe-output-items.jsonl + if-no-files-found: ignore + + tier-check: + needs: activation + runs-on: ubuntu-latest + timeout-minutes: 30 + outputs: + conformance_branch: ${{ steps.params.outputs.conformance_branch }} + conformance_repo: ${{ steps.params.outputs.conformance_repo }} + csharp_sdk_branch: ${{ steps.params.outputs.csharp_sdk_branch }} + csharp_sdk_repo: ${{ steps.params.outputs.csharp_sdk_repo }} + output: ${{ steps.params.outputs.output }} + scope: ${{ steps.params.outputs.scope }} + tier_check_json: ${{ steps.tier-check.outputs.result || steps.tier-check-health.outputs.result }} + steps: + - name: Configure GH_HOST for enterprise compatibility + id: ghes-host-config + shell: bash + run: | + # Derive GH_HOST from GITHUB_SERVER_URL so the gh CLI targets the correct + # GitHub instance (GHES/GHEC). On github.com this is a harmless no-op. + GH_HOST="${GITHUB_SERVER_URL#https://}" + GH_HOST="${GH_HOST#http://}" + echo "GH_HOST=${GH_HOST}" >> "$GITHUB_ENV" + - name: Resolve parameters + id: params + run: | + echo "scope=${INPUT_SCOPE}" >> "$GITHUB_OUTPUT" + echo "output=${INPUT_OUTPUT}" >> "$GITHUB_OUTPUT" + SDK_REPO="${INPUT_SDK%%:*}" + SDK_BRANCH="${INPUT_SDK#*:}" + echo "csharp_sdk_repo=${SDK_REPO}" >> "$GITHUB_OUTPUT" + echo "csharp_sdk_branch=${SDK_BRANCH}" >> "$GITHUB_OUTPUT" + CONF_REPO="${INPUT_CONF%%:*}" + CONF_BRANCH="${INPUT_CONF#*:}" + echo "conformance_repo=${CONF_REPO}" >> "$GITHUB_OUTPUT" + echo "conformance_branch=${CONF_BRANCH}" >> "$GITHUB_OUTPUT" + env: + INPUT_CONF: ${{ github.event.inputs.conformance || 'modelcontextprotocol/conformance:main' }} + INPUT_OUTPUT: ${{ github.event.inputs.output || 'Create Issue' }} + INPUT_SCOPE: ${{ github.event.inputs.scope || 'Conformance + Repo Health' }} + INPUT_SDK: ${{ github.event.inputs.csharp_sdk || 'modelcontextprotocol/csharp-sdk:main' }} + - name: Setup .NET SDK + uses: actions/setup-dotnet@c2fa09f4bde5ebb9d1777cf28262a3eb3db3ced7 # v5 + with: + dotnet-version: "10.0" + - name: Setup Node.js + uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6 + with: + node-version: "22" + - name: "Clone C# SDK" + run: git clone --depth 1 -b "$SDK_BRANCH" "https://github.com/${SDK_REPO}.git" /tmp/csharp-sdk + env: + SDK_BRANCH: ${{ steps.params.outputs.csharp_sdk_branch }} + SDK_REPO: ${{ steps.params.outputs.csharp_sdk_repo }} + - name: Clone conformance repo + run: git clone --depth 1 -b "$CONF_BRANCH" "https://github.com/${CONF_REPO}.git" /tmp/conformance + env: + CONF_BRANCH: ${{ steps.params.outputs.conformance_branch }} + CONF_REPO: ${{ steps.params.outputs.conformance_repo }} + - name: Build conformance CLI + run: cd /tmp/conformance && npm ci && npm run build + - name: "Build C# SDK" + if: steps.params.outputs.scope == 'Conformance + Repo Health' + run: cd /tmp/csharp-sdk && dotnet build --verbosity quiet + - name: Start conformance server + if: steps.params.outputs.scope == 'Conformance + Repo Health' + run: | + cd /tmp/csharp-sdk + nohup dotnet run --no-build --project tests/ModelContextProtocol.ConformanceServer --framework net9.0 -- --urls http://localhost:3003 > /tmp/conformance-server.log 2>&1 & + timeout 60 bash -c 'until curl -sf http://localhost:3003/health; do sleep 1; done' + - name: Run tier-check (Conformance + Repo Health) + id: tier-check + if: steps.params.outputs.scope == 'Conformance + Repo Health' + run: | + cd /tmp/conformance + REPO=$(cd /tmp/csharp-sdk && git remote get-url origin | sed 's#.*github.com[:/]##; s#\.git$##') + BRANCH=$(cd /tmp/csharp-sdk && git rev-parse --abbrev-ref HEAD) + set +e + RESULT=$(node dist/index.js tier-check \ + --repo "$REPO" \ + --branch "$BRANCH" \ + --conformance-server-url http://localhost:3003 \ + --client-cmd "dotnet run --no-build --project /tmp/csharp-sdk/tests/ModelContextProtocol.ConformanceClient --framework net9.0 -- \$MCP_CONFORMANCE_SCENARIO" \ + --output json 2>/tmp/tier-check-stderr.txt) + EXIT_CODE=$? + set -e + cat /tmp/tier-check-stderr.txt >&2 + if [ $EXIT_CODE -ne 0 ]; then + echo "::error::tier-check CLI failed with exit code $EXIT_CODE" + exit $EXIT_CODE + fi + EOF_MARKER=$(uuidgen) + echo "result<<${EOF_MARKER}" >> "$GITHUB_OUTPUT" + echo "$RESULT" >> "$GITHUB_OUTPUT" + echo "${EOF_MARKER}" >> "$GITHUB_OUTPUT" + env: + GITHUB_TOKEN: ${{ github.token }} + - name: Run tier-check (Repo Health) + id: tier-check-health + if: steps.params.outputs.scope == 'Repo Health' + run: | + cd /tmp/conformance + REPO=$(cd /tmp/csharp-sdk && git remote get-url origin | sed 's#.*github.com[:/]##; s#\.git$##') + BRANCH=$(cd /tmp/csharp-sdk && git rev-parse --abbrev-ref HEAD) + set +e + RESULT=$(node dist/index.js tier-check \ + --repo "$REPO" \ + --branch "$BRANCH" \ + --skip-conformance \ + --output json 2>/tmp/tier-check-stderr.txt) + EXIT_CODE=$? + set -e + cat /tmp/tier-check-stderr.txt >&2 + if [ $EXIT_CODE -ne 0 ]; then + echo "::error::tier-check CLI failed with exit code $EXIT_CODE" + exit $EXIT_CODE + fi + EOF_MARKER=$(uuidgen) + echo "result<<${EOF_MARKER}" >> "$GITHUB_OUTPUT" + echo "$RESULT" >> "$GITHUB_OUTPUT" + echo "${EOF_MARKER}" >> "$GITHUB_OUTPUT" + env: + GITHUB_TOKEN: ${{ github.token }} + diff --git a/.github/workflows/sdk-tier-audit.md b/.github/workflows/sdk-tier-audit.md new file mode 100644 index 000000000..8109e2517 --- /dev/null +++ b/.github/workflows/sdk-tier-audit.md @@ -0,0 +1,443 @@ +--- +description: "SDK Tier Audit" + +permissions: + contents: read + issues: read + pull-requests: read + +safe-outputs: + create-issue: + title-prefix: "[C# SDK Tier Audit] " + labels: [automation] + close-older-issues: true + max: 1 + +tools: + github: + min-integrity: approved + +if: github.repository_owner == 'modelcontextprotocol' || github.event_name == 'workflow_dispatch' + +env: + FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true" + +concurrency: + group: tier-audit-${{ github.event.inputs.scope || 'Conformance + Repo Health' }} + cancel-in-progress: true + +runtimes: + node: + version: "22" + dotnet: + version: "10.0" + +network: + allowed: + - defaults + - node + - dotnet + - github + +timeout-minutes: 120 + +steps: + - name: Write tier-check outputs to files + env: + TIER_CHECK_JSON: ${{ needs.tier-check.outputs.tier_check_json }} + AUDIT_SCOPE: ${{ needs.tier-check.outputs.scope }} + AUDIT_OUTPUT: ${{ needs.tier-check.outputs.output }} + AUDIT_SDK_REPO: ${{ needs.tier-check.outputs.csharp_sdk_repo }} + AUDIT_SDK_BRANCH: ${{ needs.tier-check.outputs.csharp_sdk_branch }} + AUDIT_CONF_REPO: ${{ needs.tier-check.outputs.conformance_repo }} + AUDIT_CONF_BRANCH: ${{ needs.tier-check.outputs.conformance_branch }} + run: | + echo "$TIER_CHECK_JSON" > /tmp/tier-check-scorecard.json + mkdir -p /tmp/audit-params + echo "$AUDIT_SCOPE" > /tmp/audit-params/scope + echo "$AUDIT_OUTPUT" > /tmp/audit-params/output + echo "$AUDIT_SDK_REPO" > /tmp/audit-params/csharp-sdk-repo + echo "$AUDIT_SDK_BRANCH" > /tmp/audit-params/csharp-sdk-branch + echo "$AUDIT_CONF_REPO" > /tmp/audit-params/conformance-repo + echo "$AUDIT_CONF_BRANCH" > /tmp/audit-params/conformance-branch + +post-steps: + - name: Upload audit report + if: always() + uses: actions/upload-artifact@v4 + with: + name: audit-report + path: /tmp/audit-report.md + retention-days: 90 + if-no-files-found: ignore + +on: + schedule: weekly on thursday around 6:30am utc-5 + workflow_dispatch: + inputs: + scope: + description: "Audit scope" + required: true + type: choice + options: + - Conformance + Repo Health + - Repo Health + default: "Conformance + Repo Health" + output: + description: "Where to publish results" + required: true + type: choice + options: + - Create Issue + - Action Summary + default: "Create Issue" + csharp_sdk: + description: "C# SDK (owner/repo:branch)" + required: true + type: string + default: "modelcontextprotocol/csharp-sdk:main" + conformance: + description: "Conformance repo (owner/repo:branch)" + required: true + type: string + default: "modelcontextprotocol/conformance:main" + + # ############################################################### + # Override the COPILOT_GITHUB_TOKEN secret usage for the workflow + # with a randomly-selected token from a pool of secrets. + # + # As soon as organization-level billing is offered for Agentic + # Workflows, this stop-gap approach will be removed. + # + # See: /.github/actions/select-copilot-pat/README.md + # ############################################################### + + # Add the pre-activation step of selecting a random PAT from the supplied secrets + steps: + - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 + name: Checkout the select-copilot-pat action folder + with: + persist-credentials: false + sparse-checkout: .github/actions/select-copilot-pat + sparse-checkout-cone-mode: true + fetch-depth: 1 + + - id: select-copilot-pat + name: Select Copilot token from pool + uses: ./.github/actions/select-copilot-pat + env: + SECRET_0: ${{ secrets.AUDIT_PAT_0 }} + SECRET_1: ${{ secrets.AUDIT_PAT_1 }} + SECRET_2: ${{ secrets.AUDIT_PAT_2 }} + SECRET_3: ${{ secrets.AUDIT_PAT_3 }} + SECRET_4: ${{ secrets.AUDIT_PAT_4 }} + SECRET_5: ${{ secrets.AUDIT_PAT_5 }} + SECRET_6: ${{ secrets.AUDIT_PAT_6 }} + SECRET_7: ${{ secrets.AUDIT_PAT_7 }} + SECRET_8: ${{ secrets.AUDIT_PAT_8 }} + SECRET_9: ${{ secrets.AUDIT_PAT_9 }} + +jobs: + tier-check: + runs-on: ubuntu-latest + timeout-minutes: 30 + outputs: + tier_check_json: ${{ steps.tier-check.outputs.result || steps.tier-check-health.outputs.result }} + scope: ${{ steps.params.outputs.scope }} + output: ${{ steps.params.outputs.output }} + csharp_sdk_repo: ${{ steps.params.outputs.csharp_sdk_repo }} + csharp_sdk_branch: ${{ steps.params.outputs.csharp_sdk_branch }} + conformance_repo: ${{ steps.params.outputs.conformance_repo }} + conformance_branch: ${{ steps.params.outputs.conformance_branch }} + steps: + - name: Resolve parameters + id: params + env: + INPUT_SCOPE: ${{ github.event.inputs.scope || 'Conformance + Repo Health' }} + INPUT_OUTPUT: ${{ github.event.inputs.output || 'Create Issue' }} + INPUT_SDK: ${{ github.event.inputs.csharp_sdk || 'modelcontextprotocol/csharp-sdk:main' }} + INPUT_CONF: ${{ github.event.inputs.conformance || 'modelcontextprotocol/conformance:main' }} + run: | + echo "scope=${INPUT_SCOPE}" >> "$GITHUB_OUTPUT" + echo "output=${INPUT_OUTPUT}" >> "$GITHUB_OUTPUT" + SDK_REPO="${INPUT_SDK%%:*}" + SDK_BRANCH="${INPUT_SDK#*:}" + echo "csharp_sdk_repo=${SDK_REPO}" >> "$GITHUB_OUTPUT" + echo "csharp_sdk_branch=${SDK_BRANCH}" >> "$GITHUB_OUTPUT" + CONF_REPO="${INPUT_CONF%%:*}" + CONF_BRANCH="${INPUT_CONF#*:}" + echo "conformance_repo=${CONF_REPO}" >> "$GITHUB_OUTPUT" + echo "conformance_branch=${CONF_BRANCH}" >> "$GITHUB_OUTPUT" + + - name: Setup .NET SDK + uses: actions/setup-dotnet@v5 + with: + dotnet-version: "10.0" + + - name: Setup Node.js + uses: actions/setup-node@v6 + with: + node-version: "22" + + - name: Clone C# SDK + env: + SDK_BRANCH: ${{ steps.params.outputs.csharp_sdk_branch }} + SDK_REPO: ${{ steps.params.outputs.csharp_sdk_repo }} + run: git clone --depth 1 -b "$SDK_BRANCH" "https://github.com/${SDK_REPO}.git" /tmp/csharp-sdk + + - name: Clone conformance repo + env: + CONF_BRANCH: ${{ steps.params.outputs.conformance_branch }} + CONF_REPO: ${{ steps.params.outputs.conformance_repo }} + run: git clone --depth 1 -b "$CONF_BRANCH" "https://github.com/${CONF_REPO}.git" /tmp/conformance + + - name: Build conformance CLI + run: cd /tmp/conformance && npm ci && npm run build + + - name: Build C# SDK + if: steps.params.outputs.scope == 'Conformance + Repo Health' + run: cd /tmp/csharp-sdk && dotnet build --verbosity quiet + + - name: Start conformance server + if: steps.params.outputs.scope == 'Conformance + Repo Health' + run: | + cd /tmp/csharp-sdk + nohup dotnet run --no-build --project tests/ModelContextProtocol.ConformanceServer --framework net9.0 -- --urls http://localhost:3003 > /tmp/conformance-server.log 2>&1 & + timeout 60 bash -c 'until curl -sf http://localhost:3003/health; do sleep 1; done' + + - name: Run tier-check (Conformance + Repo Health) + id: tier-check + if: steps.params.outputs.scope == 'Conformance + Repo Health' + env: + GITHUB_TOKEN: ${{ github.token }} + run: | + cd /tmp/conformance + REPO=$(cd /tmp/csharp-sdk && git remote get-url origin | sed 's#.*github.com[:/]##; s#\.git$##') + BRANCH=$(cd /tmp/csharp-sdk && git rev-parse --abbrev-ref HEAD) + set +e + RESULT=$(node dist/index.js tier-check \ + --repo "$REPO" \ + --branch "$BRANCH" \ + --conformance-server-url http://localhost:3003 \ + --client-cmd "dotnet run --no-build --project /tmp/csharp-sdk/tests/ModelContextProtocol.ConformanceClient --framework net9.0 -- \$MCP_CONFORMANCE_SCENARIO" \ + --output json 2>/tmp/tier-check-stderr.txt) + EXIT_CODE=$? + set -e + cat /tmp/tier-check-stderr.txt >&2 + if [ $EXIT_CODE -ne 0 ]; then + echo "::error::tier-check CLI failed with exit code $EXIT_CODE" + exit $EXIT_CODE + fi + EOF_MARKER=$(uuidgen) + echo "result<<${EOF_MARKER}" >> "$GITHUB_OUTPUT" + echo "$RESULT" >> "$GITHUB_OUTPUT" + echo "${EOF_MARKER}" >> "$GITHUB_OUTPUT" + + - name: Run tier-check (Repo Health) + id: tier-check-health + if: steps.params.outputs.scope == 'Repo Health' + env: + GITHUB_TOKEN: ${{ github.token }} + run: | + cd /tmp/conformance + REPO=$(cd /tmp/csharp-sdk && git remote get-url origin | sed 's#.*github.com[:/]##; s#\.git$##') + BRANCH=$(cd /tmp/csharp-sdk && git rev-parse --abbrev-ref HEAD) + set +e + RESULT=$(node dist/index.js tier-check \ + --repo "$REPO" \ + --branch "$BRANCH" \ + --skip-conformance \ + --output json 2>/tmp/tier-check-stderr.txt) + EXIT_CODE=$? + set -e + cat /tmp/tier-check-stderr.txt >&2 + if [ $EXIT_CODE -ne 0 ]; then + echo "::error::tier-check CLI failed with exit code $EXIT_CODE" + exit $EXIT_CODE + fi + EOF_MARKER=$(uuidgen) + echo "result<<${EOF_MARKER}" >> "$GITHUB_OUTPUT" + echo "$RESULT" >> "$GITHUB_OUTPUT" + echo "${EOF_MARKER}" >> "$GITHUB_OUTPUT" + + pre-activation: + outputs: + copilot_pat_number: ${{ steps.select-copilot-pat.outputs.copilot_pat_number }} + +engine: + id: copilot + env: + # We cannot use line breaks in this expression as it leads to a syntax error in the compiled workflow + # If none of the `AUDIT_PAT_#` secrets were selected, then the default COPILOT_GITHUB_TOKEN is used + COPILOT_GITHUB_TOKEN: ${{ case(needs.pre_activation.outputs.copilot_pat_number == '0', secrets.AUDIT_PAT_0, needs.pre_activation.outputs.copilot_pat_number == '1', secrets.AUDIT_PAT_1, needs.pre_activation.outputs.copilot_pat_number == '2', secrets.AUDIT_PAT_2, needs.pre_activation.outputs.copilot_pat_number == '3', secrets.AUDIT_PAT_3, needs.pre_activation.outputs.copilot_pat_number == '4', secrets.AUDIT_PAT_4, needs.pre_activation.outputs.copilot_pat_number == '5', secrets.AUDIT_PAT_5, needs.pre_activation.outputs.copilot_pat_number == '6', secrets.AUDIT_PAT_6, needs.pre_activation.outputs.copilot_pat_number == '7', secrets.AUDIT_PAT_7, needs.pre_activation.outputs.copilot_pat_number == '8', secrets.AUDIT_PAT_8, needs.pre_activation.outputs.copilot_pat_number == '9', secrets.AUDIT_PAT_9, secrets.COPILOT_GITHUB_TOKEN) }} +--- + +# SDK Tier Audit + +Perform a tier audit of the C# MCP SDK. The deterministic tier-check scorecard has already been computed by the `tier-check` job. Your job is to perform the AI-assisted evaluation (documentation coverage, policy review), apply tier logic, generate the report, and publish the results. + +## Inputs + +All parameters are written to `/tmp/audit-params/` by a pre-agent step. Read them: + +```bash +SCOPE=$(cat /tmp/audit-params/scope) +OUTPUT_MODE=$(cat /tmp/audit-params/output) +SDK_REPO=$(cat /tmp/audit-params/csharp-sdk-repo) +SDK_BRANCH=$(cat /tmp/audit-params/csharp-sdk-branch) +CONF_REPO=$(cat /tmp/audit-params/conformance-repo) +CONF_BRANCH=$(cat /tmp/audit-params/conformance-branch) +``` + +The tier-check scorecard is at `/tmp/tier-check-scorecard.json`. + +## Tier-Check Scorecard (pre-computed) + +The deterministic tier-check scorecard has been pre-computed by the `tier-check` job and written to `/tmp/tier-check-scorecard.json`. Read this file to get the full JSON scorecard. **Do not re-run the tier-check CLI.** + +```bash +cat /tmp/tier-check-scorecard.json +``` + +## Step 1: Setup + +Read the parameters from `/tmp/audit-params/` and clone both repositories for the AI-assisted evaluations. Use shallow clones. + +```bash +SDK_REPO=$(cat /tmp/audit-params/csharp-sdk-repo) +SDK_BRANCH=$(cat /tmp/audit-params/csharp-sdk-branch) +CONF_REPO=$(cat /tmp/audit-params/conformance-repo) +CONF_BRANCH=$(cat /tmp/audit-params/conformance-branch) + +git clone --depth 1 -b "$SDK_BRANCH" "https://github.com/${SDK_REPO}.git" /tmp/csharp-sdk +git clone --depth 1 -b "$CONF_BRANCH" "https://github.com/${CONF_REPO}.git" /tmp/conformance +``` + +You do NOT need to build either repo or start any servers — the conformance tests have already run. + +## Step 2: AI-Assisted Evaluation + +Read the **"Any Other AI Coding Agent"** section from `/tmp/conformance/.claude/skills/mcp-sdk-tier-audit/README.md`. Follow steps **2 through 5** only (skip step 1 — the CLI has already run): + +2. **Evaluate documentation coverage** using the prompt in `references/docs-coverage-prompt.md` +3. **Evaluate policies** using the prompt in `references/policy-evaluation-prompt.md` — pass the `policy_signals` section from the tier-check JSON above +4. **Apply tier logic** using the thresholds in `references/tier-requirements.md` — combine the scorecard above with your evaluation results +5. **Generate report** using the template in `references/report-template.md` + +The SDK checkout at `/tmp/csharp-sdk` is the local path for documentation and policy evaluations. + +Write the assessment and remediation reports to `/tmp/conformance/results/`: +- `results/-csharp-sdk-assessment.md` +- `results/-csharp-sdk-remediation.md` + +## Step 3: Compose the Audit Report + +After the evaluation completes, compose the full audit report as a single markdown file at `/tmp/audit-report.md`. This file is used for both issue creation and the action summary artifact. + +### Report structure + +The report must contain these sections in order: + +1. **Executive summary** — Use bullet points (not a paragraph). Include: + - The tier classification result (e.g., "**Tier 1** — all requirements met") + - Key conformance results (server and client pass rates) + - Repo health highlights (triage compliance, P0 status, stable release version) + - Documentation coverage (N/48 features) + - Any blockers for the next tier up + +2. **Full assessment** — Include the complete contents of `results/-csharp-sdk-assessment.md` verbatim. Do not modify, summarize, or reformat. + +3. **Remediation guide** — Include the complete contents of `results/-csharp-sdk-remediation.md` verbatim. Do not modify, summarize, or reformat. + +Use horizontal rules (`---`) to separate sections. + +### Write the report + +Write the composed report to `/tmp/audit-report.md`. This file will be uploaded as a workflow artifact by a post-execution step. + +### Write the action summary + +Always write the full audit report to the GitHub Step Summary so it appears on the workflow run's summary page: + +```bash +cat /tmp/audit-report.md >> "$GITHUB_STEP_SUMMARY" +``` + +## Step 4: Publish Results + +Read the output mode: `cat /tmp/audit-params/output` + +### If output mode is "Create Issue" + +Create a GitHub issue using the `create-issue` safe output. + +**Issue title** — Read the scope from `cat /tmp/audit-params/scope`. The dynamic part (after the `[C# SDK Tier Audit] ` prefix): + +- **"Conformance + Repo Health" scope**: ` - Tier ` +- **"Repo Health" scope**: ` - Tier (Repo Health)` + +Where `` is today's date and `` is the computed tier number (1, 2, or 3). + +**Issue body** — Use the contents of `/tmp/audit-report.md` as the issue body. + +### If output mode is "Action Summary" + +Do NOT create an issue. The report is already written to `$GITHUB_STEP_SUMMARY` and will be uploaded as an artifact. No further action is needed. + +## Failure Handling + +If the evaluation fails at any step, or if the audit does not produce assessment/remediation results: + +1. **Do NOT create an issue.** Do not use the `create-issue` safe output. +2. **Write a GitHub Step Summary** explaining what happened: + +```bash +echo "## Tier Audit: No Results" >> "$GITHUB_STEP_SUMMARY" +echo "" >> "$GITHUB_STEP_SUMMARY" +echo "The tier audit did not produce results. Reason: " >> "$GITHUB_STEP_SUMMARY" +echo "" >> "$GITHUB_STEP_SUMMARY" +echo "No issue was filed for this run." >> "$GITHUB_STEP_SUMMARY" +``` + +3. The `noop` safe output will apply automatically when no `create-issue` output is produced. + +--- + +## PAT Pool Setup + +> **Note:** This section is for **repository maintainers** setting up PAT pool secrets. It is not consumed by the agent. + +This workflow uses its own PAT pool (`AUDIT_PAT_0` through `AUDIT_PAT_9`) separate from the standard `COPILOT_PAT` pool. This allows audit-specific PATs with the minimum required scopes. + +### Required PAT configuration + +| Setting | Value | +|---------|-------| +| **Resource owner** | Your **user account** (not an organization) | +| **Repository access** | **Public Repositories (read-only)** — or, if either the C# SDK or conformance repos are private, add those repos explicitly | +| **Copilot Requests** | **Read** | + +The PAT needs **Copilot Requests (Read)** for the Copilot engine. It also needs read access to the C# SDK and conformance repositories used by this workflow (configured via the `csharp_sdk_repo` and `conformance_repo` inputs, defaulting to `modelcontextprotocol/csharp-sdk` and `modelcontextprotocol/conformance`). If those repositories are public, selecting **Public Repositories (read-only)** is sufficient. If either repository is private, add it explicitly under **Repository access** in the PAT configuration. + +### Create a PAT + +[Use this link to prefill the PAT creation form][create-pat] with the required settings: + +1. Set **Resource owner** to your **user account** (not an organization). +2. Ensure **Copilot Requests (Read)** is the only permission granted. +3. Set **Repository access** to **Public Repositories (read-only)** — or add the C# SDK and conformance repos explicitly if they are not public. +4. The **Token Name** does not need to match the secret name — use something recognizable like `MCP C# SDK: AUDIT_PAT_0`. + +### Add the PAT as a repository secret + +Add your PAT as a repository secret named `AUDIT_PAT_0` through `AUDIT_PAT_9` in the repository's **Settings > Secrets and variables > Actions** page: + +```bash +gh secret set "AUDIT_PAT_0" --body "" --repo +``` + +### Renewal + +Set a recurring reminder to regenerate and update your PAT before it expires. PATs can be renewed on the same day each cycle. + +[create-pat]: https://github.com/settings/personal-access-tokens/new?name=MCP+C%23+SDK%3A+AUDIT_PAT_%23&description=PAT+for+the+MCP+C%23+SDK+tier+audit+agentic+workflow.+Must+be+configured+with+Copilot+Requests+(Read)+permission+and+read+access+to+the+conformance+and+csharp-sdk+repos+(Public+Repositories+if+both+are+public).+User+account+as+resource+owner.&user_copilot_requests=read From c9d0d90947be53fa8bed7ed99f21f5dc96d07361 Mon Sep 17 00:00:00 2001 From: Jeff Handley Date: Sat, 4 Apr 2026 21:52:12 -0700 Subject: [PATCH 5/7] Add output mode, artifact upload, and condensed inputs - Add 'output' input: 'Create Issue' (default) or 'Action Summary' When 'Action Summary', the report is written to the step summary and uploaded as an artifact but no issue is created. - Condense repo/branch inputs into owner/repo:branch format: csharp_sdk: 'modelcontextprotocol/csharp-sdk:main' conformance: 'modelcontextprotocol/conformance:main' Parsed with bash parameter expansion (${var%%:*} / ${var#*:}) - Add post-steps to upload /tmp/audit-report.md as an artifact (90-day retention, ignore if missing) - Executive summary now uses bullet points instead of a paragraph - Audit report always written to $GITHUB_STEP_SUMMARY regardless of output mode, so the summary page always shows results - Agent writes a single /tmp/audit-report.md combining executive summary + assessment + remediation, used for both issue body and action summary Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/aw/actions-lock.json | 23 +---- .github/workflows/sdk-tier-audit.lock.yml | 107 ++++++++++++---------- .github/workflows/sdk-tier-audit.md | 58 +++--------- 3 files changed, 72 insertions(+), 116 deletions(-) diff --git a/.github/aw/actions-lock.json b/.github/aw/actions-lock.json index 138b2808a..503987c4f 100644 --- a/.github/aw/actions-lock.json +++ b/.github/aw/actions-lock.json @@ -1,30 +1,15 @@ { "entries": { - "actions/download-artifact@v8.0.1": { - "repo": "actions/download-artifact", - "version": "v8.0.1", - "sha": "3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c" - }, - "actions/github-script@v8": { - "repo": "actions/github-script", - "version": "v8", - "sha": "ed597411d8f924073f98dfc5c65a23a2325f34cd" - }, - "actions/setup-dotnet@v5.2.0": { + "actions/setup-dotnet@v5": { "repo": "actions/setup-dotnet", - "version": "v5.2.0", + "version": "v5", "sha": "c2fa09f4bde5ebb9d1777cf28262a3eb3db3ced7" }, - "actions/setup-node@v6.3.0": { + "actions/setup-node@v6": { "repo": "actions/setup-node", - "version": "v6.3.0", + "version": "v6", "sha": "53b83947a5a98c8d113130e565377fae1a50d02f" }, - "actions/upload-artifact@v7": { - "repo": "actions/upload-artifact", - "version": "v7", - "sha": "bbbca2ddaa5d8feaa63e36b76fdaad77386f024f" - }, "github/gh-aw-actions/setup@v0.66.1": { "repo": "github/gh-aw-actions/setup", "version": "v0.66.1", diff --git a/.github/workflows/sdk-tier-audit.lock.yml b/.github/workflows/sdk-tier-audit.lock.yml index 37976ecd1..2dc53657c 100644 --- a/.github/workflows/sdk-tier-audit.lock.yml +++ b/.github/workflows/sdk-tier-audit.lock.yml @@ -22,7 +22,7 @@ # # SDK Tier Audit # -# gh-aw-metadata: {"schema_version":"v3","frontmatter_hash":"70bf24be23625c84fbdbec90dff9f6066a02da072946c9525fedc95b49a7de92","compiler_version":"v0.66.1","strict":true,"agent_id":"copilot"} +# gh-aw-metadata: {"schema_version":"v3","frontmatter_hash":"fca932f26ce84a81e3648efc77465df23784aafd9cb2bbd62e841560d7a3405d","compiler_version":"v0.66.1","strict":true,"agent_id":"copilot"} name: "SDK Tier Audit" "on": @@ -93,12 +93,11 @@ concurrency: run-name: "SDK Tier Audit" -env: - FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true" - jobs: activation: - needs: pre_activation + needs: + - pre_activation + - tier-check if: > needs.pre_activation.outputs.activated == 'true' && (github.repository_owner == 'modelcontextprotocol' || github.event_name == 'workflow_dispatch') @@ -181,6 +180,12 @@ jobs: env: GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt GH_AW_SAFE_OUTPUTS: ${{ runner.temp }}/gh-aw/safeoutputs/outputs.jsonl + GH_AW_EXPR_0DFFBBBC: ${{ needs.tier-check.outputs.csharp_sdk_repo }} + GH_AW_EXPR_3F2EBE85: ${{ needs.tier-check.outputs.csharp_sdk_branch }} + GH_AW_EXPR_771B4966: ${{ needs.tier-check.outputs.scope }} + GH_AW_EXPR_9CED0D62: ${{ needs.tier-check.outputs.conformance_branch }} + GH_AW_EXPR_BDA3F902: ${{ needs.tier-check.outputs.output }} + GH_AW_EXPR_DC7F6CEA: ${{ needs.tier-check.outputs.conformance_repo }} GH_AW_GITHUB_ACTOR: ${{ github.actor }} GH_AW_GITHUB_EVENT_COMMENT_ID: ${{ github.event.comment.id }} GH_AW_GITHUB_EVENT_DISCUSSION_NUMBER: ${{ github.event.discussion.number }} @@ -193,14 +198,14 @@ jobs: run: | bash ${RUNNER_TEMP}/gh-aw/actions/create_prompt_first.sh { - cat << 'GH_AW_PROMPT_cffe1da634462c6b_EOF' + cat << 'GH_AW_PROMPT_bc128154aed53b51_EOF' - GH_AW_PROMPT_cffe1da634462c6b_EOF + GH_AW_PROMPT_bc128154aed53b51_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/xpia.md" cat "${RUNNER_TEMP}/gh-aw/prompts/temp_folder_prompt.md" cat "${RUNNER_TEMP}/gh-aw/prompts/markdown.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_prompt.md" - cat << 'GH_AW_PROMPT_cffe1da634462c6b_EOF' + cat << 'GH_AW_PROMPT_bc128154aed53b51_EOF' Tools: create_issue, missing_tool, missing_data, noop @@ -232,17 +237,23 @@ jobs: {{/if}} - GH_AW_PROMPT_cffe1da634462c6b_EOF + GH_AW_PROMPT_bc128154aed53b51_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/github_mcp_tools_with_safeoutputs_prompt.md" - cat << 'GH_AW_PROMPT_cffe1da634462c6b_EOF' + cat << 'GH_AW_PROMPT_bc128154aed53b51_EOF' {{#runtime-import .github/workflows/sdk-tier-audit.md}} - GH_AW_PROMPT_cffe1da634462c6b_EOF + GH_AW_PROMPT_bc128154aed53b51_EOF } > "$GH_AW_PROMPT" - name: Interpolate variables and render templates uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 env: GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt + GH_AW_EXPR_9CED0D62: ${{ needs.tier-check.outputs.conformance_branch }} + GH_AW_EXPR_DC7F6CEA: ${{ needs.tier-check.outputs.conformance_repo }} + GH_AW_EXPR_3F2EBE85: ${{ needs.tier-check.outputs.csharp_sdk_branch }} + GH_AW_EXPR_0DFFBBBC: ${{ needs.tier-check.outputs.csharp_sdk_repo }} + GH_AW_EXPR_BDA3F902: ${{ needs.tier-check.outputs.output }} + GH_AW_EXPR_771B4966: ${{ needs.tier-check.outputs.scope }} with: script: | const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); @@ -253,6 +264,12 @@ jobs: uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 env: GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt + GH_AW_EXPR_0DFFBBBC: ${{ needs.tier-check.outputs.csharp_sdk_repo }} + GH_AW_EXPR_3F2EBE85: ${{ needs.tier-check.outputs.csharp_sdk_branch }} + GH_AW_EXPR_771B4966: ${{ needs.tier-check.outputs.scope }} + GH_AW_EXPR_9CED0D62: ${{ needs.tier-check.outputs.conformance_branch }} + GH_AW_EXPR_BDA3F902: ${{ needs.tier-check.outputs.output }} + GH_AW_EXPR_DC7F6CEA: ${{ needs.tier-check.outputs.conformance_repo }} GH_AW_GITHUB_ACTOR: ${{ github.actor }} GH_AW_GITHUB_EVENT_COMMENT_ID: ${{ github.event.comment.id }} GH_AW_GITHUB_EVENT_DISCUSSION_NUMBER: ${{ github.event.discussion.number }} @@ -273,6 +290,12 @@ jobs: return await substitutePlaceholders({ file: process.env.GH_AW_PROMPT, substitutions: { + GH_AW_EXPR_0DFFBBBC: process.env.GH_AW_EXPR_0DFFBBBC, + GH_AW_EXPR_3F2EBE85: process.env.GH_AW_EXPR_3F2EBE85, + GH_AW_EXPR_771B4966: process.env.GH_AW_EXPR_771B4966, + GH_AW_EXPR_9CED0D62: process.env.GH_AW_EXPR_9CED0D62, + GH_AW_EXPR_BDA3F902: process.env.GH_AW_EXPR_BDA3F902, + GH_AW_EXPR_DC7F6CEA: process.env.GH_AW_EXPR_DC7F6CEA, GH_AW_GITHUB_ACTOR: process.env.GH_AW_GITHUB_ACTOR, GH_AW_GITHUB_EVENT_COMMENT_ID: process.env.GH_AW_GITHUB_EVENT_COMMENT_ID, GH_AW_GITHUB_EVENT_DISCUSSION_NUMBER: process.env.GH_AW_GITHUB_EVENT_DISCUSSION_NUMBER, @@ -312,7 +335,6 @@ jobs: permissions: contents: read issues: read - pull-requests: read concurrency: group: "gh-aw-copilot-${{ github.workflow }}" env: @@ -361,23 +383,9 @@ jobs: env: GH_TOKEN: ${{ github.token }} - env: - AUDIT_CONF_BRANCH: ${{ needs.tier-check.outputs.conformance_branch }} - AUDIT_CONF_REPO: ${{ needs.tier-check.outputs.conformance_repo }} - AUDIT_OUTPUT: ${{ needs.tier-check.outputs.output }} - AUDIT_SCOPE: ${{ needs.tier-check.outputs.scope }} - AUDIT_SDK_BRANCH: ${{ needs.tier-check.outputs.csharp_sdk_branch }} - AUDIT_SDK_REPO: ${{ needs.tier-check.outputs.csharp_sdk_repo }} TIER_CHECK_JSON: ${{ needs.tier-check.outputs.tier_check_json }} - name: Write tier-check outputs to files - run: |- - echo "$TIER_CHECK_JSON" > /tmp/tier-check-scorecard.json - mkdir -p /tmp/audit-params - echo "$AUDIT_SCOPE" > /tmp/audit-params/scope - echo "$AUDIT_OUTPUT" > /tmp/audit-params/output - echo "$AUDIT_SDK_REPO" > /tmp/audit-params/csharp-sdk-repo - echo "$AUDIT_SDK_BRANCH" > /tmp/audit-params/csharp-sdk-branch - echo "$AUDIT_CONF_REPO" > /tmp/audit-params/conformance-repo - echo "$AUDIT_CONF_BRANCH" > /tmp/audit-params/conformance-branch + name: Write tier-check scorecard to file + run: echo "$TIER_CHECK_JSON" > /tmp/tier-check-scorecard.json - name: Configure Git credentials env: @@ -411,13 +419,16 @@ jobs: GH_HOST: github.com - name: Install AWF binary run: bash ${RUNNER_TEMP}/gh-aw/actions/install_awf_binary.sh v0.25.13 - - name: Parse integrity filter lists - id: parse-guard-vars + - name: Determine automatic lockdown mode for GitHub MCP Server + id: determine-automatic-lockdown + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 env: - GH_AW_BLOCKED_USERS_VAR: ${{ vars.GH_AW_GITHUB_BLOCKED_USERS || '' }} - GH_AW_TRUSTED_USERS_VAR: ${{ vars.GH_AW_GITHUB_TRUSTED_USERS || '' }} - GH_AW_APPROVAL_LABELS_VAR: ${{ vars.GH_AW_GITHUB_APPROVAL_LABELS || '' }} - run: bash ${RUNNER_TEMP}/gh-aw/actions/parse_guard_list.sh + GH_AW_GITHUB_TOKEN: ${{ secrets.GH_AW_GITHUB_TOKEN }} + GH_AW_GITHUB_MCP_SERVER_TOKEN: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN }} + with: + script: | + const determineAutomaticLockdown = require('${{ runner.temp }}/gh-aw/actions/determine_automatic_lockdown.cjs'); + await determineAutomaticLockdown(github, context, core); - name: Download container images run: bash ${RUNNER_TEMP}/gh-aw/actions/download_docker_images.sh ghcr.io/github/gh-aw-firewall/agent:0.25.13 ghcr.io/github/gh-aw-firewall/api-proxy:0.25.13 ghcr.io/github/gh-aw-firewall/squid:0.25.13 ghcr.io/github/gh-aw-mcpg:v0.2.12 ghcr.io/github/github-mcp-server:v0.32.0 node:lts-alpine - name: Write Safe Outputs Config @@ -425,12 +436,12 @@ jobs: mkdir -p ${RUNNER_TEMP}/gh-aw/safeoutputs mkdir -p /tmp/gh-aw/safeoutputs mkdir -p /tmp/gh-aw/mcp-logs/safeoutputs - cat > ${RUNNER_TEMP}/gh-aw/safeoutputs/config.json << 'GH_AW_SAFE_OUTPUTS_CONFIG_c6fd50f1e250c752_EOF' + cat > ${RUNNER_TEMP}/gh-aw/safeoutputs/config.json << 'GH_AW_SAFE_OUTPUTS_CONFIG_125bf17eed223b0c_EOF' {"create_issue":{"close_older_issues":true,"labels":["automation"],"max":1,"title_prefix":"[C# SDK Tier Audit] "},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"}} - GH_AW_SAFE_OUTPUTS_CONFIG_c6fd50f1e250c752_EOF + GH_AW_SAFE_OUTPUTS_CONFIG_125bf17eed223b0c_EOF - name: Write Safe Outputs Tools run: | - cat > ${RUNNER_TEMP}/gh-aw/safeoutputs/tools_meta.json << 'GH_AW_SAFE_OUTPUTS_TOOLS_META_5ec9df3bfed4f693_EOF' + cat > ${RUNNER_TEMP}/gh-aw/safeoutputs/tools_meta.json << 'GH_AW_SAFE_OUTPUTS_TOOLS_META_2a3a0c9c86f0eae4_EOF' { "description_suffixes": { "create_issue": " CONSTRAINTS: Maximum 1 issue(s) can be created. Title will be prefixed with \"[C# SDK Tier Audit] \". Labels [\"automation\"] will be automatically added." @@ -438,8 +449,8 @@ jobs: "repo_params": {}, "dynamic_tools": [] } - GH_AW_SAFE_OUTPUTS_TOOLS_META_5ec9df3bfed4f693_EOF - cat > ${RUNNER_TEMP}/gh-aw/safeoutputs/validation.json << 'GH_AW_SAFE_OUTPUTS_VALIDATION_2ba4f188e5e9c93d_EOF' + GH_AW_SAFE_OUTPUTS_TOOLS_META_2a3a0c9c86f0eae4_EOF + cat > ${RUNNER_TEMP}/gh-aw/safeoutputs/validation.json << 'GH_AW_SAFE_OUTPUTS_VALIDATION_dc6089ad4494a421_EOF' { "create_issue": { "defaultMax": 1, @@ -532,7 +543,7 @@ jobs: } } } - GH_AW_SAFE_OUTPUTS_VALIDATION_2ba4f188e5e9c93d_EOF + GH_AW_SAFE_OUTPUTS_VALIDATION_dc6089ad4494a421_EOF node ${RUNNER_TEMP}/gh-aw/actions/generate_safe_outputs_tools.cjs - name: Generate Safe Outputs MCP Server Config id: safe-outputs-config @@ -580,6 +591,8 @@ jobs: GH_AW_SAFE_OUTPUTS: ${{ steps.set-runtime-paths.outputs.GH_AW_SAFE_OUTPUTS }} GH_AW_SAFE_OUTPUTS_API_KEY: ${{ steps.safe-outputs-start.outputs.api_key }} GH_AW_SAFE_OUTPUTS_PORT: ${{ steps.safe-outputs-start.outputs.port }} + GITHUB_MCP_GUARD_MIN_INTEGRITY: ${{ steps.determine-automatic-lockdown.outputs.min_integrity }} + GITHUB_MCP_GUARD_REPOS: ${{ steps.determine-automatic-lockdown.outputs.repos }} GITHUB_MCP_SERVER_TOKEN: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN || secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} run: | set -eo pipefail @@ -600,7 +613,7 @@ jobs: export MCP_GATEWAY_DOCKER_COMMAND='docker run -i --rm --network host -v /var/run/docker.sock:/var/run/docker.sock -e MCP_GATEWAY_PORT -e MCP_GATEWAY_DOMAIN -e MCP_GATEWAY_API_KEY -e MCP_GATEWAY_PAYLOAD_DIR -e MCP_GATEWAY_PAYLOAD_SIZE_THRESHOLD -e DEBUG -e MCP_GATEWAY_LOG_DIR -e GH_AW_MCP_LOG_DIR -e GH_AW_SAFE_OUTPUTS -e GH_AW_SAFE_OUTPUTS_CONFIG_PATH -e GH_AW_SAFE_OUTPUTS_TOOLS_PATH -e GH_AW_ASSETS_BRANCH -e GH_AW_ASSETS_MAX_SIZE_KB -e GH_AW_ASSETS_ALLOWED_EXTS -e DEFAULT_BRANCH -e GITHUB_MCP_SERVER_TOKEN -e GITHUB_MCP_GUARD_MIN_INTEGRITY -e GITHUB_MCP_GUARD_REPOS -e GITHUB_REPOSITORY -e GITHUB_SERVER_URL -e GITHUB_SHA -e GITHUB_WORKSPACE -e GITHUB_TOKEN -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e GITHUB_JOB -e GITHUB_ACTION -e GITHUB_EVENT_NAME -e GITHUB_EVENT_PATH -e GITHUB_ACTOR -e GITHUB_ACTOR_ID -e GITHUB_TRIGGERING_ACTOR -e GITHUB_WORKFLOW -e GITHUB_WORKFLOW_REF -e GITHUB_WORKFLOW_SHA -e GITHUB_REF -e GITHUB_REF_NAME -e GITHUB_REF_TYPE -e GITHUB_HEAD_REF -e GITHUB_BASE_REF -e GH_AW_SAFE_OUTPUTS_PORT -e GH_AW_SAFE_OUTPUTS_API_KEY -v /tmp/gh-aw/mcp-payloads:/tmp/gh-aw/mcp-payloads:rw -v /opt:/opt:ro -v /tmp:/tmp:rw -v '"${GITHUB_WORKSPACE}"':'"${GITHUB_WORKSPACE}"':rw ghcr.io/github/gh-aw-mcpg:v0.2.12' mkdir -p /home/runner/.copilot - cat << GH_AW_MCP_CONFIG_567e7781e5d7a545_EOF | bash ${RUNNER_TEMP}/gh-aw/actions/start_mcp_gateway.sh + cat << GH_AW_MCP_CONFIG_51b4796376e01c6c_EOF | bash ${RUNNER_TEMP}/gh-aw/actions/start_mcp_gateway.sh { "mcpServers": { "github": { @@ -614,11 +627,8 @@ jobs: }, "guard-policies": { "allow-only": { - "approval-labels": ${{ steps.parse-guard-vars.outputs.approval_labels }}, - "blocked-users": ${{ steps.parse-guard-vars.outputs.blocked_users }}, - "min-integrity": "approved", - "repos": "all", - "trusted-users": ${{ steps.parse-guard-vars.outputs.trusted_users }} + "min-integrity": "$GITHUB_MCP_GUARD_MIN_INTEGRITY", + "repos": "$GITHUB_MCP_GUARD_REPOS" } } }, @@ -644,7 +654,7 @@ jobs: "payloadDir": "${MCP_GATEWAY_PAYLOAD_DIR}" } } - GH_AW_MCP_CONFIG_567e7781e5d7a545_EOF + GH_AW_MCP_CONFIG_51b4796376e01c6c_EOF - name: Download activation artifact uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 with: @@ -831,8 +841,6 @@ jobs: /tmp/gh-aw/sandbox/agent/logs/ /tmp/gh-aw/redacted-urls.log /tmp/gh-aw/mcp-logs/ - /tmp/gh-aw/proxy-logs/ - !/tmp/gh-aw/proxy-logs/proxy-tls/ /tmp/gh-aw/agent_usage.json /tmp/gh-aw/agent-stdio.log /tmp/gh-aw/agent/ @@ -1218,7 +1226,6 @@ jobs: if-no-files-found: ignore tier-check: - needs: activation runs-on: ubuntu-latest timeout-minutes: 30 outputs: diff --git a/.github/workflows/sdk-tier-audit.md b/.github/workflows/sdk-tier-audit.md index 8109e2517..dac4c8763 100644 --- a/.github/workflows/sdk-tier-audit.md +++ b/.github/workflows/sdk-tier-audit.md @@ -4,24 +4,15 @@ description: "SDK Tier Audit" permissions: contents: read issues: read - pull-requests: read safe-outputs: create-issue: title-prefix: "[C# SDK Tier Audit] " labels: [automation] close-older-issues: true - max: 1 - -tools: - github: - min-integrity: approved if: github.repository_owner == 'modelcontextprotocol' || github.event_name == 'workflow_dispatch' -env: - FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true" - concurrency: group: tier-audit-${{ github.event.inputs.scope || 'Conformance + Repo Health' }} cancel-in-progress: true @@ -42,24 +33,10 @@ network: timeout-minutes: 120 steps: - - name: Write tier-check outputs to files + - name: Write tier-check scorecard to file env: TIER_CHECK_JSON: ${{ needs.tier-check.outputs.tier_check_json }} - AUDIT_SCOPE: ${{ needs.tier-check.outputs.scope }} - AUDIT_OUTPUT: ${{ needs.tier-check.outputs.output }} - AUDIT_SDK_REPO: ${{ needs.tier-check.outputs.csharp_sdk_repo }} - AUDIT_SDK_BRANCH: ${{ needs.tier-check.outputs.csharp_sdk_branch }} - AUDIT_CONF_REPO: ${{ needs.tier-check.outputs.conformance_repo }} - AUDIT_CONF_BRANCH: ${{ needs.tier-check.outputs.conformance_branch }} - run: | - echo "$TIER_CHECK_JSON" > /tmp/tier-check-scorecard.json - mkdir -p /tmp/audit-params - echo "$AUDIT_SCOPE" > /tmp/audit-params/scope - echo "$AUDIT_OUTPUT" > /tmp/audit-params/output - echo "$AUDIT_SDK_REPO" > /tmp/audit-params/csharp-sdk-repo - echo "$AUDIT_SDK_BRANCH" > /tmp/audit-params/csharp-sdk-branch - echo "$AUDIT_CONF_REPO" > /tmp/audit-params/conformance-repo - echo "$AUDIT_CONF_BRANCH" > /tmp/audit-params/conformance-branch + run: echo "$TIER_CHECK_JSON" > /tmp/tier-check-scorecard.json post-steps: - name: Upload audit report @@ -278,18 +255,12 @@ Perform a tier audit of the C# MCP SDK. The deterministic tier-check scorecard h ## Inputs -All parameters are written to `/tmp/audit-params/` by a pre-agent step. Read them: +- **Scope**: ${{ needs.tier-check.outputs.scope }} +- **Output mode**: ${{ needs.tier-check.outputs.output }} +- **C# SDK**: ${{ needs.tier-check.outputs.csharp_sdk_repo }} (branch: ${{ needs.tier-check.outputs.csharp_sdk_branch }}) +- **Conformance**: ${{ needs.tier-check.outputs.conformance_repo }} (branch: ${{ needs.tier-check.outputs.conformance_branch }}) -```bash -SCOPE=$(cat /tmp/audit-params/scope) -OUTPUT_MODE=$(cat /tmp/audit-params/output) -SDK_REPO=$(cat /tmp/audit-params/csharp-sdk-repo) -SDK_BRANCH=$(cat /tmp/audit-params/csharp-sdk-branch) -CONF_REPO=$(cat /tmp/audit-params/conformance-repo) -CONF_BRANCH=$(cat /tmp/audit-params/conformance-branch) -``` - -The tier-check scorecard is at `/tmp/tier-check-scorecard.json`. +The scorecard is at `/tmp/tier-check-scorecard.json` (written by a pre-agent step from the `tier-check` job output). ## Tier-Check Scorecard (pre-computed) @@ -301,16 +272,11 @@ cat /tmp/tier-check-scorecard.json ## Step 1: Setup -Read the parameters from `/tmp/audit-params/` and clone both repositories for the AI-assisted evaluations. Use shallow clones. +Clone both repositories for the AI-assisted evaluations. Use shallow clones. ```bash -SDK_REPO=$(cat /tmp/audit-params/csharp-sdk-repo) -SDK_BRANCH=$(cat /tmp/audit-params/csharp-sdk-branch) -CONF_REPO=$(cat /tmp/audit-params/conformance-repo) -CONF_BRANCH=$(cat /tmp/audit-params/conformance-branch) - -git clone --depth 1 -b "$SDK_BRANCH" "https://github.com/${SDK_REPO}.git" /tmp/csharp-sdk -git clone --depth 1 -b "$CONF_BRANCH" "https://github.com/${CONF_REPO}.git" /tmp/conformance +git clone --depth 1 -b https://github.com/.git /tmp/csharp-sdk +git clone --depth 1 -b https://github.com/.git /tmp/conformance ``` You do NOT need to build either repo or start any servers — the conformance tests have already run. @@ -365,13 +331,11 @@ cat /tmp/audit-report.md >> "$GITHUB_STEP_SUMMARY" ## Step 4: Publish Results -Read the output mode: `cat /tmp/audit-params/output` - ### If output mode is "Create Issue" Create a GitHub issue using the `create-issue` safe output. -**Issue title** — Read the scope from `cat /tmp/audit-params/scope`. The dynamic part (after the `[C# SDK Tier Audit] ` prefix): +**Issue title** — The dynamic part (after the `[C# SDK Tier Audit] ` prefix): - **"Conformance + Repo Health" scope**: ` - Tier ` - **"Repo Health" scope**: ` - Tier (Repo Health)` From 3ca689b7e053a9724ca9ad949df1a9b0ddbb5a19 Mon Sep 17 00:00:00 2001 From: Jeff Handley Date: Sat, 4 Apr 2026 22:22:32 -0700 Subject: [PATCH 6/7] Add guard policy, Node 24 opt-in, and safe output max - Add tools.github.min-integrity: approved for content guard policy - Add pull-requests: read permission (required by default toolsets) - Add FORCE_JAVASCRIPT_ACTIONS_TO_NODE24=true env var to opt into Node 24 early and suppress the Node 20 deprecation warning - Add max: 1 to create-issue safe output for explicit limit Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/workflows/sdk-tier-audit.lock.yml | 113 ++++++++++------------ .github/workflows/sdk-tier-audit.md | 11 +++ 2 files changed, 64 insertions(+), 60 deletions(-) diff --git a/.github/workflows/sdk-tier-audit.lock.yml b/.github/workflows/sdk-tier-audit.lock.yml index 2dc53657c..d14f2613a 100644 --- a/.github/workflows/sdk-tier-audit.lock.yml +++ b/.github/workflows/sdk-tier-audit.lock.yml @@ -22,7 +22,7 @@ # # SDK Tier Audit # -# gh-aw-metadata: {"schema_version":"v3","frontmatter_hash":"fca932f26ce84a81e3648efc77465df23784aafd9cb2bbd62e841560d7a3405d","compiler_version":"v0.66.1","strict":true,"agent_id":"copilot"} +# gh-aw-metadata: {"schema_version":"v3","frontmatter_hash":"e5e92f19005699a3df07e55b4d0dddd2e82445da2a87934f640f0ac50d9ad1cc","compiler_version":"v0.66.1","strict":true,"agent_id":"copilot"} name: "SDK Tier Audit" "on": @@ -93,11 +93,12 @@ concurrency: run-name: "SDK Tier Audit" +env: + FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true" + jobs: activation: - needs: - - pre_activation - - tier-check + needs: pre_activation if: > needs.pre_activation.outputs.activated == 'true' && (github.repository_owner == 'modelcontextprotocol' || github.event_name == 'workflow_dispatch') @@ -180,12 +181,6 @@ jobs: env: GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt GH_AW_SAFE_OUTPUTS: ${{ runner.temp }}/gh-aw/safeoutputs/outputs.jsonl - GH_AW_EXPR_0DFFBBBC: ${{ needs.tier-check.outputs.csharp_sdk_repo }} - GH_AW_EXPR_3F2EBE85: ${{ needs.tier-check.outputs.csharp_sdk_branch }} - GH_AW_EXPR_771B4966: ${{ needs.tier-check.outputs.scope }} - GH_AW_EXPR_9CED0D62: ${{ needs.tier-check.outputs.conformance_branch }} - GH_AW_EXPR_BDA3F902: ${{ needs.tier-check.outputs.output }} - GH_AW_EXPR_DC7F6CEA: ${{ needs.tier-check.outputs.conformance_repo }} GH_AW_GITHUB_ACTOR: ${{ github.actor }} GH_AW_GITHUB_EVENT_COMMENT_ID: ${{ github.event.comment.id }} GH_AW_GITHUB_EVENT_DISCUSSION_NUMBER: ${{ github.event.discussion.number }} @@ -198,14 +193,14 @@ jobs: run: | bash ${RUNNER_TEMP}/gh-aw/actions/create_prompt_first.sh { - cat << 'GH_AW_PROMPT_bc128154aed53b51_EOF' + cat << 'GH_AW_PROMPT_21d20e14acbcf8de_EOF' - GH_AW_PROMPT_bc128154aed53b51_EOF + GH_AW_PROMPT_21d20e14acbcf8de_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/xpia.md" cat "${RUNNER_TEMP}/gh-aw/prompts/temp_folder_prompt.md" cat "${RUNNER_TEMP}/gh-aw/prompts/markdown.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_prompt.md" - cat << 'GH_AW_PROMPT_bc128154aed53b51_EOF' + cat << 'GH_AW_PROMPT_21d20e14acbcf8de_EOF' Tools: create_issue, missing_tool, missing_data, noop @@ -237,23 +232,17 @@ jobs: {{/if}} - GH_AW_PROMPT_bc128154aed53b51_EOF + GH_AW_PROMPT_21d20e14acbcf8de_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/github_mcp_tools_with_safeoutputs_prompt.md" - cat << 'GH_AW_PROMPT_bc128154aed53b51_EOF' + cat << 'GH_AW_PROMPT_21d20e14acbcf8de_EOF' {{#runtime-import .github/workflows/sdk-tier-audit.md}} - GH_AW_PROMPT_bc128154aed53b51_EOF + GH_AW_PROMPT_21d20e14acbcf8de_EOF } > "$GH_AW_PROMPT" - name: Interpolate variables and render templates uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 env: GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt - GH_AW_EXPR_9CED0D62: ${{ needs.tier-check.outputs.conformance_branch }} - GH_AW_EXPR_DC7F6CEA: ${{ needs.tier-check.outputs.conformance_repo }} - GH_AW_EXPR_3F2EBE85: ${{ needs.tier-check.outputs.csharp_sdk_branch }} - GH_AW_EXPR_0DFFBBBC: ${{ needs.tier-check.outputs.csharp_sdk_repo }} - GH_AW_EXPR_BDA3F902: ${{ needs.tier-check.outputs.output }} - GH_AW_EXPR_771B4966: ${{ needs.tier-check.outputs.scope }} with: script: | const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); @@ -264,12 +253,6 @@ jobs: uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 env: GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt - GH_AW_EXPR_0DFFBBBC: ${{ needs.tier-check.outputs.csharp_sdk_repo }} - GH_AW_EXPR_3F2EBE85: ${{ needs.tier-check.outputs.csharp_sdk_branch }} - GH_AW_EXPR_771B4966: ${{ needs.tier-check.outputs.scope }} - GH_AW_EXPR_9CED0D62: ${{ needs.tier-check.outputs.conformance_branch }} - GH_AW_EXPR_BDA3F902: ${{ needs.tier-check.outputs.output }} - GH_AW_EXPR_DC7F6CEA: ${{ needs.tier-check.outputs.conformance_repo }} GH_AW_GITHUB_ACTOR: ${{ github.actor }} GH_AW_GITHUB_EVENT_COMMENT_ID: ${{ github.event.comment.id }} GH_AW_GITHUB_EVENT_DISCUSSION_NUMBER: ${{ github.event.discussion.number }} @@ -290,12 +273,6 @@ jobs: return await substitutePlaceholders({ file: process.env.GH_AW_PROMPT, substitutions: { - GH_AW_EXPR_0DFFBBBC: process.env.GH_AW_EXPR_0DFFBBBC, - GH_AW_EXPR_3F2EBE85: process.env.GH_AW_EXPR_3F2EBE85, - GH_AW_EXPR_771B4966: process.env.GH_AW_EXPR_771B4966, - GH_AW_EXPR_9CED0D62: process.env.GH_AW_EXPR_9CED0D62, - GH_AW_EXPR_BDA3F902: process.env.GH_AW_EXPR_BDA3F902, - GH_AW_EXPR_DC7F6CEA: process.env.GH_AW_EXPR_DC7F6CEA, GH_AW_GITHUB_ACTOR: process.env.GH_AW_GITHUB_ACTOR, GH_AW_GITHUB_EVENT_COMMENT_ID: process.env.GH_AW_GITHUB_EVENT_COMMENT_ID, GH_AW_GITHUB_EVENT_DISCUSSION_NUMBER: process.env.GH_AW_GITHUB_EVENT_DISCUSSION_NUMBER, @@ -335,6 +312,7 @@ jobs: permissions: contents: read issues: read + pull-requests: read concurrency: group: "gh-aw-copilot-${{ github.workflow }}" env: @@ -383,9 +361,23 @@ jobs: env: GH_TOKEN: ${{ github.token }} - env: + AUDIT_CONF_BRANCH: ${{ needs.tier-check.outputs.conformance_branch }} + AUDIT_CONF_REPO: ${{ needs.tier-check.outputs.conformance_repo }} + AUDIT_OUTPUT: ${{ needs.tier-check.outputs.output }} + AUDIT_SCOPE: ${{ needs.tier-check.outputs.scope }} + AUDIT_SDK_BRANCH: ${{ needs.tier-check.outputs.csharp_sdk_branch }} + AUDIT_SDK_REPO: ${{ needs.tier-check.outputs.csharp_sdk_repo }} TIER_CHECK_JSON: ${{ needs.tier-check.outputs.tier_check_json }} - name: Write tier-check scorecard to file - run: echo "$TIER_CHECK_JSON" > /tmp/tier-check-scorecard.json + name: Write tier-check outputs to files + run: |- + echo "$TIER_CHECK_JSON" > /tmp/tier-check-scorecard.json + mkdir -p /tmp/audit-params + echo "$AUDIT_SCOPE" > /tmp/audit-params/scope + echo "$AUDIT_OUTPUT" > /tmp/audit-params/output + echo "$AUDIT_SDK_REPO" > /tmp/audit-params/csharp-sdk-repo + echo "$AUDIT_SDK_BRANCH" > /tmp/audit-params/csharp-sdk-branch + echo "$AUDIT_CONF_REPO" > /tmp/audit-params/conformance-repo + echo "$AUDIT_CONF_BRANCH" > /tmp/audit-params/conformance-branch - name: Configure Git credentials env: @@ -419,16 +411,13 @@ jobs: GH_HOST: github.com - name: Install AWF binary run: bash ${RUNNER_TEMP}/gh-aw/actions/install_awf_binary.sh v0.25.13 - - name: Determine automatic lockdown mode for GitHub MCP Server - id: determine-automatic-lockdown - uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + - name: Parse integrity filter lists + id: parse-guard-vars env: - GH_AW_GITHUB_TOKEN: ${{ secrets.GH_AW_GITHUB_TOKEN }} - GH_AW_GITHUB_MCP_SERVER_TOKEN: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN }} - with: - script: | - const determineAutomaticLockdown = require('${{ runner.temp }}/gh-aw/actions/determine_automatic_lockdown.cjs'); - await determineAutomaticLockdown(github, context, core); + GH_AW_BLOCKED_USERS_VAR: ${{ vars.GH_AW_GITHUB_BLOCKED_USERS || '' }} + GH_AW_TRUSTED_USERS_VAR: ${{ vars.GH_AW_GITHUB_TRUSTED_USERS || '' }} + GH_AW_APPROVAL_LABELS_VAR: ${{ vars.GH_AW_GITHUB_APPROVAL_LABELS || '' }} + run: bash ${RUNNER_TEMP}/gh-aw/actions/parse_guard_list.sh - name: Download container images run: bash ${RUNNER_TEMP}/gh-aw/actions/download_docker_images.sh ghcr.io/github/gh-aw-firewall/agent:0.25.13 ghcr.io/github/gh-aw-firewall/api-proxy:0.25.13 ghcr.io/github/gh-aw-firewall/squid:0.25.13 ghcr.io/github/gh-aw-mcpg:v0.2.12 ghcr.io/github/github-mcp-server:v0.32.0 node:lts-alpine - name: Write Safe Outputs Config @@ -436,12 +425,12 @@ jobs: mkdir -p ${RUNNER_TEMP}/gh-aw/safeoutputs mkdir -p /tmp/gh-aw/safeoutputs mkdir -p /tmp/gh-aw/mcp-logs/safeoutputs - cat > ${RUNNER_TEMP}/gh-aw/safeoutputs/config.json << 'GH_AW_SAFE_OUTPUTS_CONFIG_125bf17eed223b0c_EOF' - {"create_issue":{"close_older_issues":true,"labels":["automation"],"max":1,"title_prefix":"[C# SDK Tier Audit] "},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"}} - GH_AW_SAFE_OUTPUTS_CONFIG_125bf17eed223b0c_EOF + cat > ${RUNNER_TEMP}/gh-aw/safeoutputs/config.json << 'GH_AW_SAFE_OUTPUTS_CONFIG_ea2a2941ec57eb31_EOF' + {"create_issue":{"close_older_issues":true,"labels":["automation"],"max":1,"title_prefix":"[C# SDK Tier Audit] "},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"false"}} + GH_AW_SAFE_OUTPUTS_CONFIG_ea2a2941ec57eb31_EOF - name: Write Safe Outputs Tools run: | - cat > ${RUNNER_TEMP}/gh-aw/safeoutputs/tools_meta.json << 'GH_AW_SAFE_OUTPUTS_TOOLS_META_2a3a0c9c86f0eae4_EOF' + cat > ${RUNNER_TEMP}/gh-aw/safeoutputs/tools_meta.json << 'GH_AW_SAFE_OUTPUTS_TOOLS_META_127fe9d6e4c318c6_EOF' { "description_suffixes": { "create_issue": " CONSTRAINTS: Maximum 1 issue(s) can be created. Title will be prefixed with \"[C# SDK Tier Audit] \". Labels [\"automation\"] will be automatically added." @@ -449,8 +438,8 @@ jobs: "repo_params": {}, "dynamic_tools": [] } - GH_AW_SAFE_OUTPUTS_TOOLS_META_2a3a0c9c86f0eae4_EOF - cat > ${RUNNER_TEMP}/gh-aw/safeoutputs/validation.json << 'GH_AW_SAFE_OUTPUTS_VALIDATION_dc6089ad4494a421_EOF' + GH_AW_SAFE_OUTPUTS_TOOLS_META_127fe9d6e4c318c6_EOF + cat > ${RUNNER_TEMP}/gh-aw/safeoutputs/validation.json << 'GH_AW_SAFE_OUTPUTS_VALIDATION_07628e897611a621_EOF' { "create_issue": { "defaultMax": 1, @@ -543,7 +532,7 @@ jobs: } } } - GH_AW_SAFE_OUTPUTS_VALIDATION_dc6089ad4494a421_EOF + GH_AW_SAFE_OUTPUTS_VALIDATION_07628e897611a621_EOF node ${RUNNER_TEMP}/gh-aw/actions/generate_safe_outputs_tools.cjs - name: Generate Safe Outputs MCP Server Config id: safe-outputs-config @@ -591,8 +580,6 @@ jobs: GH_AW_SAFE_OUTPUTS: ${{ steps.set-runtime-paths.outputs.GH_AW_SAFE_OUTPUTS }} GH_AW_SAFE_OUTPUTS_API_KEY: ${{ steps.safe-outputs-start.outputs.api_key }} GH_AW_SAFE_OUTPUTS_PORT: ${{ steps.safe-outputs-start.outputs.port }} - GITHUB_MCP_GUARD_MIN_INTEGRITY: ${{ steps.determine-automatic-lockdown.outputs.min_integrity }} - GITHUB_MCP_GUARD_REPOS: ${{ steps.determine-automatic-lockdown.outputs.repos }} GITHUB_MCP_SERVER_TOKEN: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN || secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} run: | set -eo pipefail @@ -613,7 +600,7 @@ jobs: export MCP_GATEWAY_DOCKER_COMMAND='docker run -i --rm --network host -v /var/run/docker.sock:/var/run/docker.sock -e MCP_GATEWAY_PORT -e MCP_GATEWAY_DOMAIN -e MCP_GATEWAY_API_KEY -e MCP_GATEWAY_PAYLOAD_DIR -e MCP_GATEWAY_PAYLOAD_SIZE_THRESHOLD -e DEBUG -e MCP_GATEWAY_LOG_DIR -e GH_AW_MCP_LOG_DIR -e GH_AW_SAFE_OUTPUTS -e GH_AW_SAFE_OUTPUTS_CONFIG_PATH -e GH_AW_SAFE_OUTPUTS_TOOLS_PATH -e GH_AW_ASSETS_BRANCH -e GH_AW_ASSETS_MAX_SIZE_KB -e GH_AW_ASSETS_ALLOWED_EXTS -e DEFAULT_BRANCH -e GITHUB_MCP_SERVER_TOKEN -e GITHUB_MCP_GUARD_MIN_INTEGRITY -e GITHUB_MCP_GUARD_REPOS -e GITHUB_REPOSITORY -e GITHUB_SERVER_URL -e GITHUB_SHA -e GITHUB_WORKSPACE -e GITHUB_TOKEN -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e GITHUB_JOB -e GITHUB_ACTION -e GITHUB_EVENT_NAME -e GITHUB_EVENT_PATH -e GITHUB_ACTOR -e GITHUB_ACTOR_ID -e GITHUB_TRIGGERING_ACTOR -e GITHUB_WORKFLOW -e GITHUB_WORKFLOW_REF -e GITHUB_WORKFLOW_SHA -e GITHUB_REF -e GITHUB_REF_NAME -e GITHUB_REF_TYPE -e GITHUB_HEAD_REF -e GITHUB_BASE_REF -e GH_AW_SAFE_OUTPUTS_PORT -e GH_AW_SAFE_OUTPUTS_API_KEY -v /tmp/gh-aw/mcp-payloads:/tmp/gh-aw/mcp-payloads:rw -v /opt:/opt:ro -v /tmp:/tmp:rw -v '"${GITHUB_WORKSPACE}"':'"${GITHUB_WORKSPACE}"':rw ghcr.io/github/gh-aw-mcpg:v0.2.12' mkdir -p /home/runner/.copilot - cat << GH_AW_MCP_CONFIG_51b4796376e01c6c_EOF | bash ${RUNNER_TEMP}/gh-aw/actions/start_mcp_gateway.sh + cat << GH_AW_MCP_CONFIG_95be2b88947e780e_EOF | bash ${RUNNER_TEMP}/gh-aw/actions/start_mcp_gateway.sh { "mcpServers": { "github": { @@ -627,8 +614,11 @@ jobs: }, "guard-policies": { "allow-only": { - "min-integrity": "$GITHUB_MCP_GUARD_MIN_INTEGRITY", - "repos": "$GITHUB_MCP_GUARD_REPOS" + "approval-labels": ${{ steps.parse-guard-vars.outputs.approval_labels }}, + "blocked-users": ${{ steps.parse-guard-vars.outputs.blocked_users }}, + "min-integrity": "approved", + "repos": "all", + "trusted-users": ${{ steps.parse-guard-vars.outputs.trusted_users }} } } }, @@ -654,7 +644,7 @@ jobs: "payloadDir": "${MCP_GATEWAY_PAYLOAD_DIR}" } } - GH_AW_MCP_CONFIG_51b4796376e01c6c_EOF + GH_AW_MCP_CONFIG_95be2b88947e780e_EOF - name: Download activation artifact uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 with: @@ -841,6 +831,8 @@ jobs: /tmp/gh-aw/sandbox/agent/logs/ /tmp/gh-aw/redacted-urls.log /tmp/gh-aw/mcp-logs/ + /tmp/gh-aw/proxy-logs/ + !/tmp/gh-aw/proxy-logs/proxy-tls/ /tmp/gh-aw/agent_usage.json /tmp/gh-aw/agent-stdio.log /tmp/gh-aw/agent/ @@ -907,7 +899,7 @@ jobs: GH_AW_WORKFLOW_NAME: "SDK Tier Audit" GH_AW_RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} GH_AW_AGENT_CONCLUSION: ${{ needs.agent.result }} - GH_AW_NOOP_REPORT_AS_ISSUE: "true" + GH_AW_NOOP_REPORT_AS_ISSUE: "false" with: github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} script: | @@ -1209,7 +1201,7 @@ jobs: GH_AW_ALLOWED_DOMAINS: "*.githubusercontent.com,*.vsblob.vsassets.io,api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.npms.io,api.nuget.org,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,azuresearch-usnc.nuget.org,azuresearch-ussc.nuget.org,builds.dotnet.microsoft.com,bun.sh,cdn.jsdelivr.net,ci.dot.net,codeload.github.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,dc.services.visualstudio.com,deb.nodesource.com,deno.land,dist.nuget.org,docs.github.com,dot.net,dotnet.microsoft.com,dotnetcli.blob.core.windows.net,esm.sh,get.pnpm.io,github-cloud.githubusercontent.com,github-cloud.s3.amazonaws.com,github.blog,github.com,github.githubassets.com,googleapis.deno.dev,googlechromelabs.github.io,host.docker.internal,json-schema.org,json.schemastore.org,jsr.io,keyserver.ubuntu.com,lfs.github.com,nodejs.org,npm.pkg.github.com,npmjs.com,npmjs.org,nuget.org,nuget.pkg.github.com,nugetregistryv2prod.blob.core.windows.net,objects.githubusercontent.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,oneocsp.microsoft.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,pkgs.dev.azure.com,ppa.launchpad.net,raw.githubusercontent.com,registry.bower.io,registry.npmjs.com,registry.npmjs.org,registry.yarnpkg.com,repo.yarnpkg.com,s.symcb.com,s.symcd.com,security.ubuntu.com,skimdb.npmjs.com,storage.googleapis.com,telemetry.enterprise.githubcopilot.com,telemetry.vercel.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com,www.microsoft.com,www.npmjs.com,www.npmjs.org,yarnpkg.com" GITHUB_SERVER_URL: ${{ github.server_url }} GITHUB_API_URL: ${{ github.api_url }} - GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"create_issue\":{\"close_older_issues\":true,\"labels\":[\"automation\"],\"max\":1,\"title_prefix\":\"[C# SDK Tier Audit] \"},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"}}" + GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"create_issue\":{\"close_older_issues\":true,\"labels\":[\"automation\"],\"max\":1,\"title_prefix\":\"[C# SDK Tier Audit] \"},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"false\"}}" with: github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} script: | @@ -1226,6 +1218,7 @@ jobs: if-no-files-found: ignore tier-check: + needs: activation runs-on: ubuntu-latest timeout-minutes: 30 outputs: diff --git a/.github/workflows/sdk-tier-audit.md b/.github/workflows/sdk-tier-audit.md index dac4c8763..3c7550f3e 100644 --- a/.github/workflows/sdk-tier-audit.md +++ b/.github/workflows/sdk-tier-audit.md @@ -4,15 +4,26 @@ description: "SDK Tier Audit" permissions: contents: read issues: read + pull-requests: read safe-outputs: create-issue: title-prefix: "[C# SDK Tier Audit] " labels: [automation] close-older-issues: true + max: 1 + noop: + report-as-issue: false + +tools: + github: + min-integrity: approved if: github.repository_owner == 'modelcontextprotocol' || github.event_name == 'workflow_dispatch' +env: + FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true" + concurrency: group: tier-audit-${{ github.event.inputs.scope || 'Conformance + Repo Health' }} cancel-in-progress: true From 2e5451cc60f258f435d570f83da49e3727195c8b Mon Sep 17 00:00:00 2001 From: Jeff Handley Date: Sat, 4 Apr 2026 23:25:35 -0700 Subject: [PATCH 7/7] Remove own tier logic; fix action summary and artifact output MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two issues fixed: 1. Tier logic: The prompt was telling the agent to 'Apply tier logic' itself (Step 2.4). Now the prompt explicitly says: 'Do not apply your own tier logic or scoring — use only the conformance skill's thresholds, rules, and templates.' All tier determination is delegated to the conformance repo's mcp-sdk-tier-audit skill. 2. Action summary: The report was not appearing on the workflow summary page because the agent wasn't writing to $GITHUB_STEP_SUMMARY reliably. Now Step 3 is restructured with explicit requirements: - MUST write /tmp/audit-report.md (artifact) - MUST cat it to $GITHUB_STEP_SUMMARY (action summary) - Both required BEFORE Step 4 (publish) - 'Action Summary' mode now explicitly calls noop - Issue body must be identical to the action summary content Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/workflows/sdk-tier-audit.lock.yml | 41 +++---- .github/workflows/sdk-tier-audit.md | 133 ++++++++++++---------- 2 files changed, 89 insertions(+), 85 deletions(-) diff --git a/.github/workflows/sdk-tier-audit.lock.yml b/.github/workflows/sdk-tier-audit.lock.yml index d14f2613a..9aeca5f28 100644 --- a/.github/workflows/sdk-tier-audit.lock.yml +++ b/.github/workflows/sdk-tier-audit.lock.yml @@ -22,7 +22,7 @@ # # SDK Tier Audit # -# gh-aw-metadata: {"schema_version":"v3","frontmatter_hash":"e5e92f19005699a3df07e55b4d0dddd2e82445da2a87934f640f0ac50d9ad1cc","compiler_version":"v0.66.1","strict":true,"agent_id":"copilot"} +# gh-aw-metadata: {"schema_version":"v3","frontmatter_hash":"9057f6bdae085366734f4af5f0b5f55a96968d48627f96d0374910f0d29ac92c","compiler_version":"v0.66.1","strict":true,"agent_id":"copilot"} name: "SDK Tier Audit" "on": @@ -193,14 +193,14 @@ jobs: run: | bash ${RUNNER_TEMP}/gh-aw/actions/create_prompt_first.sh { - cat << 'GH_AW_PROMPT_21d20e14acbcf8de_EOF' + cat << 'GH_AW_PROMPT_c2561529c84de816_EOF' - GH_AW_PROMPT_21d20e14acbcf8de_EOF + GH_AW_PROMPT_c2561529c84de816_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/xpia.md" cat "${RUNNER_TEMP}/gh-aw/prompts/temp_folder_prompt.md" cat "${RUNNER_TEMP}/gh-aw/prompts/markdown.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_prompt.md" - cat << 'GH_AW_PROMPT_21d20e14acbcf8de_EOF' + cat << 'GH_AW_PROMPT_c2561529c84de816_EOF' Tools: create_issue, missing_tool, missing_data, noop @@ -232,12 +232,12 @@ jobs: {{/if}} - GH_AW_PROMPT_21d20e14acbcf8de_EOF + GH_AW_PROMPT_c2561529c84de816_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/github_mcp_tools_with_safeoutputs_prompt.md" - cat << 'GH_AW_PROMPT_21d20e14acbcf8de_EOF' + cat << 'GH_AW_PROMPT_c2561529c84de816_EOF' {{#runtime-import .github/workflows/sdk-tier-audit.md}} - GH_AW_PROMPT_21d20e14acbcf8de_EOF + GH_AW_PROMPT_c2561529c84de816_EOF } > "$GH_AW_PROMPT" - name: Interpolate variables and render templates uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 @@ -352,7 +352,7 @@ jobs: - name: Setup Node.js uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0 with: - node-version: '22' + node-version: '24' package-manager-cache: false - name: Create gh-aw temp directory run: bash ${RUNNER_TEMP}/gh-aw/actions/create_gh_aw_tmp_dir.sh @@ -425,12 +425,12 @@ jobs: mkdir -p ${RUNNER_TEMP}/gh-aw/safeoutputs mkdir -p /tmp/gh-aw/safeoutputs mkdir -p /tmp/gh-aw/mcp-logs/safeoutputs - cat > ${RUNNER_TEMP}/gh-aw/safeoutputs/config.json << 'GH_AW_SAFE_OUTPUTS_CONFIG_ea2a2941ec57eb31_EOF' + cat > ${RUNNER_TEMP}/gh-aw/safeoutputs/config.json << 'GH_AW_SAFE_OUTPUTS_CONFIG_86ce87cfaa56791d_EOF' {"create_issue":{"close_older_issues":true,"labels":["automation"],"max":1,"title_prefix":"[C# SDK Tier Audit] "},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"false"}} - GH_AW_SAFE_OUTPUTS_CONFIG_ea2a2941ec57eb31_EOF + GH_AW_SAFE_OUTPUTS_CONFIG_86ce87cfaa56791d_EOF - name: Write Safe Outputs Tools run: | - cat > ${RUNNER_TEMP}/gh-aw/safeoutputs/tools_meta.json << 'GH_AW_SAFE_OUTPUTS_TOOLS_META_127fe9d6e4c318c6_EOF' + cat > ${RUNNER_TEMP}/gh-aw/safeoutputs/tools_meta.json << 'GH_AW_SAFE_OUTPUTS_TOOLS_META_8590df58ad39dcf8_EOF' { "description_suffixes": { "create_issue": " CONSTRAINTS: Maximum 1 issue(s) can be created. Title will be prefixed with \"[C# SDK Tier Audit] \". Labels [\"automation\"] will be automatically added." @@ -438,8 +438,8 @@ jobs: "repo_params": {}, "dynamic_tools": [] } - GH_AW_SAFE_OUTPUTS_TOOLS_META_127fe9d6e4c318c6_EOF - cat > ${RUNNER_TEMP}/gh-aw/safeoutputs/validation.json << 'GH_AW_SAFE_OUTPUTS_VALIDATION_07628e897611a621_EOF' + GH_AW_SAFE_OUTPUTS_TOOLS_META_8590df58ad39dcf8_EOF + cat > ${RUNNER_TEMP}/gh-aw/safeoutputs/validation.json << 'GH_AW_SAFE_OUTPUTS_VALIDATION_66482c4b5a6b576c_EOF' { "create_issue": { "defaultMax": 1, @@ -532,7 +532,7 @@ jobs: } } } - GH_AW_SAFE_OUTPUTS_VALIDATION_07628e897611a621_EOF + GH_AW_SAFE_OUTPUTS_VALIDATION_66482c4b5a6b576c_EOF node ${RUNNER_TEMP}/gh-aw/actions/generate_safe_outputs_tools.cjs - name: Generate Safe Outputs MCP Server Config id: safe-outputs-config @@ -600,7 +600,7 @@ jobs: export MCP_GATEWAY_DOCKER_COMMAND='docker run -i --rm --network host -v /var/run/docker.sock:/var/run/docker.sock -e MCP_GATEWAY_PORT -e MCP_GATEWAY_DOMAIN -e MCP_GATEWAY_API_KEY -e MCP_GATEWAY_PAYLOAD_DIR -e MCP_GATEWAY_PAYLOAD_SIZE_THRESHOLD -e DEBUG -e MCP_GATEWAY_LOG_DIR -e GH_AW_MCP_LOG_DIR -e GH_AW_SAFE_OUTPUTS -e GH_AW_SAFE_OUTPUTS_CONFIG_PATH -e GH_AW_SAFE_OUTPUTS_TOOLS_PATH -e GH_AW_ASSETS_BRANCH -e GH_AW_ASSETS_MAX_SIZE_KB -e GH_AW_ASSETS_ALLOWED_EXTS -e DEFAULT_BRANCH -e GITHUB_MCP_SERVER_TOKEN -e GITHUB_MCP_GUARD_MIN_INTEGRITY -e GITHUB_MCP_GUARD_REPOS -e GITHUB_REPOSITORY -e GITHUB_SERVER_URL -e GITHUB_SHA -e GITHUB_WORKSPACE -e GITHUB_TOKEN -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e GITHUB_JOB -e GITHUB_ACTION -e GITHUB_EVENT_NAME -e GITHUB_EVENT_PATH -e GITHUB_ACTOR -e GITHUB_ACTOR_ID -e GITHUB_TRIGGERING_ACTOR -e GITHUB_WORKFLOW -e GITHUB_WORKFLOW_REF -e GITHUB_WORKFLOW_SHA -e GITHUB_REF -e GITHUB_REF_NAME -e GITHUB_REF_TYPE -e GITHUB_HEAD_REF -e GITHUB_BASE_REF -e GH_AW_SAFE_OUTPUTS_PORT -e GH_AW_SAFE_OUTPUTS_API_KEY -v /tmp/gh-aw/mcp-payloads:/tmp/gh-aw/mcp-payloads:rw -v /opt:/opt:ro -v /tmp:/tmp:rw -v '"${GITHUB_WORKSPACE}"':'"${GITHUB_WORKSPACE}"':rw ghcr.io/github/gh-aw-mcpg:v0.2.12' mkdir -p /home/runner/.copilot - cat << GH_AW_MCP_CONFIG_95be2b88947e780e_EOF | bash ${RUNNER_TEMP}/gh-aw/actions/start_mcp_gateway.sh + cat << GH_AW_MCP_CONFIG_718955b16c63d18a_EOF | bash ${RUNNER_TEMP}/gh-aw/actions/start_mcp_gateway.sh { "mcpServers": { "github": { @@ -644,7 +644,7 @@ jobs: "payloadDir": "${MCP_GATEWAY_PAYLOAD_DIR}" } } - GH_AW_MCP_CONFIG_95be2b88947e780e_EOF + GH_AW_MCP_CONFIG_718955b16c63d18a_EOF - name: Download activation artifact uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 with: @@ -812,13 +812,8 @@ jobs: echo '{"items":[]}' > /tmp/gh-aw/agent_output.json fi - if: always() - name: Upload audit report - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4 - with: - if-no-files-found: ignore - name: audit-report - path: /tmp/audit-report.md - retention-days: 90 + name: Write audit report to action summary + run: "if [ -f /tmp/audit-report.md ]; then\n cat /tmp/audit-report.md >> \"$GITHUB_STEP_SUMMARY\"\nelse\n echo \"## Tier Audit: No Report\" >> \"$GITHUB_STEP_SUMMARY\"\n echo \"The agent did not produce /tmp/audit-report.md.\" >> \"$GITHUB_STEP_SUMMARY\"\nfi\n" - name: Upload agent artifacts if: always() diff --git a/.github/workflows/sdk-tier-audit.md b/.github/workflows/sdk-tier-audit.md index 3c7550f3e..ccb3e3afb 100644 --- a/.github/workflows/sdk-tier-audit.md +++ b/.github/workflows/sdk-tier-audit.md @@ -6,6 +6,13 @@ permissions: issues: read pull-requests: read +network: + allowed: + - defaults + - node + - dotnet + - github + safe-outputs: create-issue: title-prefix: "[C# SDK Tier Audit] " @@ -21,43 +28,51 @@ tools: if: github.repository_owner == 'modelcontextprotocol' || github.event_name == 'workflow_dispatch' -env: - FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true" - concurrency: group: tier-audit-${{ github.event.inputs.scope || 'Conformance + Repo Health' }} cancel-in-progress: true +timeout-minutes: 120 + runtimes: node: - version: "22" + version: "24" dotnet: version: "10.0" -network: - allowed: - - defaults - - node - - dotnet - - github - -timeout-minutes: 120 +env: + FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true" steps: - - name: Write tier-check scorecard to file + - name: Write tier-check outputs to files env: TIER_CHECK_JSON: ${{ needs.tier-check.outputs.tier_check_json }} - run: echo "$TIER_CHECK_JSON" > /tmp/tier-check-scorecard.json + AUDIT_SCOPE: ${{ needs.tier-check.outputs.scope }} + AUDIT_OUTPUT: ${{ needs.tier-check.outputs.output }} + AUDIT_SDK_REPO: ${{ needs.tier-check.outputs.csharp_sdk_repo }} + AUDIT_SDK_BRANCH: ${{ needs.tier-check.outputs.csharp_sdk_branch }} + AUDIT_CONF_REPO: ${{ needs.tier-check.outputs.conformance_repo }} + AUDIT_CONF_BRANCH: ${{ needs.tier-check.outputs.conformance_branch }} + run: | + echo "$TIER_CHECK_JSON" > /tmp/tier-check-scorecard.json + mkdir -p /tmp/audit-params + echo "$AUDIT_SCOPE" > /tmp/audit-params/scope + echo "$AUDIT_OUTPUT" > /tmp/audit-params/output + echo "$AUDIT_SDK_REPO" > /tmp/audit-params/csharp-sdk-repo + echo "$AUDIT_SDK_BRANCH" > /tmp/audit-params/csharp-sdk-branch + echo "$AUDIT_CONF_REPO" > /tmp/audit-params/conformance-repo + echo "$AUDIT_CONF_BRANCH" > /tmp/audit-params/conformance-branch post-steps: - - name: Upload audit report + - name: Write audit report to action summary if: always() - uses: actions/upload-artifact@v4 - with: - name: audit-report - path: /tmp/audit-report.md - retention-days: 90 - if-no-files-found: ignore + run: | + if [ -f /tmp/audit-report.md ]; then + cat /tmp/audit-report.md >> "$GITHUB_STEP_SUMMARY" + else + echo "## Tier Audit: No Report" >> "$GITHUB_STEP_SUMMARY" + echo "The agent did not produce /tmp/audit-report.md." >> "$GITHUB_STEP_SUMMARY" + fi on: schedule: weekly on thursday around 6:30am utc-5 @@ -266,12 +281,18 @@ Perform a tier audit of the C# MCP SDK. The deterministic tier-check scorecard h ## Inputs -- **Scope**: ${{ needs.tier-check.outputs.scope }} -- **Output mode**: ${{ needs.tier-check.outputs.output }} -- **C# SDK**: ${{ needs.tier-check.outputs.csharp_sdk_repo }} (branch: ${{ needs.tier-check.outputs.csharp_sdk_branch }}) -- **Conformance**: ${{ needs.tier-check.outputs.conformance_repo }} (branch: ${{ needs.tier-check.outputs.conformance_branch }}) +All parameters are written to `/tmp/audit-params/` by a pre-agent step. Read them: -The scorecard is at `/tmp/tier-check-scorecard.json` (written by a pre-agent step from the `tier-check` job output). +```bash +SCOPE=$(cat /tmp/audit-params/scope) +OUTPUT_MODE=$(cat /tmp/audit-params/output) +SDK_REPO=$(cat /tmp/audit-params/csharp-sdk-repo) +SDK_BRANCH=$(cat /tmp/audit-params/csharp-sdk-branch) +CONF_REPO=$(cat /tmp/audit-params/conformance-repo) +CONF_BRANCH=$(cat /tmp/audit-params/conformance-branch) +``` + +The tier-check scorecard is at `/tmp/tier-check-scorecard.json`. ## Tier-Check Scorecard (pre-computed) @@ -283,33 +304,33 @@ cat /tmp/tier-check-scorecard.json ## Step 1: Setup -Clone both repositories for the AI-assisted evaluations. Use shallow clones. +Read the parameters from `/tmp/audit-params/` and clone both repositories for the AI-assisted evaluations. Use shallow clones. ```bash -git clone --depth 1 -b https://github.com/.git /tmp/csharp-sdk -git clone --depth 1 -b https://github.com/.git /tmp/conformance +SDK_REPO=$(cat /tmp/audit-params/csharp-sdk-repo) +SDK_BRANCH=$(cat /tmp/audit-params/csharp-sdk-branch) +CONF_REPO=$(cat /tmp/audit-params/conformance-repo) +CONF_BRANCH=$(cat /tmp/audit-params/conformance-branch) + +git clone --depth 1 -b "$SDK_BRANCH" "https://github.com/${SDK_REPO}.git" /tmp/csharp-sdk +git clone --depth 1 -b "$CONF_BRANCH" "https://github.com/${CONF_REPO}.git" /tmp/conformance ``` You do NOT need to build either repo or start any servers — the conformance tests have already run. ## Step 2: AI-Assisted Evaluation -Read the **"Any Other AI Coding Agent"** section from `/tmp/conformance/.claude/skills/mcp-sdk-tier-audit/README.md`. Follow steps **2 through 5** only (skip step 1 — the CLI has already run): - -2. **Evaluate documentation coverage** using the prompt in `references/docs-coverage-prompt.md` -3. **Evaluate policies** using the prompt in `references/policy-evaluation-prompt.md` — pass the `policy_signals` section from the tier-check JSON above -4. **Apply tier logic** using the thresholds in `references/tier-requirements.md` — combine the scorecard above with your evaluation results -5. **Generate report** using the template in `references/report-template.md` +Read the **"Any Other AI Coding Agent"** section from `/tmp/conformance/.claude/skills/mcp-sdk-tier-audit/README.md`. Follow those instructions exactly — steps 2 through 5 (skip step 1, the CLI has already run). The conformance skill's instructions are the single source of truth for tier logic, documentation evaluation criteria, policy evaluation criteria, and report templates. **Do not apply your own tier logic or scoring — use only the conformance skill's thresholds, rules, and templates.** -The SDK checkout at `/tmp/csharp-sdk` is the local path for documentation and policy evaluations. +The tier-check scorecard JSON is at `/tmp/tier-check-scorecard.json`. The SDK checkout at `/tmp/csharp-sdk` is the local path for documentation and policy evaluations. -Write the assessment and remediation reports to `/tmp/conformance/results/`: +The conformance skill will produce assessment and remediation reports. Write them to `/tmp/conformance/results/`: - `results/-csharp-sdk-assessment.md` - `results/-csharp-sdk-remediation.md` -## Step 3: Compose the Audit Report +## Step 3: Compose and Save the Audit Report -After the evaluation completes, compose the full audit report as a single markdown file at `/tmp/audit-report.md`. This file is used for both issue creation and the action summary artifact. +After the evaluation completes, compose a single markdown file at `/tmp/audit-report.md`. This same content is used for both output modes: issue body and action summary. ### Report structure @@ -328,53 +349,41 @@ The report must contain these sections in order: Use horizontal rules (`---`) to separate sections. -### Write the report +### Save the report file -Write the composed report to `/tmp/audit-report.md`. This file will be uploaded as a workflow artifact by a post-execution step. +Write the composed report to `/tmp/audit-report.md`. A post-execution step will write it to the GitHub Action Summary (visible on the workflow run's summary page). -### Write the action summary - -Always write the full audit report to the GitHub Step Summary so it appears on the workflow run's summary page: - -```bash -cat /tmp/audit-report.md >> "$GITHUB_STEP_SUMMARY" -``` +Verify the file exists before proceeding to Step 4. ## Step 4: Publish Results +Read the output mode: `cat /tmp/audit-params/output` + ### If output mode is "Create Issue" Create a GitHub issue using the `create-issue` safe output. -**Issue title** — The dynamic part (after the `[C# SDK Tier Audit] ` prefix): +**Issue title** — Read the scope from `cat /tmp/audit-params/scope`. The dynamic part (after the `[C# SDK Tier Audit] ` prefix): - **"Conformance + Repo Health" scope**: ` - Tier ` - **"Repo Health" scope**: ` - Tier (Repo Health)` Where `` is today's date and `` is the computed tier number (1, 2, or 3). -**Issue body** — Use the contents of `/tmp/audit-report.md` as the issue body. +**Issue body** — Use the exact contents of `/tmp/audit-report.md` as the issue body. The issue body must be identical to what was written to the action summary. ### If output mode is "Action Summary" -Do NOT create an issue. The report is already written to `$GITHUB_STEP_SUMMARY` and will be uploaded as an artifact. No further action is needed. +Do NOT create an issue. Call `noop` with a message like "Audit complete — results in Action Summary." The report will be displayed on the workflow run's summary page by a post-execution step. ## Failure Handling If the evaluation fails at any step, or if the audit does not produce assessment/remediation results: 1. **Do NOT create an issue.** Do not use the `create-issue` safe output. -2. **Write a GitHub Step Summary** explaining what happened: - -```bash -echo "## Tier Audit: No Results" >> "$GITHUB_STEP_SUMMARY" -echo "" >> "$GITHUB_STEP_SUMMARY" -echo "The tier audit did not produce results. Reason: " >> "$GITHUB_STEP_SUMMARY" -echo "" >> "$GITHUB_STEP_SUMMARY" -echo "No issue was filed for this run." >> "$GITHUB_STEP_SUMMARY" -``` - -3. The `noop` safe output will apply automatically when no `create-issue` output is produced. +2. **Write a failure report** to `/tmp/audit-report.md` explaining what happened. The post-execution step will display it on the action summary page. +3. Call `noop` with a message describing the failure. +4. The post-execution step will handle displaying the failure on the action summary page. ---