Skip to content

MAINT Retire stale CodeQL runs - #3124

Open
Roman Lutz (romanlutz) wants to merge 2 commits into
microsoft:mainfrom
romanlutz:romanlutz-codeql-obsolete-runs
Open

Roman Lutz (romanlutz) wants to merge 2 commits into
microsoft:mainfrom
romanlutz:romanlutz-codeql-obsolete-runs

Conversation

@romanlutz

Copy link
Copy Markdown
Contributor

Description

CodeQL scans can finish after a merge candidate is evicted and GitHub deletes its temporary gh-readonly-queue branch. Publishing the results then fails with a missing-ref error, even though analysis succeeded. This addresses that secondary failure, following the scanning restoration in #3049.

  • Check out the exact tested SHA and separate analysis from publishing. Literal upload: failure-only preserves the pinned CodeQL action's failed-analysis diagnostics.
  • Keep the pinned native SARIF uploader for PRs, pushes, schedules and manual runs. Merge groups use a small CommonJS helper with the authenticated client and context supplied by pinned actions/github-script@v9.0.0, without a custom HTTP client or Python setup.
  • Require a successful GitHub response proving that the exact queue ref is absent or points to a different SHA before retiring a candidate. Recheck a specific missing-ref upload rejection to cover deletion between the initial check and upload, and check the ref during and after SARIF processing.
  • Verify the current CodeQL run and existing matrix-job failures before requesting cancellation. Cancellation cannot return a successful check. Authentication, rate-limit, network, analysis and genuine upload/processing failures remain explicit failures.

Required check names, all three languages, categories/fingerprints and existing triggers remain unchanged. There is no continue-on-error, skipped-success fallback, rerouting of results to main, or branch-protection change.

Self-cancellation requires actions: write on the existing CodeQL job. Workflow-wide permissions remain empty; contents stays read-only and security-events stays write. The helper only executes on merge groups; fork PRs retain the native uploader and GitHub's fork-token policy.

The final upstream diff has 600 added lines, down from the initial 1,021. The helper is 178 lines rather than 309; most remaining additions are shared offline scenarios and workflow contracts. No dependency manifest or lockfile changes are included.

Tests and Documentation

Updated the contributor testing documentation. Regression coverage uses one scripted fake GitHub client and a deterministic clock. Cases cover live/absent/replaced refs, upload and processing races, all language categories, API/network errors, analyzer and sibling-job failures, cancellation/conflicts/timeouts, and non-merge/fork workflow paths. The pytest entry point also executes the actual workflow script with mocked GitHub inputs rather than only inspecting its text.

Commands were run on Windows:

  • uv run --frozen --no-sync pytest tests\unit\build_scripts\test_codeql_merge_group.py -q: 7 passed, including the 135 Node scenarios executed through the workflow entry point.
  • node --test --experimental-test-coverage --test-coverage-include=build_scripts\codeql_merge_group.cjs --test-coverage-lines=90 --test-coverage-functions=90 --test-coverage-branches=80 --test-reporter=spec tests\unit\build_scripts\test_codeql_merge_group.cjs: 135 passed, with 100% helper line, branch and function coverage.
  • & .\frontend\node_modules\.bin\tsc.cmd --noEmit --allowJs --checkJs --strict --module node16 --target es2022 --types node --typeRoots frontend\node_modules\@types build_scripts\codeql_merge_group.cjs: passed.
  • & .\frontend\node_modules\.bin\eslint.cmd --no-config-lookup --no-warn-ignored --global 'require,module,Buffer,__dirname,process' --rule 'no-unused-vars:error' --rule 'no-undef:error' --rule 'eqeqeq:[error,smart]' build_scripts\codeql_merge_group.cjs tests\unit\build_scripts\test_codeql_merge_group.cjs: passed.
  • uv run --frozen --no-sync ty check tests\unit\build_scripts\test_codeql_merge_group.py: passed.
  • uv run --frozen --no-sync ruff check tests\unit\build_scripts\test_codeql_merge_group.py: passed.
  • uv run --frozen --no-sync pre-commit run --files .github\workflows\codeql.yml build_scripts\codeql_merge_group.cjs tests\unit\build_scripts\test_codeql_merge_group.py tests\unit\build_scripts\test_codeql_merge_group.cjs doc\contributing\4_running_tests.md: all applicable hooks passed, including YAML and documentation checks. Commit hooks passed as well.
  • node --check build_scripts\codeql_merge_group.cjs and node --check tests\unit\build_scripts\test_codeql_merge_group.cjs: passed. Actionlint v1.7.12 also passed on .github\workflows\codeql.yml using a digest-verified release binary.
  • git diff --check and git diff --cached --check: passed before committing. The normal fork push was verified against the remote HEAD.

Live workflow cancellation is mocked, not exercised against existing Actions runs. No runs were manually cancelled or rerun and no queue settings were changed. The full application suites were not run because the change is confined to CI handling.

JupyText was not run; no notebooks or documentation code samples were changed.

Separate analysis from publishing, preserve failed-scan diagnostics, and cancel only positively verified obsolete merge candidates. Keep native CodeQL publishing for non-merge events and cover the upload race with offline regression checks.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Use the pinned GitHub Script client instead of custom Python HTTP and CLI plumbing. Consolidate failure and cancellation regressions into shared offline scenarios while preserving native non-merge uploads and fail-closed merge-group behavior.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant