docs: upload coverage when CI only tests the modules that changed by heliocodacy · Pull Request #2824 · codacy/docs

heliocodacy · 2026-11-10T20:53:13Z

Codacy needs coverage data for all tested files on every commit and doesn't reuse coverage from previous commits. Monorepos whose CI only tests the modules a commit changed therefore end up with incomplete coverage and unexpected coverage changes. Codacy doesn't support this natively yet, so this PR documents a workaround.

Adds the section Uploading coverage when you only test the modules that changed (#changed-modules) to Uploading coverage in advanced scenarios, after the section on uploading multiple reports. The page description already mentions monorepos, but the page had no monorepo guidance. The section covers:

The problem, and the workaround in four steps: detect changed modules, test only those and store each report as an artifact, fetch one report per module, upload each with --partial, then send final
A GitHub Actions example, with a fetch script, a workflow with a Python and a Go module, and a first manual run that stores a report for every module
Limitations: dependencies outside a module, expired artifacts, failed tests reusing the previous report, and pull requests that are behind the default branch

Notes for reviewers:

The approach comes from an internal demo repository. The fetch script here fixes two issues in that demo's script. It filters artifacts by name, so modules don't drop out of the 100 newest artifacts. It only reuses reports from the current run or the default branch, so an unmerged pull request can't become the baseline. The script was tested against real artifacts through the GitHub API. The full workflow hasn't run end to end with this script yet.
The workflow YAML is wrapped in {% raw %}, like other pages that show GitHub Actions expressions, so the macros plugin doesn't render ${{ }}.
Commit detection on pull requests relies on the reporter's GitHubActionProvider, which reads the pull request head SHA. GitHub Actions is missing from the platform list in Troubleshooting Codacy Coverage Reporter issues. That's out of scope for this PR.

Checks: mkdocs build --strict passes locally. Vale wasn't run locally, so CI will be its first run.

👀 Live preview

Pending the branch preview deployment: coverage-reporter/uploading-coverage-in-advanced-scenarios/#changed-modules

🚧 To do

If relevant, include the Jira issue key at the end of the pull request title
Perform a self-review of the changes
Fix any issues reported by the CI/CD

🤖 Generated with Claude Code

Codacy needs coverage data for all tested files on every commit, so monorepos whose CI only tests changed modules end up with incomplete coverage. Document a workaround on GitHub Actions: store each module's report as an artifact, reuse the most recent report for unchanged modules, and upload every report with --partial followed by final. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

github-actions · 2026-11-10T20:54:08Z

Overall readability score: 54.56 (🔴 -0.09)

File	Readability
uploading-coverage-in-advanced-scenarios.md	64.37 (🟢 +3.46)

View detailed metrics

🟢 - Shows an increase in readability
🔴 - Shows a decrease in readability

File	Readability	FRE	GF	ARI	CLI	DCRS
uploading-coverage-in-advanced-scenarios.md	64.37	45.15	9.15	12	11.66	6.29
	🟢 +3.46	🔴 -0.71	🟢 +1.02	🔴 -0.2	🟢 +0.34	🟢 +0.51

Averages:

	Readability	FRE	GF	ARI	CLI	DCRS
Average	54.56	42.99	10.8	12.28	12.31	7.94
	🔴 -0.09	🔴 -0.06	🔴 -0.02	🔴 -0.02	🟢 +0	🔴 -0.01

View metric targets

Metric	Range	Ideal score
Flesch Reading Ease	100 (very easy read) to 0 (extremely difficult read)	60
Gunning Fog	6 (very easy read) to 17 (extremely difficult read)	8 or less
Auto. Read. Index	6 (very easy read) to 14 (extremely difficult read)	8 or less
Coleman Liau Index	6 (very easy read) to 17 (extremely difficult read)	8 or less
Dale-Chall Readability	4.9 (very easy read) to 9.9 (extremely difficult read)	6.9 or less

codacy-production · 2026-11-10T20:56:43Z

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

AI Reviewer: first review requested successfully. AI can make mistakes. Always validate suggestions.

_{TIP This summary will be updated as you push new changes.}

codacy-production

Pull Request Overview

⚠️ Resolve issues before merging

The documented artifact-selection workflow can reuse an older coverage report when overlapping runs complete out of order, producing incorrect aggregate coverage. This is a high-severity correctness issue that should be addressed before merging.

The workflow also has lower-severity operational gaps: artifact lookup is not paginated, pull-request change detection may lack required token permissions, and fork-based pull requests cannot access the upload token. The documented limitations should cover the fork case or provide a trusted upload path.

Test suggestions

Fetch the current workflow's artifact when available and otherwise select the newest non-expired artifact from the default branch.
Fail clearly when no artifact exists for a module.
Run tests only for changed modules while always uploading coverage for every module.
Upload reports as partial submissions and send the final notification after all reports are uploaded.

Prompt proposal for missing tests

Consider implementing these tests if applicable:
1. Fetch the current workflow's artifact when available and otherwise select the newest non-expired artifact from the default branch.
2. Fail clearly when no artifact exists for a module.
3. Run tests only for changed modules while always uploading coverage for every module.
4. Upload reports as partial submissions and send the final notification after all reports are uploaded.

Make this review better

_{TIP Add a ticket key to the branch or title so the reviewer can check the PR against it. Or tell us which ticket tracker you use here.}

_{TIP How was this review? Give us feedback}

codacy-production · 2026-11-10T20:57:12Z

+          + ([$all[]
+              | select(.workflow_run.head_branch == $branch)
+              | select(.workflow_run.head_repository_id == .workflow_run.repository_id)]
+             | sort_by(.created_at) | reverse)


_{🔴 HIGH RISK}

If an older default-branch run finishes after a newer one, subsequent commits reuse the older commit's report because it has the later creation time. Consider selecting reports by commit ancestry and handling runs whose required predecessor reports are still pending.

_{TIP React 👍 if this review is right, or reply with what's wrong and how it could be better to help improve the reviewer.}

codacy-production · 2026-11-10T20:57:12Z

+        env:
+          GH_TOKEN: ${{ github.token }}
+          DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
+          CODACY_PROJECT_TOKEN: ${{ secrets.CODACY_PROJECT_TOKEN }}


_{🟡 MEDIUM RISK}

Pull requests from forks cannot authenticate the coverage upload because CODACY_PROJECT_TOKEN is unavailable. Consider documenting this limitation or using a separate trusted workflow to upload fork coverage without executing untrusted code with the secret.

_{TIP React 👍 if this review is right, or reply with what's wrong and how it could be better to help improve the reviewer.}

codacy-production · 2026-11-10T20:57:12Z

+          gateway: ${{ steps.filter.outputs.gateway }}
+        steps:
+          - uses: actions/checkout@v4
+          - uses: dorny/paths-filter@v3


_{🟡 MEDIUM RISK}

Pull-request runs fail to detect changes when the repository's default token permissions don't include pull-request access. Consider granting pull-requests: read and contents: read to this job.

_{TIP React 👍 if this review is right, or reply with what's wrong and how it could be better to help improve the reviewer.}

codacy-production · 2026-11-10T20:57:12Z

+    # Prefer the report from the current run. Otherwise, use the most recent
+    # report from the default branch of this repository.
+    artifact_id=$(
+      gh api "repos/$GITHUB_REPOSITORY/actions/artifacts?name=$artifact_name&per_page=100" |


_{🟡 MEDIUM RISK}

If 100 newer pull-request artifacts exist, this lookup misses an unexpired default-branch report and fails the upload. Consider paginating the artifact listing before selecting a report.

_{TIP React 👍 if this review is right, or reply with what's wrong and how it could be better to help improve the reviewer.}

heliocodacy requested a review from a team as a code owner November 10, 2026 20:53

github-actions Bot temporarily deployed to Netlify November 10, 2026 20:54 Inactive

codacy-production Bot reviewed Nov 10, 2026

View reviewed changes

Navigation Menu

Repository navigation

docs: upload coverage when CI only tests the modules that changed - #2824

docs: upload coverage when CI only tests the modules that changed#2824
heliocodacy wants to merge 1 commit into
masterfrom
docs-coverage-for-changed-modules-only

heliocodacy commented Nov 10, 2026

Uh oh!

github-actions Bot commented Nov 10, 2026

Uh oh!

codacy-production Bot commented Nov 10, 2026

Uh oh!

codacy-production Bot left a comment

Uh oh!

codacy-production Bot Nov 10, 2026

Uh oh!

codacy-production Bot Nov 10, 2026

Uh oh!

codacy-production Bot Nov 10, 2026

Uh oh!

codacy-production Bot Nov 10, 2026

Uh oh!

This branch was previously deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

heliocodacy commented Nov 10, 2026

👀 Live preview

🚧 To do

Uh oh!

github-actions Bot commented Nov 10, 2026

Uh oh!

codacy-production Bot commented Nov 10, 2026

Up to standards ✅

Uh oh!

codacy-production Bot left a comment

Choose a reason for hiding this comment

Pull Request Overview

Test suggestions

Uh oh!

codacy-production Bot Nov 10, 2026

Choose a reason for hiding this comment

Uh oh!

codacy-production Bot Nov 10, 2026

Choose a reason for hiding this comment

Uh oh!

codacy-production Bot Nov 10, 2026

Choose a reason for hiding this comment

Uh oh!

codacy-production Bot Nov 10, 2026

Choose a reason for hiding this comment

Uh oh!

This branch was previously deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant