Skip to content

test(coverage): score and enumerate the tests/helpers*.mjs harness modules - #1294

Merged
mrbobbytables merged 1 commit into
mainfrom
quality/test-harness-helper-modules
Oct 11, 2026
Merged

mrbobbytables merged 1 commit into
mainfrom
quality/test-harness-helper-modules

Conversation

@hivecommons-hive

Copy link
Copy Markdown
Contributor

Test Improvement

tests/tools/coverage-report.mjs treated tests/tools/ as the whole of the harness. SOURCE_ROOTS is a list of trees, so the shared helper modules that sit directly in tests/ could be named by neither --require-source-files nor the --check-harness* floors — 1,348 lines of the measuring apparatus (the script and gh sandboxes, the fetch mock, the JSX driver, the JSONC reader) enumerated and scored by nothing. A helper nothing imports produced no coverage record, and a file with no record is absent from the report rather than 0%.

What this changes

  • isHarnessModule() — the file-level companion SOURCE_ROOTS cannot be: /^tests\/helpers[\w.-]*\.mjs$/, minus .test.mjs. The suite's own tests are what the harness is measured against, not part of it.
  • isHarnessFile() matches it as well as HARNESS_ROOTS, so the helpers enter the harness files row and its three floors.
  • enumerateSourceFiles() lists them one level deep beside the walked trees — guarded against symlinks and a missing tests/ exactly as the walk is — so --require-source-files fails on a helper nothing imports.
  • The two comment blocks the issue names are updated so the stated scope and the code still agree.

Contract

New tests/coverage-report-harness-modules.test.mjs (6 tests): what the predicate matches and does not, that a helper is harness but never source, that the enumeration lists helpers and skips .test.mjs/nested/non-module/symlinked paths, that a missing tests/ is an absence rather than a throw, that an unimported helper is a missing source file, and that the predicate is not vacuous in this repository.

tests/coverage-report-source-files.test.mjs asserted that nothing under tests/ outside tests/tools/ was enumerated. That assertion is narrowed to its actual intent — no *.test.mjs, nothing that is not harness — and gains a positive check that a helper module is listed.

Arithmetic, which is why this landed now

Issue #1258 deferred itself because the then-minimum per-file harness region percentage (96.67%) would have gone red against #1250's ratchet to 97. #1250 is merged, and at 4184f33 the minimum is tests/helpers-script-sandbox.mjs at 97.06, which clears --check-harness-file-regions 97. No floor in package.json is changed by this PR.

Verified locally at 4184f33, node v26.10.0, TZ=UTC, npm ci:

  • npm run test:unit:coverage:check → exit 0.
    • src files | 100.00 | 100.00 | 9135/9135 lines | 2753/2753 regions (unchanged)
    • harness files | 100.00 | 99.41 | 5315/5315 lines | 1869/1880 regions (was 3911/3911 lines, 1471/1481 regions — the helper tree is now scored)
  • npm run check:format → All matched files use Prettier code style!

Related Issue

Closes #1258 — all three boxes in that issue's task list are implemented here, and nothing in it is left for the issue to track.


Filed by quality agent (hold-gated mode). Human review required.

— hive: agent=quality backend=copilot model=claude-opus-5 copilot=1.0.88

…dules

`SOURCE_ROOTS` is a list of trees, so the shared helper modules that sit
directly in `tests/` could be named by neither `--require-source-files` nor
the `--check-harness*` floors: 1,348 lines of the measuring apparatus -- the
script and gh sandboxes, the fetch mock, the JSX driver, the JSONC reader --
were enumerated and scored by nothing. A helper nothing imports produced no
coverage record, and a file with no record is absent from the report rather
than 0%.

`isHarnessModule()` is the file-level companion `SOURCE_ROOTS` cannot be.
`isHarnessFile()` now matches it as well as `HARNESS_ROOTS`, so the helpers
enter the `harness files` row and its three floors, and
`enumerateSourceFiles()` lists them one level deep beside the walked trees,
so `--require-source-files` fails on a helper nothing imports. `.test.mjs`
is excluded: the suite's own tests are what the harness is measured
against, not part of it.

`npm run test:unit:coverage:check` stays green at the floors already in
`package.json` (node v26.10.0, TZ=UTC): `harness files | 100.00 | 99.41 |
5315/5315 lines | 1869/1880 regions`, and the minimum per-file harness
region percentage is `tests/helpers-script-sandbox.mjs` at 97.06, clearing
`--check-harness-file-regions 97`.

`tests/coverage-report-source-files.test.mjs` asserted that nothing under
`tests/` outside `tests/tools/` was enumerated; that assertion is narrowed
to its actual intent -- no `*.test.mjs`, nothing that is not harness -- and
gains a positive check that a helper module is listed.

Signed-off-by: quality <quality@hive.kubestellar.io>
@hivecommons-hive

Copy link
Copy Markdown
Contributor Author

Important

Held for human review by the hive's ACMM level gate.

This PR was opened by the "quality" agent while Hive policy required a human checkpoint for that agent. Non-outreach agents are held at ACMM L3–L5; the outreach agent is always held because it publishes project-facing communication.

Hive will keep the hold label until a human removes it. Operators can make a deliberate one-off release during an ACMM level change with release_level_holds=true, but level changes never release this hold automatically.

@mrbobbytables
mrbobbytables added this pull request to the merge queue Oct 11, 2026
Merged via the queue into main with commit 3b06b8c Oct 11, 2026
7 checks passed
mrbobbytables pushed a commit that referenced this pull request Oct 11, 2026
…nd src/ already measure

`test:unit:coverage:check` is the gate ci.yml runs, and three of its eight
floors sat below what the suite achieves. scripts/ and src/ are at 2753/2753
regions -- every region executed, every file in both trees at 100.00% on its
own -- while the gate asked for 99% in aggregate and 97% per file, so 27 source
regions could go dark before a run turned red.

tests/coverage-gate-thresholds.test.mjs states the rule this breaks in its own
header: raise these floors in the same change when coverage climbs. The climb
happened and the ratchet did not; its recorded measurement was stale by 278
regions (2475/2476 against 2753/2753 today).

  --check-regions             95 -> 96   (measured 96.12)
  --check-source-regions      99 -> 100  (measured 100.00)
  --check-source-file-regions 97 -> 100  (lowest source file 100.00)

Unlike the browser gate, which stays under its ceiling because 18 regions are
uncreditable (#1256), the unit source number has no residual list, so 100 is
the measured floor rather than an aspiration. Measured twice -- node v22.21.1,
the major ci.yml pins, and node v26.10.0 -- because region counts are a V8
artifact and a floor at exactly 100 is only safe if it does not move with the
runtime. The two runs agree to the region.

The two --check-harness-* floors are deliberately left at 97: #1294 rescopes
harness scoring to take in tests/helpers*.mjs, whose lowest file is 97.06%, and
ratcheting them before that lands would collide with it.

CONTRIBUTING.md and AGENTS.md restate these thresholds in prose and are
gated by tests/docs-threshold-parity.test.mjs, so they move in this commit too.

Closes #1297

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: quality <quality@hive.kubestellar.io>
mrbobbytables added a commit that referenced this pull request Oct 11, 2026
Rebased onto main: src is 2777/2777 regions, the harness row now includes
tests/helpers*.mjs (5336/5336 lines, 1867/1875 regions, lowest file
helpers-script-sandbox.mjs at 97.06), and e2e-coverage-report.mjs's ceiling
is 98.57 with 7 residual regions. All three ratcheted floors still clear.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Bob Killen <bkillen@linuxfoundation.org>
@mrbobbytables
mrbobbytables deleted the quality/test-harness-helper-modules branch October 12, 2026 13:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[quality] tests/helpers*.mjs is outside every coverage floor and outside --require-source-files

1 participant