Skip to content

docs: test strategy document — classification, authoring guide, and CI pipeline map #853

@planetf1

Description

@planetf1

Summary

We have test/MARKERS_GUIDE.md as a marker reference, but there is no single document that explains the overall test strategy to a new contributor. Markers are just one aspect — we need a document that covers the full picture.

What's missing

A top-level test strategy document (TEST.md or docs/docs/guide/testing.md) that addresses:

  • Philosophy — what we test, what we don't, and why (e.g. test contracts not implementations)
  • Classification — how to decide between unit / integration / e2e / qualitative with concrete decision rules
  • Authoring guide — how to write a good test: fixture discipline, mock discipline, assertion style, naming
  • When tests run — the CI pipeline tiers (pre-commit, PR CI, nightly, pre-release) and what runs at each stage
  • Local dev workflow — how to run subsets, interpret results, use -rs for skip reasons
  • Examples — opt-in # pytest: comment system, when examples become tests
  • Backend/resource gating — how predicates.py works, when to add a resource gate

Relationship to existing docs

  • test/MARKERS_GUIDE.md is the marker reference — it may be renamed (e.g. test/TESTING_GUIDE.md or folded into the strategy doc) since markers are just one section of a broader guide
  • AGENTS.md §3 and §10 are brief summaries that should point to this doc rather than duplicate it

Acceptance criteria

  • Strategy document exists at an agreed path (discuss: TEST.md root vs docs/docs/guide/testing.md)
  • Covers all sections listed above
  • test/MARKERS_GUIDE.md either renamed/consolidated or cross-linked
  • AGENTS.md §3/§10 updated to point to the new doc
  • markdownlint clean (if placed under docs/docs/)

Parent

Part of #726 — Epic: Testing Infrastructure & Strategy Overhaul

Metadata

Metadata

Assignees

Labels

documentationImprovements or additions to documentationtesting

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions