Measure CI/CD time to live after the v2.3 work: 1090 s median before #569

Part of #554. Rulings: #554 rulings 1, 2, 3, 4, 5, 7. Work on main. This issue changes no repo file. Its output is a "Measured" comment here and the same text on #554.

Summary

• What. Measure the pipeline after the v2.3 CI/CD work with one fixed method. Compare it with the frozen baseline below.
• Why. The tracker computes about 620 s per (build): deploy after Build the deploy images while the quality gates run #565 lands, and a stretch of about 540 s after set E, from measured steps. This issue is the end-to-end check.
• Before. Time to live (M1) has a median of 1090 s and a p90 of 1386 s, over 14 deploys from 2026-09-22 to 2026-10-20. Run 37431581531 took 1510 s; run 39398067267 took 1132 s.
• Status, 2026-11-12. Window 1 closed at run 42344958557, the 8th qualifying deploy (2026-11-07). FIRST_AFTER is 41608014517. The 10th qualifying row is 42582921035 (2026-11-09), so the p90 was judged too. The window 1 verdict is posted (Measured) and ruled a clear win (ruling). The 620 s median target is missed by 10 s. Window 2 (set E) is not started.
• Owns. The metric definitions, sample rules, statistics rules, baseline, "after" protocol and targets. It is the only source of an M1 number. Every other issue quotes M1 from this issue's stats.ts output only.

Benchmark

Depends on / Blocks

• Depends on Build the deploy images while the quality gates run #565. The "after" sample starts at the first deploy that contains it.
• Depends on Write a timing table to every CI/CD run summary #556. Each run's 📊 Run Report summary is the quick read, and it shows free_gb and load1.
• Reads, does not wait for: Start the extension and webapp unit tests without waiting for backend validation #558 and Prune Docker build cache toward a free-disk target and keep rollback images #564. Every after row contains both. Run only the CI suites a pull request or push can affect #563, because the docs-only push target needs it. Cut the webapp image installs from 158 s to about 25 s per deploy #559, Skip the duplicate type check inside the webapp Docker build #560, Keep the Actions cache under its limit and pin Bun in CI #561 and Stop extension README media from triggering extension test suites #562, because each changes a measured stage.
• Window 2 reads: Run the backend and webapp image builds at the same time, to cut the image build step #567, or the Measure whether the backend image build still needs --no-cache #566 drop commit if Run the backend and webapp image builds at the same time, to cut the image build step #567 reverts. Window 2 starts at that commit (Step 3.6).
• Blocks: nothing. [Tracking] Faster, change-aware CI/CD pipeline #554 and Publish one Bun 1.4.x benchmark report for the marketing team #660 quote M1 only from this issue's stats.ts output.
• Related: Find why the deploy roll-out step (M5) got slower after 2026-11-03 #760 owns the M5 rise. Measure CI/CD time to live after the v2.3 work: 1090 s median before #569 records M5 and does not fix it.

Current behaviour

Links are pinned to d0a53756f8676556358b6adc6c2397d2ed7c64c8. They are the "before" state; #565 moves some on purpose.

• Triage reads only the tip commit subject (prod.docs.plus.yml:108-112; AGENTS.md:416). So the script reads head_commit.message. displayTitle can show the workflow name, as in run 36437965916.
• Every number comes from production runs, read with read-only gh and git calls. Docker timings do not transfer from a laptop (AGENTS.md:293), and a cache fix cannot be measured under --no-cache (AGENTS.md:301). Since Write a timing table to every CI/CD run summary #556, each run's 📊 Run Report summary is the quick read, not the result.

What we measure

• createdAt is 2–6 s after the push. Record queue_s, the first job's startedAt minus createdAt, so a GitHub queue delay stays visible.
• M1 starts at the run created_at, not the attempt created_at. This answers the 2026-11-03 question. On run 41677887088 the two differ by 1 s.
• Today M1 minus (M2 + M3 + M4 + M5) is 19–39 s: checkout, 🔧 Ensure Infrastructure, migrate and verify.
• After Build the deploy images while the quality gates run #565, M1 equals the later of M2g and M2i, plus W, plus the deploy job time.

Extra fields per deploy separate a real gain from a lucky cache.

"Previous deployed SHA" is the last earlier push or workflow_dispatch run whose deploy succeeded on any attempt. A re-run that deployed counts as a base. Example: run 37976785981 is warm only when re-run 37906965365 is its base. Without that base, it reads yes, yes, 81.9 h.

Sample rules

• Count a push run when its 🚀 Deploy Production job ended success on attempt 1.
• Read the subject from head_commit.message (gh api repos/docs-plus/docs.plus/actions/runs/<id> -q .head_commit.message).
• Leave out failed runs, cancelled runs and re-runs. A re-run carries a human gap: run 37906965365, attempt 2, reads 2569 s.
• Split by "Ext suite ran" and by cache class. Warm means gap_h is 24 or less, with no bun.lock change and no extension change. Every other deploy is cold.
• Report a (build): back run on its own row. It runs no front gates, so it stays out of the pooled median.
• Statistics. The median of an even n is the mean of the two middle values, rounded half up. The p90 is nearest-rank: sort ascending, then take item ceil(0.9 × n). Show the p90 only when n is 10 or more.
• A slice with n below 5 is a hint, not a result.
• The baseline sample ends with run 39398067267 (2026-10-20 09:04 UTC).
• The 📊 Run Report line benchmark sample: yes must agree with these rules. If it disagrees, these rules win, and the comment names the run under "Left out".
• Rows from after the Move production to a new Debian 13 droplet #747 host move form a new sample. Never pool them with earlier rows.
• M6 with n = 5. Take the 5 samples from 5 pushes, for example 5 empty commits on a draft PR. Never take a sample from a gh run rerun attempt (house rule). A re-run keeps the run createdAt, so attempts 2 to 5 include the wait since attempt 1.

Baseline: (build): deploys

Seconds after run start, attempt 1. Read on 2026-10-20 with the Step 1 script; free_gb is the "Disk after prune" log value. queue_s was 2–3 s on every row except 35723691814 (11 s).

• The first row has no diff base, so its diff fields are unknown.
• Both webapp installs were CACHED on the three fastest image builds (M3 316–472 s). Neither was CACHED on the other 11 deploys.
• Left out: 37906965365 (re-run), 39274130383 and 38689468498 (the (build):front subject typo), 37904552781 (flaky test), 35723391885 (cancelled).

Baseline Summary

W was 2–3 s on every row, because the image build ran inside the deploy job.

Inside M3, from the production BuildKit logs

Parsed from the logs of six baseline deploy jobs. #559 and #567 now own these parts.

Baseline: PR and non-deploy push runs (M6)

Successful runs from 2026-09-20 to 2026-10-20:

• Every PR ran 5 or 6 Ext suites and the backend unit suite. Backend E2E does not run on PRs.
• Run 39283678757 changed a README and a GIF, yet ran the mermaid suite (194 s) and Build Verification (161 s).

Target behaviour

One "Measured, " comment here and on #554 holds the "after" result. It uses the metrics and rules above. It shows the pooled median, min–max and every slice. It gives a verdict per target and per slice, with run ids. It names the stage behind each miss.

The "after" protocol

1. Measure normal deploys only. Measure whether the backend image build still needs --no-cache #566 is the one exception, with its own procedure.
2. Each change set gets a quick read from the 📊 Run Report of its next one or two deploys. The sibling issue owns it. It is not the result.
3. The sample window starts at the first (build): push run that contains the Build the deploy images while the quality gates run #565 merge commit (Step 3.2). It ends at the 8th qualifying deploy, or 30 days after the start, whichever comes first. Post the median verdict at 8. Judge the p90 at n of 10 or more. Window 1 already holds 10 rows, so one "Measured" comment carries both verdicts.
   • Window 1 ("Measured, Build the deploy images while the quality gates run #565") uses runs 41608014517 to 42344958557 for the median, against 620 s, and through 42582921035 for the p90, against 720 s.
   • Window 2 (set E, Step 3.6) is the only window that judges the 540 s stretch.
4. A change set landing inside the window does not restart the count. Each row records the later change sets it contains.
5. A slice is a clear win when its "after" median is below its "before" minimum: All 777, Ext suite ran 1052, No Ext suite 777, Warm 777, Cold 971.

Targets

The targets are the author's call, from the tracker estimate.

Implementation steps

Work in a scratch folder outside the repo. Never commit the scripts. "Comment here" means a comment on this issue.

Step 0. Read and check the base

On current main, check names and behaviour, not pinned line numbers.

1. Update the clone: git -C <repo> fetch origin main. The script needs every deployed SHA locally.
2. Confirm the two dependencies are on main.

   git -C <repo> log origin/main --format='%H %cs %s' -S '📦 Verify images present' -- .github/workflows/prod.docs.plus.yml | tail -1
   git -C <repo> log origin/main --format='%H %cs %s' -S '📊 Run Report' -- .github/workflows/prod.docs.plus.yml | tail -1

   Each command must print one commit. The first SHA is IMAGES_EARLY_SHA. If either prints nothing, stop and comment here.
3. Confirm each step name exists exactly once.

   for n in '🩺 Verify Deployment' '🏗️ Build Docker Images' '💾 Pre-deploy disk guard' '🚀 Deploy Services (rolling replace)' '🚀 Deploy Production' '🏗️ Build Verification' '📊 Run Report'; do
     printf '%s: ' "$n"; git -C <repo> show origin/main:.github/workflows/prod.docs.plus.yml | grep -cF "name: $n"
   done

   Expect 1 on every line. If a name changed, update its regex in bench.ts and note it in the result comment.
4. Confirm the job names fit the script's regexes. On the first after run, run gh run view <runId> -R docs-plus/docs.plus --json jobs -q '.jobs[].name'.
   • The images job (🧱 Build Images in Build the deploy images while the quality gates run #565) must not match /Deploy Production/ or /Build Verification/.
   • The report job must match /report/i, or M6 counts it.
   • Only 🚀 Deploy Production may match /Deploy Production/.
     If any rule fails, fix the regex in bench.ts and note it.
5. Confirm the parsed log text. With the images job id from Step 0.4, run gh run view -R docs-plus/docs.plus --job <imagesJobId> --log | grep -E 'Disk after prune: [0-9]+ GB free|\[webapp (build-extensions|builder) [^]]*\] RUN bun install'.
   • Expect one Disk after prune line. If Prune Docker build cache toward a free-disk target and keep rollback images #564 rewrote it, change the free_gb regex, or read free_gb from the 📊 Run Report table.
   • Expect both [webapp build-extensions …] and [webapp builder …] install lines. If Cut the webapp image installs from 158 s to about 25 s per deploy #559 renamed a stage, change the install regex.
   • If Run the backend and webapp image builds at the same time, to cut the image build step #567 is on main, install lines start with [front] , which bench.ts matches. Expect each end line to carry the same prefix.
6. Confirm log retention: gh api repos/docs-plus/docs.plus/actions/permissions/artifact-and-log-retention. On 2026-10-20 it printed {"days":90,"maximum_allowed_days":90}, so the 2026-09-22 logs expire about 2026-12-21. Then cached_installs and free_gb read null for old rows; the frozen values stay the baseline.
7. Confirm the window has closed (Step 3.3) before Step 4.

Step 1. Write the two scripts

Write both files in ~/ci-bench/ or another scratch folder. They call only gh and git diff. bench.ts prints one JSON row per run.

// bench.ts: bun bench.ts <repoDir> <runId> ...  → one JSON row per run, oldest first
// Read-only. Diff fields compare each run with the last earlier run in the list that deployed on any
// attempt (a re-run that deployed is a valid base). Pass every push and dispatch run, oldest first.
const REPO = "docs-plus/docs.plus";
const [repoDir, ...ids] = Bun.argv.slice(2);
const sh = (cmd: string[]) => {
  const p = Bun.spawnSync(cmd);
  if (p.exitCode !== 0) throw new Error(`${cmd.join(" ")}: ${p.stderr.toString()}`);
  return p.stdout.toString();
};
// GitHub reports a step that never ran with a zero date (0001-01-01). Treat it as missing.
const ms = (s?: string | null) => (s && !s.startsWith("0001") ? Date.parse(s) : null);
const rows: any[] = [];
let prev: { sha: string; t0: number } | null = null;
for (const id of ids) {
  // Run createdAt comes from the run, not from the attempt: `--attempt 1` reports the attempt's own time.
  const run = JSON.parse(sh(["gh", "api", `repos/${REPO}/actions/runs/${id}`]));
  const d = JSON.parse(sh(["gh", "run", "view", id, "-R", REPO, "--attempt", "1", "--json", "jobs"]));
  const subject = run.head_commit.message.split("\n")[0];
  const t0 = Date.parse(run.created_at);
  const off = (s?: string | null) => { const v = ms(s); return v === null ? null : Math.round((v - t0) / 1000); };
  const dur = (s: any) => { const a = ms(s?.startedAt), b = ms(s?.completedAt); return a === null || b === null ? null : Math.round((b - a) / 1000); };
  // Search the steps of every job: after the images-early change, the image steps live in a new job.
  const all = d.jobs.flatMap((j: any) => (j.steps ?? []).map((s: any) => ({ ...s, job: j })));
  const step = (re: RegExp) => all.find((s: any) => re.test(s.name) && s.conclusion === "success");
  const job = (re: RegExp) => d.jobs.find((j: any) => re.test(j.name));
  const deployJob = job(/Deploy Production/);
  const build = step(/Build Docker Images/);
  const m2g = off(job(/Build Verification/)?.completedAt);
  const m2i = off(build?.completedAt);
  const wStart = off(deployJob?.startedAt);
  const done = d.jobs.filter((j: any) => j.conclusion === "success" && !/report/i.test(j.name));
  const row: any = {
    id, sha: run.head_sha, subject, event: run.event, conclusion: run.conclusion, attempts: run.run_attempt,
    deploy: deployJob?.conclusion ?? null,
    ext: d.jobs.filter((j: any) => /Ext \(/.test(j.name) && j.conclusion === "success").length,
    queue_s: Math.min(...d.jobs.map((j: any) => off(j.startedAt) ?? Infinity)),
    m1: off(step(/Verify Deployment/)?.completedAt),
    m2g, m2i,
    m3: dur(build), m4: dur(step(/disk guard/)), m5: dur(step(/Deploy Services/)),
    m2: wStart,
    // Before images-early the image build runs inside the deploy job, so W is the gap after M2g only.
    w: wStart === null || m2g === null ? null
      : build?.job.databaseId === deployJob?.databaseId ? wStart - m2g : wStart - Math.max(m2g, m2i ?? 0),
    m6: done.length ? Math.max(...done.map((j: any) => off(j.completedAt) ?? 0)) : null,
    typecheck: d.jobs.some((j: any) => /Type Check/.test(j.name) && j.conclusion === "success"),
    obs: d.jobs.some((j: any) => /Deploy Observability/.test(j.name) && j.conclusion === "success"),
    lock_changed: null, ext_src_changed: null, gap_h: null, cached_installs: null, free_gb: null,
  };
  if (build) {
    const log = sh(["gh", "run", "view", "-R", REPO, "--job", String(build.job.databaseId), "--log"]).split("\n");
    let hits = 0;
    for (const [i, line] of log.entries()) {
      const m = line.match(/(\[(?:front|backend)\] )?(#\d+) \[webapp (build-extensions|builder) [^\]]*\] RUN bun install/);
      if (!m) continue;
      const [, pre = "", num, stage] = m;
      if (log.slice(0, i).some((l) => l.includes(`${pre}${num} [webapp ${stage} `))) continue;
      const end = log.slice(i + 1).find((l) => l.includes(`${pre}${num} CACHED`) || l.includes(`${pre}${num} DONE`));
      if (end?.includes("CACHED")) hits++;
    }
    row.cached_installs = hits;
    const free = log.map((l) => l.match(/Disk after prune: (\d+) GB free/)).find(Boolean);
    row.free_gb = free ? Number(free[1]) : null;
  }
  if (prev) {
    const same = (path: string) => {
      const code = Bun.spawnSync(["git", "-C", repoDir, "diff", "--quiet", prev!.sha, run.head_sha, "--", path]).exitCode;
      if (code !== 0 && code !== 1) throw new Error(`git diff failed for ${id}; run git fetch origin main`);
      return code === 0;
    };
    row.lock_changed = !same("bun.lock");
    row.ext_src_changed = !same("extensions/");
    row.gap_h = Math.round(((t0 - prev.t0) / 3600000) * 10) / 10;
  }
  rows.push(row);
  const last = run.run_attempt > 1
    ? JSON.parse(sh(["gh", "run", "view", id, "-R", REPO, "--json", "jobs"])).jobs.find((j: any) => /Deploy Production/.test(j.name))
    : deployJob;
  if (last?.conclusion === "success") prev = { sha: run.head_sha, t0 };
}
console.log(JSON.stringify(rows, null, 1));

stats.ts applies the sample and statistics rules. --from and --to fail on an unknown run id. --before adds the before minimum and clear-win verdict per slice.

// stats.ts: bun stats.ts rows.json [--from <runId>] [--to <runId>] [--before before.json]  → Markdown
// Applies the sample rules and the statistics rules to bench.ts output. --from and --to bound the window.
const args = Bun.argv.slice(2);
const opt = (k: string) => { const i = args.indexOf(k); return i < 0 ? null : args[i + 1]; };
const load = async (p: string) => JSON.parse(await Bun.file(p).text()) as any[];
const at = (rows: any[], id: string) => {
  const i = rows.findIndex((r) => r.id === id);
  if (i < 0) throw new Error(`run ${id} is not in the input`);
  return i;
};
const cut = (rows: any[], from: string | null, to: string | null) =>
  rows.slice(from ? at(rows, from) : 0, to ? at(rows, to) + 1 : rows.length);
const backOnly = (r: any) => /^\(build\):/.test(r.subject) && !/front/.test(r.subject);
const warm = (r: any) => r.gap_h !== null && r.gap_h <= 24 && r.lock_changed === false && r.ext_src_changed === false;
// Sample rule: a push whose deploy job ended success on attempt 1. bench.ts reads attempt 1 only.
const pooled = (rows: any[]) => rows.filter((r) => r.event === "push" && r.deploy === "success" && r.m1 !== null && !backOnly(r));
const slices = (rs: any[]): [string, any[]][] => [
  ["All", rs],
  ["Ext suite ran", rs.filter((r) => r.ext > 0)],
  ["No Ext suite", rs.filter((r) => r.ext === 0)],
  ["Warm", rs.filter(warm)],
  ["Cold", rs.filter((r) => !warm(r))],
];
const median = (v: number[]) => {
  const s = [...v].sort((a, b) => a - b), n = s.length, h = Math.floor(n / 2);
  return n === 0 ? null : n % 2 ? s[h] : Math.floor((s[h - 1] + s[h]) / 2 + 0.5); // even n: round half up
};
const p90 = (v: number[]) => { const s = [...v].sort((a, b) => a - b); return s[Math.ceil(0.9 * s.length) - 1]; };
const col = (rs: any[], k: string) => rs.map((r) => r[k]).filter((x) => typeof x === "number");

const rows = cut(await load(args[0]), opt("--from"), opt("--to"));
const before = opt("--before") ? new Map(slices(pooled(await load(opt("--before")!)))) : null;
console.log("| Slice | n | M1 median | M1 p90 | M1 min–max | M2 median | M2g median | M2i median | W median | M3 median |" + (before ? " Before min | Clear win |" : ""));
console.log("|---|---|---|---|---|---|---|---|---|---|" + (before ? "---|---|" : ""));
for (const [name, rs] of slices(pooled(rows))) {
  const m1 = col(rs, "m1"), med = median(m1);
  const p = m1.length >= 10 ? String(p90(m1)) : `— (max ${m1.length ? Math.max(...m1) : "—"})`;
  const range = m1.length ? `${Math.min(...m1)}–${Math.max(...m1)}` : "—";
  let tail = "";
  if (before) {
    const bmin = Math.min(...col(before.get(name)!, "m1"));
    const verdict = med === null ? "—" : med < bmin ? (rs.length < 5 ? "yes (hint, n<5)" : "yes") : "no";
    tail = ` ${bmin} | ${verdict} |`;
  }
  console.log(`| ${name} | ${rs.length} | ${med ?? "—"} | ${p} | ${range} | ${median(col(rs, "m2")) ?? "—"} | ${median(col(rs, "m2g")) ?? "—"} | ${median(col(rs, "m2i")) ?? "—"} | ${median(col(rs, "w")) ?? "—"} | ${median(col(rs, "m3")) ?? "—"} |${tail}`);
}
for (const r of rows.filter((r) => r.event === "push" && r.deploy === "success" && backOnly(r)))
  console.log(`\n(build): back row: run ${r.id}, M1 ${r.m1} s`);

Check: bun --version prints 1.4.0 or later.

M1 starts at the run created_at, not the attempt created_at. bench.ts already reads run.created_at.

Step 2. Prove the scripts on the baseline

Run this step before 2026-12-15, so all 14 baseline logs are still readable. Run the scripts on every baseline push run, oldest first. Re-run 37906965365 must be in the list, as a diff base.

cd ~/ci-bench
gh run list --repo docs-plus/docs.plus --workflow prod.docs.plus.yml --event push --limit 400 \
  --json databaseId,createdAt > pushes.json
bun -e 'const r = JSON.parse(await Bun.file("pushes.json").text());
console.log(r.filter((x) => x.createdAt >= "2026-09-22T11:50:00Z" && x.createdAt <= "2026-10-20T09:05:00Z")
  .reverse().map((x) => x.databaseId).join(" "))' | xargs bun bench.ts <repo> > before.json
bun stats.ts before.json

• Pass: every slice row equals the Baseline Summary to the second, and every before.json row matches the 14-row table. This passed on 2026-10-20.
• If the 2026-09 logs have expired (Step 0.6), compare timing fields only.
• If any timing value differs, stop and comment here with the row.

Step 3. List the after runs

1. List every push and workflow_dispatch run from 39398067267 to now, oldest first, so the first after deploy has its real diff base.

   gh run list --repo docs-plus/docs.plus --workflow prod.docs.plus.yml --limit 400 \
     --json databaseId,createdAt,event -q '[.[] | select(.event == "push" or .event == "workflow_dispatch")]' > pushes.json
   bun -e 'const r = JSON.parse(await Bun.file("pushes.json").text());
   console.log(r.filter((x) => x.createdAt >= "2026-10-20T09:04:00Z").reverse().map((x) => x.databaseId).join(" "))' \
     | xargs bun bench.ts <repo> > after-all.json

   Dispatch rows are diff bases only; stats.ts keeps push rows. List each under "Left out" as "A/B dispatch (Measure whether the backend image build still needs --no-cache #566)".
2. FIRST_AFTER is the first push row whose subject starts with (build): and for which git -C <repo> merge-base --is-ancestor <IMAGES_EARLY_SHA> <row sha> && echo contains prints contains.
3. LAST_AFTER. A qualifying row has event push, deploy success, a numeric m1, and a subject that starts with (build): and contains front. A (build): back row does not count; stats.ts prints it on its own line. Count from FIRST_AFTER, oldest first.
   • LAST_AFTER is the 8th qualifying row, or, if day 30 comes first, the last qualifying row created before FIRST_AFTER createdAt plus 30 days. Read createdAt from pushes.json.
   • END_TIME is the createdAt of LAST_AFTER. The second comment uses the same rule with 10 in place of 8. If day 30 comes first, it reports "p90 not measured, n = ".
   • Check: bun stats.ts after-all.json --from <FIRST_AFTER> --to <LAST_AFTER> prints an All row with n of 8, or the day-30 count.
4. Tag each qualifying row with the later change sets it contains. Find each merge SHA, then run git -C <repo> merge-base --is-ancestor <merge SHA> <row sha> && echo contains.

   git -C <repo> log origin/main --format='%H %cs %s' -S 'wait "${BACK_PID}"' -- .github/workflows/prod.docs.plus.yml | tail -1   # parallel-compose
   git -C <repo> log origin/main --format='%H %cs %s' -S 'outputFileTracingExcludes' -- apps/webapp/next.config.js | tail -1   # build-traces trim
   git -C <repo> log origin/main --format='%H %cs %s' -S 'backend_no_cache' -- .github/workflows/prod.docs.plus.yml | head -1   # backend-nocache-ab Step 6, input removed

   An empty line means the change set is not on main; tag it "no". If a sibling used another string, read its merge commit there.
   • The parallel-compose string wait "${BACK_PID}" exists only in Run the backend and webapp image builds at the same time, to cut the image build step #567 Step 1b. A Step 1a commit must not match it.
   • For Measure whether the backend image build still needs --no-cache #566: if git -C <repo> show origin/main:.github/workflows/prod.docs.plus.yml | grep -c backend_no_cache prints more than 0, the A/B test is still open. Tag every row "A/B only".
   • Otherwise the head -1 commit is its Step 6 change. Tag a row 'drop' only when it contains that commit and the maintainer ruled 'drop' there. A 'keep' ruling changes no deploy, so tag it 'no change'.
5. Tag the confounds. Give each row obs, db_region and traefik_timeout. Report the All slice twice: with and without the traefik_timeout rows.
6. Set E window (window 2). It starts at the first (build): push that contains the Run the backend and webapp image builds at the same time, to cut the image build step #567 commit, or the Measure whether the backend image build still needs --no-cache #566 drop commit if Run the backend and webapp image builds at the same time, to cut the image build step #567 reverts. It ends at the 5th qualifying deploy. Use the same stats.ts call with --from and --to set to those runs. Only this comment judges 540 s. Rows after the Move production to a new Debian 13 droplet #747 move start a new sample.
   • Judge the Run the backend and webapp image builds at the same time, to cut the image build step #567 effect on M2i and M3. Report the deploy job time by db_region beside the 540 s verdict. Name the London and PostgreSQL 18.6 move (2026-11-11) as a confound in the window 2 Claim.
7. If the window has not ended (Step 3.3), stop and come back later.

Step 4. Add free_gb and load1

• For each qualifying row, read free_gb and load1 from the 📊 Run Report table at https://github.com/docs-plus/docs.plus/actions/runs/<id>.
• The bench.ts free_gb is a cross-check. If they differ, use the report value and note it. If the report shows n/a, use the bench.ts value and say so.
• If load1 reads n/a, record n/a. Do not read the host.

Step 5. Compute the result

1. Deploys: bun stats.ts after-all.json --from <FIRST_AFTER> --to <LAST_AFTER> --before before.json. The p90 prints only at n of 10 or more; judge it in the second comment.
2. M6. Run bench.ts on the PR runs in the same window.

   gh run list --repo docs-plus/docs.plus --workflow prod.docs.plus.yml --event pull_request --status success --limit 200 \
     --json databaseId,createdAt -q '[.[] | select(.createdAt >= "<FIRST_AFTER createdAt>" and .createdAt <= "<END_TIME>")] | reverse | .[].databaseId' \
     | xargs bun bench.ts <repo> > after-pr.json

   • PR median: the median of m6 over after-pr.json. Also report how many PRs ran 0 Ext suites.
   • Root docs-only push: an after-all.json row between FIRST_AFTER and LAST_AFTER with event push, conclusion success, deploy not success, typecheck false and ext 0. A push can hold more than one commit, so confirm with git -C <repo> diff --name-only <previous push row sha> <row sha> that every changed file is docs-only. Report its m6, or "no sample".
3. Name the stage that missed, for every missed target. The later of the M2g and M2i medians is the long path. A W median above 10 s (an author threshold) means the prod runner queued the deploy. The rest is the deploy job: M1 minus the later of M2g and M2i, minus W. Split the deploy job time by db_region.

Step 6. Draft the result comment

Post it as this issue's SP3 (#554) record. This layout extends the SP3 template with the window, the row table and the target verdicts. Draft it in the scratch folder. Use stats.ts output only; do not hand-compute medians.

## Measured, <YYYY-MM-DD>

Setup: gates on GitHub-hosted ubuntu-latest; images and deploy on the 4-vCPU self-hosted production runner; window <date> to <date>; Bun <version>; n = <n>.
Window: <FIRST_AFTER> (<date>) to <last run id> (<date>). n = <n> qualifying deploys.
Left out: <run id (reason)>, …

| Run | Date | Ext suites | M2g | M2i | W | M4 | M3 | M5 | **M1** | queue_s | lock_changed | ext_src_changed | gap_h | cached_installs | obs | db_region | traefik_timeout | free_gb | load1 | Class | Later sets |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|

<stats.ts output with --before>

| Target | Before | After | Verdict |
|---|---|---|---|
| M1 median ≤ 620 s (stretch 540 s, rows with set E) | 1090 | | |
| M1 p90 ≤ 720 s (n ≥ 10) | 1386 | | |
| M6 PR median lower | 519 | | |
| M6 root docs-only push ≤ 120 s | 217 | | |
| M5 record (missed stage if the after median is above 70 s) | 59 (57–70, n = 14) | | |

Missed stage: <M2g / M2i / W / deploy job, with medians>, or "none".
Stop condition: <not met / met, with the value>.

Review checkpoint

• git -C <repo> status --short must show no change from this work.
• Show the maintainer the draft comment, before.json and after-all.json. Then stop.
• After the maintainer reads it, post it here and on [Tracking] Faster, change-aware CI/CD pipeline #554, with a link back here.
• No git add, git commit, git push, git stash or branch.

Acceptance criteria

• Step 0 found IMAGES_EARLY_SHA and the 📊 Run Report commit on main, and every step name once (Measured).
• Step 2 reproduced the Baseline Summary to the second, or the comment states which fields the expired logs blocked (Measured).
• FIRST_AFTER is the first (build): push run that contains IMAGES_EARLY_SHA, checked with git merge-base --is-ancestor (Measured).
• LAST_AFTER follows Step 3.3, and stats.ts ran with --from <FIRST_AFTER> --to <LAST_AFTER> (Measured).
• Every left-out run in the window is listed with its reason (Measured).
• Every qualifying row shows every metric and extra field, or n/a (Measured).
• All, Ext suite ran, No Ext suite, Warm and Cold each show n, median, min–max and the clear-win verdict. Each slice with n below 5 is marked as a hint (Measured).
• Each of the four targets has a verdict, and each miss names its stage (Measured).
• The p90 verdict is in the window 1 Measured comment, which carries both verdicts (2026-11-12) (Measured).
• The same comment exists on [Tracking] Faster, change-aware CI/CD pipeline #554, with a link back here (copy).
• Window 2 (set E) has its own comment against 540 s, or the stretch row reads "not measured". It reads "not measured" (close rule, 2026-11-14).
• A "Correction, 2026-11-11" comment is posted. Under the statistics rule, the 2026-11-04 interim median of 6 rows is 622 s, not 621.5 s (Correction).
• git status --short shows no change from this work (read 2026-11-14).

Verification

• No workflow or triage code changes here, so actionlint with .github/actionlint.yaml, bash .github/scripts/parse-build-trigger.test.sh and a dry_run_subject dispatch do not apply.
• Script check: Step 2.
• The measurement: read with bench.ts, stats.ts and the 📊 Run Report table. M1 and M6 rows are in Targets; clear-win minimums are in the protocol.

Accountability

• Who. Agent in the repo: reads the runs and drafts the comment. Maintainer: reads the draft before posting and rules on the verdict.
• When. At the 8th qualifying deploy or day 30, whichever comes first. A second comment follows at n of 10 for the p90. Window 1 already holds 10 rows, so the window 1 comment carries both verdicts. Window 1 was posted and ruled on 2026-11-12. Window 2 posts after its 5th qualifying deploy.
• Close rule (ruled 2026-11-12). Measure CI/CD time to live after the v2.3 work: 1090 s median before #569 closes after the window 2 comment. If Run the backend and webapp image builds at the same time, to cut the image build step #567 Step 1b is not on main by the v2.3.6 release audit, the stretch row reads "not measured", and Measure CI/CD time to live after the v2.3 work: 1090 s median before #569 closes.
• Stop condition, which means "revert". Any one of these:
  • the after All median of M1 is above 1090 s;
  • a slice with n of 5 or more has an after median above its before median: Ext suite ran 1354, No Ext suite 994, Warm 946, Cold 1132;
  • any qualifying after deploy has M1 above 1510 s, the before maximum.
• When it is met: name the missed stage, from M2g, M2i and W. Comment on Build the deploy images while the quality gates run #565 and recommend the revert of the Build the deploy images while the quality gates run #565 commit only. The maintainer decides. Change no workflow from this issue.
• Disk check. If any after row has free_gb below 20, comment on Prune Docker build cache toward a free-disk target and keep rollback images #564 with the run id and the value.

Rollback

• This issue changes no repo file and no host.
• Fix a wrong result with a new comment headed "Correction, " that names the wrong and right value. Do not delete the first comment.
• On the stop condition, Build the deploy images while the quality gates run #565 reverts per SP4 ([Tracking] Faster, change-aware CI/CD pipeline #554): revert only IMAGES_EARLY_SHA, and keep Prune Docker build cache toward a free-disk target and keep rollback images #564. The next run reads the old workflow from the new tip.

Docs to update

None. #557 owns the stale CACHED and deploy-gap facts in AGENTS.md §Docker.

Out of scope

• Extra deploys to make numbers (ruling 4), a dashboard, a metrics stack or a new CI vendor (ruling 5).
• Committing the scripts, or any workflow edit. Write a timing table to every CI/CD run summary #556 owns the in-run timing table.
• Fixing a missed target here. Comment on the owning sibling issue.
• Laptop timings, and judging a cache fix under --no-cache.
• Reading or touching the prod host. Do not ssh.
• Reopening the GHCR question. The tracker reopens it only when free disk stays under the Prune Docker build cache toward a free-disk target and keep rollback images #564 target across a week of deploys.
• Skipping the backend images on (build): front (ruling 7), Turbopack, a BuildKit cache mount with a 168h prune, and a deploy gate on chatroom E2E. The tracker rejected or deferred each one.

Read first

• [Tracking] Faster, change-aware CI/CD pipeline #554, Build the deploy images while the quality gates run #565 and Write a timing table to every CI/CD run summary #556, in full.
• AGENTS.md §Docker (lines 286–307 at the pinned commit) and §Learned Workspace Facts (line 416).
• .github/workflows/prod.docs.plus.yml on current main: the images, deploy and report jobs.
• .github/WORKFLOWS.md, for the current job order.