test.sh: refuse the cli suite against a real HOME

[lgerard42]

Why

The cli suite installs and uninstalls agent configurations for real — it calls cbm_cmd_install and cbm_cmd_uninstall, not mocks, across 53 call sites.

HOME isolation is per-test rather than global: tests/test_cli.c calls cbm_setenv("HOME", tmpdir, 1) 58 separate times, and scripts/test.sh sets HOME zero times. So the suite acts on whatever HOME it inherits, and the safety of a run depends on every individual test remembering — including the ones that fail partway.

The observable consequence is that results outside isolation are not trustworthy. Running the suite against a real $HOME on my machine gave different failure counts on consecutive runs, and the tests that moved were exactly the agent-config ones — OpenClaw, VS Code, Gemini, Augment, CodeBuddy/Pochi. The same tree under HOME=$(mktemp -d) was stable at the baseline every time.

I am not claiming a specific test damages a real config — I tried to reproduce that and could not, twice, against increasingly faithful copies of the home directory in question. The argument here is narrower and I think uncontroversial: a suite that installs and uninstalls agent configurations should not be pointed at a developer's own one by default, and its results there are unreliable regardless.

What this does

scripts/test.sh exits 2 when the cli suite is selected and $HOME contains .claude or .claude.json, printing the isolated command to use. Default mode (no --suites) is covered too, since it runs every suite. CBM_ALLOW_REAL_HOME=1 overrides for anyone who wants the current behaviour.

CI is unaffected in the normal case: a runner whose $HOME has no .claude never trips it, and the override is available if yours does.

Verified in this tree: refuses --suites cli on a real HOME with exit 2, permits --suites discover there, and permits --suites cli under an isolated HOME.

Happy to gate it differently — a warning instead of a refusal, or a different env var name — if you would rather.

[github-actions]

Thanks for opening this — it has been seen, and it is queued.

This note is automated, but it is not a brush-off: it exists so you know where your PR stands instead of having to guess from silence.

Current review status: working through a backlog. 0.9.1-rc.1 is out, so the release freeze that held reviews is over — but it left a large queue of open pull requests behind it, and we are reading through them oldest-first. The background is in discussion #1144.

What that means for this PR, concretely:

• It will not be closed for inactivity. No stale bot touches pull requests here.
• It may still sit a while before a human reads it. That is on us, not on you.
• Older PRs are read first, so a recent one is not being skipped — it is behind a queue.

Things that will genuinely speed it up whenever review does happen:

• Keep it rebased on main — the tree is moving quickly right now, and a conflicting branch cannot be reviewed as the diff you intended.
• Get CI green, or say which failures you believe are pre-existing.
• Keep the change to one claim. Bundled features and refactors get split before they get merged, which costs you a round trip.
• Every commit needs a sign-off (git commit -s) — CI enforces DCO.

If this fixes a bug, a reproduction we can run is worth more than a description of the symptom.

Thanks for contributing, and sorry in advance for the wait.

[DeusData]

Thank you for the careful framing here, especially for separating a confirmed isolation problem from damage you could not reproduce. I checked current main: scripts/test.sh selects and runs the cli suite without a runner-level HOME isolation guard, while the suite exercises real install and uninstall paths. The safety and result-reliability premise is therefore grounded.

I have labeled this as a high-priority security and maintainer-safety item. The exact policy still needs a maintainer direction decision before detailed review: hard refusal with an escape hatch versus making isolation unconditional at the runner boundary. The current DCO check is also red, so the commit will need a valid Signed-off-by line. Our review queue is full, but this is now classified and routed. Thank you for raising it without overstating the evidence.