[FEAT]: Automated quality gate for AI-generated Rails code

[rodrigovirgilio]

Thank you for taking the time to help us improve the app!

• I have searched the existing issues

Is there a pain point in the app driving this feature request?

Human Essentials is a large monolith (70+ controllers, 60+ models, 60+ services) with a lot of code generated and maintained with the help of AI coding assistants. Today the only automated safeguards are CI runs of RuboCop (with most Metrics cops disabled via Standard), Brakeman (no threshold) and RSpec. There is no single command an AI agent or contributor can run to know whether a change is safe to ship beyond "the tests pass".

Desired Behavior

Implement an automated quality gate for the Rails app, following the "Rails Playbook" described by Paulo Tarso (CodeMiner42):

1. Test coverage (SimpleCov) — line and branch coverage with a ratcheted threshold.
2. Complexity / method & module sizes (RuboCop Metrics cops + Flog) — ceiling on cyclomatic complexity, ABC size, method/class/module length for new code.
3. Mutation testing (Mutant) — kill ratio as the "truth detector" for whether tests assert real behavior; threshold ratcheted from the first run.
4. Security (Brakeman) — JSON report parsed into the gate with a hard warnings_max threshold.
5. N+1 detection (Bullet) — Bullet.raise = true wired into the spec suite so N+1s fail the gate.

All of it collapses into one command: bin/rake quality, with a fast local variant (bin/rake quality:local, everything except mutation) and a full CI variant. A config/quality_thresholds.yml holds every threshold, ratcheted on first run. Diff-aware execution (e.g. mutant --since main) should be used where full-suite runs are infeasible for a codebase this size.

Criteria for Completion

• bin/rake quality runs coverage, rubocop metrics, flog, mutation, brakeman and reports a pass/fail table with thresholds from config/quality_thresholds.yml
• bin/rake quality:local runs everything except mutation (~15s) for fast AI/contributor iteration
• Thresholds are ratcheted on first run (no hardcoded impossible numbers on a codebase this size)
• Bullet enabled in the test suite and N+1 regressions fail the gate
• Brakeman warnings are gated (new warnings fail) via JSON parsing
• GitHub Actions workflow runs the full gate on PRs and uploads tmp/quality/ artifacts for inspection
• CLAUDE.md documents that the gate must be run before declaring a task complete

Anything else?

Reference articles (the playbook this issue implements):

• https://blog.codeminer42.com/stop-reading-ai-code-start-measuring-it-a-rails-playbook/
• https://blog.codeminer42.com/how-far-can-ai-self-validate-rails-code/

The follow-up post also documents two gates the author deliberately removed (suite-wide allocations via GC.stat and per-action SQL counts via ActiveSupport::Notifications) because they are "continuous-drift" numbers that only produce threshold-bumping rituals — those should NOT be implemented. Packwerk/dependency-structure gates were also skipped by the author as architecture-theatre for a Rails monolith.

• I've read the Code of Conduct and understand my responsibilities as a member of the Ruby for Good community

[dorner]

@rodrigovirgilio thank you for the suggestion. However, we aren't quite at the point where we're overwhelmed by AI contributions right now. Don't get me wrong, I really wish people wouldn't use AI to write their PR and issue descriptions (hint hint) but the actual code is fine. We still rely on human code review, and we're still at a few PRs a week, well within our limits.

I'd rather not introduce a whole whack of additional gates that we have to maintain and ensure don't get flaky. This is solving a problem we don't have as of now.

[dorner]

cc: @awwaiid what do you think?

[awwaiid]

Yes... I agree. Looking at the PR, I don't see anything particularly objectionable individually, but doing it as one whole thing and then having all of everything failing all at once isn't immediately helpful.

You don't need an issue for a PR, but they are good for discussion. If you're passionate about adding some more of these checks -- like mutation checking or bullet compliance -- then better would be to do it in much smaller steps and maybe with ratcheting (like a way to incrementally move toward compliance one PR at a time), then that would be ok. I'm going to go ahead and close the large red PR, but in case @rodrigovirgilio wants to discuss further we can leave this issue here for comments for a bit.