cache: shared-memory-backed Dir for fast restart

[masaori335]

Motivation

Cold-start cache initialization rebuilds each stripe's in-memory directory from disk on every
restart — multi-minute on large caches.

Approach

Host the directory (raw_dir) in POSIX shared memory so the next process start attaches the
existing segment in milliseconds instead of rebuilding it. Recovery is binary and fail-safe:
anything untrustworthy (crash, reboot, ABI/schema mismatch, failed validation, bad disk) falls
back to the existing disk-rebuild path, and reads still validate Doc magic + key so a stale
entry is a miss, never served corruption.

New configs & traffic_ctl commands

Opt-in behind proxy.config.cache.shm.enabled (default 0, a functional no-op). Also adds
proxy.config.cache.shm.{name_prefix,use_hugepages,purge_stale_on_start} and a
traffic_ctl cache shm status|clear command.

Details

Design, recovery model, configuration, and platform notes are in
doc/developer-guide/cache-architecture/shm-fast-restart.en.rst (added in this PR) and the
proxy.config.cache.shm.* entries in records.yaml.

Testing

• Unit: test_CacheShm (ABI hash, storage signature, control round-trip, name length, prefix
  normalization, process liveness).
• AuTest: 7 cache_shm_* suites — fast restart, unclean shutdown, schema/storage mismatch,
  bad-disk drop + orphan reclaim, concurrent-attach refusal, purge-on-disable.

Caveats

• No automated test yet exercises the _shm_directory_is_valid() rejection branches or the
  bad-disk / flush-failure invalidate_stripe_directory() paths (planned follow-up).
• macOS concurrent-attach is best-effort: the flock guard is authoritative on Linux but a
  no-op on macOS POSIX shm, where it falls back to an owner-pid liveness check.

[Copilot]

Pull request overview

This PR introduces an opt-in “fast restart” cache directory implementation by hosting each stripe’s in-memory directory (raw_dir) in POSIX shared memory, allowing subsequent traffic_server starts to attach the prior directory quickly instead of rebuilding from disk. It also adds operator tooling (traffic_ctl cache shm status|clear), configuration records, extensive AuTest coverage, unit tests for trust gates, and design/admin documentation.

Changes:

• Add shared-memory directory infrastructure (CacheShm*) plus integration into cache startup, stripe directory allocation, and shutdown/clean-marking paths.
• Add traffic_ctl cache shm status|clear commands and new proxy.config.cache.shm.* records.
• Add unit + AuTest suites and documentation for the shm fast-restart feature.

Reviewed changes

Copilot reviewed 32 out of 32 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
tests/gold_tests/logging/ts_process_handler.py	Hardens psutil cmdline handling for macOS/permission-restricted processes.
tests/gold_tests/cache/shm_poke.py	Adds Linux-only helper to tamper with /dev/shm segments for trust-gate testing.
tests/gold_tests/cache/replay/cache-shm-fast-restart.replay.yaml	Proxy Verifier replay traffic used to validate hit-after-restart behavior.
tests/gold_tests/cache/gold/cache_shm_state_after_shutdown.gold	Gold output for traffic_ctl cache shm status validation.
tests/gold_tests/cache/cache_shm_unclean_shutdown.test.py	AuTest for rejecting dirty segments after SIGKILL and rebuilding from disk.
tests/gold_tests/cache/cache_shm_storage_mismatch.test.py	AuTest for partial attach behavior when storage layout changes.
tests/gold_tests/cache/cache_shm_schema_mismatch.test.py	Linux-only AuTest that pokes schema_version and verifies drop+rebuild.
tests/gold_tests/cache/cache_shm_purge_on_disable.test.py	AuTest for purge-on-disabled-start behavior and traffic_ctl exit codes.
tests/gold_tests/cache/cache_shm_fast_restart.test.py	End-to-end AuTest for clean shutdown -> shm attach -> cache HIT without origin contact.
tests/gold_tests/cache/cache_shm_concurrent_attach.test.py	AuTest for concurrent-attach guard (flock vs owner_pid liveness backstop).
tests/gold_tests/cache/cache_shm_bad_disk_dropped.test.py	AuTest for partial attach and orphan reclaim when a disk is removed from storage.
src/traffic_ctl/traffic_ctl.cc	Adds `traffic_ctl cache shm status
src/traffic_ctl/CMakeLists.txt	Builds the new traffic_ctl command source and adds cache include path for shared headers.
src/traffic_ctl/CacheShmCommand.h	Declares the traffic_ctl cache shm command handler.
src/traffic_ctl/CacheShmCommand.cc	Implements shm status/clear via direct shm_open/mmap, shared purge primitive, and exit codes.
src/records/RecordsConfig.cc	Registers proxy.config.cache.shm.* configuration records.
src/iocore/cache/unit_tests/test_CacheShm.cc	Adds unit tests for ABI hash, storage signature, layout round-trip, prefix normalization, and liveness checks.
src/iocore/cache/StripeSM.cc	Adds fast-attach path that can skip disk dir read + recovery when shm directory is trusted; adjusts shutdown behavior.
src/iocore/cache/Stripe.h	Adds _shm_directory_is_valid() and strengthens flush API via [[nodiscard]].
src/iocore/cache/Stripe.cc	Uses shm-backed directory allocation; adds shm bounds validation; detaches shm mappings safely in destructor.
src/iocore/cache/CMakeLists.txt	Builds new CacheShm.cc and adds test_CacheShm to the cache unit tests.
src/iocore/cache/CacheShmPurge.h	Introduces shared header-only purge/enumerate/unlink primitive for server startup and traffic_ctl.
src/iocore/cache/CacheShmLayout.h	Defines the shared control-segment layout and prefix normalization utilities.
src/iocore/cache/CacheShm.h	Declares the CacheShm facade for lifecycle + stripe attach/create + trust gates.
src/iocore/cache/CacheShm.cc	Implements shm control segment lifecycle, trust gates, stripe attach/create, orphan reclaim, and clean-shutdown marking.
src/iocore/cache/CacheProcessor.cc	Calls CacheShm initialization before stripe construction and finalization after cache init.
src/iocore/cache/CacheDir.cc	Marks shm control segment clean only after shutdown sync has quiesced writers.
src/iocore/cache/AggregateWriteBuffer.h	Marks flush() as [[nodiscard]] to encourage handling short-write failures.
src/iocore/cache/AggregateWriteBuffer.cc	Converts flush failure from an assertion to a false return for graceful handling.
doc/developer-guide/cache-architecture/shm-fast-restart.en.rst	Adds detailed design doc: layout, gates, attach modes, shutdown semantics, tooling, and platform notes.
doc/developer-guide/cache-architecture/index.en.rst	Hooks shm fast-restart document into the cache architecture index.
doc/admin-guide/files/records.yaml.en.rst	Documents new proxy.config.cache.shm.* records and operator-facing behavior/tooling.

[Copilot]

Pull request overview

Copilot reviewed 32 out of 32 changed files in this pull request and generated 2 comments.