fix: exit with _exit after a requested shutdown (low priority) - #197
Merged
Merged
Conversation
Low priority: only affects stopping the server while a request is in flight.
The shutdown handlers called Darwin.exit(0), which runs C++ static
destructors and atexit handlers while an inference thread may still be
inside an MLX GPU eval. Sending SIGTERM mid-generation therefore crashed
right after the exiting{reason:"requested"} event (SIGSEGV in
CustomKernel::eval_gpu's kernel map, or SIGABRT), leaving a crash report
and a non-zero exit status for what the Engine Protocol reports as a
clean stop.
Flush stdout/stderr and _exit(0) instead. Nothing needs those destructors
at this point, and the exiting event is already flushed by emitEvent.
Repro (Qwen3.6-35B-A3B, --stream-experts --ssd-prefetch): SIGTERM 25 s
into a streaming 2000-token generation. Before: 3/3 crashed (exit 139/134,
one .ips each). After: 3/3 exit 0, no crash report, exiting event present.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Nn7Du4yB5jtocihUNxxBGs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Low priority. This only affects stopping the server while a request is in flight. Normal operation is unaffected.
Problem
Both shutdown handlers (SIGTERM and SIGINT) emit
exiting{reason:"requested"}and then callDarwin.exit(0).exit()runs C++ static destructors andatexithandlers while an inference thread may still be inside an MLX GPU eval. So stopping the server mid-generation crashes right after it announces a clean shutdown:mlx::core::fast::CustomKernel::eval_gpu, dereferencing its kernelunordered_mapafter it's destroyed. One case was SIGABRT instead..ipsfile for every stop.A daemon following the Engine Protocol sees
exiting{requested}, then a crash, for what should be a clean stop.Change
Flush stdout and stderr, then call
_exit(0). Nothing needs those destructors at that point, andemitEventhas already flushed the exiting event. The helper isexitAfterShutdownRequest().Verification
Repro: Qwen3.6-35B-A3B with
--stream-experts --ssd-prefetch. Start a streaming 2000-token generation, then send SIGTERM 25 s in.exitingevent.ipsper runSwiftLMTests: 194 passed.🤖 Generated with Claude Code
https://claude.ai/code/session_01Nn7Du4yB5jtocihUNxxBGs