DOC-404: Document orphaned containers after Docker Compose stop - #894
DOC-404: Document orphaned containers after Docker Compose stop#894quetzalliwrites wants to merge 3 commits into
Conversation
Add an FAQ entry explaining that Docker Compose's default 10s SIGTERM-to-SIGKILL grace period can be too short for LocalStack to finish tearing down auxiliary containers (including k3d containers backing an EKS cluster), leaving them behind as orphans, and how to raise stop_grace_period to avoid it. The CLI is not affected since it waits for shutdown to finish. Cross-link the same behavior from the EKS docs where k3d-backed cluster creation is introduced.
Deploying localstack-docs with
|
| Latest commit: |
81124a6
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://f33bed34.localstack-docs.pages.dev |
| Branch Preview URL: | https://doc-404-doc-faq-addition-of.localstack-docs.pages.dev |
|
@nik-localstack @pinzon heads up, looks like this PR is still pending eng review |
nik-localstack
left a comment
There was a problem hiding this comment.
Thanks for putting this together, the entry describes a real problem, and it came up in an internal discussion recently too.
Some things from that are worth folding in:
- There's a second setting involved, and it isn't in our docs. Alongside docker-compose stop_grace_period, LocalStack applies its own
SHUTDOWN_TIMEOUT(5 seconds by default) to its cleanup, and that one runs out first. Raising onlystop_grace_perioddoesn't give the cleanup more time and it has a side effect worth warning about: docker compose stop then takes the full three minutes before the container exits, whereas raising both settings gets a clean shutdown in well under a minute. The inline suggestions reflect that.SHUTDOWN_TIMEOUTisn't listed in the configuration reference today, so this entry would be the first place we mention it. We should probably expose it in the configuration page. - It isn't only EKS. The same shutdown budget is shared by every service that manages containers. ECS is the closest example, where a task's own stop timeout can consume the entire budget by itself.
I think we need more eyes before merging: shutdown behavior is being actively discussed internally right now, and how these settings interact may change. I don't think that blocks the changes but we should consider what should be advertised as a permanent solution and what not.
cc @purcell who was involved in this discussion about shutdown
btw I have added some inline suggestions but it may not be what we want to document in the end, so feel free to disregard them.
|
Hey @purcell we need confirmation from ENG about whether this docs ticket request is still needed or if it's been cancelled, per @nik-localstack earlier Q: If you're not the right person to confirm, can you let us know who is? thx ! |
- Generalize the FAQ entry away from EKS/k3d specifics (Lambda, ECS, RDS, and EKS all start auxiliary containers); keep the k3d-specific framing in the EKS doc's cross-referencing note instead. - Explain both timing windows that can cause orphaned containers: Docker Compose's stop_grace_period and LocalStack's own SHUTDOWN_TIMEOUT, and note both need to be raised together. - Clarify the CLI callout refers to lstk specifically, not the legacy localstack CLI. - Fix the cleanup instruction to actually say how to find the containers (docker ps) before removing them.
|
Thanks @nik-localstack, while we wait confirmation on whether this ticket is cancelled, I still went ahead and pushed
One thing worth double-checking before merge (independent of the cancellation question): I couldn't find |
purcell
left a comment
There was a problem hiding this comment.
This is all accurate and reasonable, so approving this in principle, with the few proposed edits below to fix some formatting and idioms.
Co-authored-by: Steve Purcell <steve@sanityinc.com>
Summary
stop_grace_periodto avoid it.Linear ticket
https://linear.app/localstack/issue/DOC-404/doc-faq-addition-of-why-are-some-containers-left-behind-after-i-stop
Test plan
npx astro buildcompletes successfully (401 pages built)eks.mdxandfaq.mdx, are valid