Skip to content

fix(server): pin locally-built swarm services to the build node - #4829

Open
ilysorc wants to merge 1 commit into
Dokploy:canaryfrom
ilysorc:fix/pin-local-build-to-build-node
Open

fix(server): pin locally-built swarm services to the build node#4829
ilysorc wants to merge 1 commit into
Dokploy:canaryfrom
ilysorc:fix/pin-local-build-to-build-node

Conversation

@ilysorc

@ilysorc ilysorc commented Jul 14, 2026

Copy link
Copy Markdown

What & Why

While testing Dokploy on a multi-node Docker Swarm, I noticed that applications built locally (Git, Dockerfile, or Buildpacks without a registry) don't always update correctly after a redeploy.

The deployment finishes successfully and is marked as done, but the application may continue serving the previous image.

Root cause

The problem only appears when the image is built locally and never pushed to a registry.

In that case, the newly built image exists only on the build node. However, if the service has no placement constraints (generateConfigContainer returns Placement.Constraints: [] when there are no mounts or user-defined placement rules), Swarm is free to schedule the replacement task on any node.

During a redeploy, ForceUpdate correctly creates a new task, but that task may land on another node that doesn't have the newly built image. Swarm rejects it with No such image, while the existing task continues running. Since the old task never stops, the deployment still reports success even though nothing has actually changed.

Reproduction (2-node swarm: manager + worker)

$ docker service ps <app> --no-trunc --format 'table {{.Node}}\t{{.CurrentState}}\t{{.Error}}'

NODE      CURRENT STATE            ERROR
worker    Rejected 9 minutes ago   "No such image: <app>:latest"
manager   Running  9 hours ago
$ docker inspect <running container> --format '{{.Image}}'   # old image
$ docker images -q <app>:latest                              # newly built image

Inspecting the service shows that Spec.TaskTemplate.Placement is simply {}, so Swarm is free to place the replacement task on the worker.

Accessing the origin directly (bypassing any CDN) confirms that the previous build is still being served. The only reliable workaround is recreating the service (Stop → Deploy), which causes the task to be scheduled back onto the build node.

The fix

The solution is to automatically pin locally built application services to the build node by adding a node.id==<build-node> placement constraint.

The node ID is obtained from docker info (Swarm.NodeID) using the same getRemoteDocker(serverId) connection that builds the image and creates the service.

This follows the same idea already used for bind mounts, which are pinned to the manager node, but instead pins the service to the actual build node. That makes it work correctly even when deployments are initiated from a worker.

The constraint is added only when:

  • the image is built locally (sourceType !== "docker"),
  • no registry is configured (registry, buildRegistry, and rollbackRegistry are all unset), and
  • the user has not configured a custom placementSwarm.

Registry-backed images, external Docker images, and database services are unaffected. If the user explicitly defines placement rules, those always take precedence.

Verification

The change was verified on a real two-node Swarm cluster.

With the placement constraint in place:

  • Redeploy schedules the replacement task on the build node.
  • The running container uses the newly built :latest image.
  • A normal Redeploy updates the application correctly without requiring a Stop → Deploy cycle.

Additional validation:

  • packages/server passes tsc --noEmit with no errors.

  • Added unit tests in mechanizeDockerContainer.test.ts covering:

    • local builds are pinned,
    • external Docker images are skipped,
    • registry-backed images are skipped,
    • user-defined placement is respected.
  • biome check passes cleanly.

Notes

  • The implementation uses docker.info().Swarm.NodeID from the existing deployment connection. If the node ID cannot be resolved, deployment continues exactly as before without adding a placement constraint.
  • Single-node deployments and registry-based workflows are unchanged.
  • Using a registry is still the recommended approach for true multi-node high availability. This change simply makes the common local-build workflow behave reliably instead of failing silently.

A locally built application image (git/dockerfile/buildpacks, no registry
configured) only exists on the node that built it. The swarm service is
created with no placement constraint, so on a multi-node swarm the
scheduler can place a task on another node that lacks the image. That task
is rejected with "No such image" while the previous task keeps running, so
the deployment reports success ("done") but the app is never updated.

Pin such services to the build node by adding a `node.id==<build node>`
placement constraint (resolved from the deploy connection's
`docker info` Swarm.NodeID), mirroring how bind mounts are already pinned.
Only applied for locally-built images and only when the user has not
configured their own placement; images pushed to a registry and external
(docker) images are unaffected.
@ilysorc
ilysorc requested a review from Siumauricio as a code owner July 14, 2026 19:14
@dosubot dosubot Bot added size:S This PR changes 10-29 lines, ignoring generated files. bug Something isn't working labels Jul 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working size:S This PR changes 10-29 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant