From fc6652eea0ef9c6cda147effa522853e08398368 Mon Sep 17 00:00:00 2001 From: Saifuddin Rangwala Date: Tue, 8 Sep 2026 12:32:45 +0530 Subject: [PATCH] Minion/master connection-resilience defaults, dedicated namespaces, GHCR-first quickstart Minion resilience (salt-minion-vcf chart + salt-minion-kubernetes chart/ entrypoint, and salt-minion-vcf's own entrypoint for Docker/env users): - auth_timeout: 60 - allow more time for minions to authenticate against the master before timing out and retrying, reducing thundering-herd retry storms. - master_alive_interval: 60 - proactively checks the master TCP connection and reconnects if it's gone stale; disabled (0) by default in Salt itself, so minions previously only noticed a dead/rescheduled master at the next job dispatch attempt. - recon_default/recon_max/recon_randomize - ZeroMQ reconnect backoff with jitter, Salt's own documented fix for reconnect thundering herds when many minions reconnect at once (e.g. after a master restart). Master resilience (salt-master-kubernetes chart + docker/salt-master entrypoint): - presence_events: true - fires salt/presence/present|change events so manage.present/manage.status reflect which minions are actually connected right now, useful in Kubernetes where minion pods come and go independently of their accepted-key status. Dedicated default namespaces (previously kube-system for both): - salt-master-kubernetes now defaults to namespace `salt-master`. - salt-minion-kubernetes now defaults to namespace `salt` (both its own `namespace` and `kubeBench.namespace`, which must stay in sync with each other and with kube-bench-job's own namespace - see updated docs/kubernetes-compliance-guide.md). - salt-minion-vcf docs updated from the `vcf-salt` example namespace to `salt` for consistency (this chart has no explicit namespace value of its own - it deploys wherever `helm install -n` targets). README/docs: - Main README now leads with GHCR quick-install commands (Helm OCI charts + a plain `docker run` against the published salt-minion-vcf image) instead of local build instructions. - New docs/building.md consolidates the from-source build path (Docker images, chart packaging) that used to live in the main README. Verified: helm lint/template on all 3 charts, hadolint on all 4 Dockerfiles, shellcheck on both entrypoint scripts, bash -n on the master entrypoint - all clean. --- README.md | 75 ++++++++------- docker/salt-master/entrypoint.sh | 6 ++ docs/building.md | 95 +++++++++++++++++++ docs/kubernetes-compliance-guide.md | 18 ++-- salt-master-kubernetes/README.md | 3 +- .../templates/deployment.yaml | 2 + salt-master-kubernetes/values.yaml | 8 +- salt-minion-kubernetes/README.md | 8 +- .../scripts/docker-entrypoint.sh | 5 + .../templates/deployment.yaml | 10 ++ salt-minion-kubernetes/values.yaml | 26 ++++- salt-minion-vcf/README.md | 2 +- .../helm/salt-minion-vcf/README.md | 6 +- .../salt-minion-vcf/templates/configmap.yaml | 5 + .../helm/salt-minion-vcf/values.yaml | 21 ++++ salt-minion-vcf/kubernetes/README.md | 20 ++-- salt-minion-vcf/scripts/docker-entrypoint.sh | 5 + salt-minion-vcf/scripts/onboarding/README.md | 4 +- 18 files changed, 252 insertions(+), 67 deletions(-) create mode 100644 docs/building.md diff --git a/README.md b/README.md index fb6a2e6..ed5a508 100644 --- a/README.md +++ b/README.md @@ -10,66 +10,65 @@ Helm charts for deploying different flavours of Salt images to Kubernetes. | [salt-minion-kubernetes](salt-minion-kubernetes) | Installs Salt Minion and RBAC. Has built-in support to run CIS Kubernetes compliance assessments via kube-bench on-demand Jobs. Supports in-cluster (minion runs as a pod) and external (RBAC only) modes. Like `salt-minion-vcf`, this is a full project directory - its own `Dockerfile` builds a Salt minion preloaded with `saltext.vault` and `saltext.kubernetes`, with `kubectl` bundled in. | | [salt-minion-vcf](salt-minion-vcf) | Extensible Salt Minion image (Docker, Docker Compose, Kubernetes, and Helm) preloaded with configurable Salt extensions - `saltext.vcf` (VMware Cloud Foundation automation: vCenter, NSX, SDDC-M, VCF Ops) by default, but not limited to it. Includes `saltext.vault` integration for sourcing credentials from HashiCorp Vault into Pillar instead of storing them on disk. Unlike the other entries here, this directory is the full project (Dockerfile, Docker Compose, scripts, docs), not a chart-only directory - the Helm chart itself lives at [`salt-minion-vcf/helm/salt-minion-vcf`](salt-minion-vcf/helm/salt-minion-vcf). | -## Usage +## Quick install (published images/charts) -### Kubernetes / Helm +The fastest path to a working deployment: install straight from the +published GHCR OCI charts, no local Docker/Helm build required. See +[`docs/releasing.md`](docs/releasing.md) for the full list of published +artifacts, and each component's own `CHANGELOG.md` for exactly which +Salt/extension versions a given tag carries (the tag itself is just that +component's own semver, independent of Salt's version). -```bash -helm install salt-master-kubernetes ./salt-master-kubernetes -f my-values.yaml -``` +### salt-master-kubernetes ```bash -helm install salt-minion-kubernetes ./salt-minion-kubernetes -f my-values.yaml +helm install salt-master-kubernetes \ + oci://ghcr.io/saltstack/salt-helm/charts/salt-master-kubernetes \ + --version 0.1.0 \ + --set agent.image.repository=ghcr.io/saltstack/salt-helm/salt-master \ + --set agent.image.tag=1.0.0 ``` +### salt-minion-kubernetes + ```bash -helm install salt-minion-vcf ./salt-minion-vcf/helm/salt-minion-vcf \ - --set salt.master=salt-master.example.com +helm install salt-minion-kubernetes \ + oci://ghcr.io/saltstack/salt-helm/charts/salt-minion-kubernetes \ + --version 0.1.0 \ + --set agent.saltMasterHost=salt-master.example.com ``` -See each chart's `values.yaml` for configurable parameters. +(`agent.image.repository` already defaults to the published +`ghcr.io/saltstack/salt-helm/salt-minion-kubernetes` image - no override +needed there.) -Charts and images are also published to GHCR — see -[`docs/releasing.md`](docs/releasing.md) for the full list, and each -component's own `CHANGELOG.md` for exactly which Salt/extension versions a -given release tag carries (the tag itself is just that component's own -semver, independent of Salt's version): +### salt-minion-vcf ```bash -helm install salt-master-kubernetes \ - oci://ghcr.io/saltstack/salt-helm/charts/salt-master-kubernetes \ - --version 0.1.0 +helm install salt-minion-vcf \ + oci://ghcr.io/saltstack/salt-helm/charts/salt-minion-vcf \ + --version 0.1.0 \ + --set image.repository=ghcr.io/saltstack/salt-helm/salt-minion-vcf \ + --set image.tag=0.1.0 \ + --set salt.master=salt-master.example.com ``` -### Docker - -`salt-minion-vcf` also runs as a plain Docker container or via Docker -Compose, without Kubernetes - see +`salt-minion-vcf` also runs as a plain Docker container, pulling the +published image directly - no Kubernetes required. See [`salt-minion-vcf/README.md`](salt-minion-vcf/README.md) for the full guide, -including air-gapped builds and the local Vault-backed testing workflow. +including Docker Compose and the local Vault-backed testing workflow. ```bash -docker build -t salt-minion-vcf:0.1.0 ./salt-minion-vcf - docker run -d \ --name salt-minion-vcf \ -e SALT_MASTER=salt-master.example.com \ -v salt-minion-vcf-pki:/etc/salt/pki/minion \ - salt-minion-vcf:0.1.0 + ghcr.io/saltstack/salt-helm/salt-minion-vcf:0.1.0 ``` -or with Docker Compose: - -```bash -cd salt-minion-vcf -cp .env.example .env # set SALT_MASTER, etc. -docker compose up -d --build -``` +See each chart's `values.yaml` for the full list of configurable parameters. -`salt-minion-kubernetes` and `docker/salt-master` build the same way, each -with its own directory as context: +## Building from source -```bash -docker build -t salt-minion-kubernetes:0.1.0 ./salt-minion-kubernetes -docker build -t salt-master:3008.2 ./docker/salt-master -``` +Need a patched image, a different Salt version, or an air-gapped build with +no route to GHCR? See [`docs/building.md`](docs/building.md). diff --git a/docker/salt-master/entrypoint.sh b/docker/salt-master/entrypoint.sh index e178e35..381714b 100644 --- a/docker/salt-master/entrypoint.sh +++ b/docker/salt-master/entrypoint.sh @@ -8,6 +8,12 @@ mkdir -p "$CONF_DIR" [ -z "${SALT_AUTO_ACCEPT:-}" ] || printf 'auto_accept: %s\n' "$SALT_AUTO_ACCEPT" [ -z "${SALT_MASTER_ID:-}" ] || printf 'id: %s\n' "$SALT_MASTER_ID" + # Fires salt/presence/present (and /change) events on the event bus so + # `salt-run manage.present`/`manage.status` reflect which minions are + # actually connected right now - useful in Kubernetes, where minion pods + # come and go independently of their accepted-key status. + printf 'presence_events: %s\n' "${SALT_PRESENCE_EVENTS:-True}" + # Keep pidfile/sock_dir off /var/run: the container runtime remounts # /run fresh (root-owned, 0755) on every pod start regardless of what's # baked into the image, so a non-root master can never mkdir under diff --git a/docs/building.md b/docs/building.md new file mode 100644 index 0000000..8858fec --- /dev/null +++ b/docs/building.md @@ -0,0 +1,95 @@ +# Building From Source + +Most users don't need this — the published images and charts on GHCR (see +the main [`README.md`](../README.md)) are the fastest path to a working +deployment. Build from source instead when you need a patched/local image, +a Salt version other than what's published, or an air-gapped build with no +route to GHCR. + +## Prerequisites + +- Docker (or another OCI builder) for the images +- Helm 3+ for the charts + +## Docker images + +Each image is self-contained (pip-installed Salt extensions from PyPI rather +than a checkout of the extension's own repo, where applicable), so all three +build directly from this repo with that image's own directory as the build +context: + +```bash +docker build -t salt-master:3008.2 docker/salt-master +docker build -t salt-minion-vcf:0.1.0 salt-minion-vcf +docker build -t salt-minion-kubernetes:0.1.0 salt-minion-kubernetes +``` + +See each image's own README for the full build-arg reference (Salt version, +extension selection, internal PyPI/apt mirrors for air-gapped builds, etc.): + +- [`docker/salt-master/README.md`](../docker/salt-master/README.md) +- [`salt-minion-vcf/README.md`](../salt-minion-vcf/README.md) +- [`salt-minion-kubernetes/README.md`](../salt-minion-kubernetes/README.md) + +### Running a locally-built image + +`salt-minion-vcf` also runs as a plain Docker container or via Docker +Compose, without Kubernetes — see +[`salt-minion-vcf/README.md`](../salt-minion-vcf/README.md) for the full +guide, including the local Vault-backed testing workflow: + +```bash +docker run -d \ + --name salt-minion-vcf \ + -e SALT_MASTER=salt-master.example.com \ + -v salt-minion-vcf-pki:/etc/salt/pki/minion \ + salt-minion-vcf:0.1.0 +``` + +or with Docker Compose: + +```bash +cd salt-minion-vcf +cp .env.example .env # set SALT_MASTER, etc. +docker compose up -d --build +``` + +### Getting a locally-built image onto a cluster + +- **With a registry:** tag and push, then point the chart's + `agent.image.repository`/`image.repository` value at your registry path. +- **Without a registry** (air-gapped/test clusters): `docker save` the image + and `ctr -n k8s.io images import` it directly into every node's containerd + store, then set `image.pullPolicy: Never` (or `agent.image.pullPolicy`) so + Kubernetes doesn't try to pull. + +## Helm charts + +The charts themselves need no build step — install straight from a checkout +of this repo: + +```bash +helm install salt-master-kubernetes ./salt-master-kubernetes -f my-values.yaml +helm install salt-minion-kubernetes ./salt-minion-kubernetes -f my-values.yaml +helm install salt-minion-vcf ./salt-minion-vcf/helm/salt-minion-vcf \ + --set salt.master=salt-master.example.com +``` + +Remember to also point each chart's image value at whatever you built above +(the charts otherwise default to either the published GHCR image or an +unqualified local tag — check `values.yaml`). + +To produce a distributable `.tgz` instead (e.g. for your own chart repo or +registry): + +```bash +helm package salt-master-kubernetes +helm package salt-minion-kubernetes +helm package salt-minion-vcf/helm/salt-minion-vcf +``` + +## Cutting an actual release + +Building locally doesn't publish anything. To push a build to +`ghcr.io/saltstack/salt-helm/...` under a real version tag, see +[`docs/releasing.md`](releasing.md). diff --git a/docs/kubernetes-compliance-guide.md b/docs/kubernetes-compliance-guide.md index 66bb894..d6c00cb 100644 --- a/docs/kubernetes-compliance-guide.md +++ b/docs/kubernetes-compliance-guide.md @@ -104,8 +104,8 @@ for the full list): Verify the master came up healthy: ```bash -kubectl -n kube-system get pods -l app=salt-master-kubernetes -kubectl -n kube-system exec -it deploy/salt-master-kubernetes -- salt-key -L +kubectl -n salt-master get pods -l app=salt-master-kubernetes +kubectl -n salt-master exec -it deploy/salt-master-kubernetes -- salt-key -L ``` ### 2b. salt-minion-kubernetes @@ -134,9 +134,9 @@ jobs), and both must be reachable at whatever address you set Accept the new minion's key from the master: ```bash -kubectl -n kube-system exec -it deploy/salt-master-kubernetes -- salt-key -L -kubectl -n kube-system exec -it deploy/salt-master-kubernetes -- salt-key -a -kubectl -n kube-system exec -it deploy/salt-master-kubernetes -- salt '' test.ping +kubectl -n salt-master exec -it deploy/salt-master-kubernetes -- salt-key -L +kubectl -n salt-master exec -it deploy/salt-master-kubernetes -- salt-key -a +kubectl -n salt-master exec -it deploy/salt-master-kubernetes -- salt '' test.ping ``` `test.ping` returning `True` confirms both ports are wired up correctly. @@ -179,6 +179,10 @@ or `run_assessment` won't find the CronJob it's looking for: | `namespace` | `namespace` | | `cronJob.name` | `kubeBench.cronJobName` | +`salt-minion-kubernetes` now defaults `namespace` to `salt` (not `kube-system`), +so install `kube-bench-job` with `--set namespace=salt` unless you've +overridden `salt-minion-kubernetes`'s own `namespace` value too. + See [`salt-k8s-compliance/helm/kube-bench-job/README.md`](https://github.com/saltstack/salt-k8s-compliance/blob/main/helm/kube-bench-job/README.md) for the full configuration reference, including host-path mounts for non-kubeadm distros (RKE2/k3s) and the RBAC gaps to watch for on multi-node @@ -192,7 +196,7 @@ execution module (installed on the minion as part of `saltext.kubernetes`). Force a fresh assessment right now, regardless of any cached result: ```bash -kubectl -n kube-system exec -it deploy/salt-master-kubernetes -- \ +kubectl -n salt-master exec -it deploy/salt-master-kubernetes -- \ salt '' kube_bench_cache.run_assessment ``` @@ -205,7 +209,7 @@ assessment first only if the cache is stale — see `pillar.ttlSeconds` below): ```bash -kubectl -n kube-system exec -it deploy/salt-master-kubernetes -- \ +kubectl -n salt-master exec -it deploy/salt-master-kubernetes -- \ salt '' kube_bench_cache.status_for_check test_number=1.1.11 ``` diff --git a/salt-master-kubernetes/README.md b/salt-master-kubernetes/README.md index 509ebe5..a95b51c 100644 --- a/salt-master-kubernetes/README.md +++ b/salt-master-kubernetes/README.md @@ -40,11 +40,12 @@ The following table lists the most commonly overridden values. See | Parameter | Description | Default | | --- | --- | --- | -| `namespace` | Namespace for all chart resources. | `kube-system` | +| `namespace` | Namespace for all chart resources. | `salt-master` | | `agent.image.repository` | Salt master image repository. | `salt-master` | | `agent.image.tag` | Salt master image tag. | `3007.1` | | `agent.autoAccept` | Auto-accept new minion keys instead of requiring `salt-key -a` per minion. Leave `false` for a production master. | `false` | | `agent.masterId` | Sets `id:` on the master itself. Empty uses the pod hostname. | `""` | +| `agent.presenceEvents` | Fires `salt/presence/present`/`change` events so `manage.present`/`manage.status` reflect which minions are actually connected right now. | `true` | | `agent.persistence.enabled` | Persist the master's `/etc/salt/pki` (its own keypair *and* the accepted-minion key list) across pod restarts. | `false` | | `agent.persistence.type` | `pvc` or `hostPath`. `hostPath` requires `agent.nodeSelector`. | `pvc` | | `agent.nodeSelector` | Pins the pod to a node. Required when `agent.persistence.type=hostPath`. | `{}` | diff --git a/salt-master-kubernetes/templates/deployment.yaml b/salt-master-kubernetes/templates/deployment.yaml index 6f5f899..7504e3c 100644 --- a/salt-master-kubernetes/templates/deployment.yaml +++ b/salt-master-kubernetes/templates/deployment.yaml @@ -48,6 +48,8 @@ spec: env: - name: SALT_AUTO_ACCEPT value: {{ .Values.agent.autoAccept | quote }} + - name: SALT_PRESENCE_EVENTS + value: {{ .Values.agent.presenceEvents | quote }} {{- if .Values.agent.masterId }} - name: SALT_MASTER_ID value: {{ .Values.agent.masterId | quote }} diff --git a/salt-master-kubernetes/values.yaml b/salt-master-kubernetes/values.yaml index 65d81b1..b13b11e 100644 --- a/salt-master-kubernetes/values.yaml +++ b/salt-master-kubernetes/values.yaml @@ -1,5 +1,5 @@ # Namespace where all salt-master-kubernetes objects are deployed. -namespace: kube-system +namespace: salt-master agent: image: @@ -56,6 +56,12 @@ agent: # Sets `id:` on the master itself. Leave empty to use the pod hostname. masterId: "" + # Fires salt/presence/present (and /change) events on the event bus, so + # `salt-run manage.present`/`manage.status` reflect which minions are + # actually connected right now - useful in Kubernetes, where minion pods + # come and go independently of their accepted-key status. + presenceEvents: true + # Name of an EXISTING Kubernetes Secret (created out-of-band - e.g. by # whatever generates and registers this master's identity with RaaS/the # minion side before this chart is ever installed - never via Helm diff --git a/salt-minion-kubernetes/README.md b/salt-minion-kubernetes/README.md index c951d8a..b1b9d37 100644 --- a/salt-minion-kubernetes/README.md +++ b/salt-minion-kubernetes/README.md @@ -54,7 +54,7 @@ Then issue a token for the created ServiceAccount and use it in the external minion's kubeconfig: ```bash -kubectl create token salt-minion-kubernetes -n kube-system +kubectl create token salt-minion-kubernetes -n salt ``` ## Uninstalling the chart @@ -70,7 +70,7 @@ The following table lists the most commonly overridden values. See | Parameter | Description | Default | | --- | --- | --- | -| `namespace` | Namespace for all chart resources. Must match `kube-bench-job`'s namespace. | `kube-system` | +| `namespace` | Namespace for all chart resources. Must match `kube-bench-job`'s namespace. | `salt` | | `agent.authMode` | `in_cluster` or `external`. | `in_cluster` | | `agent.image.repository` | Salt minion image repository. | `ghcr.io/saltstack/salt-helm/salt-minion-kubernetes` | | `agent.image.tag` | Salt minion image tag. | `0.1.0` | @@ -78,6 +78,10 @@ The following table lists the most commonly overridden values. See | `agent.saltMasterHost` | Salt master address. Required for `in_cluster` mode. | `""` | | `agent.saltMasterPort` | Salt master "ret" port (`master_port`). Override alongside `agent.saltPublishPort` when the master isn't reachable on its default ports, e.g. behind a Kubernetes NodePort Service. | `4506` | | `agent.saltPublishPort` | Salt master "publish" port (`publish_port`). | `4505` | +| `agent.authTimeout` | Seconds to wait for master auth before retrying - reduces thundering-herd retry storms. | `60` | +| `agent.masterAliveInterval` | Seconds between checks that the master TCP connection is still alive; reconnects if not. | `60` | +| `agent.reconDefault` / `agent.reconMax` | ZeroMQ transport reconnect backoff range (ms). | `1000` / `5000` | +| `agent.reconRandomize` | Jitters reconnect delay so minions don't all retry in lockstep. | `true` | | `agent.minion.id` | Salt minion ID. Empty uses the pod hostname. | `""` | | `agent.persistence.enabled` | Persist the minion's generated keypair (`/etc/salt/pki`) across pod restarts. | `false` | | `agent.persistence.type` | `pvc` or `hostPath`. `hostPath` requires `agent.nodeSelector`. | `pvc` | diff --git a/salt-minion-kubernetes/scripts/docker-entrypoint.sh b/salt-minion-kubernetes/scripts/docker-entrypoint.sh index 31d3078..3798c2b 100755 --- a/salt-minion-kubernetes/scripts/docker-entrypoint.sh +++ b/salt-minion-kubernetes/scripts/docker-entrypoint.sh @@ -35,6 +35,11 @@ master_port: ${SALT_MASTER_PORT} publish_port: ${SALT_PUBLISH_PORT} master_tries: -1 retry_dns: 30 +auth_timeout: ${SALT_AUTH_TIMEOUT:-60} +master_alive_interval: ${SALT_MASTER_ALIVE_INTERVAL:-60} +recon_default: ${SALT_RECON_DEFAULT:-1000} +recon_max: ${SALT_RECON_MAX:-5000} +recon_randomize: ${SALT_RECON_RANDOMIZE:-True} EOF # Preferred: pre-seed the master's actual public key so the minion trusts diff --git a/salt-minion-kubernetes/templates/deployment.yaml b/salt-minion-kubernetes/templates/deployment.yaml index 20edc5d..44ce45d 100644 --- a/salt-minion-kubernetes/templates/deployment.yaml +++ b/salt-minion-kubernetes/templates/deployment.yaml @@ -78,6 +78,16 @@ spec: value: {{ .Values.agent.saltMasterPort | quote }} - name: SALT_PUBLISH_PORT value: {{ .Values.agent.saltPublishPort | quote }} + - name: SALT_AUTH_TIMEOUT + value: {{ .Values.agent.authTimeout | quote }} + - name: SALT_MASTER_ALIVE_INTERVAL + value: {{ .Values.agent.masterAliveInterval | quote }} + - name: SALT_RECON_DEFAULT + value: {{ .Values.agent.reconDefault | quote }} + - name: SALT_RECON_MAX + value: {{ .Values.agent.reconMax | quote }} + - name: SALT_RECON_RANDOMIZE + value: {{ .Values.agent.reconRandomize | quote }} {{- if .Values.agent.minion.id }} - name: SALT_MINION_ID value: {{ .Values.agent.minion.id | quote }} diff --git a/salt-minion-kubernetes/values.yaml b/salt-minion-kubernetes/values.yaml index 376450f..72c7446 100644 --- a/salt-minion-kubernetes/values.yaml +++ b/salt-minion-kubernetes/values.yaml @@ -1,6 +1,6 @@ # Namespace where all salt-minion-kubernetes objects are deployed. # Must match the namespace used in kube-bench-job. -namespace: kube-system +namespace: salt agent: # Deployment mode: @@ -74,6 +74,28 @@ agent: saltMasterPort: 4506 # "publish" channel (publish_port). saltPublishPort: 4505 + + # Allow more time for minions to authenticate against the master before + # timing out and retrying - reduces "thundering herd" retry storms when + # many minions (re)connect at once, e.g. right after a master restart. + # Matches Salt's own default, set explicitly so it's documented here. + authTimeout: 60 + + # Proactively checks the TCP connection to the master every N seconds and + # reconnects if it's gone stale. Disabled (0) by default in Salt itself - + # without this, a minion sitting on a dead/rescheduled master's connection + # only notices at the next job dispatch attempt. + masterAliveInterval: 60 + + # ZeroMQ transport reconnect backoff: attempts start at reconDefault ms + # and back off up to reconMax ms; reconRandomize jitters the actual delay + # so many minions reconnecting at once (e.g. after a master restart) + # don't all retry in lockstep - Salt's own documented fix for reconnect + # thundering herds. + reconDefault: 1000 + reconMax: 5000 + reconRandomize: true + minion: # Salt minion ID. Leave empty to use the pod hostname. id: "" @@ -135,7 +157,7 @@ rbac: # kube-bench coordination settings — must match kube-bench-job chart values. kubeBench: - namespace: kube-system + namespace: salt # Must match cronJob.name in the kube-bench-job chart. cronJobName: kube-bench # Label applied to assessment Jobs and their pods for tracking. diff --git a/salt-minion-vcf/README.md b/salt-minion-vcf/README.md index c02db6b..2f6ea57 100644 --- a/salt-minion-vcf/README.md +++ b/salt-minion-vcf/README.md @@ -419,7 +419,7 @@ Do not put VCF or other target-system credentials in the ConfigMap. ```bash helm upgrade --install vcf-executor \ ./helm/salt-minion-vcf \ - --namespace vcf-salt \ + --namespace salt \ --create-namespace \ --set salt.master=salt-master.example.com \ --set image.repository=my-registry/salt-minion-vcf \ diff --git a/salt-minion-vcf/helm/salt-minion-vcf/README.md b/salt-minion-vcf/helm/salt-minion-vcf/README.md index 0dbdb11..a21aafb 100644 --- a/salt-minion-vcf/helm/salt-minion-vcf/README.md +++ b/salt-minion-vcf/helm/salt-minion-vcf/README.md @@ -7,7 +7,7 @@ maps naturally to the runtime. ```bash helm upgrade --install vcf-executor ./helm/salt-minion-vcf \ - --namespace vcf-salt \ + --namespace salt \ --create-namespace \ --set salt.master=salt-master.example.com \ --set image.repository=registry.example.com/salt-minion-vcf \ @@ -24,7 +24,7 @@ Scale to three independent Minions: ```bash helm upgrade --install vcf-executor ./helm/salt-minion-vcf \ - --namespace vcf-salt \ + --namespace salt \ --set workload.kind=StatefulSet \ --set workload.replicas=3 \ --set salt.master=salt-master.example.com @@ -41,7 +41,7 @@ A Deployment is also supported when exactly one Minion is wanted. ```bash helm upgrade --install vcf-executor ./helm/salt-minion-vcf \ - --namespace vcf-salt \ + --namespace salt \ --set workload.kind=Deployment \ --set workload.replicas=1 \ --set salt.master=salt-master.example.com diff --git a/salt-minion-vcf/helm/salt-minion-vcf/templates/configmap.yaml b/salt-minion-vcf/helm/salt-minion-vcf/templates/configmap.yaml index 517e801..d7994a2 100644 --- a/salt-minion-vcf/helm/salt-minion-vcf/templates/configmap.yaml +++ b/salt-minion-vcf/helm/salt-minion-vcf/templates/configmap.yaml @@ -11,6 +11,11 @@ data: publish_port: {{ .Values.salt.publishPort }} master_tries: {{ .Values.salt.masterTries }} retry_dns: {{ .Values.salt.retryDns }} + auth_timeout: {{ .Values.salt.authTimeout }} + master_alive_interval: {{ .Values.salt.masterAliveInterval }} + recon_default: {{ .Values.salt.reconDefault }} + recon_max: {{ .Values.salt.reconMax }} + recon_randomize: {{ .Values.salt.reconRandomize }} {{- if .Values.salt.masterFinger }} master_finger: {{ .Values.salt.masterFinger | quote }} {{- end }} diff --git a/salt-minion-vcf/helm/salt-minion-vcf/values.yaml b/salt-minion-vcf/helm/salt-minion-vcf/values.yaml index 75f5f2d..70d5fda 100644 --- a/salt-minion-vcf/helm/salt-minion-vcf/values.yaml +++ b/salt-minion-vcf/helm/salt-minion-vcf/values.yaml @@ -44,6 +44,27 @@ salt: masterTries: -1 retryDns: 30 + # Allow more time for minions to authenticate against the master before + # timing out and retrying - reduces "thundering herd" retry storms when + # many minions (re)connect at once, e.g. right after a master restart. + # Matches Salt's own default, set explicitly so it's documented here. + authTimeout: 60 + + # Proactively checks the TCP connection to the master every N seconds and + # reconnects if it's gone stale. Disabled (0) by default in Salt itself - + # without this, a minion sitting on a dead/rescheduled master's connection + # only notices at the next job dispatch attempt. + masterAliveInterval: 60 + + # ZeroMQ transport reconnect backoff: attempts start at reconDefault ms and + # back off up to reconMax ms; reconRandomize jitters the actual delay so + # many minions reconnecting at once (e.g. after a master restart) don't + # all retry in lockstep - Salt's own documented fix for reconnect + # thundering herds. + reconDefault: 1000 + reconMax: 5000 + reconRandomize: true + extraMinionConfig: {} # Example: # extraMinionConfig: diff --git a/salt-minion-vcf/kubernetes/README.md b/salt-minion-vcf/kubernetes/README.md index e65dffb..3d9913e 100644 --- a/salt-minion-vcf/kubernetes/README.md +++ b/salt-minion-vcf/kubernetes/README.md @@ -8,10 +8,10 @@ Use StatefulSet when you want the strongest mapping between Salt identity and persistent storage, or when you may scale to multiple execution Minions. ```bash -kubectl create namespace vcf-salt -kubectl -n vcf-salt apply -f configmap.yaml -kubectl -n vcf-salt apply -f service.yaml -kubectl -n vcf-salt apply -f statefulset.yaml +kubectl create namespace salt +kubectl -n salt apply -f configmap.yaml +kubectl -n salt apply -f service.yaml +kubectl -n salt apply -f statefulset.yaml ``` The first Minion ID is the stable Pod name: @@ -26,10 +26,10 @@ A normal Deployment is supported for exactly one Salt Minion. The explicit Minion ID plus PVC keep its Salt identity stable even when the Pod name changes. ```bash -kubectl create namespace vcf-salt -kubectl -n vcf-salt apply -f configmap.yaml -kubectl -n vcf-salt apply -f pvc.yaml -kubectl -n vcf-salt apply -f deployment.yaml +kubectl create namespace salt +kubectl -n salt apply -f configmap.yaml +kubectl -n salt apply -f pvc.yaml +kubectl -n salt apply -f deployment.yaml ``` The example Minion ID is: @@ -52,9 +52,9 @@ Edit `configmap.yaml` before deployment. It is mounted as: After changing the raw ConfigMap, restart the workload so Salt reloads it: ```bash -kubectl -n vcf-salt rollout restart statefulset/salt-minion-vcf +kubectl -n salt rollout restart statefulset/salt-minion-vcf # or -kubectl -n vcf-salt rollout restart deployment/salt-minion-vcf +kubectl -n salt rollout restart deployment/salt-minion-vcf ``` ## Accept the Minion diff --git a/salt-minion-vcf/scripts/docker-entrypoint.sh b/salt-minion-vcf/scripts/docker-entrypoint.sh index fa58fdf..7479f04 100755 --- a/salt-minion-vcf/scripts/docker-entrypoint.sh +++ b/salt-minion-vcf/scripts/docker-entrypoint.sh @@ -55,6 +55,11 @@ master_port: ${SALT_MASTER_PORT} publish_port: ${SALT_PUBLISH_PORT} master_tries: -1 retry_dns: 30 +auth_timeout: ${SALT_AUTH_TIMEOUT:-60} +master_alive_interval: ${SALT_MASTER_ALIVE_INTERVAL:-60} +recon_default: ${SALT_RECON_DEFAULT:-1000} +recon_max: ${SALT_RECON_MAX:-5000} +recon_randomize: ${SALT_RECON_RANDOMIZE:-True} EOF # Preferred: pre-seed the master's actual public key so the minion trusts diff --git a/salt-minion-vcf/scripts/onboarding/README.md b/salt-minion-vcf/scripts/onboarding/README.md index 7d29a78..82bd68d 100644 --- a/salt-minion-vcf/scripts/onboarding/README.md +++ b/salt-minion-vcf/scripts/onboarding/README.md @@ -133,7 +133,7 @@ python3 scripts/onboarding/vcf-ops-onboard.py \ --ops-host vcfops.example.com \ --ops-user admin \ --deployment kubernetes \ - --namespace vcf-salt \ + --namespace salt \ --release-name vcf-executor # Skip the interactive master picker if you already know the master ID @@ -153,7 +153,7 @@ python3 scripts/onboarding/vcf-ops-onboard.py \ python3 scripts/onboarding/vcf-ops-onboard.py \ --action rotate --deployment kubernetes \ --ops-host vcfops.example.com --ops-user admin \ - --namespace vcf-salt --release-name vcf-executor + --namespace salt --release-name vcf-executor # List trusted minions python3 scripts/onboarding/vcf-ops-onboard.py \