Operator config rootCA should be configurable via secret

[Laraakaa]

In my current understanding, to get the Operator to work with strict TLS verification, one needs to do the following:

1. Have a valid certificate issued with a CA, key and a certificate.
2. Add the file openssl.xml to the "files" configuration in the CHI (provided below)

<clickhouse>
  <openSSL>
    <server>
      <certificateFile>/etc/clickhouse-server/certs/tls.crt</certificateFile>
      <privateKeyFile>/etc/clickhouse-server/certs/tls.key</privateKeyFile>
      <verificationMode>strict</verificationMode>
      <caConfig>/etc/clickhouse-server/certs/ca.crt</caConfig>
      <loadDefaultCAFile>true</loadDefaultCAFile>
      <cacheSessions>true</cacheSessions>
      <disableProtocols>sslv2,sslv3</disableProtocols>
      <preferServerCiphers>true</preferServerCiphers>
    </server>
    <client>
      <loadDefaultCAFile>true</loadDefaultCAFile>
      <cacheSessions>true</cacheSessions>
      <disableProtocols>sslv2,sslv3</disableProtocols>
      <preferServerCiphers>true</preferServerCiphers>
      <verificationMode>strict</verificationMode>
      <caConfig>/etc/clickhouse-server/certs/ca.crt</caConfig>
    </client>
  </openSSL>
</clickhouse>

Those certs can be mounted by a secret using the podTemplate or by specifying it like described here: https://altinity.com/blog/clickhouse-confidential-using-kubernetes-secrets-with-the-altinity-operator "Using Secrets to transmit X509 certificates and private keys".

3. Add the rootCA in the operator config.yaml at clickhouse.access.rootCA, in my understanding to verify the certificate provided by the server against the CA. The issue is that this clickhouse.access.rootCA seems to only accept a certificate as string, and in our case it's generated by cert-manager. So we would have to either copy paste the generated certificates into the config.yaml as a string - which is not nice as it might expire and this would force us to do this manual thing again.
   For reference, it works when I don't set it at all but then I suppose it's not being verified.

A possible solution could be to add a property access.rootCAFile or similar, that would instead look for a certificate file on the disk. That would allow us to mount the dynamically generated CA as a secret.

Specifically I've looked at this part of the code: https://github.com/Altinity/clickhouse-operator/blob/master/pkg/model/clickhouse/connection.go#L124

And this old issue: #1073, which doesn't seem to fully answer the access.rootCA property in the operator config.yaml, and rather focuses on how to mount certificates in a CHI rather than for the operator itself.

I'm happy to provide a MR contribution if the issue is acknowledged 👍

Environment:
Chart 0.25.6
Kubernetes 1.33.7 (EKS)

[sunsingerus]

Thanks for the detailed write-up and for offering to contribute.

The request is valid and the code confirms the gap exactly as you described. clickhouse.access.rootCA is a plain string field with no file-path alternative today.

What already exists as a pattern

The operator already does something similar for credentials: clickhouse.access.secret lets you reference a k8s Secret by namespace/name, and fetchSecretCredentials() reads it at startup and populates the runtime username/password. The same pattern can be applied for the CA cert.

Two approaches worth considering

Option A — rootCAFile (simplest)

Add a rootCAFile string field alongside the existing rootCA. At startup, read the file and feed its content into the existing setupTLSAdvanced() pipeline, which already handles PEM, base64, and DER. Works with any mounted file — Secrets, ConfigMaps, hostPath. Implementation is minimal: a new field in the config struct and a file-read before the existing cert parsing logic.

Option B — Secret reference (more Kubernetes-native)

Add a rootCASecret struct with namespace, name, and key fields (similar to the existing access.secret). The operator fetches the cert directly from the k8s API at startup — no volume mount on the operator pod required. For cert-manager specifically this is the cleanest fit, since cert-manager already writes the CA to a Secret under the ca.crt key.

Both approaches are acceptable. Option A is easier to implement; Option B avoids the need to mount anything extra into the operator pod.

A separate bug to fix at the same time

There is an existing TODO in setupTLSAdvanced() (connection.go:167) that you should be aware of:

err = goch.RegisterTLSConfig(tlsSettings, &tls.Config{
    RootCAs:            rootCAs,
    InsecureSkipVerify: true, // TODO: Make it configurable
})

InsecureSkipVerify: true is set unconditionally even when a rootCA is configured. This means TLS server certificate verification is currently always skipped, regardless of what you put in rootCA. Your observation that "it works when I don't set it at all" is consistent with this — the behavior is identical either way today.

Any PR adding rootCAFile or rootCASecret support should also fix this TODO: set InsecureSkipVerify: false (or make it a separate configurable field) when a CA is provided, so the cert pool actually gets used.

Summary of what a PR should include

1. New config field(s): rootCAFile and/or rootCASecret { namespace, name, key }
2. Config normalization/loading: read the file or fetch the secret at startup, populate a runtime field
3. Wire the runtime value into setupTLSAdvanced() alongside the existing rootCA string
4. Fix InsecureSkipVerify: true → false when a root CA is configured
5. Update the operator config CRD/YAML documentation

Happy to review a PR when ready.

[Laraakaa]

Perfect, I feel like option B) makes it easier to use overall (no manual mount when deploying) and as you say more kubernetes-native. I will likely give that a shot.

As for your other points, I saw that and already intended to add that at the same time. Happy to submit the PR when I get around it, thanks for your insightful response!