Native routing: Renaming servers in in hcloud but not Kubernetes (which you can't) results in a bad time (TM)

[andsens]

TL;DR

Relevant log entries from hccm:

E0417 10:34:19.308084       1 route_controller.go:452] Could not create route 1af3279e-37d7-4ad4-b6c1-30a2f5aaa046 10.1.131.0/24 for node worker-nbg1-1: hcloud/CreateRoute: hcops/AllServersCache.ByName: worker-nbg1-1 hcops/AllServersCache.getCache: not found
I0417 10:34:19.308205       1 event.go:389] "Event occurred" object="worker-nbg1-1" fieldPath="" kind="Node" apiVersion="v1" type="Warning" reason="FailedToCreateRoute" message="Could not create route 1af3279e-37d7-4ad4-b6c1-30a2f5aaa046 10.1.131.0/24 for node worker-nbg1-1 after 32.457156ms: hcloud/CreateRoute: hcops/AllServersCache.ByName: worker-nbg1-1 hcops/AllServersCache.getCache: not found"

This happened after I changed the naming scheme of our nodes in hcloud, but not in Kubernetes.
hccm seems to use node names to figure out routes to add/keep/delete. IMO depending on a mutable property doesn't seem like a sensible approach. Would it be possible to somehow add a server-id annotation instead and use that?
Without some kind of DaemonSet that interacts with the metadata service (like hcloud-csi) I don't see how though.

Expected behavior

.

Observed behavior

.

Minimal working example

No response

Log output

Additional information

No response

[lukasmetzner]

This is not totally trivial to solve. For context, the Kubernetes nodes have a field in the spec called providerID. This allows us to identify the underlying Hetzner server via its ID. Unfortunately, the CreateRoute method of the cloudprovider interface, which we implement, only provides us with the node name and not the full object. We could initialize a Kubernetes client and fetch the node via its name. Another idea would be to evaluate, if the “is this a Hetzner server” check is really necessary here, or if we can just rely on the routes' controller to create a valid network definition.

I will investigate this topic. For the time being I can just recommend keeping the server and node names in sync :/