Skip to content

Design: NetworkEgressPolicy CRD + gateway egress status fields (865 Phase A) #16

Description

@privateip

Part of datum-cloud/enhancements#865. Phase A of the egress (masquerade) implementation plan (docs/plans/865-edge-gateway-nat66-egress.md in datum-cloud/galactic). Implemented in #15.

Purpose

Extend the BGP API to support internet egress for tenant VPC backends through the edge gateway's masquerade (SNAT/PAT) datapath. This is the API surface every later phase builds on.

Design

NetworkEgressPolicy (new type)

Namespaced, tenant-writable, gated by the same ownership-verification admission webhook NetworkRule already requires. Unlike NetworkRule, it carries no VIP/backend/port — egress is on or off for a (vpcRef, vpcAttachmentRef) pair, existence-implies-enabled, because an egress flow's destination is an arbitrary internet address, not a pre-configured backend list.

type NetworkEgressPolicySpec struct {
    VPCRef           string
    VPCAttachmentRef string
}

type NetworkEgressPolicyStatus struct {
    ObservedGeneration  int64
    AssignedGatewayNode string // set once, see below
    Conditions          []metav1.Condition // Accepted, same reasons as NetworkRule
}

AssignedGatewayNode mirrors NetworkRuleStatus.PrimaryNode's set-once contract exactly, for the identical reason: gateway.AssignPrimaryNode is deterministic given a fixed gateway-node pool, but recomputing it on a later reconcile (once a periodic reconciler is involved, as Phase D's route installer is) can flip which node a tenant's egress traffic routes through whenever that pool changes — an avoidable traffic flap. This field did not exist in the original plan text; it surfaced as necessary once Phase D's route reconciler was actually implemented (see #384 and galactic#385).

NetworkGatewayStatus additions

  • EgressAddress — the gateway node's publicly-routable masquerade SNAT source. Populated and advertised into BGP the same way SRv6Address already is.
  • EgressSID — the gateway node's egress_sid uSID locator (design plan §3.1). Also not in the original plan text: RouteEgressAdd (the function that installs a tenant VRF's default route toward it) requires a real kernel route to the destination SID to already exist, which only happens via BGP advertisement — the same requirement SRv6Address already satisfies for the ingress path. Without publishing this, no compute node could ever route toward a gateway's egress locator.

Both are optional independently of SRv6Address (a gateway node not offering egress is a valid deployment) but always set together as a pair (config-validated on the galactic side).

Key decisions

  1. Existence-implies-enabled, not a richer policy spec — matches NetworkRule's own sparse-spec precedent and keeps the enablement question purely routing-layer (does the tenant's VRF have a default route), not a datapath-level lookup.
  2. AssignedGatewayNode and EgressSID are both retroactive additions to the plan's own §4.1 text — flagging this plainly since they weren't part of the original design and only became apparent once Phase D's implementation forced the question of "how does a compute node actually resolve where to route."

References

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Fields

Priority

None yet

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions