Repository navigation
Egress replies that are not TCP or UDP are dropped, including path MTU discovery #404
Description
What needs to happen
The gateway claims a dedicated public address as the source that tenant egress traffic is masqueraded to. Every packet arriving for that address is claimed by the gateway program, and anything that is not TCP or UDP is dropped rather than handed to the kernel stack. That mirrors the fail closed choice already made for the gateway's own internal reachability address.
The stakes are not the same. The masquerade address is reachable from the entire internet, and the internet sends more than TCP and UDP to it. Path MTU discovery messages, destination unreachable, time exceeded, and echo replies all arrive as ICMP, and all of them are dropped today.
Desired outcome: decide which of these the return path should translate back to the tenant that originated the flow, and pass or handle the rest rather than dropping every one of them.
Why this matters
Path MTU discovery is the one with teeth. A tenant connection that crosses a smaller link somewhere on the internet stalls instead of adapting, and the failure presents as a hang rather than an error, on large transfers only, intermittently. That is close to the hardest class of network problem to attribute back to its cause.
Echo replies being dropped means the simplest reachability check a tenant can run from inside their own workload never works, which will read as a broken network long before anyone suspects the gateway.
This was called out by the author when the datapath was written, so it is a known deferral rather than an oversight, but nothing tracked it.
Activity
ecv commentedon Aug 14, 2026
Detail.
Both egress branches reject non TCP/UDP:
internal/plumbing/ebpf/edgeprog/edgenat.c:1234(handle_egress_return, the internet facing one this issue is about) drops toDROP_REASON_MALFORMED_EGRESS_RETURN.internal/plumbing/ebpf/edgeprog/edgenat.c:1116(handle_egress_forward) drops toDROP_REASON_MALFORMED_EGRESS_FORWARD, which is the tenant-outbound side of the same restriction.Flagged by the author under "Flagged for review" in #381: "a real cost for an internet-facing address that
gw_addr's internal-only precedent did not have to weigh." Agreed, and the ingress precedent does not transfer:gw_addris a fabric-internal address that only ever sees traffic this gateway itself sourced.Two smaller things to fold in whenever this is picked up:
MALFORMEDis the wrong reason name for a well formed ICMPv6 packet. An operator reading drop counters while debugging a stalled transfer gets pointed at packet parsing rather than at a protocol policy decision. A distinct reason would pay for itself the first time.Translating ICMP errors back to the originating tenant needs the embedded original datagram to be parsed to recover the masqueraded port, since ICMP has no ports of its own to key
egress_conn_tableon. That is the standard NAT approach and it is more work than the TCP/UDP paths, which is worth knowing before this gets sized.Related: #865 in the enhancements repo, plan section 3.4. Not covered by the plan's non goals list.
privateip commentedon Aug 17, 2026
Closed by #419