Skip to content

Do not send past the right edge of the peer window - #99

Open
hypnosis wants to merge 1 commit into
narrowlink:mainfrom
hypnosis:fix-send-window
Open

hypnosis wants to merge 1 commit into
narrowlink:mainfrom
hypnosis:fix-send-window

Conversation

@hypnosis

@hypnosis hypnosis commented Oct 5, 2026

Copy link
Copy Markdown

Problem

poll_write checks is_send_buffer_full() before sending, but then sends the whole buffer (up to the MSS) regardless of how much of the peer window is left. The last segment can end past the right edge of the window.

Example: a macOS peer, tun2proxy writing 8192-byte chunks, ipstack 1.0.1 (no window scaling).

  1. The peer advertises a window of 16383 bytes.
  2. Two 8192-byte segments are sent, 16384 bytes in flight: one byte past the edge.
  3. The peer trims the segment and acknowledges end - 1. The next two segments produce two duplicate ACKs, one short of the fast-retransmit threshold.
  4. In 1.0.1 the retransmission timer is checked only when a packet arrives, so the connection stalls until the application sends something.

Measurements

Path: macOS application → utun → tun2proxy 0.8.3 → SOCKS proxy. A 15 MB HTTPS download, 20 s limit per attempt, TCP headers captured on utun.

ipstack Downloads Stalled Longest gap
1.0.1 6 4 18–19 s
main @ 15751c2 12 0 0.46 s

Every stall on 1.0.1 matched the sequence above: an ACK at segment end − 1, two duplicate ACKs, then silence. On main the timer-driven retransmission (#92) and window scaling (#91) remove the stall, but the overshoot is still reachable whenever the peer window shrinks below the in-flight cap; it now costs a retransmission instead of a stall.

To make the window shrink on purpose, the reader was rate-limited to 200 KB/s (curl --limit-rate). Same path, six 4 MB downloads per build:

ipstack Windows < 8 KB Zero windows ACKs inside a sent segment Retransmitted segments Retransmitted bytes
main @ 15751c2 2736 22 93, in 6 of 6 connections 24 106 589
this PR 3055 28 0 0 0

The peer window shrank just as often with the change. Every segment now ends at or before its right edge, and nothing is resent. Throughput is set by the reader in both runs, about 0.2 MB/s.

Change

  • Tcb::get_usable_send_window(): min(max_unacked_bytes, send_window) minus the bytes in flight.
  • poll_write cuts the buffer to the usable window. is_send_buffer_full() is now usable == 0, same condition as before.
  • Retransmissions are not affected: they resend data that is already inside the window.

Tests

  • New writer_stops_at_the_right_edge_of_the_peer_window: peer window 2000 bytes, two 1460-byte writes. The second is cut to 540 bytes, the third waits for an ACK. Fails on main, passes with this change.
  • cargo fmt --all -- --check, cargo clippy --all-targets --all-features -- -D warnings and cargo test pass.

🤖 Generated with Claude Code

poll_write checked that the send buffer was not full, then sent the whole
buffer up to the MSS. The last segment could end past the right edge of the
peer window; the peer trims it and the tail has to be retransmitted.

Cut each write to the usable window: min(max_unacked_bytes, send_window)
minus the bytes in flight.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@SajjadPourali
SajjadPourali self-requested a review October 6, 2026 05:44
@SajjadPourali SajjadPourali self-assigned this Oct 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants