Skip to content

Enable Streaming Chunks for responses to improve TTFB on TS #320

@jevansnyc

Description

@jevansnyc

What it does: Flush to the browser immediately. Run the auction concurrently. Stream the body as origin HTML arrives, inject auction results when they're ready.

What changes in TS: The process_response_with_pipeline function in [proxy.rs:141-169] currently buffers the entire body before returning. This needs to become a true streaming pipeline, parse and forward HTML in chunks, holding only the ad-config injection point. Fastly's Rust SDK supports streaming response bodies via Body streaming APIs so we should be good.

Browser caching changes: With streaming, the response headers go out immediately, so something like this:

Cache-Control: max-age=60, stale-while-revalidate=300
Surrogate-Control: max-age=300, stale-while-revalidate=3600
Browser caches for 60s, serves stale (instantly) for 300s while TS revalidates in background. On Fastly's side, CDN holds it 300s with 1hr SWR. Real users see TTFB of edge cache latency (~5-30ms) on virtually every visit.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions