Skip to content

fix(web): read the stream whole once, so a host can compress it - #303

Merged
MoLow merged 1 commit into
mainfrom
fix/web-unranged-first-read
Sep 9, 2026
Merged

MoLow merged 1 commit into
mainfrom
fix/web-unranged-first-read

Conversation

@MoLow

@MoLow MoLow commented Sep 9, 2026

Copy link
Copy Markdown
Owner

What

createNdjsonReader sent Range: bytes=${offset}- on every read, including the first β€” where the offset is 0 and the range is the entire stream.

Browsers drop Accept-Encoding to identity on any request that carries a Range header, so that first read forfeited compression and pulled the whole report raw. The bigger the report, the worse it got.

Now a Range is sent only from a non-zero offset. Incremental reads are untouched.

Measured

Same 4.6MB report, same session, back to back, through a compressing host (fastify + @fastify/compress, zstd):

Read Status Transferred Wall
Range: bytes=0- 206, identity 4,617,968 B 6,323 ms
no Range 200, zstd 252,525 B 739 ms

18x fewer bytes, 8.5x faster. Parsing the 7,357 lines takes 8 ms, so transfer was the whole cost.

Worth noting there was no host-side fix available: @fastify/compress (and any correct proxy) refuses to compress a 206 at all, since Content-Range describes unencoded offsets. The only way to get a compressed first read is not to range it.

Correctness

  • Offsets are unaffected. offset += byteLength(text) counts decoded bytes, so a compressed transfer still yields the right uncompressed offset for the next Range.
  • reset got more precise. It now fires only when a read that did ask for a Range got the whole body back β€” the case where events repeats what the caller already applied. The unranged first read has nothing to rebuild, so it no longer claims one. PullResult.reset documents this.
  • An empty stream stays unranged until it has bytes to skip, so a report that does not exist yet behaves as before.
  • Custom fetch implementations are handed no Range on the first read. The fetch docs on TestReportViewerProps, ReportSource and in the README now say so β€” an implementation must pass through whatever headers it is given rather than assume a Range is always there to forward.

Tests

packages/web/tests/poll.test.ts, 10/10 green locally:

  • new: the first read sends no Range, and the second ranges from the byte count it reached
  • new: an empty stream stays unranged until it has bytes to skip
  • rewritten: a source that ignores Range is re-read in full and flags a reset on the second pull (the first, being unranged, correctly does not)
  • the remaining mocks now answer the unranged first read with 200, as a real host would

I could not run viewer-component.test.ts locally (it loads the built dist/ and needs jsdom; no yarn on this machine), so CI is the check on that one. Its in-memory source already answers start === 0 with a 200, so it should be unaffected.

πŸ€– Generated with Claude Code

@codecov

codecov Bot commented Sep 9, 2026 •

Copy link
Copy Markdown

Codecov Report

βœ… All modified and coverable lines are covered by tests.
βœ… Project coverage is 99.68%. Comparing base (6200e7a) to head (80fd0bd).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #303   +/-   ##
=======================================
  Coverage   99.68%   99.68%           
=======================================
  Files          43       43           
  Lines        4690     4699    +9     
  Branches      659      660    +1     
=======================================
+ Hits         4675     4684    +9     
  Misses         13       13           
  Partials        2        2           
Flag Coverage Ξ”
v22 99.46% <100.00%> (+<0.01%) ⬆️
v24 99.46% <100.00%> (+<0.01%) ⬆️
v26 99.46% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

β˜” View full report in Codecov by Harness.
πŸ“’ Have feedback on the report? Share it here.

πŸš€ New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • πŸ“¦ JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

The reader asked for `Range: bytes=${offset}-` on every read, the first one
included, where the offset is 0 and the range is the whole stream. A browser
drops `Accept-Encoding` to `identity` on any request carrying a Range, so that
first read forfeited compression and pulled the report raw.

Range only from a non-zero offset. On a 4.6MB report served by a compressing
host that is 4,617,968 bytes in 6.3s before and 252,525 bytes in 0.7s after -
18x fewer bytes, 8.5x faster. Incremental reads are unchanged.

Offsets are unaffected: they count decoded bytes, so a compressed transfer still
yields the right uncompressed offset. `reset` now fires only when a read that
did ask for a Range got the whole body back, which is the case that repeats
already-applied events; the unranged first read has nothing to rebuild.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@MoLow
MoLow force-pushed the fix/web-unranged-first-read branch from d1aa072 to 80fd0bd Compare September 9, 2026 10:14
@MoLow
MoLow merged commit 5cdc35e into main Sep 9, 2026
6 checks passed
@pr-opener-bot pr-opener-bot Bot mentioned this pull request Sep 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant