Repository navigation
fix(web): read the stream whole once, so a host can compress it - #303
Merged
Merged
Conversation
Codecov Reportβ
All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #303 +/- ##
=======================================
Coverage 99.68% 99.68%
=======================================
Files 43 43
Lines 4690 4699 +9
Branches 659 660 +1
=======================================
+ Hits 4675 4684 +9
Misses 13 13
Partials 2 2
Flags with carried forward coverage won't be shown. Click here to find out more. β View full report in Codecov by Harness. π New features to boost your workflow:
|
The reader asked for `Range: bytes=${offset}-` on every read, the first one
included, where the offset is 0 and the range is the whole stream. A browser
drops `Accept-Encoding` to `identity` on any request carrying a Range, so that
first read forfeited compression and pulled the report raw.
Range only from a non-zero offset. On a 4.6MB report served by a compressing
host that is 4,617,968 bytes in 6.3s before and 252,525 bytes in 0.7s after -
18x fewer bytes, 8.5x faster. Incremental reads are unchanged.
Offsets are unaffected: they count decoded bytes, so a compressed transfer still
yields the right uncompressed offset. `reset` now fires only when a read that
did ask for a Range got the whole body back, which is the case that repeats
already-applied events; the unranged first read has nothing to rebuild.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
MoLow
force-pushed
the
fix/web-unranged-first-read
branch
from
September 9, 2026 10:14
d1aa072 to
80fd0bd
Compare
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
createNdjsonReadersentRange: bytes=${offset}-on every read, including the first β where the offset is0and the range is the entire stream.Browsers drop
Accept-Encodingtoidentityon any request that carries aRangeheader, so that first read forfeited compression and pulled the whole report raw. The bigger the report, the worse it got.Now a
Rangeis sent only from a non-zero offset. Incremental reads are untouched.Measured
Same 4.6MB report, same session, back to back, through a compressing host (fastify +
@fastify/compress, zstd):Range: bytes=0-identityRangezstd18x fewer bytes, 8.5x faster. Parsing the 7,357 lines takes 8 ms, so transfer was the whole cost.
Worth noting there was no host-side fix available:
@fastify/compress(and any correct proxy) refuses to compress a206at all, sinceContent-Rangedescribes unencoded offsets. The only way to get a compressed first read is not to range it.Correctness
offset += byteLength(text)counts decoded bytes, so a compressed transfer still yields the right uncompressed offset for the nextRange.resetgot more precise. It now fires only when a read that did ask for aRangegot the whole body back β the case whereeventsrepeats what the caller already applied. The unranged first read has nothing to rebuild, so it no longer claims one.PullResult.resetdocuments this.fetchimplementations are handed noRangeon the first read. Thefetchdocs onTestReportViewerProps,ReportSourceand in the README now say so β an implementation must pass through whatever headers it is given rather than assume aRangeis always there to forward.Tests
packages/web/tests/poll.test.ts, 10/10 green locally:Range, and the second ranges from the byte count it reachedRangeis re-read in full and flags a reset on the second pull (the first, being unranged, correctly does not)200, as a real host wouldI could not run
viewer-component.test.tslocally (it loads the builtdist/and needsjsdom; no yarn on this machine), so CI is the check on that one. Its in-memory source already answersstart === 0with a200, so it should be unaffected.π€ Generated with Claude Code