Repository navigation
Implement WebAPI streams' WritableStream - #6132
Conversation
Findings(by Codex)
If a user sets
|
| +// FIXME: This test passes in isolation but not when run together with the rest of the file. | ||
| +// It asserts the relative settlement order of the writer's ready and closed promises. In the | ||
| +// k6/Sobek event loop, settling a promise from Go drains the whole job queue synchronously, | ||
| +// so when dozens of streams from the surrounding tests have work pending, their reactions | ||
| +// interleave differently than the specification's per-microtask ordering, and closed happens | ||
| +// to settle before ready here. The stream still ends up errored with the right error. |
There was a problem hiding this comment.
Agreed we don't need to fix this in this PR. Good catch flagging it yourself in the patch comment.
One ask though β do you think it's worth opening an issue for the ready/closed ordering thing and reference it in the aborting.any.js.patch comment (e.g. FIXME(grafana/k6#XXXX))? Right now it's just commented out with no link back, so the WPT suite will report green with two known-failing assertions and nobody reading the code later would know that. Just want it discoverable down the line rather than only living in this PR's history.
There was a problem hiding this comment.
Yeah, we had the same issue within ReadableStream at the time. The key learning here is that there is a clear difference between how Sobek and, say the kind of engine WPT expects, handle those scenarios.
I'm not sure if we can do anything about it, or even would consider a bug.
Down the road, I would prefer those implementations to be living outside of k6, as sobek extensions, like in sobek-webapi-encoding. We couldn't do it from the start, or now, because we have in the module built dependency on k6 itself that I'd rather address later when migrating the module out of k6; first we should complete it.
ankur22
left a comment
There was a problem hiding this comment.
Really nice work β the spec-step annotations made this a pleasure to review, and wiring in
13 of the 14 upstream WPT files is great coverage. I ran a deeper pass on this (including
repros against a build of this branch) and found five functional issues I think need
addressing before merge β inline comments have the details. Short version:
- calling the controller's exposed
constructorwithoutnewand then.error()SIGSEGVs the whole k6 process (nilstreamderef), uncatchable from JS - a strategy
size()returning a value whosevalueOfthrows makeswriter.write()throw synchronously and permanently wedges the stream (no promise on it ever settles again) ensureReadyPromiseRejectedchecks the live promiseState()while settlements are transaction-deferred, sowriter.readycan fulfil where the spec requires reject (and can reject with the wrong error)- the
controllerobject passed tostart()and eachwrite()is a different wrapper every time, so identity checks and expando props break {start: undefined}throws instead of using the default algorithm, and{type: null}constructs instead of throwing RangeError
Happy to share the repro scripts.
|
Thanks for the review @Ankur, on it ππ» |
0d748c2 to
79c0aea
Compare
ankur22
left a comment
There was a problem hiding this comment.
Thanks @oleiade, this is looking much better β I re-ran all my earlier repros against the new head and every one of the five is fixed now (the constructor() crash, the size() panic + wedge, the writer.ready re-entrancy outcomes, controller identity, and the {start: undefined}/{type: null} cases). Nice work on the settlement-tracking rework in particular.
One new thing though, and I think it's a blocker: the writer-prototype change means calling the constructor with a foreign this pollutes Object.prototype. And separately, the prototype fix doesn't actually take effect in a normal imported script β only in the WPT harness where the classes are globals β so instanceof is still false where users run. Both come from the same spot (resolving the prototype by global name lookup). Details inline.
ankur22
left a comment
There was a problem hiding this comment.
LGTM, thanks for taking my comments into account!
I've left some non-blocking comments, ignore them if you feel they're not needed.
Move the stream-local promise wrapper next to the existing Sobek helpers and rename goja.go to reflect the runtime used by k6. This keeps promise creation, settlement, and handled-state behavior in one place instead of maintaining parallel readable and writable implementations. Represent ReadableStream reader closed promises with the same wrapper used by WritableStream. These promises settle synchronously on the runtime thread, so the event-loop-oriented js/promises helper is intentionally not used. WritableStream transaction scheduling remains local because it implements writable-specific settlement ordering.
c6a8e84 to
848c9b4
Compare
ReadableStream's closed-promise accessors return the package-private promise wrapper and are only used inside the streams package. Keep the methods private as well so the API reflects their scope and satisfies revive's unexported-return check.
k6 2.2.0 Created-by: HarmonybrewBot Commit-by: HarmonybrewBot Merged-by: HarmonybrewBot Description: Created by `brew bump` --- Created with `brew bump-formula-pr`.<details> <summary>release notes</summary> <pre>k6 `v2.2.0` is here π! This release includes: - `k6 cloud run --local-execution` now streams k6's logs to Grafana Cloud, so the test run's log view works for local execution too. - `chromium.connectOverCDP()`, which connects browser tests to an already-running Chromium instance. - `TextEncoder` and `TextDecoder` available as globals, and `WritableStream` support in `k6/experimental/streams`. - A `k6 cloud load-zone list` command. - Two new experimental feature flags: `merge-run-tags` and `freeze-env`. ## Breaking changes There are no breaking changes in this release. ## New features ### `k6 cloud run --local-execution` streams logs to Grafana Cloud [#6171](grafana/k6#6171) When a cloud test runs locally with `k6 cloud run --local-execution`, k6's logs now stream to the Grafana Cloud test run, so the run's log view is populated the same way it is for cloud execution. Previously, local-execution logs stayed on the machine running k6 and never reached the cloud. Work with Grafana's secrets management to safely work with secrets and redact them if they're accidentally leaked into logs are pushed. Use the new `--no-cloud-logs` flag opts out to opt out of streaming of logs when working with `--local-execution`: ```shell k6 cloud run --local-execution script.js k6 cloud run --local-execution --no-cloud-logs script.js ``` ### Connect to a running browser with `chromium.connectOverCDP()` [#6165](grafana/k6#6165) The browser module can now attach to an existing Chromium-based browser over the Chrome DevTools Protocol, mirroring Playwright's `browserType.connectOverCDP()`. Pass the browser's WebSocket endpoint and k6 manages the returned browser's connection β it's auto-closed at the end of the iteration, though you can call `close()` earlier to release the connection on demand. ```js import { chromium } from 'k6/browser'; export default async function () { const browser = await chromium.connectOverCDP('ws://localhost:9222/devtools/browser/<id>'); const page = await browser.newPage(); try { await page.goto('https://quickpizza.grafana.com/'); } finally { await page.close(); await browser.close(); } } ``` Unlike the `K6_BROWSER_WS_URL` environment variable, the endpoint is a runtime value β you can, for example, request a fresh session URL from a browser provider's API in `setup()` and connect to it from the iterations. ### `TextEncoder` and `TextDecoder` globals [#6182](grafana/k6#6182) `TextEncoder` and `TextDecoder` are now available as standard globals in both the init and VU contexts, no import required β matching how they are exposed in browsers and other JavaScript runtimes. ```js const encoded = new TextEncoder().encode('Hello, world!'); const decoded = new TextDecoder().decode(encoded); ``` ### `WritableStream` in `k6/experimental/streams` [#6132](grafana/k6#6132) The experimental streams module now implements `WritableStream` and `WritableStreamDefaultWriter` following the WHATWG Streams specification, complementing the existing `ReadableStream` and paving the way for a future `TransformStream` implementation. ```js import { WritableStream } from 'k6/experimental/streams'; export default async function () { const stream = new WritableStream({ write(chunk) { console.log(`wrote ${chunk}`); }, }); const writer = stream.getWriter(); await writer.write('hello'); await writer.close(); } ``` ### `k6 cloud load-zone list` command [#6142](grafana/k6#6142) A new `k6 cloud load-zone list` subcommand lists the load zones β public and private β available in the configured Grafana Cloud k6 stack, mirroring the existing `k6 cloud project list` command. Output defaults to a human-readable table; pass `--json` to emit a JSON array instead. ```console $ k6 cloud load-zone list Load zones for https://example.grafana.net: ID NAME TYPE AVAILABLE amazon:us:ashburn Ashburn, US (Amazon) public yes amazon:sa:cape town Cape Town, SA (Amazon) public yes ``` ### Configurable `handleSummary()` timeout [#5854](grafana/k6#5854) The time budget for the `handleSummary()` callback β previously hardcoded to 120 seconds β is now configurable through the `handleSummaryTimeout` option or the `K6_HANDLE_SUMMARY_TIMEOUT` environment variable, so long-running tests with heavy summaries no longer fail with `handleSummary() execution timed out`. Thanks, @LBaronceli! ```js export const options = { handleSummaryTimeout: '5m', }; ``` ### New experimental feature flags: `merge-run-tags` and `freeze-env` Two new experimental flags join the feature-flag system introduced in v2.1.0: - [#5714](grafana/k6#5714) `merge-run-tags` merges run tags per key across config layers, so `options.tags` in a script is no longer silently discarded when `--tag` or `K6_TAGS` is also used β higher-priority layers win on conflicting keys instead of replacing the whole map. Thanks, @yordis! - [#6032](grafana/k6#6032) `freeze-env` freezes the `__ENV` object, so modifications from script code throw a `TypeError` (in strict mode) instead of silently persisting across iterations and scenarios. Thanks, @lohitkolluri! ```shell k6 run --features merge-run-tags,freeze-env script.js ``` ## UX improvements and enhancements - [#5631](grafana/k6#5631) Makes the browser module's header accessors β `response.allHeaders()`, `headerValue()`, `headerValues()`, and `headersArray()` β return the raw wire headers (including `Set-Cookie` and security-related headers), correctly paired with each hop of a redirect chain instead of Chrome's provisional headers. As part of this, `headerValues()` now matches header names case-insensitively and splits repeated values on newlines rather than commas, and the `browser_data_sent`/`browser_data_received` metrics now include the raw header bytes and no longer vary run-to-run with CDP event ordering. - [#6208](grafana/k6#6208) Makes `k6 cloud` reject the run flags (for example, `--vus`) with an `unknown flag` error and a non-zero exit code. Previously `k6 cloud --vus 10 script.js` accepted the flags, printed the help text, and exited 0 β running tests with `k6 cloud` directly was deprecated in v2.0.0 in favor of `k6 cloud run`. - [#6096](grafana/k6#6096) Points the cloud secrets error at `K6_CLOUD_SECRETS_TOKEN` and `K6_CLOUD_SECRETS_ENDPOINT` when a test run is reused via `K6_CLOUD_PUSH_REF_ID` under `--local-execution`, instead of suggesting the `--local-execution` flag the user is already using. - [#6196](grafana/k6#6196) Adds `catch` blocks to the browser examples so a failing iteration reports the original error instead of a subsequent `page.close()` failure. Thanks, @locker95! ## Bug fixes - [#6234](grafana/k6#6234) Classifies HTTP/2 errors by message so the `error_code` metric tag stays correct when k6 is built with Go 1.27 (whose `x/net/http2` delegates to the standard library), and explicitly enables HTTP/2 negotiation on VU transports. - [#6232](grafana/k6#6232) Drains queued log entries in the Loki hook at shutdown so `--out loki` and cloud log streaming no longer lose the final batch, and emits a `k6 dropped N log messages` warning when the cloud log buffer overflows instead of dropping logs silently. - [#6125](grafana/k6#6125) Serializes the first concurrent open of a file in the caching filesystem so parallel `fs.open()` calls on the same file no longer read zero or truncated bytes. - [#6147](grafana/k6#6147) Fixes a data race and inconsistent request-interception state when browser routes are added or removed concurrently. Thanks, @somak2kai! - [#6070](grafana/k6#6070) Flushes buffered file log output once per second so recent logs aren't lost when k6 is killed before shutdown. Thanks, @rohan-patnaik! - [#6205](grafana/k6#6205) Stops sending an invalid `Sec-WebSocket-Protocol` header when tailing Grafana Cloud logs; spec-strict servers rejected the handshake with `websocket: bad handshake`. - [#6200](grafana/k6#6200) Leaves a counter's `rate` unset when the observed duration is zero, instead of computing `+Inf` and spuriously failing `rate` thresholds. Thanks, @samarth70! - [#6195](grafana/k6#6195) Initializes a gauge's maximum from the first sample so all-negative gauge series no longer report `max=0`. Thanks, @Solaris-star! - [#6145](grafana/k6#6145) Prevents the OpenTelemetry output from panicking at startup when basic auth is configured without `K6_OTEL_HEADERS`. Thanks, @lukdz! - [#6140](grafana/k6#6140) Stops `SharedArray` deep-freezing JS primitives, which needlessly wrapped large strings in `String` objects β cutting memory usage in the reported reproduction from roughly 1 GB to 100 MB. ## Maintenance and internal improvements - [#6126](grafana/k6#6126), [#6224](grafana/k6#6224), [#6229](grafana/k6#6229) Adds anonymous extension usage to the k6 usage report: a run reports the Go module path, version, and type of registry-cataloged extensions it actually uses (imported `k6/x/` modules, output extensions selected with `--out`, and `k6 x` subcommands). Private and unlisted extensions are never reported, and the existing `--no-usage-report` opt-out covers it. - [#6183](grafana/k6#6183), [#6218](grafana/k6#6218) Updates Sobek and regexp2, making `W See merge request: Harmonybrew/homebrew-core!16299
What?
This pull request adds support for
WritableStreamandWritableStreamDefaultWriterto the experimental streams module in k6, bringing the implementation closer to the WHATWG Streams specification. It introduces constructors, updates exports, and integrates new and patched web platform tests to validate the implementation. Some specification tests are commented out due to current limitations in the k6/Sobek event loop and lack of certain web APIs.Why?
This improves the streams module coverage of the WebApi specification, and opens the door to the implementation of the
TransformStreamwhich depends onWritableStream.ref #2974
ref #3919
Most important changes
Feature Implementation:
WritableStreamandWritableStreamDefaultWriterininternal/js/modules/k6/experimental/streams/module.go, including argument validation, strategy handling, and controller setup according to the WHATWG Streams spec.WritableStreamandWritableStreamDefaultWriterfrom the experimental streams module, making them available to JavaScript code in k6.Testing and Examples:
examples/experimental/writable-streams.jsdemonstrating how to use the newWritableStreamimplementation, including backpressure and logging.resources/recording-streams.jsin the test runtime setup.Web Platform Test Integration:
aborting.any.jsandreentrant-strategy.any.js. Some tests are commented out with detailed explanations due to differences in microtask scheduling and missing APIs (e.g.,AbortSignal,DOMException) in the current environment. [1] [2]Checklist
make check) and all pass.Checklist: Documentation (only for k6 maintainers and if relevant)
Please do not merge this PR until the following items are filled out.
Related PR(s)/Issue(s)