Related: 1248897

Paul is currently managing the development of this.

This will take some time to complete. Possibly into 2026. Hard to judge at this point as the infrastructure to do speech recognition is still being fleshed out by the ML team.

This allows skipping unnecessary copies if the caller has the data in
nsTArray.

We'll use it for speech recognition, and that doesn't need any quality
at all.

This includes the Metal shader source in asm and compiles it. Specific
symbols are then picked up by the implementation, and the byte range is
fed to the Metal shader compiler. The resulting object file is
subsequently cached by the OS.

First compilation is in the order of 4-5s, subsequent instantiation is
10-15ms.

This backend will always ever run in a dedicated process, that has a
sandbox very similar to the GPU process, but doesn't have access to the
display server, fonts, and other facilities related to rendering. It can
access one more folder than the GPU process (read/write) to cache the
compiled shaders.

This benefits llama.cpp, but also whisper.cpp, that is introduced in a
later patch. This can multiply the performance by multiple dozen times
(depending on the hardware and use-case). More importantly, it
completely offload inference to the GPU. While (e.g.) real-time local
speech recognition is running with this backend, about:processes
show all thread of the HWInference process as "idle".

Whisper.cpp is a C++ speech recognition library that uses models of the
Whisper family. It allows performing local real-time and faster-than
real-time local speech recognition in many languages.

This library uses libggml (that we already vendor in llama.cpp) as its
backend for most computations. I have synchronized the version so it
works well with what we have, and will have to synchronize updates as
well.

This is compiled into libmozinference.

A patch is added on top of the upstream library, to be able to load a speech
recognition model from a FILE*, instead of a file path. This is passed
down by the main process when starting recognition, similarly to what we
do for llama.cpp. This patch is likely upstreamable.

The HWInference process is a new type of process, that is created from a
content process (e.g. to implement a Web API), but can also talk to the
main process (e.g. to interact with ModelHub and acquire ML models).

It doesn't run script. It has a dedicated sandbox policy (in a
subsequent patch), that is roughly similar to the GPU process policy,
with everything related to graphics rendering removed (connection to the
display server, access to fonts, misc system calls and privileges,
etc.), as it is just for hardware accelerated computations.

This patch only contains the scaffolding, and the first actor
(PSpeechRecognition) will be introduced in another patch.q

This has never been shipped and the endpoints aren't functional anymore,
we can just remove everything. I don't think we'll want to ship
non-local speech recognition at this point.

This is mostly to ease review, we'll fill in the blanks in a later
patch.

https://webaudio.github.io/web-speech-api/#speechreco-section

The SpeechRecognitionBackend object (to be introduced in a subsequent
patch) handles all the "actual" recognition, but this object handles the
DOM side.

A noted in the code, a number of things are unclear in the standard,
that we're working on addressing.

This removes a little bit of the previous backend (online recognition)
that was left after the previous patch removed most of it, and implements:

• available()
• install()
• SpeechRecognitionPhrase
• a new error code
• final attribute of SpeechRecognitionResult

most of the WPTs are manual, and need work anyway, this will happen in
another patch as a followup.

This object handles:

• Getting audio from a MediaTrackGraph, downmixing it, queuing it to a ring
  buffer
• Running a processing loop on a dedicated thread (called the resampling
  thread), that dequeues from the ring buffer, downsamples the audio to 16kHz
  (which is the rate at which the inference is running), and sends it over to the
  IPC thread, that will send it to the HWInference process in chunks of (by
  default) 500ms
• Receive events from the HWInference process, and relay them to the main thread
  (notably speech recognition events, but also errors and such) object,
  SpeechRecognition
• Running another (static) thread (called the IPC thread) on which the IPC will
  run. This is done this way because available and install calls are static, while
  the actual speech recognition process require an instance, and our IPC system
  requires all IPC calls to be on the same thread.

This also contains some code removal and simplifications for the graph listener.

This allows calling into ModelHub from native code. This also adds a
couple testing facilities, and a little bit of formatting fixes.

The testing facilities allow almost end-to-end testing of some
Web Speech API feature, testing all the IPC, but sidestepping the very
last part about having an actual model.

Only the facilities SpeechRecognition needs are exposed via this small
XPCOM component: check availability, download a mode, with progress
callbacks, and getting an fd for a model file stored in ModelHub.

SpeechRecognition.available() allows knowing if a particular language is
supported. This patch starts in the content process, instantiate a
SpeechRecognition actor, sending a request to the HWInference process,
that then asks the parent process, at which point the ModelHub can be
queried (on the main thread).

This is very similar to the previous patch (available()), but leverages
the ModelHub to download a model, with progress callback. The download
progress isn't relayed to content (intentionnaly, for fingerprinting
reasons), but a completion callback is (error or success).

Again, this starts in the content process, uses a transient
PSpeechRecognition actor to reach the HWInference process, that asks the
main process (on its main thread) to download a model (or resolve the
promise immediately if already downloaded -- ModelHub implements a
cache).

This is be later used when initializing the speech recognition backend.
The files range from a hundreds megabyte to multiple gigabytes
(depending on the quality and how many languages they can handle), so we
really need to avoid a copy here.

This is essentially a copy-paste of the GPU process policy, trimmed down
to removed everything that wasn't needed. One thing that has been added
is read/write permissions to the /plugincontainer directory so shader
compilation caching works.

We have discussed changing the executable name to relinquish allowing
executable page, this hasn't been implemented.

This also include just enough code so that it compiles and run on Linux,
but that side needs work.

We could imagine renaming this LlamaRuntimeLinker but I didn't bother.

This is similar to what we do for llama.cpp in the Inference process,
but native-IPC style instead of JS style.

This requires bouncing through a couple threads, but we create a
dedicated thread, that will then run the actual recognition process, in
this patch.

This patch receives the audio via IPC, puts it in an SPSC queue. This is
periodically dequeued, and prepared for recognition. The actual
recognition loop is still being tuned and will change, but this is the
general idea.

Two types of result can be sent back: interim result (also called
non-final in the spec), and final results. Interim results are often
early recognition of a complete segment of text. Final results are
usually recognition on a larger part of audio, and it is often the case
that interim results are substring of final results. Sometimes however,
the context brought by new words recognized allow getting a better
recognition in the early part of the sentence. After some time, we
consider the recognition correct, a final result is sent, and the audio
is reset, keeping a little bit, in case we were cutting a word.

Again, this is work in progress, but largely works.

I'll add more test later, but this passes already.

I'll turn this into a real documentation for this module later, but lots
of information here can help reviewering the code, so I'm pushing it now
so it can be read by reviewers, as phabricator builds and renders the
docs.

Just for convenience

To all reviewers, first, thanks, second, I've written https://phabricator.services.mozilla.com/D268417 to ease the reviews and offer higher level consideration and information about what is done what isn't done yet, but there is also a lot of info in the commit messages. http://gecko-docs.mozilla.org-l1.s3-website.us-west-2.amazonaws.com/main/b741edf7-cec5-11f0-a2b7-76a1964522a4/media/SpeechRecognition.html is that doc rendered. The diagrams are in svg so are zoomable despite looking small.

I'm on PTOs for some time, then will be at TPAC, so I will look at the comments in the second part of November at the earliest, hopefully leaving enough time to review all this code.

It generally works well though, https://paul.cx/public/speech.html is a small testing page that transcribes a famous speech in real-time. It is working on macOS (hardware accelerated) and Linux (with inference running on the CPU, will be accelerated of course), and probably Windows (also accelerated).

This patch stasrts filling in the blanks in SpeechRecognitionBackend. This
object sits behind the SpeechRecogntion DOM object, and implements some light
audio processing, and IPC communication.

Audio processing is left for the next patch. This one implements the static
methods, available and install, that call the IPDL methods of the same name.

More importantly this patch implement a threading architecture that works for
real-time audio, and DOM API, static and instance methods.

The constraints are the following:

• We're going to send method calls (available, install, start, stop, etc.) to
  the HWInference process, those come from the main thread (Web Speech API)
• We're also going to need to send real-time audio to the same process (with a
  graduingly lower and lower packet size, to reduce end-to-end latency). This come
  from a hard-real-time thread (from our MediaTrackGraph)
• Our IPC system requires all method calls to be on a single thread
• The real-time audio thread can't do any system calls or unbounded operations
  (can't do IPC, locking, allocations, etc.)
• We really don't want to bounce the real-time audio via the main thread for IPC
  or make it more busy if we can avoid it (it's already a nightmare in a typical
  media processing web app)
• We can't affort scheduling hazards associated with using a shared thread (e.g.
  the shared background thread), because we want to have low and constant latency
• There are both static and instance methods to implement

The design is therefore the following:

• A dedicated SpeechIPC thread is created, and shut down as soon as it's not
  needed. It is kept as a static member on the class
• IPC only ever runs on that dedicated thread
• Static method calls are dispatched to that thread, and the result is
  dispatched back to the main thread.
• In the next patch, processed and packetized audio is dispatched to that
  thread as well
• Whenever a Speech Recognition session is active, or while an available or
  install call is in flight, it is kept alive
• Otherwise, we shut it down as soon as we can (session stop, result of
  available or install)

This patch adds:

• Getting audio from a MediaTrackGraph, downmixing it, queuing it to a ring
  buffer
• Running a processing loop on a dedicated thread (called the resampling
  thread), that dequeues from the ring buffer, downsamples the audio to 16kHz
  (which is the rate at which the inference is running), and sends it over to the
  IPC thread, that will send it to the HWInference process in chunks of (by
  default) 500ms
• Receive events from the HWInference process, and relay them to the main thread
  (notably speech recognition events, but also errors and such) object,
  SpeechRecognition

The backend now has a weak ref on the DOM object.

The DOM object gets DisconnectFromOwner, and that clears out the backend (and
that cascades to everything behind, such as the HWInference process).

NotifyTrackEnded() implementation is more or less as if "Stop()" had been
called.

This will probably get folded in the right patches.

The following patches are waiting for review from a reviewer who resigned from the review:

:padenot, could you please find another reviewer?

For more information, please visit BugBot documentation.

Comment on attachment 9519754 [details]
Bug 1940906 - Sandbox policy for the HWInference utility process. r?haik

Revision D268412 was moved to bug 2051970. Setting attachment 9519754 [details] to obsolete.

Comment on attachment 9519744 [details]
Bug 1940906 - Introduce the HWInference utility process. r?#ipc-reviewers,gerard-majax

Revision D268403 was moved to bug 2051970. Setting attachment 9519744 [details] to obsolete.

Comment on attachment 9519750 [details]
Bug 1940906 - Add an XPCOM interface to ModelHub.sys.mjs, add available() to ModelHub. r?tarek,#xpcom-reviewers

Revision D268408 was moved to bug 2051970. Setting attachment 9519750 [details] to obsolete.

Comment on attachment 9519753 [details]
Bug 1940906 - Allow getting a Blob of a model file in the HWInference process, cross-process, without copies. r?#ipc-reviewers

Revision D268411 was moved to bug 2051970. Setting attachment 9519753 [details] to obsolete.

Comment on attachment 9519742 [details]
Bug 1940906 - Vendor and compile the Metal backend for libggml. r?#firefox-build-system-reviewers,atossou

Revision D268401 was moved to bug 2051973. Setting attachment 9519742 [details] to obsolete.

The interface was renamed to SpeechRecognitionErrorEvent in the Web Speech
API spec; align the Gecko interface, its binding, and the (now non-nullable)
message attribute / required init members accordingly.

Send audio to the inference process in ~40ms blocks instead of 0.5s, so
end-to-end latency is bounded by the streaming model (~180ms) rather than IPC
batching. Implement the chunked downmixing path in DataCallback (slice-wise,
RT-safe) instead of crashing, and stop clearing the graph-thread scratch
buffer from the main thread in Stop() (a data race the lower latency exposed).

A development bench that drives the full local recognition pipeline: it plays
a 60s clip through a MediaStreamTrack into SpeechRecognition and compares the
transcript against a reference. It relies on a local CORS model server
(testing/tools/serve_model.py) and bundled audio, so it is not wired for CI yet.

Feeds audio with lang="auto" to route to nvidia/nemotron-3.5-asr-streaming-0.6b
and exercise the larger prompt-conditioned model end-to-end: fd load, cache-aware
streaming, language resolution and inline <lang>-tag stripping.

parakeet-stream-words.patch adds parakeet_capi_stream_drain_words, returning the
words finalized each streaming step as {text, start, end, conf} structs so the
host gets per-word timing + confidence without parsing the JSON feed. Exposed
through the runtime linker wrapper and mozinference.symbols.

The streaming backend drains per-word confidences and emits each step's words
as one final result with the mean confidence, plumbed through
OnRecognitionResult(transcript, isFinal, confidence) to SpeechRecognitionResult
Per-word timestamps are kept internal, but are logged.

This will allow synchronizing recognition and sound tightly, possibly
by delaying the audio a little bit.

Timestamps are estimated from the actual audio capture position rather
than stamped at dispatch/processing time, to avoid folding buffering
and recognition latency into the reported time.

A fetch manifest (static-url from the moz-model-hub bucket, the same pattern
as the onnxruntime/wllama fetches) downloads the two streaming GGUFs into
MOZ_FETCHES_DIR so the tests never hit the network. The local model-hub
server (serve_model.py) now resolves model requests by basename against
MOZ_FETCHES_DIR / MOZ_ML_LOCAL_DIR, falling back to the source tree for dev,
so a flat directory of fetched .gguf files satisfies ModelHub's nested
<modelName>/<revision>/<file> path.

runtests.py starts serve_model.py (bundled into the harness via
TEST_HARNESS_FILES) when a test tagged 'parakeet-asr' runs, mirroring the
websocketprocessbridge pattern, and stops it afterwards.

The parakeet tests move to their own 'speech-recognition' subsuite so the
large GGUF model artifacts are fetched (via the parakeet-models fetch tasks)
only for this task rather than the whole media suite. Registers the suite in
the test resolver, defines the taskcluster task (loopback-video, model
fetches), and registers it in mozharness.

Heavily instrument both tests: check, in order, that the on-device model is
reachable (SpeechRecognition.available/install), the AudioContext actually
reaches "running" (not just created), real audio is flowing into the
captured MediaStreamTrack (FFT levels, not merely that a track object
exists), and every recognition lifecycle event fires, with timestamps. Print
a periodic status line plus a final summary, and drop the now-unused debug
canvas code. Load the audio as a same-origin support file instead of an
external fetch.

Add a models.yaml manifest (gen_speech_models.py) so
LanguagesToModelIdentifier picks a model per locale prefix, overridable
via media.webspeech.recognition.model.<prefix> prefs, instead of always
using a single hardcoded parakeet model.

Add a SpeechRecognitionModelDownloadPermissionPrompt shown before
SpeechRecognition.install() downloads the model, skipping it when the
model is already cached (IsModelInstalled), and guard the install flow
against the window detaching mid-flight.

Reject the install() promise in the caller's realm (not the dead frame's) when
the document is not fully active, so the promise reliably settles, while
still building the DOMException in the frame's realm for correct
instanceof behaviour. Add mUnspokenPunctuation as a stored (currently
inert) field, and IsLanguageDownloading().

Enable media.webspeech.recognition.enable (plus the testing/mock prefs)
for the speech-api WPT directory and update the expectation metadata for
the now-passing recognition tests.

Behind media.webspeech.recognition.testing, resolve model availability
and install requests in HWInferenceParent from an in-process set instead
of the ModelHub (no network, IndexedDB or downloads), while still
exercising the full content->utility->parent IPC chain. Used by the WPTs.

Also adds media.webspeech.recognition.model.en and
media.webspeech.recognition.model.multilingual prefs to override the
default model per locale prefix.

Per spec, unspokenPunctuation defaults to false; the value is stored for
round-tripping but does not change recognition behaviour, since Gecko's
recognizer is LLM-based and only infers punctuation that was not spoken.

Covers the doorhanger shown by SpeechRecognition.install(): accept/dismiss,
and the regression where install() must still show the prompt even when
available() already reports the model as available (available() means
'possible to download', not 'already installed').

serve_model.py also gets verbose diagnostic logging, useful when a test
fails because the model server couldn't resolve a file.

Stress tests that fire overlapping available()/install()/start()/stop()/abort()
calls, including from iframes torn down mid-flight, using the in-process mock
backend. These found the two crashes fixed in the previous commit; they are
nets for concurrency bugs in the IPC/session lifecycle, not targeted
regression tests for a single issue.

test_fuzz_iframe_teardown.html surfaces a known-but-unfixed issue: a promise
from a detached frame's available()/install() call can fail to ever settle.
Bounded with a timeout so it reports as a TODO instead of hanging the test.

Also add test_stress_init_abort_race.html, a targeted, deterministic
regression test for a deadlock in SpeechRecognitionParent::ActorDestroy():
it used to hold mLock across a blocking mRecognitionThread->Shutdown() while
InitializeParakeetContext() (running on that same thread) also acquires
mLock during startup. A new testing-only pref,
media.webspeech.recognition.testing.parakeet_init_delay_ms, delays thread
init so abort() reliably races ActorDestroy() against it.

Add shared helpers to head.js: createResumedAudioContext() (bounded, fails
fast if resume() itself doesn't settle), AudioStreamAnalyser, and
waitForAudioFlowing() (fails fast instead of hanging on a downstream event
that will never fire), used by the mochitests here and de-duplicated from
the parakeet e2e/multilingual tests.

Audio capture doesn't produce real signal under Wayland in CI, so
recognition always returns an empty transcript and the tests fail.

isModelAvailable implicitely installs it, which isn't what we want for Web Speech.

When on-device speech recognition is disabled in AI Controls, a page's
install()/start() now shows a notification bar (matching the EME/DRM
drmContentDisabled pattern) whose "Open Settings" button deep-links to the AI
Controls setting (about:preferences#ai, spotlighting speechRecognition). The
gate is read live, so re-enabling lets a subsequent call proceed.

The bar is driven by the SpeechRecognitionUI JSWindowActor: SpeechRecognition
fires a "speech-recognition-ai-blocked" observer notification, the actor child
forwards it to the parent, which shows the bar on the requesting tab.

Blocking the feature (via the UI toggle, enterprise policy, about:config or
sync) also deletes the downloaded model: a parent-process observer of
browser.ai.control.speechRecognition / .default evicts the parakeet-gguf engine,
matching the teardown other AI-Controls features already do.

Comment on attachment 9615285 [details]
Bug 1940906 - Add HWInference architecture and security documentation. r?#ai-platform-reviewers

Revision D313816 was moved to bug 2051970. Setting attachment 9615285 [details] to obsolete.

Speech recognition decides how long the HWInference process lives, via the
generic keep-alive introduced earlier in the stack. sIPCActorUsers counts
IPCActorUserGuards, and one is held:

• for the lifetime of every SpeechRecognition object, taken in its constructor
  via AcquireProcessKeepAlive() (which only counts, and does not establish the
  connection) and dropped in DisconnectFromOwner() as well as the destructor,
  so a torn-down window releases it without waiting for GC;
• for each in-flight static available()/install()/IsModelInstalledNative()
  call, via RunWithTransientSession();
• for an active recognition session, from EnsureIPC().

Reaching zero does not close the connection immediately: it arms a
media.webspeech.recognition.idle_shutdown_grace_ms (5s) timer, cancelled by the
next Increment(), so a stop()/start() cycle or a burst of static calls reuses
the warm process instead of paying for a relaunch. 0 closes immediately, which
is what tests use. The grace period is a speech recognition policy and lives
here rather than in UtilityProcessManager, which stays a plain refcount so
other HWInference consumers can choose differently.

IPCActorUserGuard moves out of SpeechRecognitionBackend to namespace scope so
SpeechRecognition can hold one by forward declaration.

Adds keepOpen action support to PopupNotifications, threaded through
PermissionUI's promptActions, with a browser_popupNotification.js case.

Also adds missing @param JSDoc tags to a handful of nearby PopupNotifications
functions (eslint drive-by).

SpeechModelResolver expands a model id (from LanguagesToSpeechModelId) to its
concrete ModelHub artifact for HWInferenceParent's generic model-resolution
lookup (@mozilla.org/ml/model-resolver;1?task=speech-recognition). Needed
starting with the next patch's RecvIsModelInstalled, which resolves ids
through that lookup and does nothing useful without a registered resolver.

browser_speech_recognition_process_lifetime.js covers the original bug (run a
session, close the tab, the process is gone), a live-but-idle
SpeechRecognition object holding the process up, a static call releasing it
once it settles, and grace-period reuse.

It sits here rather than with the keep-alive code because it needs
browser.ml.modelHub.testing, which does nothing until the in-process mock
below it: without that, available() would hit the real ModelHub and start()
would try to fetch a real model.

pangrams.opus is two sentences separated by ~4s of silence. The test snapshots
the transcript inside that silence and checks it already ends in the first
sentence's last word, rather than that word only surfacing once the second
sentence starts being decoded.

Currently, about:addons relies on a specific way to organize ModelHub
and also how Hugging Face works: ModelHub slugs mirror HuggingFace slugs
(same org/model, a bit like on github).

For SpeechRecognition, it's a little bit different: we take multiple
models that come from different Hugging Face pages from one Hugging Face
repo (https://huggingface.co/mudler/parakeet-cpp-gguf/). They are
conversions of models from Nvidia. We'd like to link to the original
repo (that has much more informative info), and we also want better
names, so we'll be in a position to say "Speech Recognition model
(multilingual)" instead of the inscrutable model name (with a link to go
further for the nerds / interested people).

This is a simple idirection layer that any developer of a feature that
uses AI models can use to display things better. For Speech Recognition,
the information comes from a small js module generated from a yaml. This
yaml also generates some C++ for other things, so we can't risk being
out of sync here.

Covers the abort() half of what this file already covers for stop(): a session
that gets both reports nothing but "end", including once the flush the stop()
asked for has finished.

https://hg.mozilla.org/mozilla-central/rev/76a91a8b51b5

Oops we need 40 more patches here.

When on-device speech recognition is turned off in AI Controls, the three
entry points already refuse, but silently as far as the page author is
concerned: start() throws a NotAllowedError that reads like a permission
problem, and available() just says "unavailable". Point at the actual cause
once per document, in the web console.

install() resolved a NotAllowedError rejection for this case, which made the
AI Controls state observable as a distinct failure and differed from
available(). It now resolves false, like every other reason a model cannot be
installed.

The metal backend had drifted a little bit, and we made a couple
unconsequential mistakes while vendoring. Bug 2068580 is a followup to
clean this up.

JUnitTestRunner reuses MochitestDesktop.startServers but has no test
manifest, so needsParakeetModelServer's getActiveTests call throws.

We haven't enabled SpeechRecognition on mobile yet.

Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/62404 for changes under testing/web-platform/tests

https://hg.mozilla.org/mozilla-central/rev/ad20bb8144ee
https://hg.mozilla.org/mozilla-central/rev/764ecdb8b327
https://hg.mozilla.org/mozilla-central/rev/723281f599fa
https://hg.mozilla.org/mozilla-central/rev/fc71997e19c0
https://hg.mozilla.org/mozilla-central/rev/b9b16a80f385
https://hg.mozilla.org/mozilla-central/rev/aaa52ccd2da0
https://hg.mozilla.org/mozilla-central/rev/3fa5389d6adc
https://hg.mozilla.org/mozilla-central/rev/c773092a2695
https://hg.mozilla.org/mozilla-central/rev/352f6ac0a01f
https://hg.mozilla.org/mozilla-central/rev/c8d2011bcfe3
https://hg.mozilla.org/mozilla-central/rev/fcdd2169fed0
https://hg.mozilla.org/mozilla-central/rev/facbf30b71e6
https://hg.mozilla.org/mozilla-central/rev/c98c596d6788
https://hg.mozilla.org/mozilla-central/rev/17d6fb5e38b2
https://hg.mozilla.org/mozilla-central/rev/af985efef11d
https://hg.mozilla.org/mozilla-central/rev/f4747bcbe2ee
https://hg.mozilla.org/mozilla-central/rev/b161ee94e683
https://hg.mozilla.org/mozilla-central/rev/a5abe06b3bd9
https://hg.mozilla.org/mozilla-central/rev/53e21fa3cccb
https://hg.mozilla.org/mozilla-central/rev/6650833ea34c
https://hg.mozilla.org/mozilla-central/rev/1c42c02def21
https://hg.mozilla.org/mozilla-central/rev/22caefe37cf5
https://hg.mozilla.org/mozilla-central/rev/5f1db0126254
https://hg.mozilla.org/mozilla-central/rev/3452467e2c22
https://hg.mozilla.org/mozilla-central/rev/06a79cfa8831
https://hg.mozilla.org/mozilla-central/rev/818efb029e12
https://hg.mozilla.org/mozilla-central/rev/98f0f5557b59
https://hg.mozilla.org/mozilla-central/rev/0c2f411b465c
https://hg.mozilla.org/mozilla-central/rev/d675db79d04f
https://hg.mozilla.org/mozilla-central/rev/040efbb92305
https://hg.mozilla.org/mozilla-central/rev/fa7c2c36c186
https://hg.mozilla.org/mozilla-central/rev/27eea798e905
https://hg.mozilla.org/mozilla-central/rev/2b956adb2fa6
https://hg.mozilla.org/mozilla-central/rev/beb3938be317
https://hg.mozilla.org/mozilla-central/rev/403b90ae2bbd
https://hg.mozilla.org/mozilla-central/rev/91084438f8f0
https://hg.mozilla.org/mozilla-central/rev/3aa7f36dc009
https://hg.mozilla.org/mozilla-central/rev/a7a500ba2acf
https://hg.mozilla.org/mozilla-central/rev/217bf069d487
https://hg.mozilla.org/mozilla-central/rev/07a9ee2d38aa
https://hg.mozilla.org/mozilla-central/rev/4e54da63cad0
https://hg.mozilla.org/mozilla-central/rev/04671bba8e69
https://hg.mozilla.org/mozilla-central/rev/062036d9cd4d
https://hg.mozilla.org/mozilla-central/rev/7a169d5827c2
https://hg.mozilla.org/mozilla-central/rev/e24aef9aa4e8
https://hg.mozilla.org/mozilla-central/rev/f1996b83eecd
https://hg.mozilla.org/mozilla-central/rev/960ac7f60e12

Upstream PR merged by moz-wptsync-bot

Paul, is that something we should mention in our nightly release notes? Thanks
https://wiki.mozilla.org/Release_Management/Release_Notes_Nomination

(In reply to Pascal Chevrel (relman team) -> :pascalc from comment #97)

Paul, is that something we should mention in our nightly release notes? Thanks
https://wiki.mozilla.org/Release_Management/Release_Notes_Nomination

When preffed-on, certainly, but we're missing a couple patches. I just filed https://bugzilla.mozilla.org/show_bug.cgi?id=2069803 with dev-doc-needed, thanks!

Perfherder has detected a talos performance change from push 04671bba8e6911391c55e8cebf1adedb4b276710.

No action is required from the author; this comment is provided for informational purposes only.

Need Help or Information?

If you have any questions, please reach out to fbilt@mozilla.com. Alternatively, you can find help on Slack by joining #perf-help, and on Matrix you can find help by joining #perftest.

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests.

Hi there,

I'm currently looking into updating the MDN compat data to mark on-device speech recognition as supported, and I have a few questions:

1. Which prefs need to be set to true to test this at present? Just setting media.webspeech.recognition.enable to true seems to enable the features, but the demo I wrote around the time that Chrome started to support this (https://mdn.github.io/dom-examples/web-speech-api/on-device-speech-color-changer/) doesn't seem to work as expected. The recognition seems to be a bit flaky, and once recognition has occurred, the recognition server doesn't seem to stop as expected (I'm calling stop() on speechend).
2. Which features does Firefox support at present in terms of the on-device stuff? I'm assuming SpeechRecognition.[available|install|processLocally], but do we also include the contextual biasing features?
3. We don't support Permissions-Policy yet on Fx, right? So the on-device-speech-recognition is ignored, and the functionality always just works, for now?