Migrate from SDL2 to SDL3

[rom1v]

First, this PR contains some fixes that could be merged directly into dev.

Then, there are some refactors to make the transition easier, especially the handling of integer scrolling (#6156 (comment)) and the configuration of the color space at texture creation.

Finally, there's a "big" migration commit, followed by the changes for the build script and the GitHub Actions script.

Links:

• SDL3 migration docs: https://wiki.libsdl.org/SDL3/README-migration
• Relevant discussion about the new SDL3 audio API for scrcpy: SDL3 audio API: missing SDL_AudioSpec.samples feature libsdl-org/SDL#13319
• Release binaries from GitHub Actions (download scrcpy-release-sdl3): https://github.com/rom1v/scrcpy/actions/runs/16226120027

---

Tests and reviews welcome.

Remaining questions:

• check if the "continuous resizing" workaround is still necessary, or if it can be removed (0b1e591) (answer: ❌ no Migrate from SDL2 to SDL3 #6216 (comment))
• is it possible with SDL3 to keep the window aspect ratio while resizing (instead of adding black bars)? (Option to lock aspect ratio of window to screen #3460)

---

I don't plan to merge this immediately, SDL3 is not available yet on many distros (not even in current Debian stable).

I'd like to implement hardware decoding first (#3800 | libsdl-org/SDL#5405 (comment) | libsdl-org/SDL#8366), and if everything works as expected, release a new major version (scrcpy 4.0).

EDIT: hardware decoding will not be for 4.0. Flex display was implemented instead (#6772).

[yume-chan]

Quickly tried it on Windows (using the prebuilt binary) and it keeps printing Unexpected SDL audio behavior: too much data requested

┖[~\Downloads\scrcpy-win64-sdl3]> .\scrcpy.exe
scrcpy 3.3.1 <https://github.com/Genymobile/scrcpy>
INFO: ADB device found:
INFO:     -->   (usb)  73294bba                        device  M2011K2C
C:\Users\simon\Downloads\scrcpy-win64-sdl3\scrcpy-server: ...file pushed, 0 skipped. 112.9 MB/s (90788 bytes in 0.001s)
[server] INFO: Device: [Xiaomi] Xiaomi M2011K2C (Android 13)
INFO: Renderer: direct3d11
INFO: Texture: 1440x3200
ERROR: Unexpected SDL audio behavior: too much data requested
ERROR: Unexpected SDL audio behavior: too much data requested
ERROR: Unexpected SDL audio behavior: too much data requested
ERROR: Unexpected SDL audio behavior: too much data requested
ERROR: Unexpected SDL audio behavior: too much data requested
ERROR: Unexpected SDL audio behavior: too much data requested
ERROR: Unexpected SDL audio behavior: too much data requested
ERROR: Unexpected SDL audio behavior: too much data requested
ERROR: Unexpected SDL audio behavior: too much data requested
ERROR: Unexpected SDL audio behavior: too much data requested
ERROR: Unexpected SDL audio behavior: too much data requested
ERROR: Unexpected SDL audio behavior: too much data requested

Enabling -Vverbose shows more [Audio] Buffering threshold exceeded messages

ERROR: Unexpected SDL audio behavior: too much data requested
DEBUG: [Audio] Buffering threshold exceeded, skipping 240 samples
ERROR: Unexpected SDL audio behavior: too much data requested
DEBUG: [Audio] Buffering threshold exceeded, skipping 480 samples
ERROR: Unexpected SDL audio behavior: too much data requested
ERROR: Unexpected SDL audio behavior: too much data requested
ERROR: Unexpected SDL audio behavior: too much data requested
DEBUG: [Audio] Buffering threshold exceeded, skipping 480 samples
ERROR: Unexpected SDL audio behavior: too much data requested
ERROR: Unexpected SDL audio behavior: too much data requested
DEBUG: [Audio] Buffering threshold exceeded, skipping 240 samples
ERROR: Unexpected SDL audio behavior: too much data requested
ERROR: Unexpected SDL audio behavior: too much data requested
DEBUG: [Audio] Buffering threshold exceeded, skipping 480 samples
ERROR: Unexpected SDL audio behavior: too much data requested
ERROR: Unexpected SDL audio behavior: too much data requested
DEBUG: [Audio] Buffering threshold exceeded, skipping 480 samples
ERROR: Unexpected SDL audio behavior: too much data requested
ERROR: Unexpected SDL audio behavior: too much data requested
DEBUG: [Audio] Buffering threshold exceeded, skipping 480 samples
DEBUG: [Audio] Buffering threshold exceeded, skipping 960 samples

[rom1v]

Thank you for your tests!

I assumed (on purpose, to detect early if that was false) that the additional_amount requested by SDL3 never exceeds the "audio output buffer" hint. It is true on my machine, but apparently not everywhere (I only tested on Linux).

How much data does it require:

diff --git app/src/audio_player.c app/src/audio_player.c
index e1a4134e7..6ca13a1e0 100644
--- app/src/audio_player.c
+++ app/src/audio_player.c
@@ -21,7 +21,8 @@ sc_audio_player_stream_callback(void *userdata, SDL_AudioStream *stream,
         assert(len <= ap->aout_buffer_size);
         if (len > ap->aout_buffer_size) {
             // Just in case for release builds
-            LOGE("Unexpected SDL audio behavior: too much data requested");
+            LOGE("Unexpected SDL audio behavior: too much data requested: %d",
+                 additional_amount);
             len = ap->aout_buffer_size;
         }
 

?

Is it different with --audio-output-buffer=10 or --audio-output-buffer=20? (default is 5)

On SDL2, do you need to use --audio-output-buffer=xx to get audio working correctly?

[yume-chan]

How much data does it require:

additional_amount is always 3840, and audio sounds like a robot

Is it different with --audio-output-buffer=10 or --audio-output-buffer=20?

There is no error with either --audio-output-buffer=10 or --audio-output-buffer=20, additional_amount is still 3840, and audio is working

On SDL2, do you need to use --audio-output-buffer=xx to get audio working correctly?

No

[rom1v]

There is no error with either --audio-output-buffer=10 or --audio-output-buffer=20, and audio is working

What is the value of additional_amount in these cases?

@icculus Sorry to ping you here, but do you know why setting SDL_HINT_AUDIO_DEVICE_SAMPLE_FRAMES to 240 (5 milliseconds of samples, 1920 bytes for 2 channels of 4 bytes each) can result in the callback being called with additional_amount = 3840 (instead of 1920) on Windows?

Apparently, on SDL2, the callback was correctly called with 1920 bytes. @yume-chan, this can be confirmed on scrcpy 3.3.1 with:

diff --git app/src/audio_player.c app/src/audio_player.c
index 9413c2ea8..07979b31b 100644
--- app/src/audio_player.c
+++ app/src/audio_player.c
@@ -11,6 +11,8 @@ static void SDLCALL
 sc_audio_player_sdl_callback(void *userdata, uint8_t *stream, int len_int) {
     struct sc_audio_player *ap = userdata;
 
+    LOGI("=== SDL2 audio callback: %d bytes (%d samples)", len_int, len_int / 8);
+
     assert(len_int > 0);
     size_t len = len_int;
 

[icculus]

Not awake yet, but I'll try the Windows build today and see what happens.

[metayan]

scrcpy with SDL3 can even be built on macOS Catalina with MacOSX_SDK11 using

SDKROOT=/Library/Developer/CommandLineTools/SDKs/MacOSX11.1.sdk ./release/build_macos.sh x86_64

and it works splendidly, both with OpenGL and Metal.
I have an older Android phone, so can't test audio.

[metayan]

diff --git a/app/deps/sdl.sh b/app/deps/sdl.sh
index ae79ae9c..a308ae9e 100755
--- a/app/deps/sdl.sh
+++ b/app/deps/sdl.sh
@@ -28,7 +28,7 @@ export CXXFLAGS="$CFLAGS"
 
 if [[ -d "$DIRNAME" ]]
 then
-    echo "'$PWD/$HDIRNAME' already exists, not reconfigured"
+    echo "'$PWD/$DIRNAME' already exists, not reconfigured"
     cd "$DIRNAME"
 else
     mkdir "$DIRNAME"
@@ -41,7 +41,7 @@ else
     if [[ "$HOST" == linux ]]
     then
         conf+=(
-            -DSDL_WAYLOAD=ON
+            -DSDL_WAYLAND=ON
             -DSDL_X11=ON
         )
     fi

[metayan]

Also tested with SDL 3.2.18 and it works fine on macOS. Audio not tested.

diff --git a/app/deps/sdl.sh b/app/deps/sdl.sh
index ae79ae9c..970fcbfa 100755
--- a/app/deps/sdl.sh
+++ b/app/deps/sdl.sh
@@ -5,10 +5,10 @@ cd "$DEPS_DIR"
 . common
 process_args "$@"
 
-VERSION=3.2.16
+VERSION=3.2.18
 FILENAME=SDL-$VERSION.tar.gz
 PROJECT_DIR=SDL-release-$VERSION
-SHA256SUM=a1af401fb49b824ba721683862439e014498f79753602827e174f3613357aff0
+SHA256SUM=51539fa13e546bc50c632beed3f34257de2baa38a4c642048de56377903b4265
 
 cd "$SOURCES_DIR"

[yume-chan]

What is the value of additional_amount in these cases?

It's still 3840

diff --git a/app/src/audio_player.c b/app/src/audio_player.c
index e1a4134e..71fbef2a 100644
--- a/app/src/audio_player.c
+++ b/app/src/audio_player.c
@@ -16,6 +16,8 @@ sc_audio_player_stream_callback(void *userdata, SDL_AudioStream *stream,
     struct sc_audio_player *ap = userdata;
 
     if (additional_amount > 0) {
+        LOGE("additional_amount: %d", additional_amount);
+
         size_t len = additional_amount;
 
         assert(len <= ap->aout_buffer_size);
@@ -82,6 +84,9 @@ sc_audio_player_frame_sink_open(struct sc_frame_sink *sink,
     int r = snprintf(str, sizeof(str), "%" PRIu16, (uint16_t) aout_samples);
     assert(r >= 0 && (size_t) r < sizeof(str));
     (void) r;
+
+    LOGE("setting SDL_HINT_AUDIO_DEVICE_SAMPLE_FRAMES to %s", str);
+
     if (!SDL_SetHint(SDL_HINT_AUDIO_DEVICE_SAMPLE_FRAMES, str)) {
         LOGE("Could not set audio output buffer");
         sc_audio_regulator_destroy(&ap->audioreg);

> ./scrcpy --audio-output-buffer=10
scrcpy 3.3.1 <https://github.com/Genymobile/scrcpy>
INFO: ADB device found:
INFO:     -->   (usb)  73294bba                        device  M2011K2C
D:\dev\yume-chan\scrcpy-devcontainer\scrcpy\server\build\o...file pushed, 0 skipped. 102.8 MB/s (91004 bytes in 0.001s)
[server] INFO: Device: [Xiaomi] Xiaomi M2011K2C (Android 13)
INFO: Renderer: direct3d11
ERROR: setting SDL_HINT_AUDIO_DEVICE_SAMPLE_FRAMES to 480
ERROR: additional_amount: 3840
ERROR: additional_amount: 3840
ERROR: additional_amount: 3840
ERROR: additional_amount: 3840
ERROR: additional_amount: 3840
ERROR: additional_amount: 3840

> ./scrcpy --audio-output-buffer=20
scrcpy 3.3.1 <https://github.com/Genymobile/scrcpy>
INFO: ADB device found:
INFO:     -->   (usb)  73294bba                        device  M2011K2C
D:\dev\yume-chan\scrcpy-devcontainer\scrcpy\server\build\o...file pushed, 0 skipped. 107.7 MB/s (91004 bytes in 0.001s)
[server] INFO: Device: [Xiaomi] Xiaomi M2011K2C (Android 13)
INFO: Renderer: direct3d11
ERROR: setting SDL_HINT_AUDIO_DEVICE_SAMPLE_FRAMES to 960
ERROR: additional_amount: 3840
ERROR: additional_amount: 3840
ERROR: additional_amount: 3840
INFO: Texture: 1440x3200
ERROR: additional_amount: 3840
ERROR: additional_amount: 3840

---

this can be confirmed on scrcpy 3.3.1

> ./scrcpy
scrcpy 3.3.1 <https://github.com/Genymobile/scrcpy>
INFO: ADB device found:
INFO:     -->   (usb)  73294bba                        device  M2011K2C
D:\dev\yume-chan\scrcpy-devcontainer\scrcpy\server\build\o... file pushed, 0 skipped. 99.5 MB/s (91004 bytes in 0.001s)
[server] INFO: Device: [Xiaomi] Xiaomi M2011K2C (Android 13)
INFO: Renderer: direct3d
INFO: Texture: 1440x3200
ERROR: === SDL2 audio callback: 1920 bytes (240 samples)
ERROR: === SDL2 audio callback: 1920 bytes (240 samples)
ERROR: === SDL2 audio callback: 1920 bytes (240 samples)
ERROR: === SDL2 audio callback: 1920 bytes (240 samples)
ERROR: === SDL2 audio callback: 1920 bytes (240 samples)

> ./scrcpy --audio-output-buffer=10
scrcpy 3.3.1 <https://github.com/Genymobile/scrcpy>
INFO: ADB device found:
INFO:     -->   (usb)  73294bba                        device  M2011K2C
D:\dev\yume-chan\scrcpy-devcontainer\scrcpy\server\build\o...file pushed, 0 skipped. 104.3 MB/s (91004 bytes in 0.001s)
[server] INFO: Device: [Xiaomi] Xiaomi M2011K2C (Android 13)
INFO: Renderer: direct3d
INFO: Texture: 1440x3200
ERROR: === SDL2 audio callback: 3840 bytes (480 samples)
ERROR: === SDL2 audio callback: 3840 bytes (480 samples)
ERROR: === SDL2 audio callback: 3840 bytes (480 samples)
ERROR: === SDL2 audio callback: 3840 bytes (480 samples)
ERROR: === SDL2 audio callback: 3840 bytes (480 samples)
ERROR: === SDL2 audio callback: 3840 bytes (480 samples)

> ./scrcpy --audio-output-buffer=20
scrcpy 3.3.1 <https://github.com/Genymobile/scrcpy>
INFO: ADB device found:
INFO:     -->   (usb)  73294bba                        device  M2011K2C
D:\dev\yume-chan\scrcpy-devcontainer\scrcpy\server\build\o...file pushed, 0 skipped. 103.7 MB/s (91004 bytes in 0.001s)
[server] INFO: Device: [Xiaomi] Xiaomi M2011K2C (Android 13)
INFO: Renderer: direct3d
INFO: Texture: 1440x3200
ERROR: === SDL2 audio callback: 7680 bytes (960 samples)
ERROR: === SDL2 audio callback: 7680 bytes (960 samples)
ERROR: === SDL2 audio callback: 7680 bytes (960 samples)
ERROR: === SDL2 audio callback: 7680 bytes (960 samples)
ERROR: === SDL2 audio callback: 7680 bytes (960 samples)

[metayan]

Numbers and punctuation now only work with --raw-key-events.

[yume-chan]

https://github.com/libsdl-org/SDL/blob/bc5c9a686ca01c7cffa45e223efcd0cb7ef1efb7/src/audio/wasapi/SDL_wasapi.c#L759-L760

SDL3 calls IAudioClient3::GetSharedModeEnginePeriod and clamps device->sample_frames. min_period_in_frames and max_period_in_frames are both 480, so additional_amount is always 3840. Not sure what it means, nor how does SDL2 works.

---

directsound audio driver doesn't work with the default --audio-output-buffer=5 neither, additional_amount is 1920 as expected, but audio played is broken:

log

> ./scrcpy --audio-output-buffer=5
scrcpy 3.3.1 <https://github.com/Genymobile/scrcpy>
INFO: ADB device found:
INFO:     -->   (usb)  73294bba                        device  M2011K2C
DEBUG: Device serial: 73294bba
DEBUG: Using SCRCPY_SERVER_PATH: D:\dev\yume-chan\scrcpy-devcontainer\scrcpy\server\build\outputs\apk\release\server-release-unsigned.apk
D:\dev\yume-chan\scrcpy-devcontainer\scrcpy\server\build\o...file pushed, 0 skipped. 104.2 MB/s (91004 bytes in 0.001s)
[server] INFO: Device: [Xiaomi] Xiaomi M2011K2C (Android 13)
[server] DEBUG: Using audio encoder: 'c2.android.opus.encoder'
[server] DEBUG: Using video encoder: 'c2.qti.avc.encoder'
[server] DEBUG: Display: using SurfaceControl API
DEBUG: Server connected
DEBUG: Starting controller thread
DEBUG: Starting receiver thread
DEBUG: Using icon (portable): D:\dev\yume-chan\scrcpy-devcontainer\scrcpy\release\work\build-win64\dist\icon.png
INFO: Renderer: direct3d11
DEBUG: Trilinear filtering disabled (not an OpenGL renderer)
DEBUG: Demuxer 'video': starting thread
DEBUG: Demuxer 'audio': starting thread
INFO: Texture: 1440x3200
ERROR: additional_amount: 1920
ERROR: additional_amount: 1920
ERROR: additional_amount: 1920
ERROR: additional_amount: 1920
ERROR: additional_amount: 1920
ERROR: additional_amount: 1920
ERROR: additional_amount: 1920
ERROR: additional_amount: 1920
DEBUG: [Audio] Buffering threshold exceeded, skipping 240 samples
ERROR: additional_amount: 1920
ERROR: additional_amount: 1920
DEBUG: [Audio] Buffering threshold exceeded, skipping 480 samples
ERROR: additional_amount: 1920
ERROR: additional_amount: 1920
DEBUG: [Audio] Buffering threshold exceeded, skipping 480 samples
ERROR: additional_amount: 1920
ERROR: additional_amount: 1920
DEBUG: [Audio] Buffering threshold exceeded, skipping 480 samples
ERROR: additional_amount: 1920
ERROR: additional_amount: 1920
ERROR: additional_amount: 1920
DEBUG: [Audio] Buffering threshold exceeded, skipping 480 samples
ERROR: additional_amount: 1920
ERROR: additional_amount: 1920
DEBUG: [Audio] Buffering threshold exceeded, skipping 240 samples

[rom1v]

Numbers and punctuation now only work with --raw-key-events.

Oh, SDL_EVENT_TEXT_INPUT is never received:

scrcpy/app/src/input_manager.c

Line 1019 in 3615c75

case SDL_EVENT_TEXT_INPUT:

Reported here: libsdl-org/SDL#13380

[rom1v]

directsound audio driver doesn't work with the default --audio-output-buffer=5 neither, additional_amount is 1920 as expected, but audio played is broken

Is it also the case on SDL2?

--audio-output-buffer was added because some audio outputs on Windows did not support 5ms, so if the behavior was the same on SDL2 (and respects the sample_frames value), it's ok I guess: #3793

[rom1v]

Oh, SDL_EVENT_TEXT_INPUT is never received

Should be fixed on the new branch.

I added:

diff --git app/src/screen.c app/src/screen.c
index 27259e830..6ecb64167 100644
--- app/src/screen.c
+++ app/src/screen.c
@@ -394,6 +394,12 @@ sc_screen_init(struct sc_screen *screen,
         goto error_destroy_fps_counter;
     }
 
+    ok = SDL_StartTextInput(screen->window);
+    if (!ok) {
+        LOGE("Could not enable text input: %s", SDL_GetError());
+        goto error_destroy_window;
+    }
+
     SDL_Surface *icon = scrcpy_icon_load();
     if (icon) {
         SDL_SetWindowIcon(screen->window, icon);

(cf libsdl-org/SDL#13380)

As a consequence, it might open an IME on scrcpy start (libsdl-org/SDL#13380 (comment)). If that's the case (to be tested), we need a workaround.

[yume-chan]

directsound audio driver doesn't work with the default --audio-output-buffer=5 neither, additional_amount is 1920 as expected, but audio played is broken

Is it also the case on SDL2?

Yes, with SDL2, directsound driver, and --audio-output-buffer=5, audio playback is also broken. wasapi driver works.

[yume-chan]

check if the "continuous resizing" workaround is still necessary, or if it can be removed (0b1e591)

It still doesn't work, video stream pauses when moving or resizing window.

I think libsdl-org/SDL#1059 (comment) only applies automatically when using main callbacks, otherwise the SC_EVENT_NEW_FRAME custom events are still not delivered (#3458 (comment))

[rom1v]

Thank you 👍

Maybe it can make sense to use SDL main callbacks (to be tested). This will require to rework the way the scrcpy-server is started, but in the end in might simplify (or not).

[rom1v]

@yume-chan

=== SDL2 audio callback: 1920 bytes (240 samples)

Could you please also add the timestamp of the call, to see if the callback is called twice in a row: libsdl-org/SDL#13397

[szmarczak]

I don't plan to merge this immediately, SDL3 is not available yet on many distros (not even in current Debian stable).

Why not use a statically linked lib? Unless it's lots of megabytes in size but I'd still accept the tradeoff for better color accuracy.

[rom1v]

Why not use a statically linked lib

This is one of the possibilities: https://github.com/Genymobile/scrcpy/blob/master/doc/linux.md#from-the-official-release

But I would like to avoid a dependency "too recent", which is not even packaged (Debian stable is my reference, if it's in stable, it's not "too recent").

Anyway, it will take some time to resolve the remaining issues with SDL3.

[adamponi]

Hi,

I compiled the SDL3 branch in Termux, and it works well with the default settings. However, when using the --render=vulkan option, the colors appear inverted, although scrcpy remains usable.

In contrast, the precompiled build linked in the first post, which I ran on Windows, performs much worse. When launched with the --render=vulkan parameter, the screen displays only glitches, and the image is completely unreadable.

Downloads\scrcpy-release-sdl3\scrcpy-win64-sdl3> ./scrcpy --render=vulkan --no-audio
scrcpy 3.3.1 <https://github.com/Genymobile/scrcpy>
INFO: ADB device found:
INFO:     -->   (usb)  ec8faaa3                        device  xxxxxxxxxxx
C:\Users\xxxxxxx\Downloads\scrcpy-release-sdl3\scrcpy-... file pushed, 0 skipped. 27.7 MB/s (90788 bytes in 0.003s)
[server] INFO: Device: [Xiaomi] Redmi xxxxxxxxxxxxxxxxxx (Android 13)
INFO: Renderer: vulkan
INFO: Texture: 1224x2712

If there’s anything I can do to help diagnose or resolve this issue, please let me know. I’m happy to provide logs or any other information needed.

For reference, on Windows 11 I have the latest graphics drivers installed for the Intel® Core™ Ultra 5 Processor 135U.

Thanks!

[rom1v]

Looks like there is an alignment issue. Maybe because in 1224x2712, 1224 and 2712 are not multiple of 16. (see #55)
Try -m1024 or something else until you get both dimensions multiple of 16.

Can you reproduce with scrcpy + SDL2?

[rom1v]

🚀