Skip to content

viewer: eliminate UI freeze and slider snap-back on option change - #15029

Merged
remibettan merged 10 commits into
realsenseai:developmentfrom
remibettan:viewer-ui-exp-change-avoid-stream-pause
Jun 11, 2026
Merged

remibettan merged 10 commits into
realsenseai:developmentfrom
remibettan:viewer-ui-exp-change-avoid-stream-pause

Conversation

@remibettan

@remibettan remibettan commented May 6, 2026 •

Copy link
Copy Markdown
Contributor

Tracked by: RSDSO-20803

Summary

Eliminates a ~1.5 s UI freeze in realsense-viewer on every option change while streaming, plus a slider snap-back regression (slider would briefly jump back to the old cached value before settling on the new one).

Three independent root causes, each fixed in its own commit, plus one peer-review-feedback commit:

  1. 1fd9686 — async option-write dispatch. New option_async_setter worker takes the FW round-trip off the UI thread; last_user_set_stopwatch gate (500 ms) keeps per-frame get_option_value() polling from contending on the per-device USB lock.
  2. 4fd1ff0 — async JSON config save. Moves save_processing_block_to_config_file x N + per-pp save_to_config_file (~400 ms of disk I/O) to a singleton config_save_worker with void*-keyed coalescing.
  3. 28a2f8f — user-request cache for slider feedback. Locally masks the stale option_model::value with the user's just-requested value until the FW echo arrives via options_watcher / update_all_fields (capped at 2 s for FW reject/clamp).
  4. 8dee2d9 — review feedback. _has_user_request atomic, dead error_message / ignore_period params dropped from set_option, LOG_WARNING for FW errors in the async setter, gate scoped to just the two paths it protects, set_option_async's redundant opt removed, config_save_worker lifted to a private nested class in subdevice-model.h, and on_chip_calib bypasses option_model for its synchronous set+verify path.

Thread-safety for rs2::config_file (recursive_mutex around all _j / on-disk access) has been split out to #15092 so it can land independently.

Each commit compiles independently.

Test plan

  • Drag Exposure slider rapidly - no freeze, no snap-back
  • Toggle AE / Backlight Compensation - no pause
  • Slider settles to FW-applied value if clamped (cache clears on echo)
  • On-chip calibration laser-emitter / thermal-loop verify still synchronous
  • All commits build (Release, MSVC) on their own

Copilot AI review requested due to automatic review settings May 6, 2026 10:58

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review this pull request because it exceeds the maximum number of files (300). Try reducing the number of changed files and requesting a review from Copilot again.

@remibettan
remibettan changed the base branch from master to development May 6, 2026 11:01
@remibettan
remibettan force-pushed the viewer-ui-exp-change-avoid-stream-pause branch from a39c375 to bfc3392 Compare May 6, 2026 11:07
Comment thread common/option-model.cpp Resolved

@Nir-Az Nir-Az left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.

@Nir-Az Nir-Az left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.

@Nir-Az Nir-Az left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.

@Nir-Az Nir-Az left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.

@Nir-Az Nir-Az left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.

@Nir-Az Nir-Az left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.

@Nir-Az Nir-Az left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.

@Nir-Az Nir-Az left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.

@Nir-Az Nir-Az left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.

@remibettan
remibettan requested a review from Copilot May 6, 2026 11:09

@Nir-Az Nir-Az left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.

@Nir-Az Nir-Az left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.

@Nir-Az Nir-Az left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.

@Nir-Az Nir-Az left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.

@Nir-Az Nir-Az left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.

@Nir-Az Nir-Az left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.

@Nir-Az Nir-Az left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.

@Nir-Az Nir-Az left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.

@Nir-Az Nir-Az left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.

@Nir-Az Nir-Az left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.

@Nir-Az Nir-Az left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.

@Nir-Az Nir-Az left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.

@Nir-Az Nir-Az left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.

@Nir-Az Nir-Az left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.

@Nir-Az Nir-Az left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.

Comment thread common/subdevice-model.h Resolved
@remibettan
remibettan force-pushed the viewer-ui-exp-change-avoid-stream-pause branch 2 times, most recently from 7273b3b to 8dee2d9 Compare May 25, 2026 11:45
Comment thread common/option-model.cpp Resolved
@remibettan
remibettan requested a review from Nir-Az May 26, 2026 10:33
@remibettan

Copy link
Copy Markdown
Contributor Author

Refactored the async-write layer to rsutils::dispatcher per the suggestion (8dff3e8).

What changed

  • One dispatcher per subdevice_model replaces the per-option option_async_setter threads. Single FIFO worker per sensor → cross-option ordering serialized, no more races on the per-device USB bus, fewer threads.
  • option_async_setter class deleted entirely (~70 LOC).
  • option_model::set_option_async is now a CAS + dispatcher::invoke. Per-option coalescing via two shared_ptr<std::atomic> members (_latest_pending_value, _has_pending_job): the action clears _has_pending_job at entry, reads back _latest_pending_value, calls endpoint->set_option, then cancellable_timer::try_sleep(200ms). A 60 Hz slider drag produces ~5 Hz of FW writes, never queues stale values, never blocks the UI thread.
  • The 200 ms gate in slider_selected (restored last commit) is removed again — the dispatcher's try_sleep enforces the same floor at the right layer, uniformly across slider / checkbox / calibration paths. slider_unselected is back to a no-op.
  • on_chip_calib bypass undone: set_laser_emitter_state / set_thermal_loop_state now call new option_model::set_option_sync, which invoke_and_waits through the same dispatcher — calibration writes can't interleave with concurrent UI writes anymore.

Net diff in this commit: −14 LOC across the 5 touched files (175 + / 189 −).

Lifetime safety: _set_dispatcher is declared after options_metadata in subdevice_model, so ~dispatcher stops/joins the worker before any option_model is destroyed. The dispatcher action captures only shared_ptrs — never this — so an in-flight FW call is UAF-safe regardless of option_model destruction order.

Let me know if the device-level (vs subdevice-level) variant is preferred — easy to lift the dispatcher one level up if cross-sensor serialization is wanted.

Comment thread common/option-model.h Resolved
remibettan and others added 7 commits May 31, 2026 15:20
… blocks on FW round-trip

Each option_model::set_option used to call endpoint->set_option synchronously
on the UI thread. That call is ~200 ms on UVC and serializes on the per-device
USB lock against options_watcher's 1 s poll cycle, so any slider drag or
checkbox toggle visibly froze the viewer.

Introduces option_async_setter, a per-option worker thread that coalesces
posted values (only the latest between two FW writes wins, no stale-value
queue), and routes set_option / slider_selected through it.

Adds last_user_set_stopwatch on subdevice_model: option_model::set_option_async
resets it on every dispatch, and subdevice_model::update() returns early for
500 ms after the last user write so its per-frame get_option_value() polling
and the JSON config-save block don't fight the in-flight worker write on the
same USB bus. The cached value still refreshes during the gate via
options_watcher -> on_options_changed -> update_value.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
When _options_invalidated is set, subdevice_model::update() previously called
save_processing_block_to_config_file 5x plus once per post-processing model on
the UI thread. Each call iterates the block's options and runs config_file::set
per option, and every set rewrites the entire JSON config to disk synchronously
(see common/rs-config.cpp). For a typical depth+color setup that's tens of full
file rewrites per change — ~400 ms of disk-IO blocking the render loop.

Adds an anonymous-namespace config_save_worker singleton with one persistent
thread. update() now captures the processing-block shared_ptrs by value and
posts the save lambda to the worker; the UI thread returns immediately. Posts
coalesce by opaque void* key (subdevice identity), so rapid invalidation events
collapse into at most one save pass per wake-up cycle.

The lambda never dereferences `this` — UAF-safe even if the subdevice is
destroyed mid-save. ~subdevice_model still calls cancel(this) to drop any
not-yet-started job for tidy shutdown.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
… in flight

After dispatching an option write asynchronously, the cached
option_model::value isn't refreshed until either the options_watcher callback
fires (~1 s later) or the post-gate update_all_fields poll reads it back. In
the meantime, draw_slider re-reads value_as_float() on the next render frame
and the slider visually snaps back to the old value, then jumps to the new
value once the echo finally arrives — a regression introduced by making the
write async.

Adds an optimistic local cache (_optimistic_value + stopwatch) seeded by
set_option_async. value_as_float() returns the optimistic value while it's
fresh (capped at 2 s in case the FW rejects/clamps and never echoes the
requested value), so the slider stays pinned to the user's choice from the
moment they release.

The flag is cleared as soon as either authoritative path refreshes value:
update_value (options_watcher callback) or update_all_fields (per-frame poll).
After clearing, value_as_float() returns whatever FW actually applied — so a
clamped/rejected value shows real applied state instead of the user's request.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Make _has_optimistic atomic via a shared_ptr<atomic<bool>> wrapper. The flag
  is read on the UI thread (value_as_float) and written on both the UI thread
  (set_option_async, update_all_fields) and the on_options_changed callback
  thread (update_value); shared_ptr keeps option_model copyable since
  std::atomic deletes its copy/move ctors. Reorder writes in set_option_async
  so observers that load the flag set also see matching _optimistic_value /
  _optimistic_stopwatch.
- Drop dead error_message / ignore_period parameters from
  option_model::set_option (now fire-and-forget). Update all call sites
  accordingly. Reduce slider_unselected to a no-op since the rate-limit retry
  path it guarded is now unreachable.
- Bypass option_model in on_chip_calib_manager: calibration needs synchronous
  set/verify semantics, so call sensor->set_option / sensor->get_option
  directly instead of going through the (now async) UI cache.
- Log FW errors from option_async_setter::run with LOG_WARNING for rs2::error
  / std::exception / unknown - silent rejections/clamps were undiagnosable
  before.
- Scope the 500 ms USB-bus contention gate to just the two paths it protects
  (config save + per-frame option polling) instead of returning early from
  the entire subdevice_model::update().

Config-file thread-safety (recursive_mutex inside config_file) has been split
out to PR realsenseai#15092 so it can be reviewed independently.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@remibettan
remibettan force-pushed the viewer-ui-exp-change-avoid-stream-pause branch from 4c60fe6 to 33ec131 Compare May 31, 2026 12:21
Comment thread common/option-model.cpp Resolved
Comment thread common/option-model.cpp Resolved
Comment thread common/on-chip-calib.cpp Resolved
@Nir-Az

Nir-Az commented May 31, 2026 •

Copy link
Copy Markdown
Collaborator

Suggestion: tie invalidate + log to the actual FW write, and floor writes at 200 ms

Right now the three things below are decoupled from when the FW write actually happens:

  • Invalidate — if (invalidate_flag) *invalidate_flag = true; is set unconditionally on the UI thread at dispatch time, i.e. before the queued set_option runs. So subdevice_model::update() can reset next_option = 0 and re-poll all options from FW while the write is still in flight, reading back the old value.
  • Log — model.add_log(... "Setting " << opt << " to " << value) is throttled to ~5 Hz inside slider_selected, independent of whether a FW write was actually issued.
  • FW write spacing — the action's try_sleep floor is currently 50 ms (commit 33ec131), so writes can go out at ~20 Hz. Note the 200 ms last_set_stopwatch gate in slider_selected only throttles add_log, not the FW write.

The pre-PR behavior was: invalidate + log happened only after set_option returned OK (an actual FW write), and the 200 ms ignore_period gate kept writes/invalidates to ~5 Hz. I'd like to restore that with the async setter:

  1. At most one FW write per 200 ms.
  2. Invalidate only after the write was sent to FW.
  3. Log only when the set is actually called, with the value that was sent.

All three should be driven by the worker action. We can reuse the option_async_state channel that already carries last_error back to the UI thread via draw_option.

1. Raise the FW-write floor back to 200 ms in the action (it's currently 50 ms at line 895, not absent):

c.try_sleep( std::chrono::milliseconds( 200 ) );   // currently 50 ms

2. Extend option_async_state to report a completed write:

struct option_async_state
{
    std::mutex   mutex;
    std::string  last_error;
    bool         did_write = false;   // a FW write was actually performed
    float        written_value = 0.f; // the value that was sent to FW
};

3. In the worker lambda, record the write on success (next to the existing error handling):

if( ! err_msg.empty() )
{
    std::lock_guard< std::mutex > lk( state->mutex );
    state->last_error = err_msg;
    has_user_request->store( false );
}
else
{
    std::lock_guard< std::mutex > lk( state->mutex );
    state->did_write     = true;
    state->written_value = v;
}

4. Remove the four unconditional UI-thread invalidate lines (draw_slider, draw_checkbox, draw_combobox, slider_selected) and the throttled log in slider_selected.

5. Drain the new flags in draw_option (already on the UI thread, already swapping last_error):

std::string async_err;
bool  did_write = false;
float written_value = 0.f;
{
    std::lock_guard< std::mutex > lk( _async_state->mutex );
    async_err.swap( _async_state->last_error );
    did_write     = _async_state->did_write;
    written_value = _async_state->written_value;
    _async_state->did_write = false;
}
// ... existing async_err -> error_message handling ...
if( did_write )
{
    if( invalidate_flag )
        *invalidate_flag = true;
    model.add_log( rsutils::string::from() << "Setting " << opt << " to " << written_value );
}

This keeps the write to *invalidate_flag / _options_invalidated and add_log on the UI thread (no data race, no UAF — the worker only touches the shared _async_state), and because the dispatcher coalesces, a 60 Hz drag collapses to ~1 FW write / 200 ms, each producing exactly one invalidate + one log — matching the old gated cadence.

Note: a successful write is also echoed per-option via options_watcher → update_value, so the only thing invalidate adds is the cross-option re-poll (e.g. AE → exposure/gain coupling). That's the bit we want to keep firing only after FW actually applied the value.

Comment thread common/subdevice-model.h Resolved
@remibettan

Copy link
Copy Markdown
Contributor Author

Done in 3bfeda3 - followed the plan you laid out almost verbatim:

  1. FW-write floor back to 200 ms in the action's c.try_sleep.
  2. option_async_state extended with did_write + written_value.
  3. Worker lambda records both on the success branch (else of the existing error capture), under the same _async_state->mutex.
  4. Removed the four unconditional UI-thread *invalidate_flag = true; lines - draw_slider (edit-field submit), draw_checkbox, draw_combobox, slider_selected. Also removed the throttled add_log in slider_selected.
  5. draw_option drains did_write / written_value in the same swap that already handles last_error, then fires *invalidate_flag = true; and model.add_log("Setting " << opt << " to " << written_value) only when an actual FW write completed.

Result: a 60 Hz slider drag now produces ~5 Hz of FW writes, each producing exactly one invalidate + one log - matching the pre-PR gated cadence. Cross-option re-polls (AE ? exposure/gain) only fire after the FW has actually applied the value.

Kept the leading add_log("Setting X to Y (label)") lines in draw_checkbox / draw_combobox / draw_slider edit-field as-is - they fire on user intent and carry the human-readable label, which the did_write log doesn't. Happy to remove those too if you'd prefer a single log per FW write.

Comment thread common/option-model.cpp Resolved

@Nir-Az Nir-Az left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!

@remibettan
remibettan merged commit 13c8ccc into realsenseai:development Jun 11, 2026
29 of 30 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants