Repository navigation
FPS drop and stream latency on Jetson ROS #9879
Description
| Required Info | |
|---|---|
| Camera Model | D435i |
| Firmware Version | 5.12.07.100 |
| Operating System & Version | L4T-32.4.4 |
| Kernel Version (Linux Only) | 4.9.140-tegra |
| Platform | Jetson Xavier AGX |
| SDK Version | 2.49.0 |
| Language | ROS melodic + realsense ROS wrapper 2.3.1 |
| Segment | Robot |
Issue Description
Stream FPS drop and latencies (hundreds of microseconds up to a few seconds) occur in ROS Jetson when pointcloud or align depth is enabled.
The issue can be reproduced by running roslaunch realsense2_camera rs_camera.launch. Below are given some cases which reproduce the issue. FPS drop was measured with rostopic hz <topic> and latencies are observed by displaying stream with rosurun rviz rviz and rosrun image_view image_view image:=<image topic>.
| enable pointcloud | align depth | filters | RGB Hz | depth Hz | Comment |
|---|---|---|---|---|---|
| false | false | - | 30 | 30 | - |
| true | false | - | 10 | 24 | issue present |
| true | false | colorizer | 9 | 22 | issue present |
| true | false | decimation | 30 | 30 | - |
| true | false | disparity | 17 | 30 | issue present |
| true | true | - | 20 | 40 | issue present |
| true | true | colorizer | 16 | 27 | issue present |
| true | true | decimation | 12 | 34 | issue present |
| true | true | disparity | 11 | 45 | issue present |
| false | true | - | 30 | 30 | - |
| false | true | colorizer | 24 | 36 | issue present |
| false | true | decimation | 30 | 30 | - |
| false | true | disparity | 17 | 40 | issue present |
LibRealSense
Realsense is built from source with CUDA support cmake .. -DBUILD_EXAMPLES=true -DCMAKE_BUILD_TYPE=release -DFORCE_RSUSB_BACKEND=false -DBUILD_WITH_CUDA=true. Native kernel drivers are used and the L4T kernel is patched with ./scripts/patch-realsense-ubuntu-L4T.sh.
ROS launch
Lunch enables two camera streams: RGB (1280x720, RGB8, 30FPS) and, depth (848x480, Z16, 30FPS)
The launch is built and run with librealsense v2.49.0.
CPU load
On the issue, the system has increased CPU consumption (sometimes one core is fully loaded). Our Jetson AGX system is configured to full power mode (30W) with all cores enabled.
CPU load when pointcloud and align depth used
CPU load when pointcloud and align depth not used
Reproducing in RealSense Viewer
In the viewer, similar could be reproduced when GLSL processing and rendering were disabled. Is it possible that ROS uses LibRealSense without GLSL support?
Similar issues
A similar issue has already been reported #9519, but it is closed without an adequate solution.
Activity
MartyG-RealSense commentedon Oct 20, 2021
Hi @nkorenicFR The RealSense ROS wrapper version used should ideally be matched as closely as possible with the librealsense SDK version listed for that particular wrapper in its release notes. The librealsense version to match with wrapper 2.31 is 2.48.0. There is not a wrapper for matching with 2.49.0, and the next wrapper version should be for upcoming librealsense version 2.50.0.
In regard to GLSL, it is not enabled in RealSense applications by default and has to be deliberately incorporated into programs, like it is in the RealSense Viewer.
nkorenicFR commentedon Oct 20, 2021
Hi @MartyG-RealSense ,
Thank you for your fast response. I will try with librealsense 2.48.0.
Regarding GLSL in ROS wrapper, i guess that
realsense2-glmodule (#3654) is not used, as i could not locaters2::gl::pointcloudorrs2::gl::colorize. Could we expect better ROS performances (FPS, stream latency) by adding GLSL to the RealSense ROS wrapper?MartyG-RealSense commentedon Oct 20, 2021
I am not personally aware of anyone who has tried adjusting the rs2:: instructions in the ROS wrapper to convert them to rs2::gl:: instructions. The key lines in the wrapper's base_realsense_node.cpp to look at for each type of rs2:: operation are likely to be the ones below (I say 'likely' as I am not involved in the wrapper's development).
Pointcloud
https://github.com/IntelRealSense/realsense-ros/blob/development/realsense2_camera/src/base_realsense_node.cpp#L1273
Align
https://github.com/IntelRealSense/realsense-ros/blob/development/realsense2_camera/src/base_realsense_node.cpp#L1253
Colorizer
https://github.com/IntelRealSense/realsense-ros/blob/development/realsense2_camera/src/base_realsense_node.cpp#L1258
nkorenicFR commentedon Oct 21, 2021
Same behavior with RealSense SDK 2.48.0, the issue is still present. Any idea what else could we try?
Does Intel have any benchmark results on ROS jetson that can be used for comparison? Is this expected behavior on the Jetson platform?
MartyG-RealSense commentedon Oct 21, 2021
Hi @nkorenicFR Dips in performance when enabling pointcloud or align in the RealSense ROS wrapper with Jetson boards is an issue that has previously been reported by a small number of Jetson users, as mentioned in #9519 (comment)
There are not ROS benchmark results available from Intel that compare pointcloud and align performance on Jetson with non-Jetson systems. Intel did publish a CPU performance table of alignment with CUDA versus non-CUDA when CUDA support was first introduced in the librealsense SDK though. That table can be found at the link below.
#2670
MartyG-RealSense commentedon Oct 25, 2026
This open RealSense GitHub issue is being closed because of its significant age. Thank you.