Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

realsense viewer unable to receive imu data #11548

Closed
Karin-Sugi opened this issue Mar 9, 2023 · 5 comments
Closed

realsense viewer unable to receive imu data #11548

Karin-Sugi opened this issue Mar 9, 2023 · 5 comments

Comments

@Karin-Sugi
Copy link

Karin-Sugi commented Mar 9, 2023

Hi, I'm using realsense d435i with Ubuntu18.04 on RaspberryPi 4.
I installed librealsense with following commands because using sudo apt install command didn't work as document says.

$ git clone https://github.com/IntelRealSense/librealsense.git
$ cd librealsense/
$ sudo apt install -y git libssl-dev libusb-1.0-0-dev pkg-config libgtk-3-dev
$ sudo apt install -y libglfw3-dev libgl1-mesa-dev libglu1-mesa-dev
$ sudo apt install -y cmake
$ ./scripts/setup_udev_rules.sh
$ ./scripts/patch-realsense-ubuntu-lts.sh
$ mkdir build && cd build
$ cmake ../ -DFORCE_RSUSB_BACKEND=true -DCMAKE_BUILD_TYPE=release -DBUILD_EXAMPLES=true -DBUILD_GRAPHICAL_EXAMPLES=true
$ sudo make clean && make && sudo make install

I can see depth&RGB image correctly. But about imu data, viewer says "Nothing is streaming! toggle to start" even when the toggle is ON. Also, when i motion module turns on, everything (include depth&RGB image) dissapear from the viewer and says "Nothing is streaming!"

How can I fix it?? Thanks in advance.

error messages are below. First, I started the viewer, then turn on depth image -> RGB image -> motion module.

ubuntu@ubuntu:~$ realsense-viewer
 09/03 19:50:22,850 INFO [281473142566928] (synthetic-stream-gl.cpp:80) Initializing rendering, GLSL=0
 09/03 19:50:22,850 INFO [281473142566928] (synthetic-stream-gl.cpp:89)  0 GPU objects initialized
 09/03 19:50:23,069 INFO [281473142566928] (context.cpp:382) Found 1 RealSense devices (mask 0xff)
 09/03 19:50:23,213 INFO [281473142566928] (rs.cpp:2701) Framebuffer size changed to 1344 x 896
 09/03 19:50:23,214 INFO [281473142566928] (rs.cpp:2701) Scale Factor is now 1
 09/03 19:50:24,397 INFO [281473142566928] (context.cpp:382) Found 1 RealSense devices (mask 0xfe)
 09/03 19:50:41,067 INFO [281473142566928] (sensor.cpp:1586) Request: Z16 Depth, 
Resolved to: Z16 Depth, 
 09/03 19:50:41,109 INFO [281473142566928] (uvc-streamer.cpp:28) endpoint 82 read buffer size: 815104
 09/03 19:50:41,126 WARNING [281472351068608] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
 09/03 19:50:41,795 ERROR [281471940022720] (synthetic-stream.cpp:48) Exception was thrown during callback: Out of frame resources!
 09/03 19:50:41,795 ERROR [281471940022720] (synthetic-stream.cpp:48) Exception was thrown during callback: Error occured during execution of the processing block! See the log for more info
 09/03 19:50:41,832 ERROR [281471940022720] (synthetic-stream.cpp:48) Exception was thrown during callback: Out of frame resources!
 09/03 19:50:41,832 ERROR [281471940022720] (synthetic-stream.cpp:48) Exception was thrown during callback: Error occured during execution of the processing block! See the log for more info
 09/03 19:50:41,865 ERROR [281471940022720] (synthetic-stream.cpp:48) Exception was thrown during callback: Out of frame resources!
 09/03 19:50:41,865 ERROR [281471940022720] (synthetic-stream.cpp:48) Exception was thrown during callback: Error occured during execution of the processing block! See the log for more info
 09/03 19:50:41,899 ERROR [281471940022720] (synthetic-stream.cpp:48) Exception was thrown during callback: Out of frame resources!
 09/03 19:50:41,899 ERROR [281471940022720] (synthetic-stream.cpp:48) Exception was thrown during callback: Error occured during execution of the processing block! See the log for more info
 09/03 19:50:41,928 ERROR [281471940022720] (synthetic-stream.cpp:48) Exception was thrown during callback: Out of frame resources!
 09/03 19:50:41,928 ERROR [281471940022720] (synthetic-stream.cpp:48) Exception was thrown during callback: Error occured during execution of the processing block! See the log for more info
 09/03 19:50:41,963 ERROR [281471940022720] (synthetic-stream.cpp:48) Exception was thrown during callback: Out of frame resources!
 09/03 19:50:41,963 ERROR [281471940022720] (synthetic-stream.cpp:48) Exception was thrown during callback: Error occured during execution of the processing block! See the log for more info
 09/03 19:50:41,999 ERROR [281471940022720] (synthetic-stream.cpp:48) Exception was thrown during callback: Out of frame resources!
 09/03 19:50:42,000 ERROR [281471940022720] (synthetic-stream.cpp:48) Exception was thrown during callback: Error occured during execution of the processing block! See the log for more info
 09/03 19:50:42,031 ERROR [281471940022720] (synthetic-stream.cpp:48) Exception was thrown during callback: Out of frame resources!
 09/03 19:50:42,032 ERROR [281471940022720] (synthetic-stream.cpp:48) Exception was thrown during callback: Error occured during execution of the processing block! See the log for more info
 09/03 19:50:42,066 ERROR [281471940022720] (synthetic-stream.cpp:48) Exception was thrown during callback: Out of frame resources!
 09/03 19:50:42,066 ERROR [281471940022720] (synthetic-stream.cpp:48) Exception was thrown during callback: Error occured during execution of the processing block! See the log for more info
 09/03 19:50:42,099 ERROR [281471940022720] (synthetic-stream.cpp:48) Exception was thrown during callback: Out of frame resources!
 09/03 19:50:42,099 ERROR [281471940022720] (synthetic-stream.cpp:48) Exception was thrown during callback: Error occured during execution of the processing block! See the log for more info
 09/03 19:50:42,132 ERROR [281471940022720] (synthetic-stream.cpp:48) Exception was thrown during callback: Out of frame resources!
 09/03 19:50:42,132 ERROR [281471940022720] (synthetic-stream.cpp:48) Exception was thrown during callback: Error occured during execution of the processing block! See the log for more info
 09/03 19:50:42,165 ERROR [281471940022720] (synthetic-stream.cpp:48) Exception was thrown during callback: Out of frame resources!
 09/03 19:50:42,165 ERROR [281471940022720] (synthetic-stream.cpp:48) Exception was thrown during callback: Error occured during execution of the processing block! See the log for more info
 09/03 19:50:42,198 ERROR [281471940022720] (synthetic-stream.cpp:48) Exception was thrown during callback: Out of frame resources!
 09/03 19:50:42,198 ERROR [281471940022720] (synthetic-stream.cpp:48) Exception was thrown during callback: Error occured during execution of the processing block! See the log for more info
 09/03 19:50:42,233 ERROR [281471940022720] (synthetic-stream.cpp:48) Exception was thrown during callback: Out of frame resources!
 09/03 19:50:42,233 ERROR [281471940022720] (synthetic-stream.cpp:48) Exception was thrown during callback: Error occured during execution of the processing block! See the log for more info
 09/03 19:50:42,265 ERROR [281471940022720] (synthetic-stream.cpp:48) Exception was thrown during callback: Out of frame resources!
 09/03 19:50:42,265 ERROR [281471940022720] (synthetic-stream.cpp:48) Exception was thrown during callback: Error occured during execution of the processing block! See the log for more info
 09/03 19:50:42,299 ERROR [281471940022720] (synthetic-stream.cpp:48) Exception was thrown during callback: Out of frame resources!
 09/03 19:50:42,299 ERROR [281471940022720] (synthetic-stream.cpp:48) Exception was thrown during callback: Error occured during execution of the processing block! See the log for more info
 09/03 19:50:42,827 ERROR [281471940022720] (synthetic-stream.cpp:48) Exception was thrown during callback: Out of frame resources!
 09/03 19:50:42,828 ERROR [281471940022720] (synthetic-stream.cpp:48) Exception was thrown during callback: Error occured during execution of the processing block! See the log for more info
 09/03 19:50:42,861 ERROR [281471940022720] (synthetic-stream.cpp:48) Exception was thrown during callback: Out of frame resources!
 09/03 19:50:42,861 ERROR [281471940022720] (synthetic-stream.cpp:48) Exception was thrown during callback: Error occured during execution of the processing block! See the log for more info
 09/03 19:50:42,894 ERROR [281471940022720] (synthetic-stream.cpp:48) Exception was thrown during callback: Out of frame resources!
 09/03 19:50:42,894 ERROR [281471940022720] (synthetic-stream.cpp:48) Exception was thrown during callback: Error occured during execution of the processing block! See the log for more info
 09/03 19:50:42,937 ERROR [281471940022720] (synthetic-stream.cpp:48) Exception was thrown during callback: Out of frame resources!
 09/03 19:50:42,938 ERROR [281471940022720] (synthetic-stream.cpp:48) Exception was thrown during callback: Error occured during execution of the processing block! See the log for more info
 09/03 19:50:43,731 INFO [281473142566928] (sensor.cpp:1586) Request: RGB8 Color, 
Resolved to: YUYV Color, 
 09/03 19:50:43,769 INFO [281473142566928] (uvc-streamer.cpp:28) endpoint 132 read buffer size: 1844224
 09/03 19:50:44,305 INFO [281471923237312] (metadata-parser.h:334) Frame counter reset
 09/03 19:50:44,306 INFO [281471923237312] (metadata-parser.h:334) Frame counter reset
 09/03 19:50:44,306 INFO [281471923237312] (metadata-parser.h:334) Frame counter reset
 09/03 19:50:44,307 INFO [281471923237312] (sensor.cpp:40) Frame Callback Color #0 overdue. (FPS: 30, max duration: 32.258064 ms)
 09/03 19:50:44,328 INFO [281471923237312] (metadata-parser.h:334) Frame counter reset
 09/03 19:50:44,342 INFO [281471956808128] (metadata-parser.h:334) Frame counter reset
 09/03 19:50:44,400 INFO [281471923237312] (sensor.cpp:40) Frame Callback Color #3 overdue. (FPS: 30, max duration: 32.258064 ms)
 09/03 19:50:44,599 INFO [281471923237312] (sensor.cpp:40) Frame Callback Color #8 overdue. (FPS: 30, max duration: 32.258064 ms)
 09/03 19:50:44,716 INFO [281471923237312] (sensor.cpp:40) Frame Callback Color #11 overdue. (FPS: 30, max duration: 32.258064 ms)
 09/03 19:50:44,795 INFO [281471923237312] (sensor.cpp:40) Frame Callback Color #14 overdue. (FPS: 30, max duration: 32.258064 ms)
 09/03 19:50:46,627 INFO [281471923237312] (sensor.cpp:40) Frame Callback Color #69 overdue. (FPS: 30, max duration: 32.258064 ms)
 09/03 19:50:46,715 INFO [281471923237312] (sensor.cpp:40) Frame Callback Color #72 overdue. (FPS: 30, max duration: 32.258064 ms)
 09/03 19:50:46,814 WARNING [281473142566928] (ds5-motion.cpp:573) IMU Calibration is not available, default intrinsic and extrinsic will be used.
 09/03 19:50:46,814 INFO [281473142566928] (ds5-motion.h:188) IMU extrinsic table not found; using CAD values
 09/03 19:50:46,814 INFO [281473142566928] (ds5-motion.h:219) Depth Module V2 IMU Accelno valid intrinsic available, use default values.
 09/03 19:50:46,814 INFO [281473142566928] (ds5-motion.h:231) Depth Module V2 IMU Gyrointrinsic not valid, use default values.
 09/03 19:50:46,814 INFO [281473142566928] (sensor.cpp:1586) Request: MOTION_XYZ32F Accel, 
Resolved to: MOTION_XYZ32F Accel, 
 09/03 19:50:46,817 INFO [281473142566928] (ds5-motion.h:219) Depth Module V2 IMU Accelno valid intrinsic available, use default values.
 09/03 19:50:46,818 INFO [281473142566928] (ds5-motion.h:231) Depth Module V2 IMU Gyrointrinsic not valid, use default values.
 09/03 19:50:46,818 INFO [281473142566928] (sensor.cpp:1586) Request: MOTION_XYZ32F Gyro, 
Resolved to: MOTION_XYZ32F Gyro, 
 09/03 19:50:47,140 ERROR [281471931630016] (uvc-streamer.cpp:106) uvc streamer watchdog triggered on endpoint: 132
 09/03 19:50:47,456 ERROR [281472107778496] (uvc-streamer.cpp:106) uvc streamer watchdog triggered on endpoint: 130
 09/03 19:50:49,502 INFO [281473142566928] (metadata-parser.h:334) Frame counter reset
 09/03 19:50:49,502 INFO [281472857940416] (metadata-parser.h:334) Frame counter reset
 09/03 19:50:49,558 INFO [281471923237312] (sensor.cpp:40) Frame Callback Color #4 overdue. (FPS: 30, max duration: 32.258064 ms)
 09/03 19:50:50,241 INFO [281471923237312] (sensor.cpp:40) Frame Callback Color #24 overdue. (FPS: 30, max duration: 32.258064 ms)
 09/03 19:50:50,562 INFO [281471923237312] (sensor.cpp:40) Frame Callback Color #34 overdue. (FPS: 30, max duration: 32.258064 ms)
 09/03 19:50:50,600 INFO [281471923237312] (sensor.cpp:40) Frame Callback Color #35 overdue. (FPS: 30, max duration: 32.258064 ms)
 09/03 19:50:50,770 INFO [281471923237312] (sensor.cpp:40) Frame Callback Color #40 overdue. (FPS: 30, max duration: 32.258064 ms)
 09/03 19:50:52,247 INFO [281471923237312] (sensor.cpp:40) Frame Callback Color #84 overdue. (FPS: 30, max duration: 32.258064 ms)
 09/03 19:50:53,275 INFO [281471923237312] (sensor.cpp:40) Frame Callback Color #115 overdue. (FPS: 30, max duration: 32.258064 ms)
 09/03 19:50:53,377 INFO [281471923237312] (sensor.cpp:40) Frame Callback Color #118 overdue. (FPS: 30, max duration: 32.258064 ms)
 09/03 19:50:54,540 INFO [281471923237312] (sensor.cpp:40) Frame Callback Color #153 overdue. (FPS: 30, max duration: 32.258064 ms)
 09/03 19:50:55,265 INFO [281471923237312] (sensor.cpp:40) Frame Callback Color #175 overdue. (FPS: 30, max duration: 32.258064 ms)
 09/03 19:50:56,338 INFO [281471923237312] (sensor.cpp:40) Frame Callback Color #207 overdue. (FPS: 30, max duration: 32.258064 ms)

@Karin-Sugi Karin-Sugi changed the title realsense viewer sometimes unable receive imu data realsense viewer unable receive imu data Mar 9, 2023
@Karin-Sugi Karin-Sugi changed the title realsense viewer unable receive imu data realsense viewer unable to receive imu data Mar 9, 2023
@MartyG-RealSense
Copy link
Collaborator

Hi @Karin-Sugi The problem is not related to the installation method. It is a known issue that enabling three stream types in the RealSense Viewer including the Motion Module (such as depth, RGB and Motion Module) can cause one of the streams to become blank and display the No Frames Received message for some RealSense users.

Changing the order that the streams are enabled in makes no difference, because when enabling three streams it is the enabling of the Motion Module that is the trigger for the problem. There is not a fix for this in the RealSense Viewer unfortunately except to only enable two streams (such as Depth and Motion Module, or RGB and Motion Module).

There are workarounds for the issue though when creating your own script. In C++ you can use callbacks, and in Python you can create two separate pipelines and place IMU on its own on one pipeline and depth + RGB on the other pipeline. The best example of this Python method is at #5628 (comment)

@Karin-Sugi
Copy link
Author

Thanks @MartyG-RealSense ! Now I understood it is unable to view three stream types at once. However, even when I stream only imu data, sometimes says "no data recieved!". How can I keep streaming? or is it because of ability of RaspberryPi?

There is error messages below. Thanks in advance!

ubuntu@ubuntu:~$ realsense-viewer
 10/03 12:06:39,897 INFO [281473789898768] (synthetic-stream-gl.cpp:80) Initializing rendering, GLSL=0
 10/03 12:06:39,897 INFO [281473789898768] (synthetic-stream-gl.cpp:89)  0 GPU objects initialized
 10/03 12:06:40,117 INFO [281473789898768] (context.cpp:382) Found 1 RealSense devices (mask 0xff)
 10/03 12:06:40,163 INFO [281473789898768] (rs.cpp:2701) Framebuffer size changed to 1344 x 896
 10/03 12:06:40,166 INFO [281473789898768] (rs.cpp:2701) Scale Factor is now 1
 10/03 12:06:41,311 INFO [281473789898768] (context.cpp:382) Found 1 RealSense devices (mask 0xfe)
 10/03 12:06:48,175 WARNING [281473789898768] (ds5-motion.cpp:573) IMU Calibration is not available, default intrinsic and extrinsic will be used.
 10/03 12:06:48,175 INFO [281473789898768] (ds5-motion.h:188) IMU extrinsic table not found; using CAD values
 10/03 12:06:48,175 INFO [281473789898768] (ds5-motion.h:219) Depth Module V2 IMU Accelno valid intrinsic available, use default values.
 10/03 12:06:48,175 INFO [281473789898768] (ds5-motion.h:231) Depth Module V2 IMU Gyrointrinsic not valid, use default values.
 10/03 12:06:48,175 INFO [281473789898768] (sensor.cpp:1586) Request: MOTION_XYZ32F Accel, 
Resolved to: MOTION_XYZ32F Accel, 
 10/03 12:06:48,177 INFO [281473789898768] (ds5-motion.h:219) Depth Module V2 IMU Accelno valid intrinsic available, use default values.
 10/03 12:06:48,177 INFO [281473789898768] (ds5-motion.h:231) Depth Module V2 IMU Gyrointrinsic not valid, use default values.
 10/03 12:06:48,177 INFO [281473789898768] (sensor.cpp:1586) Request: MOTION_XYZ32F Gyro, 
Resolved to: MOTION_XYZ32F Gyro, 
 10/03 12:06:54,803 INFO [281472669827520] (sensor.cpp:40) Frame Callback Gyro #840 overdue. (FPS: 200, max duration: 4.975124 ms)
 10/03 12:07:02,608 INFO [281472669827520] (sensor.cpp:40) Frame Callback Accel #1184 overdue. (FPS: 100, max duration: 9.900990 ms)
 10/03 12:07:05,166 INFO [281472669827520] (sensor.cpp:40) Frame Callback Gyro #2862 overdue. (FPS: 200, max duration: 4.975124 ms)
 10/03 12:07:08,470 ERROR [281472669827520] (synthetic-stream.cpp:48) Exception was thrown during callback: Out of frame resources!
 10/03 12:07:10,674 ERROR [281472669827520] (synthetic-stream.cpp:48) Exception was thrown during callback: Out of frame resources!
 10/03 12:07:10,675 ERROR [281472669827520] (synthetic-stream.cpp:48) Exception was thrown during callback: Out of frame resources!
 10/03 12:07:10,691 ERROR [281472669827520] (synthetic-stream.cpp:48) Exception was thrown during callback: Out of frame resources!
 10/03 12:07:10,693 ERROR [281472669827520] (synthetic-stream.cpp:48) Exception was thrown during callback: Out of frame resources!
 10/03 12:07:10,694 ERROR [281472669827520] (synthetic-stream.cpp:48) Exception was thrown during callback: Out of frame resources!
 10/03 12:07:31,438 INFO [281472669827520] (sensor.cpp:40) Frame Callback Gyro #7983 overdue. (FPS: 200, max duration: 4.975124 ms)
 10/03 12:07:33,364 INFO [281472669827520] (sensor.cpp:40) Frame Callback Accel #4196 overdue. (FPS: 100, max duration: 9.900990 ms)
 10/03 12:07:43,245 INFO [281472669827520] (sensor.cpp:40) Frame Callback Gyro #10317 overdue. (FPS: 200, max duration: 4.975124 ms)
 10/03 12:07:43,406 INFO [281472669827520] (sensor.cpp:40) Frame Callback Accel #5187 overdue. (FPS: 100, max duration: 9.900990 ms)

@MartyG-RealSense
Copy link
Collaborator

RealSense users with Raspberry Pi do not often report using the IMU with it as depth + color are more often used, but on the rare occasions when they have, there have been reports of problems that do not occur when the IMU is used with other computers / computing devices.

It has previously been theorized at #4979 (comment) that it is a kernel related problem, in which case building the SDK on Pi from source code in RSUSB backend mode with the -DFORCE_RSUSB_BACKEND=true build flag may help if you have not tried this already, as an RSUSB build of the SDK bypasses the kernel.

@Karin-Sugi
Copy link
Author

I see. I've already built my SDK from source code in RSUSB backend mode. It looks difficult to get imu data by RaspberryPi...
OK. I'll use another imu sensor on my robot instead. Thanks @MartyG-RealSense

@MartyG-RealSense
Copy link
Collaborator

You are very welcome!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants