Error rtabmap: malloc.c:2401 with Realsense T265

[novadr]

Hello,

I'm trying to run Rtabmap with the two fisheyes of an Intel Realsense T265.
I successfully installed rtabmap and rtabmap_ros on my ubuntu 18.04 with ROS Melodic and with the OpenCV 3.2 version (built in ROS) and CV bridge.
Furthermore, I installed Realsense sdk 2.42.0 and calibrate the T265's fisheyes successfully by executing the instruction mentioned here : http://official-rtab-map-forum.206.s1.nabble.com/Slam-using-Intel-RealSense-tracking-camera-T265-td6333.html#a6343

My .yaml files seems correct when I compare it with the values given by rs-enumerate-devices -c and I remove the "%YAML:1.0" as mentioned in the previous post.

I try to launch rtab with the following lines :

• roscore
• rosparam set /camera/tracking_module/enable_mapping false
• roslaunch realsense2_camera rs_t265.launch
• python camera_infpub.py _url:=/home/robot/Desktop/all_yaml_files/t265_with_rtabmap_left.yaml image:=/camera/fisheye1/image_raw camera_info:=/camera/fisheye1/camera_info_calib
• python camera_infpub.py _url:=/home/robot/Desktop/all_yaml_files/t265_with_rtabmap_right.yaml image:=/camera/fisheye2/image_raw camera_info:=/camera/fisheye2/camera_info_calib
• roslaunch rtabmap_ros rtabmap.launch \ args:="-d --Rtabmap/ImagesAlreadyRectified false" \ stereo:=true \ left_image_topic:=/camera/fisheye1/image_raw \ right_image_topic:=/camera/fisheye2/image_raw \ left_camera_info_topic:=/camera/fisheye1/camera_info_calib \ right_camera_info_topic:=/camera/fisheye2/camera_info_calib \ visual_odometry:=false \ odom_frame_id:=camera_odom_frame

Then I have the following issue :

Has somebody else encounter this error, please ? I don't have any idea about the origin of this issue and how to fix that..

Thank you.

[matlabbe]

Seems related to introlab/rtabmap#872, what are the dependencies of rtabmap?

rtabmap --version

You may give a try with rtabmap binary version:

sudo apt install ros-melodic-rtabmap-ros

[novadr]

The rtapmap dependencies are the following :

RTAB-Map:              0.20.23
PCL:                     1.8.1
With VTK:                6.3.0
OpenCV:                  3.2.0
With OpenCV xfeatures2d: false
With OpenCV nonfree:     false
With ORB OcTree:          true
With SuperPoint Torch:   false
With Python3:            false
With FastCV:             false
With OpenGV:             false
With Madgwick:            true
With PDAL:               false
With TORO:                true
With g2o:                 true
With GTSAM:               true
With Vertigo:             true
With CVSBA:              false
With Ceres:              false
With OpenNI2:             true
With Freenect:            true
With Freenect2:          false
With K4W2:               false
With K4A:                false
With DC1394:              true
With FlyCapture2:        false
With ZED:                false
With ZED Open Capture:   false
With RealSense:          false
With RealSense SLAM:     false
With RealSense2:          true
With MYNT EYE S:         false
With DepthAI:            false
With libpointmatcher:     true
With CCCoreLib:          false
With octomap:             true
With cpu-tsdf:           false
With open chisel:        false
With Alice Vision:       false
With LOAM:               false
With FLOAM:              false
With FOVIS:              false
With Viso2:              false
With DVO:                false
With ORB_SLAM:           false
With OKVIS:              false
With MSCKF_VIO:          false
With VINS-Fusion:        false
With OpenVINS:           false

I tried to install the binary version as you advised and now it gives an opencv error :

OpenCV Error: Assertion failed (fabs(nt) > 0) in cvStereoRectifyFisheye, file /tmp/binarydeb/ros-melodic-rtabmap-0.20.22/corelib/src/../include/rtabmap/core/stereo/stereoRectifyFisheye.h, line 1016
terminate called after throwing an instance of 'cv::Exception'
  what():  /tmp/binarydeb/ros-melodic-rtabmap-0.20.22/corelib/src/../include/rtabmap/core/stereo/stereoRectifyFisheye.h:1016: error: (-215) fabs(nt) > 0 in function cvStereoRectifyFisheye

OpenCV Error: Assertion failed (fabs(nt) > 0) in cvStereoRectifyFisheye, file /tmp/binarydeb/ros-melodic-rtabmap-0.20.22/corelib/src/../include/rtabmap/core/stereo/stereoRectifyFisheye.h, line 1016
terminate called after throwing an instance of 'cv::Exception'
  what():  /tmp/binarydeb/ros-melodic-rtabmap-0.20.22/corelib/src/../include/rtabmap/core/stereo/stereoRectifyFisheye.h:1016: error: (-215) fabs(nt) > 0 in function cvStereoRectifyFisheye

[rtabmap/rtabmap-2] process has died [pid 6357, exit code -6, cmd /home/robot/rtab_ws/devel/lib/rtabmap_ros/rtabmap -d --Rtabmap/ImagesAlreadyRectified false rgb/image:=/camera/rgb/image_rect_color depth/image:=/camera/depth_registered/image_raw rgb/camera_info:=/camera/rgb/camera_info rgbd_image:=rgbd_image_relay left/image_rect:=/camera/fisheye1/image_raw right/image_rect:=/camera/fisheye2/image_raw left/camera_info:=/camera/fisheye1/camera_info_calib right/camera_info:=/camera/fisheye2/camera_info_calib scan:=/scan scan_cloud:=/scan_cloud scan_descriptor:=/scan_descriptor user_data:=/user_data user_data_async:=/user_data_async gps/fix:=/gps/fix tag_detections:=/tag_detections fiducial_transforms:=/fiducial_transforms odom:=odom imu:=/imu/data __name:=rtabmap __log:=/home/robot/.ros/log/fab39eaa-a2d5-11ed-a4f3-a44cc87c946f/rtabmap-rtabmap-2.log].
log file: /home/robot/.ros/log/fab39eaa-a2d5-11ed-a4f3-a44cc87c946f/rtabmap-rtabmap-2*.log
[rtabmap/rtabmapviz-3] process has died [pid 6358, exit code -6, cmd /home/robot/rtab_ws/devel/lib/rtabmap_ros/rtabmapviz -d ~/.ros/rtabmap_gui.ini rgb/image:=/camera/rgb/image_rect_color depth/image:=/camera/depth_registered/image_raw rgb/camera_info:=/camera/rgb/camera_info rgbd_image:=rgbd_image_relay left/image_rect:=/camera/fisheye1/image_raw right/image_rect:=/camera/fisheye2/image_raw left/camera_info:=/camera/fisheye1/camera_info_calib right/camera_info:=/camera/fisheye2/camera_info_calib scan:=/scan scan_cloud:=/scan_cloud scan_descriptor:=/scan_descriptor odom:=odom __name:=rtabmapviz __log:=/home/robot/.ros/log/fab39eaa-a2d5-11ed-a4f3-a44cc87c946f/rtabmap-rtabmapviz-3.log].
log file: /home/robot/.ros/log/fab39eaa-a2d5-11ed-a4f3-a44cc87c946f/rtabmap-rtabmapviz-3*.log

The error does not appear when I launch the realsense T265 camera but when I launch the camera_info with these command :

python camera_infpub.py _url:=/home/robot/Desktop/all_yaml_files/t265_with_rtabmap_left.yaml image:=/camera/fisheye1/image_raw camera_info:=/camera/fisheye1/camera_info_calib
python camera_infpub.py _url:=/home/robot/Desktop/all_yaml_files/t265_with_rtabmap_right.yaml image:=/camera/fisheye2/image_raw camera_info:=/camera/fisheye2/camera_info_calib

[cy-2022]

Mine seems similar...
And this was with one of the TUM datasets.

[cy-2022]

Hi Mathieu @matlabbe, thanks a lot for helping close issue 882 which has been solved by resolving the boost-related conflicts. Here is some follow-up of the aforementioned problem in this issue. Basically I've tried all I could think of and also what you've suggested in a few previous issues where similar problems have been reported:

1. To first resolve OpenCV-related conflicts (there were indeed two versions of OpenCV on my PC, version 3.2 installed with ROS melodic, and version 3.3.1 required by the Indemind SDK).
   (left - before; right - after)
   

2. Try switching off g2o and ORB-SLAM while building (I did have ORB-SLAM3 on my PC and used to work on it a lot, but I checked what was printed out after cmake, ORB-SLAM3 was not found)

3. Try updating ros binaries with sudo apt upgrade before building rtabmap_ros again

But the problem remains (this time with ZED2).

So it has to have something to do with ROS, right, as it works totally fine with rtabmap alone. I'm attaching here a few screen-shots and logs. Please let me know if you notice anything. In the meantime, I'll try gdb and see what it brings me. Thank you very much!

-- MOBILE_BUILD=OFF
-- The imported target "vtkRenderingPythonTkWidgets" references the file
"/usr/lib/x86_64-linux-gnu/libvtkRenderingPythonTkWidgets.so"
but this file does not exist. Possible reasons include:

• The file was deleted, renamed, or moved to another location.
• An install or uninstall procedure did not complete successfully.
• The installation package was faulty and contained
  "/usr/lib/cmake/vtk-6.3/VTKTargets.cmake"
  but not all the files it references.

-- The imported target "vtk" references the file
"/usr/bin/vtk"
but this file does not exist. Possible reasons include:

• The file was deleted, renamed, or moved to another location.
• An install or uninstall procedure did not complete successfully.
• The installation package was faulty and contained
  "/usr/lib/cmake/vtk-6.3/VTKTargets.cmake"
  but not all the files it references.

-- PCL definitions don't contain "-march=native", make sure all libraries using Eigen are also compiled without that flag to avoid some segmentation faults (with gdb referring to some Eigen functions).
-- Found Sqlite3: /usr/include /usr/lib/x86_64-linux-gnu/libsqlite3.so
-- Found OpenMP: /usr/lib/gcc/x86_64-linux-gnu/7/libgomp.so;/usr/lib/x86_64-linux-gnu/libpthread.so
-- Add PCL_OMP to definitions
-- Found OpenCV: /usr/include;/usr/include/opencv
-- Found PCL: /usr/include/pcl-1.8;/usr/local/include/eigen3;/usr/include;/usr/include/ni;/usr/include/openni2;/usr/include/vtk-6.3;/usr/include/freetype2;/usr/lib/x86_64-linux-gnu/openmpi/include/openmpi;/usr/lib/x86_64-linux-gnu/openmpi/include/openmpi/opal/mca/event/libevent2022/libevent;/usr/lib/x86_64-linux-gnu/openmpi/include/openmpi/opal/mca/event/libevent2022/libevent/include;/usr/lib/x86_64-linux-gnu/openmpi/include;/usr/include/python2.7;/usr/include/x86_64-linux-gnu;/usr/include/hdf5/openmpi;/usr/include/libxml2;/usr/include/jsoncpp;/usr/include/tcl
-- Found ZLIB: /usr/include
-- Could NOT find Qt5Svg (missing: Qt5Svg_DIR)
-- VTK_RENDERING_BACKEND=OpenGL
-- Found Freenect: /usr/include
-- Found OpenNI2: /usr/include/openni2
-- Found DC1394: /usr/include/dc1394
-- FlyCapture2_INCLUDE_DIR=FlyCapture2_INCLUDE_DIR-NOTFOUND
-- FlyCapture2_LIBRARY=FlyCapture2_LIBRARY-NOTFOUND
-- Triclops_INCLUDE_DIR=Triclops_INCLUDE_DIR-NOTFOUND
-- Triclops_LIBRARY=Triclops_LIBRARY-NOTFOUND
-- FlyCaptureBridge_LIBRARY=FlyCaptureBridge_LIBRARY-NOTFOUND
-- Boost version: 1.65.1
-- Found the following Boost libraries:
-- thread
-- filesystem
-- system
-- program_options
-- date_time
-- chrono
-- atomic
-- Boost version: 1.65.1
-- Found the following Boost libraries:
-- thread
-- filesystem
-- system
-- program_options
-- date_time
-- chrono
-- atomic
-- Found libpointmatcher: /opt/ros/melodic/include;/usr/include/eigen3
-- Boost version: 1.65.1
-- Found the following Boost libraries:
-- thread
-- filesystem
-- system
-- program_options
-- date_time
-- chrono
-- atomic
-- Boost version: 1.65.1
-- Found the following Boost libraries:
-- thread
-- filesystem
-- system
-- program_options
-- date_time
-- chrono
-- timer
-- serialization
-- atomic
-- A library with BLAS API found.
-- Found CUDA: /usr/local/cuda-10.2 (found suitable version "10.2", minimum required is "10.2")
-- Found ZED sdk: /usr/local/zed/include
-- Found CUDA: /usr/local/cuda-10.2 (found version "10.2")
-- Found CUDA: /usr/local/cuda-10.2/include
-- Found RealSense2:
-- Found octomap 1.9.8: /opt/ros/melodic/include
-- Architecture: x86_64
-- Performing Test COMPILER_SUPPORTS_CXX11
-- Performing Test COMPILER_SUPPORTS_CXX11 - Success
-- Performing Test COMPILER_SUPPORTS_CXX0X
-- Performing Test COMPILER_SUPPORTS_CXX0X - Success
-- Found Pthreads

---

-- Info :
-- RTAB-Map Version = 0.20.23
-- CMAKE_VERSION = 3.10.2
-- CMAKE_INSTALL_PREFIX = /home/cy/rtabmap_ws/devel
-- CMAKE_BUILD_TYPE = Release
-- CMAKE_INSTALL_LIBDIR = lib
-- BUILD_APP = ON
-- BUILD_TOOLS = ON
-- BUILD_EXAMPLES = ON
-- BUILD_SHARED_LIBS = ON
-- CMAKE_CXX_FLAGS = -fmessage-length=0 -fopenmp -std=c++11
-- FLANN_KDTREE_MEM_OPT = OFF
-- PCL_DEFINITIONS = ;-DDISABLE_ENSENSO;-DDISABLE_DAVIDSDK;-DDISABLE_DSSDK;-DDISABLE_PCAP;-DDISABLE_PNG;-DDISABLE_LIBUSB_1_0;-DFLANN_STATIC;-Dqh_QHpointer;-DDISABLE_RSSDK
-- PCL_VERSION = 1.8.1

-- Optional dependencies ('*' affects some default parameters) :
-- *With OpenCV 3.2.0 xfeatures2d = NO, nonfree = NO (License: BSD)
-- With Qt 5.9.5 = YES (License: Open Source or Commercial)
-- With VTK 6.3 = YES (License: BSD)
-- With external SQLite3 = YES (License: Public Domain)
-- With ORB OcTree = YES (License: GPLv3)
-- With SupertPoint = NO (WITH_TORCH=OFF)
-- With Python3 = NO (WITH_PYTHON=OFF)
-- With Madgwick = YES (License: GPL)
-- With FastCV = NO (FastCV not found)
-- With PDAL = NO (PDAL not found)

-- Solvers:
-- With TORO = YES (License: Creative Commons [Attribution-NonCommercial-ShareAlike])
-- *With g2o = NO (WITH_G2O=OFF)
-- *With GTSAM = NO (GTSAM not found)
-- *With Ceres = NO (WITH_CERES=OFF)
-- With VERTIGO = NO (GTSAM or g2o required)
-- With cvsba = NO (WITH_CVSBA=OFF)
-- *With libpointmatcher = YES (License: BSD)
-- With CCCoreLib = NO (WITH_CCCORELIB=OFF)
-- With Open3D = NO (WITH_OPEN3D=OFF)
-- With OpenGV = NO (WITH_OPENGV=OFF)

-- Reconstruction Approaches:
-- With OCTOMAP = YES (License: BSD)
-- With CPUTSDF = NO (WITH_CPUTSDF=OFF)
-- With OpenChisel = NO (WITH_OPENCHISEL=OFF)
-- With AliceVision = NO (WITH_ALICE_VISION=OFF)

-- Camera Drivers:
-- With Freenect = YES (License: Apache v2 and/or GPLv2)
-- With OpenNI2 = YES (License: Apache v2)
-- With Freenect2 = NO (libfreenect2 not found)
-- With Kinect for Windows 2 = NO (Kinect for Windows 2 SDK not found)
-- With Kinect for Azure = NO (Kinect for Azure SDK not found)
-- With dc1394 = YES (License: LGPL)
-- With FlyCapture2/Triclops = NO (Point Grey SDK not found)
-- With ZED = YES
-- With ZEDOC = NO (ZED Open Capture not found)
-- With RealSense = NO (librealsense not found)
-- With RealSense2 = YES (License: Apache-2)
-- With MyntEyeS = NO (mynteye s sdk not found)
-- With DepthAI = NO (WITH_DEPTHAI=OFF)

-- Odometry Approaches:
-- With loam_velodyne = NO (WITH_LOAM=OFF)
-- With floam = NO (WITH_FLOAM=OFF)
-- With libfovis = NO (WITH_FOVIS=OFF)
-- With libviso2 = NO (WITH_VISO2=OFF)
-- With dvo_core = NO (WITH_DVO=OFF)
-- With okvis = NO (WITH_OKVIS=OFF)
-- With msckf_vio = NO (WITH_MSCKF_VIO=OFF)
-- With VINS-Fusion = NO (WITH_VINS=OFF)
-- With OpenVINS = NO (WITH_OPENVINS=OFF)
-- With ORB_SLAM = NO (WITH_ORB_SLAM=OFF)
-- Show all options with: cmake -LA | grep WITH_

---

-- Configuring done
-- Generating done
-- Build files have been written to: /home/cy/rtabmap/build

[saplesss]

have you solved this problem? ? I get the same...
works fine with rtabmap alone , but the ros crash...

[cy-2022]

have you solved this problem? ? I get the same... works fine with rtabmap alone , but the ros crash...

Yes and no :P. We actually tried with multiple devices. On my laptop, the problem is still there. On another device, where the settings and stuff are similar, launching the stereo odometry still led to errors mentioned above. BUT the RGBD odometry works just fine. For that we were using an Orbbec Astra mini s camera.

We are still trying to figure out what is going on. Once I find time, I'll try debugging.

[cy-2022]

Hi Mathieu @matlabbe, one further question: as I have ROS melodic on all my devices, should I clone from the 0.20.23-melodic tag of rtabmap?

[matlabbe]

@cy-2022 From the cmake screenshot from #881 (comment), there is a mixed up of two OpenCV versions in loaded libraries. This could cause malloc opencv related issues. Note that if you have another opencv installed, rtabmap standalone may build with it (running fine), but rtabmap_ros would likely fail (unless it is the same OpencV version than the system binaries) because rtabmap_ros depends on cv_bridge, which imports system opencv libraries. Thus rtabmap_ros has to load local opencv and system opencv libraries at the same time, causing crashes. This is also explained here, step 1.

@novadr From this post: #881 (comment), it seems that rtabmap_ros binaries are loading the locally built rtabmap libraries. To try ros binaries, uninstall the local rtabmap one (make uninstall from rtabmap/build directory, or manually remove librtabmap* from /usr/local/lib)

[cy-2022]

Hi Mathieu @matlabbe, thank you very much for your reply and the detailed explanation. I tried resolving the OpenCV conflict, in the end even by removing OpenCV 3.3.1 (installed for the Indemind camera SDK) completely. But the error is still there. Here I'm attaching the logs and stuff saved while building. Would you please help take a look when you have time? Thanks a lot!
rtabmap_cmake.txt
rtabmap_version.txt
rtabmap_ros_catkin_make.txt
tmp_lld_rtabmap.txt

And in the meantime, I've tried installing the rtabmap binary version as you suggested #881 (comment). It worked well, and on three of our devices on hand. So now we're finally able to try the framework out. Thank you very much for the advice.

With the only device of ours on which building rtabmap from source has worked well, we wanted to try using ORB-SLAM2 (already there on the device) as the visual odometry. Hopefully it'd work out. And I also wonder, with the binary version, how one can achieve this, so is ORB-SLAM2 supported, right? How about ORB-SLAM3? Can we simply modify the launch file (I saw related comments from you in a certain issue) to use these specific visual odometries? Thanks a lot!

[novadr]

Hi @matlabbe ! I just tried to install rtabmap binaries in a clean environment (Ubuntu 18.04 with ROS Melodic and Realsense, I just run the command : sudo apt install ros-melodic-rtabmap-ros to install it)

I got the same error as last time I tried rtabmap binaries : The Opencv assertion failed :

/rtabmap/rtabmap subscribed to (exact sync):
   /camera/fisheye1/image_raw \
   /camera/fisheye2/image_raw \
   /camera/fisheye1/camera_info_calib \
   /camera/fisheye2/camera_info_calib
[ INFO] [1676035837.610390324]: rtabmap 0.20.22 started...
OpenCV Error: Assertion failed (fabs(nt) > 0) in cvStereoRectifyFisheye, file /tmp/binarydeb/ros-melodic-rtabmap-0.20.22/corelib/src/../include/rtabmap/core/stereo/stereoRectifyFisheye.h, line 1016
terminate called after throwing an instance of 'cv::Exception'
  what():  /tmp/binarydeb/ros-melodic-rtabmap-0.20.22/corelib/src/../include/rtabmap/core/stereo/stereoRectifyFisheye.h:1016: error: (-215) fabs(nt) > 0 in function cvStereoRectifyFisheye

libpng warning: iCCP: known incorrect sRGB profile
libpng warning: iCCP: known incorrect sRGB profile
libpng warning: iCCP: known incorrect sRGB profile
[ INFO] [1676035837.992390464]: rtabmapviz: Reading parameters from the ROS server...

For the new Ubuntu, I got some problems with the installation of librealsense 2.42, so I installed the last version from source (v2.53.1) and I fixed the problem of the synchronization of the fisheyes according to your indication here : introlab/rtabmap#716 (comment)

[cy-2022]

Hi Mathieu @matlabbe, I've tried debugging with gbd on one of our devices where the error occurred. The results are as follows, with stereo_mapping (with a ZED2 camera) and rgbd_mapping (with an Orbbec Astra mini s), respectively. These reminded me of one previous issue I saw while checking if similar problems had been reported before #664 (comment)
There you've replied that it crashed in libimage_transport_plugins.so and suggested that one should try sudo apt upgrade to make sure all ros binaries are at the latest version. I'll try it. Thanks a lot. And a further favor to ask, if you have time, could you please explain how you could tell where the crashing happened from these gdb results? Thank you very much!

[cy-2022]

Hi Mathieu @matlabbe, I just did sudo apt upgrade and deleted ~/rtabmap_ws/build and catkin_make again. The problem was still there, and gbd gave the same results. Any idea how I should proceed? Would re-install ROS help? What do you think? Thank you very much!

[novadr]

I solved the malloc problem. It came from the librealsense version. I installed SDK 2.42.0 and now I don't have the malloc error.
I do however have an error due to the T265 fisheyes rectification. I close this issue and I will open a new one.

[matlabbe]

Confirmed I saw same error:
rgbd_odometry: malloc.c:2379: sysmalloc: Assertion '(old_top == initial_top (av) && old_size == 0) || ((unsigned long) (old_size) >= MINSIZE && prev_inuse (old_top) && ((unsigned long) old_end & (pagesize - 1)) == 0)' failed.

when building against realsense2 (2.53.1). The workaround is to build rtabmap library without realsense (if you use only ros):

cd rtabmap/build
cmake -DWITH_REALSENSE2=OFF ..
make