CPU lidar (no rendering)

[chapulina]

Ignition already supports GPU Lidars, which use ign-rendering and detect visuals in the scene.

Besides the GPU version of this sensor, Gazebo-classic also supported a CPU-based ray sensor, which used the physics engine to detect objects. Even though that sensor is less performant than the GPU version, over the years many users have relied on it to have a ray sensor on settings where using a GPU / rendering engine was not ideal, like on cloud machines. Another difference is that these sensors would see collision shapes instead of visuals.

Based on this past experience, we should also consider adding a render-less lidar sensor to Ignition.

[caioaamaral]

@chapulina Seems like the hard work is in the implementation of a MultiRayShape an then somehow instantiate it inside a CpuLidar sensor plugin.

What is the ignition equivalent of calling gazebo::physics::PhysicsEngine::CreateCollision(...)?

[chapulina]

One way that I can think of this sensor being implemented is:

• Adding a MultiRayShape to ign-physics similar to the existing shapes, i.e. CapsuleShape
• Adding the implementation for DART
• Create a component in ign-gazebo to hold Lidar data
• On ignition::gazebo::systems::Physics, populate that data similar to how we populate WorldLinearAcceleration.
• Create a CpuLidar sensor here
• Create a CpiuLidar Gazebo system similar to systems::Imu that instantiates sensors::CpuLidar and updates it with the component data

[azeey]

In the meantime, a workaround for those who need to simulate lidars in an environment where a GPU is not available is to use gpu_lidar but with software rendering. Running gz-sim as is with a gpu_lidar in such an environment would currently cause a crash with an error: [GUI] [Err] [Ogre2RenderEngine.cc:1098] Unable to create the rendering window: .... The workaround is to set the environment variable MESA_GL_VERSION_OVERRIDE=3.3 and LIBGL_ALWAYS_SOFTWARE=true (this may not be necessary).

This is also how we have setup some of our CI machines (eg. Github Actions). There, since there is also no X server, we start one using Xvfb:

Xvfb :1 -noreset  -screen 0 1280x1024x24 &
export DISPLAY=:1.0
export MESA_GL_VERSION_OVERRIDE=3.3

The caveat with this workaround is that, unlike the cpu_lidar, it would still use visuals instead of collisions. The performance will also be poorer than gpu_lidar.

[romulogcerqueira]

Hi all,

it seems that DART does support ray collision via Bullet backend, which is also supported by gz-physics (being configured through SDF <collision_dectector>).

This way, one high-level implementation proposal for the ray-based collisions would be:

• [gz-physics]
  • Create GetRaycasts (similar to GetContacts)
    • Create structs to store ray information (Ray, RaycastResult and RayHit)
    • Create GetRaycastsFromLastStepFeature simulation feature
    • Create GetRaycastsFromLastStep() to get the raycast results from last simulation step
  • Incorporate this feature into gz::physics::dartsim::SimulationFeatures
• [gz-sensors]
  • Create sensor RaySensor here
• [gz-sim]
  • Create component RaycastResult to hold raycast data
  • Create UpdateRaycastCollisions() into Physics.cc
    • Call GetRaycastsFromLastStep() feature to get the raycast results
    • Populate RaycastResult component data with output of this feature
    • Write component to ECM
  • Create system plugin Raycast that instanciates RaySensor and updates it with component data from ECM.

Any suggestions/recommendations?

[ggari-robotnik]

I'm trying to run 2D laser simulations without GPU inside a container using this launch:

ros2 launch ros_gz_sim_demos gpu_lidar_bridge.launch.py

But the 2D laser scans and 3D beams do not update. (I checked that simulation is not paused).

The workaround proposed by @azeey do not work for me. Any idea how make it work?

[azeey]

Do you see any errors when launching? Can you try running https://github.com/gazebosim/gz-sim/blob/b8bbf1d9dee47af68fd1c85fa8382afc47732ff2/examples/worlds/gpu_lidar_sensor.sdf in Gazebo and with -v4 to get debug output?

[ggari-robotnik]

Here you are the output, But v4 do not seem to work, I think I'm doing something wrong about that, but I cannot figure it out:

ros2 launch ros_gz_sim_demos gpu_lidar_bridge.launch.py  gz_args:=v4 
[INFO] [launch]: All log files can be found below /home/robot/.ros/log/2024-07-12-10-09-33-274163-rob-id-002-2648
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [ruby $(which ign) gazebo-1]: process started with pid [2651]
[INFO] [parameter_bridge-2]: process started with pid [2653]
[INFO] [rviz2-3]: process started with pid [2656]
[rviz2-3] QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-robot'
[parameter_bridge-2] [INFO] [1720771773.350842014] [ros_gz_bridge]: Creating GZ->ROS Bridge: [lidar (gz.msgs.LaserScan) -> lidar (sensor_msgs/msg/LaserScan)] (Lazy 0)
[parameter_bridge-2] [INFO] [1720771773.352148142] [ros_gz_bridge]: Creating ROS->GZ Bridge: [lidar (sensor_msgs/msg/LaserScan) -> lidar (gz.msgs.LaserScan)] (Lazy 0)
[parameter_bridge-2] [INFO] [1720771773.352544802] [ros_gz_bridge]: Creating GZ->ROS Bridge: [/lidar/points (gz.msgs.PointCloudPacked) -> /lidar/points (sensor_msgs/msg/PointCloud2)] (Lazy 0)
[parameter_bridge-2] [INFO] [1720771773.352772269] [ros_gz_bridge]: Creating ROS->GZ Bridge: [/lidar/points (sensor_msgs/msg/PointCloud2) -> /lidar/points (gz.msgs.PointCloudPacked)] (Lazy 0)
[ruby $(which ign) gazebo-1] QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-robot'

[azeey]

Oh, I meant directly using gz sim -v4 to isolate the issue to Gazebo.

Anyway, I was able to reproduce this issue. It's happening because the UserCommands plugin is missing. When you translated the box, it gave the appearance that it translated, but it didn't actually get translated. The GUI sends calls the translate service, but since UserCommands is missing, the service is not available. If you click off, you'd see the box would snap back to its original position.

I think we have two action items:

1. The translate tool should either be disabled if the UserCommands plugin is not available or request for the plugin to be added (The translate tool should be disabled if the UserCommands plugin is not available  gz-sim#2478)
2. The gpu_lidar_sensor.sdf example should include the UserCommands plugin. (Add UserCommands plugin to GPU lidar sensor example gz-sim#2479)

I'll go ahead and create issues for these. Any help to fix them would be greatly appreciated.

[VoidDruid]

Hi all! I am running gazebo in a VM with GPU passthrough, I got everything to work with --render-engine ogre (world rendering, camera sensor) except for LiDAR. The workaround with software rendering works, but significantly reduces simulation performance.

Since in my case GPU not being used for lidar is probably a virtualization problem and I don't have the expertise to fix it, an acceptable workaround would be to use GPU rendering and CPU lidar. But from looking at the issues and PRs mentioned above by @azeey, I understand that CPU lidar is not ready yet. Will it be included in future releases or will it come as an update? (I use gz harmonic)