Map does not appear anymore in sample demo

[VRichardJP]

Checklist

• I've read the contribution guidelines.
• I've searched other issues and no duplicate issues were found.
• I'm convinced that this is not my fault but a bug.

Description

I am trying to run the planning simulator on the latest main branch. I can't get to visualize the PCD/lanelet maps on Rviz anymore (it worked just fine just a few weeks ago). I have tested several maps, including the sample map:

$ ros2 launch  autoware_launch planning_simulator.launch.xml map_path:=/home/sig/data/autoware_data_universe/sample-map-planning/ vehicle_model:=sample_vehicle sensor_model:=sample_sensor_kit

The 3d view screen just stays black, moving around does not make a difference:

According to the coordinates in the /map tf, the camera is looking at map location, so I should be able to see something.

I am using latest main from github :

$ git log --oneline -1 && vcs custom src --args log --oneline -1
d34355c8 (HEAD -> main, origin/main, origin/humble, origin/HEAD) chore(pre-commit workflows): add comments to recommend using pre-commit.ci (#3160)
.....................
=== src/core/autoware.core (git) ===
6bafedf ci(pre-commit): autoupdate (#62)
=== src/core/autoware_adapi_msgs (git) ===
9679b5a ci(pre-commit): autoupdate (#10)
=== src/core/autoware_common (git) ===
14a9975 ci(pre-commit): autoupdate (#138)
=== src/core/autoware_msgs (git) ===
90ed3f1 feat(autoware_planning_msgs): move header to the top in PoseWithUuidStamped (#50)
=== src/core/external/autoware_auto_msgs (git) ===
f664237 fix(autoware_auto_planning_msgs): added action_msgs dependency (#37)
=== src/launcher/autoware_launch (git) ===
5fa613b feat(mpc_lateral_controller): add steering bias removal (#190)
=== src/param/autoware_individual_params (git) ===
79cff0b ci(pre-commit): autoupdate (#35)
=== src/sensor_component/external/sensor_component_description (git) ===
475857d chore: sync files (#32)
=== src/sensor_component/external/tamagawa_imu_driver (git) ===
28ad3cd fix: remove init options (#7)
=== src/sensor_component/external/velodyne_vls (git) ===
baeafaf fix: sign error in return value of getPacket() (#38)
=== src/sensor_kit/external/awsim_sensor_kit_launch (git) ===
d9022ee Merge pull request #5 from RobotecAI/feat/shorter_timeout
=== src/sensor_kit/sample_sensor_kit_launch (git) ===
6275063 feat(sample_sensor_kit_launch): pass container to velodyne nodes (#48)
=== src/universe/autoware.universe (git) ===
5940ba77d feat(tier4_state_rviz_plugin): add planning API visualization (#2632)
=== src/universe/external/morai_msgs (git) ===
6fd6a71 Merge pull request #5 from Soohyeok-Park/main
=== src/universe/external/muSSP (git) ===
c79e98f Merge pull request #2 from tier4/feature/change_compile_option to avoid muSP
=== src/universe/external/ndt_omp (git) ===
d97eb41 feat(ndt_omp): add multigrid_ndt_omp for dynamic map loading  (#19)
=== src/universe/external/pointcloud_to_laserscan (git) ===
948a4fc fix: replace deprecated header for humble (#5)
=== src/universe/external/tier4_ad_api_adaptor (git) ===
e490986 ci: add ci to sync tier4/universe -> micro-bus  (#92)
=== src/universe/external/tier4_autoware_msgs (git) ===
2e43884 feat(tier4_rtc_msgs): add external request lane change module (#66)
=== src/vehicle/external/pacmod_interface (git) ===
c1a8eec ci(pre-commit): autoupdate (#44)
=== src/vehicle/sample_vehicle_launch (git) ===
157238c fix: suppress xacro warning (#47)

The logs do not seem to report any map related error:
log.txt

Any idea?

Expected behavior

Map appears in Rviz

Actual behavior

Black screen

Steps to reproduce

Run sample demo?

$ ros2 launch  autoware_launch planning_simulator.launch.xml map_path:=/home/sig/data/autoware_data_universe/sample-map-planning/ vehicle_model:=sample_vehicle sensor_kit:=sample_sensor_model

Versions

• Ubuntu 22.04
• ROS2 Humble
• main

Possible causes

No response

Additional context

No response

[VRichardJP]

The lanelet and pcd maps are received according to Rviz, but they just wont show:

It could be a bug from Rviz2 side. This is the version I have in my container:

$ sudo apt list --installed | grep rviz

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

ros-humble-rviz-assimp-vendor/jammy,now 11.2.5-1jammy.20230112.141217 amd64 [installed,automatic]
ros-humble-rviz-common/jammy,now 11.2.5-1jammy.20230120.222053 amd64 [installed,automatic]
ros-humble-rviz-default-plugins/jammy,now 11.2.5-1jammy.20230124.071257 amd64 [installed,automatic]
ros-humble-rviz-ogre-vendor/jammy,now 11.2.5-1jammy.20230112.141237 amd64 [installed,automatic]
ros-humble-rviz-rendering/jammy,now 11.2.5-1jammy.20230112.145716 amd64 [installed,automatic]
ros-humble-rviz2/jammy,now 11.2.5-1jammy.20230124.081000 amd64 [installed,automatic]

I will try to downgrade rviz2 and see if it fixes

[VRichardJP]

(Maybe) related issue: ros2/rviz#948

[VRichardJP]

OpenGL is broken in Ubuntu 22.04 containers because some mesa libs are too old. Upgrading the mesa libs in the container with the third-party repository ppa:kisak/kisak-mesa fixes the issue for me (from mesa version 22.0.1 to 22.3.5). See ros2/rviz#948 (comment)

Should we consider updating autoware humble container with that fix?

[kenji-miyake]

Should we consider updating autoware humble container with that fix?

If the fixed release won't be released or if you're in a hurry, I think it's allowed.

[VRichardJP]

@mitsudome-r do you want to propose a fix for this?

Out of curiousity, I have tested glxgears on ubuntu 22.10 and 23.04 containers. The OpenGL app crashes in both cases:

docker run --rm --network=host -e DISPLAY -e XAUTHORITY=/tmp/.Xauthority -v ~/.Xauthority:/tmp/.Xauthority -v /tmp/.X11-unix:/tmp/.X11-unix ubuntu:23.04 /bin/bash -c "apt update && apt install -y mesa-utils && glxgears"

On Ubuntu 23.04, the latest mesa version is 22.2.5. So the bug has been fixed somewhere between 22.2.5 and 22.3.5. I don't see many people complaining about it though (mostly people on Windows). So there may be something specific on my machine (like local mesa version, docker version, etc).

In any case, I have added these lines to my dev dockerfile and everything works fine now:

# for apt-add-repository
RUN apt update && apt install -y software-properties-common \
  && rm -rf /var/lib/apt/lists/*

# fix OpenGL issues with old mesa libs on Ubuntu 22.04 (e.g. black view in rviz2)
RUN apt-add-repository ppa:kisak/kisak-mesa && apt install -y \
  libegl-mesa0 libegl1-mesa-dev libgbm-dev libgbm1 libgl1-mesa-dev \
  libgl1-mesa-dri libglapi-mesa libglx-mesa0 \
  && rm -rf /var/lib/apt/lists/*

[mitsudome-r]

@VRichardJP Sorry for the late reply.
Could you create a PR on this? We can merge it once I check it on my machine.

[isouf]

@VRichardJP, @mitsudome-r - The same problem appears in the awsim-stable branch. I was able to to fix it by applying your recent PR. It would be good to update autoware/awsim-stable with the same fix.

[isouf]

@VRichardJP, @mitsudome-r - The same problem appears in the awsim-stable branch. I was able to to fix it by applying your recent PR. It would be good to update autoware/awsim-stable with the same fix.

[Gowtham-Raj-M]

@VRichardJP , I am also facing the same issue. Is the above fix is applied to autoware/awsim-stable. ?