Point GZ_CONFIG_PATH to source install when not building from fetched sources

[srmainwaring]

Summary

This issue relates to building the gz-*-vendor packages when the underlying libraries have been built from source in a separate workspace. In this case the GZ_CONFIG_PATH points to the libraries in the vendor package workspace rather than the packages in the source workspace. As a result the CLI tools that are expected with the command gz are not visible.

See also: thread on Discord starting here: https://discord.com/channels/1077825543698927656/1077836768176517203/1239512156681011270

Details

System: macOS Sonoma 14.4.1, Xcode 15.3, Command Line Tools 15.3
Machine: MacBook Pro M1

How to replicate

1. workspace gz_source_ws contains a build of the gz-* libraries from source.
2. workspace gz_vendor_ws contains a build of the gz-*-vendor libraries.
3. build workspace gz_source_ws
4. change to workspace gz_vendor_ws, source gz_source_ws (i.e. . ~gz_source_ws/install/setup.zsh)
5. build workspace gz_vendor_ws
6. run the gz command and inspect the available tools

Expected behaviour

$ gz
The 'gz' command provides a command line interface to the Gazebo Tools.

  gz <command> [options]

List of available commands:

  help:          Print this help text.
  fuel:          Manage simulation resources.
  gui:           Launch graphical interfaces.
  log:           Record or playback topics.
  model:         Print information about models.
  msg:           Print information about messages.
  param:         List, get or set parameters.
  plugin:        Print information about plugins.
  sdf:           Utilities for SDF files.
  service:       Print information about services.
  sim:           Run and manage the Gazebo Simulator.
  topic:         Print information about topics.

Options:

  --force-version <VERSION>  Use a specific library version.
  --versions                 Show the available versions.
  --commands                 Show the available commands.
Use 'gz help <command>' to print help for a command.

Actual behaviour

$ gz
The 'gz' command provides a command line interface to the Gazebo Tools.

  gz <command> [options]

List of available commands:

  help:          Print this help text.
  sdf:           Utilities for SDF files.

Options:

  --force-version <VERSION>  Use a specific library version.
  --versions                 Show the available versions.
  --commands                 Show the available commands.
Use 'gz help <command>' to print help for a command.

[azeey]

I believe this is an issue with gz-tools captured in gazebosim/gz-tools#61. My assumption is that you've built gz_source_ws with --merge-install which puts gz-tools and all of the other libraries in the same merged install space. The gz tool is then able to find the necessary .yaml files since by default it looks for those files relative to where the gz file is installed (https://github.com/gazebosim/gz-tools/blob/2b228e5b956f1e966053dd860374670573580b41/src/gz.in#L50-L51). The other assumption is that one of the vendor packages was not able to find the required version of the vendored library, so it went ahead and fetched the source and built it. I'm guessing it's sdformat since that's the only command available. I say this because if a vendor packages finds the underlying library via find_package, it will not set the GZ_CONFIG_PATH environment variable (

gz_sim_vendor/CMakeLists.txt

Lines 90 to 96 in f6d32ab

	if(NOT ${${LIB_NAME_FULL}_FOUND})
	ament_environment_hooks("${CMAKE_CURRENT_SOURCE_DIR}/${PROJECT_NAME}.dsv.in")
	# Create a dummy .sh file needed for ament_package to source the .dsv file.
	# See https://github.com/ament/ament_package/issues/145
	file(WRITE ${CMAKE_CURRENT_BINARY_DIR}/${PROJECT_NAME}.sh "# Dummy .sh file needed for .dsv file to be sourced.")
	ament_environment_hooks("${CMAKE_CURRENT_BINARY_DIR}/${PROJECT_NAME}.sh")
	endif()

).

With GZ_CONFIG_PATH set, gz will only look for .yaml files in the paths defined by that environment variable and not look for the onces in gz_source_ws, which is what gazebosim/gz-tools#61 is about. @srmainwaring can you confirm my assumptions? Does gz work if you set GZ_CONFIG_PATH=~/gz_source_ws/install/share/gz?

The quickest workaround is to ensure that all the vendor packages actually find the packages from the gz_source_ws. By default, the vendor packages look for an exact match, but you can set GZ_BUILD_FROM_SOURCE=true to tell it to find any version with the same major version.

[srmainwaring]

My assumption is that you've built gz_source_ws with --merge-install which puts gz-tools and all of the other libraries in the same merged install space

Correct.

I'm guessing it's sdformat since that's the only command available.

Also correct.

Does gz work if you set GZ_CONFIG_PATH=~/gz_source_ws/install/share/gz?

Yes this works.

but you can set GZ_BUILD_FROM_SOURCE=true to tell it to find any version with the same major version.

exported GZ_BUILD_FROM_SOURCE=true then rebuilt all vendor packages and sourced the workspace. This time GZ_CONFIG_PATH is not set, and the CLI tools work.

Thanks! We should get this documented (the GZ_BUILD_FROM_SOURCE=true setting), as this is good to know.

[azeey]

Thanks! We should get this documented (the GZ_BUILD_FROM_SOURCE=true setting), as this is good to know.

This is now documented in https://gazebosim.org/docs/all/ros2_gz_vendor_pkgs