VTK Dependency Audit

[svenevs]

External dependencies for the VTK builds need to be inspected. From #16502 (comment)

Relates: #17962

There are three external dependencies shared between drake and the VTK tarball build:

• eigen
• png
• zlib

There are also third party dependencies that the VTK build vendors that we can reduce:

• freetype

    # extract from: tools/workspace/vtk/iamge/vtk_cmake_configure_args.py

    # Dependencies other parts of drake depend on directly get shared.  For the
    # .tar.gz builds these come from apt-get / brew.  Files involved:
    # - macOS
    #     - setup/mac/binary_distribution/Brewfile
    #     - tools/wheel/image/packages-macos
    # - ubuntu
    #     - tools/workspace/vtk/image/prereqs (for the docker build)
    #     - setup/ubuntu/binary_distribution/packages-focal.txt
    #     - setup/ubuntu/binary_distribution/packages-jammy.txt
    #     - tools/wheel/image/packages-focal
    #
    # These packages are the only ones we should require -dev from in apt.
    shared_external_dependencies = [
        "eigen",
        "png",
        "zlib",
    ]

Eigen

Drake installs this via the package manager (apt / brew). However, there are additional considerations.

https://gitlab.kitware.com/vtk/vtk/-/blob/fcc6af2c07bffcd918e89917f9aad592424dc846/ThirdParty/eigen/vtk_eigen.h.in#L26-L30

The implications:

• Forces EIGEN_MPL2_ONLY (for the vtk build).

• Does not affect VTK headers, only source files.

  $ git grep VTK_EIGEN
  Filters/FlowPaths/vtkLinearTransformCellLocator.cxx:#include VTK_EIGEN(Geometry)
  Filters/FlowPaths/vtkParallelVectors.cxx:#include VTK_EIGEN(Eigenvalues)
  Filters/FlowPaths/vtkParallelVectors.cxx:#include VTK_EIGEN(Geometry)
  Filters/FlowPaths/vtkVectorFieldTopology.cxx:#include VTK_EIGEN(Eigenvalues)
  Filters/FlowPaths/vtkVortexCore.cxx:#include VTK_EIGEN(Eigenvalues)
  Filters/FlowPaths/vtkVortexCore.cxx:#include VTK_EIGEN(Geometry)
  Filters/Statistics/vtkPCAStatistics.cxx:#include VTK_EIGEN(Dense)
  ThirdParty/eigen/vtk_eigen.h.in:# define VTK_EIGEN(x) <Eigen/x>
  ThirdParty/eigen/vtk_eigen.h.in:# define VTK_EIGEN(x) <vtkeigen/eigen/x>

• If the user does "normal" drake,

  • The linux apt build will have used the same thing.
  • The macOS brew install might have been the same (depends on when we build vs user installed things).

• If th user supplies their own Eigen, all bets are off

• Switching to VTK vendored Eigen might be the right call?

  • It's in its own namespace: https://gitlab.kitware.com/vtk/vtk/-/blob/fcc6af2c07bffcd918e89917f9aad592424dc846/ThirdParty/eigen/vtkeigen/README.kitware.md
  • The wheel could stop doing its own thing (which is likely diverged from both linux and macOS at this point): https://github.com/RobotLocomotion/drake/blob/894ac55aa493ecca4bf43379b61447fcc99fbe0c/tools/wheel/image/dependencies/projects/eigen.cmake

png

Considerations for removing:

• @jwnimmer-tri mentioned we might be able to update this file to use vtkPNGReader instead:

drake/systems/sensors/lcm_image_array_to_images.cc

Line 5 in 894ac55

#include <png.h>

• Instead of using vtkPNGReader, we could switch to vtkpng library and use the mangled symbols in drake: https://gitlab.kitware.com/vtk/vtk/-/blob/fcc6af2c07bffcd918e89917f9aad592424dc846/ThirdParty/png/vtkpng/vtk_png_mangle.h
• The wheel build can stop doing its own thing: https://github.com/RobotLocomotion/drake/blob/894ac55aa493ecca4bf43379b61447fcc99fbe0c/tools/wheel/image/dependencies/projects/png.cmake

zlib

I don't think we should be doing anything about this one. There is, as far as I know, no way for the user to override the zlib choice. On Ubuntu it comes from apt-get, on macOS it comes from the system (xcode bundled and linked in statically).

If we did switch to the VTK one, it would be mangled, and therefore we could stop building it in the wheel: https://github.com/RobotLocomotion/drake/blob/894ac55aa493ecca4bf43379b61447fcc99fbe0c/tools/wheel/image/dependencies/projects/zlib.cmake

Freetype

While developing, we noticed that the vtkIOExport module forces a lot of dependencies on us that we do not want, e.g., vtkRenderingFreetype:

https://gitlab.kitware.com/vtk/vtk/-/blob/fcc6af2c07bffcd918e89917f9aad592424dc846/IO/Export/vtk.module#L15

We need that module for vtkGLTFExporter, and nothing else.

1. (@svenevs) For drake, revisit creating a patch to strip out everything else from vtkIOExport to only build the exporter we need. This should be simple to create and maintain, but the original investigation took place while sorting out all of the other dependencies and I made a mistake.
2. (@svenevs) The drake-specific hack will not be acceptable for VTK, in order to get something upstreamed and not do this we will want to.
   • Create a discourse post describing the desired change and why it is helpful.
   • Decide on how the modules will change (likely solution, e.g., is vtkGLTFExporter becomes its own module).
   • Decide on how deprecations for the module moves are going to work.
   • The deprecations are needed because we're changing the VTK build system interface.
   • Sticky: this may make the python bindings difficult during the transition process.
     • from vtkmodules.vtkA import vtkX now needs to be from vtkmodules.vtkB import vtkX when a class moves around
     • VTK would need some compat bridge to make vtkA still have vtkX (preferably with a deprecation warning)

Showing people how (1) works is the easiest way to get the discussion started, in our case it lets us get rid of the freetype dependency.

[EricCousineau-TRI]

Per platform checklist, assigning @jwnimmer-tri as lead for delegation etc.

[jwnimmer-tri]

Updates from f2f today:

Most likely the only remaining work is to deal with these three libraries:

• libpng
• libtiff
• libjpeg

See TODOs in @vtk_internal:

drake/tools/workspace/vtk_internal/settings.bzl

Lines 526 to 532 in d718147

	"deps_extra": [
	# TODO(jwnimmer-tri) VTK is the only user of this library.
	# We should write our own WORKSPACE rule to build it sensibly,
	# or switch to VTK's vendored version.
	"@libjpeg",
	],
	},

drake/tools/workspace/vtk_internal/settings.bzl

Lines 550 to 555 in d718147

	"deps_extra": [
	# TODO(jwnimmer-tri) VTK is the only user of this library.
	# We should write our own WORKSPACE rule to build it sensibly,
	# or switch to VTK's vendored version.
	"@libpng",
	],

drake/tools/workspace/vtk_internal/settings.bzl

Lines 561 to 566 in d718147

	"deps_extra": [
	# TODO(jwnimmer-tri) VTK is the only user of this library.
	# We should write our own WORKSPACE rule to build it sensibly,
	# or switch to VTK's vendored version.
	"@libtiff",
	],

The goal is to make sure all three of those are properly vendored, ideally with drake_vendor_foo prefix not just vtkfoo.

Remember that libtiff got angry last time we tried this. Figure out why.

[jwnimmer-tri]

@libpng was fixed in #20276
@libtiff was fixed in #20331.
@libjpeg was fixed in #20343.