Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix overlinking of MATLAB bindings to MatlabEngine #840

Merged
merged 6 commits into from
Mar 10, 2021

Conversation

traversaro
Copy link
Member

@traversaro traversaro commented Mar 9, 2021

This PR fixes the problem described in robotology/robotology-superbuild#653 for iDynTree, by locally implementing the CMake fix proposed in https://gitlab.kitware.com/cmake/cmake/-/issues/21912 .

Before (the library links libMatlabEngine.so and libMatlabDataArray.so):

ldd ./build/iDynTreeMEX.mexa64
linux-vdso.so.1 (0x00007fff9bf9a000)
	libMatlabEngine.so => /home/runner/work/idyntree/idyntree/msdk_R2020b_mexa64/extern/bin/glnxa64/libMatlabEngine.so (0x00007fde698a1000)
	libMatlabDataArray.so => /home/runner/work/idyntree/idyntree/msdk_R2020b_mexa64/bin/glnxa64/libMatlabDataArray.so (0x00007fde6987c000)
	libmex.so => /home/runner/work/idyntree/idyntree/msdk_R2020b_mexa64/bin/glnxa64/libmex.so (0x00007fde69763000)
	libmx.so => /home/runner/work/idyntree/idyntree/msdk_R2020b_mexa64/bin/glnxa64/libmx.so (0x00007fde6950d000)
	libidyntree-solid-shapes.so => /home/runner/work/idyntree/idyntree/build/lib/libidyntree-solid-shapes.so (0x00007fde694ff000)
	libidyntree-inverse-kinematics.so => /home/runner/work/idyntree/idyntree/build/lib/libidyntree-inverse-kinematics.so (0x00007fde694c0000)
	libidyntree-visualization.so => /home/runner/work/idyntree/idyntree/build/lib/libidyntree-visualization.so (0x00007fde69478000)
	libidyntree-high-level.so => /home/runner/work/idyntree/idyntree/build/lib/libidyntree-high-level.so (0x00007fde69455000)
	libidyntree-estimation.so => /home/runner/work/idyntree/idyntree/build/lib/libidyntree-estimation.so (0x00007fde693c7000)
	libidyntree-modelio-urdf.so => /home/runner/work/idyntree/idyntree/build/lib/libidyntree-modelio-urdf.so (0x00007fde6937b000)
	libidyntree-sensors.so => /home/runner/work/idyntree/idyntree/build/lib/libidyntree-sensors.so (0x00007fde69355000)
	libidyntree-model.so => /home/runner/work/idyntree/idyntree/build/lib/libidyntree-model.so (0x00007fde692de000)
	libidyntree-core.so => /home/runner/work/idyntree/idyntree/build/lib/libidyntree-core.so (0x00007fde69266000)
	libstdc++.so.6 => /usr/share/miniconda/envs/test/lib/libstdc++.so.6 (0x00007fde690f1000)
	libgcc_s.so.1 => /usr/share/miniconda/envs/test/lib/libgcc_s.so.1 (0x00007fde690dd000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fde68edc000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fde68ed4000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fde68eb1000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fde69b6e000)
	libmwfl.so => not found
	libut.so => not found
	libmwservices.so => not found
	libmwfoundation_matlabdata_matlab.so => not found
	libmwfoundation_matlabdata.so => not found
	libmwmvm.so => not found
	libmwm_dispatcher_interfaces.so => not found
	libmwm_dispatcher.so => not found
	libmwi18n.so => not found
	libmwfoundation_filesystem.so => not found
	libmwmlutil.so => not found
	libmwfoundation_usm.so => not found
	libmwgenerate_diag_message.so => not found
	libmwmcos.so => not found
	libmwcpp11compat.so => not found
	libmwboost_date_time.so.1.70.0 => not found
	libmwboost_log.so.1.70.0 => not found
	libmwboost_thread.so.1.70.0 => not found
	libmwi18n.so => not found
	libmwmfl_permute.so => not found
	libut.so => not found
	libmwfl.so => not found
	libmwfoundation_usm.so => not found
	libmwindexingapimethods.so => not found
	libmwcpp11compat.so => not found
	libmwboost_log.so.1.70.0 => not found
	libmwboost_thread.so.1.70.0 => not found
	libicuuc.so.64 => not found
	libtbbmalloc.so.2 => /lib/x86_64-linux-gnu/libtbbmalloc.so.2 (0x00007fde68e65000)
	libz.so.1 => /usr/share/miniconda/envs/test/lib/libz.so.1 (0x00007fde68e4b000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fde68cfa000)
	libassimp.so.5 => /usr/share/miniconda/envs/test/lib/libassimp.so.5 (0x00007fde6809e000)
	libipopt.so.3 => /usr/share/miniconda/envs/test/lib/libipopt.so.3 (0x00007fde67e3e000)
	libIrrlicht.so => /usr/share/miniconda/envs/test/lib/libIrrlicht.so (0x00007fde678f7000)
	libidyntree-modelio-xml.so => /home/runner/work/idyntree/idyntree/build/lib/libidyntree-modelio-xml.so (0x00007fde678e2000)
	libxml2.so.2 => /usr/share/miniconda/envs/test/lib/libxml2.so.2 (0x00007fde67778000)
	librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fde6776d000)
	liblapack.so.3 => /usr/share/miniconda/envs/test/lib/./liblapack.so.3 (0x00007fde657a8000)
	libdmumps_seq-5.2.1.so => /usr/share/miniconda/envs/test/lib/./libdmumps_seq-5.2.1.so (0x00007fde65582000)
	libmumps_common_seq-5.2.1.so => /usr/share/miniconda/envs/test/lib/./libmumps_common_seq-5.2.1.so (0x00007fde65521000)
	libpord_seq-5.2.1.so => /usr/share/miniconda/envs/test/lib/./libpord_seq-5.2.1.so (0x00007fde65508000)
	libmpiseq_seq-5.2.1.so => /usr/share/miniconda/envs/test/lib/./libmpiseq_seq-5.2.1.so (0x00007fde654fa000)
	libesmumps-6.so => /usr/share/miniconda/envs/test/lib/./libesmumps-6.so (0x00007fde654f3000)
	libscotch-6.so => /usr/share/miniconda/envs/test/lib/./libscotch-6.so (0x00007fde6544e000)
	libscotcherr-6.so => /usr/share/miniconda/envs/test/lib/./libscotcherr-6.so (0x00007fde65449000)
	libmetis.so => /usr/share/miniconda/envs/test/lib/./libmetis.so (0x00007fde653d4000)
	libgfortran.so.5 => /usr/share/miniconda/envs/test/lib/./libgfortran.so.5 (0x00007fde65234000)
	libpng16.so.16 => /usr/share/miniconda/envs/test/lib/./libpng16.so.16 (0x00007fde651f9000)
	libjpeg.so.8 => /usr/share/miniconda/envs/test/lib/./libjpeg.so.8 (0x00007fde6515f000)
	libbz2.so.1.0 => /usr/share/miniconda/envs/test/lib/./libbz2.so.1.0 (0x00007fde6514b000)
	libX11.so.6 => /usr/share/miniconda/envs/test/lib/./libX11.so.6 (0x00007fde65008000)
	libGL.so.1 => /lib/x86_64-linux-gnu/libGL.so.1 (0x00007fde64f80000)
	libXxf86vm.so.1 => /lib/x86_64-linux-gnu/libXxf86vm.so.1 (0x00007fde64f77000)
	libicuuc.so.68 => /usr/share/miniconda/envs/test/lib/./libicuuc.so.68 (0x00007fde64d8d000)
	liblzma.so.5 => /usr/share/miniconda/envs/test/lib/./liblzma.so.5 (0x00007fde64d64000)
	libiconv.so.2 => /usr/share/miniconda/envs/test/lib/./libiconv.so.2 (0x00007fde64c7e000)
	libgomp.so.1 => /usr/share/miniconda/envs/test/lib/././libgomp.so.1 (0x00007fde64c51000)
	libquadmath.so.0 => /usr/share/miniconda/envs/test/lib/./libquadmath.so.0 (0x00007fde64c15000)
	libxcb.so.1 => /usr/share/miniconda/envs/test/lib/././libxcb.so.1 (0x00007fde64be9000)
	libGLdispatch.so.0 => /lib/x86_64-linux-gnu/libGLdispatch.so.0 (0x00007fde64b31000)
	libGLX.so.0 => /lib/x86_64-linux-gnu/libGLX.so.0 (0x00007fde64afd000)
	libXext.so.6 => /usr/share/miniconda/envs/test/lib/./libXext.so.6 (0x00007fde64ae8000)
	libicudata.so.68 => /usr/share/miniconda/envs/test/lib/././libicudata.so.68 (0x00007fde62fa5000)
	libXau.so.6 => /usr/share/miniconda/envs/test/lib/./././libXau.so.6 (0x00007fde62fa0000)

After (for MATLAB only libmex and libmx are linked):

ldd ./build/iDynTreeMEX.mexa64
	linux-vdso.so.1 (0x00007ffc351b4000)
	libidyntree-solid-shapes.so => /home/runner/work/idyntree/idyntree/build/lib/libidyntree-solid-shapes.so (0x00007fca144f6000)
	libidyntree-inverse-kinematics.so => /home/runner/work/idyntree/idyntree/build/lib/libidyntree-inverse-kinematics.so (0x00007fca144b9000)
	libidyntree-visualization.so => /home/runner/work/idyntree/idyntree/build/lib/libidyntree-visualization.so (0x00007fca14471000)
	libmex.so => /home/runner/work/idyntree/idyntree/msdk_R2020b_mexa64/bin/glnxa64/libmex.so (0x00007fca14358000)
	libmx.so => /home/runner/work/idyntree/idyntree/msdk_R2020b_mexa64/bin/glnxa64/libmx.so (0x00007fca14102000)
	libidyntree-high-level.so => /home/runner/work/idyntree/idyntree/build/lib/libidyntree-high-level.so (0x00007fca140dd000)
	libidyntree-estimation.so => /home/runner/work/idyntree/idyntree/build/lib/libidyntree-estimation.so (0x00007fca1404f000)
	libidyntree-modelio-urdf.so => /home/runner/work/idyntree/idyntree/build/lib/libidyntree-modelio-urdf.so (0x00007fca14003000)
	libidyntree-sensors.so => /home/runner/work/idyntree/idyntree/build/lib/libidyntree-sensors.so (0x00007fca13fdd000)
	libidyntree-model.so => /home/runner/work/idyntree/idyntree/build/lib/libidyntree-model.so (0x00007fca13f68000)
	libidyntree-core.so => /home/runner/work/idyntree/idyntree/build/lib/libidyntree-core.so (0x00007fca13ef0000)
	libstdc++.so.6 => /usr/share/miniconda/envs/test/lib/libstdc++.so.6 (0x00007fca13d79000)
	libgcc_s.so.1 => /usr/share/miniconda/envs/test/lib/libgcc_s.so.1 (0x00007fca13d65000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fca13b64000)
	libassimp.so.5 => /usr/share/miniconda/envs/test/lib/libassimp.so.5 (0x00007fca12f08000)
	libipopt.so.3 => /usr/share/miniconda/envs/test/lib/libipopt.so.3 (0x00007fca12ca8000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fca12b57000)
	libIrrlicht.so => /usr/share/miniconda/envs/test/lib/libIrrlicht.so (0x00007fca12610000)
	libmwfl.so => not found
	libut.so => not found
	libmwservices.so => not found
	libmwfoundation_matlabdata_matlab.so => not found
	libmwfoundation_matlabdata.so => not found
	libmwmvm.so => not found
	libmwm_dispatcher_interfaces.so => not found
	libmwm_dispatcher.so => not found
	libmwi18n.so => not found
	libmwfoundation_filesystem.so => not found
	libmwmlutil.so => not found
	libmwfoundation_usm.so => not found
	libmwgenerate_diag_message.so => not found
	libmwmcos.so => not found
	libmwcpp11compat.so => not found
	libmwboost_date_time.so.1.70.0 => not found
	libmwboost_log.so.1.70.0 => not found
	libmwboost_thread.so.1.70.0 => not found
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fca125e7000)
	libmwi18n.so => not found
	libmwmfl_permute.so => not found
	libut.so => not found
	libmwfl.so => not found
	libmwfoundation_usm.so => not found
	libmwindexingapimethods.so => not found
	libmwcpp11compat.so => not found
	libmwboost_log.so.1.70.0 => not found
	libmwboost_thread.so.1.70.0 => not found
	libicuuc.so.64 => not found
	libtbbmalloc.so.2 => /lib/x86_64-linux-gnu/libtbbmalloc.so.2 (0x00007fca125a1000)
	libz.so.1 => /usr/share/miniconda/envs/test/lib/libz.so.1 (0x00007fca12587000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fca147c5000)
	libidyntree-modelio-xml.so => /home/runner/work/idyntree/idyntree/build/lib/libidyntree-modelio-xml.so (0x00007fca12570000)
	libxml2.so.2 => /usr/share/miniconda/envs/test/lib/libxml2.so.2 (0x00007fca12408000)
	librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fca123fd000)
	liblapack.so.3 => /usr/share/miniconda/envs/test/lib/./liblapack.so.3 (0x00007fca10438000)
	libdmumps_seq-5.2.1.so => /usr/share/miniconda/envs/test/lib/./libdmumps_seq-5.2.1.so (0x00007fca10212000)
	libmumps_common_seq-5.2.1.so => /usr/share/miniconda/envs/test/lib/./libmumps_common_seq-5.2.1.so (0x00007fca101af000)
	libpord_seq-5.2.1.so => /usr/share/miniconda/envs/test/lib/./libpord_seq-5.2.1.so (0x00007fca10196000)
	libmpiseq_seq-5.2.1.so => /usr/share/miniconda/envs/test/lib/./libmpiseq_seq-5.2.1.so (0x00007fca1018a000)
	libesmumps-6.so => /usr/share/miniconda/envs/test/lib/./libesmumps-6.so (0x00007fca10183000)
	libscotch-6.so => /usr/share/miniconda/envs/test/lib/./libscotch-6.so (0x00007fca100de000)
	libscotcherr-6.so => /usr/share/miniconda/envs/test/lib/./libscotcherr-6.so (0x00007fca100d9000)
	libmetis.so => /usr/share/miniconda/envs/test/lib/./libmetis.so (0x00007fca10062000)
	libgfortran.so.5 => /usr/share/miniconda/envs/test/lib/./libgfortran.so.5 (0x00007fca0fec2000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fca0febc000)
	libpng16.so.16 => /usr/share/miniconda/envs/test/lib/./libpng16.so.16 (0x00007fca0fe83000)
	libjpeg.so.8 => /usr/share/miniconda/envs/test/lib/./libjpeg.so.8 (0x00007fca0fde9000)
	libbz2.so.1.0 => /usr/share/miniconda/envs/test/lib/./libbz2.so.1.0 (0x00007fca0fdd3000)
	libX11.so.6 => /usr/share/miniconda/envs/test/lib/./libX11.so.6 (0x00007fca0fc90000)
	libGL.so.1 => /lib/x86_64-linux-gnu/libGL.so.1 (0x00007fca0fc08000)
	libXxf86vm.so.1 => /lib/x86_64-linux-gnu/libXxf86vm.so.1 (0x00007fca0fc01000)
	libicuuc.so.68 => /usr/share/miniconda/envs/test/lib/./libicuuc.so.68 (0x00007fca0fa17000)
	liblzma.so.5 => /usr/share/miniconda/envs/test/lib/./liblzma.so.5 (0x00007fca0f9ec000)
	libiconv.so.2 => /usr/share/miniconda/envs/test/lib/./libiconv.so.2 (0x00007fca0f906000)
	libgomp.so.1 => /usr/share/miniconda/envs/test/lib/././libgomp.so.1 (0x00007fca0f8d9000)
	libquadmath.so.0 => /usr/share/miniconda/envs/test/lib/./libquadmath.so.0 (0x00007fca0f89f000)
	libxcb.so.1 => /usr/share/miniconda/envs/test/lib/././libxcb.so.1 (0x00007fca0f871000)
	libGLdispatch.so.0 => /lib/x86_64-linux-gnu/libGLdispatch.so.0 (0x00007fca0f7b9000)
	libGLX.so.0 => /lib/x86_64-linux-gnu/libGLX.so.0 (0x00007fca0f785000)
	libXext.so.6 => /usr/share/miniconda/envs/test/lib/./libXext.so.6 (0x00007fca0f770000)
	libicudata.so.68 => /usr/share/miniconda/envs/test/lib/././libicudata.so.68 (0x00007fca0dc2f000)
	libXau.so.6 => /usr/share/miniconda/envs/test/lib/./././libXau.so.6 (0x00007fca0dc28000)

@traversaro traversaro changed the title [DO NOT MERGE] Test Fix overlinking of MATLAB bindings to MatlabEngine Mar 10, 2021
@traversaro traversaro requested a review from prashanthr05 March 10, 2021 11:34
@prashanthr05
Copy link
Contributor

@traversaro could you also add the command you are calling in the first comment? (I suppose it is ldd iDynTreeMEX.mexa64) Could you also highlight the differences?

@traversaro
Copy link
Member Author

Done.

Copy link
Contributor

@prashanthr05 prashanthr05 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not an expert, but overall the cmake logic relating to NO_IMPLICIT_LINK_TO_MATLAB_LIBRARIES LGTM.

If you think it'd be safe to have a Matlab bindings/cmake expert's review regarding this, please request another review.

Minor comments regarding documentation.

Comment on lines +1 to +2
# Vendored from https://gitlab.kitware.com/cmake/cmake/-/raw/v3.20.0-rc3/Modules/FindMatlab.cmake
# With local modification to implement https://gitlab.kitware.com/cmake/cmake/-/issues/21912
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to document this change in Changelog and also the README so that we remember that we are not using a vanilla FindMatlab.cmake. Also the Matlab versions that we support and this is affecting .

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be honest, I don't think so. As long as for the users the cmake -DIDYNTREE_USES_MATLAB .. works, I don't think we should document internal details such as which CMake scripts we vendor. Interested advanced user can just look in PR.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also the Matlab versions that we support and this is affecting .

This is something that we could do, but to be honest I don't know which MATLAB version we explicitly support (as we don't test them) so I don't know how can we do this in a proper way.

Copy link
Contributor

@prashanthr05 prashanthr05 Mar 10, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was actually thinking about the developers. iDynTree is a project that a few developers we know (including myself) use as a reference for software engineering. In case some other project faces something similar and come looking for answers here. That was the scenario I had in mind.

Added the ``ENGINE_LIBRARY``, ``DATAARRAY_LIBRARY`` and ``MCC_COMPILER``
components.

.. versionchanged:: 3.14
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what do these versionchanged tags imply?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is something that is part of the imported CMake file. I think it is used internally by CMake, but I don't have other details.

Comment on lines +1092 to +1102
if(NOT ${prefix}_NO_IMPLICIT_LINK_TO_MATLAB_LIBRARIES)
if(Matlab_HAS_CPP_API)
if(Matlab_ENGINE_LIBRARY)
target_link_libraries(${${prefix}_NAME} ${Matlab_ENGINE_LIBRARY})
endif()
if(Matlab_DATAARRAY_LIBRARY)
target_link_libraries(${${prefix}_NAME} ${Matlab_DATAARRAY_LIBRARY})
endif()
endif()

target_link_libraries(${${prefix}_NAME} ${Matlab_MEX_LIBRARY} ${Matlab_MX_LIBRARY})
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the logic affecting the first comment in the PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants