Matlab version not found when compiling yarp-matlab-bindings #363

EWouters · 2020-03-10T10:43:59Z

Hi,
I'm trying to compile this awesome project using this Dockerfile.

I created a Dockerfile to install Matlab into the Tools image and use that image as a base for the Development image. Matlab runs fine in that image and I have a single-machine-licence.

When I try to compile the robotology-superbuild, it fails on compiling the yarp-matlab-bindings with the following error:

[165/224] Performing configure step for 'yarp-matlab-bindings'
FAILED: robotology/yarp-matlab-bindings/CMakeFiles/YCMStamp/yarp-matlab-bindings-configure 
cd /iit/sources/robotology-superbuild/build/robotology/yarp-matlab-bindings && /usr/bin/cmake --no-warn-unused-cli -DCMAKE_PREFIX_PATH:PATH=/iit/sources/robotology-superbuild/build/install -DYARP_USES_MATLAB:BOOL=ON -DYARP_USES_OCTAVE:BOOL=ON -C/iit/sources/robotology-superbuild/build/robotology/yarp-matlab-bindings/CMakeFiles/YCMTmp/yarp-matlab-bindings-cache-Release.cmake -GNinja /iit/sources/robotology-superbuild/robotology/yarp-matlab-bindings && /usr/bin/cmake -E touch /iit/sources/robotology-superbuild/build/robotology/yarp-matlab-bindings/CMakeFiles/YCMStamp/yarp-matlab-bindings-configure
Not searching for unused variables given on the command line.
loading initial cache file /iit/sources/robotology-superbuild/build/robotology/yarp-matlab-bindings/CMakeFiles/YCMTmp/yarp-matlab-bindings-cache-Release.cmake
-- The C compiler identification is GNU 7.4.0
-- The CXX compiler identification is GNU 7.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found YARP: /iit/sources/robotology-superbuild/build/install/lib/cmake/YARP (found version "3.3.2+25-20200305.6+git4d3c8b098")
CMake Deprecation Warning at /iit/sources/robotology-superbuild/build/install/lib/cmake/YARP/YARPConfig.cmake:258 (message):
  The YARP_INCLUDE_DIRS variable is deprecated
Call Stack (most recent call first):
  CMakeLists.txt:9999 (_yarp_deprecated_variable_warning)
  CMakeLists.txt:38 (get_target_property)


-- Found YCM: /iit/sources/robotology-superbuild/build/install/lib/cmake/YCM (found version "0.11.0.1-20191204+gitad8bb25")
CMake Deprecation Warning at /iit/sources/robotology-superbuild/build/install/lib/cmake/YARP/YARPConfig.cmake:258 (message):
  The YARP_INCLUDE_DIRS variable is deprecated
Call Stack (most recent call first):
  CMakeLists.txt:9999 (_yarp_deprecated_variable_warning)
  CMakeLists.txt:51 (include_directories)


CMake Error at cmake/FindMatlab.cmake:1033 (list):
  list GET given empty list
Call Stack (most recent call first):
  cmake/FindMatlab.cmake:1239 (_Matlab_get_version_from_root)
  matlab/CMakeLists.txt:96 (find_package)


-- Found Matlab: /usr/local/MATLAB/R2017b/extern/include (found version "NOTFOUND") found components: MX_LIBRARY MAIN_PROGRAM

It looks like there were a few related issues to this 3 weeks ago, but I couldn't solve this when reading them.

The text was updated successfully, but these errors were encountered:

traversaro · 2020-03-10T10:47:15Z

Hi @EWouters ,
CMake's FindMatlab works fine when matlab is available in the path. In your Docker, is matlab available in the path?

traversaro · 2020-03-10T10:49:10Z

In the meantime, it seems that diegoferigo/dockerfiles#8 mentioned in the Docker image that you posted is a related issue, and it points to https://github.com/diegoferigo/devenv for Matlab support, but I never tried it.

fyi @diegoferigo

traversaro · 2020-03-10T11:05:25Z

Hi @EWouters ,
CMake's FindMatlab works fine when matlab is available in the path. In your Docker, is matlab available in the path?

If for any reason the logic in FindMatlab fails, you can explicitly set the directory where Matlab is installed via the Matlab_ROOT_DIR CMake variable, see https://cmake.org/cmake/help/v3.16/module/FindMatlab.html . Unfortunately this variable needs to be set in the yarp-matlab-bindings build, so you would need to first make the superbuild create the build, then when it fails manually go in the build directory and set it via, but that is probably not feasible in Docker.

diegoferigo · 2020-03-10T11:11:04Z

@EWouters Honestly I never tried to install Matlab into a Docker image since distributing such image (under the assumption that you also included the licence) would not be permitted. I recommend the usage of volumes in this case. This approach requires that you install and activate Matlab in the host OS, and then mount the Matlab directory inside your container.

As @traversaro mentioned, you need also to add the <matlab_prefix>/bin/matlab directory to your PATH.

With the images stored in the repo development-iit, using devenv is recommended since those images derive from devenv:nvidia. With devenv, the operations reported above are done automatically executed. Refer to the DevenvConfProcessor._process_matlab and setup.sh for details.

EWouters · 2020-03-10T11:12:00Z

Hi @EWouters ,
CMake's FindMatlab works fine when matlab is available in the path. In your Docker, is matlab available in the path?

It seems Matlab is available in my path:

erik@erik-Virtual-Machine:~/git/development-iit$ make run-matlab 
docker run --rm --init --name matlab -it --runtime runc -e USERNAME=matlab diegoferigo/matlab bash
==> Configuring tools image
==> Configuring the parent image
==> Creating the runtime user
Adding user matlab to group runtimeusers
==> Setting the default root password
==> Setting the default user password
==> Configuring the support of extending PATH system-wide
==> Parent devenv image configured
==> Tools container ready

/ 
❯ which matlab
/usr/local/bin/matlab

/ 
❯ whoami
root

I link the binary to my path with these lines in my Dockerfile:

ARG MATLAB_RELEASE=R2017b

# Add a script to start MATLAB
COPY --from=matlab /opt/startscript/startmatlab.sh /opt/startscript/startmatlab.sh
RUN chmod 755 /opt/startscript/startmatlab.sh && \
    ln -s /usr/local/MATLAB/$MATLAB_RELEASE/bin/matlab /usr/local/bin/matlab

# Desktop icon
COPY matlab.png /usr/share/icons/matlab.png
COPY matlab.desktop /usr/share/applications/matlab.desktop

# Tell the container what version of MATLAB is being used
ENV MATLAB_RELEASE $MATLAB_RELEASE

# # Add a user other than root to run MATLAB
# RUN useradd -ms /bin/bash matlab
# # Add bless that user with sudo powers
# RUN echo "matlab ALL=(ALL) NOPASSWD: ALL" > /etc/sudoers.d/matlab \
#     && chmod 0440 /etc/sudoers.d/matlab

Also mex-wholebodymodel seems to be able to detect matlab just fine.

Edit: removed user creation

diegoferigo · 2020-03-10T11:23:16Z

Few fixes on your dockerfile. Since diegoferigo/tools is based on devenv:nvidia, a user is already created when you start the container. This avoids hardcoding in the image user information as you did (you cannot assume that the host's UID is 1000 if you want to distribute the image). Using your dockerfile, the user is created twice.

If you don't want to use devenv, can you set in the Dockerfile the environment variable Matlab_ROOT_DIR to the path where you installed Matlab? @traversaro suggested that you could pass to CMake an option with the same name, however if the same variable already exists in your environment, CMake automatically reads it and you don't need to pass it through command line.

traversaro · 2020-03-10T11:32:16Z

Also mex-wholebodymodel seems to be able to detect matlab just fine.

Slightly OT: mex-wholebodymodel is not part anymore of the superbuild, and in general has been deprecated since a long time. Do you need it for any particular reason?

traversaro · 2020-03-10T11:36:15Z

It seems Matlab is available in my path:

Actually I was not precise in my previous comment. FindMatlab.cmake finds matlab if it is in the path, and if it is able to be executed correctly. In fact, the problem (see the CMake error in your log) seems indeed in this function: https://github.com/robotology/yarp-matlab-bindings/blob/master/cmake/FindMatlab.cmake#L1031 .
Can you verify that
matlab -nosplash -nojvm-logfile "matlabVersionLog.cmaketmp" -nodesktop -nodisplay -r "version, exit (script from https://github.com/robotology/yarp-matlab-bindings/blob/master/cmake/FindMatlab.cmake#L643) is able to run correctly. I guess that mex-wholebodymodel is just using an old FindMatlab, and for this reason everything works.

EWouters · 2020-03-10T11:37:07Z

Slightly OT: mex-wholebodymodel is not part anymore of the superbuild, and in general has been deprecated since a long time. Do you need it for any particular reason?

No, I was just trying to debug the problem, searched the repos for FindMatlab.CMake and found that other repos had slightly other versions of the file.

EWouters · 2020-03-10T11:54:43Z

Actually I was not precise in my previous comment. FindMatlab.cmake finds matlab if it is in the path, and if it is able to be executed correctly. In fact, the problem (see the CMake error in your log) seems indeed in this function: https://github.com/robotology/yarp-matlab-bindings/blob/master/cmake/FindMatlab.cmake#L1031 .
Can you verify that
matlab -nosplash -nojvm-logfile "matlabVersionLog.cmaketmp" -nodesktop -nodisplay -r "version, exit" (script from https://github.com/robotology/yarp-matlab-bindings/blob/master/cmake/FindMatlab.cmake#L643) is able to run correctly. I guess that mex-wholebodymodel is just using an old FindMatlab, and for this reason everything works.

It seems that this is indeed the command that fails! If I run that command as root I get an Activation error, however if I run it as the matlab user I get:

❯ su matlab

/ 
❯ matlab -nosplash -nojvm-logfile "matlabVersionLog.cmaketmp" -nodesktop -nodisplay -r "version, exit"
Unrecognized MATLAB option "nojvm-logfile".

                                                         < M A T L A B (R) >
                                               Copyright 1984-2017 The MathWorks, Inc.
                                               R2017b (9.3.0.713579) 64-bit (glnxa64)
                                                         September 14, 2017

 
To get started, type one of these: helpwin, helpdesk, or demo.
For product information, visit www.mathworks.com.
 

ans =

    '9.3.0.713579 (R2017b)'

So I should either activate matlab as root or build robotology as the matlab user.

EWouters · 2020-03-10T15:09:10Z

Looks like when compiling using the matlab user and sudo the yarp-matlab-bindings do compile.

[165/224] Performing configure step for 'yarp-matlab-bindings'
Not searching for unused variables given on the command line.
loading initial cache file /iit/sources/robotology-superbuild/build/robotology/yarp-matlab-bindings/CMakeFiles/YCMTmp/yarp-matlab-bindings-cache-Release.cmake
-- The C compiler identification is GNU 7.4.0
-- The CXX compiler identification is GNU 7.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found YARP: /iit/sources/robotology-superbuild/build/install/lib/cmake/YARP (found version "3.3.2+28-20200310.3+git841852272")
CMake Deprecation Warning at /iit/sources/robotology-superbuild/build/install/lib/cmake/YARP/YARPConfig.cmake:258 (message):
  The YARP_INCLUDE_DIRS variable is deprecated
Call Stack (most recent call first):
  CMakeLists.txt:9999 (_yarp_deprecated_variable_warning)
  CMakeLists.txt:38 (get_target_property)


-- Found YCM: /iit/sources/robotology-superbuild/build/install/lib/cmake/YCM (found version "0.11.0.1-20191204+gitad8bb25")
CMake Deprecation Warning at /iit/sources/robotology-superbuild/build/install/lib/cmake/YARP/YARPConfig.cmake:258 (message):
  The YARP_INCLUDE_DIRS variable is deprecated
Call Stack (most recent call first):
  CMakeLists.txt:9999 (_yarp_deprecated_variable_warning)
  CMakeLists.txt:51 (include_directories)


-- Found Matlab: /usr/local/MATLAB/R2017b/extern/include (found version "NOTFOUND") found components: MX_LIBRARY MAIN_PROGRAM 
-- Found Octave: /usr/bin/octave (found version "4.2.2") 
CMake Deprecation Warning at /iit/sources/robotology-superbuild/build/install/share/YCM/modules/InstallBasicPackageFiles.cmake:313 (message):
  TARGETS is deprecated.  Use EXPORT instead
Call Stack (most recent call first):
  matlab/CMakeLists.txt:181 (install_basic_package_files)


CMake Warning (dev) at /iit/sources/robotology-superbuild/build/install/share/YCM/modules/InstallBasicPackageFiles.cmake:385 (message):
  Compatibility variables are no longer generated.  Use
  ENABLE_COMPATIBILITY_VARS to re-enable them (deprecated) or define them
  using either INCLUDE_FILE or INCLUDE_CONTENT (recommended).
Call Stack (most recent call first):
  matlab/CMakeLists.txt:181 (install_basic_package_files)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Configuring done
-- Generating done
-- Build files have been written to: /iit/sources/robotology-superbuild/build/robotology/yarp-matlab-bindings

I have to do some more testing to confirm it works, and that takes some time on my laptop.

traversaro · 2020-03-10T15:14:43Z

I think that to avoid this workaround (that is not easy) we should have some way to pass a given set of CMake arguments to all the CMake-based projects in the superbuild, something similar to colcon's --cmake-args .

traversaro · 2020-04-26T09:22:37Z

I think that to avoid this workaround (that is not easy) we should have some way to pass a given set of CMake arguments to all the CMake-based projects in the superbuild, something similar to colcon's --cmake-args .

This has been added to YCM as the YCM_EP_ADDITIONAL_CMAKE_ARGS cmake cache variable in robotology/ycm-cmake-modules#329 . Unfortunately it still needs to make it to a released version of YCM so it is still not available with the default value of Stable ROBOTOLOGY_PROJECT_TAGS , but if you are on Unstable ROBOTOLOGY_PROJECT_TAGS you can already specify the Matlab_ROOT_DIR for all the superbuild CMake subprojects via:

cd robotology-superbuild
mkdir build
cd build
cmake -DROBOTOLOGY_ENABLE_DYNAMICS:BOOL=ON  -DROBOTOLOGY_USES_MATLAB:BOOL=ON -DYCM_EP_ADDITIONAL_CMAKE_ARGS:STRING="-DMatlab_ROOT_DIR=<matlab_root_dir>" ..

Note that YCM_EP_ADDITIONAL_CMAKE_ARGS is a cache variable, so if you specify it once from the terminal you don't need to specify it again (unless you edit or removed it from the cache).

For more info on this feature, check YCM's docs: http://robotology.github.io/ycm/gh-pages/git-master/manual/ycm-superbuild.7.html#specifying-additional-cmake-arguments-for-all-subprojects .

traversaro · 2020-08-22T12:27:39Z

The definitive solution to this is blocked by robotology/ycm-cmake-modules#352 , and probably updating the docs.

traversaro · 2020-10-11T14:07:46Z

YCM 0.12 has been released and is now used by default in robotology-superbuild Stable and Unstable builds, and will also be included in the 2020.11 release. For this reason I think that the solution proposed in #363 (comment) should now work, if it does not, @EWouters feel free to reopen the issue, thanks!

traversaro mentioned this issue Sep 16, 2020

Add the possibility to override the module path in the autogenerated config robotology/ycm-cmake-modules#356

Closed

traversaro closed this as completed Oct 11, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Matlab version not found when compiling yarp-matlab-bindings #363

Matlab version not found when compiling yarp-matlab-bindings #363

EWouters commented Mar 10, 2020 •

edited

Loading

traversaro commented Mar 10, 2020

traversaro commented Mar 10, 2020

traversaro commented Mar 10, 2020

diegoferigo commented Mar 10, 2020

EWouters commented Mar 10, 2020 •

edited

Loading

diegoferigo commented Mar 10, 2020

traversaro commented Mar 10, 2020

traversaro commented Mar 10, 2020

EWouters commented Mar 10, 2020 •

edited

Loading

EWouters commented Mar 10, 2020 •

edited

Loading

EWouters commented Mar 10, 2020 •

edited

Loading

traversaro commented Mar 10, 2020

traversaro commented Apr 26, 2020

traversaro commented Aug 22, 2020

traversaro commented Oct 11, 2020

Matlab version not found when compiling yarp-matlab-bindings #363

Matlab version not found when compiling yarp-matlab-bindings #363

Comments

EWouters commented Mar 10, 2020 • edited Loading

traversaro commented Mar 10, 2020

traversaro commented Mar 10, 2020

traversaro commented Mar 10, 2020

diegoferigo commented Mar 10, 2020

EWouters commented Mar 10, 2020 • edited Loading

diegoferigo commented Mar 10, 2020

traversaro commented Mar 10, 2020

traversaro commented Mar 10, 2020

EWouters commented Mar 10, 2020 • edited Loading

EWouters commented Mar 10, 2020 • edited Loading

EWouters commented Mar 10, 2020 • edited Loading

traversaro commented Mar 10, 2020

traversaro commented Apr 26, 2020

traversaro commented Aug 22, 2020

traversaro commented Oct 11, 2020

EWouters commented Mar 10, 2020 •

edited

Loading

EWouters commented Mar 10, 2020 •

edited

Loading

EWouters commented Mar 10, 2020 •

edited

Loading

EWouters commented Mar 10, 2020 •

edited

Loading

EWouters commented Mar 10, 2020 •

edited

Loading