Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OsqpEigen, YCM and robot-testing-framework: Switch to use conda-forge binary package when generating conda packages #807

Merged
merged 9 commits into from
Jun 28, 2021

Conversation

traversaro
Copy link
Member

@traversaro traversaro commented Jun 25, 2021

As osqp-eigen and YCM are now available in conda-forge (see conda-forge/staged-recipes#15309 and https://github.com/conda-forge/osqp-eigen-feedstock) we can now avoid building our own osqp-eigen, ycm-cmake-modules and robot-testing-framework packages in the robotology channel, and just rely on the one provided by conda-forge.

This is done by defining in Build<project>.cmake the <project>_CONDA_PKG_NAME and <project>_CONDA_PKG_CONDA_FORGE_OVERRIDE variables, as described in https://github.com/robotology/robotology-superbuild/blob/master/doc/developers-faqs.md#how-to-ensure-that-binary-packages-are-correctly-generated-for-a-new-package . We also removed the <project>_CONDA_DEPENDENCIES as that was used just in the conda binary generation phase.

Part of #752 .

@traversaro
Copy link
Member Author

To avoid regressions, it is a good idea to run generate-conda-packages CI job on this branch. However, we need to first merge #800 otherwise the CI job will not even work.

@traversaro traversaro force-pushed the traversaro-patch-2 branch from 89954d6 to 7f5efba Compare June 25, 2021 11:21
@traversaro
Copy link
Member Author

To avoid regressions, it is a good idea to run generate-conda-packages CI job on this branch. However, we need to first merge #800 otherwise the CI job will not even work.

I did that in https://github.com/robotology/robotology-superbuild/actions/runs/971226693, and indeed it revelated another point that needed to be modified (it is a point where osqp-eigen was explicitly referred to workaround a bug in conda build), done in c684917

@traversaro
Copy link
Member Author

I created a new job in https://github.com/robotology/robotology-superbuild/actions/runs/971334957, but it turns out that there was a regression due to #805 . Now event-driven is able to find OpenCV, and in Linux in the part of event driven that depends on OpenCV there is this failure:

2021-06-25T14:34:29.1037459Z [20/26] Linking CXX executable bin/vFramerLite
2021-06-25T14:34:33.2693751Z FAILED: bin/vFramerLite 
2021-06-25T14:34:33.2711947Z : && $BUILD_PREFIX/bin/x86_64-conda-linux-gnu-c++ -fvisibility-inlines-hidden -std=c++17 -fmessage-length=0 -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem $PREFIX/include -fdebug-prefix-map=$SRC_DIR=/usr/local/src/conda/event-driven-1.5 -fdebug-prefix-map=$PREFIX=/usr/local/src/conda-prefix -O3 -DNDEBUG -Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags -Wl,--gc-sections -Wl,-rpath,$PREFIX/lib -Wl,-rpath-link,$PREFIX/lib -L$PREFIX/lib src/processing/vFramer/CMakeFiles/vFramerLite.dir/src/vFramerLite.cpp.o src/processing/vFramer/CMakeFiles/vFramerLite.dir/src/other_drawers.cpp.o -o bin/vFramerLite  -Wl,-rpath,$SRC_DIR/build/lib:  $PREFIX/lib/libYARP_init.so.3.4.5  lib/libevent-driven.so.1.3  $PREFIX/lib/libYARP_sig.so.3.4.5  $PREFIX/lib/libYARP_os.so.3.4.5  $PREFIX/lib/libopencv_gapi.so.4.5.1  $PREFIX/lib/libopencv_stitching.so.4.5.1  $PREFIX/lib/libopencv_alphamat.so.4.5.1  $PREFIX/lib/libopencv_aruco.so.4.5.1  $PREFIX/lib/libopencv_bgsegm.so.4.5.1  $PREFIX/lib/libopencv_bioinspired.so.4.5.1  $PREFIX/lib/libopencv_ccalib.so.4.5.1  $PREFIX/lib/libopencv_cvv.so.4.5.1  $PREFIX/lib/libopencv_dnn_objdetect.so.4.5.1  $PREFIX/lib/libopencv_dnn_superres.so.4.5.1  $PREFIX/lib/libopencv_dpm.so.4.5.1  $PREFIX/lib/libopencv_face.so.4.5.1  $PREFIX/lib/libopencv_freetype.so.4.5.1  $PREFIX/lib/libopencv_fuzzy.so.4.5.1  $PREFIX/lib/libopencv_hdf.so.4.5.1  $PREFIX/lib/libopencv_hfs.so.4.5.1  $PREFIX/lib/libopencv_img_hash.so.4.5.1  $PREFIX/lib/libopencv_intensity_transform.so.4.5.1  $PREFIX/lib/libopencv_line_descriptor.so.4.5.1  $PREFIX/lib/libopencv_mcc.so.4.5.1  $PREFIX/lib/libopencv_quality.so.4.5.1  $PREFIX/lib/libopencv_rapid.so.4.5.1  $PREFIX/lib/libopencv_reg.so.4.5.1  $PREFIX/lib/libopencv_rgbd.so.4.5.1  $PREFIX/lib/libopencv_saliency.so.4.5.1  $PREFIX/lib/libopencv_stereo.so.4.5.1  $PREFIX/lib/libopencv_structured_light.so.4.5.1  $PREFIX/lib/libopencv_phase_unwrapping.so.4.5.1  $PREFIX/lib/libopencv_superres.so.4.5.1  $PREFIX/lib/libopencv_optflow.so.4.5.1  $PREFIX/lib/libopencv_surface_matching.so.4.5.1  $PREFIX/lib/libopencv_tracking.so.4.5.1  $PREFIX/lib/libopencv_highgui.so.4.5.1  $PREFIX/lib/libopencv_datasets.so.4.5.1  $PREFIX/lib/libopencv_plot.so.4.5.1  $PREFIX/lib/libopencv_text.so.4.5.1  $PREFIX/lib/libopencv_videostab.so.4.5.1  $PREFIX/lib/libopencv_videoio.so.4.5.1  $PREFIX/lib/libopencv_xfeatures2d.so.4.5.1  $PREFIX/lib/libopencv_ml.so.4.5.1  $PREFIX/lib/libopencv_shape.so.4.5.1  $PREFIX/lib/libopencv_ximgproc.so.4.5.1  $PREFIX/lib/libopencv_video.so.4.5.1  $PREFIX/lib/libopencv_dnn.so.4.5.1  $PREFIX/lib/libopencv_xobjdetect.so.4.5.1  $PREFIX/lib/libopencv_imgcodecs.so.4.5.1  $PREFIX/lib/libopencv_objdetect.so.4.5.1  $PREFIX/lib/libopencv_calib3d.so.4.5.1  $PREFIX/lib/libopencv_features2d.so.4.5.1  $PREFIX/lib/libopencv_flann.so.4.5.1  $PREFIX/lib/libopencv_xphoto.so.4.5.1  $PREFIX/lib/libopencv_photo.so.4.5.1  $PREFIX/lib/libopencv_imgproc.so.4.5.1  $PREFIX/lib/libopencv_core.so.4.5.1 && :
2021-06-25T14:34:33.2727803Z $BUILD_PREFIX/bin/../lib/gcc/x86_64-conda-linux-gnu/9.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: warning: libGL.so.1, needed by $PREFIX/lib/libQt5Widgets.so.5, not found (try using -rpath or -rpath-link)
2021-06-25T14:34:33.2729886Z $BUILD_PREFIX/bin/../lib/gcc/x86_64-conda-linux-gnu/9.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: $PREFIX/lib/libQt5Gui.so.5: undefined reference to `glLoadIdentity'
2021-06-25T14:34:33.2731793Z $BUILD_PREFIX/bin/../lib/gcc/x86_64-conda-linux-gnu/9.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: $PREFIX/lib/libQt5Gui.so.5: undefined reference to `glMatrixMode'
2021-06-25T14:34:33.2733636Z $BUILD_PREFIX/bin/../lib/gcc/x86_64-conda-linux-gnu/9.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: $PREFIX/lib/libQt5Gui.so.5: undefined reference to `glOrtho'
2021-06-25T14:34:33.2735584Z $BUILD_PREFIX/bin/../lib/gcc/x86_64-conda-linux-gnu/9.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: $PREFIX/lib/libQt5Gui.so.5: undefined reference to `glLoadMatrixf'
2021-06-25T14:34:33.2736727Z collect2: error: ld returned 1 exit status

@traversaro traversaro changed the title OsqpEigen: Switch to use conda-forge binary package when generating conda packages OsqpEigen and YCM: Switch to use conda-forge binary package when generating conda packages Jun 26, 2021
@traversaro
Copy link
Member Author

As YCM was merged as well in conda-forge (see conda-forge/staged-recipes#15404), I modified the PR to also get ycm-cmake-modules from conda-forge.

@traversaro
Copy link
Member Author

New generate-conda-packages test run: https://github.com/robotology/robotology-superbuild/actions/runs/973915021 .

@traversaro traversaro changed the title OsqpEigen and YCM: Switch to use conda-forge binary package when generating conda packages OsqpEigen, YCM and robot-testing-framework: Switch to use conda-forge binary package when generating conda packages Jun 26, 2021
@traversaro
Copy link
Member Author

As YCM was merged as well in conda-forge (see conda-forge/staged-recipes#15404), I modified the PR to also get ycm-cmake-modules from conda-forge.

Same for robot-testing-framework .

@traversaro
Copy link
Member Author

iDynTree now fails:

2021-06-26T15:21:24.3661330Z Turn off this advice by setting config variable advice.detachedHead to false
2021-06-26T15:21:24.3662052Z 
2021-06-26T15:21:24.3662623Z HEAD is now at 8bf1795df Release v3.2.1 (#875)
2021-06-26T15:21:27.9257331Z CMake Error at /usr/share/miniconda/envs/test/conda-bld/idyntree_1624720813385/_build_env/share/cmake-3.20/Modules/CMakeFindDependencyMacro.cmake:47 (find_package):
2021-06-26T15:21:27.9259243Z   By not providing "FindYCM.cmake" in CMAKE_MODULE_PATH this project has
2021-06-26T15:21:27.9260321Z   asked CMake to find a package configuration file provided by "YCM", but
2021-06-26T15:21:27.9261155Z   CMake did not find one.
2021-06-26T15:21:27.9261681Z 
2021-06-26T15:21:27.9262729Z   Could not find a package configuration file provided by "YCM" (requested
2021-06-26T15:21:27.9263746Z   version 0.11.0) with any of the following names:
2021-06-26T15:21:27.9264334Z 
2021-06-26T15:21:27.9264945Z     YCMConfig.cmake
2021-06-26T15:21:27.9265865Z     ycm-config.cmake
2021-06-26T15:21:27.9266400Z 
2021-06-26T15:21:27.9267207Z   Add the installation prefix of "YCM" to CMAKE_PREFIX_PATH or set "YCM_DIR"
2021-06-26T15:21:27.9268294Z   to a directory containing one of the above files.  If "YCM" provides a
2021-06-26T15:21:27.9269379Z   separate development package or SDK, be sure it has been installed.

This happens as YCM is a build time dependency for any package depending on YARP or ICUB, but it is does not have any run_exports instruction as it is not a runtime dependency in strict stance, but rather a transitive build-time dependency. As a workaround, we can just add it as a runtime depenendency explicitly.

@traversaro
Copy link
Member Author

This happens as YCM is a build time dependency for any package depending on YARP or ICUB, but it is does not have any run_exports instruction as it is not a runtime dependency in strict stance, but rather a transitive build-time dependency. As a workaround, we can just add it as a runtime depenendency explicitly.

This was fixed with the latest two commits. With this commits, the PR is finally able to generate conda packages with no errors: https://github.com/robotology/robotology-superbuild/runs/2931185261 .

@traversaro traversaro merged commit 30b0f27 into master Jun 28, 2021
@traversaro traversaro deleted the traversaro-patch-2 branch June 28, 2021 15:05
Nicogene pushed a commit to Nicogene/robotology-superbuild that referenced this pull request Feb 14, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants