sonic-slave-stretch build failed

[tieguoevan]

Description

sonic-slave-stretch cannot be built because debhelper from stretch-backports is conflicted with cmake from stretch

Step 10/42 : RUN apt-get -t stretch-backports install -y debhelper
 ---> Running in b4298563bd0d
Reading package lists...
Building dependency tree...
Reading state information...
The following packages were automatically installed and are no longer required:
  cmake-data libarchive13 libuv1
Use 'apt autoremove' to remove them.
The following additional packages will be installed:
  dh-autoreconf dpatch dwz quilt
The following packages will be REMOVED:
  cmake
The following NEW packages will be installed:
  dwz
The following packages will be upgraded:
  debhelper dh-autoreconf dpatch quilt
4 upgraded, 1 newly installed, 1 to remove and 147 not upgraded.
Need to get 1515 kB of archives.
After this operation, 15.9 MB disk space will be freed.

Without cmake, subsequent compilation will fail

Step 31/42 : RUN cd /usr/src/gtest && cmake . && make -C /usr/src/gtest
 ---> Running in fc9ae8ff35ea
�[91m/bin/sh: 1: cmake: not found
�[0mThe command '/bin/sh -c cd /usr/src/gtest && cmake . && make -C /usr/src/gtest' returned a non-zero code: 127
Sending build context to Docker daemon  28.67kB

sonic-jenkins can also see related logs
https://sonic-jenkins.westus2.cloudapp.azure.com/job/bldenv/job/sonic-slave-stretch/110/consoleFull

Steps to reproduce the issue:

1. remove existed sonic-slave-stretch
2. make configure

[qiluo-msft]

I think related to this bug https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=954852
The upstream is broken.

[lguohan]

I suspect not. it seems to be cause by debhelper

lgh@8bb9636d98fc:/sonic$ apt show debhelper
Package: debhelper
Version: 12.1.1~bpo9+1
Priority: optional
Section: devel
Maintainer: Debhelper Maintainers <[email protected]>
Installed-Size: 1641 kB
Provides: debhelper-compat (= 9), debhelper-compat (= 10), debhelper-compat (= 11), debhelper-compat (= 12), dh-sequence-dwz, dh-sequence-installinitramfs, dh-sequence-systemd
Depends: autotools-dev, dh-autoreconf (>= 17~), dh-strip-nondeterminism (>= 0.028~), dpkg (>= 1.18.0~), dpkg-dev (>= 1.18.2~), dwz, file (>= 3.23), libdpkg-perl (>= 1.17.14), man-db, po-debconf, perl
Suggests: dh-make
Breaks: cmake (<< 3.9~), dh-systemd (<< 1.38), meson (<< 0.40.0~)
Replaces: dh-systemd (<< 1.38)
Download-Size: 1012 kB
APT-Manual-Installed: yes
APT-Sources: http://debian-archive.trafficmanager.net/debian stretch-backports/main amd64 Packages
Description: helper programs for debian/rules

N: There is 1 additional record. Please use the '-a' switch to see it

[lguohan]

this is a tricky one. here is my analysis:

• smartmontools requires debhelper >= 11, therefore, we need to install debhelper 12.1.1 from stretch-backports.

• debhelper in requires cmake (>= 3.9)

• previously, we install cmake v13.3.2, which is ok. recently, cmake in debhelper is upgraded to v13.6.3 which requires libarchive13 (>= 3.3.3).

• but libarchive13 is not available in stretch-backports.

Now, we cannot go back to cmake v13.3.2 since it is no longer available.

it looks like libarchive13 (3.4.0-2~bpo9+1) is in stretch-backports-sloppy

[gpaussabrcm]

With this change, please keep a close watch on building src/libyang. When I tried this, the newer version of cmake seems to behave differently regarding SWIG, causing the following error:

make[3]: Entering directory '/sonic/src/libyang/libyang-1.0.73/build'                                                                                 
[ 96%] Building CXX object swig/python2/CMakeFiles/_yang2.dir/yangPYTHON_wrap.cxx.o                                                                   
/sonic/src/libyang/libyang-1.0.73/build/swig/python2/yangPYTHON_wrap.cxx:3292:33: error: expected initializer before '.' token                        
 #  define SWIG_init    init_yang.so                                                                                                                  
                                 ^                                                                                                                    
/sonic/src/libyang/libyang-1.0.73/build/swig/python2/yangPYTHON_wrap.cxx:102537:1: note: in expansion of macro 'SWIG_init'                            
 SWIG_init(void) {                                                                                                                                    
 ^                                                                                                                                                    
/sonic/src/libyang/libyang-1.0.73/build/swig/python2/yangPYTHON_wrap.cxx:101971:24: warning: 'swig_const_table' defined but not used [-Wunused-variable]                                                                                                                                                    
 static swig_const_info swig_const_table[] = {                                                                                                        
                        ^~~~~~~~~~~~~~~~                                                                                                              
/sonic/src/libyang/libyang-1.0.73/build/swig/python2/yangPYTHON_wrap.cxx:99421:20: warning: 'SwigMethods' defined but not used [-Wunused-variable]    
 static PyMethodDef SwigMethods[] = {                                                                                                                 
                    ^~~~~~~~~~~                                                                                                                       
swig/python2/CMakeFiles/_yang2.dir/build.make:62: recipe for target 'swig/python2/CMakeFiles/_yang2.dir/yangPYTHON_wrap.cxx.o' failed
make[3]: *** [swig/python2/CMakeFiles/_yang2.dir/yangPYTHON_wrap.cxx.o] Error 1
make[3]: Leaving directory '/sonic/src/libyang/libyang-1.0.73/build'
CMakeFiles/Makefile2:3866: recipe for target 'swig/python2/CMakeFiles/_yang2.dir/all' failed

Not sure if there are any other build errors in SONiC, but this is the first one I encountered.

For reference, here are the versions installed in my sonic-slave-stretch build container after making the Dockerfile.j2 changes:

# dpkg -l cmake libarchive13 debhelper  
Desired=Unknown/Install/Remove/Purge/Hold                           
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)                    
||/ Name                            Version              Architecture         Description
+++-===============================-====================-====================-====================================================================
ii  cmake                           3.16.3-1~bpo9+1      amd64                cross-platform, open-source make system                             
ii  debhelper                       12.1.1~bpo9+1        all                  helper programs for debian/rules                                    
ii  libarchive13:amd64              3.4.0-2~bpo9+1       amd64                Multi-format archive and compression library (shared library)       

-Greg

[lguohan]

ack. two solutions in my mind

1. upgrade libyang to latest release 1.0.130 to see if there is issue.
2. downgrade debhelper, and only install debhelper 12 before smartmontools and then download debhelper after build smartmontools.

[gpaussabrcm]

Consider the following change as an additional patch in the src/libyang/patch series:

diff --git a/swig/CMakeLists.txt b/swig/CMakeLists.txt
index 0baa69ab..c0b75228 100644
--- a/swig/CMakeLists.txt
+++ b/swig/CMakeLists.txt
@@ -31,7 +31,9 @@ if(GEN_PYTHON_BINDINGS AND SWIG_FOUND)
             unset(PYTHON_INCLUDE_PATH CACHE)
             unset(PYTHON_EXT_SUFFIX CACHE)
             unset(PYTHON_MODULE_PATH CACHE)
-            set(PYTHON_EXT_SUFFIX ".so")
+            if(${CMAKE_VERSION} VERSION_LESS "3.16.0")
+                set(PYTHON_EXT_SUFFIX ".so")
+            endif()

         find_program(PYTHON_EXECUTABLE NAMES python${CUR_PYTHON_VERSION})
         execute_process(COMMAND ${PYTHON_EXECUTABLE} -c

The VERSION_LESS check most likely isn't exactly right, but this is meant as an alternative to upgrading the libyang version, and is in addition to the sonic-slave-stretch/Dockerfile.j2 changes already proposed as a fix. I've not tried this with anything other than my one build that was previously unable to create the stretch base image, so it'll need some vetting.

[lguohan]

@gpaussabrcm, thanks for the investigation. @qiluo-msft has found the pre-built binary for 3.13.2. we are fixing using this approach. It is probably better to keep cmake the same version.

i think this time lesson learned. I did not realized so many project relies on the cmake. Did not realize cmake is such a basic tool. it is probably not a good idea to cmake from stretch-backports in the first place.

We should probably be more careful on accepting the pr that upgrades debhelper in the first place. after all, only one project requires debhelper >= 11.