Support GLIBC 2.17 in the build binaries #1708

guolinke · 2018-09-28T02:33:12Z

It seems Azure Pipeline uses a higher version glibc to build the binaries.
Therefore, some old os cannot be supportted.

guolinke · 2018-09-28T06:29:50Z

@StrikerRUS
can we use the same glibc version for both Azure Pipeline and Travis ?
I think our binaries should build for the max compatibility.

StrikerRUS · 2018-09-28T19:48:52Z

@guolinke Sorry, I'm lost a bit. Is the problem in glibc or in gcc? Linux only or macOS too?

Can the problem be solved by installing appropriate (downgrading) glibc version on Azure (I believe that it's possible)?

skysun99 · 2018-09-28T20:13:28Z

How this possible I cloudy account unlock id

guolinke · 2018-09-29T03:26:33Z

@StrikerRUS i think downgrading glibc is okay.

guolinke · 2018-09-29T03:54:42Z

ping @davidstaheli @brcrista .
Can we use ubuntu 14.04 for the Microsoft hosted agents?
I try to use docker images, but all of them failed.

refer to https://lightgbm-ci.visualstudio.com/lightgbm-ci/_build/results?buildId=264&view=logs

guolinke · 2018-09-30T04:51:30Z

some updates:

I manage to decouple with glibc 2.23, by adding this line in the source code: https://github.com/Microsoft/LightGBM/blob/vsts-version/include/LightGBM/meta.h#L65-L67

However, there are some other higher version dependencies when build with vsts-linux-agent.

./lib_lightgbm.so: /usr/lib/x86_64-linux-gnu/libgomp.so.1: version `GOMP_4.0' not found (required by ./lib_lightgbm.so)
./lib_lightgbm.so: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by ./lib_lightgbm.so)
./lib_lightgbm.so: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by ./lib_lightgbm.so)

Since there are some new APIs in these new dependencies, I cannot fix them by the above solution for glibc.

For example, for the omp dependency, the 4.0 version only uses GOMP_parallel, while 1.0 version uses GOMP_parallel_start and GOMP_parallel_end.

guoke@test-vm3:~$ objdump -T lib_lightgbm.so | fgrep GOMP
0000000000000000      DF *UND*  0000000000000000  GOMP_1.0    GOMP_loop_guided_start
0000000000000000      DF *UND*  0000000000000000  GOMP_1.0    GOMP_loop_end_nowait
0000000000000000      DF *UND*  0000000000000000  GOMP_1.0    GOMP_atomic_start
0000000000000000      DF *UND*  0000000000000000  GOMP_1.0    GOMP_atomic_end
0000000000000000      DF *UND*  0000000000000000  GOMP_1.0    GOMP_loop_guided_next
0000000000000000      DF *UND*  0000000000000000  GOMP_4.0    GOMP_parallel

guoke@test-vm3:~/LightGBM$ objdump -T lib_lightgbm.so | fgrep GOMP
0000000000000000      DF *UND*  0000000000000000  GOMP_1.0    GOMP_loop_guided_start
0000000000000000      DF *UND*  0000000000000000  GOMP_1.0    GOMP_atomic_end
0000000000000000      DF *UND*  0000000000000000  GOMP_1.0    GOMP_atomic_start
0000000000000000      DF *UND*  0000000000000000  GOMP_1.0    GOMP_parallel_end
0000000000000000      DF *UND*  0000000000000000  GOMP_1.0    GOMP_loop_guided_next
0000000000000000      DF *UND*  0000000000000000  GOMP_1.0    GOMP_parallel_start
0000000000000000      DF *UND*  0000000000000000  GOMP_1.0    GOMP_loop_end_nowait

And there are lots of new apis in GLIBCXX.

guoke@test-vm3:~$ objdump -T lib_lightgbm.so | fgrep GLIBCXX_3.4.20
0000000000000000      DF *UND*  0000000000000000  GLIBCXX_3.4.20 _ZSt24__throw_out_of_range_fmtPKcz
guoke@test-vm3:~$ objdump -T lib_lightgbm.so | fgrep GLIBCXX_3.4.21
0000000000000000      DF *UND*  0000000000000000  GLIBCXX_3.4.21 _ZNSt7__cxx1119basic_ostringstreamIcSt11char_traitsIcESaIcEED1Ev
0000000000000000      DF *UND*  0000000000000000  GLIBCXX_3.4.21 _ZNKSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE7compareEPKc
0000000000000000      DF *UND*  0000000000000000  GLIBCXX_3.4.21 _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE9_M_appendEPKcm
0000000000000000      DO *UND*  0000000000000000  GLIBCXX_3.4.21 _ZTTNSt7__cxx1118basic_stringstreamIcSt11char_traitsIcESaIcEEE
0000000000000000      DF *UND*  0000000000000000  GLIBCXX_3.4.21 _ZNSt13runtime_errorC1EPKc
0000000000000000      DF *UND*  0000000000000000  GLIBCXX_3.4.21 _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE9_M_assignERKS4_
0000000000000000      DF *UND*  0000000000000000  GLIBCXX_3.4.21 _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEED1Ev
0000000000000000      DF *UND*  0000000000000000  GLIBCXX_3.4.21 _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE4swapERS4_
0000000000000000      DF *UND*  0000000000000000  GLIBCXX_3.4.21 _ZNSt7__cxx1118basic_stringstreamIcSt11char_traitsIcESaIcEED1Ev
0000000000000000      DF *UND*  0000000000000000  GLIBCXX_3.4.21 _ZNSt7__cxx1118basic_stringstreamIcSt11char_traitsIcESaIcEEC1ESt13_Ios_Openmode
0000000000000000      DF *UND*  0000000000000000  GLIBCXX_3.4.21 _ZNSt7__cxx1119basic_ostringstreamIcSt11char_traitsIcESaIcEEC1ESt13_Ios_Openmode
0000000000000000      DF *UND*  0000000000000000  GLIBCXX_3.4.21 _ZNSt6thread15_M_start_threadESt10shared_ptrINS_10_Impl_baseEEPFvvE
0000000000000000      DO *UND*  0000000000000000  GLIBCXX_3.4.21 _ZTVNSt7__cxx1118basic_stringstreamIcSt11char_traitsIcESaIcEEE
0000000000000000      DF *UND*  0000000000000000  GLIBCXX_3.4.21 _ZNKSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE6substrEmm
0000000000000000      DF *UND*  0000000000000000  GLIBCXX_3.4.21 _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE8_M_eraseEmm
0000000000000000      DF *UND*  0000000000000000  GLIBCXX_3.4.21 _ZNKSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE7compareEmmRKS4_
0000000000000000      DF *UND*  0000000000000000  GLIBCXX_3.4.21 _ZNSt7__cxx1115basic_stringbufIcSt11char_traitsIcESaIcEE7_M_syncEPcmm
0000000000000000      DF *UND*  0000000000000000  GLIBCXX_3.4.21 _ZNKSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE4findEcm
0000000000000000      DF *UND*  0000000000000000  GLIBCXX_3.4.21 _ZNKSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE13find_first_ofEPKcmm
0000000000000000      DF *UND*  0000000000000000  GLIBCXX_3.4.21 _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE10_M_replaceEmmPKcm
0000000000000000      DF *UND*  0000000000000000  GLIBCXX_3.4.21 _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE7reserveEm
0000000000000000      DF *UND*  0000000000000000  GLIBCXX_3.4.21 _ZNKSt7__cxx1115basic_stringbufIcSt11char_traitsIcESaIcEE3strEv
0000000000000000      DF *UND*  0000000000000000  GLIBCXX_3.4.21 _ZNSt13runtime_errorC1ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE
0000000000000000      DF *UND*  0000000000000000  GLIBCXX_3.4.21 _ZNKSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE16find_last_not_ofEPKcmm
0000000000000000      DF *UND*  0000000000000000  GLIBCXX_3.4.21 _ZNKSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE17find_first_not_ofEPKcmm
0000000000000000      DF *UND*  0000000000000000  GLIBCXX_3.4.21 _ZNSt13random_device7_M_initERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE
0000000000000000      DF *UND*  0000000000000000  GLIBCXX_3.4.21 _ZNKSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE4findEPKcmm
0000000000000000      DF *UND*  0000000000000000  GLIBCXX_3.4.21 _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE9_M_createERmm
0000000000000000      DF *UND*  0000000000000000  GLIBCXX_3.4.21 _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE9_M_mutateEmmPKcm
0000000000000000      DF *UND*  0000000000000000  GLIBCXX_3.4.21 _ZSt7getlineIcSt11char_traitsIcESaIcEERSt13basic_istreamIT_T0_ES7_RNSt7__cxx1112basic_stringIS4_S5_T1_EES4_
0000000000000000      DF *UND*  0000000000000000  GLIBCXX_3.4.21 _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE14_M_replace_auxEmmmc
0000000000000000      DO *UND*  0000000000000000  GLIBCXX_3.4.21 _ZTVNSt7__cxx1115basic_stringbufIcSt11char_traitsIcESaIcEEE

StrikerRUS · 2018-09-30T10:57:46Z

@guolinke I've got VM with Ubuntu 16 and tried to compile LightGBM (without your last fix, master branch) with max compatibility (I can repeat all steps in Azure script). Can you please check the attached file (especially libgomp)?

lib_lightgbm.zip

guolinke · 2018-09-30T11:19:31Z

@StrikerRUS thanks so much, it seems everything is fine.

guolinke · 2018-09-30T11:30:37Z

@StrikerRUS can you create a PR for the fix ?

guolinke · 2018-09-30T13:34:53Z

@StrikerRUS I also manage to use docker for building from ubuntu 14.04 in azure pipeline.
However, it always met this error:

##[Error 1]
The agent: Hosted Ubuntu 1604 2 lost communication with the server. Verify the machine is running and has a healthy network connection. For more information, see: https://go.microsoft.com/fwlink/?linkid=846610 

##[Error 2]
The request was failed due to an internal service error. Please try again.

It seems the Hosted Ubuntu 1604 is not stable for use.
ping @davidstaheli @brcrista to confirm.

StrikerRUS · 2018-09-30T13:47:08Z

@guolinke

can you create a PR for the fix ?

Sure.

I also manage to use docker for building from ubuntu 14.04 in azure pipeline.

What about the time?

guolinke · 2018-09-30T14:08:59Z

It needs an additional 1 mins to initialization. But strangly, the conda environemnt is faster...

And I prefer the downgrade solution.

StrikerRUS · 2018-09-30T15:12:13Z

@guolinke Please check this file: https://lightgbm-ci.visualstudio.com/_apis/resources/Containers/1683333?itemPath=PackageAssets%2Flib_lightgbm.so

guolinke · 2018-09-30T16:38:57Z

@StrikerRUS thanks so much !
BTW, i find the the "Hosted Ubuntu 1604" is faster than "Hosted Linux Preview". Maybe we can move to it.

StrikerRUS · 2018-09-30T17:12:16Z

@guolinke No problem! Glad to help you.

BTW, i find the the "Hosted Ubuntu 1604" is faster than "Hosted Linux Preview". Maybe we can move to it.

I think we should ask someone competent in this (from Azure team).

brcrista · 2018-10-01T11:57:36Z

@guolinke correct, you need to use Hosted Ubuntu 1604 to run containers. I would recommend replacing Hosted Linux Preview with Hosted Ubuntu 1604. The "preview" is basically done and you can think of Hosted Ubuntu as the GA release.

As for the issues you are seeing with that pool, I see you opened microsoft/azure-pipelines-tasks#8455, so I'll let that question be answered there.

guolinke · 2018-10-03T17:25:52Z

fixed in the latest binaries.

chjq201410695 · 2018-12-06T05:44:59Z

fixed in the latest binaries.

Excuse me. I find the lastest version of lightgbm need glibc >=2.14. However, the version of glibc in my cluster is 2.12, and they will not update it in near future. How can I use lightgbm in my cluster?

guolinke · 2018-12-06T08:40:02Z

@chjq201410695 you can compile lightgbm by yourself.

guolinke mentioned this issue Sep 30, 2018

ubuntu 14.04 in Microsoft Hosted Agent microsoft/azure-pipelines-tasks#8455

Closed

StrikerRUS mentioned this issue Sep 30, 2018

[ci] dowgrade gcc version for Linux on Azure #1718

Merged

guolinke closed this as completed Oct 3, 2018

alois-bissuel mentioned this issue Oct 17, 2018

New release with LightGBM 2.2.1 microsoft/SynapseML#390

Closed

lock bot locked as resolved and limited conversation to collaborators Mar 11, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support GLIBC 2.17 in the build binaries #1708

Support GLIBC 2.17 in the build binaries #1708

guolinke commented Sep 28, 2018 •

edited

Loading

guolinke commented Sep 28, 2018 •

edited

Loading

StrikerRUS commented Sep 28, 2018 •

edited

Loading

skysun99 commented Sep 28, 2018

guolinke commented Sep 29, 2018

guolinke commented Sep 29, 2018 •

edited

Loading

guolinke commented Sep 30, 2018

StrikerRUS commented Sep 30, 2018 •

edited

Loading

guolinke commented Sep 30, 2018

guolinke commented Sep 30, 2018

guolinke commented Sep 30, 2018

StrikerRUS commented Sep 30, 2018

guolinke commented Sep 30, 2018

StrikerRUS commented Sep 30, 2018

guolinke commented Sep 30, 2018

StrikerRUS commented Sep 30, 2018

brcrista commented Oct 1, 2018

guolinke commented Oct 3, 2018

chjq201410695 commented Dec 6, 2018

guolinke commented Dec 6, 2018

Support GLIBC 2.17 in the build binaries #1708

Support GLIBC 2.17 in the build binaries #1708

Comments

guolinke commented Sep 28, 2018 • edited Loading

guolinke commented Sep 28, 2018 • edited Loading

StrikerRUS commented Sep 28, 2018 • edited Loading

skysun99 commented Sep 28, 2018

guolinke commented Sep 29, 2018

guolinke commented Sep 29, 2018 • edited Loading

guolinke commented Sep 30, 2018

StrikerRUS commented Sep 30, 2018 • edited Loading

guolinke commented Sep 30, 2018

guolinke commented Sep 30, 2018

guolinke commented Sep 30, 2018

StrikerRUS commented Sep 30, 2018

guolinke commented Sep 30, 2018

StrikerRUS commented Sep 30, 2018

guolinke commented Sep 30, 2018

StrikerRUS commented Sep 30, 2018

brcrista commented Oct 1, 2018

guolinke commented Oct 3, 2018

chjq201410695 commented Dec 6, 2018

guolinke commented Dec 6, 2018

guolinke commented Sep 28, 2018 •

edited

Loading

guolinke commented Sep 28, 2018 •

edited

Loading

StrikerRUS commented Sep 28, 2018 •

edited

Loading

guolinke commented Sep 29, 2018 •

edited

Loading

StrikerRUS commented Sep 30, 2018 •

edited

Loading