Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make the testing executable dynamic linking. #1401

Closed
Shylock-Hg opened this issue Dec 9, 2019 · 3 comments
Closed

Make the testing executable dynamic linking. #1401

Shylock-Hg opened this issue Dec 9, 2019 · 3 comments
Assignees
@dutor

Comments

@Shylock-Hg
Copy link
Contributor

Is your feature request related to a problem? Please describe.
The testing executables disk consumption is too large to development and CI/CD.

Describe the solution you'd like
Make testing dynamic link.

Describe alternatives you've considered

Additional context
Block the coverage because the action disk consumption over.
@Shylock-Hg Shylock-Hg mentioned this issue Dec 10, 2019
Closed
@Shylock-Hg Shylock-Hg mentioned this issue Dec 12, 2019
Closed
@whitewum
Copy link
Contributor

what's the minimum disk capacity required?

@Shylock-Hg Shylock-Hg removed their assignment Dec 17, 2019
@dutor
Copy link
Contributor

dutor commented Dec 18, 2019

what's the minimum disk capacity required?

#1332 reduces the build size from 28GB to about 5GB, in the Debug mode.

@dutor
Copy link
Contributor

dutor commented Dec 18, 2019

It is the debug info that makes the executables huge, instead of code:

----------------------------------------
_build/nebula-graphd
----------------------------------------
.debug_info                79.5MB  39.2%
.debug_loc                 47.2MB  23.3%
.debug_str                 20.8MB  10.3%
.text                      13.4MB   6.6%
.debug_ranges              11.3MB   5.6%
.debug_line                9.30MB   4.6%
.strtab                    6.45MB   3.2%
.dynstr                    4.29MB   2.1%
.eh_frame                  2.04MB   1.0%
.debug_abbrev              1.79MB   0.9%
.symtab                    1.59MB   0.8%
.rodata                    1.48MB   0.7%
.dynsym                    1.05MB   0.5%
.gcc_except_table           588KB   0.3%
.data.rel.ro                470KB   0.2%
.eh_frame_hdr               372KB   0.2%
.gnu.hash                   338KB   0.2%
.hash                       308KB   0.1%
.debug_aranges              250KB   0.1%
TOTAL                       203MB 100.0%
----------------------------------------

Considering the build size has been reduced a lot in #1332, I will close this issue for now.

BTW. I have to mention that to link third party libraries dynamically could further make the build size smaller( by 15% I guess), but it takes a little more effort, which is not worthy for the time being.

@dutor dutor mentioned this issue Dec 18, 2019
Merged
@dutor dutor closed this as completed Dec 18, 2019
dutor added a commit that referenced this issue Jan 1, 2020
@dutor
Refactor building (#1332)
875e074 
This PR is trying to improve the building experience, by:
 1. Providing `third-party/install-cmake.sh` to download and install a prebuilt CMake
 2. Providing `third-party/build-gcc.sh` to automatically download and build GCC 7.5.0
 3. Providing `third-party/install-gcc.sh` to download and install a prebuilt GCC 7.5.0 according to the current distro and libc
 4. Providing `third-party/build-third-party.sh` to build and optionally package our third party dependencies
 4. Providing `third-party/build-all-third-party.sh` to invoke `build-third-party.sh` using several preset versions of GCC
 5. Providing `third-party/install-third-party.sh` to download and install a prebuilt third party according to the current version of glibc, GCC(along with its ABI version)
 6. Invoke `install-third-party.sh` automatically during the configure stage(cmake)

Besides, all installing scripts support an `--prefix` option to allow one customizing the install location. For the third party installation, works are done to make *Bison* and *krb5* relocatable.

The fresh new building shceme has been tested under various environments:
 1. CentOS 6/7/8
 2. Debian 7/8/9
 3. Ubuntu 16.04/18.04/19.04
 4. GCC 7/8/9, and devtoolset-7 of CentOS 6/7

Since most of these testing are from inside a Docker image, so the Linux Kernel used is nearly the newest one(5.0). There might be other unfound issues.

To try out this PR, run
```sh
$ git clone --single-branch --depth=1 --branch refactor-building https://github.com/dutor/nebula.git nebula-refactor-building
$ cd nebula-refactor-building
$ mkdir build && cd build
$ ../third-party/install-cmake.sh             # Can be skipped if you have CMake 3.5.0+
$ source cmake-3.15.5/bin/enable-cmake.sh     # Only necessary if you invoked install-cmake.sh
$ sudo ../third-party/install-gcc.sh          # Can be skipped if you have GCC 7.1.0+
$ source /opt/vesoft/toolset/gcc/7.5.0/enable # Only necessary if you invoked install-gcc.sh
$ cmake ..
$ make
```

BTW. This PR also fixes #1449 
BTW. This PR reduces the build size a lot, as mentioned in #1401
tong-hao pushed a commit to tong-hao/nebula that referenced this issue Jun 1, 2021
@dutor
Refactor building (vesoft-inc#1332)
afa9a2a 
This PR is trying to improve the building experience, by:
 1. Providing `third-party/install-cmake.sh` to download and install a prebuilt CMake
 2. Providing `third-party/build-gcc.sh` to automatically download and build GCC 7.5.0
 3. Providing `third-party/install-gcc.sh` to download and install a prebuilt GCC 7.5.0 according to the current distro and libc
 4. Providing `third-party/build-third-party.sh` to build and optionally package our third party dependencies
 4. Providing `third-party/build-all-third-party.sh` to invoke `build-third-party.sh` using several preset versions of GCC
 5. Providing `third-party/install-third-party.sh` to download and install a prebuilt third party according to the current version of glibc, GCC(along with its ABI version)
 6. Invoke `install-third-party.sh` automatically during the configure stage(cmake)

Besides, all installing scripts support an `--prefix` option to allow one customizing the install location. For the third party installation, works are done to make *Bison* and *krb5* relocatable.

The fresh new building shceme has been tested under various environments:
 1. CentOS 6/7/8
 2. Debian 7/8/9
 3. Ubuntu 16.04/18.04/19.04
 4. GCC 7/8/9, and devtoolset-7 of CentOS 6/7

Since most of these testing are from inside a Docker image, so the Linux Kernel used is nearly the newest one(5.0). There might be other unfound issues.

To try out this PR, run
```sh
$ git clone --single-branch --depth=1 --branch refactor-building https://github.com/dutor/nebula.git nebula-refactor-building
$ cd nebula-refactor-building
$ mkdir build && cd build
$ ../third-party/install-cmake.sh             # Can be skipped if you have CMake 3.5.0+
$ source cmake-3.15.5/bin/enable-cmake.sh     # Only necessary if you invoked install-cmake.sh
$ sudo ../third-party/install-gcc.sh          # Can be skipped if you have GCC 7.1.0+
$ source /opt/vesoft/toolset/gcc/7.5.0/enable # Only necessary if you invoked install-gcc.sh
$ cmake ..
$ make
```

BTW. This PR also fixes vesoft-inc#1449 
BTW. This PR reduces the build size a lot, as mentioned in vesoft-inc#1401
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dutor dutor

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@dutor @Shylock-Hg @whitewum