Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Log files in robot build leak into each other #3533

Closed
zao opened this issue Dec 21, 2020 · 3 comments · Fixed by easybuilders/easybuild-easyblocks#2702
Closed

Log files in robot build leak into each other #3533

zao opened this issue Dec 21, 2020 · 3 comments · Fixed by easybuilders/easybuild-easyblocks#2702
Labels
bug report
Milestone
4.5.4

Comments

@zao
Copy link
Contributor

zao commented Dec 21, 2020

I built with eb R-4.0.0-foss-2020a.eb -r on Ubuntu 20.04, Python 3.8.5, framework 5d8747a and ran out of disk space to many gigabytes of logs.

Investigation shows that there are multiple log files in the uniquely-named directory in /tmp; each individual log file like easybuild-Xvfb-1.20.9-20201221.170919.PHYQe.log and easybuild-xauth-1.1-20201221.170914.UrvMk.log contains the output of many other builds in that session, including full build output from libraries later in the build sequence.

For example, the Xvfb log, being rather early in the dependency chain, contains the following entries spread out across the file:

== 2020-12-21 17:09:19,391 easyblock.py:918 INFO Build dir set to /tmp/eb-build/Xvfb/1.20.9/GCCcore-9.3.0
== 2020-12-21 17:10:21,602 easyblock.py:918 INFO Build dir set to /tmp/eb-build/NLopt/2.6.1/GCCcore-9.3.0
== 2020-12-21 17:10:40,852 easyblock.py:918 INFO Build dir set to /tmp/eb-build/libsndfile/1.0.28/GCCcore-9.3.0
== 2020-12-21 17:10:55,912 easyblock.py:918 INFO Build dir set to /tmp/eb-build/ICU/66.1/GCCcore-9.3.0
== 2020-12-21 17:11:59,858 easyblock.py:918 INFO Build dir set to /tmp/eb-build/UDUNITS/2.2.26/GCCcore-9.3.0
== 2020-12-21 17:12:10,757 easyblock.py:918 INFO Build dir set to /tmp/eb-build/Szip/2.1.1/GCCcore-9.3.0
== 2020-12-21 17:12:18,272 easyblock.py:918 INFO Build dir set to /tmp/eb-build/HDF5/1.10.6/gompi-2020a
== 2020-12-21 17:15:12,229 easyblock.py:918 INFO Build dir set to /tmp/eb-build/GSL/2.6/GCC-9.3.0
== 2020-12-21 17:16:33,380 easyblock.py:918 INFO Build dir set to /tmp/eb-build/GLPK/4.65/GCCcore-9.3.0
== 2020-12-21 17:16:50,629 easyblock.py:918 INFO Build dir set to /tmp/eb-build/Ghostscript/9.52/GCCcore-9.3.0
== 2020-12-21 17:20:10,191 easyblock.py:918 INFO Build dir set to /tmp/eb-build/JasPer/2.0.14/GCCcore-9.3.0
== 2020-12-21 17:20:17,955 easyblock.py:918 INFO Build dir set to /tmp/eb-build/LittleCMS/2.9/GCCcore-9.3.0
== 2020-12-21 17:20:30,676 easyblock.py:918 INFO Build dir set to /tmp/eb-build/ImageMagick/7.0.10-1/GCCcore-9.3.0
== 2020-12-21 17:22:18,498 easyblock.py:918 INFO Build dir set to /tmp/eb-build/R/4.0.0/foss-2020a

The transitions in that log file claim to be closing the log for various applications and carries on logging nevertheless:

== 2020-12-21 17:10:21,448 easyblock.py:295 INFO Closing log for application name Xvfb version 1.20.9
== 2020-12-21 17:10:21,449 filetools.py:1602 DEBUG Not creating existing path /eb/software/Xvfb/1.20.9-GCCcore-9.3.0/easybuild
== 2020-12-21 17:10:21,450 filetools.py:1602 DEBUG Not creating existing path /eb/software/Xvfb/1.20.9-GCCcore-9.3.0/easybuild
== 2020-12-21 17:10:21,462 filetools.py:2324 INFO /tmp/eb-l05ci078/easybuild-Xvfb-1.20.9-20201221.170833.VIAer.log moved to /eb/software/Xvfb/1.20.9-GCCcore-9.3.0/easybuild/easybuild-Xvfb-1.20.9-20201221.171021.log
== 2020-12-21 17:10:21,463 filetools.py:1848 INFO Moved log file /tmp/eb-l05ci078/easybuild-Xvfb-1.20.9-20201221.170833.VIAer.log to /eb/software/Xvfb/1.20.9-GCCcore-9.3.0/easybuild/easybuild-Xvfb-1.20.9-20201221.171021.log
== 2020-12-21 17:10:21,464 filetools.py:1602 DEBUG Not creating existing path /eb/software/Xvfb/1.20.9-GCCcore-9.3.0/easybuild
== 2020-12-21 17:10:21,468 filetools.py:2093 INFO /eb/develop/easybuild-easyconfigs/easybuild/easyconfigs/x/Xvfb/Xvfb-1.20.9-GCCcore-9.3.0.eb copied to /eb/software/Xvfb/1.20.9-GCCcore-9.3.0/easybuild/Xvfb-1.20.9-GCCcore-9.3.0.eb
== 2020-12-21 17:10:21,468 easyblock.py:3408 DEBUG Copied easyconfig file /eb/develop/easybuild-easyconfigs/easybuild/easyconfigs/x/Xvfb/Xvfb-1.20.9-GCCcore-9.3.0.eb to /eb/software/Xvfb/1.20.9-GCCcore-9.3.0/easybuild/Xvfb-1.20.9-GCCcore-9.3.0.eb
== 2020-12-21 17:10:21,469 filetools.py:1602 DEBUG Not creating existing path /eb/software/Xvfb/1.20.9-GCCcore-9.3.0/easybuild
== 2020-12-21 17:10:21,472 filetools.py:2093 INFO /eb/develop/easybuild-easyconfigs/easybuild/easyconfigs/x/Xvfb/xvfb-run copied to /eb/software/Xvfb/1.20.9-GCCcore-9.3.0/easybuild/xvfb-run
== 2020-12-21 17:10:21,473 easyblock.py:3414 DEBUG Copied patch /eb/develop/easybuild-easyconfigs/easybuild/easyconfigs/x/Xvfb/xvfb-run to /eb/software/Xvfb/1.20.9-GCCcore-9.3.0/easybuild/xvfb-run
== 2020-12-21 17:10:21,473 build_log.py:265 INFO COMPLETED: Installation ended successfully (took 1 min 47 sec)
== 2020-12-21 17:10:21,474 build_log.py:265 INFO Results of the build can be found in the log file(s) /eb/software/Xvfb/1.20.9-GCCcore-9.3.0/easybuild/easybuild-Xvfb-1.20.9-20201221.171021.log
== 2020-12-21 17:10:21,476 filetools.py:1602 DEBUG Not creating existing path /eb/software/Xvfb/1.20.9-GCCcore-9.3.0/easybuild
== 2020-12-21 17:10:21,477 build_log.py:265 INFO processing EasyBuild easyconfig /eb/develop/easybuild-easyconfigs/easybuild/easyconfigs/n/NLopt/NLopt-2.6.1-GCCcore-9.3.0.eb
== 2020-12-21 17:10:21,478 easyblock.py:3280 INFO Resetting environment

The full log set obtained from the uniquely-named directory in /tmp can be found at https://zao.se/~zao/eb/R-huge-logs.tar.zst (844 MiB compressed!) expanding to 42.7 GiB.

Two individual log files and the overall build log:

Notable here is that the lingering logs seem to be either Bundle configs or bundled in such a config.

Log file names:

-rw-r--r-- easybuild/folk 701345792 2020-12-21 20:25 easybuild-R-4.0.0-20201221.172218.yoxgS.log
-rw-r--r-- easybuild/folk 727330816 2020-12-21 20:25 easybuild-Xvfb-1.20.9-20201221.170919.PHYQe.log
-rw-r--r-- easybuild/folk 727654400 2020-12-21 20:25 easybuild-bdftopcf-1.1-20201221.170850.kRWUC.log
-rw-r--r-- easybuild/folk 727592960 2020-12-21 20:25 easybuild-font-misc-misc-1.1.2-20201221.170858.glPXm.log
-rw-r--r-- easybuild/folk 727617536 2020-12-21 20:25 easybuild-font-util-1.3.2-20201221.170854.PFcPC.log
-rw-r--r-- easybuild/folk 769847296 2020-12-21 20:25 easybuild-libFS-1.0.8-20201221.163624.yxZWM.log
-rw-r--r-- easybuild/folk 769814528 2020-12-21 20:25 easybuild-libICE-1.0.10-20201221.163631.EUvbc.log
-rw-r--r-- easybuild/folk 769781760 2020-12-21 20:25 easybuild-libSM-1.2.3-20201221.163639.QXrfX.log
-rw-r--r-- easybuild/folk 770048000 2020-12-21 20:25 easybuild-libX11-1.6.9-20201221.163526.iVgUn.log
-rw-r--r-- easybuild/folk 769753088 2020-12-21 20:25 easybuild-libXScrnSaver-1.2.3-20201221.163645.VgAAY.log
-rw-r--r-- easybuild/folk 770396160 2020-12-21 20:25 easybuild-libXau-1.0.9-20201221.163439.xwtUf.log
-rw-r--r-- easybuild/folk 769556480 2020-12-21 20:25 easybuild-libXaw-1.0.13-20201221.163720.Xcaqr.log
-rw-r--r-- easybuild/folk 769007616 2020-12-21 20:25 easybuild-libXcomposite-0.4.5-20201221.163737.RSAhl.log
-rw-r--r-- easybuild/folk 768950272 2020-12-21 20:25 easybuild-libXcursor-1.2.0-20201221.163749.gYjiV.log
-rw-r--r-- easybuild/folk 768917504 2020-12-21 20:25 easybuild-libXdamage-1.1.5-20201221.163755.ouDGK.log
-rw-r--r-- easybuild/folk 770355200 2020-12-21 20:25 easybuild-libXdmcp-1.1.3-20201221.163445.NJEfI.log
-rw-r--r-- easybuild/folk 769880064 2020-12-21 20:25 easybuild-libXext-1.3.4-20201221.163617.SQARn.log
-rw-r--r-- easybuild/folk 769036288 2020-12-21 20:25 easybuild-libXfixes-5.0.3-20201221.163732.XpgeM.log
-rw-r--r-- easybuild/folk 768856064 2020-12-21 20:25 easybuild-libXfont-1.5.4-20201221.163806.zFMsC.log
-rw-r--r-- easybuild/folk 768819200 2020-12-21 20:25 easybuild-libXfont2-2.0.3-20201221.163820.AzYWL.log
-rw-r--r-- easybuild/folk 768786432 2020-12-21 20:25 easybuild-libXft-2.3.3-20201221.163830.wVqBI.log
-rw-r--r-- easybuild/folk 768757760 2020-12-21 20:25 easybuild-libXi-1.7.10-20201221.163837.gAVNp.log
-rw-r--r-- easybuild/folk 768720896 2020-12-21 20:25 easybuild-libXinerama-1.1.4-20201221.163845.uPKSY.log
-rw-r--r-- easybuild/folk 769662976 2020-12-21 20:25 easybuild-libXmu-1.1.3-20201221.163702.SfxVf.log
-rw-r--r-- easybuild/folk 768110592 2020-12-21 20:25 easybuild-libXp-1.0.3-20201221.164033.ODrlc.log
-rw-r--r-- easybuild/folk 769597440 2020-12-21 20:25 easybuild-libXpm-3.5.13-20201221.163711.DSnub.log
-rw-r--r-- easybuild/folk 768692224 2020-12-21 20:25 easybuild-libXrandr-1.5.2-20201221.163851.WSsgY.log
-rw-r--r-- easybuild/folk 768978944 2020-12-21 20:25 easybuild-libXrender-0.9.10-20201221.163743.FNmQY.log
-rw-r--r-- easybuild/folk 768663552 2020-12-21 20:25 easybuild-libXres-1.2.0-20201221.163857.VNMrU.log
-rw-r--r-- easybuild/folk 769724416 2020-12-21 20:25 easybuild-libXt-1.2.0-20201221.163650.IlCvt.log
-rw-r--r-- easybuild/folk 768634880 2020-12-21 20:25 easybuild-libXtst-1.2.3-20201221.163903.jNDDN.log
-rw-r--r-- easybuild/folk 768602112 2020-12-21 20:25 easybuild-libXv-1.0.11-20201221.163908.sJUCA.log
-rw-r--r-- easybuild/folk 768569344 2020-12-21 20:25 easybuild-libXvMC-1.0.12-20201221.163914.ZvCmZ.log
-rw-r--r-- easybuild/folk 768532480 2020-12-21 20:25 easybuild-libXxf86dga-1.1.5-20201221.163921.jlZwm.log
-rw-r--r-- easybuild/folk 768503808 2020-12-21 20:25 easybuild-libXxf86vm-1.1.4-20201221.163927.dLabb.log
-rw-r--r-- easybuild/folk 768475136 2020-12-21 20:25 easybuild-libdmx-1.1.4-20201221.163933.NNhEw.log
-rw-r--r-- easybuild/folk 768888832 2020-12-21 20:25 easybuild-libfontenc-1.1.4-20201221.163800.XDeqB.log
-rw-r--r-- easybuild/folk 770527232 2020-12-21 20:25 easybuild-libpthread-stubs-0.4-20201221.163431.OaGVj.log
-rw-r--r-- easybuild/folk 770306048 2020-12-21 20:25 easybuild-libxcb-1.14-20201221.163453.glbMO.log
-rw-r--r-- easybuild/folk 770084864 2020-12-21 20:25 easybuild-libxkbcommon-0.10.0-20201221.163522.CfZHZ.log
-rw-r--r-- easybuild/folk 768446464 2020-12-21 20:25 easybuild-libxkbfile-1.1.0-20201221.163939.SbjGA.log
-rw-r--r-- easybuild/folk 768397312 2020-12-21 20:25 easybuild-libxshmfence-1.3-20201221.163945.wPcgH.log
-rw-r--r-- easybuild/folk 727703552 2020-12-21 20:25 easybuild-mkfontdir-1.0.7-20201221.170845.okbov.log
-rw-r--r-- easybuild/folk 727773184 2020-12-21 20:25 easybuild-mkfontscale-1.2.1-20201221.170841.WLcSt.log
-rw-r--r-- easybuild/folk 768131072 2020-12-21 20:25 easybuild-printproto-1.0.5-20201221.164031.ipFaW.log
-rw------- easybuild/folk 973336576 2020-12-21 20:25 easybuild-w26s6i0a.log
-rw-r--r-- easybuild/folk 727355392 2020-12-21 20:25 easybuild-xauth-1.1-20201221.170914.UrvMk.log
-rw-r--r-- easybuild/folk 768081920 2020-12-21 20:25 easybuild-xbitmaps-1.1.2-20201221.164039.WlMXf.log
-rw-r--r-- easybuild/folk 770322432 2020-12-21 20:25 easybuild-xcb-proto-1.14-20201221.163451.YaHUR.log
-rw-r--r-- easybuild/folk 768368640 2020-12-21 20:25 easybuild-xcb-util-0.4.0-20201221.163951.NDwtC.log
-rw-r--r-- easybuild/folk 768217088 2020-12-21 20:25 easybuild-xcb-util-cursor-0.1.3-20201221.164020.jvfVf.log
-rw-r--r-- easybuild/folk 768339968 2020-12-21 20:25 easybuild-xcb-util-image-0.4.0-20201221.163956.AAzjr.log
-rw-r--r-- easybuild/folk 768311296 2020-12-21 20:25 easybuild-xcb-util-keysyms-0.4.0-20201221.164003.PXOrp.log
-rw-r--r-- easybuild/folk 768282624 2020-12-21 20:25 easybuild-xcb-util-renderutil-0.3.9-20201221.164008.bJMTX.log
-rw-r--r-- easybuild/folk 768253952 2020-12-21 20:25 easybuild-xcb-util-wm-0.4.1-20201221.164013.hVDZN.log
-rw-r--r-- easybuild/folk 727527424 2020-12-21 20:25 easybuild-xkbcomp-1.4.3-20201221.170906.xGxHZ.log
-rw-r--r-- easybuild/folk 768188416 2020-12-21 20:25 easybuild-xkeyboard-config-2.28-20201221.164026.wAebk.log
-rw-r--r-- easybuild/folk 727461888 2020-12-21 20:25 easybuild-xkeyboard-config-2.31-20201221.170910.KUYyU.log
-rw-r--r-- easybuild/folk 770465792 2020-12-21 20:25 easybuild-xorgproto-2019.2-20201221.163433.wojKr.log
-rw-r--r-- easybuild/folk 770105344 2020-12-21 20:25 easybuild-xtrans-1.4.0-20201221.163518.hBofy.log
@zao
Copy link
Contributor Author

zao commented Dec 22, 2020

Building on Ubuntu 16.04 with Python 2.7.12, I get less mention of other builds in my logs, mostly limited to the X11 bundle, but it still seems a bit fishy:

easybuild-bdftopcf-1.1-20201221.225034.QGEpj.log:== 2020-12-21 22:52:04,809 easyblock.py:295 INFO Closing log for application name Xvfb version 1.20.9
easybuild-font-misc-misc-1.1.2-20201221.225041.JHwop.log:== 2020-12-21 22:52:04,809 easyblock.py:295 INFO Closing log for application name Xvfb version 1.20.9
easybuild-font-util-1.3.2-20201221.225038.BkVej.log:== 2020-12-21 22:52:04,809 easyblock.py:295 INFO Closing log for application name Xvfb version 1.20.9
easybuild-libdmx-1.1.4-20201221.224234.zjLCI.log:== 2020-12-21 22:44:09,517 easyblock.py:295 INFO Closing log for application name X11 version 20200222
easybuild-libdmx-1.1.4-20201221.224234.zjLCI.log:== 2020-12-21 22:44:29,638 easyblock.py:295 INFO Closing log for application name libdrm version 2.4.100
easybuild-libdmx-1.1.4-20201221.224234.zjLCI.log:== 2020-12-21 22:44:49,060 easyblock.py:295 INFO Closing log for application name libglvnd version 1.2.0
easybuild-libdmx-1.1.4-20201221.224234.zjLCI.log:== 2020-12-21 22:49:55,242 easyblock.py:295 INFO Closing log for application name Mesa version 20.0.2
easybuild-libdmx-1.1.4-20201221.224234.zjLCI.log:== 2020-12-21 22:50:15,959 easyblock.py:295 INFO Closing log for application name libGLU version 9.0.1
easybuild-libdmx-1.1.4-20201221.224234.zjLCI.log:== 2020-12-21 22:52:04,809 easyblock.py:295 INFO Closing log for application name Xvfb version 1.20.9
easybuild-libfontenc-1.1.4-20201221.224109.ccRQO.log:== 2020-12-21 22:44:09,517 easyblock.py:295 INFO Closing log for application name X11 version 20200222
easybuild-libfontenc-1.1.4-20201221.224109.ccRQO.log:== 2020-12-21 22:44:29,638 easyblock.py:295 INFO Closing log for application name libdrm version 2.4.100
easybuild-libfontenc-1.1.4-20201221.224109.ccRQO.log:== 2020-12-21 22:44:49,060 easyblock.py:295 INFO Closing log for application name libglvnd version 1.2.0
easybuild-libfontenc-1.1.4-20201221.224109.ccRQO.log:== 2020-12-21 22:49:55,242 easyblock.py:295 INFO Closing log for application name Mesa version 20.0.2
easybuild-libfontenc-1.1.4-20201221.224109.ccRQO.log:== 2020-12-21 22:50:15,959 easyblock.py:295 INFO Closing log for application name libGLU version 9.0.1
easybuild-libfontenc-1.1.4-20201221.224109.ccRQO.log:== 2020-12-21 22:52:04,809 easyblock.py:295 INFO Closing log for application name Xvfb version 1.20.9

@boegel boegel added this to the next release (4.3.3?) milestone Dec 23, 2020
@boegel boegel modified the milestones: 4.3.3, release after 4.3.3 Feb 3, 2021
@boegel boegel modified the milestones: 4.4.0, release after 4.4.0 May 25, 2021
@boegel boegel modified the milestones: 4.4.2, release after 4.4.2 Sep 1, 2021
@boegel boegel modified the milestones: 4.5.0, release after 4.5.0 Oct 13, 2021
@boegel boegel modified the milestones: 4.5.1, release after 4.5.1 Dec 7, 2021
@boegel boegel modified the milestones: 4.5.2, release after 4.5.2 Jan 14, 2022
@boegel boegel modified the milestones: 4.5.3, release after 4.5.3 Feb 8, 2022
@akesandgren
Copy link
Contributor

This is indeed easily triggered by something that uses Bundle easyblock.
Take the X11 .eb, drop all components except the first three and build. The first components build log will contain the output from the second component after its own.

@akesandgren
Copy link
Contributor

EB 4.3.2 has the problem, so it's not a new thing

boegel added a commit to boegel/easybuild-easyblocks that referenced this issue Mar 25, 2022
@boegel
close log after installing Bundle component (fixes easybuilders/easyb…
3ea5572 
…uild-framework#3533)
@boegel boegel mentioned this issue Mar 25, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug report
Projects
None yet
Milestone
4.5.4
Development

Successfully merging a pull request may close this issue.

close log after installing Bundle component boegel/easybuild-easyblocks
3 participants
@akesandgren @zao @boegel