build compat layer software.eessi.io/2023.06 #191

trz42 · 2023-09-09T18:38:16Z

changed repository name to software.eessi.io
fixed creation of tarball at the end of bot/build.sh
- should be improved to handle errors when some tmp directory couldn't be determined
added bot/check-build.sh that prepares .result file including a status update for the PR comment
added a bit more debug output in install_compatibility_layer.sh, should help with setting up debug sessions

Did not change CI pilot_repo.yml yet.

- changed repository name to software.eessi.io - fixed creation of tarball at the end of `bot/build.sh` - should be improved to handle errors when some tmp directory couldn't be determined - added `bot/check-build.sh` that prepares `.result` file including a status update for the PR comment - added a bit more debug output in `install_compatibility_layer.sh`, should help with setting up debug sessions Not changed CI `pilot_repo.yml` yet.

eessi-bot · 2023-09-09T18:38:20Z

Instance eessi-bot-citc-aws is configured to build:

arch x86_64/generic for repo eessi-2021.12
arch x86_64/generic for repo eessi-2023.06-compat
arch x86_64/generic for repo eessi-2023.06-software
arch x86_64/intel/haswell for repo eessi-2021.12
arch x86_64/intel/haswell for repo eessi-2023.06-compat
arch x86_64/intel/haswell for repo eessi-2023.06-software
arch x86_64/intel/skylake_avx512 for repo eessi-2021.12
arch x86_64/intel/skylake_avx512 for repo eessi-2023.06-compat
arch x86_64/intel/skylake_avx512 for repo eessi-2023.06-software
arch x86_64/amd/zen2 for repo eessi-2021.12
arch x86_64/amd/zen2 for repo eessi-2023.06-compat
arch x86_64/amd/zen2 for repo eessi-2023.06-software
arch x86_64/amd/zen3 for repo eessi-2021.12
arch x86_64/amd/zen3 for repo eessi-2023.06-compat
arch x86_64/amd/zen3 for repo eessi-2023.06-software
arch aarch64/generic for repo eessi-2021.12
arch aarch64/generic for repo eessi-2023.06-compat
arch aarch64/generic for repo eessi-2023.06-software
arch aarch64/neoverse_n1 for repo eessi-2021.12
arch aarch64/neoverse_n1 for repo eessi-2023.06-compat
arch aarch64/neoverse_n1 for repo eessi-2023.06-software
arch aarch64/neoverse_v1 for repo eessi-2021.12
arch aarch64/neoverse_v1 for repo eessi-2023.06-compat
arch aarch64/neoverse_v1 for repo eessi-2023.06-software

trz42 · 2023-09-09T18:54:28Z

bot: build arch:generic repo:compat

eessi-bot · 2023-09-09T18:54:31Z

Updates by the bot instance eessi-bot-citc-aws (click for details)

received bot command build arch:generic repo:compat from trz42
- expanded format: build architecture:generic repository:compat
handling command build architecture:generic repository:compat resulted in:
- submitted job 7240, for details & status see build compat layer software.eessi.io/2023.06 #191 (comment)
- submitted job 7241, for details & status see build compat layer software.eessi.io/2023.06 #191 (comment)

eessi-bot · 2023-09-09T18:54:46Z

New job on instance eessi-bot-citc-aws for architecture x86_64-generic for repository eessi-2023.06-compat in job dir /mnt/shared/home/bot/eessi-bot-software-layer/jobs/2023.09/pr_191/7240

date	job status	comment
Sep 09 18:54:46 UTC 2023	submitted	job id `7240` awaits release by job manager
Sep 09 18:55:43 UTC 2023	released	job awaits launch by Slurm scheduler
Sep 09 18:58:47 UTC 2023	running	job `7240` is running
Sep 09 23:51:40 UTC 2023	finished	😢 FAILURE (click triangle for details) Details ✅ job output file `slurm-7240.out` ❌ some task failed ✅ found tarball Artefacts `eessi-2023.06-compat-linux-x86_64-1694303346.tar.gz` size: 1701 MiB (1783649468 bytes) entries: 190965

eessi-bot · 2023-09-09T18:54:48Z

New job on instance eessi-bot-citc-aws for architecture aarch64-generic for repository eessi-2023.06-compat in job dir /mnt/shared/home/bot/eessi-bot-software-layer/jobs/2023.09/pr_191/7241

date	job status	comment
Sep 09 18:54:47 UTC 2023	submitted	job id `7241` awaits release by job manager
Sep 09 18:55:41 UTC 2023	released	job awaits launch by Slurm scheduler
Sep 09 18:58:46 UTC 2023	running	job `7241` is running
Sep 09 23:48:36 UTC 2023	finished	😢 FAILURE (click triangle for details) Details ✅ job output file `slurm-7241.out` ❌ some task failed ✅ found tarball Artefacts `eessi-2023.06-compat-linux-aarch64-1694303123.tar.gz` size: 1735 MiB (1820214175 bytes) entries: 190919

trz42 · 2023-09-10T06:31:55Z

The building "worked", the task that failed was the ReFrame test at the end. This tests (among other things) if a startprefix under the correct prefix is found. It wasn't. It seems, re-using the repo config (repo name) from eessi-2023.06-compat didn't work. The old prefix was used.

Reconfigured the bot to be able to build for two repositories eessi.org-2023.06-compat (renamed from eessi-2023.06-compat) and eessi.io-2023.06-compat (for the new one).

bot: show_config

eessi-bot · 2023-09-10T06:31:58Z

Updates by the bot instance eessi-bot-citc-aws (click for details)

received bot command show_config from trz42
- expanded format: show_config
handling command show_config resulted in:
- added comment build compat layer software.eessi.io/2023.06 #191 (comment) to show configuration

eessi-bot · 2023-09-10T06:32:01Z

Instance eessi-bot-citc-aws is configured to build:

arch x86_64/generic for repo eessi-2021.12
arch x86_64/generic for repo eessi.org-2023.06-compat
arch x86_64/generic for repo eessi.io-2023.06-compat
arch x86_64/generic for repo eessi-2023.06-software
arch x86_64/intel/haswell for repo eessi-2021.12
arch x86_64/intel/haswell for repo eessi.org-2023.06-compat
arch x86_64/intel/haswell for repo eessi.io-2023.06-compat
arch x86_64/intel/haswell for repo eessi-2023.06-software
arch x86_64/intel/skylake_avx512 for repo eessi-2021.12
arch x86_64/intel/skylake_avx512 for repo eessi.org-2023.06-compat
arch x86_64/intel/skylake_avx512 for repo eessi.io-2023.06-compat
arch x86_64/intel/skylake_avx512 for repo eessi-2023.06-software
arch x86_64/amd/zen2 for repo eessi-2021.12
arch x86_64/amd/zen2 for repo eessi.org-2023.06-compat
arch x86_64/amd/zen2 for repo eessi.io-2023.06-compat
arch x86_64/amd/zen2 for repo eessi-2023.06-software
arch x86_64/amd/zen3 for repo eessi-2021.12
arch x86_64/amd/zen3 for repo eessi.org-2023.06-compat
arch x86_64/amd/zen3 for repo eessi.io-2023.06-compat
arch x86_64/amd/zen3 for repo eessi-2023.06-software
arch aarch64/generic for repo eessi-2021.12
arch aarch64/generic for repo eessi.org-2023.06-compat
arch aarch64/generic for repo eessi.io-2023.06-compat
arch aarch64/generic for repo eessi-2023.06-software
arch aarch64/neoverse_n1 for repo eessi-2021.12
arch aarch64/neoverse_n1 for repo eessi.org-2023.06-compat
arch aarch64/neoverse_n1 for repo eessi.io-2023.06-compat
arch aarch64/neoverse_n1 for repo eessi-2023.06-software
arch aarch64/neoverse_v1 for repo eessi-2021.12
arch aarch64/neoverse_v1 for repo eessi.org-2023.06-compat
arch aarch64/neoverse_v1 for repo eessi.io-2023.06-compat
arch aarch64/neoverse_v1 for repo eessi-2023.06-software

trz42 · 2023-09-10T06:33:08Z

bot: build arch:generic repo:eessi.io-2023.06-compat

eessi-bot · 2023-09-10T06:33:10Z

Updates by the bot instance eessi-bot-citc-aws (click for details)

received bot command build arch:generic repo:eessi.io-2023.06-compat from trz42
- expanded format: build architecture:generic repository:eessi.io-2023.06-compat
handling command build architecture:generic repository:eessi.io-2023.06-compat resulted in:
- submitted job 7302, for details & status see build compat layer software.eessi.io/2023.06 #191 (comment)
- submitted job 7303, for details & status see build compat layer software.eessi.io/2023.06 #191 (comment)

eessi-bot · 2023-09-10T06:33:29Z

New job on instance eessi-bot-citc-aws for architecture x86_64-generic for repository eessi.io-2023.06-compat in job dir /mnt/shared/home/bot/eessi-bot-software-layer/jobs/2023.09/pr_191/7302

date	job status	comment
Sep 10 06:33:28 UTC 2023	submitted	job id `7302` awaits release by job manager
Sep 10 06:33:31 UTC 2023	released	job awaits launch by Slurm scheduler
Sep 10 06:37:35 UTC 2023	running	job `7302` is running
Sep 10 11:31:33 UTC 2023	finished	😁 SUCCESS (click triangle for details) Details ✅ job output file `slurm-7302.out` ✅ no task failed ✅ found tarball Artefacts `eessi-2023.06-compat-linux-x86_64-1694345352.tar.gz` size: 1700 MiB (1783583498 bytes) entries: 190969

eessi-bot · 2023-09-10T06:33:30Z

New job on instance eessi-bot-citc-aws for architecture aarch64-generic for repository eessi.io-2023.06-compat in job dir /mnt/shared/home/bot/eessi-bot-software-layer/jobs/2023.09/pr_191/7303

date	job status	comment
Sep 10 06:33:29 UTC 2023	submitted	job id `7303` awaits release by job manager
Sep 10 06:34:33 UTC 2023	released	job awaits launch by Slurm scheduler
Sep 10 06:38:38 UTC 2023	running	job `7303` is running
Sep 10 11:23:24 UTC 2023	finished	😁 SUCCESS (click triangle for details) Details ✅ job output file `slurm-7303.out` ✅ no task failed ✅ found tarball Artefacts `eessi-2023.06-compat-linux-aarch64-1694344846.tar.gz` size: 1736 MiB (1820372418 bytes) entries: 190923

boegel · 2023-11-18T10:12:49Z

@trz42 @bedroge This can be merged too now right, since the compat layer has already been ingested (and even package updates have been installed, see #193)?

bedroge

@trz42 Just two small comments, but other than that I guess it's ready to be merged?

bedroge · 2023-11-20T10:51:30Z

bot/check-build.sh

+# eessi_os=linux
+job_version=$(cfg_get_value "repository" "repo_version")
+eessi_version=${job_version:-2023.09}
+# job_repo=$(cfg_get_value "repository" "repo_name")
+# eessi_repo=${job_repo:-pilot.nessi.no}
+# tar_topdir=/cvmfs/${eessi_repo}/versions


Can these commented lines be removed?

bedroge · 2023-11-20T10:52:20Z

bot/check-build.sh

+    summary=":cry: FAILURE"
+fi
+
+# TODO adjust format to what NESSI bot uses


Not sure what needs to happen here, but this mentions NESSI? Can it be removed, or is there still something that needs to be changed?

boegel · 2023-12-20T19:31:22Z

I prefer to have this merged, to avoid surprises going forward, since this is active in software.eessi.io/versions/2023.06/compat.

If any cleanup is still required, it can be done in a subsequent PR...

bypassing suggested changes

bedroge previously requested changes Nov 20, 2023

View reviewed changes

bedroge mentioned this pull request Nov 20, 2023

initial version of bot/check-result.sh script #179

Draft

boegel approved these changes Dec 20, 2023

View reviewed changes

boegel merged commit 10f1673 into EESSI:main Dec 20, 2023
1 check passed

boegel added the 2023.06-software.eessi.io 2023.06 version of software.eessi.io label Jan 20, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

build compat layer software.eessi.io/2023.06 #191

build compat layer software.eessi.io/2023.06 #191

trz42 commented Sep 9, 2023 •

edited

Loading

eessi-bot bot commented Sep 9, 2023

trz42 commented Sep 9, 2023

eessi-bot bot commented Sep 9, 2023 •

edited

Loading

eessi-bot bot commented Sep 9, 2023 •

edited

Loading

eessi-bot bot commented Sep 9, 2023 •

edited

Loading

trz42 commented Sep 10, 2023

eessi-bot bot commented Sep 10, 2023 •

edited

Loading

eessi-bot bot commented Sep 10, 2023

trz42 commented Sep 10, 2023

eessi-bot bot commented Sep 10, 2023 •

edited

Loading

eessi-bot bot commented Sep 10, 2023 •

edited

Loading

eessi-bot bot commented Sep 10, 2023 •

edited

Loading

boegel commented Nov 18, 2023

bedroge left a comment

bedroge Nov 20, 2023

bedroge Nov 20, 2023

boegel commented Dec 20, 2023

build compat layer software.eessi.io/2023.06 #191

build compat layer software.eessi.io/2023.06 #191

Conversation

trz42 commented Sep 9, 2023 • edited Loading

eessi-bot bot commented Sep 9, 2023

trz42 commented Sep 9, 2023

eessi-bot bot commented Sep 9, 2023 • edited Loading

eessi-bot bot commented Sep 9, 2023 • edited Loading

eessi-bot bot commented Sep 9, 2023 • edited Loading

trz42 commented Sep 10, 2023

eessi-bot bot commented Sep 10, 2023 • edited Loading

eessi-bot bot commented Sep 10, 2023

trz42 commented Sep 10, 2023

eessi-bot bot commented Sep 10, 2023 • edited Loading

eessi-bot bot commented Sep 10, 2023 • edited Loading

eessi-bot bot commented Sep 10, 2023 • edited Loading

boegel commented Nov 18, 2023

bedroge left a comment

Choose a reason for hiding this comment

bedroge Nov 20, 2023

Choose a reason for hiding this comment

bedroge Nov 20, 2023

Choose a reason for hiding this comment

boegel commented Dec 20, 2023

trz42 commented Sep 9, 2023 •

edited

Loading

eessi-bot bot commented Sep 9, 2023 •

edited

Loading

eessi-bot bot commented Sep 9, 2023 •

edited

Loading

eessi-bot bot commented Sep 9, 2023 •

edited

Loading

eessi-bot bot commented Sep 10, 2023 •

edited

Loading

eessi-bot bot commented Sep 10, 2023 •

edited

Loading

eessi-bot bot commented Sep 10, 2023 •

edited

Loading

eessi-bot bot commented Sep 10, 2023 •

edited

Loading