k8: add linux-aarch64 build #46559

martin-g · 2024-03-19T08:48:27Z

Describe your pull request here

Please read the guidelines for Bioconda recipes before opening a pull request (PR).

General instructions

If this PR adds or updates a recipe, use "Add" or "Update" appropriately as the first word in its title.
New recipes not directly relevant to the biological sciences need to be submitted to the conda-forge channel instead of Bioconda.
PRs require reviews prior to being merged. Once your PR is passing tests and ready to be merged, please issue the @BiocondaBot please add label command.
Please post questions on Gitter or ping @bioconda/core in a comment.

Instructions for avoiding API, ABI, and CLI breakage issues

Conda is able to record and lock (a.k.a. pin) dependency versions used at build time of other recipes.
This way, one can avoid that expectations of a downstream recipe with regards to API, ABI, or CLI are violated by later changes in the recipe.
If not already present in the meta.yaml, make sure to specify run_exports (see here for the rationale and comprehensive explanation).
Add a run_exports section like this:

build:
  run_exports:
    - ...

with ... being one of:

Case	run_exports statement
semantic versioning	`{{ pin_subpackage("myrecipe", max_pin="x") }}`
semantic versioning (0.x.x)	`{{ pin_subpackage("myrecipe", max_pin="x.x") }}`
known breakage in minor versions	`{{ pin_subpackage("myrecipe", max_pin="x.x") }}` (in such a case, please add a note that shortly mentions your evidence for that)
known breakage in patch versions	`{{ pin_subpackage("myrecipe", max_pin="x.x.x") }}` (in such a case, please add a note that shortly mentions your evidence for that)
calendar versioning	`{{ pin_subpackage("myrecipe", max_pin=None) }}`

while replacing "myrecipe" with either name if a name|lower variable is defined in your recipe or with the lowercase name of the package in quotes.

Bot commands for PR management

Please use the following BiocondaBot commands:

Everyone has access to the following BiocondaBot commands, which can be given in a comment:

`@BiocondaBot please update`	Merge the master branch into a PR.
`@BiocondaBot please add label`	Add the `please review & merge` label.
`@BiocondaBot please fetch artifacts`	Post links to CI-built packages/containers. You can use this to test packages locally.

Note that the @BiocondaBot please merge command is now depreciated. Please just squash and merge instead.

Also, the bot watches for comments from non-members that include @bioconda/<team> and will automatically re-post them to notify the addressed <team>.

martin-g · 2024-03-20T09:55:04Z

The build on Linux ARM64 has passed !
The Linux x86_64 build fails with

07:35:31 BIOCONDA INFO (OUT) ../src/node_credentials.cc:16:10: fatal error: sys/auxv.h: No such file or directory
07:35:31 BIOCONDA INFO (OUT)    16 | #include <sys/auxv.h>

This header file should come from glibc-devel, and I think I use 2.17 now (https://github.com/bioconda/bioconda-recipes/pull/46559/files#diff-cbe8b71c6a5405b6548f57a5dff509864239159b58be047a4d535b58ea5b27eb) as the Linux ARM64 image (see #46186 (comment) for more info).
Any ideas how to deal with this error ?

martin-g · 2024-03-20T14:00:19Z

@mfansler Apologies for pinging you personally!
Could you please take a look at the issue with the Linux x86_64 build ? I found several other similar issues where you explained the problem but none of them worked for me here.
Thanks in advance!

mfansler · 2024-03-21T17:44:45Z

@martin-g no worries about the ping! Unfortunately, nothing I immediately recognize to suggest to resolve this. I'd need to inspect some local builds to debug, and with my workload right now I likely can't dig in until next week. I'll try to swing back when I get some free time and see it isn't yet resolved.

martin-g · 2024-03-22T14:35:09Z

Magic! The build passed today on both x86_64 and aarch64 !

martin-g · 2024-03-22T14:35:13Z

@BiocondaBot please fetch artifacts

BiocondaBot · 2024-03-22T14:35:32Z

Azure

Package(s) built on Azure are ready for inspection:

Arch	Package	Zip File
linux-64	k8-1.0-hdcf5f25_0.tar.bz2	LinuxArtifacts
osx-64	k8-1.0-hdf58011_0.tar.bz2	OSXArtifacts

You may also use conda to install these after downloading and extracting the appropriate zip file. From the LinuxArtifacts or OSXArtifacts directories:

conda install -c ./packages <package name>

Docker image(s) built (images for Azure are in the LinuxArtifacts zip file above):

Package	Tag	Install with `docker`
k8	1.0--hdcf5f25_0	show `gzip -dc LinuxArtifacts/images/k8:1.0--hdcf5f25_0.tar.gz \| docker load`

CircleCI

Package(s) built on CircleCI are ready for inspection:

Arch	Package	Repodata
linux-aarch64	k8-1.0-h7f4e536_0.tar.bz2	repodata.json

You may also use conda to install these:

conda install -c https://output.circle-artifacts.com/output/job/b8793ed1-503c-4a72-baee-c791380b70c0/artifacts/0/tmp/artifacts/packages <package name>

martin-g · 2024-03-22T14:37:10Z


mgrigorov in 🌐 euler-arm-22 in /tmp/k8 took 8s 
❯ tar jxvf k8-1.0-h7f4e536_0.tar.bz2 
info/files
info/run_exports.json
info/test/run_test.sh
info/hash_input.json
info/paths.json
info/index.json
info/recipe/nodejs-x86_64.patch
info/recipe/meta.yaml.template
info/licenses/LICENSE.txt
info/recipe/build.sh
info/recipe/meta.yaml
info/recipe/conda_build_config.yaml
info/about.json
info/git
bin/k8

mgrigorov in 🌐 euler-arm-22 in /tmp/k8 
❯ file bin/*
bin/k8: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (GNU/Linux), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, for GNU/Linux 3.7.0, with debug_info, not stripped

mgrigorov in 🌐 euler-arm-22 in /tmp/k8 
❯ ./bin/k8
Usage: k8 [options] <script.js> [arguments]
Options:
  -e STR      execute STR
  -E STR      execute STR and print results
  -v          print version number

LGTM!

martin-g · 2024-03-22T14:37:29Z

@BiocondaBot please add label

mfansler · 2024-03-22T14:57:23Z

@martin-g good to hear!

BTW, on Conda Forge in the near future declaring sysroot will be replaced by - {{ stdlib("c") }} (see forthcoming announcement).

martin-g · 2024-03-23T07:04:33Z

.circleci/config.yml

@@ -9,7 +9,7 @@ executors:
  linux-aarch64:
    machine:
      image: ubuntu-2204:current
-    resource_class: arm.medium
+    resource_class: arm.large


I needed to switch to a large VM because the build of Node.js takes more than one hour on medium and the job is killed.
If Bioconda team thinks that it should stay medium then I'd suggest to merge this PR with large so that the recipe is published/released and then revert to medium in a follow-up PR. And also add a build instructions to k8 recipe that it needs large for a release.

In the meantime the Azure builds started failing:

12:47:04 �[32mBIOCONDA INFO�[0m (OUT) $BUILD_PREFIX/bin/x86_64-conda-linux-gnu-c++ -o $SRC_DIR/node-v18.19.1/out/Release/obj.target/v8_initializers/gen/torque-generated/third_party/v8/builtins/array-sort-tq-csa.o $SRC_DIR/node-v18.19.1/out/Release/obj/gen/torque-generated/third_party/v8/builtins/array-sort-tq-csa.cc '-D_GLIBCXX_USE_CXX11_ABI=1' '-DNODE_OPENSSL_CONF_NAME=nodejs_conf' '-DNODE_OPENSSL_HAS_QUIC' '-DICU_NO_USER_DATA_OVERRIDE' '-DV8_GYP_BUILD' '-DV8_TYPED_ARRAY_MAX_SIZE_IN_HEAP=64' '-D__STDC_FORMAT_MACROS' '-DOPENSSL_NO_PINSHARED' '-DOPENSSL_THREADS' '-DV8_TARGET_ARCH_X64' '-DV8_HAVE_TARGET_OS' '-DV8_TARGET_OS_LINUX' '-DV8_EMBEDDER_STRING="-node.28"' '-DENABLE_DISASSEMBLER' '-DV8_PROMISE_INTERNAL_FIELD_COUNT=1' '-DV8_ENABLE_PRIVATE_MAPPING_FORK_OPTIMIZATION' '-DV8_SHORT_BUILTIN_CALLS' '-DOBJECT_PRINT' '-DV8_INTL_SUPPORT' '-DV8_ATOMIC_OBJECT_FIELD_WRITES' '-DV8_ENABLE_LAZY_SOURCE_POSITIONS' '-DV8_USE_SIPHASH' '-DV8_SHARED_RO_HEAP' '-DV8_WIN64_UNWINDING_INFO' '-DV8_ENABLE_REGEXP_INTERPRETER_THREADE... ##[warning]Free memory is lower than 5%; Currently used: 95.51% ##[warning]Free memory is lower than 5%; Currently used: 95.51%

They were passing last week but apparently the build of Node.js from sources is too heavy for the CI agents...

Signed-off-by: Martin Tzvetanov Grigorov <[email protected]>

It should solve the issue with the missing <sys/auxv.h> on Linux x86_64 Signed-off-by: Martin Tzvetanov Grigorov <[email protected]>

martin-g · 2024-05-07T07:01:15Z

Yey! The build is green on all platforms! Thank you, @mencian !

martin-g · 2024-05-07T07:01:28Z

@BiocondaBot please fetch artifacts

BiocondaBot · 2024-05-07T07:01:58Z

Package(s) built are ready for inspection:

Arch	Package	Zip File / Repodata	CI	Instructions
linux-64	k8-1.0-h6618dc6_0.tar.bz2	LinuxArtifacts.zip	Azure	show You may also use `conda` to install after downloading and extracting the zip file. From the LinuxArtifacts directory: `conda install -c ./packages <package name>`
osx-64	k8-1.0-hdf58011_0.tar.bz2	OSXArtifacts.zip	Azure	show You may also use `conda` to install after downloading and extracting the zip file. From the OSXArtifacts directory: `conda install -c ./packages <package name>`
linux-aarch64	k8-1.0-h7f4e536_0.tar.bz2	repodata.json	CircleCI	show You may also use `conda` to install:`conda install -c https://output.circle-artifacts.com/output/job/00a1f95e-7cd9-4a22-9396-89affec8e443/artifacts/0/tmp/artifacts/packages <package name>`

Docker image(s) built:

Package	Tag	CI	Install with `docker`
k8	1.0--h6618dc6_0	Azure	show Images for Azure are in the LinuxArtifacts zip file above.`gzip -dc LinuxArtifacts/images/k8:1.0--h6618dc6_0.tar.gz \| docker load`

martin-g · 2024-05-07T07:02:43Z

mgrigorov in 🌐 euler-arm-22 in /tmp/k8 via 🅒 test123 took 7s 
❯ tar xvf k8-1.0-h7f4e536_0.tar.bz2                                                                                                                                                                        (test123) 
info/files
info/run_exports.json
info/test/run_test.sh
info/hash_input.json
info/paths.json
info/index.json
info/recipe/nodejs-x86_64.patch
info/recipe/meta.yaml.template
info/licenses/LICENSE.txt
info/recipe/build.sh
info/recipe/meta.yaml
info/recipe/conda_build_config.yaml
info/about.json
info/git
bin/k8

mgrigorov in 🌐 euler-arm-22 in /tmp/k8 via 🅒 test123 
❯ file bin/*                                                                                                                                                                                               (test123) 
bin/k8: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (GNU/Linux), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, for GNU/Linux 3.7.0, with debug_info, not stripped

mgrigorov in 🌐 euler-arm-22 in /tmp/k8 via 🅒 test123 
❯ ./bin/k8                                                                                                                                                                                                 (test123) 
Usage: k8 [options] <script.js> [arguments]
Options:
  -e STR      execute STR
  -E STR      execute STR and print results
  -v          print version number

LGTM!

martin-g · 2024-05-07T07:04:32Z

Linux x86_64 looks good too:

/tmp/k8 took 4s 
❯ unzip content\?format=zip 
Archive:  content?format=zip
  inflating: LinuxArtifacts/images/k8:1.0--h6618dc6_0.tar.gz  
  inflating: LinuxArtifacts/packages/channeldata.json  
  inflating: LinuxArtifacts/packages/index.html  
  inflating: LinuxArtifacts/packages/linux-64/current_repodata.json  
  inflating: LinuxArtifacts/packages/linux-64/current_repodata.json.bz2  
  inflating: LinuxArtifacts/packages/linux-64/index.html  
  inflating: LinuxArtifacts/packages/linux-64/k8-1.0-h6618dc6_0.tar.bz2  
  inflating: LinuxArtifacts/packages/linux-64/repodata.json  
  inflating: LinuxArtifacts/packages/linux-64/repodata.json.bz2  
  inflating: LinuxArtifacts/packages/linux-64/repodata_from_packages.json  
  inflating: LinuxArtifacts/packages/linux-64/repodata_from_packages.json.bz2  
  inflating: LinuxArtifacts/packages/noarch/current_repodata.json  
  inflating: LinuxArtifacts/packages/noarch/current_repodata.json.bz2  
  inflating: LinuxArtifacts/packages/noarch/index.html  
  inflating: LinuxArtifacts/packages/noarch/repodata.json  
  inflating: LinuxArtifacts/packages/noarch/repodata.json.bz2  
  inflating: LinuxArtifacts/packages/noarch/repodata_from_packages.json  
  inflating: LinuxArtifacts/packages/noarch/repodata_from_packages.json.bz2  

/tmp/k8 
❯ tar xvf LinuxArtifacts/packages/linux-64/k8-1.0-h6618dc6_0.tar.bz2
info/files
info/run_exports.json
info/test/run_test.sh
info/hash_input.json
info/paths.json
info/index.json
info/recipe/nodejs-x86_64.patch
info/recipe/meta.yaml.template
info/licenses/LICENSE.txt
info/recipe/build.sh
info/recipe/meta.yaml
info/recipe/conda_build_config.yaml
info/about.json
info/git
bin/k8

/tmp/k8 
❯ file bin/k8 
bin/k8: ELF 64-bit LSB pie executable, x86-64, version 1 (GNU/Linux), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, with debug_info, not stripped

/tmp/k8 
❯ ./bin/k8 
Usage: k8 [options] <script.js> [arguments]
Options:
  -e STR      execute STR
  -E STR      execute STR and print results
  -v          print version number

martin-g · 2024-05-07T07:04:42Z

@BiocondaBot please add label

* k8: add linux-aarch64 build Signed-off-by: Martin Tzvetanov Grigorov <[email protected]> * Try to build k8 from sources Signed-off-by: Martin Tzvetanov Grigorov <[email protected]> * Set CXXFLAGS * Add dependency to sysroot_linux-64 for a newer glibc It should solve the issue with the missing <sys/auxv.h> on Linux x86_64 Signed-off-by: Martin Tzvetanov Grigorov <[email protected]> * clean up recipe * clean up recipe * edit meta.yaml * add gcc * edit meta.yaml --------- Signed-off-by: Martin Tzvetanov Grigorov <[email protected]> Co-authored-by: Joshua Zhuang <[email protected]> Co-authored-by: mencian <[email protected]>

martin-g force-pushed the aarch64-k8 branch 8 times, most recently from e956c13 to f95cc0b Compare March 19, 2024 13:53

martin-g mentioned this pull request Mar 19, 2024

add k8-1.0: enable linux-aarch64 build #46387

Closed

martin-g force-pushed the aarch64-k8 branch 3 times, most recently from e1bfc4a to 256f5de Compare March 20, 2024 07:10

martin-g force-pushed the aarch64-k8 branch 5 times, most recently from 807b3a4 to 87cd7fd Compare March 20, 2024 13:35

rpetit3 added the aarch64 Related to adding linux-aarch64 support label Mar 20, 2024

martin-g force-pushed the aarch64-k8 branch from 87cd7fd to 11b25e0 Compare March 22, 2024 11:09

BiocondaBot added the please review & merge set to ask for merge label Mar 22, 2024

martin-g commented Mar 23, 2024

View reviewed changes

martin-g mentioned this pull request Mar 27, 2024

medaka: add linux-aarch64 build #46801

Closed

martin-g force-pushed the aarch64-k8 branch from f5406e0 to 4d65c21 Compare April 3, 2024 08:39

martin-g and others added 4 commits April 4, 2024 13:55

k8: add linux-aarch64 build

513e3f3

Signed-off-by: Martin Tzvetanov Grigorov <[email protected]>

Try to build k8 from sources

a3e3f9a

Signed-off-by: Martin Tzvetanov Grigorov <[email protected]>

Set CXXFLAGS

d38995a

Add dependency to sysroot_linux-64 for a newer glibc

90b7a52

It should solve the issue with the missing <sys/auxv.h> on Linux x86_64 Signed-off-by: Martin Tzvetanov Grigorov <[email protected]>

martin-g force-pushed the aarch64-k8 branch from 2aa30a6 to 90b7a52 Compare April 4, 2024 10:55

mencian and others added 9 commits April 9, 2024 22:54

Merge branch 'master' into aarch64-k8

18e4c2d

Merge branch 'master' into aarch64-k8

abf4fb6

Merge branch 'master' into aarch64-k8

01e760a

clean up recipe

ba9a239

clean up recipe

d860bd3

edit meta.yaml

320f8d1

add gcc

5431749

edit meta.yaml

b82c15b

Merge branch 'master' into aarch64-k8

fbc3fde

mencian approved these changes May 7, 2024

View reviewed changes

mencian merged commit 1c5a88a into bioconda:master May 7, 2024
6 checks passed

martin-g deleted the aarch64-k8 branch May 8, 2024 06:43

peterjc mentioned this pull request Aug 26, 2024

Build k8 on osx-arm64 #49946

Merged

peterjc mentioned this pull request Sep 24, 2024

minimap2 with tiledbvcf #50888

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

k8: add linux-aarch64 build #46559

k8: add linux-aarch64 build #46559

martin-g commented Mar 19, 2024

martin-g commented Mar 20, 2024

martin-g commented Mar 20, 2024

mfansler commented Mar 21, 2024

martin-g commented Mar 22, 2024

martin-g commented Mar 22, 2024

BiocondaBot commented Mar 22, 2024

martin-g commented Mar 22, 2024

martin-g commented Mar 22, 2024

mfansler commented Mar 22, 2024

martin-g Mar 23, 2024

martin-g Apr 4, 2024

martin-g commented May 7, 2024

martin-g commented May 7, 2024

BiocondaBot commented May 7, 2024

martin-g commented May 7, 2024

martin-g commented May 7, 2024

martin-g commented May 7, 2024

k8: add linux-aarch64 build #46559

k8: add linux-aarch64 build #46559

Conversation

martin-g commented Mar 19, 2024

General instructions

Instructions for avoiding API, ABI, and CLI breakage issues

Bot commands for PR management

martin-g commented Mar 20, 2024

martin-g commented Mar 20, 2024

mfansler commented Mar 21, 2024

martin-g commented Mar 22, 2024

martin-g commented Mar 22, 2024

BiocondaBot commented Mar 22, 2024

Azure

CircleCI

martin-g commented Mar 22, 2024

martin-g commented Mar 22, 2024

mfansler commented Mar 22, 2024

martin-g Mar 23, 2024

Choose a reason for hiding this comment

martin-g Apr 4, 2024

Choose a reason for hiding this comment

martin-g commented May 7, 2024

martin-g commented May 7, 2024

BiocondaBot commented May 7, 2024

martin-g commented May 7, 2024

martin-g commented May 7, 2024

martin-g commented May 7, 2024