[gompi/2022.05] in-place update to netCDF v4.9.0 #15677

boegel · 2022-06-13T18:15:09Z

I intended to actually open #15676 with netCDF v4.9.0, but apparently I managed to screw that up 😅

(opened as draft PR, because compilation fails even though it worked fine with v4.8.1...)

boegel · 2022-06-13T18:19:45Z

@boegelbot please test @ generoso

boegelbot · 2022-06-13T18:20:09Z

@boegel: Request for testing this PR well received on login1

PR test command 'EB_PR=15677 EB_ARGS= /opt/software/slurm/bin/sbatch --job-name test_PR_15677 --ntasks=4 ~/boegelbot/eb_from_pr_upload_generoso.sh' executed!

exit code: 0
output:

Submitted batch job 8707

Test results coming soon (I hope)...

- notification for comment with ID 1154235429 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

boegelbot · 2022-06-13T18:22:04Z

Test report by @boegelbot
FAILED
Build succeeded for 0 out of 1 (1 easyconfigs in total)
cns1 - Linux Rocky Linux 8.5, x86_64, Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz (haswell), Python 3.6.8
See https://gist.github.com/d6033fc36e12f40370953eee16f8293f for a full test report.

jfgrimm · 2022-06-16T14:58:36Z

/project/boegelbot/Rocky8/haswell/software/binutils/2.38-GCCcore-11.3.0/bin/ld: ../liblib/libnetcdf.a(ncjson.c.o): in function `NCJreclaim':
ncjson.c:(.text+0xa50): multiple definition of `NCJreclaim'; CMakeFiles/nczhdf5filters.dir/NCZhdf5filters.c.o:NCZhdf5filters.c:(.text+0xac0): first defined here
/project/boegelbot/Rocky8/haswell/software/binutils/2.38-GCCcore-11.3.0/bin/ld: ../liblib/libnetcdf.a(ncjson.c.o): in function `NCJnew':
ncjson.c:(.text+0xb10): multiple definition of `NCJnew'; CMakeFiles/nczhdf5filters.dir/NCZhdf5filters.c.o:NCZhdf5filters.c:(.text+0xb80): first defined here
/project/boegelbot/Rocky8/haswell/software/binutils/2.38-GCCcore-11.3.0/bin/ld: ../liblib/libnetcdf.a(ncjson.c.o): in function `NCJnewstringn':
ncjson.c:(.text+0xb60): multiple definition of `NCJnewstringn'; CMakeFiles/nczhdf5filters.dir/NCZhdf5filters.c.o:NCZhdf5filters.c:(.text+0xbd0): first defined here
/project/boegelbot/Rocky8/haswell/software/binutils/2.38-GCCcore-11.3.0/bin/ld: ../liblib/libnetcdf.a(ncjson.c.o): in function `NCJnewstring':
ncjson.c:(.text+0xc20): multiple definition of `NCJnewstring'; CMakeFiles/nczhdf5filters.dir/NCZhdf5filters.c.o:NCZhdf5filters.c:(.text+0xc90): first defined here
/project/boegelbot/Rocky8/haswell/software/binutils/2.38-GCCcore-11.3.0/bin/ld: ../liblib/libnetcdf.a(ncjson.c.o): in function `NCJparse':
ncjson.c:(.text+0x1160): multiple definition of `NCJparse'; CMakeFiles/nczhdf5filters.dir/NCZhdf5filters.c.o:NCZhdf5filters.c:(.text+0x1080): first defined here
/project/boegelbot/Rocky8/haswell/software/binutils/2.38-GCCcore-11.3.0/bin/ld: ../liblib/libnetcdf.a(ncjson.c.o): in function `NCJdictget':
ncjson.c:(.text+0x1190): multiple definition of `NCJdictget'; CMakeFiles/nczhdf5filters.dir/NCZhdf5filters.c.o:NCZhdf5filters.c:(.text+0x11a0): first defined here
/project/boegelbot/Rocky8/haswell/software/binutils/2.38-GCCcore-11.3.0/bin/ld: ../liblib/libnetcdf.a(ncjson.c.o): in function `NCJcvt':
ncjson.c:(.text+0x1220): multiple definition of `NCJcvt'; CMakeFiles/nczhdf5filters.dir/NCZhdf5filters.c.o:NCZhdf5filters.c:(.text+0x1230): first defined here

branfosj · 2022-06-21T20:31:44Z

It is not the cause of the build failure we are seeing, but we must add https://patch-diff.githubusercontent.com/raw/Unidata/netcdf-c/pull/2412.patch in to get the correct library name.

branfosj · 2022-06-21T20:57:21Z

The:

# HDF5 version detection is missed in netCDF 4.8.1 when HDF5_C_LIBRARY, HDF5_INCLUDE_DIR, and HDF5_HL_LIBRARY are set
local_hdf5_version_fix = '-DHDF5_VERSION=$EBVERSIONHDF5'

is no longer needed.

Also, this issue is in building the static libs (-DBUILD_SHARED_LIBS=OFF). The failure does not happen when building the shared libs (-DBUILD_SHARED_LIBS=ON).

…ERSION configure option which is no longer needed

boegel · 2022-06-22T12:47:53Z

I've opened an issue upstream for this: Unidata/netcdf-c#2419

zao · 2022-06-22T17:34:08Z

I commented with a bit of analysis in the upstream issue, a workaround for us may be to patch netcdf_json.h to have the function declarations and definitions static to prevent them from having linkage. Haven't tested this outside of a standalone repro case.

…e via 'make test'

boegel · 2022-06-23T08:30:54Z

@boegelbot please test @ generoso

boegelbot · 2022-06-23T08:35:09Z

@boegel: Request for testing this PR well received on login1

PR test command 'EB_PR=15677 EB_ARGS= /opt/software/slurm/bin/sbatch --job-name test_PR_15677 --ntasks=4 ~/boegelbot/eb_from_pr_upload_generoso.sh' executed!

exit code: 0
output:

Submitted batch job 8762

Test results coming soon (I hope)...

- notification for comment with ID 1164114153 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

boegel · 2022-06-23T08:46:49Z

Test report by @boegel
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in total)
node3138.skitty.os - Linux RHEL 8.4, x86_64, Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz (skylake_avx512), Python 3.6.8
See https://gist.github.com/e76f04add8a812e808e59da149a0d69f for a full test report.

boegelbot · 2022-06-23T08:47:19Z

Test report by @boegelbot
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in total)
cns1 - Linux Rocky Linux 8.5, x86_64, Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz (haswell), Python 3.6.8
See https://gist.github.com/956d3a95dba88d789168a4e89aaa4de8 for a full test report.

branfosj · 2022-06-23T09:14:05Z

Test report by @branfosj
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in total)
bear-pg0105u36b.bear.cluster - Linux RHEL 8.6, x86_64, Intel(R) Xeon(R) Platinum 8360Y CPU @ 2.40GHz (icelake), Python 3.6.8
See https://gist.github.com/d35ddebedb4d347ed05e0e7bd614a719 for a full test report.

branfosj · 2022-06-23T09:19:23Z

Test report by @branfosj
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in total)
bear-pg0211u03a.bear.cluster - Linux RHEL 8.6, x86_64, Intel(R) Xeon(R) Gold 6248 CPU @ 2.50GHz (cascadelake), Python 3.6.8
See https://gist.github.com/5069393d6e047eca078099ad4e25c203 for a full test report.

branfosj · 2022-06-23T09:20:35Z

@boegelbot please test @ jsc-zen2

boegelbot · 2022-06-23T09:22:08Z

@branfosj: Request for testing this PR well received on jsczen2l1.int.jsc-zen2.easybuild-test.cluster

PR test command 'EB_PR=15677 EB_ARGS= /opt/software/slurm/bin/sbatch --job-name test_PR_15677 --ntasks=8 ~/boegelbot/eb_from_pr_upload_jsc-zen2.sh' executed!

exit code: 0
output:

Submitted batch job 1326

Test results coming soon (I hope)...

- notification for comment with ID 1164165503 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

branfosj · 2022-06-23T09:23:22Z

Test report by @branfosj
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in total)
bear-pg0211u03a.bear.cluster - Linux Ubuntu 20.04.2 LTS (Focal Fossa), x86_64, Intel(R) Xeon(R) Gold 6248 CPU @ 2.50GHz (cascadelake), Python 3.8.5
See https://gist.github.com/4ad92b4dfecf8fc97272159ae476e08c for a full test report.

boegelbot · 2022-06-23T09:32:34Z

Test report by @boegelbot
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in total)
jsczen2c1.int.jsc-zen2.easybuild-test.cluster - Linux Rocky Linux 8.5, x86_64, AMD EPYC 7742 64-Core Processor (zen2), Python 3.6.8
See https://gist.github.com/62200f598fc4a7a8f1f9a8d7c7d92d15 for a full test report.

branfosj

lgtm

branfosj · 2022-06-23T09:36:37Z

Going in, thanks @boegel!

boegel · 2022-06-23T09:39:50Z

Test report by @boegel
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in total)
node3601.doduo.os - Linux RHEL 8.4, x86_64, AMD EPYC 7552 48-Core Processor (zen2), Python 3.6.8
See https://gist.github.com/89f8e5a4dc87489ded04da5a791de61b for a full test report.

boegel · 2022-06-23T09:59:28Z

Test report by @boegel
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in total)
fair-mastodon-c6g-2xlarge-0001 - Linux Rocky Linux 8.5, AArch64, ARM UNKNOWN (graviton2), Python 3.6.8
See https://gist.github.com/338898ca1d25185e90bd36c9cd895a45 for a full test report.

[gompi/2022.05] in-place update to netCDF v4.9.0

d0e8259

boegel added update change labels Jun 13, 2022

boegel added this to the next release (4.5.6?) milestone Jun 13, 2022

boegel added 2 commits June 22, 2022 13:42

add patch to fix library name for netCDF v4.9.0 + get rid of -DHDF5_V…

08cc268

…ERSION configure option which is no longer needed

add missing zstd dependency for netCDF v4.9.0

a33ec53

branfosj mentioned this pull request Jun 22, 2022

{data}[foss/2022.05] GDAL v3.5.0 w/ Python 3.10.4 #15746

Merged

1 task

add patch file to fix linking error for netCDF v4.9.0 + run test suit…

001aa3e

…e via 'make test'

boegel marked this pull request as ready for review June 23, 2022 08:30

branfosj approved these changes Jun 23, 2022

View reviewed changes

branfosj merged commit fc56c2b into easybuilders:develop Jun 23, 2022

boegel deleted the 20220613164029_new_pr_netCDF481 branch June 23, 2022 10:04

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[gompi/2022.05] in-place update to netCDF v4.9.0 #15677

[gompi/2022.05] in-place update to netCDF v4.9.0 #15677

boegel commented Jun 13, 2022

boegel commented Jun 13, 2022

boegelbot commented Jun 13, 2022

boegelbot commented Jun 13, 2022

jfgrimm commented Jun 16, 2022

branfosj commented Jun 21, 2022

branfosj commented Jun 21, 2022

boegel commented Jun 22, 2022

zao commented Jun 22, 2022

boegel commented Jun 23, 2022

boegelbot commented Jun 23, 2022

boegel commented Jun 23, 2022

boegelbot commented Jun 23, 2022

branfosj commented Jun 23, 2022

branfosj commented Jun 23, 2022

branfosj commented Jun 23, 2022

boegelbot commented Jun 23, 2022

branfosj commented Jun 23, 2022

boegelbot commented Jun 23, 2022

branfosj left a comment

branfosj commented Jun 23, 2022

boegel commented Jun 23, 2022

boegel commented Jun 23, 2022

[gompi/2022.05] in-place update to netCDF v4.9.0 #15677

[gompi/2022.05] in-place update to netCDF v4.9.0 #15677

Conversation

boegel commented Jun 13, 2022

boegel commented Jun 13, 2022

boegelbot commented Jun 13, 2022

boegelbot commented Jun 13, 2022

jfgrimm commented Jun 16, 2022

branfosj commented Jun 21, 2022

branfosj commented Jun 21, 2022

boegel commented Jun 22, 2022

zao commented Jun 22, 2022

boegel commented Jun 23, 2022

boegelbot commented Jun 23, 2022

boegel commented Jun 23, 2022

boegelbot commented Jun 23, 2022

branfosj commented Jun 23, 2022

branfosj commented Jun 23, 2022

branfosj commented Jun 23, 2022

boegelbot commented Jun 23, 2022

branfosj commented Jun 23, 2022

boegelbot commented Jun 23, 2022

branfosj left a comment

Choose a reason for hiding this comment

branfosj commented Jun 23, 2022

boegel commented Jun 23, 2022

boegel commented Jun 23, 2022