build: use DPDK_MACHINE=haswell when testing dpdk build on github-hosted runner #2243
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tchaikov
force-pushed
the
dpdk-build-avx10
branch
5 times, most recently
from
06eed8e to
b029875
Compare
we dropped the translation in 369d166, when upgrading to DPDK v23.07, as newer DPDK recognizes these names respected by `-march`, so there is no point to preserve this translation in `configure.py` anymore. in this change, the helper of `infer_dpdk_machine()` along with its caller are dropped. Signed-off-by: Kefu Chai <[email protected]>
this allows user and our toolings to specify the target architecture when building Seastar. in particular, it prepares us to avoid using the default "-march=native" setting when building Seastar with DPDK enabled, which could fail the build when building with a compiler supporting AVX512, while the building host does not. Signed-off-by: Kefu Chai <[email protected]>
tchaikov
force-pushed
the
dpdk-build-avx10
branch
2 times, most recently
from
d6512ed to
98b4090
Compare
…ted runner
when building dpdk on a building host which has AVX10.1 ISA support,
with Clang 18, we have following failure
```
/usr/lib/llvm-18/bin/clang -Ilib/net/libnet_crc_avx512_lib.a.p -Ilib/net -I../../../../../../dpdk/lib/net -I. -I../../../../../../dpdk -Iconfig -I../../../../../../dpdk/config -Ilib/eal/include -I../../../../../../dpdk/lib/eal/include -Ilib/eal/linux/include -I../../../../../../dpdk/lib/eal/linux/include -Ilib/eal/x86/include -I../../../../../../dpdk/lib/eal/x86/include -Ilib/eal/common -I../../../../../../dpdk/lib/eal/common -Ilib/eal -I../../../../../../dpdk/lib/eal -Ilib/kvargs -I../../../../../../dpdk/lib/kvargs -Ilib/metrics -I../../../../../../dpdk/lib/metrics -Ilib/telemetry -I../../../../../../dpdk/lib/telemetry -fcolor-diagnostics -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -Wextra -O3 -include rte_config.h -Wcast-qual -Wdeprecated -Wformat -Wformat-nonliteral -Wformat-security -Wmissing-declarations -Wmissing-prototypes -Wnested-externs -Wold-style-definition -Wpointer-arith -Wsign-compare -Wstrict-prototypes -Wundef -Wwrite-strings -Wno-address-of-packed-member -Wno-missing-field-initializers -D_GNU_SOURCE -Wno-error -fPIC -march=native -DALLOW_EXPERIMENTAL_API -DALLOW_INTERNAL_API -Wno-format-truncation -DCC_X86_64_SSE42_PCLMULQDQ_SUPPORT -DCC_X86_64_AVX512_VPCLMULQDQ_SUPPORT -mavx512f -mavx512bw -mavx512dq -mavx512vl -mvpclmulqdq -mavx2 -mavx -MD -MQ lib/net/libnet_crc_avx512_lib.a.p/net_crc_avx512.c.o -MF lib/net/libnet_crc_avx512_lib.a.p/net_crc_avx512.c.o.d -o lib/net/libnet_crc_avx512_lib.a.p/net_crc_avx512.c.o -c ../../../../../../dpdk/lib/net/net_crc_avx512.c
Error: ../../../../../../dpdk/lib/net/net_crc_avx512.c:324:22: error: always_inline function '_mm512_broadcast_i32x4' requires target feature 'evex512', but would be inlined into function 'crc32_load_init_constants' that is compiled without support for 'evex512'
324 | crc32_eth.rk1_rk2 = _mm512_broadcast_i32x4(a);
| ^
```
according to https://gcc.gnu.org/onlinedocs/gcc/x86-Options.html
and
https://github.com/llvm/llvm-project/blob/release/18.x/clang/docs/UsersManual.rst#x86,
we should either use `-mevex512` or use `-mavx10.1-512` for accessing
these new vectorized instructions provided by AVX10.1. but we also
pass "-march=native" to the compiler. it turns out that at least some
github-hosted runners do not support AVX10.1 ISA, but the Clang 18
compiler does. so this combination breaks the build, as dpdk tries
to check if compiler supports AVX512, and if it does, dpdk builds
the AVX512 C files with this unsupported combination. that's why we
have build failures recently.
to address this issue, instead of using "native" as the value of
`Seastar_DPDK_MACHINE`, let's use a more conservative but still
capable architecture supported by github-hosted runner: "haswell",
which supports the instruction set that is used by quite a few
DPDK optimizations. so we can still have a decent coverage for
building with DPDK, and for testing the build. please note,
we don't test Seastar with the DPDK backend yet.
to enable the workflow to pass extra options to `configure.py`,
we change the "cooks" parameter in the matrix to "options",
and append "--dpdk-machine haswell" to it, so that we use
this option only when testing the dpdk build.
Fixes scylladb#2242
Signed-off-by: Kefu Chai <[email protected]>
tchaikov
force-pushed
the
dpdk-build-avx10
branch
from
98b4090 to
ba395a2
Compare
tchaikov
changed the title
This comment was marked as outdated.
This comment was marked as outdated.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
when building dpdk on a building host which has AVX10.1 ISA support,
with Clang 18, we have following failure
according to https://gcc.gnu.org/onlinedocs/gcc/x86-Options.html
and
https://github.com/llvm/llvm-project/blob/release/18.x/clang/docs/UsersManual.rst#x86,
we should either use
-mevex512or use-mavx10.1-512for accessing these new vectorized instructions provided by AVX10.1.we also pass
-march=nativeto the compiler. and it turns out that at least some github-hosted runners do not support AVX10.1 ISA, but the Clang 18 compiler does. so this combination breaks the build, as dpdk tries to check if compiler supports AVX512, and if it does, dpdk builds the AVX512 C files with this unsupported combination. that's why wehave build failures recently.
to address this issue, instead of using "native" as the value of
Seastar_DPDK_MACHINE, let's use a more conservative but still capable architecture supported by github-hosted runner: "haswell", which supports the instruction set that is used by quite a few DPDK optimizations. so we can still have a decent coverage for building with DPDK, and for testing the build. please note, we don't test Seastar with the DPDK backend yet.to enable the workflow to pass extra options to
configure.py, we change the "cooks" parameter in the matrix to "options", and append "--dpdk-machine haswell" to it, so that we use this option only when testing the dpdk build.Fixes #2242