Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Patch Tuesday June 2024 #39325

Merged
merged 7 commits into from
Jul 4, 2024
Merged

Patch Tuesday June 2024 #39325

merged 7 commits into from
Jul 4, 2024

Conversation

BillyONeal
Copy link
Member

@BillyONeal BillyONeal commented Jun 17, 2024 •
edited
Loading

Updates:

  • VS 2022 17.8.11
  • CUDA 12.5 (on Windows)
  • ifort 2024.1
  • PSExec removed
  • ICC removed

Also includes a number of improvements from @StephanTLavavej in the microsoft/STL repo from which vcpkg's infrastructure here was originally forked.

@BillyONeal BillyONeal added info:internal This PR or Issue was filed by the vcpkg team. category:infrastructure Pertaining to the CI/Testing infrastrucutre labels Jun 17, 2024
@BillyONeal
Copy link
Member Author

I skipped updating the android container because we just did in #38975

@LilyWangLL LilyWangLL self-assigned this Jun 17, 2024
LilyWangLL
LilyWangLL previously approved these changes Jun 17, 2024
View reviewed changes
@LilyWangLL LilyWangLL added the info:reviewed Pull Request changes follow basic guidelines label Jun 17, 2024
@BillyONeal
Copy link
Member Author

@BillyONeal
Copy link
Member Author

REGRESSION: opencv4:x64-windows failed with BUILD_FAILED. If expected, add opencv4:x64-windows=fail to D:\a_work\1\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: vcpkg-ci-opencv:x64-windows cascaded, but it is required to pass. (D:\a_work\1\s\scripts\azure-pipelines/../ci.baseline.txt).

[...]
D:\b\opencv4\src\4.8.0-8d756cdf2d.clean\modules\cudev\include\opencv2\cudev\grid\detail/reduce_to_column.hpp(91): error: no instance of overloaded function "cv::cudev::blockReduce" matches the argument list
[...]

I believe this is opencv/opencv_contrib#3690

REGRESSION: qtwebengine:x64-windows failed with BUILD_FAILED. If expected, add qtwebengine:x64-windows=fail to D:\a_work\1\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: qt:x64-windows cascaded, but it is required to pass. (D:\a_work\1\s\scripts\azure-pipelines/../ci.baseline.txt).

ninja -t msvc -e environment.x64 -- "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.40.33807\bin\Hostx64\x64\cl.exe" /c ../../../../../src/here-src-6-dad1a1db84.clean/src/3rdparty/chromium/v8/src/wasm/turboshaft-graph-interface.cc /Foobj/v8/v8_base_without_compiler/turboshaft-graph-interface.obj /nologo /showIncludes -DUSE_AURA=1 -DTOOLKIT_QT -D_CRT_NONSTDC_NO_WARNINGS -D_WINSOCK_DEPRECATED_NO_WARNINGS -D_HAS_EXCEPTIONS=0 -D__STD_C -D_CRT_RAND_S -D_CRT_SECURE_NO_DEPRECATE -D_SCL_SECURE_NO_DEPRECATE -D_ATL_NO_OPENGL -D_WINDOWS -DCERT_CHAIN_PARA_HAS_EXTRA_FIELDS -DPSAPI_VERSION=2 -DWIN32 -D_SECURE_ATL -DWINAPI_FAMILY=WINAPI_FAMILY_DESKTOP_APP -DWIN32_LEAN_AND_MEAN -DNOMINMAX -D_UNICODE -DUNICODE -DNTDDI_VERSION=NTDDI_WIN10_NI -D_WIN32_WINNT=0x0A00 -DWINVER=0x0A00 -D_DEBUG -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DCPPGC_VERIFY_HEAP -DENABLE_DISASSEMBLER -DV8_TYPED_ARRAY_MAX_SIZE_IN_HEAP=64 -DOBJECT_PRINT -DVERIFY_HEAP -DV8_TRACE_MAPS -DV8_ENABLE_ALLOCATION_TIMEOUT -DV8_ENABLE_FORCE_SLOW_PATH -DV8_ENABLE_DOUBLE_CONST_STORE_CHECK -DV8_INTL_SUPPORT -DENABLE_HANDLE_ZAPPING -DV8_CODE_COMMENTS -DV8_ENABLE_DEBUG_CODE -DV8_ENABLE_HEAP_SNAPSHOT_VERIFY -DV8_SNAPSHOT_NATIVE_CODE_COUNTERS -DV8_USE_EXTERNAL_STARTUP_DATA -DV8_ATOMIC_OBJECT_FIELD_WRITES -DV8_ENABLE_LAZY_SOURCE_POSITIONS -DV8_SHARED_RO_HEAP -DV8_WIN64_UNWINDING_INFO -DV8_ENABLE_REGEXP_INTERPRETER_THREADED_DISPATCH -DV8_SHORT_BUILTIN_CALLS -DV8_EXTERNAL_CODE_SPACE -DV8_ENABLE_MAGLEV -DV8_ENABLE_TURBOFAN -DV8_ENABLE_SYSTEM_INSTRUMENTATION -DV8_ENABLE_ETW_STACK_WALKING -DV8_ENABLE_WEBASSEMBLY -DV8_ALLOCATION_FOLDING -DV8_ALLOCATION_SITE_TRACKING -DV8_ADVANCED_BIGINT_ALGORITHMS -DV8_STATIC_ROOTS -DV8_USE_ZLIB -DV8_ENABLE_WASM_SIMD256_REVEC -DV8_CODE_POINTER_SANDBOXING -DV8_ENABLE_SLOW_TRACING -DV8_ENABLE_BUILTIN_JUMP_TABLE_SWITCH -DV8_ENABLE_EXTENSIBLE_RO_SNAPSHOT -DV8_ENABLE_CHECKS -DV8_COMPRESS_POINTERS -DV8_COMPRESS_POINTERS_IN_SHARED_CAGE -DV8_31BIT_SMIS_ON_64BIT_ARCH -DV8_ENABLE_SANDBOX -DV8_DEPRECATION_WARNINGS -DCPPGC_CAGED_HEAP -DCPPGC_YOUNG_GENERATION -DCPPGC_POINTER_COMPRESSION -DCPPGC_SLIM_WRITE_BARRIER -DV8_TARGET_ARCH_X64 -DV8_HAVE_TARGET_OS -DV8_TARGET_OS_WIN -DDEBUG -DABSL_ALLOCATOR_NOTHROW=1 -D_ENABLE_EXTENDED_ALIGNED_STORAGE -DU_USING_ICU_NAMESPACE=0 -DU_ENABLE_DYLOAD=0 -DUSE_CHROMIUM_ICU=1 -DU_ENABLE_TRACING=1 -DU_ENABLE_RESOURCE_TRACING=0 -DU_STATIC_IMPLEMENTATION -DICU_UTIL_DATA_IMPL=ICU_UTIL_DATA_FILE -Igen -I../../../../../src/here-src-6-dad1a1db84.clean/src/3rdparty/chromium -I../../../../../src/here-src-6-dad1a1db84.clean/src/3rdparty/chromium/v8 -I../../../../../src/here-src-6-dad1a1db84.clean/src/3rdparty/chromium/v8/include -Igen/v8 -Igen/v8/include -I../../../../../src/here-src-6-dad1a1db84.clean/src/3rdparty/chromium/third_party/abseil-cpp -I../../../../../src/here-src-6-dad1a1db84.clean/src/3rdparty/chromium/third_party/icu/source/common -I../../../../../src/here-src-6-dad1a1db84.clean/src/3rdparty/chromium/third_party/icu/source/i18n -I../../../../../src/here-src-6-dad1a1db84.clean/src/3rdparty/chromium/third_party/zlib /wd4091 /wd4127 /wd4251 /wd4275 /wd4312 /wd4324 /wd4351 /wd4355 /wd4503 /wd4589 /wd4611 /wd4100 /wd4121 /wd4244 /wd4505 /wd4510 /wd4512 /wd4610 /wd4838 /wd4995 /wd4996 /wd4456 /wd4457 /wd4458 /wd4459 /wd4200 /wd4201 /wd4204 /wd4221 /wd4245 /wd4267 /wd4305 /wd4389 /wd4702 /wd4701 /wd4703 /wd4661 /wd4706 /wd4715 /wd4003 /wd4068 /wd5051 /wd4244 /Gy /FS /bigobj /utf-8 /Zc:preprocessor /wd5105 /wd4117 /D__DATE__= /D__TIME__= /D__TIMESTAMP__= /MDd /wd4245 /wd4267 /wd4324 /wd4701 /wd4702 /wd4703 /wd4709 /wd4714 /wd4715 /wd4716 /wd4718 /wd4723 /wd4724 /wd4800 /wd4506 /wd4091 /wd4127 /wd4251 /wd4275 /wd4312 /wd4324 /wd4351 /wd4355 /wd4503 /wd4589 /wd4611 /wd4100 /wd4121 /wd4244 /wd4505 /wd4510 /wd4512 /wd4610 /wd4838 /wd4995 /wd4996 /wd4456 /wd4457 /wd4458 /wd4459 /wd4200 /wd4201 /wd4204 /wd4221 /wd4245 /wd4267 /wd4305 /wd4389 /wd4702 /wd4701 /wd4703 /wd4661 /wd4706 /wd4715 /O2 /Ob2 /Oy- /Zc:inline /Gw /wd4577 /std:c++20 /TP /GR- /Fd"obj/v8/v8_base_without_compiler_cc.pdb"
../../../../../src/here-src-6-dad1a1db84.clean/src/3rdparty/chromium/v8\src/compiler/turboshaft/operations.h(1336): error C2100: you cannot dereference an operand of type 'v8::internal::compiler::turboshaft::WordRepresentation'
../../../../../src/here-src-6-dad1a1db84.clean/src/3rdparty/chromium/v8\src/compiler/turboshaft/operations.h(1336): note: the template instantiation context (the oldest one first) is
../../../../../src/here-src-6-dad1a1db84.clean/src/3rdparty/chromium/v8\src/compiler/turboshaft/operations.h(1336): note: see reference to alias template instantiation 'v8::internal::compiler::turboshaft::MaskBuilder<v8::internal::compiler::turboshaft::WordBinopOp,v8::internal::compiler::turboshaft::OpMaskField<v8::internal::compiler::turboshaft::WordBinopOp::Kind,4>,v8::internal::compiler::turboshaft::OpMaskField<v8::internal::compiler::turboshaft::UnwrapRepresentation<v8::internal::compiler::turboshaft::WordRepresentation>::type,5>>::For<v8::internal::compiler::turboshaft::WordBinopOp::Kind::kSub,v8::internal::compiler::turboshaft::WordRepresentation{v8::internal::compiler::turboshaft::RegisterRepresentation{v8::internal::compiler::turboshaft::MaybeRegisterRepresentation{v8::internal::compiler::turboshaft::MaybeRegisterRepresentation::Enum:v8::internal::compiler::turboshaft::MaybeRegisterRepresentation::Enum::kWord32}}}>' being compiled
../../../../../src/here-src-6-dad1a1db84.clean/src/3rdparty/chromium/v8\src/compiler/turboshaft/operations.h(940): fatal error C1907: unable to recover from previous error(s); stopping compilation
../../../../../src/here-src-6-dad1a1db84.clean/src/3rdparty/chromium/v8\src/compiler/turboshaft/representations.h(379): note: while evaluating constexpr function 'v8::internal::compiler::turboshaft::WordRepresentation::value'
../../../../../src/here-src-6-dad1a1db84.clean/src/3rdparty/chromium/v8\src/compiler/turboshaft/operations.h(1336): note: while evaluating constexpr function 'v8::internal::compiler::turboshaft::WordRepresentation::operator v8::internal::compiler::turboshaft::WordRepresentation::Enum'
INTERNAL COMPILER ERROR in 'C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.40.33807\bin\Hostx64\x64\cl.exe'
    Please choose the Technical Support command on the Visual C++
    Help menu, or open the Technical Support help file for more information

Neumann-A
Neumann-A reviewed Jun 18, 2024
View reviewed changes
scripts/ci.baseline.txt Outdated Show resolved Hide resolved
@BillyONeal BillyONeal added requires:vcpkg-team-review This PR or issue requires someone on the vcpkg team to take a further look. and removed info:reviewed Pull Request changes follow basic guidelines labels Jun 18, 2024
@BillyONeal
Copy link
Member Author

I want other maintainers' thoughts about the qtwebengine skip

@BillyONeal BillyONeal marked this pull request as draft June 18, 2024 23:12
@BillyONeal BillyONeal removed the requires:vcpkg-team-review This PR or issue requires someone on the vcpkg team to take a further look. label Jun 18, 2024
@BillyONeal
Copy link
Member Author

We'll go back to 17.8.

@BillyONeal BillyONeal mentioned this pull request Jun 19, 2024
Closed
@BillyONeal
Copy link
Member Author

Cherry picking #38996 had the nice effect of invalidating all the Windows SHAs so I don't need to issue a separate CI run anymore

@BillyONeal BillyONeal marked this pull request as ready for review June 19, 2024 20:29
@cenit
Copy link
Contributor

cenit commented Jun 20, 2024

REGRESSION: opencv4:x64-windows failed with BUILD_FAILED. If expected, add opencv4:x64-windows=fail to D:\a_work\1\s\scripts\azure-pipelines/../ci.baseline.txt. REGRESSION: vcpkg-ci-opencv:x64-windows cascaded, but it is required to pass. (D:\a_work\1\s\scripts\azure-pipelines/../ci.baseline.txt).

I believe this is opencv/opencv_contrib#3690

It's that issue and more (compatibility with cuda 12.4+ was seriously compromised).
All the patches should be in here #38928 but i am very slow on it... any help would be very appreciated

@BillyONeal
Copy link
Member Author

It's that issue and more (compatibility with cuda 12.4+ was seriously compromised).
All the patches should be in here #38928 but i am very slow on it... any help would be very appreciated

Taking updates is good but I don't want to make more than the minimum number of changes in an infrastructure security patch like this

@BillyONeal
Copy link
Member Author

I'm trying to update libtorch to fix that break. (And before you go 'why didn't you update opencv then': opencv's fixes were only 3-4 commits, libtorch's fixes seem far more broad/widespread since cuda removed a major subsystem libtorch used)

@Neumann-A
Copy link
Contributor

I have the update to libtorch here:
https://github.com/Neumann-A/my-vcpkg-ports/blob/master/libtorch/
Also requires newer xnnpack and something else.

Didn't update it yet in vcpkg since my port has a working python behind building the python bindings which have not yet made it into vcpkg. So it will need some adjustments in that regard.

@BillyONeal BillyONeal mentioned this pull request Jun 25, 2024
Merged
7 tasks
@LilyWangLL LilyWangLL mentioned this pull request Jun 25, 2024
Open
@BillyONeal
Update pool.
ad298cf
@BillyONeal
[opencv] More cuda fixes.
c4bdd95
@BillyONeal
[unittest-cpp] Disable trying to pass -Wextra to msvc
3cddd22
@BillyONeal
[libtorch] Fix CUDA 12.5, patch whitespace errors, and patched patches
2f41e43 
fix-onnx-case.patch merged into fix-build.patch
Several patches had whitespace errors which have been corrected
cuda-adjustments..patch had the commented-out-patch antipattern which has been changed to just delete the whole relevant blocks instead. (Unfortunately git changed the order of the resulting patch :( )
@BillyONeal
Copy link
Member Author

I have the update to libtorch here:
https://github.com/Neumann-A/my-vcpkg-ports/blob/master/libtorch/
Also requires newer xnnpack and something else.

I gave this a shot and it turned out to be WAY too involved :( But luckily after replaying all the patches to fix whitespace errors upstream's fix for the cudnn incompatibility seems to be fixed now.

@BillyONeal BillyONeal merged commit 0dab813 into microsoft:master Jul 4, 2024
17 checks passed
@BillyONeal BillyONeal deleted the patch-tuesday-june-2024 branch July 4, 2024 03:01
@LilyWangLL LilyWangLL added the info:reviewed Pull Request changes follow basic guidelines label Jul 4, 2024
@Cheney-W Cheney-W mentioned this pull request Jul 15, 2024
Closed
7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Neumann-A Neumann-A Neumann-A left review comments

@LilyWangLL LilyWangLL LilyWangLL left review comments

Assignees

@LilyWangLL LilyWangLL

Labels
category:infrastructure Pertaining to the CI/Testing infrastrucutre info:internal This PR or Issue was filed by the vcpkg team. info:reviewed Pull Request changes follow basic guidelines
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@BillyONeal @cenit @Neumann-A @LilyWangLL