Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove usages of native bootstrapping #65901

Merged
merged 2 commits into from
Feb 25, 2022

Conversation

hoyosjs
Copy link
Member

@hoyosjs hoyosjs commented Feb 25, 2022

No description provided.

@dotnet-issue-labeler
Copy link

I couldn't figure out the best area label to add to this PR. If you have write-permissions please help me learn by adding exactly one area label.

@ghost ghost assigned hoyosjs Feb 25, 2022
@hoyosjs
Copy link
Member Author

hoyosjs commented Feb 25, 2022

/azp run runtime-staging

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@ghost
Copy link

ghost commented Feb 25, 2022

Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.

Issue Details

null

Author: hoyosjs
Assignees: hoyosjs
Labels:

area-Infrastructure

Milestone: -

@hoyosjs hoyosjs force-pushed the juhoyosa/remove-native-bootstrap branch from fc545b0 to d6e3a31 Compare February 25, 2022 18:57
@hoyosjs
Copy link
Member Author

hoyosjs commented Feb 25, 2022

/azp run runtime-coreclr superpmi-replay

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@@ -491,6 +491,7 @@
<_MonoCMakeBuildCommand Condition="'$(_MonoUseNinja)' != 'true'">$(_MonoCMakeBuildCommand) --parallel $([System.Environment]::ProcessorCount)</_MonoCMakeBuildCommand>
<_MonoCMakeBuildCommand Condition="'$(TargetsBrowser)' != 'true' and '$(HostOS)' != 'windows'">@(_MonoBuildEnv, ' ') $(_MonoCMakeBuildCommand)</_MonoCMakeBuildCommand>
<_MonoCMakeBuildCommand Condition="'$(TargetsBrowser)' != 'true' and '$(HostOS)' == 'windows'">call &quot;$(RepositoryEngineeringDir)native\init-vs-env.cmd&quot; $(_CompilerTargetArch) &amp;&amp; cd /D &quot;$(MonoObjDir)&quot; &amp;&amp; @(_MonoBuildEnv, ' ') $(_MonoCMakeBuildCommand)</_MonoCMakeBuildCommand>
<_MonoCMakeBuildCommand Condition="'$(TargetsBrowser)' == 'true' and '$(HostOS)' == 'windows'">call &quot;$(RepositoryEngineeringDir)native\init-vs-env.cmd&quot; &amp;&amp; $(_MonoCMakeBuildCommand)</_MonoCMakeBuildCommand>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the main difference between this and the above? The addition of _MonoBuildEnv?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yup, $(_CompilerTargetArch) and the build env

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before it was using line 489 on wasm, and it was failing to find cmake on PATH. Now we are prepending it with eng\native\init-vs-env.cmd && so PATH gets updated by VS cmd init script that has cmake bin path.

@hoyosjs
Copy link
Member Author

hoyosjs commented Feb 25, 2022

The only case where I am seeing crashes is x86. This seems like a suspension context issue:

00 0785d32c 72997b8e     coreclr!EEPolicy::HandleFatalError+0x72 [D:\a\_work\1\s\src\coreclr\vm\eepolicy.cpp @ 776] 
01 0785d4c4 76f634a1     coreclr!CPFH_VerifyThreadIsInValidState+0x1210d9 [D:\a\_work\1\s\src\coreclr\vm\i386\excepx86.cpp @ 345] 
02 0785d4e8 76f63473     ntdll!ExecuteHandler2+0x26 [d:\w7rtm\minkernel\ntos\rtl\i386\xcptmisc.asm @ 233] 
03 0785d598 76f63414     ntdll!ExecuteHandler+0x24 [d:\w7rtm\minkernel\ntos\rtl\i386\xcptmisc.asm @ 192] 
04 0785d598 76f10133     ntdll!RtlDispatchException+0x127 [d:\w7rtm\minkernel\ntos\rtl\i386\exdsptch.c @ 560] 
05 0785da64 75abf4a5     ntdll!KiUserExceptionDispatcher+0xf [d:\w7rtm\minkernel\ntos\rtl\i386\userdisp.asm @ 503] 
06 0785da64 728cb671     kernel32!SetXStateFeaturesMask+0x8 [d:\w7rtm\base\win32\client\xsave.c @ 296] 
07 0785da8c 72881278     coreclr!Thread::RedirectThreadAtHandledJITCase+0x45 [D:\a\_work\1\s\src\coreclr\vm\threadsuspend.cpp @ 2917] 
08 (Inline) --------     coreclr!Thread::CheckForAndDoRedirect+0xc [D:\a\_work\1\s\src\coreclr\vm\threadsuspend.cpp @ 2998] 
09 (Inline) --------     coreclr!Thread::CheckForAndDoRedirectForGC+0xc [D:\a\_work\1\s\src\coreclr\vm\threadsuspend.cpp @ 3130] 
0a 0785db08 728ccd4c     coreclr!ThreadSuspend::SuspendRuntime+0x3aa [D:\a\_work\1\s\src\coreclr\vm\threadsuspend.cpp @ 3525] 
0b 0785dbb4 728ccbe3     coreclr!ThreadSuspend::SuspendEE+0xf9 [D:\a\_work\1\s\src\coreclr\vm\threadsuspend.cpp @ 5753] 
0c 0785dbdc 72876ba8     coreclr!GCToEEInterface::SuspendEE+0x26 [D:\a\_work\1\s\src\coreclr\vm\gcenv.ee.cpp @ 28] 
0d 0785dbdc 728f9ff9     coreclr!WKS::GCHeap::GarbageCollectGeneration+0xa8 [D:\a\_work\1\s\src\coreclr\gc\gc.cpp @ 45766] 
0e 0785dc00 7281ae50     coreclr!WKS::gc_heap::trigger_gc_for_alloc+0x19 [D:\a\_work\1\s\src\coreclr\gc\gc.cpp @ 17338] 
0f 0785dc34 728f3e68     coreclr!WKS::gc_heap::try_allocate_more_space+0x1e4 [D:\a\_work\1\s\src\coreclr\gc\gc.cpp @ 17484] 
10 0785dc4c 7281a113     coreclr!WKS::gc_heap::allocate_more_space+0x18 [D:\a\_work\1\s\src\coreclr\gc\gc.cpp @ 17948] 
11 (Inline) --------     coreclr!WKS::gc_heap::allocate+0x40 [D:\a\_work\1\s\src\coreclr\gc\gc.cpp @ 17979] 
12 0785dc6c 728183dc     coreclr!WKS::GCHeap::Alloc+0x53 [D:\a\_work\1\s\src\coreclr\gc\gc.cpp @ 44762] 
13 (Inline) --------     coreclr!Alloc+0x108 [D:\a\_work\1\s\src\coreclr\vm\gchelpers.cpp @ 226] 
14 0785dccc 72819dea     coreclr!AllocateObject+0x175 [D:\a\_work\1\s\src\coreclr\vm\gchelpers.cpp @ 979] 
15 0785dd50 06d04ba6     coreclr!JIT_New+0xaa [D:\a\_work\1\s\src\coreclr\vm\jithelpers.cpp @ 2312] 
16 0785dd7c 06d10a7e     Common_Tests!System.Net.Http.Unit.Tests.HPack.HPackDecoderTests.CreateDecoderAndTable()+0x46 [/_/src/libraries/Common/tests/Tests/System/Net/aspnetcore/Http2/HPackDecoderTest.cs @ 99] 
17 0785dddc 06d0fd4f     Common_Tests!System.Net.Http.Unit.Tests.HPack.HPackDecoderTests.TestDecode(Byte[], System.String, System.String, Boolean, Boolean)+0x1e [/_/src/libraries/Common/tests/Tests/System/Net/aspnetcore/Http2/HPackDecoderTest.cs @ 683] 
18 0785ddf8 06d141aa     Common_Tests!System.Net.Http.Unit.Tests.HPack.HPackDecoderTests.TestDecodeWithoutIndexing(Byte[], System.String, System.String)+0x1f [/_/src/libraries/Common/tests/Tests/System/Net/aspnetcore/Http2/HPackDecoderTest.cs @ 678] 
19 0785de10 72909d5f     Common_Tests!System.Net.Http.Unit.Tests.HPack.HPackDecoderTests.DecodesLiteralHeaderFieldNeverIndexed_NewName_HuffmanEncodedNameAndValue()+0x52 [/_/src/libraries/Common/tests/Tests/System/Net/aspnetcore/Http2/HPackDecoderTest.cs @ 355] 

Is this known @jkotas @VSadov? Is this safe to merge?

@jkotas
Copy link
Member

jkotas commented Feb 25, 2022

Yes, that is #65890

@hoyosjs hoyosjs merged commit 8727ac7 into dotnet:main Feb 25, 2022
@hoyosjs hoyosjs deleted the juhoyosa/remove-native-bootstrap branch February 25, 2022 22:06
@akoeplinger
Copy link
Member

akoeplinger commented Feb 28, 2022

We'll need to backport this to basically all earlier release branches right?

@hoyosjs
Copy link
Member Author

hoyosjs commented Feb 28, 2022

Yes

@hoyosjs
Copy link
Member Author

hoyosjs commented Feb 28, 2022

/backport to release/7.0-preview2

@github-actions
Copy link
Contributor

Started backporting to release/7.0-preview2: https://github.com/dotnet/runtime/actions/runs/1912448074

@agocke
Copy link
Member

agocke commented Mar 9, 2022

/backport to release/5.0

@github-actions
Copy link
Contributor

github-actions bot commented Mar 9, 2022

Started backporting to release/5.0: https://github.com/dotnet/runtime/actions/runs/1959416794

@github-actions
Copy link
Contributor

github-actions bot commented Mar 9, 2022

@agocke backporting to release/5.0 failed, the patch most likely resulted in conflicts:

$ git am --3way --ignore-whitespace --keep-non-patch changes.patch

Applying: Remove usages of native bootstrapping
Using index info to reconstruct a base tree...
M	eng/pipelines/common/global-build-job.yml
M	eng/pipelines/common/templates/runtimes/build-test-job.yml
A	eng/pipelines/coreclr/templates/build-jit-job.yml
M	eng/pipelines/coreclr/templates/build-job.yml
M	eng/pipelines/mono/templates/build-job.yml
A	eng/pipelines/mono/templates/generate-offsets.yml
M	global.json
Falling back to patching base and 3-way merge...
Auto-merging global.json
CONFLICT (content): Merge conflict in global.json
CONFLICT (modify/delete): eng/pipelines/mono/templates/generate-offsets.yml deleted in HEAD and modified in Remove usages of native bootstrapping. Version Remove usages of native bootstrapping of eng/pipelines/mono/templates/generate-offsets.yml left in tree.
Auto-merging eng/pipelines/mono/templates/build-job.yml
CONFLICT (content): Merge conflict in eng/pipelines/mono/templates/build-job.yml
Auto-merging eng/pipelines/coreclr/templates/build-job.yml
CONFLICT (content): Merge conflict in eng/pipelines/coreclr/templates/build-job.yml
CONFLICT (modify/delete): eng/pipelines/coreclr/templates/build-jit-job.yml deleted in HEAD and modified in Remove usages of native bootstrapping. Version Remove usages of native bootstrapping of eng/pipelines/coreclr/templates/build-jit-job.yml left in tree.
Auto-merging eng/pipelines/common/templates/runtimes/build-test-job.yml
CONFLICT (content): Merge conflict in eng/pipelines/common/templates/runtimes/build-test-job.yml
Auto-merging eng/pipelines/common/global-build-job.yml
CONFLICT (content): Merge conflict in eng/pipelines/common/global-build-job.yml
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
Patch failed at 0001 Remove usages of native bootstrapping
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".
Error: The process '/usr/bin/git' failed with exit code 128

Please backport manually!

agocke pushed a commit to agocke/runtime that referenced this pull request Mar 9, 2022
* Remove usages of native bootstrapping
* Make sure cmake is in the path for mono wasm builds

Co-authored-by: Adeel Mujahid <[email protected]>
(cherry picked from commit 8727ac7)
carlossanlop pushed a commit that referenced this pull request Mar 15, 2022
* Remove usages of native bootstrapping (#65901)

* Remove usages of native bootstrapping
* Make sure cmake is in the path for mono wasm builds

Co-authored-by: Adeel Mujahid <[email protected]>
(cherry picked from commit 8727ac7)

* Fix building CoreCLR for x86 with the Windows 10.0.20348.0 SDK (#57067)

The SDK now defines CONTEXT_UNWOUND_TO_CALL in more cases, so we get a macro redefinition error. Since the SDK defines it to the same value as we do, just skip our definition if it's already defined.

(cherry picked from commit ce4bf13)

* Port CMake fixes for latest version

Co-authored-by: Juan Hoyos <[email protected]>
Co-authored-by: Jeremy Koritzinsky <[email protected]>
@ghost ghost locked as resolved and limited conversation to collaborators Apr 9, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants