Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Restore Microsoft.Private.CoreFX.NETCoreApp package #58

Closed

Conversation

MichalStrehovsky
Copy link
Member

I haven't figured out what to do with it yet, but it's clear we can't package the framework out of the CoreCLR NuGet infra and we need one for Microsoft.DotNet.ILCompiler.

This is the rollback of the commit that deleted the transport package with tweaks to account for the endless build infra churn.

frameworkPackage.targets has some validation to ensure there's no cycle in assembly dependencies. I've deleted it because we now have at least one such cycle (netstandard > System.Xml.XDocument > System.Private.Xml.Linq > System.Private.Xml > System.Security.Cryptography.Algorithms > System.Formats.Asn1 > netstandard).

I haven't figured out what to do with it yet, but it's clear we can't package the framework out of the CoreCLR NuGet infra and we need one for Microsoft.DotNet.ILCompiler.

This is the rollback of the commit that deleted the transport package with tweaks to account for the endless build infra churn.

frameworkPackage.targets has some validation to ensure there's no cycle in assembly dependencies. I've deleted it because we now have at least one such cycle (`netstandard > System.Xml.XDocument > System.Private.Xml.Linq > System.Private.Xml > System.Security.Cryptography.Algorithms > System.Formats.Asn1 > netstandard`).
@MichalStrehovsky MichalStrehovsky added the area-NativeAOT-coreclr .NET runtime optimized for ahead of time compilation label Aug 19, 2020
@jkotas
Copy link
Member

jkotas commented Aug 19, 2020

we now have at least one such cycle

Does this need a bug in dotnet/runtime ?

@jkotas
Copy link
Member

jkotas commented Aug 19, 2020

it's clear we can't package the framework out of the CoreCLR NuGet infra

We would need to create the runtime packs in installer partition, same as where they are created for other runtimes.

Copy link
Member

@jkotas jkotas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks fine to me as a temporary workaround if doing the proper thing is too hard at the moment.

Do we need an issue to track redoing this without the extra package?

@MichalStrehovsky
Copy link
Member Author

Does this need a bug in dotnet/runtime ?

@ericstj I'm restoring Microsoft.Private.CoreFX.NETCoreApp package here for... reasons....

One of the things building this package did was the VerifyClosure thing that you added around dotnet/corefx@e04090e. Do we still care about cycles in the closure? We now apparently have some in .NET 5.

  Verifying closure of runtime.win-x64.Microsoft.Private.CoreFx.NETCoreApp runtime assemblies
C:\git\runtime\src\libraries\pkg\frameworkPackage.targets(110,5): error : Cycle detected for C:\git\runtime\artifacts\bin\netstandard\net5.0-Debug\netstandard.dll. [C:\git\runtime\src\libraries\pkg\Microsoft.Private.CoreFx.NETCoreApp\Microsoft.Private.CoreFx.NETCoreApp.pkgproj]
C:\git\runtime\src\libraries\pkg\frameworkPackage.targets(110,5): error : netstandard > System.Security.Cryptography.X509Certificates > System.Formats.Asn1 > netstandard [C:\git\runtime\src\libraries\pkg\Microsoft.Private.CoreFx.NETCoreApp\Microsoft.Private.CoreFx.NETCoreApp.pkgproj]
C:\git\runtime\src\libraries\pkg\frameworkPackage.targets(110,5): error : netstandard > System.ComponentModel.TypeConverter > System.Net.Security > System.Security.Cryptography.X509Certificates > System.Formats.Asn1 > netstandard [C:\git\runtime\src\libraries\pkg\Microsoft.Private.CoreFx.NETCoreApp\Microsoft.Private.CoreFx.NETCoreApp.pkgproj]
C:\git\runtime\src\libraries\pkg\frameworkPackage.targets(110,5): error : netstandard > System.Data.Common > System.Xml.ReaderWriter > System.Private.Xml > System.Security.Cryptography.Algorithms > System.Formats.Asn1 > netstandard [C:\git\runtime\src\libraries\pkg\Microsoft.Private.CoreFx.NETCoreApp\Microsoft.Private.CoreFx.NETCoreApp.pkgproj]
C:\git\runtime\src\libraries\pkg\frameworkPackage.targets(110,5): error : netstandard > System.Diagnostics.TextWriterTraceListener > System.Xml.ReaderWriter > System.Private.Xml > System.Security.Cryptography.Algorithms > System.Formats.Asn1 > netstandard [C:\git\runtime\src\libraries\pkg\Microsoft.Private.CoreFx.NETCoreApp\Microsoft.Private.CoreFx.NETCoreApp.pkgproj]
C:\git\runtime\src\libraries\pkg\frameworkPackage.targets(110,5): error : netstandard > System.IO.IsolatedStorage > System.Security.Cryptography.Algorithms > System.Formats.Asn1 > netstandard [C:\git\runtime\src\libraries\pkg\Microsoft.Private.CoreFx.NETCoreApp\Microsoft.Private.CoreFx.NETCoreApp.pkgproj]
C:\git\runtime\src\libraries\pkg\frameworkPackage.targets(110,5): error : netstandard > System.Net.Requests > System.Security.Cryptography.X509Certificates > System.Formats.Asn1 > netstandard [C:\git\runtime\src\libraries\pkg\Microsoft.Private.CoreFx.NETCoreApp\Microsoft.Private.CoreFx.NETCoreApp.pkgproj]
C:\git\runtime\src\libraries\pkg\frameworkPackage.targets(110,5): error : netstandard > System.Net.Primitives > System.Security.Cryptography.Algorithms > System.Formats.Asn1 > netstandard [C:\git\runtime\src\libraries\pkg\Microsoft.Private.CoreFx.NETCoreApp\Microsoft.Private.CoreFx.NETCoreApp.pkgproj]
C:\git\runtime\src\libraries\pkg\frameworkPackage.targets(110,5): error : netstandard > System.Net.HttpListener > System.Security.Cryptography.X509Certificates > System.Formats.Asn1 > netstandard [C:\git\runtime\src\libraries\pkg\Microsoft.Private.CoreFx.NETCoreApp\Microsoft.Private.CoreFx.NETCoreApp.pkgproj]
C:\git\runtime\src\libraries\pkg\frameworkPackage.targets(110,5): error : netstandard > System.Net.ServicePoint > System.Security.Cryptography.X509Certificates > System.Formats.Asn1 > netstandard [C:\git\runtime\src\libraries\pkg\Microsoft.Private.CoreFx.NETCoreApp\Microsoft.Private.CoreFx.NETCoreApp.pkgproj]
C:\git\runtime\src\libraries\pkg\frameworkPackage.targets(110,5): error : netstandard > System.Net.NameResolution > System.Net.Primitives > System.Security.Cryptography.Algorithms > System.Formats.Asn1 > netstandard [C:\git\runtime\src\libraries\pkg\Microsoft.Private.CoreFx.NETCoreApp\Microsoft.Private.CoreFx.NETCoreApp.pkgproj]
C:\git\runtime\src\libraries\pkg\frameworkPackage.targets(110,5): error : netstandard > System.Net.WebClient > System.Net.Primitives > System.Security.Cryptography.Algorithms > System.Formats.Asn1 > netstandard [C:\git\runtime\src\libraries\pkg\Microsoft.Private.CoreFx.NETCoreApp\Microsoft.Private.CoreFx.NETCoreApp.pkgproj]
C:\git\runtime\src\libraries\pkg\frameworkPackage.targets(110,5): error : netstandard > System.Net.Http > System.Security.Cryptography.X509Certificates > System.Formats.Asn1 > netstandard [C:\git\runtime\src\libraries\pkg\Microsoft.Private.CoreFx.NETCoreApp\Microsoft.Private.CoreFx.NETCoreApp.pkgproj]
C:\git\runtime\src\libraries\pkg\frameworkPackage.targets(110,5): error : netstandard > System.Net.WebProxy > System.Net.Primitives > System.Security.Cryptography.Algorithms > System.Formats.Asn1 > netstandard [C:\git\runtime\src\libraries\pkg\Microsoft.Private.CoreFx.NETCoreApp\Microsoft.Private.CoreFx.NETCoreApp.pkgproj]
C:\git\runtime\src\libraries\pkg\frameworkPackage.targets(110,5): error : netstandard > System.Net.Mail > System.Security.Cryptography.X509Certificates > System.Formats.Asn1 > netstandard [C:\git\runtime\src\libraries\pkg\Microsoft.Private.CoreFx.NETCoreApp\Microsoft.Private.CoreFx.NETCoreApp.pkgproj]
C:\git\runtime\src\libraries\pkg\frameworkPackage.targets(110,5): error : netstandard > System.Net.NetworkInformation > System.Net.Primitives > System.Security.Cryptography.Algorithms > System.Formats.Asn1 > netstandard [C:\git\runtime\src\libraries\pkg\Microsoft.Private.CoreFx.NETCoreApp\Microsoft.Private.CoreFx.NETCoreApp.pkgproj]
C:\git\runtime\src\libraries\pkg\frameworkPackage.targets(110,5): error : netstandard > System.Net.Ping > System.Net.Primitives > System.Security.Cryptography.Algorithms > System.Formats.Asn1 > netstandard [C:\git\runtime\src\libraries\pkg\Microsoft.Private.CoreFx.NETCoreApp\Microsoft.Private.CoreFx.NETCoreApp.pkgproj]
C:\git\runtime\src\libraries\pkg\frameworkPackage.targets(110,5): error : netstandard > System.Net.Security > System.Security.Cryptography.X509Certificates > System.Formats.Asn1 > netstandard [C:\git\runtime\src\libraries\pkg\Microsoft.Private.CoreFx.NETCoreApp\Microsoft.Private.CoreFx.NETCoreApp.pkgproj]
C:\git\runtime\src\libraries\pkg\frameworkPackage.targets(110,5): error : netstandard > System.Net.Sockets > System.Net.Primitives > System.Security.Cryptography.Algorithms > System.Formats.Asn1 > netstandard [C:\git\runtime\src\libraries\pkg\Microsoft.Private.CoreFx.NETCoreApp\Microsoft.Private.CoreFx.NETCoreApp.pkgproj]
C:\git\runtime\src\libraries\pkg\frameworkPackage.targets(110,5): error : netstandard > System.Net.WebSockets.Client > System.Security.Cryptography.X509Certificates > System.Formats.Asn1 > netstandard [C:\git\runtime\src\libraries\pkg\Microsoft.Private.CoreFx.NETCoreApp\Microsoft.Private.CoreFx.NETCoreApp.pkgproj]
C:\git\runtime\src\libraries\pkg\frameworkPackage.targets(110,5): error : netstandard > System.Net.WebSockets > System.Security.Cryptography.Algorithms > System.Formats.Asn1 > netstandard [C:\git\runtime\src\libraries\pkg\Microsoft.Private.CoreFx.NETCoreApp\Microsoft.Private.CoreFx.NETCoreApp.pkgproj]
C:\git\runtime\src\libraries\pkg\frameworkPackage.targets(110,5): error : netstandard > System.Runtime.Serialization.Xml > System.Private.DataContractSerialization > System.Xml.ReaderWriter > System.Private.Xml > System.Security.Cryptography.Algorithms > System.Formats.Asn1 > netstandard [C:\git\runtime\src\libraries\pkg\Microsoft.Private.CoreFx.NETCoreApp\Microsoft.Private.CoreFx.NETCoreApp.pkgproj]
C:\git\runtime\src\libraries\pkg\frameworkPackage.targets(110,5): error : netstandard > System.Runtime.Serialization.Json > System.Private.DataContractSerialization > System.Xml.ReaderWriter > System.Private.Xml > System.Security.Cryptography.Algorithms > System.Formats.Asn1 > netstandard [C:\git\runtime\src\libraries\pkg\Microsoft.Private.CoreFx.NETCoreApp\Microsoft.Private.CoreFx.NETCoreApp.pkgproj]
C:\git\runtime\src\libraries\pkg\frameworkPackage.targets(110,5): error : netstandard > System.Security.Cryptography.Algorithms > System.Formats.Asn1 > netstandard [C:\git\runtime\src\libraries\pkg\Microsoft.Private.CoreFx.NETCoreApp\Microsoft.Private.CoreFx.NETCoreApp.pkgproj]
C:\git\runtime\src\libraries\pkg\frameworkPackage.targets(110,5): error : netstandard > System.Security.Cryptography.Csp > System.Security.Cryptography.Algorithms > System.Formats.Asn1 > netstandard [C:\git\runtime\src\libraries\pkg\Microsoft.Private.CoreFx.NETCoreApp\Microsoft.Private.CoreFx.NETCoreApp.pkgproj]
C:\git\runtime\src\libraries\pkg\frameworkPackage.targets(110,5): error : netstandard > System.Xml.ReaderWriter > System.Private.Xml > System.Security.Cryptography.Algorithms > System.Formats.Asn1 > netstandard [C:\git\runtime\src\libraries\pkg\Microsoft.Private.CoreFx.NETCoreApp\Microsoft.Private.CoreFx.NETCoreApp.pkgproj]
C:\git\runtime\src\libraries\pkg\frameworkPackage.targets(110,5): error : netstandard > System.Xml.XDocument > System.Private.Xml.Linq > System.Private.Xml > System.Security.Cryptography.Algorithms > System.Formats.Asn1 > netstandard [C:\git\runtime\src\libraries\pkg\Microsoft.Private.CoreFx.NETCoreApp\Microsoft.Private.CoreFx.NETCoreApp.pkgproj]
C:\git\runtime\src\libraries\pkg\frameworkPackage.targets(110,5): error : netstandard > System.Xml.XmlSerializer > System.Private.Xml > System.Security.Cryptography.Algorithms > System.Formats.Asn1 > netstandard [C:\git\runtime\src\libraries\pkg\Microsoft.Private.CoreFx.NETCoreApp\Microsoft.Private.CoreFx.NETCoreApp.pkgproj]
C:\git\runtime\src\libraries\pkg\frameworkPackage.targets(110,5): error : netstandard > System.Xml.XPath.XDocument > System.Private.Xml > System.Security.Cryptography.Algorithms > System.Formats.Asn1 > netstandard [C:\git\runtime\src\libraries\pkg\Microsoft.Private.CoreFx.NETCoreApp\Microsoft.Private.CoreFx.NETCoreApp.pkgproj]
C:\git\runtime\src\libraries\pkg\frameworkPackage.targets(110,5): error : netstandard > System.Xml.XPath > System.Private.Xml > System.Security.Cryptography.Algorithms > System.Formats.Asn1 > netstandard [C:\git\runtime\src\libraries\pkg\Microsoft.Private.CoreFx.NETCoreApp\

@MichalStrehovsky
Copy link
Member Author

Looks fine to me as a temporary workaround if doing the proper thing is too hard at the moment.

https://github.com/dotnet/runtimelab/blob/NativeAOT/src/installer/pkg/projects/netcoreapp/pkg/Microsoft.NETCore.App.Runtime.pkgproj is pretty obfuscated in terms of what it is doing (I have no clue how that things turns into a package). I'll spend some time over a MSBuild binlog and try to figure out what it does and how to clone it into what we need.

Maybe the extra transport package is not worth it.

I'm afraid of touching too many files because people keep redoing this all the time. Whatever we build here might get broken more often than JitInterface and fixing the fallout will probably always involve inspecting binlogs because the build infra is too clever and churns too much to keep track of.

@ericstj
Copy link
Member

ericstj commented Aug 20, 2020

Do we still care about cycles in the closure? We now apparently have some in .NET 5.

Can you please open a bug and assign to @jkoritzinsky to re-enable this testing. Looks like we have a bug: dotnet/runtime#31805

The fix for this issue is to add a $(NetCoreAppCurrent) configuration to Asn1. I'll file an issue. cc @bartonjs

@jkoritzinsky
Copy link
Member

@MichalStrehovsky the building of the runtime pack will be significantly simplified once I get dotnet/arcade#5714 and dotnet/runtime#38457 merged.

@MichalStrehovsky
Copy link
Member Author

Closing in favor of #64.

@MichalStrehovsky the building of the runtime pack will be significantly simplified once I get dotnet/arcade#5714 and dotnet/runtime#38457 merged.

Thanks for the pointers!

scalablecory pushed a commit that referenced this pull request Sep 22, 2020
For some reason, the build issue was only caught internally.
MichalStrehovsky added a commit to MichalStrehovsky/runtimelab that referenced this pull request Oct 15, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-NativeAOT-coreclr .NET runtime optimized for ahead of time compilation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants