Building 3.0 on Arch Linux #1310

alucryd · 2019-10-12T16:34:50Z

Reopening an issue since for some reason issue #1242 was completely nuked along with all the valuable information it contained.

The latest head of the release/3.0 branch still doesn't build. Did you guys make progress on this?

omajid · 2019-10-12T20:15:56Z

issue #1242 was completely nuked

Yikes. I can't seem to access it either. @dleeapho Any chance we could get that issue back?

Did you guys make progress on this?

I tried installing Arch on a machine, but I didn't have much luck. Tried it in a container and I ran into the dreaded "too many files open" issue :(

Can you git clone https://github.com/dotnet/source-build --recrusive and make this change:

diff --git a/tools-local/arcade-services/eng/Versions.props b/tools-local/arcade-services/eng/Versions.props
index c69a2ff..6350ff1 100644
--- a/tools-local/arcade-services/eng/Versions.props
+++ b/tools-local/arcade-services/eng/Versions.props
@@ -12,7 +12,7 @@
     <CommandLineParserVersion>2.2.1</CommandLineParserVersion>
     <CredentialManagementVersion>1.0.2</CredentialManagementVersion>
     <HandlebarsNetVersion>1.9.5</HandlebarsNetVersion>
-    <LibGit2SharpVersion>0.26.0</LibGit2SharpVersion>
+    <LibGit2SharpVersion>0.27.0-preview-0020</LibGit2SharpVersion>
     <log4netVersion>2.0.8</log4netVersion>
     <SystemNetHttpVersion>4.3.3</SystemNetHttpVersion>
     <MicrosoftAzureKeyVaultVersion>3.0.3</MicrosoftAzureKeyVaultVersion>

And try a ./build.sh?

I also opened dotnet/corefx#41750 and dotnet/cli#12880

alucryd · 2019-10-13T13:39:11Z

@omajid Thanks for the PRs!

What's this newer libgit2 for? The current one works fine with openssl 1.0 installed, is 0.27.0 built with openssl 1.1?

robinchew · 2019-10-13T13:40:23Z

Here's a Google cache of #1242 https://webcache.googleusercontent.com/search?q=cache:CIeT58k-DkgJ:https://github.com/dotnet/source-build/issues/1242 and I've also attached a screenshot.
https://user-images.githubusercontent.com/7122710/66716574-d7897a80-ee01-11e9-8ea8-79f5b8a10c6d.png

omajid · 2019-10-13T17:49:54Z

is 0.27.0 built with openssl 1.1

Kind of; 0.27 is more flexible and supports both.

alucryd · 2019-10-13T18:33:33Z

Kind of; 0.27 is more flexible and supports both.

Works like a charm with just openssl 1.1 in the chroot, thanks!

omajid · 2019-10-13T22:13:56Z

Works like a charm with just openssl 1.1 in the chroot, thanks!

Does that mean you can build .NET Core 3.0 in the chroot? Or that it gets you past libgit2 erorrs?

alucryd · 2019-10-14T05:59:46Z

Just the libgit2 errors, those were previously fixed by having openssl 1.0 in the chroot, but now it works with 1.1, hopefully we can start retiring 1.0 from our repos soon.

adaggarwal · 2019-10-14T15:46:53Z

Reopening an issue since for some reason issue #1242 was completely nuked along with all the valuable information it contained.

The latest head of the release/3.0 branch still doesn't build. Did you guys make progress on this?

I think it didn't transfer cleanly. Here's the original issue - https://github.com/dotnet/core-eng/issues/7996

Here is the coreFX reference to this issue.

cc: @alucryd @omajid

omajid · 2019-10-14T15:49:32Z

I think it didn't transfer cleanly. Here's the original issue - dotnet/core-eng#7996

Unfortunately, dotnet/core-eng is, as far as I know, restricted to folks inside Microsoft. As an outsider-to-Microsoft, the optic of this change look like kicking the community out from a community-based issue. I know that's probably not intentional on anyone's part. Is there any chance we could get the issue back in a place everyone can see it?

I can see the corefx issue, but it just contains a pointer to the can't-be-seen-core-eng-issue.

adaggarwal · 2019-10-14T15:51:26Z

I think it didn't transfer cleanly. Here's the original issue - dotnet/core-eng#7996

Unfortunately, dotnet/core-eng is, as far as I know, restricted to folks inside Microsoft. As an outsider-to-Microsoft, this change looks like someone just completely restricted/hid the issue :(

I can see the corefx issue, but it just contains a pointer to the can't-be-seen-core-eng-issue.

Ah. I understand now. I tried to get the issue back. But it wasn't very consistent. Using zenhub, I could get all the comments and timestamps. It just lacks the reactions. I hope we can reference #1312 for history.

dagood · 2019-10-14T16:07:54Z

FYI, I found this in the help doc for GitHub transfers:

You can't transfer an issue from a private repository to a public repository.

I guess it makes sense from a "make it a little harder to expose secrets" sense, but unfortunate for this case. Would have been nice to be able to use the "native" move to bring it back as-was.

omajid · 2019-10-17T13:00:41Z

I have now locally seen both arch-x64 and fedora.32-x64 builds fail with failures similar to:

    chmod +x /builddir/build/BUILD/dotnet-v3.0.0/src/corefx.4ac4c0367003fe3973a3648eb0715ddb0e3bbcea/artifacts/bin/testhost/netcoreapp-Linux-Release-x64/                                                                        
    chmod: cannot access '/builddir/build/BUILD/dotnet-v3.0.0/src/corefx.4ac4c0367003fe3973a3648eb0715ddb0e3bbcea/artifacts/bin/testhost/netcoreapp-Linux-Release-x64/': No such file or directory                               
8>/builddir/build/BUILD/dotnet-v3.0.0/src/corefx.4ac4c0367003fe3973a3648eb0715ddb0e3bbcea/external/runtime/runtime.depproj(63,5): error MSB3073: The command "chmod +x /builddir/build/BUILD/dotnet-v3.0.0/src/corefx.4ac4c0367003fe3973a3648eb0715ddb0e3bbcea/artifacts/bin/testhost/netcoreapp-Linux-Release-x64/" exited with code 1.

Need to debug this some more.

Edit: I tried building on Fedora 30 (where this is known to work) with /etc/os-release modified to say 32. The build failed there with this exact same error. I doubt it's an issue in the system libraries or tools; otherwise it would have built on this hacked-Fedora-30 system too. This error seems more of an RID issue to me now.

tmds · 2019-10-23T03:31:52Z

This error seems more of an RID issue to me now.

It think like the issue is with:

<RuntimeIdentifier>$(PackageRID)</RuntimeIdentifier>

in runtime.depproj. I think it fails to restore some things when the rid is not known, causing them to be missing when they are used later (with the chmod).

SuddenGunter · 2019-11-07T12:16:47Z

@adaggarwal any info when it can be fixed?

adaggarwal · 2019-11-07T16:00:26Z

@SuddenGunter is coreFX standalone still failing on Arch Linux?

Saancreed · 2019-11-11T16:59:08Z

@adaggarwal Actually it seems like the build itself isn't failing anymore, at least the current release/3.0@c1778515a3bee34cc09c757b5563d0af0c8b1e99 with pre-rebased RID patch applied on top of it. Some tests in System.Net.Ping.Functional.Tests are of course failing, but it is somewhat expected.

Unfortunately, I was running into this issue because I'm in GMT+1 timezone and I had to hack a solution with find ~ -name _._ -exec touch {} + before running the build.

Would you like me to try and bisect which commit fixed it or can we simply wait for 3.0.1 (assuming it comes before 3.1 releases)?

adaggarwal · 2019-11-11T17:12:19Z

Unfortunately, I was running into this issue because I'm in GMT+1 timezone and I had to hack a solution with find ~ -name _._ -exec touch {} + before running the build.

Interesting. It should have gone as we are ahead of the commit mentioned in this comment.

Would you like me to try and bisect which commit fixed it or can we simply wait for 3.0.1 (assuming it comes before 3.1 releases)?

3.1 lacks some updates from 3.0 and will be updated with the merge PR. 3.0.1 update is pretty close to get in. Won't be a bad idea to try that one out. Even 3.1 would be a good baseline to try; as the merge PR goes in.

Saancreed · 2019-11-11T17:27:42Z

Unfortunately, I was running into this issue because I'm in GMT+1 timezone and I had to hack a solution with find ~ -name _._ -exec touch {} + before running the build.

Interesting. It should have gone as we are ahead of the commit mentioned in this comment.

From what I see, only the master branch of corefx has that commit for now, and current master builds fine without any hacks or additional patches required.

Would you like me to try and bisect which commit fixed it or can we simply wait for 3.0.1 (assuming it comes before 3.1 releases)?

3.1 lacks some updates from 3.0 and will be updated with the merge PR. 3.0.1 update is pretty close to get in. Won't be a bad idea to try that one out. Even 3.1 would be a good baseline to try; as the merge PR goes in.

Then I'm going to try building the release/3.1 of corefx and whatever commits are referenced by this repo's eng/Version.Details.xml for both release/3.0 and release/3.1 branches; I assume that if all of these build fine then 3.0.1 should be fine as well.

adaggarwal · 2019-11-11T17:52:25Z

Then I'm going to try building the release/3.1 of corefx and whatever commits are referenced by this repo's eng/Version.Details.xml for both release/3.0 and release/3.1 branches; I assume that if all of these build fine then 3.0.1 should be fine as well.

Yes. I think we could build once we have that. We can target specific problems we face, if any. And then try to investigate the delta with master that would get us to a successful build. Although, I hope the latter won't be needed.

Saancreed · 2019-11-11T18:45:05Z

Looks like all mentioned commits can be successfully built. What remains now is the zip timestamp issue, but maybe we can work around that by exporting TZ=Etc/UTC before restoring/building.

Small update: Exporting TZ=Etc/UTC before calling build.sh seems to be enough to fix the timestamp issue, assuming that nuking the user's home directory is enough to clear all possible package/config caches. And with extra /usr/bin/ping6 -> ping symlink, all tests that weren't skipped pass successfully as well (on commit 54d63c3e283e0c3937a35ab4e99f391c84ff119e at least).

Saancreed · 2019-11-11T23:41:43Z

Well, looks like my little celebration was a bit premature: even though I can build directly the corefx commit referenced by source-build, building the whole thing still fails at corefx step. The error message is now:

    chmod: cannot access '/build/dotnet/src/source-build/bin/src/corefx.54d63c3e283e0c3937a35ab4e99f391c84ff119e/artifacts/bin/testhost/netcoreapp-Linux-Release-x64/': No such file or directory
  /build/dotnet/src/source-build/bin/src/corefx.54d63c3e283e0c3937a35ab4e99f391c84ff119e/external/runtime/runtime.depproj(65,5): error MSB3073: The command "chmod +x /build/dotnet/src/source-build/bin/src/corefx.54d63c3e283e0c3937a35ab4e99f391c84ff119e/artifacts/bin/testhost/netcoreapp-Linux-Release-x64/" exited with code 1.

And /build/dotnet/src/source-build/bin/src/corefx.54d63c3e283e0c3937a35ab4e99f391c84ff119e/artifacts/bin folder has only the following content (all folders themselves): binplacePackages external.runtime netfx netstandard ref

So, it looks like the problem is somewhere in source-build patches for corefx or arguments/envvars passed by source-build to build.sh. Any ideas how to find the culprit?

Also it appears that passing --capability=CAP_IPC_LOCK to systemd-nspawn is again required, this time probably because of Darc running on pre-3.0 runtime.

tmds · 2019-11-12T09:03:00Z

@Saancreed see #1310 (comment). Patching RuntimeIdentifier (e.g. use linux-x64) in runtime.depproj may help.

Saancreed · 2019-11-12T11:33:10Z

@tmds With <RuntimeIdentifier>linux-x64</RuntimeIdentifier> building corefx now succeeds, but then I have another failure at core-setup with a nice exception:

  /build/dotnet/src/source-build/packages/restored/microsoft.dotnet.build.tasks.packaging/1.0.0-beta.19463.3/build/Packaging.targets(1258,5): error : Error when creating nuget lib package from /build/dotnet/src/source-build/bin/src/core-setup.0bdbe6b3a7caf32a4f594196effe56a941980d7b/artifacts/packages/Release/specs/runtime.arch-x64.Microsoft.NETCore.DotNetAppHost.nuspec. System.IO.DirectoryNotFoundException: Could not find a part of the path '/build/dotnet/src/source-build/bin/src/core-setup.0bdbe6b3a7caf32a4f594196effe56a941980d7b/artifacts/bin/arch-x64.Release/corehost'. [/build/dotnet/src/source-build/bin/src/core-setup.0bdbe6b3a7caf32a4f594196effe56a941980d7b/src/pkg/projects/Microsoft.NETCore.DotNetAppHost/Microsoft.NETCore.DotNetAppHost.pkgproj]

arch-x64.Release contains only two subfolders: crossgen and symbols. Interestingly, there also exists arch.-x64.Release folder in the same location, and that one does have corehost and corehost_test. Looks like another RID computation gone wrong for a rolling release distro.

Update: Yep, this line outputs Computed RID for native build is arch.-x64 because VERSION_ID being unset is not handled correctly here.

Saancreed · 2019-11-12T13:08:37Z

After changing line mentioned above to __rid_plat="$ID${VERSION_ID:+.$VERSION_ID}" building core-setup now succeeds and the next failure is at AspNetCore with error NETSDK1083: The specified RuntimeIdentifier 'arch-x64' is not recognized.

Saancreed · 2019-11-16T00:42:32Z

Okay, looks like I was able to hack together a solution by injecting arch and arch-x64 nodes into RID graphs of bootstrapped SDK/runtime. With this final change, I was able to build everything from current release/3.0 with only 113 warnings. Yay?

Patches and PKGBUILD I used are here. If applying patch to RuntimeIdentifierGraph.json fails, make sure that lines to be inserted have CRLF line endings.

adaggarwal · 2019-11-19T16:20:42Z

Related comment

alucryd · 2019-12-17T17:51:05Z

@Saancreed Thanks for the PKGBUILD, I was able to build the head of release/3.1 using only the core-setup patch, I pushed 3.1.0 to [community-staging] and added you to the contributor list: https://git.archlinux.org/svntogit/community.git/tree/trunk/PKGBUILD?h=packages/dotnet-core

huupoke12 · 2019-12-19T11:51:16Z

@alucryd Shouldn't we keep the version 2.1 instead of 2.2 in the repository? Since 2.1 is LTS but 2.2 is not and its support will end at 2019-12-23.

alucryd · 2019-12-19T20:31:15Z

The 2.2 package is only temporary while emby switches to 3.1 (which is due next release, beta targeting 3.1 has started). Due to the amount of backporting I already need to do to get 2.2 to build, I'm not really keen on trying to package 2.1. This will have to do.

ptrxyz · 2020-01-13T21:01:06Z

@Saancreed Thanks for the PKGBUILD, I was able to build the head of release/3.1 using only the core-setup patch, I pushed 3.1.0 to [community-staging] and added you to the contributor list: https://git.archlinux.org/svntogit/community.git/tree/trunk/PKGBUILD?h=packages/dotnet-core

@SanderSade , @alucryd , It seems this does not work anymore. At least if I install dotnet-sdk from the Arch repo in a clean environment and do a basic testing:

The template "Console Application" was created successfully.

Processing post-creation actions...
Running 'dotnet restore' on /home/ptrxyz/dev/test/test.csproj...
/usr/share/dotnet/sdk/3.1.100/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(88,5): error NETSDK1083: The specified RuntimeIdentifier 'arch-x64' is not recognized. [/home/ptrxyz/dev/test/test.csproj]
/usr/share/dotnet/sdk/3.1.100/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(88,5): error NETSDK1083: The specified RuntimeIdentifier 'arch-x64' is not recognized. [/home/ptrxyz/dev/test/test.csproj]
/usr/share/dotnet/sdk/3.1.100/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(88,5): error NETSDK1083: The specified RuntimeIdentifier 'arch-x64' is not recognized. [/home/ptrxyz/dev/test/test.csproj]

Restore failed.
Post action failed.
Description: Restore NuGet packages required by this project.
Manual instructions: Run 'dotnet restore'

Am I missing something? This is based on version 3.1.0.sdk100-1.

fmauNeko · 2020-01-13T21:09:58Z

@ptrxyz I opened https://bugs.archlinux.org/task/65128 about this.
This is probably due to dotnet/corefx#41750 being pushed back to milestone 5.0.

ptrxyz · 2020-01-13T21:14:44Z

@ptrxyz I opened https://bugs.archlinux.org/task/65128 about this.
This is probably due to dotnet/corefx#41750 being pushed back to milestone 5.0.

Ah right, just saw that. Thanks!

alucryd · 2020-01-18T12:52:18Z

That pull request is now backported into our package, thanks guys.

Saancreed · 2020-03-10T09:58:07Z

It might be a good time to finally close this issue I think — not only .NET Core 3.0 reached end of support a week ago, but we also have succeeded in building 3.0 on Arch and replaced it with 3.1 in our repos some time ago.

adaggarwal self-assigned this Oct 14, 2019

omajid mentioned this issue Oct 23, 2019

Add support for banana linux #297

Closed

4 tasks

crummel added community-support Issues related to community bring up of distros. triaged labels Oct 31, 2019

omajid mentioned this issue Nov 13, 2019

Initial support for FreeBSD. #1362

Merged

Saancreed mentioned this issue Nov 26, 2019

Fix corehost RID generation logic for versionless distros dotnet/runtime#318

Merged

klausenbusk mentioned this issue Dec 5, 2019

editorconfig c# language rules doesn't trigger build failure dotnet/roslyn#33558

Closed

s0dyy mentioned this issue Dec 12, 2019

core-sdk: error The type initializer for 'LibGit2Sharp.Core.NativeMethods' threw an exception #1422

Closed

tmds mentioned this issue Dec 19, 2019

Dotnet 3.1 Images CentOS/container-index#404

Closed

adaggarwal closed this as completed Mar 10, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Building 3.0 on Arch Linux #1310

Building 3.0 on Arch Linux #1310

alucryd commented Oct 12, 2019

omajid commented Oct 12, 2019 •

edited

Loading

alucryd commented Oct 13, 2019

robinchew commented Oct 13, 2019

omajid commented Oct 13, 2019

alucryd commented Oct 13, 2019

omajid commented Oct 13, 2019

alucryd commented Oct 14, 2019

adaggarwal commented Oct 14, 2019

omajid commented Oct 14, 2019 •

edited

Loading

adaggarwal commented Oct 14, 2019 •

edited

Loading

dagood commented Oct 14, 2019

omajid commented Oct 17, 2019 •

edited

Loading

tmds commented Oct 23, 2019

SuddenGunter commented Nov 7, 2019

adaggarwal commented Nov 7, 2019

Saancreed commented Nov 11, 2019

adaggarwal commented Nov 11, 2019

Saancreed commented Nov 11, 2019

adaggarwal commented Nov 11, 2019

Saancreed commented Nov 11, 2019 •

edited

Loading

Saancreed commented Nov 11, 2019 •

edited

Loading

tmds commented Nov 12, 2019

Saancreed commented Nov 12, 2019 •

edited

Loading

Saancreed commented Nov 12, 2019

Saancreed commented Nov 16, 2019

adaggarwal commented Nov 19, 2019 •

edited

Loading

alucryd commented Dec 17, 2019

huupoke12 commented Dec 19, 2019 •

edited

Loading

alucryd commented Dec 19, 2019

ptrxyz commented Jan 13, 2020 •

edited

Loading

fmauNeko commented Jan 13, 2020

ptrxyz commented Jan 13, 2020

alucryd commented Jan 18, 2020

Saancreed commented Mar 10, 2020

Building 3.0 on Arch Linux #1310

Building 3.0 on Arch Linux #1310

Comments

alucryd commented Oct 12, 2019

omajid commented Oct 12, 2019 • edited Loading

alucryd commented Oct 13, 2019

robinchew commented Oct 13, 2019

omajid commented Oct 13, 2019

alucryd commented Oct 13, 2019

omajid commented Oct 13, 2019

alucryd commented Oct 14, 2019

adaggarwal commented Oct 14, 2019

omajid commented Oct 14, 2019 • edited Loading

adaggarwal commented Oct 14, 2019 • edited Loading

dagood commented Oct 14, 2019

omajid commented Oct 17, 2019 • edited Loading

tmds commented Oct 23, 2019

SuddenGunter commented Nov 7, 2019

adaggarwal commented Nov 7, 2019

Saancreed commented Nov 11, 2019

adaggarwal commented Nov 11, 2019

Saancreed commented Nov 11, 2019

adaggarwal commented Nov 11, 2019

Saancreed commented Nov 11, 2019 • edited Loading

Saancreed commented Nov 11, 2019 • edited Loading

tmds commented Nov 12, 2019

Saancreed commented Nov 12, 2019 • edited Loading

Saancreed commented Nov 12, 2019

Saancreed commented Nov 16, 2019

adaggarwal commented Nov 19, 2019 • edited Loading

alucryd commented Dec 17, 2019

huupoke12 commented Dec 19, 2019 • edited Loading

alucryd commented Dec 19, 2019

ptrxyz commented Jan 13, 2020 • edited Loading

fmauNeko commented Jan 13, 2020

ptrxyz commented Jan 13, 2020

alucryd commented Jan 18, 2020

Saancreed commented Mar 10, 2020

omajid commented Oct 12, 2019 •

edited

Loading

omajid commented Oct 14, 2019 •

edited

Loading

adaggarwal commented Oct 14, 2019 •

edited

Loading

omajid commented Oct 17, 2019 •

edited

Loading

Saancreed commented Nov 11, 2019 •

edited

Loading

Saancreed commented Nov 11, 2019 •

edited

Loading

Saancreed commented Nov 12, 2019 •

edited

Loading

adaggarwal commented Nov 19, 2019 •

edited

Loading

huupoke12 commented Dec 19, 2019 •

edited

Loading

ptrxyz commented Jan 13, 2020 •

edited

Loading