Improve static library publishing #70277
Comments
ghost
added
the
|
Yeah, the experience for building static libraries could be better. It's not fully fleshed out. Maybe we could also just generate a lib that merges all the other libs in it. We're not going to document/support this option for .NET 7. I'm moving this to .NET 8. |
|
I have found an interesting approach for composite archives on SO https://stackoverflow.com/a/68407916/863980, which resorts to partial linking. However, it suffers from size regression. It is very likely that I have missing some option which could avoid this regression.. Consider a solution with two C# library projects having Compiling all archives at once like this (note: we need $ gcc -O2 glue.c dist/one.a dist/two.a \
~/.nuget/packages/runtime.linux-x64.microsoft.dotnet.ilcompiler/7.0.0-preview.6.22312.1/sdk/libbootstrapperdll.a ~/.nuget/packages/runtime.linux-x64.microsoft.dotnet.ilcompiler/7.0.0-preview.6.22312.1/sdk/libRuntime.WorkstationGC.a ~/.nuget/packages/runtime.linux-x64.microsoft.dotnet.ilcompiler/7.0.0-preview.6.22312.1/framework/libSystem.Native.a ~/.nuget/packages/runtime.linux-x64.microsoft.dotnet.ilcompiler/7.0.0-preview.6.22312.1/framework/libSystem.Globalization.Native.a ~/.nuget/packages/runtime.linux-x64.microsoft.dotnet.ilcompiler/7.0.0-preview.6.22312.1/framework/libSystem.IO.Compression.Native.a ~/.nuget/packages/runtime.linux-x64.microsoft.dotnet.ilcompiler/7.0.0-preview.6.22312.1/framework/libSystem.Net.Security.Native.a ~/.nuget/packages/runtime.linux-x64.microsoft.dotnet.ilcompiler/7.0.0-preview.6.22312.1/framework/libSystem.Security.Cryptography.Native.OpenSsl.a \
-Wall -pthread -lstdc++ -ldl -lm \
-Wl,--require-defined,NativeAOT_StaticInitialization -Wl,--allow-multiple-definitiongives us an 18 MB binary (a.out). $ gcc -r -o nativeaotruntime.o -Wl,--whole-archive \
~/.nuget/packages/runtime.linux-x64.microsoft.dotnet.ilcompiler/7.0.0-preview.6.22312.1/sdk/libbootstrapperdll.a \
~/.nuget/packages/runtime.linux-x64.microsoft.dotnet.ilcompiler/7.0.0-preview.6.22312.1/sdk/libRuntime.WorkstationGC.a \
~/.nuget/packages/runtime.linux-x64.microsoft.dotnet.ilcompiler/7.0.0-preview.6.22312.1/framework/libSystem.Native.a \
~/.nuget/packages/runtime.linux-x64.microsoft.dotnet.ilcompiler/7.0.0-preview.6.22312.1/framework/libSystem.Globalization.Native.a \
~/.nuget/packages/runtime.linux-x64.microsoft.dotnet.ilcompiler/7.0.0-preview.6.22312.1/framework/libSystem.IO.Compression.Native.a \
~/.nuget/packages/runtime.linux-x64.microsoft.dotnet.ilcompiler/7.0.0-preview.6.22312.1/framework/libSystem.Net.Security.Native.a \
~/.nuget/packages/runtime.linux-x64.microsoft.dotnet.ilcompiler/7.0.0-preview.6.22312.1/framework/libSystem.Security.Cryptography.Native.OpenSsl.a
$ ar cr libnativeaotruntime.a nativeaotruntime.o
$ gcc -O2 glue.c dist/one.a dist/two.a libnativeaotruntime.a \
-Wall -pthread -lstdc++ -ldl -lz -lm \
-Wl,--allow-multiple-definitionthis gives us 22 mb. |
That looks like an antipattern - we really only want one It works, as long as one makes sure the |
|
Turned out the 4 MB size regression is not related to number of archive files, but rather partial linking. Linking a single |
Why not make it a flag? |
|
One way is to combine all the static libs / archives (minus
Considering we can overcome that 4 MB size regression (#70277 (comment)), we won't be needing most of the individual static libs in the package ( |
|
Sort of related, but I am having issues linking with some of the stuff from the localization library. I added From my preliminary research, it seems all of these symbols are specific to MacOS? If that is the case, I am more inclined to believe its an architecture support issue. Any help is much appreciated. UPDATE: I think I figured it out (didn't pass the framework argument). Ill leave this here just in case. Thank you again for your hard work making NativeAOT possible everyone! |
|
Things have changed a bit in .NET 8. Here are the new command for daily build (net8.0 preview 6, osx-arm64): Invariant globalization# using code from top post
AOTBASE=~/.nuget/packages/runtime.osx-arm64.microsoft.dotnet.ilcompiler/8.0.0-preview.6.23279.6
dotnet publish -c Release --use-current-runtime -o dist \
-p:PublishAot=true \
-p:NativeLib=static \
-p:EnableNativeEventPipe=false \
-p:AllowUnsafeBlocks=true \
-p:InvariantGlobalization=true
cc test.c dist/lib1.a \
$AOTBASE/sdk/libbootstrapperdll.a \
$AOTBASE/sdk/libRuntime.WorkstationGC.a \
$AOTBASE/sdk/libstdc++compat.a \
$AOTBASE/sdk/libeventpipe-disabled.a \
$AOTBASE/framework/libSystem.Native.a \
$AOTBASE/framework/libSystem.IO.Compression.Native.a \
$AOTBASE/framework/libSystem.Net.Security.Native.a \
$AOTBASE/framework/libSystem.Security.Cryptography.Native.OpenSsl.a \
-Wall -pthread -ldl -lm -Wl,-u,_NativeAOT_StaticInitializationWith globalization# using code from top post
AOTBASE=~/.nuget/packages/runtime.osx-arm64.microsoft.dotnet.ilcompiler/8.0.0-preview.6.23279.6
dotnet publish -c Release --use-current-runtime -o dist \
-p:PublishAot=true \
-p:NativeLib=static \
-p:EnableNativeEventPipe=false \
-p:AllowUnsafeBlocks=true
cc test.c dist/lib1.a \
$AOTBASE/sdk/libbootstrapperdll.a \
$AOTBASE/sdk/libRuntime.WorkstationGC.a \
$AOTBASE/sdk/libeventpipe-disabled.a \
$AOTBASE/framework/libSystem.Native.a \
$AOTBASE/framework/libSystem.IO.Compression.Native.a \
$AOTBASE/framework/libSystem.Globalization.Native.a \
$AOTBASE/framework/libSystem.Net.Security.Native.a \
$AOTBASE/framework/libSystem.Security.Cryptography.Native.OpenSsl.a \
-Wall -pthread -ldl -lm -lstdc++ -Wl,-u,_NativeAOT_StaticInitialization \
-framework Foundation |
|
Well I got it to compile with all the extra flags but it segfaults, any ideas? It seems to segfault at RuntimeInstance which sounds managed |
|
You seem to be missing $ uname -v
Darwin Kernel Version 22.2.0: Fri Nov 11 02:03:51 PST 2022; root:xnu-8792.61.2~4/RELEASE_ARM64_T6000
$ ./a.out
Hello from C# Add!
Hello from C callback!
sum is: 3 |
Just added it and can confirm it ends up getting passed in to the linker correctly. Still having the same issue.
|
this part should be: and drop |
should be: |
|
Tried that, but I get: Wait, I apologize. It seems it injected a whitespace somehow. Working with rust is honestly super challenging because its very hard to get consistent results when trying to solve issues theres the new output with your recommended argument command |
Instead, try With compiler driver (clang, gcc etc.), |
|
Thank you SOOOO much! I have spent the last 2 days trying to get this to work. Really appreciate your help! |
|
Hey, unfortunately, I'm back. I rewrote my FFI library like 3 times to try to see if I simply had my types wrong for FFI, and was able to make some progress. I managed to get some basic passing of data and function calls to work, and my tests in my FFI crate all seem to pass without incident. Unfortunately, when I move the exact same code in the unit test over to a separate binary, it falls apart. I get the aforementioned error, despite linking with all the desired flags and arguments suggested above. Specifically, this method tries to load a register with a null pointer causing a segfault. If anyone could shed some light into the specific circumstances in which this might happen, it is possible I would be able to diagnose the issue further. I am suspicious of my linker because of this errors ties to previous missing linker arguments, but I am not convinced because I am passing the same arguments in both cases. As usual any help is much appreciated, UPDATE: I believe I may understand the problem, but not the solution. I think rust when compiling the unit test will compile directly against the static library with all of the linker args, whereas the intermediate crate seems to just use a I filed an issue for rust over here for anyone else experiencing similar issues: |
|
Turns out it was some strange undocumented behavior where rust did not properly transient linker arguments, ontop of the fact that Adding the linker args, I now get a segfault somewhere else in my code, but only when integrating with a specific existing crate: Seems very related to: @am11 , Any ideas on how to solve this? I saw you actively involved in the other issue so I figured you might be able to help. I am on |
|
This needs further debugging to find out what's going on. With Rust in the mix, it is hard to pinpoint the issue. Try creating a standalone C/C++ and C# sources, if it works then it's the issue with Rust integration. I'm on osx-arm64 and .NET 8 daily build ( If you have a repro project that will help the investigation. |
|
Ah, I have a suspicion that I am actually not using the latest preview of NativeAOT despite it being installed. I added that to my |
There is the full stack trace if it helps. Rust is not doing anything but calling the function that returns nothing,
from: https://github.com/dotnet/runtime/blob/main/src/coreclr/inc/corinfo.h |
You can create a new project:
If you could isolate the reproducible project (preferably one .cs file and a .c file) and make an SSCCE out of it, that would make the investigation go faster. Feel free to post your findings in a new issue (since it is getting a bit off-topic in this thread). ps - currently I'm looking into #87333 (ongoing native AOT issue on osx-arm64 in |
|
Its not a single C# file, but with: namespace Bepu {
extern "C" void SetupPyramidDemo();
}
int main() {
Bepu::SetupPyramidDemo();
return 0;
}Reproduces the issue |
|
With: $ cd bepuvy-bepu
$ dotnet8 publish -c Release --use-current-runtime -o dist \
-p:PublishAot=true \
-p:NativeLib=static \
-p:EnableNativeEventPipe=false
$ cat > glue.cpp <<EOF
namespace Bepu {
extern "C" void SetupPyramidDemo();
}
int main() {
Bepu::SetupPyramidDemo();
return 0;
}
EOF
$ AOTBASE=~/.nuget/packages/runtime.osx-arm64.microsoft.dotnet.ilcompiler/8.0.0-preview.6.23307.4
$ cc glue.cpp dist/Bepuvy.a \
$AOTBASE/sdk/libbootstrapperdll.a \
$AOTBASE/sdk/libRuntime.WorkstationGC.a \
$AOTBASE/sdk/libeventpipe-disabled.a \
$AOTBASE/framework/libSystem.Native.a \
$AOTBASE/framework/libSystem.IO.Compression.Native.a \
$AOTBASE/framework/libSystem.Globalization.Native.a \
$AOTBASE/framework/libSystem.Net.Security.Native.a \
$AOTBASE/framework/libSystem.Security.Cryptography.Native.OpenSsl.a \
-Wall -pthread -ldl -lm -lstdc++ -Wl,-u,_NativeAOT_StaticInitialization \
-framework Foundationit works: $ ./a.out
$ echo $?
0 |
|
Huh, your reproduction works. I will note that the linker error I got earlier with the latest preview was associated with a github issue you recently posted. I will take it from here, and appreciate your help on the matter. |
|
Note: when we omit # publish with invariant culture: -p:InvariantGlobalization=true
$ cc glue.cpp dist/Bepuvy.a \
$AOTBASE/sdk/libbootstrapperdll.a \
$AOTBASE/sdk/libRuntime.WorkstationGC.a \
$AOTBASE/sdk/libstdc++compat.a \
$AOTBASE/sdk/libeventpipe-disabled.a \
$AOTBASE/framework/libSystem.Native.a \
$AOTBASE/framework/libSystem.IO.Compression.Native.a \
$AOTBASE/framework/libSystem.Net.Security.Native.a \
$AOTBASE/framework/libSystem.Security.Cryptography.Native.OpenSsl.a \
-Wall -pthread -ldl -lm -Wl,-u,_NativeAOT_StaticInitialization
# publish without `-p:InvariantGlobalization=true` line
$ cc glue.cpp dist/Bepuvy.a \
$AOTBASE/sdk/libbootstrapperdll.a \
$AOTBASE/sdk/libRuntime.WorkstationGC.a \
$AOTBASE/sdk/libeventpipe-disabled.a \
$AOTBASE/framework/libSystem.Native.a \
$AOTBASE/framework/libSystem.IO.Compression.Native.a \
$AOTBASE/framework/libSystem.Globalization.Native.a \
$AOTBASE/framework/libSystem.Net.Security.Native.a \
$AOTBASE/framework/libSystem.Security.Cryptography.Native.OpenSsl.a \
-Wall -pthread -ldl -lm -lstdc++ -Wl,-u,_NativeAOT_StaticInitialization \
-framework Foundation |
|
Yeah. I am still not totally sure what is going on but it seems something to do with how rust handles linking. Rust seems to have problems with it despite being fed the same linker arguments. println!("cargo:rustc-link-lib=objc");
println!("cargo:rustc-link-lib=swiftCore");
println!("cargo:rustc-link-lib=swiftFoundation");
println!("cargo:rustc-link-lib=icucore");
println!("cargo:rustc-link-search=/usr/lib/swift")I think it might come down to this being different than I am still not totally sure what is going on but it seems something to do with how rust handles linking. Rust seems to have problems with it despite being fed the same linker arguments. Theres my build.rs script. If you create a new rust binary, apply the extern "C" {
fn SetupPyramid();
}
fn main() {
SetupPyramid();
} it will reproduce the issue. |
|
I finally seemed to have figured it out. For some really obscene reason the version of Theres some silent difference between the linker Apple ships with Mac and the one rust ships (or mold for that matter). Was never able to figure out the difference but it has to do with Thank you again with your extensive help on this long journey and I appreciate everything you have done for me. |
|
Just wanted to thank you @DrewRidley for sharing the details. I'm working on a new design for a poc and this thread helped!. |
|
No worries, if it helps I found out the problem was with rust, although I'm not really sure how changing the linker changed the behavior. Rust apparently strips unused symbols by default but this is poorly documented and not well known. |
|
I'll keep an eye on that. I have a scenario where I'd like to call c# code from Rust and there are just too many footguns lying around :) |
|
How about Windows? There aren't *.lib (for example: System.Native.lib) in |
|
@taodongl That sounds like static library support for Native AOT not working on Windows. I am nowhere near capable of giving it a go to confirm your situation but if that is that is the case I'm sure the team would appreciate a dedicated issue 😃 Though, I would be surprised if this was the case with 8.0 final release, but what do I know? |
There must be *.lib files or building EXE files wouldn't work either (we pass the same LIB files to the linker). Did you try following these instructions to get the list of paths to pass to linker? ("You can find a list of additional framework libraries by publishing the project as shared library (/p:NativeLib=Shared) with detailed verbosity (-v d), and looking at the output generated by the LinkNative target.") |
jkotas
changed the title
|
Another .NET 8 update. Since #89291, it's not necessary to add any linker flags regarding |
|
@MichalStrehovsky it would be useful if some msbuild item contained the list of static libraries and objects (either relative to and |
|
@lambdageek, I was thinking emitting a "helper script" file with |
|
Hey all, back here again unfortunately, I managed to follow the same steps from before, explicitly adding all the required paths for the static libraries needed, but am prompted with these internal symbols being undefined at link time. I had a peek around and found some Any help is much appreciated. Thanks |
These instructions should guide you to what's needed - first publish as a shared library with diagnostic verbosity and then find the required arguments to pass to clang in the verbose log of LinkNative target. |
dotnet-policy-service
bot
added
the
in-pr
Setup:
Compiling
test.cwithdist/lib1.ais not straightforward. I figured it out using hints from-p:NativeLib=shared -v:diag. This is how the working command looks like:I think we should copy all the required
.afiles into the publish directory and create a helper script (.sh) for the user.The text was updated successfully, but these errors were encountered: