MacCatalyst support

[ericsink]

the app works on iOS but crashes on maccatalyst

Originally posted by @VladislavAntonyuk in #468 (comment)

[ericsink]

Not sure yet how to deal with the native SQLite libraries. Do we need a special build, or does the iOS build or the macOS build work?

[edoust]

Are there any news on this, or is there any workaround?
I can't get EntityFramework to run on net6.0-maccatalyst, it does run on net6.0-macos (xamarin20)

[ericsink]

No progress on this yet. I need to figure out how to compile the C code for maccatalyst.

For example, the command line for iphone starts out like this:

xcrun --sdk iphoneos clang -O -miphoneos-version-min=6.0 -arch arm64 

And for macos:

xcrun --sdk macosx clang -dynamiclib -O -arch x86_64 

I assume it's similar to these, but I haven't spent any time yet looking for the answer.

[edoust]

Is there a documentation on how you did it for ios (end to end)? If not can you tell me on which code/ repo exactly you ran the above mentioned command?

Would like to have a quick look myself, since I need SQLite in maccatalyst

[ericsink]

There are no docs, but my build scripts for the native SQLite libraries are over in the ericsink/cb repo.

[VladislavAntonyuk]

it seems to be working fine now:

you can try KanbanBoard and Sqlite samples from https://github.com/VladislavAntonyuk/MauiSamples

Important note: Run the app from Visual Studio. for some reason CLI doesn't work

[ericsink]

@VladislavAntonyuk I haven't looked closely at your samples yet, but just to be clear for anybody else seeing this -- there is currently nothing in SQLitePCLRaw and nothing in the e_sqlite3 build scripts that is specifically for maccatalyst support. If anything works with net6.0-maccatalyst, it is an accident, and I don't know why it works. I recently did a bunch of stuff to get the other net6.0-workloads going in the 2.1-prerelease versions, but nothing for maccatalyst.

I do plan to give this issue some real attention as soon as I can.

[VladislavAntonyuk]

@ericsink seems like Apple did a great job to make iOS code work on macOS with maccatalyst, But definitely waiting for your final release with official maccatalyst support. Thank you!

[ericsink]

seems like Apple did a great job to make iOS code work on macOS with maccatalyst

Maybe so, but I thought I saw somewhere that things should be specifically compiled for maccatalyst. I need to confirm whether that's true. If something seems to work when the docs and specs say it should not, then I wonder why.

And if something works when run from Visual Studio but not from the CLI, then it doesn't work.

:-)

[VladislavAntonyuk]

I didn't investigate it, but probably VS restores packages differently and packs SQLite iOS lib in MacCatalyst. it's the only 1 idea that comes to my mind so far.

[edoust]

@VladislavAntonyuk For me building in VS does not work at all, what version of VS are you using? Are you on an x86 or ARM machine?

[VladislavAntonyuk]

VS for mac 2022. Intel

[edoust]

I always get the error clang++ exited with code 1 in Xamarin.Shared.Sdk.targets, that error is reproducible on two machines that are freshly installed, one with an M1 and one with an Intel processor

Both my machines have VS2022 (latest), Xamarin.Mac (latest) and the latest maui-maccatalyst workload installed

Do you have any special configuration on your system that might be related to this?

[VladislavAntonyuk]

Check XCode version.
Check MAUI version. Try install it again sudo dotnet workload install maui.
Clean bin/obj
First run the build with dotnet cli
It should build the app and restore packages
Then open VS, select your mac to run the app and click Restore nuget packages on solution
Run build from VS.
the app may ask for the access to the storage

[edoust]

@VladislavAntonyuk Can you please share your diagnostic build log (especially the xcrun xlang command from the LinkNativeCode target)? I found that in VS build this call fails, so I am interested to know against what sqlite lib you are linking

This is the error message I get: ld: in [...]/obj/Debug/net6.0-maccatalyst/maccatalyst-x64/linker-cache/e_sqlite3.a(sqlite3.o), building for Mac Catalyst, but linking in object file built for iOS Simulator, file '[...]/obj/Debug/net6.0-maccatalyst/maccatalyst-x64/linker-cache/e_sqlite3.a' for architecture x86_64

For RID maccatalyst-arm64 it says the lib was built for iOS not iOS Simulator

@ericsink Thanks for hinting at the cb repo, I could slightly modify your script to build a MacCatalyst compatible version of e_sqlite.a. Currently I can manually copy that file to the linker-cache folder, but I need this to work for CLI (agent) builds as well

Which repo is the base for the NuGet that will contain the maccatalyst support once you publish it? I would like to fork it and create a private NuGet for my CLI (agent) build, since I need EF Core to work on maccatalyst as soon as possible

[VladislavAntonyuk]

BuildLogs.txt

[edoust]

Thanks you, but this log does not contain the clang commands since it was not a full rebuild, so those targets were skipped. Can you please do a full rebuild after having deleted bin and obj folders and only building the target framework maccatalyst?

Also, please share the e_sqlite.a file that is located in obj/Debug/net6.0-maccatalyst/maccatalyst-x64/linker-cache. I would like to check whether you get a library built for maccatalyst already, and if so where that file originally came from

[VladislavAntonyuk]

1. Delete obj/bin
2. Run VS and restore packages and build app. Output: BuildLogs.txt
3. Run dotnet build -t:Run -f net6.0-ios
4. Run dotnet build -t:Run -f net6.0-maccatalyst
   output:
   BuildLogs.txt
5. Back to VS and run the app.

So from what I see it is important the iOS libs appear in maccatalyst folder, otherwise the app crashes and can't find e_sqlite3

[edoust]

Thanks for sharing the log files. From my understanding in order to be able to use the (iOS-)OS provided SQLite implementation, it would require some reference to it, for example by importing the SQLite header file into a referenced (native) library and actually using its methods. I would assume that SQLitePCLRaw.provider.dynamic_cdecl is a passthrough that relays all calls to e_sqlite3 to the OS-provided version of SQLite (??)

But then I still don't get how this would work with MacCatalyst, since as far as I know MacCatalyst is not just emulating an iOS environment (like x86 env may be emulated on ARM) but it requires a dedicated build for maccatalyst

• Do you get a running .app file in the end? If so, can you copy that to some other folder and start by double-clicking it?
• Can you please try what happens if you set only macatalyst as target framework and then clean? Does it build & run then?

On my end I got the app working using the CLI by doing the following steps:

1. Create a maccatalyst build of SQLite using the scripts provided by @ericsink
2. Referencing the generated libe_sqlite3.dylib with a <NativeReference Include="/.../libe_sqlite3.dylib" Kind="Dynamic" />
3. Build using CLI dotnet build -t:restore,build KanbanBoard.csproj -f net6.0-maccatalyst

The resulting .app packages work for both the maccatalyst-x64 and maccatalyst-arm64 RIDs, also both contain the libe_sqlite.dylib file (~2MB) inside the package

[VladislavAntonyuk]

as @ericsink mentioned this lib so far works only with net6.0-ios.

SQLitePCL.raw/src/SQLitePCLRaw.bundle_e_sqlite3/SQLitePCLRaw.bundle_e_sqlite3.csproj

Line 4 in 9d5a8b6

<TargetFrameworks>netstandard2.0;net6.0-ios;net6.0-android;xamarin.ios10;$(monoandroid_tfm);net461;net6.0-tvos10</TargetFrameworks>

there is no net6.0-maccatalyst and net6.0-mac target frameworks

[edoust]

For Catalyst it should do a fallback to netstandard2.0. I was just wondering about your logs not showing a native compilation/linking process, so I just don't see how you would get a runnable (deployable) macOS .app package out of it

For me it is working now using a privately published NuGet so I will use that approach going forward

[edoust]

@ericsink

Could you already have a look into rebuilding the NuGet packages?
Do you think integrating maccatalyst requires code changes, or should this be done by integrating the dylib files into the lib project and then compiling the bundle project with maccatalyst support?

[ericsink]

@edoust I think what you said is sufficient. I've made those changes and pushed them up. However, the CI builds are currently failing for other reasons. It looks like something in the net6.0-ios SDK broke with this week's preview release. I am investigating.

[ericsink]

Looks like the problem is xamarin/xamarin-macios/issues/14434

[ericsink]

Version 2.1.0-pre20220318192836 has been pushed up to nuget. It contains progress on maccatalyst support.

I tried a very basic test and it seems to work.

One caveat: I placed the native binaries under net6.0-maccatalyst15.2, so it is necessary to target at least that version in order to get the dylib into the build. If I just set TargetFramework to net6.0-maccatalyst (without specifying the version), or if I specify a version below 15.2, the resulting app crashes because it cannot find the dylib. I suspect I don't need to specify a minimum version that high, so another fix is probably needed here.

[VladislavAntonyuk]

it works fine. I set TargetFramework to net6.0-maccatalyst and min version to 15.2

[edoust]

For me it does not work automatically, I need to add a NativeReference explicitly, or else the libe_sqlite.dylib is not put in the app package (I can only use test using the CLI, since VS2022 for mac preview 7 does not support .NET 6 Xamarin projects at the moment)

[coolbluewater]

Thanks. I had installed Xcode 13.3.0 and had to downgrade to 13.2.1 because of a bug in Xcode. Will try to update to Xcode 13.3.1. I am running on x64, so fingers crossed.

[coolbluewater]

@edoust, what command line did you use to install the maui preview workload? I used this one from the dotnet/maui wiki:

sudo dotnet workload install maui --from-rollback-file https://aka.ms/dotnet/maui/rc.2.json --source https://aka.ms/dotnet6/nuget/index.json --source https://api.nuget.org/v3/index.json

However this gives an error that the android sdk manifest is not found:

Workload installation failed: Failed to install manifest microsoft.net.sdk.android version 32.0.300-rc.2.27: microsoft.net.sdk.android.manifest-6.0.200::32.0.300-rc.2.27 is not found in NuGet feeds https://aka.ms/dotnet6/nuget/index.json;https://api.nuget.org/v3/index.json"..

I couldn't find an open issue regarding this, and wondered if you ran into something similar.

[edoust]

I use sudo dotnet install workload maccatalyst since I only compile for Catalyst using the Uno framework...

I use this custom nuget.config (rename to nuget.config) and then execute the command in the context of that nuget config file

[coolbluewater]

@edoust Thanks - what is the source of that nuget config? Perhaps I'll find more info there about installing the maui preview...

[edoust]

The nuget file is not MAUI related it just contains the global feeds for dotnet and xamarin... Looks like if you specify the rc.2 json you also need more feeds, see here...

Also it seems like an rc.4 rollback file is available here

[VladislavAntonyuk]

use this guide to install it https://github.com/dotnet/maui/wiki/macOS-Install

[coolbluewater]

@edoust @VladislavAntonyuk Thanks guys, but I'm having no luck with installing either rc2 or rc4. Have you actually succeeded installing either maui rc2 or rc4? Specifically, the install halts at the first item (the android sdk) and says that it cannot find the mentioned version.

I've tried this to install rc2 using the instructions at https://github.com/dotnet/maui/wiki/macOS-Install:

dotnet workload install maui --from-rollback-file https://aka.ms/dotnet/maui/6.0.300/rc.2.json --source https://pkgs.dev.azure.com/dnceng/public/_packaging/darc-pub-dotnet-runtime-a21b9a2d/nuget/v3/index.json --source https://pkgs.dev.azure.com/dnceng/public/_packaging/darc-pub-dotnet-emsdk-572aeedc/nuget/v3/index.json --source https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet6/nuget/v3/index.json --source https://api.nuget.org/v3/index.json

And then I tried replacing the rollback file with the url given by @edoust for rc4:

sudo dotnet workload install maui --from-rollback-file https://maui.blob.core.windows.net/metadata/rollbacks/rc.4.json --source https://pkgs.dev.azure.com/dnceng/public/_packaging/darc-pub-dotnet-runtime-a21b9a2d/nuget/v3/index.json --source https://pkgs.dev.azure.com/dnceng/public/_packaging/darc-pub-dotnet-emsdk-572aeedc/nuget/v3/index.json --source https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet6/nuget/v3/index.json --source https://api.nuget.org/v3/index.json

But neither work - it looks like the rollback file and the feeds must be mutually compatible, i.e. the feeds need to have items with the versions mentioned in the rollback file. Any ideas?

[VladislavAntonyuk]

I use rc2, because it is the latest released version.
I know about upcoming rc3 very soon.

try to uninstall all workloads and install MAUI again

[coolbluewater]

@VladislavAntonyuk - could you please list the command line for installing maui rc2?

[edoust]

@coolbluewater
I just tested and using this nuget config it works

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <packageSources>
    <clear />
    <add key="dotnet6" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet6/nuget/v3/index.json" />
    <add key="xamarin" value="https://pkgs.dev.azure.com/azure-public/vside/_packaging/xamarin-impl/nuget/v3/index.json" />
    <add key="public"  value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-public/nuget/v3/index.json" />
    <add key="xx" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/darc-pub-dotnet-runtime-a21b9a2d/nuget/v3/index.json"/>
    <add key="xy" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/darc-pub-dotnet-emsdk-572aeedc/nuget/v3/index.json" />
    <add key="xz" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet6/nuget/v3/index.json" />
  </packageSources>
</configuration>

Then in the same directory of the nuget config execute the command sudo dotnet workload install maui --from-rollback-file https://maui.blob.core.windows.net/metadata/rollbacks/rc.4.json or whichever rollback file you choose

[coolbluewater]

@VladislavAntonyuk - that goes in a file named nuget.config, correct?
I ran the command you listed and unfortunately get the same error as I previously got:

Skip NuGet package signing validation. NuGet signing validation is not available on Linux or macOS https://aka.ms/workloadskippackagevalidation .
Updated advertising manifest microsoft.net.sdk.android.
Updated advertising manifest microsoft.net.sdk.tvos.
Updated advertising manifest microsoft.net.sdk.maui.
Updated advertising manifest microsoft.net.sdk.ios.
Updated advertising manifest microsoft.net.sdk.maccatalyst.
Updated advertising manifest microsoft.net.workload.mono.toolchain.
Updated advertising manifest microsoft.net.sdk.macos.
Updated advertising manifest microsoft.net.workload.emscripten.
Installing workload manifest microsoft.net.sdk.android version 32.0.300-rc.3.45…
Workload installation failed. Rolling back installed packs...
Installing workload manifest microsoft.net.sdk.android version 31.0.200-preview.14.106…
Installation rollback failed: Failed to install manifest microsoft.net.sdk.android version 31.0.200-preview.14.106: The transaction has aborted..
Workload installation failed: Failed to install manifest microsoft.net.sdk.android version 32.0.300-rc.3.45: microsoft.net.sdk.android.manifest-6.0.200::32.0.300-rc.3.45 is not found in NuGet feeds https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet6/nuget/v3/index.json;https://pkgs.dev.azure.com/azure-public/vside/_packaging/xamarin-impl/nuget/v3/index.json;https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-public/nuget/v3/index.json;https://pkgs.dev.azure.com/dnceng/public/_packaging/darc-pub-dotnet-runtime-a21b9a2d/nuget/v3/index.json;https://pkgs.dev.azure.com/dnceng/public/_packaging/darc-pub-dotnet-emsdk-572aeedc/nuget/v3/index.json;https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet6/nuget/v3/index.json"..

[edoust]

Do you have dotnet 6.0.300 installed? can you check by running dotnet --info?

[VladislavAntonyuk]

You can take a look how we install dotnet and Maui workload https://github.com/CommunityToolkit/Maui/blob/main/azure-pipelines.yml

[coolbluewater]

@VladislavAntonyuk @edoust - thanks! I had moved back and forth between runtimes and was on a different version. After installing 6.0.300 I was able to install maui rc4 using the above steps.

[coolbluewater]

@ericsink I built my app for maccatalyst15.4 (I think!), but still no sqlite love. Are you able to reproduce the failure?

[ericsink]

@coolbluewater Are you talking about above, where it works on the command line but not in the IDE? If so, no, I haven't found time to install VS for Mac and try it yet.

[VladislavAntonyuk]

I can just add that VS4Mac does not fully support MAUI. It will be added in 17.1.

[coolbluewater]

@VladislavAntonyuk - Yes, that's correct. Do we know whether sqlite works using Visual Studio for Windows, which fully supports maui?

[VladislavAntonyuk]

Just checked, it works on Windows.
To be honest it works on VS4Mac as well. You only need to restore packages using cli.

[coolbluewater]

@VladislavAntonyuk - thanks! Which packages? Any special instructions to restore? VsMac keeps restoring packages a lot. Do you see the same behavior?

[VladislavAntonyuk]

I first run dotnet restore. Than open VS4MAC and run build. In that case VS4MAC doesn't restore packages again. Seems like VS treats iOS and Mac catalyst as the same target framework during package restore.

Also it works on CI build because it uses dornet cli

[coolbluewater]

@VladislavAntonyuk thanks! Do you still need to change the os version for the maccatalyst target?

[VladislavAntonyuk]

yes, 15.2 is the minimum supported version

[ericsink]

Version 2.1.0 is on nuget. Any further maccatalyst-specific problems, please open a new issue.

[edoust]

I am still having the exact same issue with v2.1.0... :(

[ericsink]

I believe the latest status of this was that it works fine on the command line but doesn't work with Visual Studio for Mac. @edoust Is that true for you?

If so, that means your problem is possibly related to #489

[edoust]

No, I am doing a CLI build with this command - it does not change the problem whether I do it in VS or using dotnet

dotnet build %(SolutionFile) --framework net6.0-maccatalyst15.4

Will keep an eye on it, for now I am still using your preview package alongside the custom nuget that contains your native lib

[ericsink]

@edoust If you can construct a minimal repro project, please open a new issue and post it there.

[edoust]

Yes, will do it these days