Issues while using efcore with NativeAOT #733

hez2010 · 2021-02-23T05:12:09Z

After efcore fixed infinite recursive generics, I hit this exception while trying to use efcore daily build with SQLite using NativeAOT:

      System.InvalidOperationException: Operation is not valid due to the current state of the object.
         at System.SharedTypeExtensions.GetRequiredRuntimeMethod(Type, String, Type[]) + 0x57
         at Microsoft.EntityFrameworkCore.Sqlite.Query.Internal.SqliteDateTimeAddTranslator..ctor(ISqlExpressionFactory) + 0xd2
         at Microsoft.EntityFrameworkCore.Sqlite.Query.Internal.SqliteMethodCallTranslatorProvider..ctor(RelationalMethodCallTranslatorProviderDependencies) + 0xb0

A simple repro project:

efaottest.zip

The text was updated successfully, but these errors were encountered:

MichalStrehovsky · 2021-02-23T09:01:39Z

Looking at the code, we probably need RD.XML to keep all methods on DateTime:

https://github.com/dotnet/efcore/blob/da00fb69d615fa22a83dfee2077ad31b7bd15823/src/EFCore.Sqlite.Core/Query/Internal/SqliteDateTimeAddTranslator.cs#L33-L38

Hopefully these places will be fixed in EF as they're working on dotnet/efcore#21894.

If this reflection wasn't wrapped in the GetRequiredRuntimeMethod helper, the compiler would actually figure this out on it's own (typeof(DateTime).GetMethod("Foo", ...) is analyzed by the compiler).

MichalStrehovsky · 2021-02-23T09:04:18Z

You can add <SuppressTrimAnalysisWarnings>false</SuppressTrimAnalysisWarnings> to a PropertyGroup of your CSPROJ and the compiler will warn whenever there's reflection that can't be analyzed. I expect you'll see a warning for GetRequiredRuntimeMethod. It produces a lot of noise though.

hez2010 · 2021-02-23T13:29:50Z

Thanks @MichalStrehovsky .
After some investigation, I hit this but don't know how to deal it with rd.xml:

Generic virtual method pointer lookup failure.

Declaring type handle: EEType:0x00007FF62A2E8E90
Target type handle: EEType:0x00007FF629DB3428
Method name: CreateGeneric
Instantiation:
  Argument 00000000: EEType:0x00007FF629DD46F0
  Argument 00000001: EEType:0x00007FF629D44BC0
  Argument 00000002: EEType:0x00007FF629D44BC0

My rd.xml:

<Directives>
    <Application>

        <Assembly Name="Microsoft.EntityFrameworkCore">
            <Type Name="Microsoft.EntityFrameworkCore.ChangeTracking.EntryCurrentValueComparer`1[[System.Int32, System.Private.CoreLib]]" Dynamic="Required All" />
            <Type Name="Microsoft.EntityFrameworkCore.ChangeTracking.ValueComparer+DefaultValueComparer`1[[System.Int32, System.Private.CoreLib]]" Dynamic="Required All" />
            <Type Name="Microsoft.EntityFrameworkCore.Metadata.Internal.ClrAccessorFactory`1[[Microsoft.EntityFrameworkCore.Metadata.Internal.ClrPropertyGetter`2[[efaottest.Table1, efaottest], [System.Int32, System.Private.CoreLib]], Microsoft.EntityFrameworkCore]]" Dynamic="Required All">
                <Method Name="CreateGeneric" Dynamic="Required All">
                    <GenericArgument Name="efaottest.Table1, efaottest" />
                    <GenericArgument Name="System.Int32, System.Private.CoreLib" />
                    <GenericArgument Name="System.Int32, System.Private.CoreLib" />
                </Method>
            </Type>
        </Assembly>

        <Assembly Name="mscorlib">
            <Type Name="System.DateTime" Dynamic="Required All">
                <Method Name="AddYears" Dynamic="Required" />
                <Method Name="AddMonths" Dynamic="Required" />
                <Method Name="AddDays" Dynamic="Required" />
                <Method Name="AddHours" Dynamic="Required" />
                <Method Name="AddMinutes" Dynamic="Required" />
                <Method Name="AddSeconds" Dynamic="Required" />
            </Type>
        </Assembly>

    </Application>
</Directives>

Update:

fixed by adding:


            <Type Name="Microsoft.EntityFrameworkCore.Metadata.Internal.ClrPropertyGetterFactory" Dynamic="Required All">
                <Method Name="CreateGeneric" Dynamic="Required All">
                    <GenericArgument Name="efaottest.Table1, efaottest" />
                    <GenericArgument Name="System.Int32, System.Private.CoreLib" />
                    <GenericArgument Name="System.Int32, System.Private.CoreLib" />
                </Method>
            </Type>

hez2010 · 2021-02-24T03:20:11Z

Now I'm getting GVM resolution failure with below rd.xml:

<Directives>
    <Application>

        <Assembly Name="Microsoft.EntityFrameworkCore">
            <Type Name="Microsoft.EntityFrameworkCore.ChangeTracking.EntryCurrentValueComparer`1[[System.Int32, System.Private.CoreLib]]" Dynamic="Required All" />
            <Type Name="Microsoft.EntityFrameworkCore.ChangeTracking.ValueComparer+DefaultValueComparer`1[[System.Int32, System.Private.CoreLib]]" Dynamic="Required All" />
            <Type Name="Microsoft.EntityFrameworkCore.Metadata.Internal.ClrAccessorFactory`1[[Microsoft.EntityFrameworkCore.Metadata.Internal.ClrPropertyGetter`2[[efaottest.Table1, efaottest],[System.Int32, System.Private.CoreLib]],Microsoft.EntityFrameworkCore]]" Dynamic="Required All">
                <Method Name="CreateGeneric" Dynamic="Required All">
                    <GenericArgument Name="efaottest.Table1, efaottest" />
                    <GenericArgument Name="System.Int32, System.Private.CoreLib" />
                    <GenericArgument Name="System.Int32, System.Private.CoreLib" />
                </Method>
            </Type>
            <Type Name="Microsoft.EntityFrameworkCore.Metadata.Internal.ClrPropertyGetterFactory" Dynamic="Required All">
                <Method Name="CreateGeneric" Dynamic="Required All">
                    <GenericArgument Name="efaottest.Table1, efaottest" />
                    <GenericArgument Name="System.Int32, System.Private.CoreLib" />
                    <GenericArgument Name="System.Int32, System.Private.CoreLib" />
                </Method>
            </Type>
            <Type Name="Microsoft.EntityFrameworkCore.ChangeTracking.Internal.Snapshot`1[[System.Int32,System.Private.CoreLib]]" Dynamic="Required All" />
            <Type Name="Microsoft.EntityFrameworkCore.ChangeTracking.Internal.Snapshot`2[[System.Int32,System.Private.CoreLib],[System.String,System.Private.CoreLib]]" Dynamic="Required All" />
            <Type Name="Microsoft.EntityFrameworkCore.ChangeTracking.Internal.IdentityMapFactoryFactory" Dynamic="Required All">
                <Method Name="CreateFactory" Dynamic="Required All">
                    <GenericArgument Name="System.Int32, System.Private.CoreLib" />
                </Method>
            </Type>
            <Type Name="Microsoft.EntityFrameworkCore.Metadata.Internal.PropertyAccessorsFactory" Dynamic="Required All">
                <Method Name="CreateGeneric" Dynamic="Required All">
                    <GenericArgument Name="System.Int32, System.Private.CoreLib" />
                </Method>
            </Type>
            <Type Name="Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalEntityEntry" Dynamic="Required All">
                <Method Name="ReadStoreGeneratedValue" Dynamic="Required All">
                    <GenericArgument Name="System.Int32, System.Private.CoreLib" />
                </Method>
                <Method Name="ReadTemporaryValue" Dynamic="Required All">
                    <GenericArgument Name="System.Int32, System.Private.CoreLib" />
                </Method>
                <Method Name="ReadOriginalValue" Dynamic="Required All">
                    <GenericArgument Name="System.Int32, System.Private.CoreLib" />
                </Method>
                <Method Name="ReadRelationshipSnapshotValue" Dynamic="Required All">
                    <GenericArgument Name="System.Int32, System.Private.CoreLib" />
                </Method>
            </Type>
        </Assembly>

        <Assembly Name="mscorlib">
            <Type Name="System.DateTime" Dynamic="Required All">
                <Method Name="AddYears" Dynamic="Required" />
                <Method Name="AddMonths" Dynamic="Required" />
                <Method Name="AddDays" Dynamic="Required" />
                <Method Name="AddHours" Dynamic="Required" />
                <Method Name="AddMinutes" Dynamic="Required" />
                <Method Name="AddSeconds" Dynamic="Required" />
            </Type>
        </Assembly>

    </Application>
</Directives>

Seems that I hit this line:

runtimelab/src/coreclr/nativeaot/System.Private.CoreLib/src/Internal/Runtime/CompilerServices/GenericVirtualMethodSupport.cs

Line 52 in 9f1e27d

Environment.FailFast("GVM resolution failure");

I think if it can print which type is looking for, it will be eaiser to resolve.

MichalStrehovsky · 2021-02-24T11:53:10Z

I think there's a problem with how generic virtual methods get rooted from RD.XML. Maybe you need to root both the slot declaring method (the method that introduces the virtual method), and the override. I'll need to take a look at that at some point.

If you have this under a debugger (e.g. Visual Studio), you can look what thing the EEType pointer is pointing at. E.g. type this into the "Immediate" window of VS:

(void*)0x00007FF629DD46F0

(This is an address from the above error message - you should obviously use the current address.)

MichalStrehovsky · 2021-02-24T11:54:46Z

(And if you use WinDbg, you would use the ln command with the address.)

hez2010 · 2021-02-24T12:16:35Z

Yeah, I solved the issue of EEType:0x00007FF629DD46F0 by adding its override. However, in my last post I think it's a different issue other than the previous one, it only printed "GVM resolution failure" without any additional information so I don't know neither what's happening nor the EEType pointer address.

Full output while navigating to https://localhost:5001/weatherforecast:

info: Microsoft.Hosting.Lifetime[0]
      Now listening on: http://localhost:5000
info: Microsoft.Hosting.Lifetime[0]
      Now listening on: https://localhost:5001
info: Microsoft.Hosting.Lifetime[0]
      Application started. Press Ctrl+C to shut down.
info: Microsoft.Hosting.Lifetime[0]
      Hosting environment: Production
info: Microsoft.Hosting.Lifetime[0]
      Content root path: C:\Users\hez20\source\efaottest
GVM resolution failure

hez2010 · 2021-02-25T11:23:26Z

Can this line print the type and method signature so that I can know which it is looking for?

runtimelab/src/coreclr/nativeaot/System.Private.CoreLib/src/Internal/Runtime/CompilerServices/GenericVirtualMethodSupport.cs

Line 52 in 9f1e27d

Environment.FailFast("GVM resolution failure");

I'm stuck here because no information except a line with "GVM resolution failure" was printed.

MichalStrehovsky · 2021-02-25T11:33:42Z

Can this line print the type and method signature so that I can know which it is looking for?

Sure - #746. Thanks for your persistence!

hez2010 · 2021-02-26T04:24:56Z

It turns out that the resolution failure because the default interface method:

https://github.com/dotnet/efcore/blob/b247dedd6e9873d1aa9d5b2992bbccc98229147a/src/EFCore/Diagnostics/IDiagnosticsLogger.cs#L124

https://github.com/dotnet/efcore/blob/b247dedd6e9873d1aa9d5b2992bbccc98229147a/src/EFCore/Diagnostics/IDiagnosticsLogger.cs#L160

But I cannot make it work by adding:

            <Type Name="Microsoft.EntityFrameworkCore.Diagnostics.IDiagnosticsLogger" Dynamic="Required All">
                <Method Name="NeedsEventData" Dynamic="Required">
                    <GenericArgument Name="System.Object, System.Private.CoreLib" />
                </Method>
                <Method Name="NeedsEventData" Dynamic="Required" />
            </Type>

ilc threw error about either Method takes 0 generic argument(s) but 1 were provided. or Method takes 1 generic argument(s) but 0 were provided., ilc cannot distinguish these two methods.

hez2010 · 2021-02-26T04:41:25Z

This is a minimal repro:

using System;

namespace DimRepro
{
    interface IFoo
    {
        void Bar(string baz) => Console.WriteLine(baz);
        void Bar<T>(T what, string baz) => Console.WriteLine(what.ToString() + baz);
    }

    class Test : IFoo
    {
    }

    class Program
    {
        static void Main(string[] args)
        {
            IFoo x = new Test();
            x.Bar("test");
            x.Bar<int>(5, "test");
        }
    }
}

Output:

test
Generic virtual method pointer lookup failure.

Declaring type: EETypeRva:0x002E3D10(minimal_repo.Foo)
Target type: EETypeRva:0x002E3D38(minimal_repo.Test)
Method name: Bar
Instantiation:
  Argument 00000000: EETypeRva:0x002CB530(System.Int32)

Should I open a separate issue for this?

MichalStrehovsky · 2021-02-26T09:11:46Z

Should I open a separate issue for this?

Oh, okay this is generic virtual default interface methods. We don't support that yet. This is #161.

I added partial support for it in #368, but it only works for non-generic things.

MichalStrehovsky · 2021-08-13T22:13:33Z

Default generic interface methods support just merged. The nuget package with support should be out soon. Hopefully this will let things make progress.

hez2010 · 2021-08-14T02:50:57Z

I found there's a line which is incompatible with NativeAOT: https://github.com/dotnet/efcore/blob/b3619b49eb43fdeb8c93d167f2bc1fd67c2735ec/src/Microsoft.Data.Sqlite.Core/Utilities/BundleInitializer.cs#L20, but thanks to the try-catch block, I can directly invoke SQLitePCL.Batteries_V2.Init() in ConfigureServices.

Finally make it work as expected. For anyone wants to have a try, I created a sample project showing how to integrate efcore (SQLite) with NativeAOT: https://github.com/hez2010/EFCore.NativeAOT

Thanks @MichalStrehovsky for your great work.

hez2010 · 2021-08-15T15:12:59Z

Since major issue preventing efcore being used with NativeAOT has been resolved, I'm closing this.

BTW, is it possible to make Assembly.Load(const string) and Assembly.Load(new AssemblyName(const string)) work with NativeAOT? I'm considering this can be figured out by the compiler and link them during compilation.

MichalStrehovsky · 2021-08-15T23:54:51Z

Thank you @hez2010 for working on this! It's nice to have a sample we can point to!

Assembly.Load should work if the assembly in question was part of the closure (i.e. if you can see it passed as a -r argument to the AOT compiler in the *.ilc.rsp file in the obj directory of the project after publishing).

You might need to add the type/method in question (SQLitePCL.Batteries_V2.Init) in RD.XML.

We don't currently do analysis of Assembly.GetType. If we did it, this pattern might just work. Tracked here: dotnet/linker#1884

MichalStrehovsky added the area-NativeAOT-coreclr .NET runtime optimized for ahead of time compilation label Feb 23, 2021

hez2010 changed the title ~~System.InvalidOperationException: Operation is not valid due to the current state of the object~~ Issues while using efcore with NativeAOT Feb 23, 2021

hez2010 mentioned this issue Feb 26, 2021

Fix how root provider resolve generic methods #749

Merged

roji mentioned this issue Aug 2, 2021

Improve trimming support for EF Core apps to reduce application size dotnet/efcore#21894

Open

7 tasks

hez2010 closed this as completed Aug 15, 2021

roji mentioned this issue Oct 8, 2021

When looking up methods via reflection, be trimming/AOT-friendly dotnet/efcore#26288

Open

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Issues while using efcore with NativeAOT #733

Issues while using efcore with NativeAOT #733

hez2010 commented Feb 23, 2021 •

edited

Loading

MichalStrehovsky commented Feb 23, 2021

MichalStrehovsky commented Feb 23, 2021

hez2010 commented Feb 23, 2021 •

edited

Loading

hez2010 commented Feb 24, 2021 •

edited

Loading

MichalStrehovsky commented Feb 24, 2021

MichalStrehovsky commented Feb 24, 2021

hez2010 commented Feb 24, 2021 •

edited

Loading

hez2010 commented Feb 25, 2021 •

edited

Loading

MichalStrehovsky commented Feb 25, 2021

hez2010 commented Feb 26, 2021 •

edited

Loading

hez2010 commented Feb 26, 2021 •

edited

Loading

MichalStrehovsky commented Feb 26, 2021

MichalStrehovsky commented Aug 13, 2021

hez2010 commented Aug 14, 2021 •

edited

Loading

hez2010 commented Aug 15, 2021 •

edited

Loading

MichalStrehovsky commented Aug 15, 2021

Issues while using efcore with NativeAOT #733

Issues while using efcore with NativeAOT #733

Comments

hez2010 commented Feb 23, 2021 • edited Loading

MichalStrehovsky commented Feb 23, 2021

MichalStrehovsky commented Feb 23, 2021

hez2010 commented Feb 23, 2021 • edited Loading

hez2010 commented Feb 24, 2021 • edited Loading

MichalStrehovsky commented Feb 24, 2021

MichalStrehovsky commented Feb 24, 2021

hez2010 commented Feb 24, 2021 • edited Loading

hez2010 commented Feb 25, 2021 • edited Loading

MichalStrehovsky commented Feb 25, 2021

hez2010 commented Feb 26, 2021 • edited Loading

hez2010 commented Feb 26, 2021 • edited Loading

MichalStrehovsky commented Feb 26, 2021

MichalStrehovsky commented Aug 13, 2021

hez2010 commented Aug 14, 2021 • edited Loading

hez2010 commented Aug 15, 2021 • edited Loading

MichalStrehovsky commented Aug 15, 2021

hez2010 commented Feb 23, 2021 •

edited

Loading

hez2010 commented Feb 23, 2021 •

edited

Loading

hez2010 commented Feb 24, 2021 •

edited

Loading

hez2010 commented Feb 24, 2021 •

edited

Loading

hez2010 commented Feb 25, 2021 •

edited

Loading

hez2010 commented Feb 26, 2021 •

edited

Loading

hez2010 commented Feb 26, 2021 •

edited

Loading

hez2010 commented Aug 14, 2021 •

edited

Loading

hez2010 commented Aug 15, 2021 •

edited

Loading