Reduce EF Core application startup time via compiled models

[mikary]

This an "epic" issue for the theme of generating a compiled model with fast loading and access times. Specific pieces of work will be tracked by linked issues.

Proposed for 6.0

Used by calling new tool commands (all parameters are optional)

dotnet ef dbcontext optimize -c MyContext -o MyFolder -n My.Namespace

Optimize-DbContext -Context MyContext -OutputDir MyFolder -Namespace My.Namespace

• Add runtime annotation support to model Add runtime annotation support to model #22031
• Convert metadata extension methods to default interface implementations. Convert metadata extension methods to default interface implementations. #19213
• Create a read-optimized implementation of IModel that can be used as the base for the compiled model Create a read-optimized implementation of IModel #8258
• Add API to set custom implementation types instead of objects
  • ValueGenerator (store non-lambda configuration separately)
  • ValueComparer and ValueConverter
• Implement a generator that outputs the source code for a custom model implementation
  • Throw when the model contains non-serializable configuration like lambdas, proxy types and non-serializable expressions. Or if it's using a read-optimized implementation.
  • Generate #nullable enable
  • Warn when using a model generated by an older version.
  • Warn when using a non-default model cache key.

Backlog

• Consider discovering the model automatically instead of requiring 'UseModel', by using assembly-level attribute (Discover the compiled model automatically #24893)
• Consider adding a MsBuild task to generate the model at build-time (Generate the compiled model during build #24894)
• Either move the members from the pubternal model interfaces to the public ones, so that they can have non-dynamic implementations or add a runtime annotation-based backup implementation. (Remove IRuntime* interfaces #24895)
• Generate compiled relational model (Generate compiled relational model #24896)
  • Add tests that assert the relational model, view, query mapping and default mappings
• Generate code for query filters when possible. (Compiled model: Generate code for query filters when possible #24897)
• Generate constructor bindings (Compiled model: Generate constructor bindings #24898)
• Generate code that builds the model lazily (Compiled model: Generate code that builds the model lazily #24899)
• Rely on static binding instead of reflection for properties and fields (Compiled model: Use static binding instead of reflection for properties and fields #24900)
• Split out the mutable model implementation to a different assembly as it's not needed when using compiled model (Add feature switches to reduce app size #24901)
• Generate custom lazy loading and change tracking proxy types. (Compiled model: Allow use of lazy loading and change tracking proxy types. #24902)
• Generate lambdas used in change tracking (Compiled model: Generate lambdas used in change tracking #24904)
• Generate code that normally depends on reflection types in the model (Type, MemberInfo, etc.) to avoid all reflection at runtime (Don't use reflection at runtime #24903)

[lemkepf]

Any updates on if this will be added into EF Core any time soon?

[TonyHenrique]

Any updates on this? I am having problem on Cold Start up time, specially on a large model...

[ajcvickers]

@TonyHenrique Any chance we could get access to your large model for perf testing? You can send it privately (to avickers at my work) if you don't want to post here.

It's unlikely we will get to this in the near future--other work items have higher priority right now.

[RainingNight]

Now what progress?

[enghch]

Just want to add my experience. I'm using EFCore 1.1.3 on UWP with SQLite. I'm seeing almost 7 seconds worth of DB startup time. Two seconds to create the DbContext, about 4 seconds to check for a migration (no migrations are needed or performed) and a second or so to actually perform the first query. The actual database is essentially empty of any rows. The database has three interrelated entities and a total of about 25 columns. I'm guessing a large portion of the migration check is actually building the model under the hood.

It's a big hit because the user can't use the app until the data is loaded. Using Json files the entire app is up, loaded and running within about 1.5 seconds. When I add EFCore/SQLite to the mix it's suddenly around 9 seconds.

[ajcvickers]

@enghch Is this a release build (i.e. using .NET Native) or a debug build? Does it get significantly slower in release than debug?

[enghch]

Good point. That was with a debug build while debugging. Running a debug build without debugging seems to take it down to about 2-3 seconds. Running a release build seems to be about 1.3-2 seconds. I still wish it were faster (say, by using some kind of model caching) but I think I can make these times work.

[MarcoLoetscher]

This problem is also present in EF6 and unfortunately it was never solved. Unlike EF6, EF Core is also used for apps (UWP, Xamarin). Users do not accept long waits at the start of an app. In my opinion, the solution to this problem is much more important than e. g."Lazy Loading", so this point should be planned for the next release 2.1.
The cold start of a model with 500 entities should take a maximum of 5 seconds and not 180 seconds (3 minutes). So you would have to achieve a 36x improvement, which is probably not possible by compiling the model. So I guess there will be nothing else left but a kind of lazy model creating.
We don't use EF Core yet because we lack some features (Many-to-many without join entity, Spatial data, Graphical visualization of model, GroupBy translation, Raw SQL queries: Non-model types).

[ngoov]

Can I vote somewhere to make this a higher priority in the roadmap? I had to delete alot of the models and slim down the DbContext because of this. Still the 20 models take about 15-20 seconds to generate. This slows down development a lot!

[werwolfby]

Any updates?

[tobiasbunyan]

Never mind the load time on a live application, the crippling amount of time you have to wait each time you start a debug session slashes productivity to almost nothing!!! This needs sorting out BIG TIME!

[GeorgeThackrayWT]

Any news on this? this seems to be quite a serious problem.

[ajcvickers]

This issue is in the Backlog milestone. This means that it is not going to happen for the 2.1 release. We will re-assess the backlog following the 2.1 release and consider this item at that time. However, keep in mind that there are many other high priority features with which it will be competing for resources.

[mwe]

I also think this is a problem in a happy development workflow. In a test driven environment our developers need to wait 2 minutes for the DbContext to initialize in order to (re)run the test. This is a lot of waiting. Also our test systems where the apppool shuts down takes a long time to have a cold startup where at least 3-4 minutes are caused by EF (EF6 has indeed the same problem, but a cached model improves this EF6.2)

It seems that there is at least one problem in EFCore\Storage\TypeMappingSource.cs -> FindMappingWithConversion, when we run the application with sources, this function causes 60% of the CPU. It seems that it is called many times and very slow. Perhaps the problem is with the ConcurrentDictionary here that is getting too many calls.

Please find attached a screen of a debug session.

If more information is required, i will be happy to supply that.

[gojanpaolo]

I just wanted to share a workaround if you're using xUnit and in-memory testing.
You might notice that if you have multiple test classes that uses a DbContext then each test class will have the overhead of model creation.
You can apply this workaround so that the model creation will be done only once per test run and not once per test class.

/// <summary>
/// This is the base class for all test classes that will use <see cref="YourDbContext"/> SQLite in-memory testing.
/// </summary>
/// <remarks>
/// This uses the <see cref="DbContextCacheCollectionFixture"/> as a workaround to improve unit test performance.
/// The deriving class will automatically inherit this workaround.
/// </remarks>
[Collection(nameof(DbContextCacheCollectionFixture))]
public class DatabaseTestBase : IDisposable
{
	private readonly SqliteConnection _connection;
	protected readonly DbContextOptions<YourDbContext> _options;
	public DatabaseTestBase()
	{
		_connection = new SqliteConnection("DataSource=:memory:");
		_connection.Open();
		_options = new DbContextOptionsBuilder<YourDbContext>()
			.UseSqlite(_connection)
			.Options;
		using(var context = new YourDbContext(_options))
		{
			context.Database.EnsureCreated();
		}
	}
	public void Dispose()
	{
		_connection.Close();
	}

	[CollectionDefinition(nameof(DbContextCacheCollectionFixture))]
	private class DbContextCacheCollectionFixture : ICollectionFixture<object>
	{
		/// This is a workaround to improve unit test performance for test classes that use a <see cref="DbContext"/> object.
		/// <see cref="DbContext"/> model creation has a significant overhead.
		///     https://github.com/aspnet/EntityFrameworkCore/issues/4372
		///     https://github.com/aspnet/EntityFrameworkCore/issues/1906
		/// By applying this attribute across all the <see cref="DbContext"/> test classes, the model creation is only done once throughout the test run instead of once per test class.
	}
}

[ajcvickers]

@gojanpaolo Models are cached by default; if you are seeing otherwise then can you please file a new issue with a runnable project/solution or code listing that demonstrates this behavior?

[mwe]

I need to clarify the unit test example, in a tdd workflow, the test is executed by a developer many times while he is writing the code. This means that after every try, the DbContext needs to be re-initialized. When the test failed, appdomain is unloaded and on the next run a new DbContext is initialized and cached.

Anyhow, this issue is about slow initialisation of a large DbContext (300 entities). The unit test issue is only one usecase where this is a problem. please don't focus on that.

Aftere more research i discovered that EFCore\Storage\TypeMappingSource\FindMappingWithConversion is called 1.2 million times. In our situation this means that during initialisation the dictionary with typemappings (77000) items is scanned 1.2 million times. Our DbContext has 300 entities with an average of 16 properties per entity. It seems that the FindMappingWithConversion is executed way to many times. This week i will investigate further and try to understand the loops that are being executed here and why this is slow.

OnModelCreating takes 1.5 minutes in Released dll's and 10minutes in debug (with EF projects as project references) with performance tracing switched on.

UPDATE:
Cause of the many calls seems to be in efcore\metadata\conventions\internal\foreignkeyattributeconvention.cs

when i change the logic here to first check if the attrribute is there and when the attribute is found than run FindCandidateNavigationPropertyType my context initialises 5 times faster and the dictionary with typemappings is queried 20.000 times instead of 1.2 million times.

        /// src\efcore\metadata\conventions\internal\foreignkeyattributeconvention.cs
        [ContractAnnotation("navigationName:null => null")]
        private MemberInfo FindForeignKeyAttributeOnProperty(EntityType entityType, string navigationName)
        {
            if (string.IsNullOrWhiteSpace(navigationName)
                || !entityType.HasClrType())
            {
                return null;
            }

            MemberInfo candidateProperty = null;
            var clrType = entityType.ClrType;

            foreach (var memberInfo in clrType.GetRuntimeProperties().Cast<MemberInfo>()
                .Concat(clrType.GetRuntimeFields()))
            {
                // GITHUB: ISSUE: 1906
                var attribute = memberInfo.GetCustomAttribute<ForeignKeyAttribute>(true);

                // first check if we have the attribute
                if (attribute != null && attribute.Name == navigationName)
                {
                    // than run the FindCandidateNavigationPropertyType as this function seems to be expensive
                    if (memberInfo is PropertyInfo propertyInfo
                        && FindCandidateNavigationPropertyType(propertyInfo) != null)
                    {
                        continue;
                    }

                    if (candidateProperty != null)
                    {
                        throw new InvalidOperationException(CoreStrings.CompositeFkOnProperty(navigationName, entityType.DisplayName()));
                    }

                    candidateProperty = memberInfo;
                }
            }

[pantonis]

Are you planning to add support for GlobalQueryfilters queries? It is a very powerful feature used by most modern systems.

[AndriySvyryd]

@pantonis We'll try to add support for at least some. Tracked by #24897

[JanKotschenreuther]

I have just read the Blog-Post https://devblogs.microsoft.com/dotnet/announcing-entity-framework-core-6-0-preview-5-compiled-models/# which mentions that "Lazy loading proxies" and "Change tracking proxies" are not supported.

Are there any plans to improve the startup/initialization with lazy loading and change tracking enabled or is this the final state?
Those features are pretty important to us.

We got a model with 843 entity types (656 tables and 187 views), 12290 properties and 1137 foreign keys.
Sorry, i cannot share the model.
We are using EF Core 5.0.5.
Instanciating the model the first time takes just 2 seconds or less.

I have been observing 90 seconds of initialization on the DbContext when reaching the first LINQ-Query.
Disabling lazy loading reduced initialization to 60 seconds.
So just adding the lazy loading proxies added 30 seconds.

At the moment I am observing other timings for the same code.
The model has increased with more tables and views.
But at the moment I am observing 45 seconds for initialization with lazy loading.
Without lazy loading it only took around 7 seconds.
This observation is a bit weird and i cannot serve any information how this could have improved, because the code did not change and we did not try out previews.
Maybe due to some updates, I am not sure.
However our production servers still take 90 seconds.

Executing the same query a second time results in usual nice query times of half a second.

We are using dependency injection for DbContext in our ASP.NET Core 5 application which makes a lot of use of lazy loading and change tracking.
We were able to observe the same slow initialization timings in simple console applications.

Making use of DbContext-Pooling to decrease instanciating time is nice, but 2 seconds do not bother us as much compared to 45 or 90 seconds for initialization, the initialization is the real show-stopper here.

[AndriySvyryd]

@JanKotschenreuther See #20135 and #24902

[ZvonimirMatic]

I can't get compiled models working when I'm using ValueConverters. I have the following value converter:

public class TrimConverter : ValueConverter<string, string>
{
    public TrimConverter() : base(x => x.Trim(), x => x.Trim()) { }
}

When I'm not using the converter, running dotnet ef dbcontext optimize ... works as expected. When I'm using the converter, I get the following error: The property 'MyEntity.MyProperty' has a value converter configured. Use 'HasConversion' to configure the value converter type.

I do not understand what is wrong from the message. Is this somehow related to #24896?

[AndriySvyryd]

@ZvonimirMatic How do you configure the property to use the converter?

It should be something like propertyBuilder.HasConversion(typeof(TrimConverter), null)

[ZvonimirMatic]

@AndriySvyryd

I configured it using propertyBuilder.HasConversion(new TrimConverter());. After seeing your reply, I tried changing it to propertyBuilder.HasConversion(typeof(TrimConverter), default(ValueComparer)); (when I tried using null instead of default(ValueComparer), the method call was ambiguous).

Now I get the following error: The property 'MyEntity.MyProperty' is of type 'string' which is not supported by the current database provider. Either change the property CLR type, or ignore the property using the '[NotMapped]' attribute or by using 'EntityTypeBuilder.Ignore' in 'OnModelCreating'.. I tried doing that for multiple properties, but I always get the same error ("The property '' is of type '' which is not supported by the current database provider. ...".

[AndriySvyryd]

@ZvonimirMatic Could you file a new issue with a small repro project so we can investigate?

[ZvonimirMatic]

@AndriySvyryd Sure, no problem. Do you want me to create a repository with the repro project or do you want me to just copy and paste file contents in the issue description?

[AndriySvyryd]

@ZvonimirMatic You can paste them if they are small enough

[ZvonimirMatic]

@AndriySvyryd Filed the issue #25187.

[julielerman]

Recommendation (shall I make it an issue?): require output directory. All of those generated files landing in the main project folder is an easy mistake to make and a hard one to clean up after.

[ajcvickers]

@julielerman Covered by #25059

[Luigi6821]

Hi,
about the topic I would like to know if we can expect on next EF Core releases a feature that allows to make persistent the finalized model. This feature could be used to speedup the startup process like in the following example code:

IModel model;

if (!FirstStartup())
{
modelBuilder.Entity()
// Model definitions...
model = modelBuilder.FinalizeModel();
SerializeModelForUsingOnNextStartup(model) // This serialize the model and make it persistent for next startup load
}
else
model = DeserializeModelFromFirstStartup();

optionsBuilder.UseModel(model)

Thanks in advance
Luigi

[manigandham]

@Luigi6821 There's a blog post showing how compiled models will work with EF Core 6.0: https://devblogs.microsoft.com/dotnet/announcing-entity-framework-core-6-0-preview-5-compiled-models/

[Luigi6821]

@Luigi6821 There's a blog post showing how compiled models will work with EF Core 6.0: https://devblogs.microsoft.com/dotnet/announcing-entity-framework-core-6-0-preview-5-compiled-models/

The solution proposed, for what I understood, is not usable in some scenarios like the one I am using. The mapped entities and properties could be mapped differently depending on db schema version to which context is connecting to. Creating a static model version at compile time is not usable when I create app for different schema version. Maybe I didn't fully understand the proposed solution. Can you suggest me what I can do in the scenarios I described?
Thanks id advance

[roji]

@Luigi6821 please open a new issue with a precise description of what you're trying to do. EF Core's model does not rely on your database schema, only on your code; it's is normally built during application startup without connecting to the database.