Guide on memory leak warning ef core 3.0 preview9

[dotnetshadow]

Currently using the following project:
https://github.com/KalikoCMS/KalikoCMS.NextGen/blob/netcore3.0-experimental/KalikoCMS.Legacy/Data/Repositories/LegacyRepositoryBase.cs

Using ef core 3.0 preview8 the following code would work:

public virtual IQueryable<TEntity> GetAll() {
    return _context.Set<TSource>().Select(Map()).AsNoTracking();
}

public override Expression<Func<LegacyPageTypeEntity, ContentTypeEntity>> Map() {
            return x => new ContentTypeEntity {
                ContentTypeId = ToGuid(x.PageTypeId),
                DefaultChildSortDirection = x.DefaultChildSortDirection,
                ContentProviderId = Guid.Empty,
                DefaultChildSortOrder = x.DefaultChildSortOrder,
                DisplayName = x.DisplayName,
                Name = x.Name,
                ShowInAdmin = true
            };
        }

But in ef core 3.0 preview9 the following error occurs

Exception message:
Client projection contains reference to constant expression of type: KalikoCMS.Legacy.Data.Repositories.LegacyContentTypeRepository. This could potentially cause memory leak."}
    Data: {System.Collections.ListDictionaryInternal}
    HResult: -2146233079
    HelpLink: null
    InnerException: null
    Message: "Client projection contains reference to constant expression of type: KalikoCMS.Legacy.Data.Repositories.LegacyContentTypeRepository. This could potentially cause memory leak."
    Source: "Microsoft.EntityFrameworkCore"
    StackTrace: "   at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.ConstantVerifyingExpressionVisitor.VisitConstant(ConstantExpression constantExpression)\r\n   at System.Linq.Expressions.ExpressionVisitor.VisitMethodCall(MethodCallExpression node)\r\n   at System.Linq.Expressions.ExpressionVisitor.VisitMemberAssignment(MemberAssignment node)\r\n   at System.Linq.Expressions.ExpressionVisitor.Visit[T](ReadOnlyCollection`1 nodes, Func`2 elementVisitor)\r\n   at System.Linq.Expressions.ExpressionVisitor.VisitMemberInit(MemberInitExpression node)\r\n   at System.Dynamic.Utils.ExpressionVisitorUtils.VisitBlockExpressions(ExpressionVisitor visitor, BlockExpression block)\r\n   at System.Linq.Expressions.ExpressionVisitor.VisitBlock(BlockExpression node)\r\n   at System.Linq.Expressions.BlockExpression.Accept(ExpressionVisitor visitor)\r\n   at System.Linq.Expressions.ExpressionVisitor.VisitLambda[T](Expression`1 node)\r\n   at System.Linq.Expressions.Expression`1.Accept(ExpressionVisit
or visitor)\r\n   at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.InjectEntityMaterializers(Expression expression)\r\n   at Microsoft.EntityFrameworkCore.Query.RelationalShapedQueryCompilingExpressionVisitor.VisitShapedQueryExpression(ShapedQueryExpression shapedQueryExpression)\r\n   at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.VisitExtension(Expression extensionExpression)\r\n   at Microsoft.EntityFrameworkCore.Query.QueryCompilationContext.CreateQueryExecutor[TResult](Expression query)\r\n   at Microsoft.EntityFrameworkCore.Storage.Database.CompileQuery[TResult](Expression query, Boolean async)\r\n   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.CompileQueryCore[TResult](IDatabase database, Expression query, IModel model, Boolean async)\r\n   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.<>c__DisplayClass9_0`1.<Execute>b__0()\r\n   at Microsoft.EntityFrameworkCore.Query.Internal.CompiledQueryCache.GetOrAddQueryCore[
TFunc](Object cacheKey, Func`1 compiler)\r\n   at Microsoft.EntityFrameworkCore.Query.Internal.CompiledQueryCache.GetOrAddQuery[TResult](Object cacheKey, Func`1 compiler)\r\n   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.Execute[TResult](Expression query)\r\n   at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryProvider.Execute[TResult](Expression expression)\r\n   at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryable`1.GetEnumerator()\r\n   at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)\r\n   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)\r\n   at KalikoCMS.Services.Initialization.ContentTypeSynchronizer.SynchronizeContentTypes() in D:\\Programming\\Projects\\Libraries\\KalikoCMS.NextGen\\KalikoCMS.NextGen\\KalikoCMS\\Services\\Initialization\\ContentTypeSynchronizer.cs:line 35\r\n   at KalikoCMS.Services.Content.ContentTypeResolver.Initialize() in D:\\Programming\\Projects\\Libraries\\KalikoCMS.NextGen\\KalikoCMS.NextGen\\KalikoCMS\\S
ervices\\Content\\ContentTypeResolver.cs:line 35\r\n   at KalikoCMS.Services.Initialization.InitializationService.Initialize() in D:\\Programming\\Projects\\Libraries\\KalikoCMS.NextGen\\KalikoCMS.NextGen\\KalikoCMS\\Services\\Initialization\\InitializationService.cs:line 44"
    TargetSite: {System.Linq.Expressions.Expression VisitConstant(System.Linq.Expressions.ConstantExpression)}

I understand it could be related to
#13048
#17051

I'm just wondering if there is any guidance on how to solve the issue?

Further technical details

EF Core version: 3.0.0-preview9.19423.6
Database Provider: Microsoft.EntityFrameworkCore.SqlServer
Operating system:
IDE: (e.g. Visual Studio 2019 16.3 preview 3.0)

[smitpatel]

https://github.com/KalikoCMS/KalikoCMS.NextGen/blob/e1c08ce28c1e88ef6e12c7d63047ac60c50a98c2/KalikoCMS.Legacy/Data/Repositories/LegacyRepositoryBase.cs#L54-L59

ToGuid method which is being used in projection is defined on Repository so expression tree contains reference to object of your repository. Since your repository has DbContext also inside a field, it is actually leaking memory. Easy way to fix this is to use static method so you are not putting your repository objects inside expression tree.

[dotnetshadow]

@smitpatel Thank you so much I was looking at the Map() method not realising the ToGuid() was probably the problem. Making it static worked :)

[schmitch]

@smitpatel looks like the error happens in the Microsoft.AspNetCore.DataProtection.EntityFrameworkCore Package of AspNetCore:

(Version: 3.0.0-preview9.19424.4)

fail: Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingProvider[48]
      An error occurred while reading the key ring.
System.InvalidOperationException: Client projection contains reference to constant expression of type: Microsoft.AspNetCore.DataProtection.EntityFrameworkCore.EntityFrameworkCoreXmlRepository<Asvg.SalesTool.Web.Areas.Base.Services.Framework.DataProtectionDbContext>. This could potentially cause memory leak.
   at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.ConstantVerifyingExpressionVisitor.VisitConstant(ConstantExpression constantExpression)
   at System.Linq.Expressions.ConstantExpression.Accept(ExpressionVisitor visitor)
   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
   at System.Linq.Expressions.ExpressionVisitor.VisitMethodCall(MethodCallExpression node)
   at System.Linq.Expressions.MethodCallExpression.Accept(ExpressionVisitor visitor)
   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
   at System.Dynamic.Utils.ExpressionVisitorUtils.VisitBlockExpressions(ExpressionVisitor visitor, BlockExpression block)
   at System.Linq.Expressions.ExpressionVisitor.VisitBlock(BlockExpression node)
   at System.Linq.Expressions.BlockExpression.Accept(ExpressionVisitor visitor)
   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
   at System.Linq.Expressions.ExpressionVisitor.VisitLambda[T](Expression`1 node)
   at System.Linq.Expressions.Expression`1.Accept(ExpressionVisitor visitor)
   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
   at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.InjectEntityMaterializers(Expression expression)
   at Microsoft.EntityFrameworkCore.Query.RelationalShapedQueryCompilingExpressionVisitor.VisitShapedQueryExpression(ShapedQueryExpression shapedQueryExpression)
   at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.VisitExtension(Expression extensionExpression)
   at System.Linq.Expressions.Expression.Accept(ExpressionVisitor visitor)
   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
   at Microsoft.EntityFrameworkCore.Query.QueryCompilationContext.CreateQueryExecutor[TResult](Expression query)
   at Microsoft.EntityFrameworkCore.Storage.Database.CompileQuery[TResult](Expression query, Boolean async)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.CompileQueryCore[TResult](IDatabase database, Expression query, IModel model, Boolean async)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.<>c__DisplayClass9_0`1.<Execute>b__0()
   at Microsoft.EntityFrameworkCore.Query.Internal.CompiledQueryCache.GetOrAddQueryCore[TFunc](Object cacheKey, Func`1 compiler)
   at Microsoft.EntityFrameworkCore.Query.Internal.CompiledQueryCache.GetOrAddQuery[TResult](Object cacheKey, Func`1 compiler)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.Execute[TResult](Expression query)
   at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryProvider.Execute[TResult](Expression expression)
   at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryable`1.GetEnumerator()
   at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
   at Microsoft.AspNetCore.DataProtection.EntityFrameworkCore.EntityFrameworkCoreXmlRepository`1.GetAllElements()
   at Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager.GetAllKeys()
   at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingProvider.CreateCacheableKeyRingCore(DateTimeOffset now, IKey keyJustAdded)
   at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingProvider.Microsoft.AspNetCore.DataProtection.KeyManagement.Internal.ICacheableKeyRingProvider.GetCacheableKeyRing(DateTimeOffset now)
   at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingProvider.GetCurrentKeyRingCore(DateTime utcNow, Boolean forceRefresh)
info: Microsoft.AspNetCore.DataProtection.Internal.DataProtectionHostedService[0]
      Key ring failed to load during application startup.
System.InvalidOperationException: Client projection contains reference to constant expression of type: Microsoft.AspNetCore.DataProtection.EntityFrameworkCore.EntityFrameworkCoreXmlRepository<Asvg.SalesTool.Web.Areas.Base.Services.Framework.DataProtectionDbContext>. This could potentially cause memory leak.
   at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.ConstantVerifyingExpressionVisitor.VisitConstant(ConstantExpression constantExpression)
   at System.Linq.Expressions.ConstantExpression.Accept(ExpressionVisitor visitor)
   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
   at System.Linq.Expressions.ExpressionVisitor.VisitMethodCall(MethodCallExpression node)
   at System.Linq.Expressions.MethodCallExpression.Accept(ExpressionVisitor visitor)
   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
   at System.Dynamic.Utils.ExpressionVisitorUtils.VisitBlockExpressions(ExpressionVisitor visitor, BlockExpression block)
   at System.Linq.Expressions.ExpressionVisitor.VisitBlock(BlockExpression node)
   at System.Linq.Expressions.BlockExpression.Accept(ExpressionVisitor visitor)
   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
   at System.Linq.Expressions.ExpressionVisitor.VisitLambda[T](Expression`1 node)
   at System.Linq.Expressions.Expression`1.Accept(ExpressionVisitor visitor)
   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
   at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.InjectEntityMaterializers(Expression expression)
   at Microsoft.EntityFrameworkCore.Query.RelationalShapedQueryCompilingExpressionVisitor.VisitShapedQueryExpression(ShapedQueryExpression shapedQueryExpression)
   at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.VisitExtension(Expression extensionExpression)
   at System.Linq.Expressions.Expression.Accept(ExpressionVisitor visitor)
   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
   at Microsoft.EntityFrameworkCore.Query.QueryCompilationContext.CreateQueryExecutor[TResult](Expression query)
   at Microsoft.EntityFrameworkCore.Storage.Database.CompileQuery[TResult](Expression query, Boolean async)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.CompileQueryCore[TResult](IDatabase database, Expression query, IModel model, Boolean async)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.<>c__DisplayClass9_0`1.<Execute>b__0()
   at Microsoft.EntityFrameworkCore.Query.Internal.CompiledQueryCache.GetOrAddQueryCore[TFunc](Object cacheKey, Func`1 compiler)
   at Microsoft.EntityFrameworkCore.Query.Internal.CompiledQueryCache.GetOrAddQuery[TResult](Object cacheKey, Func`1 compiler)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.Execute[TResult](Expression query)
   at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryProvider.Execute[TResult](Expression expression)
   at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryable`1.GetEnumerator()
   at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
   at Microsoft.AspNetCore.DataProtection.EntityFrameworkCore.EntityFrameworkCoreXmlRepository`1.GetAllElements()
   at Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager.GetAllKeys()
   at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingProvider.CreateCacheableKeyRingCore(DateTimeOffset now, IKey keyJustAdded)
   at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingProvider.Microsoft.AspNetCore.DataProtection.KeyManagement.Internal.ICacheableKeyRingProvider.GetCacheableKeyRing(DateTimeOffset now)
   at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingProvider.GetCurrentKeyRingCore(DateTime utcNow, Boolean forceRefresh)
   at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingProvider.GetCurrentKeyRing()
   at Microsoft.AspNetCore.DataProtection.Internal.DataProtectionHostedService.StartAsync(CancellationToken token)

[smitpatel]

@schmitch - I filed issue dotnet/aspnetcore#13696 on aspnetcore repo. Thanks for pointing us to the error.

[JanEggers]

@smitpatel is it possible to improve the message:

Client projection contains reference to constant expression of type: KalikoCMS.Legacy.Data.Repositories.LegacyContentTypeRepository.

Client projection contains reference to constant expression ToGuid of type: KalikoCMS.Legacy.Data.Repositories.LegacyContentTypeRepository.

to better point to the expression that is causing the issue

[smitpatel]

@JanEggers - In expression tree we-encounter the constant, but it may not always possible to understand why the constant there. I will look into it, and try to improve exception message to be as useful as it can be.

[ajcvickers]

Improve the exception message and add a fwlink.

[smitpatel]

@divega - To generate fwlink.

[divega]

@smitpatel link is https://go.microsoft.com/fwlink/?linkid=2103067, for now pointing to query overview.

Do we need a docs issue to add the contents? I am thinking we need to go back and review all the fwlinks created for this release in search for gaps.

[MCFHTAGENTS]

OK- I have come across this with code like this:
` Chart decimalData = new Chart();
var bookingsBySource = this.context
.BookingView
.Include(a => a.Booking)
.ThenInclude(a => a.Source)
.Where(a => a.BookingTypeId != BookingType.Cancelled && a.StartDate.Year == Year && a.PropertyId == PropertyId)
.GroupBy(a => new { a.StartDate.Year, a.Booking.Source.SourceId, a.Booking.Source.Name })
.Select(
a => new { a.Key.Year, a.Key.SourceId, Source = a.Key.Name, Bookings = a.Count(), Total = a.Sum(b => b.Total) });

            var chartData = new BarChartDataSet<decimal>
            {
                Label = "Booking Value",
                BorderWidth = 1,
                Data = bookingsBySource.Select(d => d.Total).ToList(),
                BorderColor = bookingsBySource.Select(b => borderColour).ToList(),
                BackgroundColor = bookingsBySource.Select(c => backgroundColour).ToList(),
            };

`
The exception is being thrown on the BorderColor = bookingsBySource line.
borderColor is a static class member. That I need to clone n times to create a list the same length as 'Data'.

I don't see what is wrong with the above code and why the warning exception is thrown.

[smitpatel]

@MCFHTAGENTS - Please file a new issue and share full repro code. It would be hard to guess what the expression tree would be based on info you posted above.