Warn when the context starts tracking a non-proxy when proxies are being used

[IlyaBezus]

I have a .Net Core 3.1 application with an SQL database. I use lazy-loading proxies to automatically retrieve data from related tables. Basically, I have a table, which references some other entities via 1-to-many or 1-to-1 relation. Thing is, most of the cases, every relation is OK, every entity is loaded and I can read it's properties (name field, for example).

But, in very specific cases, those entities won't load, although relation Id is there. It looks like this:

Model is like this (I've cut out unnecessary relations and properties, there is a lot of them):

public partial class Delegation
{
    public Guid Id { get; set; }
    public int UserFrom { get; set; }
    public int UserTo { get; set; }

    public virtual User UserFromNavigation { get; set; }
    public virtual User UserToNavigation { get; set; }
}
...
public partial class User
{
    public User()
    {            
        DelegationsUserFromNavigation = new HashSet<Delegation>();
        DelegationsUserToNavigation = new HashSet<Delegation>();
    }

    public int Id { get; set; }
    public string Name { get; set; }

    public virtual ICollection<Delegation> DelegationsUserFromNavigation { get; set; }
    public virtual ICollection<Delegation> DelegationsUserToNavigation { get; set; }
}

Initialization inside database context (I've also cut out unnecessary parts of the code):

public virtual DbSet<Delegation> Delegations { get; set; }
public virtual DbSet<User> Users { get; set; }
...
modelBuilder.Entity<Delegation>(entity =>
{
    entity.Property(e => e.Id).ValueGeneratedNever();

    entity.HasOne(d => d.UserFromNavigation)
        .WithMany(p => p.DelegationsUserFromNavigation)
        .HasForeignKey(d => d.UserFrom)
        .OnDelete(DeleteBehavior.ClientSetNull)
        .HasConstraintName("FK_Delegations_Users_From");

     entity.HasOne(d => d.UserToNavigation)
        .WithMany(p => p.DelegationsUserToNavigation)
        .HasForeignKey(d => d.UserTo)
        .OnDelete(DeleteBehavior.ClientSetNull)
        .HasConstraintName("FK_Delegations_Users_To");
});

This specific example is only one of the many, there are also not loading virtual ICollections of other entities in other models.

Here, on debug screenshot, UserTo is OK, but UserFrom is NULL.

Most times, calling Eager Loading before accessing properties helps, and the property is there:

rmsContext.Delegations.Include(f => f.UserFromNavigation).ToList();
rmsContext.Delegations.Include(f => f.UserToNavigation).ToList();

Here is an example of failing property retrieving:

public async Task<bool> SendNewDelegationMail(Guid id)
{
    try
    {
        // Somewhat fixes relations
        rmsContext.Delegations.Include(f => f.UserFromNavigation).ToList();
        rmsContext.Delegations.Include(f => f.UserToNavigation).ToList();

        var delegation = rmsContext.Delegations.Find(id);
        if (delegation != null)
        {
            var emailData = new EmailData
            {
               Delegation = new DelegationData
               {
                    From = new UserData
                    {
                        Name = delegation.UserFromNavigation.Name // might fail here
                    },

                    To = new UserData
                    {
                        Name = delegation.UserToNavigation.Name // might fail here
                    }
                }
            };

            await emailSender.SendEmailAsync(
                emailData,
                userManager.GetUserMailAddress(delegation.UserToNavigation),
                MailViewType.Delegation_New_To);

            return true;
        }
    }
    catch (Exception e)
    {
        logger.LogError(e, $"An unexpected error occured while generating email: {e.Message}");
    }
    return false;
}

The main problem is that it appears so randomly that I don't know how to specify valid reproduction steps. The main rule is: the deeper required object is (entity inside entity inside entity), the higher chance of receiving NULL instead of required entity (Although users inside delegations is still first level relation).

I use database-first approach with further scaffolding of database into model classes. If required, I can provide more examples.

EF Core version: 3.1.2
Database provider: Microsoft.EntityFrameworkCore.SqlServer 3.1.2
Target framework: .NET Core 3.1
Operating system: Win10 Pro x64
IDE: Visual Studio 2019 16.4.5

[ajcvickers]

@IlyaBezus I don't see any obvious issues in the code you posted. I suspect we're going to need to be able to reproduce what you are seeing to be able to properly investigate. Can you put together a small, runnable project that, at least sometimes, generates these random issues?

[IlyaBezus]

@ajcvickers Sure thing! I will keep necessary parts of the code, keeping it as close to original as it is and will provide a project soon :)

[ajcvickers]

EF Team Triage: Closing this issue as the requested additional details have not been provided and we have been unable to reproduce it.

BTW this is a canned response and may have info or details that do not directly apply to this particular issue. While we'd like to spend the time to uniquely address every incoming issue, we get a lot traffic on the EF projects and that is not practical. To ensure we maximize the time we have to work on fixing bugs, implementing new features, etc. we use canned responses for common triage decisions.

[IlyaBezus]

@ajcvickers I'm so sorry for the delay. I've created a test project with all things you might need and removed mostly everything unnecessary.

There are 2 testing cases available, you just need to create DB from provided script (script will also load data) first. You can set DB name and server name inside appsettings.json file.

Lines of code where you might trigger an error are 79-84 and 169-178 of Services/MailMessageManager.cs file. You might need some time to reproduce the error, but for me it happens 90%+ of time. It looks like after initial retrieving objects by id (direct access), they stay available (non-null) for some time, but when accessing first time, an error occurs.

EF Error Test.zip

[ajcvickers]

@IlyaBezus When I run the app I see the page below. However, the Create button doesn't do anything. Just want to check if I'm missing something before going much further.

[IlyaBezus]

@ajcvickers Hm, it loads on my machine, it could be JavaScript error. A modal window should've popped.

I see icons are missing, so perhaps packages might be missing too? I did not include them, but I can (and I will in a few hours). I hoped they will restore automatically via libman. I tested it on this particular project provided, everything went fine.

[IlyaBezus]

@ajcvickers, the fully prebuilt project is too big and won't attach, so here is what you could do.

Either right-click on libman.json file inside visual studio and select "Restore Client-side libraries" or, please, add this (EF Test part 2.zip) to project and hit Ctrl+F5 after launch, if there are any 404 errors for resources in browser's development mode left.

Here's how it should look:

[ajcvickers]

@IlyaBezus Thanks. I was able to reproduce the issue, and I think I understand the problem. This code is used to create a new Delegation in DelegationManager:

var delegation = new Delegation
{
    Id = Guid.NewGuid(),
    CreatedOn = DateTime.Now,
    CreatedBy = currentUser.Id,
    UserTo = viewModel.UserTo,
    UserFrom = viewModel.UserFrom ?? currentUser.Id,
    StartDateTime = viewModel.StartDateTime,
    EndDateTime = viewModel.EndDateTime
};

rmsContext.Delegations.Add(delegation);
rmsContext.SaveChanges();

This same instance is then returned later by:

var delegation = rmsContext.Delegations.Find(id);

because it is still be tracked by the context.

However, this instance is not a proxy because it was created with new. Therefore lazy-loading doesn't happen.

To fix this, explicitly create a proxy instance. For example:

var delegation = rmsContext.CreateProxy<Delegation>();

delegation.Id = Guid.NewGuid();
delegation.CreatedOn = DateTime.Now;
delegation.CreatedBy = currentUser.Id;
delegation.UserTo = viewModel.UserTo;
delegation.UserFrom = viewModel.UserFrom ?? currentUser.Id;
delegation.StartDateTime = viewModel.StartDateTime;
delegation.EndDateTime = viewModel.EndDateTime;

rmsContext.Delegations.Add(delegation);
rmsContext.SaveChanges();

[IlyaBezus]

@ajcvickers, well, that's something I'd never figure out by myself. I was hoping that saving will create proxies, but now I know a little more. Sorry for taking your time and thanks a lot for your help :)

[ajcvickers]

@IlyaBezus Happy to help.

Re-opening to consider warning when attaching a non-proxy. (This is not going to be trivial because proxies are an extension package.)

[pharaf]

Hello,

I’m experiencing the same issue without doing any saving operation. Basically, I load all my table data from the database, and during my conversion process of my entity collection model to my front models, some deep navigation properties start to return null. The strangest part is that when doing a quick Watch before entering my conversion loop, I can access all navigation properties no matter how deep they are. However, after starting my conversion loop which Simply takes an entity and create it front model (so there is only read operations on the entities) and foreach dependency Inside that entity the associated front model will be created and so on…. (In my case I have up to 5 levels of deep dependency. Let say entity A reference B which also reference C etc...) So, after starting the loop, at the second iteration of my deep conversion, some navigation properties start to return null (even if they were there before the conversion process. Again, I’m only reading from my entities, so no update operation is performed here). I also tried to implement the no proxy lazyloading and the problem still occurs. I’m surprised that there is almost no topic speaking about this issue so I’m wondering if I’m not doing something wrong?
DataBase: PostgreSql
EFcore : 3.1.4

Edit: Using eager loading (with all required includes..) and flaging my query AsNoTracking() solve the problem. But when i remove the AsNoTracking the issue remains. Hope that could help someone and give you guys clues about the issue.

[VILLAN3LL3]

CreateProxy

There seems to be no method called "CreateProxy" on DbContext in .NET 7. What is the correct approach for .NET 7?

[ajcvickers]

@VILLAN3LL3 Make sure your project references the Microsoft.EntityFrameworkCore.Proxies package.

[VILLAN3LL3]

@VILLAN3LL3 Make sure your project references the Microsoft.EntityFrameworkCore.Proxies package.

It is already referenced:

[ajcvickers]

@VILLAN3LL3 In that case, please open a new issue and attach a small, runnable project or post a small, runnable code listing that reproduces what you are seeing so that we can investigate.