Throw for a shadow Many-To-Many navigation

[InspiringCode]

When calling Include() on Polymorphic Many-To-Many collections, an exception is thrown:

System.ArgumentException
  HResult=0x80070057
  Message=Method 'Boolean Add(System.Object, System.Object, Boolean)' declared on type 'Microsoft.EntityFrameworkCore.Metadata.IClrCollectionAccessor' cannot be called with instance of type 'System.Object'
  Source=System.Linq.Expressions
  StackTrace:
   at System.Linq.Expressions.Expression.ValidateCallInstanceType(Type instanceType, MethodInfo method)
   at System.Linq.Expressions.Expression.ValidateStaticOrInstanceMethod(Expression instance, MethodInfo method)
   at System.Linq.Expressions.Expression.ValidateMethodAndGetParameters(Expression instance, MethodInfo method)
   at System.Linq.Expressions.Expression.Call(Expression instance, MethodInfo method, Expression arg0, Expression arg1, Expression arg2)
   at Microsoft.EntityFrameworkCore.InMemory.Query.Internal.InMemoryShapedQueryCompilingExpressionVisitor.CustomShaperCompilingExpressionVisitor.AddToCollectionNavigation(ParameterExpression entity, ParameterExpression relatedEntity, INavigationBase navigation)
   at Microsoft.EntityFrameworkCore.InMemory.Query.Internal.InMemoryShapedQueryCompilingExpressionVisitor.CustomShaperCompilingExpressionVisitor.GenerateFixup(Type entityType, Type relatedEntityType, INavigationBase navigation, INavigationBase inverseNavigation)
   at Microsoft.EntityFrameworkCore.InMemory.Query.Internal.InMemoryShapedQueryCompilingExpressionVisitor.CustomShaperCompilingExpressionVisitor.VisitExtension(Expression extensionExpression)
   at System.Linq.Expressions.Expression.Accept(ExpressionVisitor visitor)
   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
   at System.Dynamic.Utils.ExpressionVisitorUtils.VisitBlockExpressions(ExpressionVisitor visitor, BlockExpression block)
   at System.Linq.Expressions.ExpressionVisitor.VisitBlock(BlockExpression node)
   at System.Linq.Expressions.BlockExpression.Accept(ExpressionVisitor visitor)
   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
   at System.Linq.Expressions.ExpressionVisitor.VisitLambda[T](Expression`1 node)
   at System.Linq.Expressions.Expression`1.Accept(ExpressionVisitor visitor)
   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
   at Microsoft.EntityFrameworkCore.InMemory.Query.Internal.InMemoryShapedQueryCompilingExpressionVisitor.VisitShapedQuery(ShapedQueryExpression shapedQueryExpression)
   at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.VisitExtension(Expression extensionExpression)
   at Microsoft.EntityFrameworkCore.InMemory.Query.Internal.InMemoryShapedQueryCompilingExpressionVisitor.VisitExtension(Expression extensionExpression)
   at System.Linq.Expressions.Expression.Accept(ExpressionVisitor visitor)
   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
   at Microsoft.EntityFrameworkCore.Query.QueryCompilationContext.CreateQueryExecutor[TResult](Expression query)
   at Microsoft.EntityFrameworkCore.Storage.Database.CompileQuery[TResult](Expression query, Boolean async)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.CompileQueryCore[TResult](IDatabase database, Expression query, IModel model, Boolean async)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.<>c__DisplayClass9_0`1.<Execute>b__0()
   at Microsoft.EntityFrameworkCore.Query.Internal.CompiledQueryCache.GetOrAddQuery[TResult](Object cacheKey, Func`1 compiler)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.Execute[TResult](Expression query)
   at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryProvider.Execute[TResult](Expression expression)
   at System.Linq.Queryable.Single[TSource](IQueryable`1 source)
   at EfCore5Tests.Program.Main(String[] args) in D:\Projects\Sandbox\EfCore5Tests\Program.cs:line 55

The used DbContext is:

    public class DemoContext : DbContext {

        public DbSet<User> Users { get; set; }

        protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder) {
            base.OnConfiguring(optionsBuilder);
            optionsBuilder.UseInMemoryDatabase("Test");
        }

        protected override void OnModelCreating(ModelBuilder mb) {
            base.OnModelCreating(mb);
            mb.Entity<Project>(e => e.HasKey(x => x.Id));
            mb.Entity<InternalProject>();
            mb.Entity<CustomerProject>();
            mb.Entity<User>(e => e.HasMany(x => x.Projects).WithMany("Users"));
        }
    }

The query is:

    class Program {
        static void Main(string[] args) {
            var db = new DemoContext();
            var p1 = new InternalProject();
            var p2 = new CustomerProject();
            var user = new User { Projects = { p1, p2 } };

            db.AddRange(p1, p2, user);
            db.SaveChanges();

            db = new DemoContext();
            
            User actual = db.Set<User>().Include(x => x.Projects).Single();

            foreach (var p in actual.Projects) {
                Console.WriteLine(p);
            }
        }
    }

The model classes are:

    public class User {
        public Guid Id { get; set; } = Guid.NewGuid();

        public List<Project> Projects { get; set; } = new List<Project>();
    }

    public class Project {
        public Guid Id { get; set; } = Guid.NewGuid();
    }

    public class InternalProject : Project {
        public string Name { get; set; }
    }

    public class CustomerProject : Project {
        public decimal Costs { get; set; }
    }

EF Core version: 5
Database provider: Microsoft.EntityFrameworkCore.InMemory
Target framework: .NET 5.0
Operating system: Win 10

[smitpatel]

This happens for SqlServer too

Issue is this
mb.Entity<User>(e => e.HasMany(x => x.Projects).WithMany("Users"));

"Users" navigation does not exist on CLR type. It is shadow state. @AndriySvyryd

model

Model: 
  EntityType: CustomerProject Base: Project
    Properties: 
      Costs (decimal) Required
    Annotations: 
      DiscriminatorProperty: 
      DiscriminatorValue: CustomerProject
      RelationshipDiscoveryConvention:NavigationCandidates: System.Collections.Immutable.ImmutableSortedDictionary`2[System.Reflection.PropertyInfo,System.Type]
  EntityType: InternalProject Base: Project
    Properties: 
      Name (string)
    Annotations: 
      DiscriminatorProperty: 
      DiscriminatorValue: InternalProject
      RelationshipDiscoveryConvention:NavigationCandidates: System.Collections.Immutable.ImmutableSortedDictionary`2[System.Reflection.PropertyInfo,System.Type]
  EntityType: Project
    Properties: 
      Id (Guid) Required PK AfterSave:Throw ValueGenerated.OnAdd
      Discriminator (no field, string) Shadow Required AfterSave:Throw
        Annotations: 
          AfterSaveBehavior: Throw
          ValueGeneratorFactory: System.Func`3[Microsoft.EntityFrameworkCore.Metadata.IProperty,Microsoft.EntityFrameworkCore.Metadata.IEntityType,Microsoft.EntityFrameworkCore.ValueGeneration.ValueGenerator]
    Skip navigations: 
      Users (no field, ) CollectionUser Inverse: Projects
    Keys: 
      Id PK
    Annotations: 
      DiscriminatorProperty: Discriminator
      DiscriminatorValue: Project
      RelationshipDiscoveryConvention:NavigationCandidates: System.Collections.Immutable.ImmutableSortedDictionary`2[System.Reflection.PropertyInfo,System.Type]
  EntityType: User
    Properties: 
      Id (Guid) Required PK AfterSave:Throw ValueGenerated.OnAdd
    Skip navigations: 
      Projects (List<Project>) CollectionProject Inverse: Users
    Keys: 
      Id PK
    Annotations: 
      Relational:TableName: Users
      RelationshipDiscoveryConvention:NavigationCandidates: System.Collections.Immutable.ImmutableSortedDictionary`2[System.Reflection.PropertyInfo,System.Type]
  EntityType: ProjectUser (Dictionary<string, object>) CLR Type: Dictionary<string, object>
    Properties: 
      ProjectsId (no field, Guid) Indexer Required PK FK AfterSave:Throw
      UsersId (no field, Guid) Indexer Required PK FK Index AfterSave:Throw
    Keys: 
      ProjectsId, UsersId PK
    Foreign keys: 
      ProjectUser (Dictionary<string, object>) {'ProjectsId'} -> Project {'Id'} Cascade
      ProjectUser (Dictionary<string, object>) {'UsersId'} -> User {'Id'} Cascade
    Indexes: 
      UsersId 
    Annotations: 
      RelationshipDiscoveryConvention:NavigationCandidates: System.Collections.Immutable.ImmutableSortedDictionary`2[System.Reflection.PropertyInfo,System.Type]
Annotations: 
  BaseTypeDiscoveryConvention:DerivedTypes: System.Collections.Generic.Dictionary`2[System.Type,System.Collections.Generic.List`1[Microsoft.EntityFrameworkCore.Metadata.IConventionEntityType]]
  NonNullableConventionState: Microsoft.EntityFrameworkCore.Metadata.Conventions.NonNullableConventionBase+NonNullabilityConventionState
  ProductVersion: 5.0.0
  Relational:MaxIdentifierLength: 128
  SqlServer:ValueGenerationStrategy: IdentityColumn

[AndriySvyryd]

We don't support shadow navigations yet. Duplicate of #3864

[ajcvickers]

@smitpatel @AndriySvyryd So it looks like this isn't a something we will patch, right?

[AndriySvyryd]

@ajcvickers We could patch to throw early in model validation.

[ajcvickers]

@AndriySvyryd Okay, let's discuss in triage.

[InspiringCode]

But the official doc says that it is an supported scenario. The API doc of CollectionNavigationBuilder<TEntity,TRelatedEntity>.WithMany reads:

If no property is specified, the relationship will be configured without a navigation property on the other end of the relationship.

That's how I came to try the above code.

[ajcvickers]

@InspiringCode Thanks. We'll fix the docs.

[InspiringCode]

Please fix the code instead, at least eventually. In most cases where you would use many-to-many, the bidirectional property will probably make no sense at all in the model and just introduce a DB concern into your domain model.

[ajcvickers]

@InspiringCode Yes, that is tracked by #3864, as Andriy said above.

[gojanpaolo]

What's the workaround to eagerly load many-to-many? Thanks!

[gojanpaolo]

Found it. #3864 (comment)

I think using the following code may help to hide many-to-many navigation properties from out of entity types with current EF Core version(5.0.1)

class Post
{
  public int Id { get; set; }
  public ICollection<Tag> Tags { get; set; }
}

class Tag
{
  public int Id { get; set; }
  private ICollection<Post> Posts { get; set; }
}

builder.Entity<Post>()
  .HasMany(p => p.Tags)
  .WithMany("Posts");

Just mark the Posts property as private, ef core can work well and you can still use this navigation property inside
Tag class. I prefer to field-only property, but it will make both navigations can not be loaded by explicit loading. See at #23717.