Include() does not work if used after Union()

[mbodecek]

I have the following data context:

public class Foo
{
    public long Id { get; set; }
}

public class Bar
{
    public long Id { get; set; }
    public Foo Foo { get; set; }
}

public class DataContext : DbContext
{
    public DbSet<Foo> Foos { get; set; }
    public DbSet<Bar> Bars { get; set; }

    protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
    {
        optionsBuilder.UseSqlServer(@"data source=(local);initial catalog=IncludeTest;Integrated Security=True");
    }
}

With this setup, I'm not able to use Include after I applied Union:

var list = ctx.Bars
    .Union(ctx.Bars)
    .Include(b => b.Foo)
    .ToList();

// items will have null Foo, even though the associated entity is present

This will work if Include() is used before Union(), but this workaround is less than ideal, when I want the queryable to be reusable and only care about fetching the association in specific cases.

Further technical details

EF Core version: 2.2.4
Database Provider: Microsoft.EntityFrameworkCore.SqlServer
Operating system: Windows 10
IDE: Visual Studio 2017 15.9.3

[maumar]

related to #6812

[ajcvickers]

Notes from triage: We plan to implement this in 3.0, but consider:

• Union (and likely other set operations) must operate on the same shape to be translated. This means that if the first ctx.Bars has an Include, then the second must also do so.
  So this is okay:

var list = ctx.Bars.Include(b => b.Foo)
    .Union(ctx.Bars.Include(b => b.Foo))
    .ToList();

But this will throw:

var list = ctx.Bars.Include(b => b.Foo)
    .Union(ctx.Bars)
    .ToList();

• It is likely that putting the Include at the end will work in 3.0, although semantically it can be different from two Includes on the two roots.
• Note that this implies that putting the Include on only the first query root means something different from putting at the end after the two roots are combined into a single query. In other words, this:

var list = ctx.Bars
    .Include(b => b.Foo)
    .Union(ctx.Bars)
    .ToList();

is different from this:

var list = ctx.Bars
    .Union(ctx.Bars)
    .Include(b => b.Foo)
    .ToList();

Assigning to @roji
/cc @smitpatel @divega for anything I missed.

[mbodecek]

@maumar @ajcvickers Thanks for clarification. It's a fair distinction and in makes sense this does not work with the current in-memory implementation of Union (if maybe surprising for new users coming from EF6).

[roji]

This works in the new implementation, see test Union_Include introduced in #16249.