Does EF Core support bulk-insert/update/delete?

[ZhiqiangTao]

No description provided.

[yukozh]

You could take a look:https://github.com/PomeloFoundation/Lolita

[divega]

I am going to assume that you refer to set based operations which is missing functionality that is tracked in our backlog by #795. But just in case, there are other feature that are often confused with bulk operations:

• Batching of commands (we already support it)
• SqlBulkCopy-like of fast data loading functionality (we have no plans)
• TVP support (is in our backlog at Strategy for inserting rows with table-values parameters #2484)

[divega]

Closed as a duplicate of #795.

[MgSam]

SqlBulkCopy-like of fast data loading functionality (we have no plans)"

@divega This is exactly what I was referring to on the MSDN blog. Why do you have no plans to do this? It is crazy that the EF team feels loading data into a database in a non-performance-crippled manner is not important enough to support.

[ErikEJ]

@MgSam I have been using https://github.com/borisdj/EFCore.BulkExtensions with success.

[divega]

@MgSam I think we can all see fast loading of data as an important scenario, that is in fact why SQL Server has a dedicated API for it in SqlBulkCopy. I just don't have knowledge of any object-database mapping product that have attempted to abstract this, I don't think we can pretend that SqlBulkCopy is just a faster way to achieve the same thing that inserts do (for instance, how do you recover store generated identities from a bulk insert?), and to be honest, we don't see that many customers asking for this compared to other features.

Personally I see value in a tool that leverages EF Core mappings to perform bulk inserts on SQL Server (or on any database that supports equivalent APIs). It is just not functionality that we are planning to build into EF Core, so it is a good opportunity for extensions (like the one @ErikEJ mentioned) to provide value.

[MgSam]

@divega It's definitely not the same as a regular insert and I did not mean to imply it should be treated as such. It would certainly need it's own API design.

I just don't have knowledge of any object-database mapping product that have attempted to abstract this

So because many other ORMs are relegated to mediocrity EF should be too?

To be frank, you're not seeing the customer demand because you guys are using the wrong metrics. @ErikEJ's EFCore Power Tools has 112,000 downloads. EFCore Bulk Extensions has 75,000 downloads. Entity Framework Extended had 465,000 downloads. And these are 3rd party libraries with absolutely none of the marketing power of Microsoft behind them. The community demand is clearly there.

These people may not flood your forums but the numbers speak for themselves. I would argue the people frequenting these forums and upvoting issues likely has a strong selection bias from the far, far larger set of users who need a good ORM.

[divega]

@MgSam I agree those download numbers are proof that the community projects you mentione are popular, or at least that there is significant demand for the functionality they offer. I am actually glad to hear that they are that great (I had not been checking on them later).

However I can't conclude that EF Core should then take over that functionality (assuming that is what you are suggesting). I would rather like us keep spending time building the kind of base functionality into EF Core that these tools then leverage. I also wish we could spend more time trying these extensions, talking to the teams that develop them to understand their needs, identifying possible improvements, and contributing to them.

[MgSam]

@divega The fact that these projects exist and are so popular show that there is critical "base functionality" missing from EF, and that whatever feedback mechanisms you guys use are not sufficient. Take bulk functionality for example, though perhaps not directly comparable, 75,000 downloads is 250 times more popular than the highest upvoted item on this forum.

If you look through the top NuGet packages for EF, they are all for these sorts of Enterprise features that I've mentioned. On the other hand, if you look for packages implementing the stuff the EF team is actually working on and on the roadmap they usually don't exist. If people really thought these features were so important they would be implementing themselves in their own NuGet packages and not waiting for the EF team.

The disconnect between what the EF team thinks is important to implement and what people using the product actually think is important is vast and I've shown you the clear evidence that proves this point. If you refuse to believe it you just underline my additional point on how poorly the EF team does in supporting actual enterprises that need a robust ORM solution.

[ajcvickers]

@MgSam Would you say more people need this than many-to-many and TPT? These are two things we think are super important, but are still on the backlog.

(The question isn't really about how important we think it is, but about where to put the limited resources that we have. We have a huge backlog that we believe is full of important features, but we only have the resources to work on a tiny fraction of those features in any given release.)

[sandersaares]

Would you say more people need this than many-to-many and TPT? These are two things we think are super important, but are still on the backlog.

Absolutely! Working with large data set import is far more important than more fanciful relationship models (what exist right now is perfectly fine on that angle, IMO). We need powerful and broad-scoped basic capabilities, not advanced data modeling.

[f135ta]

@sandersaares - I'm inclined to agree with the points here. Bulk data operations is a key component of any Enterprise solution that uses EF. The amount of times I've seen code hacked together on projects to do this is frightening. It would be a REALLY useful feature to have built into EF Core - I can however appreciate the demands placed on the EF team and realise that if you could wave a magic wand; I'm sure it'd appear quick-time..

Maybe what we really need is more people contributing PR's instead of moaning about 'free' software not doing the job. After all, you get a massive stack of functionality handed to you already - if there are missing pieces to the puzzle, theres nothing stopping people from contributing too.