Removing 85.000 products from backend takes 28 hours!

[rklerck]

We've added 85000 products with rest, this takes 14 hours. This seems fair because of a request for each product. But when we remove the products all at once in the backend, it takes 28 hours!!

[kandy]

First of all, better to use indexation on schedule for this case.
Second, in your case better to use import functionality that was optimized for mass update.

[rklerck]

Use import functionality for mass delete? We don't really care import takes this long, it's the initial load..

[vrann]

@rklerck hi, can you please confirm that you use "reindex on schedule" for all the indexers related to catalog?

[rklerck]

Tried that, ended up with #1833

[pboisvert]

@choukalos Chuck--i see your reference to a closed issue. Can you take a look at this one and reassign as needed?

[choukalos]

Hi @rklerck

What version of Magento are you using?
Did you use REST API to delete the entire catalog ( i.e. all 85k products )?
Is your use case the need to delete the entire catalog quickly - for what reason? Is this a merchant use case, why?
Do you have merchant use cases where you'd only want to delete portions of the catalog? What time to do so would you expect?

Thanks,
Chuck

[acidreian666]

@choukalos
Having similar issues with my large product database also. Magento 2.1 is unable to import more then 95000 products at a time if it don't fail already because it says the "URL already exists" and it ends with importing 66k of the 95k list of products then breaks the indexing ability.

Then like @rklerck says Mass Delete function for me i have let run for 3 days to delete over 85k products at once because they imported and had missing data causing an Segmentation Core error while indexing keywords.

Magmi for 1.92 was amazing at importing/updating products but this isn't an option for magento 2.x
imagine having a perfect Database CSV file made with 180k products then using magento 2.x OEM import tool and it fails everytime.

to compare to what ever @rklerck server is mine is.
Magento 2.1
ubuntu 16.04 php7.08 Nginx mysql5.7
4 core cpu 24gb ram
mysql and php both tuned to use high amounts of memory
execution times = umlimited

result = magento 2.x needs Magmi style import/update/delete system.

[piotrekkaminski]

Thank you for your submission.

We recently made some changes to the way we process GitHub submissions to more quickly identify and respond to core code issues.

Feature Requests and Improvements should now be submitted to the new Magento 2 Feature Requests and Improvements forum (see details here).

We are closing this GitHub ticket and have moved your request to the new forum.