Magento 2.4 Upgrade issue Unable to apply data patch Magento\CatalogUrlRewrite\Setup\Patch\Data\UpdateUrlKeyForProducts #29365

sunel · 2020-08-03T06:53:28Z

Preconditions (*)

Magento 2.4.0
Magento 2.4-develop

Steps to reproduce (*)

Composer upgrade from 2.3.4 to 2.4.0
php bin/magento setup:upgrade

Expected result (*)

Successful upgrade

Actual result (*)

Module 'Magento_EavGraphQl':                                                                                                                                                                                                                  
Module 'Magento_Cms':                                                                                                                                                                                                                         
Module 'Magento_CatalogImportExport':                                                                                                                                                                                                         
Module 'Magento_Catalog':                                                                                                                                                                                                                     
Module 'Magento_CatalogInventory':                                                                                                                                                                                                            
Module 'Magento_Rule':                                                                                                                                                                                                                        
Module 'Magento_Payment':                                                                                                                                                                                                                     
Module 'Magento_CatalogRuleGraphQl':                                                                                                                                                                                                          
Module 'Magento_CatalogRule':                                                                                                                                                                                                                 
Module 'Magento_CatalogUrlRewrite':                                                                                                                                                                                                           
Unable to apply data patch Magento\CatalogUrlRewrite\Setup\Patch\Data\UpdateUrlKeyForProducts for module Magento_CatalogUrlRewrite. Original exception message: SQLSTATE[23000]: Integrity constraint violation: 1452 Cannot add or update a c
hild row: a foreign key constraint fails (`m24`.`catalog_product_entity_varchar`, CONSTRAINT `CAT_PRD_ENTT_VCHR_ENTT_ID_CAT_PRD_ENTT_ENTT_ID` FOREIGN KEY (`entity_id`) REFERENCES `catalog_product_entity` (`entity_id`) ON DELETE CA), q
uery was: INSERT  INTO `catalog_product_entity_varchar` (`value_id`,`value`) VALUES (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?
, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), 
(?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?)
, (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, 
?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?
, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), 
(?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?)
, (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, 
?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?
, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), 
(?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?)
, (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, 
?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?
, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), 
(?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?)
, (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, 
?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?
, ?), (?, ?), (?, ?), (?, ?), (?, ?) ON DUPLICATE KEY UPDATE `value` = VALUES(`value`)

Please provide Severity assessment for the Issue as Reporter. This information will help during Confirmation and Issue triage processes.

Severity: S0 - Affects critical data or functionality and leaves users without workaround.
Severity: S1 - Affects critical data or functionality and forces users to employ a workaround.
Severity: S2 - Affects non-critical data or functionality and forces users to employ a workaround.
Severity: S3 - Affects non-critical data or functionality and does not force users to employ a workaround.
Severity: S4 - Affects aesthetics, professional look and feel, “quality” or “usability”.

m2-assistant · 2020-08-03T06:53:31Z

Hi @sunel. Thank you for your report.
To help us process this issue please make sure that you provided the following information:

Summary of the issue
Information on your environment
Steps to reproduce
Expected and actual results

Please make sure that the issue is reproducible on the vanilla Magento instance following Steps to reproduce. To deploy vanilla Magento instance on our environment, please, add a comment to the issue:

@magento give me 2.4-develop instance - upcoming 2.4.x release

For more details, please, review the Magento Contributor Assistant documentation.

Please, add a comment to assign the issue: @magento I am working on this

Join Magento Community Engineering Slack and ask your questions in #github channel.

⚠️ According to the Magento Contribution requirements, all issues must go through the Community Contributions Triage process. Community Contributions Triage is a public meeting.

🕙 You can find the schedule on the Magento Community Calendar page.

📞 The triage of issues happens in the queue order. If you want to speed up the delivery of your contribution, please join the Community Contributions Triage session to discuss the appropriate ticket.

🎥 You can find the recording of the previous Community Contributions Triage on the Magento Youtube Channel

✏️ Feel free to post questions/proposals/feedback related to the Community Contributions Triage process to the corresponding Slack Channel

pmclain · 2020-08-04T19:59:11Z

Wonder if this should be update on duplicate instead of insert on duplicate.
https://github.com/magento/magento2/blob/2.4.0/app/code/Magento/CatalogUrlRewrite/Setup/Patch/Data/UpdateUrlKeyForProducts.php#L73

dharake · 2020-08-05T21:08:36Z

I can confirm we are experiencing the same issue when trying to upgrade from CE 2.3.5-p2 to 2.4.0

dharake · 2020-08-05T21:09:21Z

Wonder if this should be update on duplicate instead of insert on duplicate.
https://github.com/magento/magento2/blob/2.4.0/app/code/Magento/CatalogUrlRewrite/Setup/Patch/Data/UpdateUrlKeyForProducts.php#L73

I tried this with no success unfortunately. Unrecognised class error or something similar

MichaelThessel · 2020-08-05T23:00:08Z

Can confirm issue. Upgrading from 2.3.5 to 2.4.

moni921 · 2020-08-06T16:44:59Z

Any solution, we are stuck here

sunel · 2020-08-06T17:18:41Z

@moni921 for now you can disable and re-enable foreign key constrain

...php
$setup->getConnection()->query('SET @TEMPORARY_FOREIGN_KEY_CHECKS=@@FOREIGN_KEY_CHECKS, FOREIGN_KEY_CHECKS=1');
$this->groupRepository->delete($customerGroup);
$setup->getConnection()->query('SET FOREIGN_KEY_CHECKS=IF(@TEMPORARY_FOREIGN_KEY_CHECKS=0, 0, 1)');
...

vanvasten · 2020-08-06T21:01:58Z

Can confirm issue. Upgrading from 2.3.5-p1 to 2.4.

BavoJanss · 2020-08-07T12:54:34Z

For the update to succeed all keys should be included in the query.
Change lines according below in Magento\CatalogUrlRewrite\Setup\Patch\Data\UpdateUrlKeyForProducts:

@@ -58,7 +58,7 @@
         $table = $this->moduleDataSetup->getTable('catalog_product_entity_varchar');
         $select = $this->moduleDataSetup->getConnection()->select()->from(
             $table,
-            ['value_id', 'value']
+            ['entity_id', 'attribute_id', 'store_id', 'value_id', 'value']
         )->where(
             'attribute_id = ?',
             $this->eavSetup->getAttributeId($productTypeId, 'url_key')

dharake · 2020-08-07T12:58:43Z

For the update to succeed all keys should be included in the query.
Change lines according below:

@@ -58,7 +58,7 @@
         $table = $this->moduleDataSetup->getTable('catalog_product_entity_varchar');
         $select = $this->moduleDataSetup->getConnection()->select()->from(
             $table,
-            ['value_id', 'value']
+            ['entity_id', 'attribute_id', 'store_id', 'value_id', 'value']
         )->where(
             'attribute_id = ?',
             $this->eavSetup->getAttributeId($productTypeId, 'url_key')

Can you provide more details? Which file are you changing?

vanvasten · 2020-08-07T14:29:32Z

Hi @dharake
@BavoJanss is talking about file "UpdateUrlKeyForProducts.php" located in > vendor/magento/module-catalog-url-rewrite/Setup/Patch/Data

It worked for me!!
Thanks @BavoJanss

dharake · 2020-08-07T15:21:05Z

Hi @dharake
@BavoJanss is talking about file "UpdateUrlKeyForProducts.php" located in > vendor/magento/module-catalog-url-rewrite/Setup/Patch/Data

It worked for me!!
Thanks @BavoJanss

Thanks @vanvasten, glad it worked for you. Will try it out

hostep · 2020-08-07T16:11:34Z

Tagging @engcom-Echo in here, since he wrote that patch, maybe he has some more useful insights and can push for a quick fix to be added to Magento 2.4.1 or a hotfix to be released if it is deemed necessary 🙂

dverkade · 2020-08-17T12:28:39Z

Please note that this patch is not working for commerce installations. entity_id is not a column name for commerce, it should be row_id instead.

HeiderSati · 2020-08-31T12:33:50Z

I reported the same problem in ticket 29805, closed due to being duplicated to this one.

I am posting my solution to the issue below whilst Magento is taking time to push another update:

->

Further into the problem details, I debugged the installation and can see the following in your upgrade code:

Under vendor/magento/module-catalog-url-rewrite/Setup/Patch/Data/UpdateUrlKeyForProducts.php

You have:

Line-67 says:

$result = $this->moduleDataSetup->getConnection()->fetchAll($select);

This reads all details from the catalog_product_entity_varchar table, on my installation it loads over 8000 records.

Then you have a foreach look that is updating the values within the array:

$result[$key]['value'] = $this->urlProduct->formatUrlKey($item['value']);

So far so good,...

But then after the array is finished above, you are then re-inserting the same IDs into the database with the
insertOnDuplicate function that is known to throw an error if the record already exists:

foreach (array_chunk($result, 500, true) as $pathResult) {
    $this->moduleDataSetup->getConnection()->insertOnDuplicate($table, $pathResult, ['value']);
}

This means that the upgrade will NEVER succeed whenever there is data in an existing version.

I wonder,... have you guys tested this before launching v2.4 ?

The solution is to:

Please don't re-insert/update all values into the database from the array you just loaded as it causes performance problems, or upgrade too slow etc), rather, have a logic that says:

$OLD = $item['value'];
$NEW = $this->urlProduct->formatUrlKey($item['value']);

if ($OLD !== $NEW)
{
... update the database
}
Replace the insertOnDuplicate function with an UPDATE instead.

I hope this helps, I posted the following:

A) Exact issue
B) Exact files causing this issue
C) Code snippets on what to change to fix the issue

In other words, the existing logic of:

Read database values,
Do some changes (if any)
Re-Insert back into the database with the function that throws an exception/error if the value is there already
Of course would fail, mainly because the ID/Keys are already in the database.

The solution is to change (3) into (Update if the value change)... please remove the Insert function at the end of the Apply() function.

If you require any further information, then please let me know.

Kind Regards
Heider

luka4 · 2020-09-09T12:01:02Z

For the update to succeed all keys should be included in the query.
Change lines according below in Magento\CatalogUrlRewrite\Setup\Patch\Data\UpdateUrlKeyForProducts:

@@ -58,7 +58,7 @@
         $table = $this->moduleDataSetup->getTable('catalog_product_entity_varchar');
         $select = $this->moduleDataSetup->getConnection()->select()->from(
             $table,
-            ['value_id', 'value']
+            ['entity_id', 'attribute_id', 'store_id', 'value_id', 'value']
         )->where(
             'attribute_id = ?',
             $this->eavSetup->getAttributeId($productTypeId, 'url_key')

why "entity_id" ?? that must be "row_id". e.g.:
['row_id', 'attribute_id', 'store_id', 'value_id', 'value']

ihor-sviziev · 2020-10-28T09:54:58Z

Hi @engcom-Alfa @engcom-Bravo @engcom-Charlie ,
Could you try to reproduce this issue again with configuring magento in the same way?

Two store views - en & de
Enable showing store code in the URL
Configure different URL keys for categories and for products for the DE & EN versions
Try to upgrade

paul-blundell · 2020-11-03T11:50:36Z

We had the same issue when upgrading from 2.3.4 to 2.4.1.
As someone else has mentioned, applying this pull request as a patch seemingly works.

christianbookpwood · 2020-12-16T18:29:34Z

I can confirm I'm also having this issue trying to upgrade from magento community 2.3.4 to 2.4.1:

Unable to apply data patch Magento\CatalogUrlRewrite\Setup\Patch\Data\UpdateUrlKeyForProducts for module Magento_CatalogUrlRewrite. Original exception message: SQLSTATE[23000]: Integrity constraint violation: 1452 Cannot add or update a child row: a foreign key constraint fails (XXredactedXX.catalog_product_entity_varchar, CONSTRAINT CAT_PRD_ENTT_VCHR_ENTT_ID_CAT_PRD_ENTT_ENTT_ID FOREIGN KEY (entity_id) REFERENCES catalog_product_entity (entity_id) ON DELET), query was: INSERT INTO catalog_product_entity_varchar (value_id,value) VALUES (?, ?), (--many snips--) (?, ?) ON DUPLICATE KEY UPDATE value = VALUES(value)

christianbookpwood · 2020-12-16T18:48:46Z

I can also confirm that applying the patch from @ravi-chandra3197 allowed the upgrade to proceed. https://github.com/magento/magento2/pull/29804/files

ihor-sviziev · 2020-12-16T20:08:10Z

@christianbookpwood any ideas how to reproduce this issue on clean magento installation?

christianbookpwood · 2020-12-16T20:35:15Z

@christianbookpwood any ideas how to reproduce this issue on clean magento installation?

The only thing that comes to mind is that, back when we upgraded to Magento 2.3.3, we had an issue with URL keys, as did a number of other people, described in this issue: #26499 -- it was still an issue when we upgraded to 2.3.4, which is the version we're currently on. We're now trying to upgrade from 2.3.4 to 2.4.1.

We had applied the fix specified by that issue reporter to our version of vendor/magento/module-catalog-url-rewrite/Model/ProductUrlPathGenerator.php, and thereafter our url keys were properly updated. We had to reapply this fix when we upgraded from 2.3.3 to 2.3.4.

These modules both seem to deal with Url keys, so I'm wondering if our previous fix to the ProductUrlPathGenerator.php file put our system into a state that is causing the issue with UpdateUrlKeyForProducts.php.

thomas-kl1 · 2021-01-11T09:02:37Z

I got that issue too, trying to upgrade 2.2 to 2.4.1.

Unable to apply data patch Magento\CatalogUrlRewrite\Setup\Patch\Data\UpdateUrlKeyForProducts for module Magento_CatalogUrlRewrite. Original exception message: SQLSTATE[23000]: Integrity constraint violation: 1452 Cannot add or update a child row: a foreign key constraint fails (mpbio_qa.catalog_product_entity_varchar, CONSTRAINT CAT_PRD_ENTT_VCHR_ENTT_ID_CAT_PRD_ENTT_ENTT_ID FOREIGN KEY (entity_id) REFERENCES catalog_product_entity (entity_id) ON DELETE C), query was: INSERT INTO catalog_product_entity_varchar (value_id,value) VALUES (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?) ON DUPLICATE KEY UPDATE value = VALUES(value)

thomas-kl1 · 2021-01-11T11:11:51Z

For the update to succeed all keys should be included in the query.
Change lines according below in Magento\CatalogUrlRewrite\Setup\Patch\Data\UpdateUrlKeyForProducts:

@@ -58,7 +58,7 @@
         $table = $this->moduleDataSetup->getTable('catalog_product_entity_varchar');
         $select = $this->moduleDataSetup->getConnection()->select()->from(
             $table,
-            ['value_id', 'value']
+            ['entity_id', 'attribute_id', 'store_id', 'value_id', 'value']
         )->where(
             'attribute_id = ?',
             $this->eavSetup->getAttributeId($productTypeId, 'url_key')

this seems to fix it

magento-engcom-team · 2021-02-01T11:47:28Z

✅ Confirmed by @engcom-Alfa
Thank you for verifying the issue. Based on the provided information internal tickets MC-40700 were created

Issue Available: @engcom-Alfa, You will be automatically unassigned. Contributors/Maintainers can claim this issue to continue. To reclaim and continue work, reassign the ticket to yourself.

…2.4 #29365 #29804 - Merge Pull Request #29804 from ravi-chandra3197/magento2:patch-29365 - Merged commits: 1. b2b29ea 2. a06ec39

…2.4 #29365 #29804

thanura · 2021-03-10T02:35:58Z

Upgrading from 2.3.6-p1 to 2.4.2. Still got the same issue. I have created the patch using the commit here : https://github.com/ravi-chandra3197/magento2/tree/patch-29365

` diff --git a/vendor/magento/module-catalog-url-rewrite/Setup/Patch/Data/UpdateUrlKeyForProducts.php b/vendor/magento/module-catalog-url-rewrite/Setup/Patch/Data/UpdateUrlKeyForProducts.php
index 5e70399129..f74d8ca358 100644
--- a/vendor/magento/module-catalog-url-rewrite/Setup/Patch/Data/UpdateUrlKeyForProducts.php
+++ b/vendor/magento/module-catalog-url-rewrite/Setup/Patch/Data/UpdateUrlKeyForProducts.php
@@ -34,19 +34,27 @@ class UpdateUrlKeyForProducts implements DataPatchInterface, PatchVersionInterfa
*/
private $urlProduct;

    +    /**
    +     * @var \Magento\Framework\EntityManager\MetadataPool
    +     */
    +    private $metadataPool;
    +
         /**
          * @param ModuleDataSetupInterface $moduleDataSetup
          * @param EavSetupFactory $eavSetupFactory
          * @param Url $urlProduct
    +     * @param \Magento\Framework\EntityManager\MetadataPool $metadataPool
          */
         public function __construct(
             ModuleDataSetupInterface $moduleDataSetup,
             EavSetupFactory $eavSetupFactory,
    -        Url $urlProduct
    +        Url $urlProduct,
    +        \Magento\Framework\EntityManager\MetadataPool $metadataPool
         ) {
             $this->moduleDataSetup = $moduleDataSetup;
             $this->eavSetup = $eavSetupFactory->create(['setup' => $moduleDataSetup]);
             $this->urlProduct = $urlProduct;
    +        $this->metadataPool = $metadataPool;
         }
     
         /**
    @@ -58,7 +66,7 @@ class UpdateUrlKeyForProducts implements DataPatchInterface, PatchVersionInterfa
             $table = $this->moduleDataSetup->getTable('catalog_product_entity_varchar');
             $select = $this->moduleDataSetup->getConnection()->select()->from(
                 $table,
    -            ['value_id', 'value']
    +            [$this->getProductLinkField(), 'attribute_id', 'store_id', 'value_id', 'value']
             )->where(
                 'attribute_id = ?',
                 $this->eavSetup->getAttributeId($productTypeId, 'url_key')
    @@ -99,4 +107,17 @@ class UpdateUrlKeyForProducts implements DataPatchInterface, PatchVersionInterfa
         {
             return [];
         }
    +
    +    /**
    +     * Return product id field name - entity_id|row_id
    +     *
    +     * @return string
    +     * @throws \Exception
    +     */
    +    private function getProductLinkField()
    +    {
    +        return $this->metadataPool
    +            ->getMetadata(\Magento\Catalog\Api\Data\ProductInterface::class)
    +            ->getLinkField();
    +    }
     }`

barbazul · 2021-05-12T20:31:39Z

I just noticed this fix is in the main branch but wasn't released in 2.4.2-p1 which is dissapointing

ihor-sviziev · 2021-05-12T20:44:25Z

@barbazul 2.4.2-p1 is a security-only release. That's why this change wasn't added

roman204 · 2021-07-07T15:55:48Z

For anyone who needs a ready-to-use patch file for composer patches:
here you go ->>> save it as patch file and configure it for composer patches, viola

Index: Setup/Patch/Data/UpdateUrlKeyForProducts.php
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/Setup/Patch/Data/UpdateUrlKeyForProducts.php b/Setup/Patch/Data/UpdateUrlKeyForProducts.php
--- a/Setup/Patch/Data/UpdateUrlKeyForProducts.php	(date 1625672707821)
+++ b/Setup/Patch/Data/UpdateUrlKeyForProducts.php	(date 1625672707821)
@@ -34,6 +34,11 @@
      */
     private $urlProduct;
 
+    /**
+     * @var \Magento\Framework\EntityManager\MetadataPool
+     */
+    private $metadataPool;
+
     /**
      * @param ModuleDataSetupInterface $moduleDataSetup
      * @param EavSetupFactory $eavSetupFactory
@@ -42,11 +47,13 @@
     public function __construct(
         ModuleDataSetupInterface $moduleDataSetup,
         EavSetupFactory $eavSetupFactory,
-        Url $urlProduct
+        Url $urlProduct,
+        \Magento\Framework\EntityManager\MetadataPool $metadataPool
     ) {
         $this->moduleDataSetup = $moduleDataSetup;
         $this->eavSetup = $eavSetupFactory->create(['setup' => $moduleDataSetup]);
         $this->urlProduct = $urlProduct;
+        $this->metadataPool = $metadataPool;
     }
 
     /**
@@ -58,7 +65,7 @@
         $table = $this->moduleDataSetup->getTable('catalog_product_entity_varchar');
         $select = $this->moduleDataSetup->getConnection()->select()->from(
             $table,
-            ['value_id', 'value']
+            [$this->getProductLinkField(), 'attribute_id', 'store_id', 'value_id', 'value']
         )->where(
             'attribute_id = ?',
             $this->eavSetup->getAttributeId($productTypeId, 'url_key')
@@ -99,4 +106,17 @@
     {
         return [];
     }
+
+    /**
+     * Return product id field name - entity_id|row_id
+     *
+     * @return string
+     * @throws \Exception
+     */
+    private function getProductLinkField()
+    {
+        return $this->metadataPool
+            ->getMetadata(\Magento\Catalog\Api\Data\ProductInterface::class)
+            ->getLinkField();
+    }
 }

magento-engcom-team added the Issue: Format is valid Gate 1 Passed. Automatic verification of issue format passed label Aug 3, 2020

ghost added the Progress: ready for QA label Aug 3, 2020

ravi-chandra3197 added a commit to ravi-chandra3197/magento2 that referenced this issue Aug 29, 2020

Fixed Unable to apply data patch issue magento#29365

b2b29ea

ravi-chandra3197 mentioned this issue Aug 29, 2020

Fixed Unable to apply data patch issue after upgrade in 2.4 #29365 #29804

Merged

4 tasks

ghost assigned ravi-chandra3197 Aug 29, 2020

ghost added Progress: PR in progress Severity: S1 Affects critical data or functionality and forces users to employ a workaround. and removed Progress: ready for QA labels Aug 29, 2020

krish-vivek mentioned this issue Aug 31, 2020

[Issue]:#29805 Resolved #29815

Closed

4 tasks

ihor-sviziev mentioned this issue Aug 31, 2020

Magento 2.4 upgrade fails if there is data under catalog_product_entity_varchar #29805

Closed

4 tasks

ghost added the Priority: P1 Once P0 defects have been fixed, a defect having this priority is the next candidate for fixing. label Aug 31, 2020

ghost unassigned ravi-chandra3197 Sep 11, 2020

magento-engcom-team added the Reported on 2.3.4 Indicates original Magento version for the Issue report. label Nov 13, 2020

engcom-Alfa added Reproduced on 2.4.x The issue has been reproduced on latest 2.4-develop branch Issue: Confirmed Gate 3 Passed. Manual verification of the issue completed. Issue is confirmed labels Feb 1, 2021

github-jira-sync-bot added Progress: ready for dev and removed Progress: PR in progress labels Feb 1, 2021

m2-community-project bot added Progress: PR in progress and removed Progress: ready for dev labels Feb 1, 2021

magento-engcom-team added a commit that referenced this issue Feb 5, 2021

ENGCOM-8403: Fixed Unable to apply data patch issue after upgrade in …

281a389

…2.4 #29365 #29804 - Merge Pull Request #29804 from ravi-chandra3197/magento2:patch-29365 - Merged commits: 1. b2b29ea 2. a06ec39

magento-engcom-team pushed a commit that referenced this issue Feb 5, 2021

ENGCOM-8403: Fixed Unable to apply data patch issue after upgrade in …

5749410

…2.4 #29365 #29804

magento-engcom-team closed this as completed in #29804 Feb 5, 2021

m2-community-project bot added Progress: done and removed Progress: PR in progress labels Feb 5, 2021

sdzhepa mentioned this issue May 10, 2022

2.4.3 Krish sdzhepa/magento2#67

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Magento 2.4 Upgrade issue Unable to apply data patch Magento\CatalogUrlRewrite\Setup\Patch\Data\UpdateUrlKeyForProducts #29365

Magento 2.4 Upgrade issue Unable to apply data patch Magento\CatalogUrlRewrite\Setup\Patch\Data\UpdateUrlKeyForProducts #29365

sunel commented Aug 3, 2020 •

edited by engcom-Alfa

Loading

m2-assistant bot commented Aug 3, 2020

pmclain commented Aug 4, 2020

dharake commented Aug 5, 2020

dharake commented Aug 5, 2020

MichaelThessel commented Aug 5, 2020

moni921 commented Aug 6, 2020

sunel commented Aug 6, 2020

vanvasten commented Aug 6, 2020 •

edited

Loading

BavoJanss commented Aug 7, 2020 •

edited

Loading

dharake commented Aug 7, 2020

vanvasten commented Aug 7, 2020

dharake commented Aug 7, 2020

hostep commented Aug 7, 2020

dverkade commented Aug 17, 2020

HeiderSati commented Aug 31, 2020

luka4 commented Sep 9, 2020

ihor-sviziev commented Oct 28, 2020 •

edited

Loading

paul-blundell commented Nov 3, 2020

christianbookpwood commented Dec 16, 2020

christianbookpwood commented Dec 16, 2020

ihor-sviziev commented Dec 16, 2020

christianbookpwood commented Dec 16, 2020

thomas-kl1 commented Jan 11, 2021

thomas-kl1 commented Jan 11, 2021

magento-engcom-team commented Feb 1, 2021

thanura commented Mar 10, 2021 •

edited

Loading

barbazul commented May 12, 2021

ihor-sviziev commented May 12, 2021 •

edited

Loading

roman204 commented Jul 7, 2021

Magento 2.4 Upgrade issue Unable to apply data patch Magento\CatalogUrlRewrite\Setup\Patch\Data\UpdateUrlKeyForProducts #29365

Magento 2.4 Upgrade issue Unable to apply data patch Magento\CatalogUrlRewrite\Setup\Patch\Data\UpdateUrlKeyForProducts #29365

Comments

sunel commented Aug 3, 2020 • edited by engcom-Alfa Loading

Preconditions (*)

Steps to reproduce (*)

Expected result (*)

Actual result (*)

m2-assistant bot commented Aug 3, 2020

pmclain commented Aug 4, 2020

dharake commented Aug 5, 2020

dharake commented Aug 5, 2020

MichaelThessel commented Aug 5, 2020

moni921 commented Aug 6, 2020

sunel commented Aug 6, 2020

vanvasten commented Aug 6, 2020 • edited Loading

BavoJanss commented Aug 7, 2020 • edited Loading

dharake commented Aug 7, 2020

vanvasten commented Aug 7, 2020

dharake commented Aug 7, 2020

hostep commented Aug 7, 2020

dverkade commented Aug 17, 2020

HeiderSati commented Aug 31, 2020

luka4 commented Sep 9, 2020

ihor-sviziev commented Oct 28, 2020 • edited Loading

paul-blundell commented Nov 3, 2020

christianbookpwood commented Dec 16, 2020

christianbookpwood commented Dec 16, 2020

ihor-sviziev commented Dec 16, 2020

christianbookpwood commented Dec 16, 2020

thomas-kl1 commented Jan 11, 2021

thomas-kl1 commented Jan 11, 2021

magento-engcom-team commented Feb 1, 2021

thanura commented Mar 10, 2021 • edited Loading

barbazul commented May 12, 2021

ihor-sviziev commented May 12, 2021 • edited Loading

roman204 commented Jul 7, 2021

sunel commented Aug 3, 2020 •

edited by engcom-Alfa

Loading

vanvasten commented Aug 6, 2020 •

edited

Loading

BavoJanss commented Aug 7, 2020 •

edited

Loading

ihor-sviziev commented Oct 28, 2020 •

edited

Loading

thanura commented Mar 10, 2021 •

edited

Loading

ihor-sviziev commented May 12, 2021 •

edited

Loading