#Bug? - migrate:settings doesn't preserve special characters

[MagePsycho]

I am in the process of data migration from Magento EE 1.14.2.0 to Magento EE 2.1.0.
Running the command php bin/magento migrate:settings {path/to/data-migration/config.xml} migrates the settings and store data but the values with special characters are replaced by ???.

From where it handles the UTF-8 thing?

[victor-v-rad]

Hi @MagePsycho

Try this patch #28

[victor-v-rad]

It is not really clear what you are trying to migrate settings or data. But settings should go first anyway. Data is the second step.

[MagePsycho]

Thanks for the link to the #patch.
Yeah I am running the migrate:settings in the first step (edited now).
BTW, I have one query: Do the data-migration-tool only operates on data? or it's also responsible for creating non-existing tables or fields?

[ilol]

operates with the data only. You have to create needed data table and fields before. MT provides the information for this .

[victor-v-rad]

it's also responsible for creating non-existing tables or fields?

It creates m2_cl_* tables and trg_* triggers in your Magento 1 DB. Also temporary url_rewrite_m2* table is created because you have EE

[victor-v-rad]

An internal issue MAGETWO-54492 regarding to special characters was created

[MagePsycho]

Perfect.

BTW, the patch for UTF-8 didn't work for me. Not sure what I have to reload after the change. I tried to run the command with option --reset
Is there anything specific that we have to do to reflect the changes in the data-migration-tool?

[victor-v-rad]

You could share the DB dump of the particular table where you get ??? after migration. We will try to migrate it. You can send it through the email (find it in my profile https://github.com/victor-v-rad)

[MagePsycho]

@victor-v-rad it's the core_config_data table.
Not sure what am I missing.

[victor-v-rad]

What is the Collation / Character Set in your DB of M1 and M2?

[MagePsycho]

M1 utf8 utf8_general_ci
M2 latin1 latin1_swedish_ci

[victor-v-rad]

Have you tried to make your M2 utf8 utf8_general_ci also?

[MagePsycho]

Yeah I run the ALTER DATABASE M2 CHARACTER SET utf8 COLLATE utf8_general_ci;
But nothing is helping me, getting lost.

[MagePsycho]

@victor-v-rad This issue has been fixed.
In fact I was using wrong array:
File: vendor/magento/data-migration-tool/src/Migration/ResourceModel/Destination.php > getResourceConfig()
$config['initStatements'] = $destination[$destinationType]['initStatements'];
which should be
$config['database']['initStatements'] = $destination[$destinationType]['initStatements'];

And similar change for vendor/magento/data-migration-tool/src/Migration/ResourceModel/Destination.php

[victor-v-rad]

Glad to hear that! So the fix #28 works after your change
$config['database']['initStatements'] = $destination[$destinationType]['initStatements'];
?

[MagePsycho]

Yes. that fixed it.
But not all rows were transferred from M1 to M2 (context: core_config_data).
Noticeable, base urls were not transferred. unable to login to M2 (Fix: update cookie settings), frontend showing design exception etc.
My Question here is: what kind of data are not transferred from M1 to M2 during migrate:settings ?
Thanks again for your help.

[victor-v-rad]

See etc/ce-to-ce/settings.xml.dist for details

[MagePsycho]

gotcha! Thanks a lot :)

One more Question:
In which case .xml.dist needs to be renamed to .xml?

<settings_map_file>etc/ee-to-ee/settings.xml.dist</settings_map_file> needs to be changed to
<settings_map_file>etc/ee-to-ee/settings.xml</settings_map_file> after renaming settings.xml.dist file?
or as-is will work fine.

[victor-v-rad]

both cases are possible. But renaming it is recommendation

[victor-v-rad]

The issue is a duplicate and will be fixed in #96