Add migrate_run instead of migrate_tools

[chx]

https://www.drupal.org/node/2905741 says "Add Migrate Tools to Drush (contrib)"

Instead add https://www.drupal.org/project/migrate_run

migrate_tools depends on migrate_plus which is an incredibly, incredibly broken module, full of bad ideas. Basically, after a core maintainer asked migrations not to be configuration in https://www.drupal.org/node/2462233 some people sulked off to contrib and made them one nonetheless. As migrate_run shows, there's no need for this.

[weitzman]

I'll take a look. Happy to see some fresh ideas in this area.

[weitzman]

I'd love to see more issue queue activity and usage for migrate-run before putting it in drush core.

[weitzman]

migrate-run looks great. happy to see drush9 support there.

More clear requirements for putting this in Drush core

1. A maintainer. Ideally @claudiu-cristea keeps maintaining it as part of the Drush team
2. Tests
3. At least one person posts here saying that they've used the commands successfully on a project.

[claudiu-cristea]

@weitzman, I agree to maintain it as part of Drush. Related to tests, I guess we should use the Drush testing with unish as we test console commands but how this can be done as this is still a Drupal module?

[chx]

Do I count as a person :) ?

I have three projects at least where I used migrate import successfully, https://www.drupal.org/u/opdavies and me have recently used rollback as well and that worked, too. If you want a non-chx confirmation about this module working, ask him.

There are some improvements to made but nothing blocking: when running dependents, migrate_run runs dependent migrations multiple times.

[weitzman]

@chx counts as a small nation 💪. Thanks for the positive report. Thats good enough IMO.

@claudiu-cristea I think you stop being a module and ship these commands in the src/Drupal/Commands/core dir. And yes, Unish tests.

[claudiu-cristea]

#3402 is ready for review.

[claudiu-cristea]

@weitzman @chx, #3402 is ready but there’s still one important thing to figure out. Users of migrate_run module are covered, as this is a straight port of that module into Drush core. But users of migrate_tools that were using the migrate_plus way to store migrations as config entities? I guess they are not too many because migrate_tools knows also to run migrations stored as simple YAML files (not as config entities). The problem is that till we merge #3402 they're still able run their migrations. But once the Drush core migrate commands are in, buuum! I guess we have to think some kind of BC for them. Few ideas:

• migrate_tools is renaming their Drush commands.
• We are renaming the core commands in Add the migrate runner commands #3402

[chx]

migrate_tools is renaming their Drush commands.

For a little bit of background, alexpott have asked migrations not to be config entities. Once that happened, migrate_plus decided to go against the explicit guidance of a core maintainer and still do it. They also have plugins that go against the vision I implemented in the migrate architecture. So: they are very well used to do their own thing. Can't see a problem here, they stay in their own world and rename their commands.

[weitzman]

It would be helpful if folks could share their migrate ymls that have been used with migrate_run? Were they Drupal => Drupal upgrades or not? Seems like that use case has more merit for config entities than migrations from 3rd party sources.

@chx - please keep this discussion as positive as possible.

[chx]

Yeah, I am using it for Drupal to Drupal not sure why would they be different...? When I want to change a core YAML, I just copy it and change it. https://gist.github.com/chx/329c33825ec08aa222535a7a6bddc3d8 the first is the core file migration with source_base_path filled in and a constant added and used. The second in the gist is a completely custom migration. I find it easy to have them at the same place, easy to edit et al. The way this goes (for me) by the time we finish editing them, the project is over so having them in files is best.

[claudiu-cristea]

I wonder if there are use cases in the real world that are only using the migrate_drupal core module without writing or touching any of the migrate YAML files. I mean, you have a Drupal 6 site and you want to migrate it as it is, without changing, improving, adding new features? I'm still managing some D6 sites that will never change as features and I will keep them in D6. Why would someone migrate a D6/7 site to D8 without changing anything?

However, I'll add some docs on how to run and extend such a migration. Recently I had a big D6 > D8 migration and I kept migrate_drupal disabled but I copied and adapted files from migrate_drupal in my custom module.

[weitzman]

I actually think we should document the 'copy and adapt' approach. I agree that this is much more common.

[juampynr]

I guess that until this is merge it's better to use https://www.drupal.org/project/migrate_run, right?

[weitzman]

Correct. This PR actually got stuck and I'm unsure if its a good idea. Lets keep migrate_run as a separate project for now. Drush core is going to stay out of the migrate space for a while longer. Sorry for the extra work, @claudiu-cristea

[juampynr]

Thanks for the quick feedback @weitzman. Did you just say that Drush is going into core? If so, where can I read more about this?

[weitzman]

Hah, just a typo. Fixed now.