[Data Liberation] Add WXR import CLI script

[zaerl]

Add Data Liberation import script. The script lets you import a folder with WXRs inside WordPress. Add the possibility to run PHPUnit inside Playground.

cd packages/playground/data-liberation/bin/import
bash import-wxr.sh /a-folder/with-the/wxr-files-to-import-inside

cd packages/playground/data-liberation
nx run test:wp-phpunit

The import CLI is also registered as a WP-CLI command in the init action if WP-CLI is included. So it can also be run as wp data-liberation your-wrx-file-you-want-to-import.xml.

Motivation for the change, related issues

There's no good entry point to running that import right now; we use an ad-hoc code snippet inside the Data Liberation WordPress plugin. This new CLI command will make testing the import easy.

There must be also be the possibility of running the PHPUnit test in the context of WordPress.

1. New CLI script
2. Added PHPUnit run inside Playground
3. Fix: missing require_once
4. Fix: wrong method name
5. Fix: endless loop

Implementation details

This script consists of six major parts.

The bin/import/import-wxr.sh bash script

This script accepts a folder path. You can create one and put all the WXR you want to import inside it. It starts the cli.ts server, mounts the folder specified in /wordpress/wp-content/uploads/import-wxr.

The bin/import/blueprint-import-wxr.json blueprint

The bluescript enables the Data Liberation plugin. Enumerate all the files with .xml extension inside the mounted folder and import them all using a new function created.

The PHP snippet run in the runPHP step uses the wp_visit_file_tree provided by the plugin:

<?PHP

require_once 'wordpress/wp-load.php';

$upload_dir = wp_upload_dir();

foreach ( wp_visit_file_tree( $upload_dir['basedir'] . '/import-wxr' ) as $event ) {
  foreach ( $event->files as $file ) {
    if ( $file->isFile() && pathinfo( $file->getPathname(), PATHINFO_EXTENSION ) === 'xml' ) {
      data_liberation_import( $file->getPathname() ); // Import the WXR.
    }
  }
};

A new data_liberation_import function

The new simple import function in the plugin runs WP_Stream_Importer and not much more.

The new tests/import/run.sh script

This script runs PHPUnit inside Playground. It generates an error if PHPUnit generates an error.

The new tests/import/blueprint-import.json blueprint

This blueprint runs all PHPUnit tests found in tests inside Playground. It returns success if everything goes well. It returns an error if one or more tests fail.

$base = '/wordpress/wp-content/plugins/data-liberation/';
require $base . 'vendor/autoload.php';

try {
    $arguments = [
        '--stderr',
        '--configuration', $base . 'phpunit.xml'
    ];

    $res = (new PHPUnit\TextUI\Application())->run($arguments);

    if ( $res !== 0 ) {
        trigger_error('PHPUnit failed', E_USER_ERROR);
    }
} catch (Throwable $e) {
    trigger_error('PHPUnit failed: ' . $e->getMessage(), E_USER_ERROR);
}

New unit test

The new WPStreamImporterTests class runs the first test using WP_Stream_Importer::create_for_wxr_file. It is only runnable inside WordPress, so there is a check in setUp() if it's the right environment. Otherwise, it is not run.

Testing Instructions (or ideally a Blueprint)

Import script

Example with one of the preexisting XML files:

cd packages/playground/data-liberation/bin/import
mkdir tmp
cp ../../tests/wxr/small-export.xml tmp
bash import-wxr.sh ./tmp

Then check http://127.0.0.1:9400/wp-admin/edit.php. All the WXR posts should be there.

PHPUnit run inside Playground

Run test on local:

cd packages/playground/data-liberation
nx run test:phpunit

1188 tests should succeed.

Run PHPUnit on Playground:

cd packages/playground/data-liberation
nx run test:wp-phpunit

All tests should succeed and output "Successfully ran target test:wp-phpunit for project playground-data-liberation".

[adamziel]

This is a great start Francesco, thank you! What would it take to expand this to run the actual PHPunit test in context of WordPress — similarly to what WordPress core tests do? I'm not saying this would actually be useful at this early stage, but I'm just curious maybe it wouldn't be that heavy of a lift? There's some prior art in this repo if you search for PHPUnit

[zaerl]

This is a great start Francesco, thank you! What would it take to expand this to run the actual PHPunit test in context of WordPress — similarly to what WordPress core tests do? I'm not saying this would actually be useful at this early stage, but I'm just curious maybe it wouldn't be that heavy of a lift? There's some prior art in this repo if you search for PHPUnit

I have added the possibility to run PHPUnit in Playground and updated the PR description with details. When you have time, please let me know what you think. Thanks.

[adamziel]

At this point, a dedicated WP_CLI command might make sense. It would only be a thin wrapper. The website and the unit tests would use the same underlying import library with their own dedicated logging facilities.

[zaerl]

Yes. I made this way so that a user that want to use only the plugin do not need to have WP-CLI all the times.

[adamziel]

Cool idea! This will suffice for starters, but here's something if you'd like to the next level. What would it take to go from this to something more like a typical CLI command, e.g. cli --blueprint=... --mount=... run vendor/bin/phpunit --configuration phpunit.xml --stderr?

[zaerl]

Great idea. Didn't touched the CLI in this first phase.

[adamziel]

Aside of my two notes, this looks good. Thank you Francesco! A useful next step would be adding a few actual assertions assertions and running that test in the CI

[zaerl]

I have added an issue to track the new step for the CLI. Fell free to change it if you want.