Create a build script to make a WordPress plugin

[pdfernhout]

There should be an easier way to make a WordPress plugin for installation than the manual process outlined in wordpress-plugin/README.md. This is a spinoff from issue #1.

[pdfernhout]

My original plan had been to use npm to build the WordPress plugin using this approach. As I think about the WordPress build process, there really is no specific reason a WordPress or Drupal developer might have npm installed already though. So, maybe there might be better approaches for building PHP plugins than asking someone to install npm? But at least I can start with an npm script and go from there.

Also, I'm not sure how to compile TypeScript code without first installing Node.js and npm (unless it is done via an IDE, which again, may not be installed). It sounds like the npm and tsc installation part of setup went smoothly though as discussed in issue #1.

== More details behind why this installation issue is as complex as it is, requiring a build step

Even ignoring the issue of compiling TypeScript into JavaScript, one big design conflict leading to installation complexities (like in issue #1) is that I'm trying to support both a WordPress plugin and a Node.js server.

If I was just supporting the WordPress plugin, I could just have "narrafirma.php" in the WebContent directory (and I did have it there briefly at one point). But doing that means users of the Node.js approach will have an extra php file around (unless they delete it). Also, that approach creates another issue where a zipped directory name internally is "WebContent" and not "narrafirma", where someone unzipping such a distribution file into a WordPress installation's plugins directory would expect "narrafirma". I could rename "WebContent" as narrafirma, but that creates other possible confusion. The WordPress plugin also has some extra JavaScript just for the admin interface, and it is not clear where to put it if not in its own directory.

Also, in the future, other supporting php files might be created if that code is expanded or refactored, which would increasing the clutter for Node.js users. The narrafirma.php already is big for one file, and to make a Drupal plugin at some point, it would probably make sense to split it into a support file useable by WordPress and Drupal and files specific to each platform. And then, there might be an issue of where to put things if both WordPress and Drupal shared some of the same code.

I'm not sure how to resolve this design conflict, other than to have separate directories for the php files with a separate npm (or similar) build step and an output directory. However, a build step makes it harder to develop and test quickly. There is no build step required for the Node.js server.

I'm open to design suggestions on improving the file layout to make the building process easier. Perhaps there is some standard layout for this situation? Or maybe not many people are yet writing applications that run with both Node.js and WordPress (and maybe someday Drupal)? I know CiviCRM runs with WordPress, Joomla, and Drupal, so maybe there is some design experience there that we could learn from?

[pdfernhout]

It seems that PHP-based CiviCRM used Node.js as part of its current build process (as from around the begining of 2015) and now lists Node.js as a requirement (even if it is not used in all the tools). See also here:

"Beginning with CiviCRM v4.6, developing code for CiviCRM (civicrm-core.git) will require two additional tools: Composer and NodeJS. If you develop CiviCRM, then you should install them in order to continue development. ... In recent development cycles, CiviCRM increasingly implements major features using Javascript logic and APIv3. For example, v4.5 introduces a CiviCase configuration GUI based on AngularJS, and v4.6 introduces a revised CiviMail GUI also based on AngularJS. As more code is written in Javascript, Javascript testing becomes more important. Unfortunatey, neither PHPUnit nor PHPUnit-Selenium is well-suited to testing client-side Javascript code. The Javascript community has produced some great tools for Javascript testing. Generally, these tools require NodeJS. Specifically, in AngularJS the standard practice is to write tests with Karma and Jasmine -- which are better suited for testing Javascript code (more concise, more performant, better documentation). And they require NodeJS."

So that's another datapoint suggesting it is reasonable to require Node.js for the NarraFirma build process. Still, the requirmenent seems mainly to be about testing JavaScript code which is increasingly a part of the CiviCRM frontend. So, an alternative for those interested in the WordPress plugin only might still be to have a PHP script to do the building. Still, hopefully there will be be more tests for the JavaScript code, in which case NarraFirma developers would still want to install Node.js.

So, especially given I'm more comfortable with developing JavaScript than PHP, I'm going to proceed with the Node.js npm approach, although a PR with a PHP build script would still be welcome.

[pdfernhout]

I've been thinking more about changing the file layout to help make the build process easier.

I've looked as several WordPress plugins available on GitHub here which are mirrored from the WordPress plugin SVN. Of the few I've looked at at random, it seems they all have the format of having the plugin PHP file at the top level. I am wondering if that is just a required file layout for such plugins when they are hosted in that SVN repository? I don't think the few I have looked at attempt to support more than just WordPress though.

By contrast, CiviCRM has multiple repositories and apparently has build tools for WordPress, Joomla, and Drupal. However, CiviCRM is not in the WordPress plugin directory (although a few third-party add-ons for it are).

One possible approach would be to use the WordPress plugin file layout and put narrafirma.php at the root level of the repository. The Node.js server code would then be included though if the directory was just zipped without a build step. This might be confusing though if a Drupal plugin was also added at some point. Also, there are other directories in the NarraFirma project related to documentation, testing, experiments, and so on that are not intended to be distributed with the plugin.

So, I'm still not that inclined to reorganize the code to have the narrafirma.php file at the root level, even though it would simplify the build process for now. I may have to learn more about the options for submitting a WordPress plugin in the official repository to see if there are other options.

[pdfernhout]

For future reference, here are some scripts to sync GitHub to the WordPress plugin SVN repository. Presumably they could also be modified to selectively copy files or to do other build processes like TypeScript compilation? Or, perhaps a Node.js npm command could relatively easily be created to do much the same, since most of what they do seems to be to call command line tools? The "push.php" script is written to run as a GitHub hook, but it might be good enough to have a similar script that just was run manually locally now and then.

If the SVN repository is being updated using such a script, then the specific organization of files in this repository might not matter all that much. The script updating SVN from GitHub could adjust for the different file layouts. Of course, reducing the need for adjustments is always worthwhile, all other things being equal. But changing the file layout to make WordPress installation easier might not be worth the cost of making supporting multiple platforms (Node.js, WordPress, Drupal, or whatever else) more confusing than it has to be by moving away from server module code being in self-contained subdirectories in the project.

[pdfernhout]

I have created a first cut at a build script using npm. It probably works only under Mac and Linux -- or Windows with a bash-like shell that supports commands like "cp", "mkdir", and so on. (For future reference, adding the shelljs package or something similar as a development dependency could be a way around this limitation for Windows.)

The command is "npm run build-wp" and needs to be run from the top-level project directory.

It is described in more detail in the wordpress-plugin.README.md.

Running the command creates the file "distribution/narrafirma.zip" which can be uploaded into WordPress or unzipped in the WordPress plugins directory (after copying some files to assemble the zip file contents and then deleting those extra files afterwards). The process seems to work, but it probably is brittle if anything changes and does not provide much feedback on its progress or on error conditions.

The zip file size (1,787,687 bytes) is just under 2MB, which is the default limit for PHP uploads. To get it down to under 2MB, I had to move out of the plugin some mockup images which were thankfully no longer needed. If much more content is added to the plugin, it might be worth trying hard to keep the overall zip file size under that 2MB limit for easy installation via uploading. I'm not sure if the 2MB limit applies when installing form the official WordPress plugin repository, so maybe at some point the 2MB limit will not be so important to try to stay under.

@eliza411 If you get time, it would be great if you could try the build-wp command on your system and see if the resulting zip file can be installed OK. Also, any suggestions for improving the related documentation would be appreciated.

[pdfernhout]

@eliza411 If you try the build script with your most recent setup from following the previous manual build approach, you may see a TypeScript compile error related to duplicate symbols. To prevent that error, you would need to delete the copy of WebContent you made under the narrafirma directory (assuming you have not made any changes to that code).

That error is because TypeScript seems to be trying to compile all the "*.ts" files in the project regardless of their location, and does not yet support wildcards or exclude for file names in tsconfig.json in the currently generally available tsc 1.5.3 version (although tsc 1.6 will support "exclude"). Related TypeScript issues about tsconfig.json are here and here.

A similar error shows up if a copy of WebContent is left under the "distribution" directory, which is why the build-wp script deletes that copy after making the zip file. Otherwise I would have left that directory around for review and possibly a symbolic link from the WordPress directory like you are using now to the "wordpress-plugin/narrafirma" directory. There might be other ways around this general issue, like by moving the tsconfig.json file into the WebContent/source directory and making other related adjustments including for tsc options or the directory from which it is run, but I have not tried that.

And of course, also related to your current setup from the previous manual build approach, you would also need to replace the symbolic link from the WordPress plugin directory with the unzipped contents of distribution/narrafirma.zip.

[pdfernhout]

@eliza411The build script seems to be consistently working OK for me and Cynthia. So I am closing this issue. Feel free to open another one if the build script is not working for you.