feat: CLI command to upgrade an existing Webpack config

[jaredcwhite]

As we continue to make the Webpack configuration in Bridgetown better out of the box, it highlights the need to allow existing sites' configurations to be upgraded. I shouldn't have to spin up a new site, then do a diff between the new site configuration and an older site of mine to figure out what's changed. That's the computer's job! 😉

I'm not sure exactly what to call the command, but I think if it basically copied an existing configuration to a backup copy, saved a new configuration, and then provided a visual diff between the two, that would be ideal. Maybe we even do a more rigorous job of maintaining a changelog just for the Webpack configuration, and link to that for reference.

[jaredcwhite]

P.S. I know some frameworks out there get around this sort of thing by hiding the configuration file entirely… requiring you to override it with changes only when needed.

I think that's a terrible idea. I think the Webpack config file should be completely accessible and out in the open for every project. All we need to do then is simply provide a way to upgrade it when necessary.

[ayushn21]

I was just thinking about this yesterday evening. In my mind I was toying with the idea of a super-lightweight Webpacker that would orchestrate Webpack. But that's opening one hell of a can of worms :D.

And your second comment has literally just covered what I was about to type!

[ayushn21]

I guess we can use the v4 -> v5 upgrade as a guinea pig and depending on the complexity of the changes (if any) required, we could figure out how to build this tool?

[ayushn21]

I guess we can use the v4 -> v5 upgrade as a guinea pig and depending on the complexity of the changes (if any) required, we could figure out how to build this tool?

Ignore me. I forgot about improvement we make to the config without a major version upgrade 🤦 . Last comment for now I promise!

[jaredcwhite]

I guess we can use the v4 -> v5 upgrade as a guinea pig and depending on the complexity of the changes (if any) required, we could figure out how to build this tool?

Ignore me. I forgot about improvement we make to the config without a major version upgrade 🤦 . Last comment for now I promise!

No those are really great points actually. I must admit I'm totally ignorant as to what would need to be tweaked in order to support Webpack v5. For the purposes of this conversation, how do we handle upgrades of the default state of webpack.config.js (not Webpack tooling, although that would benefit from config upgrades too)?

[KonnorRogers]

Thor's #create_file gives the option of doing a diff split merge, so you could always just overwrite what a user already has 🤷‍♂️

You can always ship an npm package like Webpacker for Rails does and then let users extend it.

https://github.com/rails/webpacker/tree/master/package

[ayushn21]

I must admit I'm totally ignorant as to what would need to be tweaked in order to support Webpack v5.

Ideally, we shouldn't have to change anything but I couldn't get the manifest plugin to work in v5 so I swapped it out (as described in #204). If we can figure out how to make the manifest plugin work then in theory no major changes should be required.

The more I think about this though I'm starting to think we need some kind of proper versioning system for the config file. An npm package as @ParamagicDev suggested could well be a good way to go.

Since the config file is essentially just a JavaScript object; I wonder if there'd be a way for us to create the baseline config in an npm package; then allow the user to add/override values in a "local" project config. Then at build time we merge the two and write out the final config file that's used for the build to disk. Could potentially get the best of both worlds in the sense that we can version the baseline config properly while still not really hiding the config from the developer.

Am I overthinking this too much?

[ayushn21]

Just tried something and it seems to work; so here's an idea:

1. We create a repo containing just the webpack config and we version it using git tags.
2. We create a bridgetown webpack or some such CLI command that can fetch this config based on a supplied version number or the latest version.
3. The above config file will be saved to disk as bridgetown.webpack.config.js or something like that.
4. In webpack.config.js we just require the above file and export it. The user is free to make additions and overrides in this file as well.

This way we don't hide the config from the user; we don't have to publish and maintain an NPM package and we can easily roll out upgrades to users.

What do you think?

[KonnorRogers]

@ayushn21 i really like this method. By modifying it a user essentially is "forking" the base config and it's up to them to maintain it properly going forward. This also allows us to ship configs without having to maintain a full fledged NPM package.

I also like the idea of not having to sift through source code to find what the configuration is actually doing.

[jaredcwhite]

I think we're on the right track here. So we basically would remove webpack.config.js.erb from the repo, relocate it to another repo (or can we just create a new folder in our main monorepo?), and then pull that in for new sites and also for a new CLI command?

[ayushn21]

or can we just create a new folder in our main monorepo?

We wouldn't be able to track the version of the config file independently in the monorepo though. Don't know if its a good idea to tie the version of the webpack config to bridgetown version?

So yeah we'd remove the webpack config to its own repo or folder in the monorepo and possibly put an automation in there along with it. The CLI would fetch and run the automation which would download config file. Advantage of having the automation is that we can handle versioning errors a bit better. For eg: v2 of the config file needs webpack v5; but the user is still on webpack v4. Using the automation; we can detect the version mismatch and ask the user if they'd like to upgrade.

One of my concerns with this approach though is it's yet another thing for the user to keep up to date in addition to bundle update and yarn update. Maybe we could add a script in that updates everything?

[jaredcwhite]

bridgetown update ?

About the config updates, I do think they should tagged alongside Bridgetown releases, unless we completely strip out all the Webpack-related code and put it in its own gem (like Webpacker). Which maybe is the ultimate answer, but I'm wary of the extra overhead in maintaining…

[ayushn21]

I do think they should tagged alongside Bridgetown releases

That would significantly simplify matters for sure so yeah probably best to go this way as a first step. So if we put the config in a folder in bridgetown-core; maybe we could avoid writing a webpack CLI took completely? It would definitely remove the need for a bridgetown update command as we're not adding a third thing to maintain.

If the file lives in bridgetown-core and is downloaded along with the gem; all we really need to do is copy it into the project directory. Maybe we could do that during the first build or something? Or is it possible to run some code after bundle install or bundle update ?

[jaredcwhite]

I had an idea… What if we make Webpack itself a "bundled configuration"? That way we can run it by default on bridgetown new (unless a flag is set to skip it), plus it can be run anytime on an existing site to update the config. If we do it the same way as we do other bundlers in the future—Snowpack, Vite, whatever—then it's a coherent story.

BTW, I wouldn't expect it to be run automatically when updating the Bridgetown gem version itself. I think the Webpack config should be updated in a manual step.

[ayushn21]

Yes! I think that would the simplest and most user-friendly way to go about it.

But at the risk of going full circle right back to the start of the discussion; I reckon we should have a bespoke CLI command for this, even if the implementation is simply triggering an automation. In my mind bridgetown webpack update is more descriptive than bridgetown configure webpack. Also I think configurations should be kept for third-party libraries/tools that are optional; webpack/snowpack etc are pretty much essential to the usage of Bridgetown.

This way when we add more bundlers we can give each of them their own command and suite of sub-commands like bridgetown snowpack install and bridgetown snowpack update.

To sum up, we implement this functionality as an automation script but create a little wrapper command to invoke it. Or am I over-engineering this?

[jaredcwhite]

I think that all makes sense. Just about everything in Rails, for example, is a "generator" and/or a Rake task, but generally it's all triggered by rails CLI commands now, so the end-user interface is nice and clean.

[jaredcwhite]

So what's the best process to get from here to there? Also #247 is waiting in the wings, which feels like a good candidate to test our approach to letting people upgrade their configs once Bridgetown v0.20 rolls out.

[ayushn21]

I reckon the first step would be to add a uses_postcss? property to site. In the future we can add a frontend_bundler property as well and these would be helpful in writing automations and producing user friendly error messages.

We can then use this property when executing the template for the webpack config. So this is how I'd implement this:

• Add uses_postcss? property.
• Create webpack CLI command that executes an automation script
• Create split webpack config files with adequate signposting in comments (one file which we control and one for user overrides)
• Write automation script to copy these files into place
• Invoke above script in bridgetown new
• Update docs and ask users to run the webpack update command in the version release announcement.

I can spend some time on this but it'll only be about 4-5 weeks from now after my current contract ends. It might be best that way so #247 can be merged and released and we can use any learnings from how people use that while building this out.

[jaredcwhite]

I think we're in good shape now to work on this whenever you're free. Your proposal above sounds good to me. ☺️

[ayushn21]

Implemented in #270