Explore unbundling of Performance Lab as canonical performance-oriented plugins

[OllieJones]

I've been thinking about Matt's pushback on webp and SQLite. I think he's right (not that it matters much what I think). Here's an approach that might work.

Problems to solve

1. Matt objects to Performance Lab's approach of bundling various proof-of-concept experiments in a single plugin. He has stated that he prefers "canonical plugins" for such things as webp support and SQLite database support.
2. It's not clear what "canonical plugin" means. The Classic Editor plugin might be an example of that, but it's not called that.
3. Techniques for performance improvement exist that don't apply uniformly to all WordPress installs. Therefore, there may be ways to add value to the overall ecosystem without insisting that all @core-performance work be destined for core inclusion.
4. Hosting providers' site reliability teams likely want features and hooks in performance-oriented code.
5. There's a lot of great work in Performance Lab, which should not be lost.

Observations

Matt seems to favor limited-scope plugins that do one thing well, rather than kitchen-sink plugins that do a lot of things. That is a viable approach. But there's one hazard: each plugin has overhead, so there's a performance cost to having multiple plugins.

The present Performance Lab plugin has a lot of stuff in it. Much of it adds sophisticated Site Health checks, but it also has a variety of operational features.

Suggested course of action

1. Define "canonical performance plugin" and socialize that definition -- get at least some buy-in from the Plugins team, other teams and Matt.
2. Work out, test, and publish guidelines for plugins themselves, to reduce the many-plugins performance penalties.
3. Adapt the Module Proposal process to serve as a Canonical Performance Plugin Proposal process.
4. Create a canonical plugin for webp handling
5. Create a canonical plugin for @aristath's SQLite compatibility work.
6. Refocus Performance Lab on Site Health.

Benefits, issues

• Benefit: These Canonical Performance Plugins can be completed and published, saving regression-test effort.
• Benefit: Hosting providers may see benefits from sponsoring developers to work on these plugins.
• Issue: Is it worthwhile to broaden the scope from "Canonical Performance Plugins" to "Canonical Plugins"?

[javiercasares]

About the "Canonical plugins", yes, there is no a clear definition about that, but there are mainly "community plugins" created by "a WordPress Team" and "maintained by some members". Also, as I understood from the SotW, maybe the Canonical Plugins may be in the HackerOne program.

Some "canonical plugins" (as said, not officially said, but from general knowledge):

• Classic Editor: https://wordpress.org/plugins/classic-editor/
• Classic Widgets: https://wordpress.org/plugins/classic-widgets/
• Gutenberg: https://wordpress.org/plugins/gutenberg/
• Health Check & Troubleshooting: https://wordpress.org/plugins/health-check/
• BuddyPress: https://wordpress.org/plugins/buddypress/
• bbPress: https://wordpress.org/plugins/bbpress/

For example, we started a plugin from the Hosting team, and we are working to "convert it" into a Canonical one... uncharted waters there :D https://wordpress.org/plugins/wpsustainable/

About the "Benefit: Hosting providers may see benefits from sponsoring developers to work on these plugins", from the Hosting team we are going to start a thing about this, looking for people to invest in some projects / plugins / whatever. We will start in the next weeks, so @WordPress/hosting-team can help with that.

[felixarntz]

@OllieJones Thank you for opening this issue to discuss the options further. I agree we need to define an approach in which the individual Performance Lab modules can become canonical plugins. However I think to start that conversation, we should take a step back and look at our potential options.

I have some concerns with your suggested course of action, particularly because it doesn't treat the different modules equally. Why should WebP Uploads and SQLite become canonical plugins, while other things remain in the Performance Lab plugin? I would also push back against refocusing the Performance Lab plugin on Site Health, since one could argue the different Site Health modules themselves could become individual plugins, or one plugin with all of them - that doesn't need to be the Performance Lab plugin though.

@ThierryA already briefly listed some potential options here: https://make.wordpress.org/core/2022/12/20/help-us-test-the-sqlite-implementation/#comment-44231

I'd like to expand on those potential approaches a bit:

1. Keep Performance Lab plugin as is, but additionally deploy modules as individual plugins
   • This is effectively copying the approach Jetpack has recently started to take.
   • The Performance Lab plugin will almost remain as is. However, every individual module will additionally be deployed as standalone plugins.
   • Users that prefer to use the overarching performance plugin can (continue to) do so, while this approach also allows us to recommend the features as canonical plugins in a more standalone fashion.
   • Likely the only change the Performance Lab plugin itself would require is checks to avoid loading a module that is already present on the site as the standalone plugin.
2. Make Performance Lab plugin a wrapper focused on central infrastructure and recommendation of individual plugins
   • This approach would keep the Performance Lab plugin around, but pull all modules out of it and deploy those exclusively as individual plugins.
   • The Performance Lab plugin would then be somewhat of a wrapper that makes it easy to navigate performance features and provide central measurement infrastructure.
   • Its module screen would change to become a screen in which the individual performance feature plugins would be surfaced, including UI to install, activate, and deactivate them - almost as a “featured performance plugins” area in WordPress.
3. Deprecate Performance Lab plugin completely in favor of individual plugins
   • In this approach the Performance Lab plugin would disappear entirely. The modules would become individual plugins.
   • There would no longer be a central way to manage or measure WordPress performance features.

Regardless of which of the above 3 approaches we'd take, all of them would come with the following:

• Canonical plugins would exist for the features.
• We could still develop these in a mono repo (this repo here).

My personal preference at this point is approach 1, for the following reasons:

• It means the Performance Lab plugin could still be used in the same way it is today, no need for any migration routine.
• End users would be able to install a single plugin for a feature if they want that feature, or if they are more generally interested in the features that the WP Performance Team is working on they could install the Performance Lab plugin.
  • In that model, any individual plugin would take precedence (e.g. if you have a hypothetical plugin WebP Uploads active but you're also using Performance Lab, then the plugin implementation would be loaded and the PL's module implementation would not be loaded).
• We would keep the central management and measurement features that the Performance Lab plugin provides. A nice side effect is that those become then somewhat of an opt-in, since you would only get those if you use the Performance Lab plugin. If you use just one (or more) of the individual plugins, you would simply get those features, but nothing of the central infrastructure.
• Effectively this approach would mirror what Jetpack does, i.e. an already established approach from a big plugin with millions of installs.

Another consideration is how we scope the individual plugins: Does each module become its own plugin, or would we somehow group certain modules / features that make sense together?

Let's discuss further, curious to hear other people's thoughts. As @bethanylang mentioned today, we'll thoroughly talk about it in next week's performance chat as well.

[javiercasares]

I'm with @felixarntz that 1 is the best option. Not everything may be a “canonical plugin”, and probably everything should be launched in a first version in the Performance Lab, then graduate it into their plugin. Checking all the functionality, most likely only WebP and SQLite may need a Canonical plugin “once left the beta period”.

WebP I think is in that case, when there is a “more or less” stable version, so it can be on its own, and left the Performance Lab. SQLite maybe is not yet at this moment. When there is some feedback, some checks and tests of this first version, probably will be that moment when “it can be used in production” although is an experimental case. Like the WebP was… many changes, but now there is a stable version.

For example, the project for the “domain color” doesn't need its plugin… it's something to test, and it's an improvement, but not a “big feature”.

[lkraav]

I'd like to note, it's possible to grossly underestimate the amount of forever-chores required to maintain individual repos, esp. by people that aren't projected to take the actual responsibility later.

[felixarntz]

@javiercasares I'm more thinking along the lines to have every current module of the PL plugin also be available as a plugin, since otherwise we would still not fully embrace the approach of canonical plugins. The idea about having some features start as a module though when they're not yet "stable enough" sounds great to me though. For example, we could say that we only publish a module as a plugin once it's no longer flagged as experimental.

@lkraav Are you referring to the WordPress.org plugin repositories? There's definitely some maintenance overhead there indeed. Just want to clarify that even with the canonical plugins approaches we would still develop all of them in this single WordPress/performance GitHub repo - only how they're deployed to WordPress.org would change.

[mxbclang]

After discussion in our January 10 performance chat, we're now ready to vote on an approach to unbundling the Performance Lab plugin. More detailed information on each option is available in this comment.

Please vote for an approach on this comment using the emojis noted below. Voting will close on Friday, January 20, 2023.

• 👍 for Option 1: Keep Performance Lab plugin as is, but additionally deploy modules as individual plugins
• 🎉 for Option 2: Make Performance Lab plugin a wrapper focused on central infrastructure and recommendation of individual plugins
• ❤️ for Option 3: Deprecate Performance Lab plugin completely in favor of individual plugins

[eclarke1]

Just to confirm the vote here is now closed.

[joemcgill]

Sorry I missed the voting period, but wanted to throw in some thoughts from what I've observed. My concern with option 1 is that it introduces some confusion and possible development overhead to have modules published as part of a bundled plugin as well as individual plugins.

Option 2, as written, doesn't fully account for the advantages @javiercasares mentioned in this comment, that there could be value in using the PL plugin for smaller experiments that would not make as much sense as standalone plugins. If we determine that there really isn't any value in keeping the plugin for that purpose (things like performance health checks, suggestions, smaller API experiments that would get merged into a future version of core, etc.) then option 3 seems like the inevitable best option.

[felixarntz]

Summarizing here the outcome of our conversation in the #core-performance team chat earlier today (also see https://make.wordpress.org/core/2023/01/31/performance-team-meeting-summary-31-january-2023/):

1. We are, for now, going to work towards option 1. This does not mean that option 1 is the decision made here, but rather that option 1 is the option that is the least disruptive and takes the least effort. Additionally, options 2 or 3 would both require the steps to accomplish option 1 as well, so no matter which option we end up going with, the steps to achieve option 1 are needed and therefore will not be a waste of time.
2. We still need to finalize on the policy for how to publish modules from the PL plugin as standalone plugins. It appears based on the discussion that we are leaning towards having each module map to a standalone plugin. Every module can be a standalone plugin, but we would decide on a case-by-case basis whether and when to publish a module as standalone plugin, based on factors like e.g. stability, whether it's an experiment etc. There would not be bundles of multiple modules in one plugin, as then, once again the features would not be available as standalone plugins, so that would miss the point.

We can already start doing some of the work needed to accomplish the above (work that would be decoupled from the pending decision in the 2nd point above), while continuing the conversation on point 2 here.

[ThierryA]

we would decide on a case-by-case basis whether and when to publish a module as standalone plugin, based on factors like e.g. stability, whether it's an experiment etc. There would not be bundles of multiple modules in one plugin, as then, once again the features would not be available as standalone plugins, so that would miss the point.

Just adding my two cents. If we have a systematic and automated approach to deploying modules as plugins, I don’t think it does any arm to do so for every module even very small. To me consistency outweighs the rest here, developers/users should know that they can find stuff in the PL plugin or individual plugins without having to find out what the Performance Team decided for each module. Also noting that should options 3 be final, it will have to be the case anyway.

[felixarntz]

@ThierryA I personally agree. I think publishing every module as a standalone plugin benefits several things:

• Addressing the original request: For any modules that we would not publish as a standalone plugin, we would effectively be ignoring the request for those individual features to be available as standalone plugins.
• Consistency: A single policy for all modules, expectations are clear.
• Workflow: No overhead in having to decide and discuss for every single module whether or when to publish it as standalone.

[joemcgill]

@felix and @ThierryA – If each module is published as a standalone plugin, would it be valuable to bundle things like the audits into a single module/plugin or split them out from the "modules" into a separate folder that is functionality that isn't intended to be shipped separate from this plugin?

[felixarntz]

@joemcgill Are you referring to the Site Health audits? I would see those as WordPress core features just like any other, so I would say those should also be in individual plugins. This is an existing practice, see for example https://github.com/audrasjb/site-health-audit-enqueued-assets.

My take is that, no matter how niche the feature is, it should still be an individual plugin per feature. Otherwise we end up bundling things again, which would to a degree miss the point of the original request to have individual features in their own plugins.

[joemcgill]

Are you referring to the Site Health audits?

Yes, those are what I'm referring to. I agree that each audit could be considered an individual feature, but there's a benefit of thinking of them as a suite of performance audits rather than individual features. Part of my thinking here is that these types of tools would be valuable to stay in a singular performance lab plugin, even if we remove the other modules in the future, essentially making this the canonical plugin for those audits.

[felixarntz]

@joemcgill I like that idea. It would still allow us to have a clear policy on which modules are published as standalone plugins (any module that's not a Site Health check), and the idea of using this plugin as a suite of performance Site Health checks intended for future core merge makes sense to me. Curious what others think.

[felixarntz]

To summarize today's performance chat discussion, there are basically 3 alternative options now:

1. Decide for every module individually whether/when to publish it as standalone plugin
2. Publish every module as standalone plugin as soon as it is ready
3. Publish every module except Site Health modules as standalone plugin as soon as it is ready

It looks like we are leaning towards option 3: It sets a clear policy that means we don't have to discuss every module individually, which is good for consistency and for not spending valuable time on such discussions.

Let's pick this up again next week.

[joemcgill]

I'd propose modifying option 3. slightly, to set policy based on the intent of a feature, rather than a specific class of functionality. As follows:

3. Publish every module as a standalone plugin that implements a feature with the intent to affect the performance of WordPress.

This means that any module created with the intent to observe, measure, or provide admin feedback about performance (e.g., Site Health checks) would remain part of the infrastructure of the Performance Lab plugin unless/until those features are included in WordPress Core.

[josephahaden]

Hi team. I know that the overwhelming preference for the team is to move forward with Option 1. After some discussions with Matt, he would much prefer that we do Option 2.

How can I help move toward that option instead?

[ThierryA]

Thanks for updating this thread with Matt's and your preference of the options discussed in this comment, @josephahaden ! I am personally happy with option 2 and think it is a good outcome from everyone, I trust/hope most folks will feel the same.

Re moving forward, I suggest adding this topic to the next core performance chat on Feb 21st (cc @eclarke1) to define the next steps to execute on option 2. This change will indeed require some work such as updating the PL plugin, spinning up individual plugins (already WIP) etc.

[felixarntz]

@josephahaden Thank you for the update. I'm personally on board with that as well, as it is a reasonable compromise.

Option 2 involves slightly more work than option 1, but the way I think about it, it's probably best to go an iterative approach with two milestones:

1. Focus on having the modules available as standalone plugins
2. Remove the modules from the plugin

I believe the 1. milestone here is the most urgent one as that was the original request and goal, so by breaking the overall work into these two milestones we can get to that goal more quickly. We would then immediately proceed working on milestone 2., just conscious that it may take a few weeks longer as it will involve some broader UI changes and a migration paths for all the existing users of the Performance Lab plugin to using the standalone plugins.

[felixarntz]

I have created additional issues for the work as mentioned above, as well as an overview issue #656 to make it easy to oversee, review, and discuss the overall progress and sequence of work.

[felixarntz]

Given that the path forward is clear now, I'm going to close this issue as completed. #656 can be used to keep track of the overarching work and for any broader follow up conversations.