Performance tests are failing against v6.3

[WunderBart]

Description

The performance job for merge commits (base branch comparison) has been failing since #54072. In consequence, no data to codevitals is being currently sent.

The following error is thrown:

✖ Error while running docker-compose command.
Creating 734421528201ba6bf4a980a8270e890d_cli_run ... 
Creating 734421528201ba6bf4a980a8270e890d_cli_run ... done
[31-Aug-2023 21:44:25 UTC] PHP Fatal error:  Cannot declare class WP_Classic_To_Block_Menu_Converter, because the name is already in use in /var/www/html/wp-content/plugins/plugin/lib/experimental/class-wp-classic-to-block-menu-converter.php on line 14
Fatal error: Cannot declare class WP_Classic_To_Block_Menu_Converter, because the name is already in use in /var/www/html/wp-content/plugins/plugin/lib/experimental/class-wp-classic-to-block-menu-converter.php on line 14
Error: Command failed: /tmp/1b7f5823-4836-4dfe-a9b1-961583e3bc1d/tests/node_modules/.bin/wp-env start
✖ Error while running docker-compose command.
Creating 734421528201ba6bf4a980a8270e890d_cli_run ... 
Creating 734421528201ba6bf4a980a8270e890d_cli_run ... done
[31-Aug-2023 21:44:25 UTC] PHP Fatal error:  Cannot declare class WP_Classic_To_Block_Menu_Converter, because the name is already in use in /var/www/html/wp-content/plugins/plugin/lib/experimental/class-wp-classic-to-block-menu-converter.php on line 14
Fatal error: Cannot declare class WP_Classic_To_Block_Menu_Converter, because the name is already in use in /var/www/html/wp-content/plugins/plugin/lib/experimental/class-wp-classic-to-block-menu-converter.php on line 14
255

/cc @youknowriad @dmsnell @kevin940726 @Mamaduka

[youknowriad]

I'm guessing that it's the commit that we use as the "base comparison" commit that is incompatible with WP 6.3? If that's the case, It seems the fix would be to change the base commit to one that is compatible (checks existence of the class before declaring it)

[Mamaduka]

Here's the PR - #54111.

We need to document this process as we end up in a similar situation after each release. See #50360 (comment).

cc @bph, @ndiego

[ndiego]

@Mamaduka, where should this info be added? Somewhere in here?

[Mamaduka]

@ndiego, there probably should be a "Post-WP release" section in the docs. Documenting:

• Removal of compat files for unsupported version. Example: Plugin: require minimum WordPress 6.2 #53451.
• Bumping minimum required WP version and changing the SHA: Example: Perf Tests: Update base point to compare #54111.
• Bumping the "Tested up to" version for a plugin.

These can be split into multiple PRs or be a single one. I personally think the last two should be combined.