Skip to content

Commit

Permalink
PHPUnit 11 | ExpectUserDeprecationtrait: polyfill the TestCase::expec…
Browse files Browse the repository at this point in the history
…tUserDeprecation*() methods

PHPUnit 11.0.0 introduces the new `TestCase::expectUserDeprecationMessage()` and `TestCase::expectUserDeprecationMessageMatches()` methods.

These methods can largely be seen as replacements for the `TestCase::expectDeprecationMessage()` and `TestCase::expectDeprecationMessageMatches()` methods which were removed in PHPUnit 10.0, though there are significant differences between the implementation details of the old vs the new methods.

As PHPUnit 10 does not have any even semi-equivalent method(s) available, the challenge was to polyfill these methods for PHPUnit 10.
And as these methods are expectation methods, not assertions, this is even more challenging as the information to evaluate a pass/fail is not available at the time of the method call, but only once the rest of the test code has run.

To do so, the following five options were considered:

**Option 1: evaluate pass/fail in an "assertPostConditions()" method**

While possible, doing so would make the use of the `Yoast\PHPUnitPolyfills\TestCases\TestCase` base class required.
It would also mean deprecating/removing the annotation-based `Yoast\PHPUnitPolyfills\TestCases\XTestCase` class as there is no annotation available to mark a method as an "assertPostConditions()"  method, so the functionality could not work with the `XTestCase` class.

As a knock-on effect of making the use of the Polyfilled `TestCase` a requirement, it would also mean that using of the individual Polyfills as stand-alone traits should be deprecated/no longer be supported.

This chain of consequences was deemed undesirable as it breaks the premise of the Polyfills being a "drop in" helper library.

**Option 2: use Events to evaluate pass/fail**

While possible, doing so would mean that the event listeners would need to be "hooked in" from the PHPUnit config file.
This would mean that every project using the Polyfills and wanting to use the `expectUserDeprecation*()` methods would need to make changes to their PHPUnit configuration file.
Additionally, adding the event listeners would make a configuration file incompatible with PHPUnit <= 9 and the event listener should not be hooked in for PHPUnit 11, in which the functionality exists in PHPUnit natively. In other words, it would mean that end-users would now need three (3) different PHPUnit configuration files, one for PHPUnit <=9, one for PHPUnit 10 and one for PHPUnit 11.

Again, this was deemed undesirable as it breaks the premise of the Polyfills being a "drop in" helper library.

**Option 3: polyfill, but don't support PHPUnit 10 in the polyfill**

This would basically mean that any test using the `expectUserDeprecation*()` polyfill would be marked as skipped on PHPUnit 10.

This is not a transparent action and the impact of this on tests was deemed too high to be acceptable.

**Option 4: don't polyfill**

It should be obvious that this is the least desirable option as this would mean the polyfills wouldn't polyfill (at least for these methods).

**Option 5: polyfill, but drop support for PHPUnit 10 completely**

This means polyfilling for PHPUnit <= 9 by dropping through to the "old" functions and not supporting installation of the Polyfills 3.x versions in combination with PHPUnit 10.

In practical terms, the net effect of this is that tests on PHP 8.1 would run on PHPUnit 9 instead of PHPUnit 10. Other than that, there is no downside.

Having considered these options carefully, option 5 was deemed most appropriate and most in line with the simple elegance of the functionality offered by the PHPUnit Polyfills until now.

Final note: when falling through to the "old" methods on PHPUnit 9.5.x, PHPUnit native deprecation notices will be shown (but won't fail a build).
Unfortunately, this is unavoidable as the `@` operator is not respected and cannot silence these deprecation notices.
Having said that, as long as the polyfilled methods are used in the actual test suite, these deprecation notices can be safely ignored anyway.

This commit implements option 5:
* Adds two traits with the same name.
    One to polyfill the methods when not available in PHPUnit.
    The other - an empty trait - to allow for `use`-ing the trait in PHPUnit versions in which the methods are already natively available.
* Adds logic to the custom autoloader which will load the correct trait depending on the PHPUnit version used.
* Adds an availability test for the functionality polyfilled.
* Adds documentation in the `README` about the differences in the functionality of the `expectUserDeprecation*()` methods in PHPUnit <= 9 versus PHPUnit 11.

Includes:
* Adding the new polyfill to the existing `TestCases` classes.

This commit also drops support for PHPUnit 10:
* Make the version drop explicit via the PHPUnit `require` settings in the `composer.json` file.
* Update the GH Actions `test` workflow to no longer run the tests against PHPUnit 10.
    This also allows for getting rid of a number of work-arounds which were previously needed for running the tests on both PHPUnit 10.0 and PHPUnit 10.1+.
* Updates the PHPUnit 10+ configuration file used by the polyfills package itself to the schema supported in PHPUnit 11.
    This includes adding some configuration options to the file, which were previously, conditionally, added in the CI run as those options were not available in PHPUnit 10.0.

Refs:
* sebastianbergmann/phpunit 5605
  • Loading branch information
jrfnl committed Sep 6, 2024
1 parent d6537e8 commit 01cfa57
Show file tree
Hide file tree
Showing 13 changed files with 199 additions and 74 deletions.
78 changes: 7 additions & 71 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,10 @@ jobs:
# it would result PHP 7.0 - 7.4 all using PHPUnit 6.4.4, which is not the intention.
# It also would run into trouble with PHP 8.5.12 being used on PHP 8.0+, while the
# 8.5.12 release still contained a bug which makes it incompatible with PHP 8.1+,
# even though it officially allows for it..
# even though it officially allows for it.
#
# Note: PHPUnit 10 is not supported for the PHPUnit Polyfills 3.x branch, so there are
# no builds against PHPUnit 10!
- php: '7.0'
phpunit: '6.4.4'
coverage: true
Expand Down Expand Up @@ -86,12 +89,6 @@ jobs:
phpunit: '9.3.0'
coverage: true
experimental: false
- php: '8.1'
# Specifically set at 10.0.12 minimum to prevent needing a toggle in the tests for something
# related to the ArrayIsList polyfill, but not necessarily relevant.
phpunit: '10.0.12'
coverage: true
experimental: false
- php: '8.2'
phpunit: '9.3.0'
coverage: true
Expand All @@ -100,10 +97,6 @@ jobs:
phpunit: '~11.1.0' # See note above about PHPUnit 11.2.
coverage: true
experimental: false
- php: '8.3'
phpunit: '10.1.0'
coverage: true
experimental: false
- php: '8.3'
phpunit: '11.0.0'
coverage: true
Expand All @@ -114,10 +107,6 @@ jobs:
phpunit: '^9.6'
coverage: false
experimental: true
- php: 'nightly'
phpunit: '^10.5'
coverage: false
experimental: true
- php: 'nightly'
phpunit: 'auto' # PHPUnit 11.x.
coverage: false
Expand All @@ -132,9 +121,6 @@ jobs:

continue-on-error: ${{ matrix.experimental }}

env:
EXTRA_PHPUNIT_CLIARGS: '--fail-on-deprecation --fail-on-notice'

steps:
- name: Checkout code
uses: actions/checkout@v4
Expand Down Expand Up @@ -178,40 +164,23 @@ jobs:
- name: "DEBUG: Show grabbed version"
run: echo ${{ steps.phpunit_version.outputs.VERSION }}

# NEEDS_MIGRATION toggle has two functions:
# 1. yes/no run PHPUnit "migrate-configuration" for PHPUnit 10.1+.
# 2. yes/no pass the "EXTRA_PHPUNIT_CLIARGS" on PHPUnit 10.1+ (which can't be added to the config as that would
# make the config incompatible with PHPUnit 10.0).
# The variable should only be set for the "yes" cases.
- name: Determine PHPUnit config
id: phpunit_config
run: |
if [ "${{ matrix.phpunit == 'dev-main' }}" == "true" ]; then
echo 'FILE=phpunit10.xml.dist' >> $GITHUB_OUTPUT
echo 'NEEDS_MIGRATION=true' >> $GITHUB_OUTPUT
elif [ "${{ steps.phpunit_version.outputs.VERSION }}" == "10.0" ]; then
elif [ "${{ startsWith( steps.phpunit_version.outputs.VERSION, '11.' ) }}" == "true" ]; then
echo 'FILE=phpunit10.xml.dist' >> $GITHUB_OUTPUT
elif [ "${{ startsWith( steps.phpunit_version.outputs.VERSION, '10.' ) || startsWith( steps.phpunit_version.outputs.VERSION, '11.' ) }}" == "true" ]; then
echo 'FILE=phpunit10.xml.dist' >> $GITHUB_OUTPUT
echo 'NEEDS_MIGRATION=true' >> $GITHUB_OUTPUT
else
echo 'FILE=phpunit.xml.dist' >> $GITHUB_OUTPUT
fi
# Migrate PHPUnit configuration to deal with changes in the coverage/source setting across PHPUnit 10.x
# versions as otherwise the warnings about these would fail the build (which to me, feels like a bug).
- name: "Migrate configuration (PHPUnit 10.1+)"
if: ${{ steps.phpunit_config.outputs.NEEDS_MIGRATION }}
continue-on-error: true
run: vendor/bin/phpunit -c ${{ steps.phpunit_config.outputs.FILE }} --migrate-configuration

- name: "Run the unit tests"
# Don't fail the build on a test run failure against a future PHPUnit version.
continue-on-error: ${{ matrix.phpunit == 'dev-main' }}
run: >
vendor/bin/phpunit -c ${{ steps.phpunit_config.outputs.FILE }}
${{ ! matrix.coverage && '--no-coverage' || '' }}
${{ steps.phpunit_config.outputs.NEEDS_MIGRATION && env.EXTRA_PHPUNIT_CLIARGS || '' }}
- name: Upload coverage results to Coveralls
if: ${{ success() && matrix.coverage == true }}
Expand Down Expand Up @@ -310,21 +279,7 @@ jobs:
phpunit: '9'
coverage: true

# PHPUnit 10 is fully supported for the officially supported PHP versions.
- php: '8.1'
phpunit: '10.0'
coverage: true
- php: '8.1'
phpunit: '10'
- php: '8.2'
phpunit: '10.0'
- php: '8.2'
phpunit: '10'
- php: '8.3'
phpunit: '10.0'
- php: '8.3'
phpunit: '10'
coverage: true
# PHPUnit 10 is NOT supported in PHPUnit Polyfills 3.x.

# PHPUnit 11 is fully supported for the officially supported PHP versions.
#
Expand Down Expand Up @@ -353,9 +308,6 @@ jobs:

continue-on-error: ${{ matrix.php == 'nightly' }}

env:
EXTRA_PHPUNIT_CLIARGS: '--fail-on-deprecation --fail-on-notice'

steps:
- name: Checkout code
uses: actions/checkout@v4
Expand Down Expand Up @@ -403,35 +355,19 @@ jobs:
- name: "DEBUG: Show grabbed version"
run: echo ${{ steps.phpunit_version.outputs.VERSION }}

# NEEDS_MIGRATION toggle has two functions:
# 1. yes/no run PHPUnit "migrate-configuration" for PHPUnit 10.1+.
# 2. yes/no pass the "EXTRA_PHPUNIT_CLIARGS" on PHPUnit 10.1+ (which can't be added to the config as that would
# make the config incompatible with PHPUnit 10.0).
# The variable should only be set for the "yes" cases.
- name: Determine PHPUnit config
id: phpunit_config
run: |
if [ "${{ steps.phpunit_version.outputs.VERSION }}" == "10.0" ]; then
echo 'FILE=phpunit10.xml.dist' >> $GITHUB_OUTPUT
elif [ "${{ startsWith( steps.phpunit_version.outputs.VERSION, '10.' ) || startsWith( steps.phpunit_version.outputs.VERSION, '11.' ) }}" == "true" ]; then
if [ "${{ startsWith( steps.phpunit_version.outputs.VERSION, '11.' ) }}" == "true" ]; then
echo 'FILE=phpunit10.xml.dist' >> $GITHUB_OUTPUT
echo 'NEEDS_MIGRATION=true' >> $GITHUB_OUTPUT
else
echo 'FILE=phpunit.xml.dist' >> $GITHUB_OUTPUT
fi
# Migrate PHPUnit configuration to deal with changes in the coverage/source setting across PHPUnit 10.x
# versions as otherwise the warnings about these would fail the build (which to me, feels like a bug).
- name: "Migrate configuration (PHPUnit 10.1+)"
if: ${{ steps.phpunit_config.outputs.NEEDS_MIGRATION }}
continue-on-error: true
run: phpunit -c ${{ steps.phpunit_config.outputs.FILE }} --migrate-configuration

- name: "Run the unit tests"
run: >
phpunit -c ${{ steps.phpunit_config.outputs.FILE }}
${{ ! matrix.coverage && '--no-coverage' || '' }}
${{ steps.phpunit_config.outputs.NEEDS_MIGRATION && env.EXTRA_PHPUNIT_CLIARGS || '' }}
- name: Upload coverage results to Coveralls
if: ${{ success() && matrix.coverage }}
Expand Down
30 changes: 30 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -443,6 +443,36 @@ Refactoring tests which still use `Assert::assertArraySubset()` to use the new a
[`Assert::assertArrayIsIdenticalToArrayOnlyConsideringListOfKeys()`]: https://docs.phpunit.de/en/main/assertions.html#assertarrayisidenticaltoarrayonlyconsideringlistofkeys
[`Assert::assertArrayIsIdenticalToArrayIgnoringListOfKeys()`]: https://docs.phpunit.de/en/main/assertions.html#assertarrayisidenticaltoarrayignoringlistofkeys

#### PHPUnit < 11.0.0: `Yoast\PHPUnitPolyfills\Polyfills\ExpectUserDeprecation`

| | |
| -------------------------------------------- | --------------------------------------------------- |
| [`TestCase::expectUserDeprecationMessage()`] | [`TestCase::expectUserDeprecationMessageMatches()`] |

These methods were introduced in PHPUnit 11.0.0.

This functionality resembles the functionality previously offered by the `TestCase::expectDeprecationMessage()` and `TestCase::expectDeprecationMessageMatches()` methods, which were removed in PHPUnit 10.0.0.

The polyfill use the old methods under the hood for PHPUnit <= 9, however, there are some pertinent differences in behaviour between the old and the new methods, which users of the polyfill should be aware of.

| PHPUnit <= 9.x | PHPUnit >= 11.0 |
| -------------- | --------------- |
| Only one deprecation can be expected per test | Multiple deprecations can be expected per test |
| The test stops running as soon as the deprecation message has been seen | The test will be executed completely, independently of the deprecation notice |
| The message passed to `expectUserDeprecationMessage()` will be compared as a substring | The message passed to `expectUserDeprecationMessage()` must be an exact match |
| Can expect both PHP native and user-land deprecation notices | Can only expect user-land deprecation notices, i.e. `E_USER_DEPRECATED`, not `E_DEPRECATED` |

Please keep these differences in mind when writing tests using the `expectUserDeprecationMessage*()` methods.

Note: on PHPUnit 9.5.x, when using the `expectUserDeprecationMessage*()` expectations, a "_Expecting E_DEPRECATED and E_USER_DEPRECATED is deprecated and will no longer be possible in PHPUnit 10._" deprecation will be shown in the test output.
As long at the actual test uses the `expectUserDeprecationMessage*()` expectations, this depreation message can be safely ignored.

> :information_source: Important: when using the `expectUserDeprecationMessage*()` expectation(s) in a test, the test should be annotated with a [`#[IgnoreDeprecations]`][ignoredeprecations-attribute] attribute.
[`TestCase::expectUserDeprecationMessage()`]: https://docs.phpunit.de/en/main/error-handling.html#expecting-deprecations-e-user-deprecated
[`TestCase::expectUserDeprecationMessageMatches()`]: https://docs.phpunit.de/en/main/error-handling.html#expecting-deprecations-e-user-deprecated
[ignoredeprecations-attribute]: https://docs.phpunit.de/en/main/attributes.html#ignoredeprecations

#### PHPUnit < 11.2.0: `Yoast\PHPUnitPolyfills\Polyfills\AssertObjectNotEquals`

Polyfills the [`Assert::assertObjectNotEquals()`] method to verify two (value) objects are **_not_** considered equal.
Expand Down
2 changes: 1 addition & 1 deletion composer.json
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@
},
"require": {
"php": ">=7.0",
"phpunit/phpunit": "^6.4.4 || ^7.0 || ^8.0 || ^9.0 || ^10.0 || ^11.0"
"phpunit/phpunit": "^6.4.4 || ^7.0 || ^8.0 || ^9.0 || ^11.0"
},
"require-dev": {
"php-parallel-lint/php-console-highlighter": "^1.0.0",
Expand Down
5 changes: 5 additions & 0 deletions phpstan.neon.dist
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,11 @@ parameters:
count: 2
path: src/Polyfills/AssertIsList.php

-
message: '`^Call to an undefined method Yoast\\PHPUnitPolyfills\\TestCases\\[X]?TestCase::expectExceptionMessageRegExp\(\)\.$`'
count: 2
path: src/Polyfills/ExpectUserDeprecation.php

# Level 5
-
# False positive, a string callback is perfectly fine, especially for static methods.
Expand Down
9 changes: 7 additions & 2 deletions phpunit10.xml.dist
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
<?xml version="1.0" encoding="UTF-8"?>
<phpunit
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/10.0/phpunit.xsd"
xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/11.0/phpunit.xsd"
backupGlobals="true"
bootstrap="./tests/bootstrap.php"
beStrictAboutOutputDuringTests="true"
Expand All @@ -14,6 +14,8 @@
displayDetailsOnTestsThatTriggerNotices="true"
displayDetailsOnTestsThatTriggerDeprecations="true"
failOnWarning="true"
failOnNotice="true"
failOnDeprecation="true"
>

<testsuites>
Expand All @@ -22,13 +24,16 @@
</testsuite>
</testsuites>

<coverage includeUncoveredFiles="true">
<source>
<include>
<directory suffix=".php">./src/</directory>
</include>
<exclude>
<file>src/Polyfills/AssertClosedResource_Empty.php</file>
</exclude>
</source>

<coverage includeUncoveredFiles="true">
<report>
<text outputFile="php://stdout" showOnlySummary="true"/>
<clover outputFile="build/logs/clover.xml"/>
Expand Down
21 changes: 21 additions & 0 deletions phpunitpolyfills-autoload.php
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,10 @@ public static function load( $className ) {
self::loadAssertArrayWithListKeys();
return true;

case 'Yoast\PHPUnitPolyfills\Polyfills\ExpectUserDeprecation':
self::loadExpectUserDeprecation();
return true;

case 'Yoast\PHPUnitPolyfills\Polyfills\AssertObjectNotEquals':
self::loadAssertObjectNotEquals();
return true;
Expand Down Expand Up @@ -339,6 +343,23 @@ public static function loadAssertArrayWithListKeys() {
require_once __DIR__ . '/src/Polyfills/AssertArrayWithListKeys_Empty.php';
}

/**
* Load the ExpectUserDeprecation polyfill or an empty trait with the same name
* if a PHPUnit version is used which already contains this functionality.
*
* @return void
*/
public static function loadExpectUserDeprecation() {
if ( \method_exists( TestCase::class, 'expectUserDeprecationMessage' ) === false ) {
// PHPUnit < 11.0.0.
require_once __DIR__ . '/src/Polyfills/ExpectUserDeprecation.php';
return;
}

// PHPUnit >= 11.0.0.
require_once __DIR__ . '/src/Polyfills/ExpectUserDeprecation_Empty.php';
}

/**
* Load the AssertObjectNotEquals polyfill or an empty trait with the same name
* if a PHPUnit version is used which already contains this functionality.
Expand Down
58 changes: 58 additions & 0 deletions src/Polyfills/ExpectUserDeprecation.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
<?php

namespace Yoast\PHPUnitPolyfills\Polyfills;

use PHPUnit\Framework\Error\Deprecated;
use PHPUnit\Framework\TestCase;

/**
* Polyfill the TestCase::expectUserDeprecationMessage() and the TestCase::expectUserDeprecationMessageMatches() methods.
*
* Introduced in PHPUnit 11.0.0.
*
* Note: PHPUnit 10 is not and will not be supported for these polyfills.
*
* @link https://github.com/sebastianbergmann/phpunit/pull/5605
*/
trait ExpectUserDeprecation {

/**
* Set expectation for the message when receiving a user defined deprecation notice.
*
* @param string $expectedUserDeprecationMessage The message to expect.
*
* @return void
*/
final protected function expectUserDeprecationMessage( string $expectedUserDeprecationMessage ) {
if ( \method_exists( TestCase::class, 'expectDeprecationMessage' ) ) {
// PHPUnit 8.4.0 - 9.x.
$this->expectDeprecation();
$this->expectDeprecationMessage( $expectedUserDeprecationMessage );
return;
}

// PHPUnit < 8.4.0.
$this->expectException( Deprecated::class );
$this->expectExceptionMessage( $expectedUserDeprecationMessage );
}

/**
* Set expectation for the message when receiving a user defined deprecation notice (regex based).
*
* @param string $expectedUserDeprecationMessageRegularExpression A regular expression which must match the message.
*
* @return void
*/
final protected function expectUserDeprecationMessageMatches( string $expectedUserDeprecationMessageRegularExpression ) {
if ( \method_exists( TestCase::class, 'expectDeprecationMessageMatches' ) ) {
// PHPUnit 8.4.0 - 9.x.
$this->expectDeprecation();
$this->expectDeprecationMessageMatches( $expectedUserDeprecationMessageRegularExpression );
return;
}

// PHPUnit < 8.4.0.
$this->expectException( Deprecated::class );
$this->expectExceptionMessageRegExp( $expectedUserDeprecationMessageRegularExpression );
}
}
8 changes: 8 additions & 0 deletions src/Polyfills/ExpectUserDeprecation_Empty.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
<?php

namespace Yoast\PHPUnitPolyfills\Polyfills;

/**
* Empty trait for use with PHPUnit >= 11.0.0 in which this polyfill is not needed.
*/
trait ExpectUserDeprecation {}
2 changes: 2 additions & 0 deletions src/TestCases/TestCasePHPUnitGte8.php
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
use Yoast\PHPUnitPolyfills\Polyfills\AssertObjectProperty;
use Yoast\PHPUnitPolyfills\Polyfills\EqualToSpecializations;
use Yoast\PHPUnitPolyfills\Polyfills\ExpectExceptionMessageMatches;
use Yoast\PHPUnitPolyfills\Polyfills\ExpectUserDeprecation;

/**
* Basic test case for use with PHPUnit >= 8.
Expand All @@ -37,6 +38,7 @@ abstract class TestCase extends PHPUnit_TestCase {
use AssertObjectProperty;
use EqualToSpecializations;
use ExpectExceptionMessageMatches;
use ExpectUserDeprecation;

/**
* This method is called before the first test of this test class is run.
Expand Down
2 changes: 2 additions & 0 deletions src/TestCases/TestCasePHPUnitLte7.php
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
use Yoast\PHPUnitPolyfills\Polyfills\AssertStringContains;
use Yoast\PHPUnitPolyfills\Polyfills\EqualToSpecializations;
use Yoast\PHPUnitPolyfills\Polyfills\ExpectExceptionMessageMatches;
use Yoast\PHPUnitPolyfills\Polyfills\ExpectUserDeprecation;

/**
* Basic test case for use with PHPUnit <= 7.
Expand All @@ -43,6 +44,7 @@ abstract class TestCase extends PHPUnit_TestCase {
use AssertStringContains;
use EqualToSpecializations;
use ExpectExceptionMessageMatches;
use ExpectUserDeprecation;

/**
* This method is called before the first test of this test class is run.
Expand Down
Loading

0 comments on commit 01cfa57

Please sign in to comment.