-
-
Notifications
You must be signed in to change notification settings - Fork 2.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Do not use a shell in proc_open()
if not really needed
#5766
Conversation
7f2ab0b
to
9c65505
Compare
proc_open()
if not really needed
@MasonM could you please double check whether this PR does not introduce regressions? could you run it against your existing test-suite (on non windows machines; I have a look into windows support when I get back to a win laptop)? |
Note that any advantage in performance is highly environment dependant. you might not see any difference with older PHP versions, old OS versions or any combination thereof. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@staabm Okay, I manually applied this patch to our codebase. It works, but the performance improvement is indistinguishable from noise (which is going to be the case for basically anything that isn't a 5%+ improvement).
Before:
$ uname -a
Linux 485b0600476b 6.6.16-linuxkit #1 SMP Fri Feb 16 11:54:02 UTC 2024 aarch64 GNU/Linux
$ php -v
PHP 8.2.10 (cli) (built: Sep 4 2023 08:13:17) (NTS)
Copyright (c) The PHP Group
Zend Engine v4.2.10, Copyright (c) Zend Technologies
with Zend OPcache v8.2.10, Copyright (c), by Zend Technologies
with Xdebug v3.2.1, Copyright (c) 2002-2023, by Derick Rethans
$ phpunit -c ./phpunit.xml-dist
PHPUnit 9.5.23 #StandWithUkraine
........................................................... 59 / 20179 ( 0%)
<SNIP>
. 20179 / 20179 (100%)
Time: 07:03.880, Memory: 776.00 MB
OK (20179 tests, 40817 assertions)
After:
$ phpunit -c ./phpunit.xml-dist
OK
PHPUnit 9.5.23 #StandWithUkraine
........................................................... 59 / 20179 ( 0%)
<SNIP>
. 20179 / 20179 (100%)
Time: 06:59.617, Memory: 776.00 MB
OK (20179 tests, 40817 assertions)
Thanks for testing @MasonM . Since we are just improving frmework overhead even this few ms are nice as it summs up across all projects beeing run on CI with phpunit. The forking change should improve a lot more but will take more work and time. These changes cannot make a slow test fast. 😅 Thanks again |
That PHP is too old. most gains will be with 8.3 vs older releases. again it is highly environment dependant. |
interessting find.. as soon as I add a sleep after diff --git a/src/Util/PHP/DefaultPhpProcess.php b/src/Util/PHP/DefaultPhpProcess.php
index b85e31048..df228024a 100644
--- a/src/Util/PHP/DefaultPhpProcess.php
+++ b/src/Util/PHP/DefaultPhpProcess.php
@@ -115,6 +115,7 @@ protected function runProcess(string $job, array $settings): array
if ($job) {
$this->process($pipes[0], $job);
+ sleep(1);
}
fclose($pipes[0]); |
so, there is a race condition.. output comes but not fast enough.. unfortunately at least according to the manual you can't stream_select on things returned by proc_open on windows.. :-| |
my local testing suggests on windows we are now having 25-30ms less overhead per isolated-test: overall end-to-end this means after mstaab@NB-COMPLEX-61 MINGW64 /c/dvl/Workspace/phpunit (no-shell)
$ php ./phpunit --testsuite end-to-end
PHPUnit 10.5.15-11-gfb8de2dcb by Sebastian Bergmann and contributors.
Runtime: PHP 8.2.12
Configuration: C:\dvl\Workspace\phpunit\phpunit.xml
............................................................... 63 / 468 ( 13%)
...........SSSSSSS........................S..................S. 126 / 468 ( 26%)
.............................................S................. 189 / 468 ( 40%)
.............................S................................. 252 / 468 ( 53%)
............................................................... 315 / 468 ( 67%)
.............................................................S. 378 / 468 ( 80%)
...................................S.......S................... 441 / 468 ( 94%)
...S....................... 468 / 468 (100%)
Time: 01:39.220, Memory: 8.00 MB
OK, but some tests were skipped!
Tests: 468, Assertions: 453, Skipped: 15. before mstaab@NB-COMPLEX-61 MINGW64 /c/dvl/Workspace/phpunit (10.5)
$ php ./phpunit --testsuite end-to-end
PHPUnit 10.5.15-10-g406a2b48d by Sebastian Bergmann and contributors.
Runtime: PHP 8.2.12
Configuration: C:\dvl\Workspace\phpunit\phpunit.xml
............................................................... 63 / 468 ( 13%)
...........SSSSSSS........................S..................S. 126 / 468 ( 26%)
.............................................S................. 189 / 468 ( 40%)
.............................S................................. 252 / 468 ( 53%)
............................................................... 315 / 468 ( 67%)
........................................................S....S. 378 / 468 ( 80%)
...................................S.......S................... 441 / 468 ( 94%)
...S....................... 468 / 468 (100%)
Time: 01:57.334, Memory: 8.00 MB
OK, but some tests were skipped!
Tests: 468, Assertions: 452, Skipped: 16. => ~25% faster on windows 🚀
as windows is no longer using a tempfile after this PR, we could now remove the whole tempfile logic which simplifies the implementation further. this could be done here in a separate commit or in a separate PR |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## 10.5 #5766 +/- ##
============================================
+ Coverage 90.09% 90.14% +0.04%
+ Complexity 6447 6443 -4
============================================
Files 680 680
Lines 19557 19543 -14
============================================
- Hits 17620 17617 -3
+ Misses 1937 1926 -11 ☔ View full report in Codecov by Sentry. |
fb8de2d
to
fbf391a
Compare
did some more testing. I locally applied this PRs changes to the main-branch and used it to run tests of Roave/BetterReflection I can see a 8% end-to-end improvement when running the test-suite. I also used this same patched phpunit@main and run the phpunit 11 tests several times. with this PR
before
all testing done on windows11 with
|
I would suggest removing the now obsolete stuff in a separate PR, which I can open immediately after this is merged |
5c2cc5b
to
eeddef5
Compare
Thanks! |
refs #5749 (comment)
in my testing I can confirm on PHP 8.3 the process starts 1-2ms faster.
a hello world process isolation test on my machine runs in 46-47ms, so 1-2ms faster is not too bad actually.
requires sebastianbergmann/environment#72