Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

benchmark: update iterations of assert/deepequal-typedarrays.js #51419

Conversation

lucshi
Copy link

@lucshi lucshi commented Jan 10, 2024

Fixes: #50571

Before applying this PR, the top functions are reading test JS file, instead of real logic code of "equal".
After increasing the iteration value, the test case behaved as expected to trigger "equal" code.

<style> </style>
        after PR before PR benefit
assert/deepequal-typedarrays.js method="deepEqual" strict=0 type="Int8Array": 824746.3493 243048.5679 339%
assert/deepequal-typedarrays.js method="deepEqual" strict=0 type="Int8Array": 779766.1781 233794.6413 334%
assert/deepequal-typedarrays.js method="notDeepEqual" strict=0 type="Int8Array": 1136986.291 294974.9832 385%
assert/deepequal-typedarrays.js method="notDeepEqual" strict=0 type="Int8Array": 1106649.347 295092.668 375%
assert/deepequal-typedarrays.js method="deepEqual" strict=1 type="Int8Array": 847645.5458 247216.344 343%
assert/deepequal-typedarrays.js method="deepEqual" strict=1 type="Int8Array": 799122.474 242063.4652 330%
assert/deepequal-typedarrays.js method="notDeepEqual" strict=1 type="Int8Array": 1133671.938 308798.0899 367%
assert/deepequal-typedarrays.js method="notDeepEqual" strict=1 type="Int8Array": 1081829.263 297812.4484 363%
assert/deepequal-typedarrays.js method="deepEqual" strict=0 type="Uint8Array": 826865.9462 247329.0931 334%
assert/deepequal-typedarrays.js method="deepEqual" strict=0 type="Uint8Array": 783388.5037 235249.7246 333%
assert/deepequal-typedarrays.js method="notDeepEqual" strict=0 type="Uint8Array": 1146704.619 295838.7324 388%
assert/deepequal-typedarrays.js method="notDeepEqual" strict=0 type="Uint8Array": 1107188.907 291751.2016 379%
assert/deepequal-typedarrays.js method="deepEqual" strict=1 type="Uint8Array": 835204.4976 245779.354 340%
assert/deepequal-typedarrays.js method="deepEqual" strict=1 type="Uint8Array": 792970.5005 241826.6229 328%
assert/deepequal-typedarrays.js method="notDeepEqual" strict=1 type="Uint8Array": 1100274.29 303353.1443 363%
assert/deepequal-typedarrays.js method="notDeepEqual" strict=1 type="Uint8Array": 1070294.228 306121.5743 350%
assert/deepequal-typedarrays.js method="deepEqual" strict=0 type="Float32Array": 134114.6596 78330.42153 171%
assert/deepequal-typedarrays.js method="deepEqual" strict=0 type="Float32Array": 3133.745148 3054.98716 103%
assert/deepequal-typedarrays.js method="notDeepEqual" strict=0 type="Float32Array": 779201.6493 156644.0577 497%
assert/deepequal-typedarrays.js method="notDeepEqual" strict=0 type="Float32Array": 31156.45764 25603.25755 122%
assert/deepequal-typedarrays.js method="deepEqual" strict=1 type="Float32Array": 841584.2251 248456.0938 339%
assert/deepequal-typedarrays.js method="deepEqual" strict=1 type="Float32Array": 575310.3581 215447.8709 267%
assert/deepequal-typedarrays.js method="notDeepEqual" strict=1 type="Float32Array": 1112535.008 303634.2592 366%
assert/deepequal-typedarrays.js method="notDeepEqual" strict=1 type="Float32Array": 932094.9793 282013.6224 331%
assert/deepequal-typedarrays.js method="deepEqual" strict=0 type="Uint32Array": 824229.3357 240765.2096 342%
assert/deepequal-typedarrays.js method="deepEqual" strict=0 type="Uint32Array": 558549.0558 210350.9495 266%
assert/deepequal-typedarrays.js method="notDeepEqual" strict=0 type="Uint32Array": 1114482.346 291626.9811 382%
assert/deepequal-typedarrays.js method="notDeepEqual" strict=0 type="Uint32Array": 976467.5614 287154.8184 340%
assert/deepequal-typedarrays.js method="deepEqual" strict=1 type="Uint32Array": 844915.4341 252636.5147 334%
assert/deepequal-typedarrays.js method="deepEqual" strict=1 type="Uint32Array": 579043.6774 213440.3381 271%
assert/deepequal-typedarrays.js method="notDeepEqual" strict=1 type="Uint32Array": 1113688.235 307862.6268 362%
assert/deepequal-typedarrays.js method="notDeepEqual" strict=1 type="Uint32Array": 941621.1107 288160.3555 327%

@nodejs-github-bot nodejs-github-bot added assert Issues and PRs related to the assert subsystem. benchmark Issues and PRs related to the benchmark subsystem. labels Jan 10, 2024
Copy link
Member

@BridgeAR BridgeAR left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The low number was intentional. First of: a micro benchmark should not take too long and increasing the iterations by a factor of 50 also increases the runtime accordingly.
That aside: it is also beneficial to know if a difference can be identified without the full optimizations of V8 are in place as it's more likely for real code to behave like that.

@lucshi
Copy link
Author

lucshi commented Jan 10, 2024

The low number was intentional. First of: a micro benchmark should not take too long and increasing the iterations by a factor of 50 also increases the runtime accordingly. That aside: it is also beneficial to know if a difference can be identified without the full optimizations of V8 are in place as it's more likely for real code to behave like that.

After applying this PR, the testing duration remains almost the same so there was little overhead for the testing duration and runtime.

The real reason for this PR is that the old iteration value does not trigger "equal" logic code enought. I pasted the time consumption of top funtions captured by perf-tools as below. It is clear that before this PR, the top 10 functions are around JS file reading. After increasing the iterations to a proper value (the specific value was carefully tested out) the "equal" logic was triggered.

Before PR:
image

After PR:
image

Copy link
Member

@BridgeAR BridgeAR left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is indeed a very valid point. Thank you for pointing that out! Do we have to increase it by a factor of 50x to reach that or would that also be hit by a lower number?

@lucshi
Copy link
Author

lucshi commented Jan 11, 2024

That is indeed a very valid point. Thank you for pointing that out! Do we have to increase it by a factor of 50x to reach that or would that also be hit by a lower number?

Yes, the factor was tested out case by case. I also modifed other cases and the factor varies across different cases. The method was to find out the least value to let case making sense and not have notable overhead for running duration.

@lucshi
Copy link
Author

lucshi commented Feb 26, 2024

Hi @BridgeAR, this PR is not merged yet, and I submitted many other PRs like this but all of them are hanging up with no reviewers for more than 6 weeks. I'm not sure if I need to assign a reviewer or invite you as the reviewer? Thanks!

@legendecas legendecas added the commit-queue Add this label to land a pull request using GitHub Actions. label Feb 28, 2024
@nodejs-github-bot nodejs-github-bot removed the commit-queue Add this label to land a pull request using GitHub Actions. label Feb 28, 2024
@nodejs-github-bot nodejs-github-bot merged commit e8de0f5 into nodejs:main Feb 28, 2024
27 checks passed
@nodejs-github-bot
Copy link
Collaborator

Landed in e8de0f5

@lucshi lucshi deleted the benchmark/assert/deepequal-typedarrays.js branch February 29, 2024 01:14
marco-ippolito pushed a commit that referenced this pull request Feb 29, 2024
Fixes: #50571
PR-URL: #51419
Reviewed-By: Ruben Bridgewater <[email protected]>
Reviewed-By: Chengzhong Wu <[email protected]>
marco-ippolito pushed a commit that referenced this pull request Feb 29, 2024
Fixes: #50571
PR-URL: #51419
Reviewed-By: Ruben Bridgewater <[email protected]>
Reviewed-By: Chengzhong Wu <[email protected]>
@marco-ippolito marco-ippolito mentioned this pull request Mar 1, 2024
richardlau pushed a commit that referenced this pull request Mar 25, 2024
Fixes: #50571
PR-URL: #51419
Reviewed-By: Ruben Bridgewater <[email protected]>
Reviewed-By: Chengzhong Wu <[email protected]>
richardlau pushed a commit that referenced this pull request Mar 25, 2024
Fixes: #50571
PR-URL: #51419
Reviewed-By: Ruben Bridgewater <[email protected]>
Reviewed-By: Chengzhong Wu <[email protected]>
@richardlau richardlau mentioned this pull request Mar 25, 2024
rdw-msft pushed a commit to rdw-msft/node that referenced this pull request Mar 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
assert Issues and PRs related to the assert subsystem. benchmark Issues and PRs related to the benchmark subsystem.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

The performance gap between node16 and node21 changes as the n of the benchmark changes
4 participants