querystring: improve parse() performance #10874

mscdex · 2017-01-18T18:12:19Z

Here are the results with existing benchmarks (and newly added input/type) for the changes in this PR:

                                                                  improvement confidence      p.value
 querystring/querystring-parse.js n=5000000 type="encodefake"          7.60 %        *** 5.588287e-48
 querystring/querystring-parse.js n=5000000 type="encodelast"         23.85 %        *** 3.772582e-43
 querystring/querystring-parse.js n=5000000 type="encodemany"         13.75 %        *** 7.027748e-27
 querystring/querystring-parse.js n=5000000 type="manypairs"           7.09 %        *** 2.271675e-49
 querystring/querystring-parse.js n=5000000 type="multicharsep"       19.71 %        *** 6.757215e-33
 querystring/querystring-parse.js n=5000000 type="multivalue"         37.01 %        *** 4.461087e-58
 querystring/querystring-parse.js n=5000000 type="multivaluemany"     33.38 %        *** 4.410104e-59
 querystring/querystring-parse.js n=5000000 type="noencode"           27.66 %        *** 8.234363e-41

CI: https://ci.nodejs.org/job/node-test-pull-request/5932/

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
commit message follows commit guidelines

Affected core subsystem(s)

querystring

targos · 2017-01-18T19:10:52Z

Have you tried to replace the uses of array[array.length] = value with array.push(value)?

mscdex · 2017-01-18T19:12:38Z

Have you tried to replace the uses of array[array.length] = value with array.push(value)?

I've looked into it before. The former is still slightly faster in master.

jasnell · 2017-01-19T16:07:02Z

lib/querystring.js

  if (typeof v === 'string')
    return v;
  if (typeof v === 'number' && isFinite(v))
    return '' + v;
  if (typeof v === 'boolean')
    return v ? 'true' : 'false';
  return '';
-};
+}


is there any reasonable perf impact moving to a switch?
e.g.

switch (typeof v) { case 'string': return v; case 'boolean': return String(v); case 'number': if (isFinite(v)) return String(v); default: return ''; }

I dunno, I didn't do anything with this particular function. However, I remember we switched away from that type of usage (no pun intended) because V8 optimizes typeof xx === yyy statements. Before we switched to separate if statements, the code was storing the result of typeof and that was causing a performance degredation. I suspect switch(typeof v) might have the same effect.

fair enough :-)

mscdex · 2017-01-19T16:27:20Z

I just noticed that this time last year I was making performance improvements to the same querystring functions. Totally coincidental!

PR-URL: nodejs#10874 Reviewed-By: James M Snell <[email protected]>

PR-URL: #10874 Reviewed-By: James M Snell <[email protected]>

PR-URL: nodejs#10874 Reviewed-By: James M Snell <[email protected]>

caub · 2017-02-03T12:36:51Z

Just a question, how does it compare with the new searchParams in performance?

jasnell · 2017-03-08T22:33:54Z

This would need a backport PR to land in v4. Landed in v6

PR-URL: #10874 Reviewed-By: James M Snell <[email protected]>

Notable changes * performance: The performance of several APIs has been improved. - `Buffer.compare()` is up to 35% faster on average. (Brian White) #10927 - `buffer.toJSON()` is up to 2859% faster on average. (Brian White) #10895 - `fs.*statSync()` functions are now up to 9.3% faster on average. (Brian White) #11522 - `os.loadavg` is up to 151% faster. (Brian White) #11516 - `process.memoryUsage()` is up to 34% faster. (Brian White) #11497 - `querystring.unescape()` for `Buffer`s is 15% faster on average. (Brian White) #10837 - `querystring.stringify()` is up to 7.8% faster on average. (Brian White) #10852 - `querystring.parse()` is up to 21% faster on average. (Brian White) #10874 * IPC: - Batched writes have been enabled for process IPC on platforms that support Unix Domain Sockets. (Alexey Orlenko) #10677 - Performance gains may be up to 40% for some workloads. * child_process: - `spawnSync` now returns a null `status` when child is terminated by a signal. (cjihrig) #11288 - This fixes the behavior to act like `spawn()` does. * http: - Control characters are now always rejected when using `http.request()`. (Ben Noordhuis) #8923 - Debug messages have been added for cases when headers contain invalid values. (Evan Lucas) #9195 * node: - Heap statistics now support values larger than 4GB. (Ben Noordhuis) #10186 * timers: - Timer callbacks now always maintain order when interacting with domain error handling. (John Barboza) #10522 PR-URL: #11759

Notable changes * performance: The performance of several APIs has been improved. - `Buffer.compare()` is up to 35% faster on average. (Brian White) nodejs/node#10927 - `buffer.toJSON()` is up to 2859% faster on average. (Brian White) nodejs/node#10895 - `fs.*statSync()` functions are now up to 9.3% faster on average. (Brian White) nodejs/node#11522 - `os.loadavg` is up to 151% faster. (Brian White) nodejs/node#11516 - `process.memoryUsage()` is up to 34% faster. (Brian White) nodejs/node#11497 - `querystring.unescape()` for `Buffer`s is 15% faster on average. (Brian White) nodejs/node#10837 - `querystring.stringify()` is up to 7.8% faster on average. (Brian White) nodejs/node#10852 - `querystring.parse()` is up to 21% faster on average. (Brian White) nodejs/node#10874 * IPC: - Batched writes have been enabled for process IPC on platforms that support Unix Domain Sockets. (Alexey Orlenko) nodejs/node#10677 - Performance gains may be up to 40% for some workloads. * child_process: - `spawnSync` now returns a null `status` when child is terminated by a signal. (cjihrig) nodejs/node#11288 - This fixes the behavior to act like `spawn()` does. * http: - Control characters are now always rejected when using `http.request()`. (Ben Noordhuis) nodejs/node#8923 - Debug messages have been added for cases when headers contain invalid values. (Evan Lucas) nodejs/node#9195 * node: - Heap statistics now support values larger than 4GB. (Ben Noordhuis) nodejs/node#10186 * timers: - Timer callbacks now always maintain order when interacting with domain error handling. (John Barboza) nodejs/node#10522 PR-URL: nodejs/node#11759 Signed-off-by: Ilkka Myller <[email protected]>

mscdex added performance Issues and PRs related to the performance of Node.js. querystring Issues and PRs related to the built-in querystring module. labels Jan 18, 2017

nodejs-github-bot added the lts-watch-v6.x label Jan 18, 2017

mscdex force-pushed the querystring-parse-perf branch from 2f8834c to db07d61 Compare January 18, 2017 19:15

nodejs-github-bot added the lts-watch-v6.x label Jan 18, 2017

mscdex force-pushed the querystring-parse-perf branch from db07d61 to c174286 Compare January 18, 2017 20:26

nodejs-github-bot added the lts-watch-v6.x label Jan 18, 2017

mscdex force-pushed the querystring-parse-perf branch from c174286 to 0b9d55d Compare January 18, 2017 20:27

nodejs-github-bot added the lts-watch-v6.x label Jan 18, 2017

jasnell reviewed Jan 19, 2017

View reviewed changes

jasnell approved these changes Jan 19, 2017

View reviewed changes

mscdex removed the lts-watch-v6.x label Jan 24, 2017

querystring: improve parse() performance

2298bc4

PR-URL: nodejs#10874 Reviewed-By: James M Snell <[email protected]>

mscdex force-pushed the querystring-parse-perf branch from 0b9d55d to 2298bc4 Compare January 25, 2017 07:06

nodejs-github-bot added the lts-watch-v6.x label Jan 25, 2017

mscdex removed the lts-watch-v6.x label Jan 25, 2017

mscdex merged commit 2298bc4 into nodejs:master Jan 25, 2017

mscdex deleted the querystring-parse-perf branch January 25, 2017 07:10

targos pushed a commit that referenced this pull request Jan 28, 2017

querystring: improve parse() performance

2d85fbc

PR-URL: #10874 Reviewed-By: James M Snell <[email protected]>

italoacasas mentioned this pull request Jan 29, 2017

v7.5.0 proposal #11062

Merged

italoacasas pushed a commit to italoacasas/node that referenced this pull request Jan 30, 2017

querystring: improve parse() performance

53421b1

PR-URL: nodejs#10874 Reviewed-By: James M Snell <[email protected]>

italoacasas pushed a commit to italoacasas/node that referenced this pull request Jan 30, 2017

querystring: improve parse() performance

6130bae

PR-URL: nodejs#10874 Reviewed-By: James M Snell <[email protected]>

joyeecheung mentioned this pull request Feb 4, 2017

benchmark: URLSearchParams v.s. querystring #11170

Closed

3 tasks

jasnell added the dont-land-on-v4.x label Mar 8, 2017

jasnell pushed a commit that referenced this pull request Mar 8, 2017

querystring: improve parse() performance

d954adf

PR-URL: #10874 Reviewed-By: James M Snell <[email protected]>

MylesBorins pushed a commit that referenced this pull request Mar 9, 2017

querystring: improve parse() performance

23f3f20

PR-URL: #10874 Reviewed-By: James M Snell <[email protected]>

MylesBorins mentioned this pull request Mar 9, 2017

v6.10.1 proposal #11759

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

querystring: improve parse() performance #10874

querystring: improve parse() performance #10874

mscdex commented Jan 18, 2017 •

edited

Loading

targos commented Jan 18, 2017

mscdex commented Jan 18, 2017 •

edited

Loading

jasnell Jan 19, 2017

mscdex Jan 19, 2017

jasnell Jan 19, 2017

mscdex commented Jan 19, 2017

caub commented Feb 3, 2017

jasnell commented Mar 8, 2017

querystring: improve parse() performance #10874

querystring: improve parse() performance #10874

Conversation

mscdex commented Jan 18, 2017 • edited Loading

Checklist

Affected core subsystem(s)

targos commented Jan 18, 2017

mscdex commented Jan 18, 2017 • edited Loading

jasnell Jan 19, 2017

Choose a reason for hiding this comment

mscdex Jan 19, 2017

Choose a reason for hiding this comment

jasnell Jan 19, 2017

Choose a reason for hiding this comment

mscdex commented Jan 19, 2017

caub commented Feb 3, 2017

jasnell commented Mar 8, 2017

mscdex commented Jan 18, 2017 •

edited

Loading

mscdex commented Jan 18, 2017 •

edited

Loading