lib: refactor internal/util #11404

jasnell · 2017-02-15T21:22:41Z

Use the more efficient module.exports = {} approach
Eliminate some uses of arguments

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
commit message follows commit guidelines

Affected core subsystem(s)

internal/util

mscdex · 2017-02-15T22:35:51Z

One thing to keep in mind when inlining functions inside module.exports = { ... } is that the extra indentation will increase the function body length, which will cause issues with runtime inlineability if the new function's length is >= 600 characters.

jasnell · 2017-02-15T22:45:43Z

Yeah i was thinking about that. Just in general it would be good to move the function bodies out of the object.

sam-github · 2017-02-16T15:50:14Z

I'm not sure the refactor makes it better, just stylistically different. What is more efficient? There are benchmarks for this?

sam-github · 2017-02-16T16:00:14Z

modules that do exports.whatever = ... through-out are common style in js, easy to understand, familiar to any js develeoper, and used widely in node.js lib/, rewriting will make backporting across this barrier pretty hard, and its just moving from one of the two most popular js module styles to the other.

I don't think this kind of sweeping stylistic change is worth doing unless its really compellingly better style or has some compelling functional difference and I'm not sure the "more efficient" style will be noticeable.

If we didn't have to backport, I'd have no particular opinion, but sweeping stylistic changes that make backports hard worries me.

In contrast, breaking up the absolutely enormous lib/crypto.js is, I think, compellingly better style, easier to read, maintain, etc., so despite the backport cost, I was +1 for that. Here, its not so clear to me.

vsemozhetbyt · 2017-02-16T20:49:59Z

@sam-github FWIW: #11430

jasnell · 2017-02-17T21:11:17Z

@mscdex @sam-github ... so this one we will definitely want to investigate further and hold off on landing. Unlike the other similar changes I've done moving to the module.exports = {} pattern (which have shown about a 5% performance improvement), the change in this module shows a significant perf regression when running in a benchmark test.

jasnell · 2017-02-17T21:11:39Z

I've added the in progress label to ensure this doesn't get landed before we complete the investigation

fhalde

Not sure what is the general practice followed in the node code base. But looks like you've been missing out on semicolons after end of the function block. Looking at the history of the file, function blocks were ended with semicolons.

jasnell · 2017-02-21T16:53:22Z

semicolons are not required to closing function blocks for regular functions with our linting rules. They are only required on assignments.

* Use the more efficient module.exports = {} approach * Eliminate some uses of arguments

jasnell · 2017-04-20T18:21:23Z

@mscdex @sam-github ... I've updated this PR. The perf diff with master compared to 7.9 is negligible.

The following is the benchmark comparison results for the normalize encoding method in internal/util:

Master to 7.9

                                                            improvement confidence      p.value
 util/normalize-encoding.js n=100000 input=""                   -4.45 %            4.711075e-01
 util/normalize-encoding.js n=100000 input="[]"                 36.27 %        *** 2.548943e-08
 util/normalize-encoding.js n=100000 input="1"                  -5.27 %            3.436246e-01
 util/normalize-encoding.js n=100000 input="base64"              3.10 %            6.309801e-01
 util/normalize-encoding.js n=100000 input="BASE64"             -7.69 %            1.484294e-01
 util/normalize-encoding.js n=100000 input="binary"              3.06 %            6.140675e-01
 util/normalize-encoding.js n=100000 input="BINARY"             -5.41 %            3.480641e-01
 util/normalize-encoding.js n=100000 input="false"               2.86 %            7.025918e-01
 util/normalize-encoding.js n=100000 input="foo"                 2.63 %            6.799095e-01
 util/normalize-encoding.js n=100000 input="group_common"      -15.97 %         ** 2.509445e-03
 util/normalize-encoding.js n=100000 input="group_misc"         -5.11 %            2.732518e-01
 util/normalize-encoding.js n=100000 input="group_uncommon"    -16.40 %         ** 2.064198e-03
 util/normalize-encoding.js n=100000 input="group_upper"        -7.97 %          * 4.742396e-02
 util/normalize-encoding.js n=100000 input="hex"                 2.49 %            6.976260e-01
 util/normalize-encoding.js n=100000 input="HEX"                -0.64 %            9.160260e-01
 util/normalize-encoding.js n=100000 input="latin1"             -0.11 %            9.873016e-01
 util/normalize-encoding.js n=100000 input="ucs2"               -4.21 %            5.362670e-01
 util/normalize-encoding.js n=100000 input="UCS2"              -12.25 %          * 3.123020e-02
 util/normalize-encoding.js n=100000 input="undefined"          -5.21 %            4.693525e-01
 util/normalize-encoding.js n=100000 input="utf-16le"            7.69 %            3.099749e-01
 util/normalize-encoding.js n=100000 input="UTF-16LE"            5.83 %            2.410658e-01
 util/normalize-encoding.js n=100000 input="utf-8"              -4.69 %            4.729617e-01
 util/normalize-encoding.js n=100000 input="utF-8"               1.09 %            8.107947e-01
 util/normalize-encoding.js n=100000 input="uTf-8"              -0.05 %            9.919185e-01
 util/normalize-encoding.js n=100000 input="UTF-8"              -2.07 %            6.396208e-01
 util/normalize-encoding.js n=100000 input="utf16le"            -3.86 %            5.127621e-01
 util/normalize-encoding.js n=100000 input="UTF16LE"            -5.05 %            3.387483e-01
 util/normalize-encoding.js n=100000 input="utf8"               -6.97 %            2.784592e-01
 util/normalize-encoding.js n=100000 input="Utf8"              -10.27 %          * 3.095776e-02
 util/normalize-encoding.js n=100000 input="UTF8"               -1.84 %            6.712303e-01

This PR to 7.9

                                                            improvement confidence      p.value
 util/normalize-encoding.js n=100000 input=""                    3.55 %            6.846831e-01
 util/normalize-encoding.js n=100000 input="[]"                 46.68 %        *** 2.736654e-09
 util/normalize-encoding.js n=100000 input="1"                   4.98 %            3.766843e-01
 util/normalize-encoding.js n=100000 input="base64"             -0.16 %            9.787752e-01
 util/normalize-encoding.js n=100000 input="BASE64"              3.66 %            5.249255e-01
 util/normalize-encoding.js n=100000 input="binary"              8.36 %            1.699122e-01
 util/normalize-encoding.js n=100000 input="BINARY"              7.63 %            1.570101e-01
 util/normalize-encoding.js n=100000 input="false"              10.93 %            1.812432e-01
 util/normalize-encoding.js n=100000 input="foo"                -3.55 %            4.628670e-01
 util/normalize-encoding.js n=100000 input="group_common"      -21.83 %         ** 3.009091e-03
 util/normalize-encoding.js n=100000 input="group_misc"         -7.03 %            2.180318e-01
 util/normalize-encoding.js n=100000 input="group_uncommon"     -9.68 %            9.350889e-02
 util/normalize-encoding.js n=100000 input="group_upper"        -4.28 %            4.121715e-01
 util/normalize-encoding.js n=100000 input="hex"                 8.06 %            1.020501e-01
 util/normalize-encoding.js n=100000 input="HEX"                -6.09 %            2.984084e-01
 util/normalize-encoding.js n=100000 input="latin1"             14.54 %            6.504796e-02
 util/normalize-encoding.js n=100000 input="ucs2"               -5.27 %            4.170471e-01
 util/normalize-encoding.js n=100000 input="UCS2"              -13.12 %          * 3.303378e-02
 util/normalize-encoding.js n=100000 input="undefined"           2.53 %            7.443536e-01
 util/normalize-encoding.js n=100000 input="utf-16le"            5.47 %            4.676833e-01
 util/normalize-encoding.js n=100000 input="UTF-16LE"            5.10 %            3.996346e-01
 util/normalize-encoding.js n=100000 input="utf-8"              -5.90 %            4.555466e-01
 util/normalize-encoding.js n=100000 input="utF-8"              -8.04 %            1.263448e-01
 util/normalize-encoding.js n=100000 input="uTf-8"              -0.35 %            9.586554e-01
 util/normalize-encoding.js n=100000 input="UTF-8"              -8.36 %            1.802233e-01
 util/normalize-encoding.js n=100000 input="utf16le"             0.59 %            9.322743e-01
 util/normalize-encoding.js n=100000 input="UTF16LE"            -6.19 %            2.138244e-01
 util/normalize-encoding.js n=100000 input="utf8"              -10.47 %            1.785193e-01
 util/normalize-encoding.js n=100000 input="Utf8"               -6.53 %            3.446999e-01
 util/normalize-encoding.js n=100000 input="UTF8"               -7.60 %            2.526299e-01

PTAL

jasnell · 2017-04-20T18:26:22Z

CI: https://ci.nodejs.org/job/node-test-pull-request/7558/

mscdex · 2017-04-20T18:26:31Z

You can actually remove toInteger() and toLength() completely, they're not used anywhere anymore.

sam-github · 2017-04-20T18:34:52Z

lib/internal/util.js

@@ -186,4 +197,27 @@ exports.convertToValidSignal = function convertToValidSignal(signal) {
  }

  throw new Error('Unknown signal: ' + signal);
+}
+
+module.exports = {


assigning to exports = as well is more future proof, it allows util functions to refer to each other more easily.

Of the 3 common export styles, at-top, at-definition (what util used to do), and at-bottom,at-bottom remains my least favorite. It lacks the visibility and readability of export at top, and lacks the cohesion of export at time of definition. But if at-bottom is how node.js goes, I'll follow along.

Avoiding the need to refer to exports.{whatever} is important and is why the various exported functions are all named top level functions (they can refer to each other without the exports. bit). That said, it's harmless to add.

One problem with having it at the top is that you can run into variable definition issues, where you're exporting something that gets defined/set later on in the file. Sometimes you might be able to just pull the definition to the top, but other times the assignment may not be a simple one-liner. So putting it at the bottom means you never have to worry about such issues and doing it the same everywhere is good for consistency.

sam-github · 2017-04-20T18:36:19Z

lib/internal/util.js

+function isError(e) {
+  return objectToString(e) === '[object Error]' || e instanceof Error;
+}
+


sometimes there is two lines beween definitions, sometimes one, should probably be consisten

sam-github

AFAICT from the diff, this is adding new APIs to util, toInteger() and toLength(). Am I misreading?

jasnell · 2017-04-20T18:40:31Z

The toInteger() and toLength() bits were remnants from the original version of this. Those were removed subsequently but did not get deleted after the rebase. I'm removing them.

jasnell · 2017-04-20T18:42:41Z

PR Updated to remove the toInteger() and toLength(). PTAL

sam-github · 2017-04-20T18:44:32Z

I would prefer for both exports and module.exports to be reassigned, to keep a consistent idiom (and I think it costs nothing). Other than that, LGTM

mscdex · 2017-04-20T18:44:45Z

lib/internal/util.js

+  return Object.prototype.toString.call(o);
+}
+
+  // Mark that a method should not be used.


These comments shouldn't be indented.

whoops! good catch :-)

jasnell · 2017-04-20T18:47:52Z

Updated!

mscdex · 2017-04-20T19:00:52Z

LGTM if CI is ok with it: https://ci.nodejs.org/job/node-test-pull-request/7560/

jasnell · 2017-04-20T19:51:55Z

Fedora on CI has been a bit challenged today in terms of speed, but everything else looks good. Will get this landed tomorrow assuming there are no issues.

* Use the more efficient module.exports = {} approach * Eliminate some uses of arguments PR-URL: #11404 Reviewed-By: Sam Roberts <[email protected]> Reviewed-By: Brian White <[email protected]

jasnell · 2017-04-21T18:25:37Z

Landed in 9077b48

gibfahn · 2017-06-18T16:51:50Z

@jasnell is this something we want on v6.x?

jasnell · 2017-06-19T14:48:17Z

Only if there's no negative perf hit.

gibfahn · 2017-06-19T15:05:49Z

Okay, I'm marking this as don't land, but if anyone is willing to backport (and test the perf) on this then go for it.

nodejs-github-bot added the util Issues and PRs related to the built-in util module. label Feb 15, 2017

jasnell force-pushed the internal-util-refactor branch from 37282b0 to ece8161 Compare February 15, 2017 21:23

jasnell changed the title ~~src: refactor internal/util~~ lib: refactor internal/util Feb 15, 2017

vsemozhetbyt mentioned this pull request Feb 15, 2017

Another case of possibly deoptimizing use of arguments in libs #11001

Closed

vsemozhetbyt mentioned this pull request Feb 16, 2017

module.exports = {} approach and objects with fast / slow properties #11430

Closed

vsemozhetbyt mentioned this pull request Feb 17, 2017

net: avoid use of arguments #11411

Closed

2 tasks

jasnell added the wip Issues and PRs that are still a work in progress. label Feb 17, 2017

jasnell force-pushed the internal-util-refactor branch from ece8161 to 7ba5f4c Compare February 17, 2017 21:08

jasnell force-pushed the internal-util-refactor branch 3 times, most recently from 3acfcb3 to 1fe0ac8 Compare February 20, 2017 15:43

fhalde suggested changes Feb 21, 2017

View reviewed changes

refack force-pushed the master branch from 16073c0 to fbe946b Compare April 14, 2017 04:11

lib: refactor internal/util

bf29544

* Use the more efficient module.exports = {} approach * Eliminate some uses of arguments

jasnell force-pushed the internal-util-refactor branch from 1fe0ac8 to bf29544 Compare April 20, 2017 17:58

jasnell removed the wip Issues and PRs that are still a work in progress. label Apr 20, 2017

sam-github reviewed Apr 20, 2017

View reviewed changes

Address nits

4b013e8

mscdex reviewed Apr 20, 2017

View reviewed changes

Address nits

3a6954a

jasnell closed this Apr 21, 2017

TimothyGu mentioned this pull request Apr 22, 2017

url: use util's getConstructorOf #12526

Closed

1 task

jasnell mentioned this pull request May 11, 2017

8.0.0 Release Proposal #12220

Closed

gibfahn mentioned this pull request Jun 15, 2017

Auditing for 6.11.1 nodejs/Release#230

Closed

3 tasks

gibfahn added the lts-watch-v6.x label Jun 18, 2017

gibfahn added land-on-v6.x and removed lts-watch-v6.x labels Jun 19, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

lib: refactor internal/util #11404

lib: refactor internal/util #11404

jasnell commented Feb 15, 2017

mscdex commented Feb 15, 2017 •

edited

Loading

jasnell commented Feb 15, 2017

sam-github commented Feb 16, 2017

sam-github commented Feb 16, 2017

vsemozhetbyt commented Feb 16, 2017

jasnell commented Feb 17, 2017

jasnell commented Feb 17, 2017

fhalde left a comment

jasnell commented Feb 21, 2017

jasnell commented Apr 20, 2017

jasnell commented Apr 20, 2017

mscdex commented Apr 20, 2017

sam-github Apr 20, 2017

jasnell Apr 20, 2017

mscdex Apr 20, 2017

sam-github Apr 20, 2017

sam-github left a comment

jasnell commented Apr 20, 2017

jasnell commented Apr 20, 2017

sam-github commented Apr 20, 2017

mscdex Apr 20, 2017

jasnell Apr 20, 2017

jasnell commented Apr 20, 2017

mscdex commented Apr 20, 2017

jasnell commented Apr 20, 2017

jasnell commented Apr 21, 2017

gibfahn commented Jun 18, 2017

jasnell commented Jun 19, 2017

gibfahn commented Jun 19, 2017

lib: refactor internal/util #11404

lib: refactor internal/util #11404

Conversation

jasnell commented Feb 15, 2017

Checklist

Affected core subsystem(s)

mscdex commented Feb 15, 2017 • edited Loading

jasnell commented Feb 15, 2017

sam-github commented Feb 16, 2017

sam-github commented Feb 16, 2017

vsemozhetbyt commented Feb 16, 2017

jasnell commented Feb 17, 2017

jasnell commented Feb 17, 2017

fhalde left a comment

Choose a reason for hiding this comment

jasnell commented Feb 21, 2017

jasnell commented Apr 20, 2017

jasnell commented Apr 20, 2017

mscdex commented Apr 20, 2017

sam-github Apr 20, 2017

Choose a reason for hiding this comment

jasnell Apr 20, 2017

Choose a reason for hiding this comment

mscdex Apr 20, 2017

Choose a reason for hiding this comment

sam-github Apr 20, 2017

Choose a reason for hiding this comment

sam-github left a comment

Choose a reason for hiding this comment

jasnell commented Apr 20, 2017

jasnell commented Apr 20, 2017

sam-github commented Apr 20, 2017

mscdex Apr 20, 2017

Choose a reason for hiding this comment

jasnell Apr 20, 2017

Choose a reason for hiding this comment

jasnell commented Apr 20, 2017

mscdex commented Apr 20, 2017

jasnell commented Apr 20, 2017

jasnell commented Apr 21, 2017

gibfahn commented Jun 18, 2017

jasnell commented Jun 19, 2017

gibfahn commented Jun 19, 2017

mscdex commented Feb 15, 2017 •

edited

Loading