node: Make builtin libs available for `--eval` #6207

addaleax · 2016-04-15T01:07:50Z

Checklist

tests and code linting passes
a test and/or benchmark is included
documentation is changed or added
the commit message follows commit guidelines

Affected core subsystem(s)

cli

Description of change

Make the builtin libraries available for the --eval and --print CLI options, using the same mechanism that the REPL uses.

This renders workarounds like node -e 'require("fs").doStuff()' unnecessary which I find myself often using.

jasnell · 2016-04-15T07:45:16Z

oh, yes please. Haven't done a complete review yet but would love for this to happen.

bnoordhuis · 2016-04-15T07:48:45Z

lib/internal/bootstrap_node.js

@@ -101,10 +101,14 @@
        delete process.env.NODE_UNIQUE_ID;
      }

+      var internalModule = NativeModule.require('internal/module');


Can you move this into the branches of the if statement below? It avoids the hit of loading the module unless it's actually used.

Fishrock123 · 2016-04-15T13:50:31Z

Hmmm, I feel like there was an issue for this, but search doesn't return anything.

Fishrock123 · 2016-04-15T14:05:01Z

lib/internal/module.js

+exports.builtinLibs = ['assert', 'buffer', 'child_process', 'cluster',
+  'crypto', 'dgram', 'dns', 'domain', 'events', 'fs', 'http', 'https', 'net',
+  'os', 'path', 'punycode', 'querystring', 'readline', 'stream',
+  'string_decoder', 'tls', 'tty', 'url', 'util', 'v8', 'vm', 'zlib'];


Another good excuse for #3307?

I guess it doesn't matter too much. The only thing that is remotely on the roadmap as a module is async_wrap.

I guess it doesn't matter too much.

Agreed. And I just noticed that the list misses the repl module itself? Should I add that myself here? 😄

+1 for having an API that exposes the list. Let's be sure to make it read-only so we don't end up with another process.config situation ;-)

Fishrock123 · 2016-04-15T14:12:29Z

CI: https://ci.nodejs.org/job/node-test-pull-request/2276/

Seems good to me, I'm unsure if more tests are necessary or not, but the repl test doesn't seem much more comprehensive either: https://github.com/nodejs/node/blob/master/test/parallel/test-repl-autolibs.js

thefourtheye · 2016-04-17T04:25:30Z

lib/internal/module.js

+    Object.defineProperty(object, name, {
+      get: () => {
+        const lib = require(name);
+        object[name] = lib;


Isn't this overriding the value set by set below?

@thefourtheye Nah, it’s a nice but weird trick. The setter below deletes the property before re-assigning it, which disables the setter/getter themselves. So only the first assignment or read of this variable uses this mechanism, and nothing gets overridden.
I was a bit confused at first, too, so I guess it’s a good idea to add a comment for explanation. 😄

(This code has essentially only been moved from lib/repl.js, btw)

addaleax · 2016-04-17T14:12:38Z

Updated again with more explanatory comments + added repl to the list of builtin modules.

bnoordhuis · 2016-04-18T09:13:03Z

lib/internal/module.js

+        delete object[name];
+        object[name] = val;
+      },
+      configurable: true


Maybe also set enumerable: true? The delete+assign setter trick is alright but it makes the property enumerable.

EDIT: I guess that could have unexpected consequences. Perhaps just make sure it stays non-enumerable.

Perhaps just make sure it stays non-enumerable.

@bnoordhuis That’s probably doable, but I think it would get a lot trickier to maintain the current behaviour of allowing to re-assign these globals. Correct me if I’m wrong, but to do this, you’d have to:

Stop invoking the setter from the getter, and

Call delete from there “manually”, and use Object.defineProperty({ value: … }) to set the reference to the module, and

Give that new property descriptor the setter that is used right now, too, so that in the case of user-defined globals with these names they are enumerable as expected

Maybe there’s a simpler way, and I agree that it’s a bit weird the way it is right now (and has been for the last three years), but I don’t think anybody really cares very deeply.

Wouldn't this work?

function get() { const value = require(name); Object.defineProperty(object, name, { value, writable: true }); return value; } function set(val) { void object[name]; // Property access for side-effect. object[name] = val; } Object.defineProperty(object, name, { get, set, configurable: true });

It would leave the property non-enumerable in all cases, even when manually overwritten by the user. If that’s okay, sure, that basically works (It has to be delete object[name] where you wrote void object[name] though, so that the setter does not enter infinite recursion).

addaleax · 2016-04-18T11:20:27Z

@bnoordhuis Updated this with the behaviour I think you had in mind

evanlucas · 2016-04-18T12:03:16Z

CI again: https://ci.nodejs.org/job/node-test-pull-request/2296/

Make the builtin libraries available for the `--eval` and `--print` CLI options, using the same mechanism that the REPL uses. This renders workarounds like `node -e 'require("fs").doStuff()'` unnecessary. As part of this, the list of builtin modules and the code for adding the corresponding properties to the target context is moved to `internal/module.js`, and the previously missing `repl` entry is added.

addaleax · 2016-04-18T13:01:29Z

Rebased & squashed into two commits that I’d consider rather orthogonal changes, PTAL :)
Also new CI because the previous one failed, probably unrelated, but it might be a good opportunity to try running that myself for the first time: https://ci.nodejs.org/job/node-test-pull-request/2297/

Make sure that the built-in modules in the repl stay non-enumerable. Previously, they would pop up as enumerable properties of the global object after having been accessed for the first time.

jasnell · 2016-04-18T17:01:40Z

LGTM

bnoordhuis · 2016-04-18T18:14:28Z

LGTM2. Traditionally, the first line of the commit log is all lowercase.

addaleax · 2016-04-18T18:26:18Z

@bnoordhuis Yeah, I already noticed that I screwed that up again… someday I’ll get it all right. 😄

cjihrig · 2016-04-18T18:33:40Z

LGTM. CI is green.

Make the builtin libraries available for the `--eval` and `--print` CLI options, using the same mechanism that the REPL uses. This renders workarounds like `node -e 'require("fs").doStuff()'` unnecessary. As part of this, the list of builtin modules and the code for adding the corresponding properties to the target context is moved to `internal/module.js`, and the previously missing `repl` entry is added. PR-URL: #6207 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Colin Ihrig <[email protected]>

Make sure that the built-in modules in the repl stay non-enumerable. Previously, they would pop up as enumerable properties of the global object after having been accessed for the first time. PR-URL: #6207 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Colin Ihrig <[email protected]>

jasnell · 2016-04-18T22:33:13Z

Landed in 39d905e and ebc8c37. Fixed the commit log nits on landing. Thanks!

Make sure that the built-in modules in the repl stay non-enumerable. Previously, they would pop up as enumerable properties of the global object after having been accessed for the first time. PR-URL: #6207 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Colin Ihrig <[email protected]>

Buffer: * Buffer.prototype.compare can now compare sub-ranges of two Buffers (James M Snell) #5880 deps: * update to http-parser 2.7.0 (Fedor Indutny) #6279 * update ESLint to 2.7.0 (silverwind) #6132 net: * adds support for passing DNS lookup hints to createConnection() (Colin Ihrig) #6000 node: * Make the builtin libraries available for the --eval and --print CLI options (Anna Henningsen) #6207 npm: * upgrade npm to 3.8.6 (Kat Marchán) #6153 repl: * Pressing enter in the repl will repeat the last command by default if no input has been received. This behaviour was in node previously and was not removed intentionally. (Rich Trott) #6090 src: * add SIGINFO to supported signals (James Reggio) #6093 streams: * Fix a regression that caused by net streams requesting multiple chunks synchronously when combined with cork/uncork (Matteo Collina) #6164 zlib: * The flushing flag is now configurable allowing for decompression of partial data (Anna Henningsen) #6069

Make the builtin libraries available for the `--eval` and `--print` CLI options, using the same mechanism that the REPL uses. This renders workarounds like `node -e 'require("fs").doStuff()'` unnecessary. As part of this, the list of builtin modules and the code for adding the corresponding properties to the target context is moved to `internal/module.js`, and the previously missing `repl` entry is added. PR-URL: #6207 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Colin Ihrig <[email protected]>

Make sure that the built-in modules in the repl stay non-enumerable. Previously, they would pop up as enumerable properties of the global object after having been accessed for the first time. PR-URL: #6207 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Colin Ihrig <[email protected]>

Buffer: * Buffer.prototype.compare can now compare sub-ranges of two Buffers (James M Snell) #5880 deps: * update to http-parser 2.7.0 (Fedor Indutny) #6279 * update ESLint to 2.7.0 (silverwind) #6132 net: * adds support for passing DNS lookup hints to createConnection() (Colin Ihrig) #6000 node: * Make the builtin libraries available for the --eval and --print CLI options (Anna Henningsen) #6207 npm: * upgrade npm to 3.8.6 (Kat Marchán) #6153 repl: * Pressing enter in the repl will repeat the last command by default if no input has been received. This behavior was in node previously and was not removed intentionally. (Rich Trott) #6090 src: * add SIGINFO to supported signals (James Reggio) #6093 streams: * Fix a regression that caused by net streams requesting multiple chunks synchronously when combined with cork/uncork (Matteo Collina) #6164 zlib: * The flushing flag is now configurable allowing for decompression of partial data (Anna Henningsen) #6069

Buffer: * Buffer.prototype.compare can now compare sub-ranges of two Buffers (James M Snell) #5880 deps: * update to http-parser 2.7.0 (Fedor Indutny) #6279 * update ESLint to 2.7.0 (silverwind) #6132 net: * adds support for passing DNS lookup hints to createConnection() (Colin Ihrig) #6000 node: * Make the builtin libraries available for the --eval and --print CLI options (Anna Henningsen) #6207 npm: * upgrade npm to 3.8.6 (Kat Marchán) #6153 repl: * Pressing enter in the repl will repeat the last command by default if no input has been received. This behaviour was in node previously and was not removed intentionally. (Rich Trott) #6090 src: * add SIGINFO to supported signals (James Reggio) #6093 streams: * Fix a regression that caused by net streams requesting multiple chunks synchronously when combined with cork/uncork (Matteo Collina) #6164 zlib: * The flushing flag is now configurable allowing for decompression of partial data (Anna Henningsen) #6069 PR-URL: #6322

cpojer · 2016-04-22T03:17:52Z

fyi this change breaks code like node -e "const path = require('path');" which might be found in existing scripts.

addaleax · 2016-04-22T10:08:01Z

@cpojer I think that went unnoticed by me because it doesn’t happen with the current master or the current Node.js v6 release candidates…

addaleax · 2016-04-22T10:16:01Z

@cpojer Also, the --eval/--print argument is executed in sloppy mode, not strict mode. In Node.js v5 that means that const doesn’t work as expected anyway.

cpojer · 2016-04-25T03:43:56Z

It would probably be nice to mention this behavior change somewhere.

addaleax · 2016-04-25T11:21:03Z

@cpojer Got any suggestions for that (esp. the “somewhere”)? The docs have been amended to reflect the fact that these modules are now predefined, and it’s listed in the changelog as a notable change.

Make the builtin libraries available for the `--eval` and `--print` CLI options, using the same mechanism that the REPL uses. This renders workarounds like `node -e 'require("fs").doStuff()'` unnecessary. As part of this, the list of builtin modules and the code for adding the corresponding properties to the target context is moved to `internal/module.js`, and the previously missing `repl` entry is added. PR-URL: nodejs#6207 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Colin Ihrig <[email protected]>

Make sure that the built-in modules in the repl stay non-enumerable. Previously, they would pop up as enumerable properties of the global object after having been accessed for the first time. PR-URL: nodejs#6207 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Colin Ihrig <[email protected]>

Buffer: * Buffer.prototype.compare can now compare sub-ranges of two Buffers (James M Snell) nodejs#5880 deps: * update to http-parser 2.7.0 (Fedor Indutny) nodejs#6279 * update ESLint to 2.7.0 (silverwind) nodejs#6132 net: * adds support for passing DNS lookup hints to createConnection() (Colin Ihrig) nodejs#6000 node: * Make the builtin libraries available for the --eval and --print CLI options (Anna Henningsen) nodejs#6207 npm: * upgrade npm to 3.8.6 (Kat Marchán) nodejs#6153 repl: * Pressing enter in the repl will repeat the last command by default if no input has been received. This behaviour was in node previously and was not removed intentionally. (Rich Trott) nodejs#6090 src: * add SIGINFO to supported signals (James Reggio) nodejs#6093 streams: * Fix a regression that caused by net streams requesting multiple chunks synchronously when combined with cork/uncork (Matteo Collina) nodejs#6164 zlib: * The flushing flag is now configurable allowing for decompression of partial data (Anna Henningsen) nodejs#6069 PR-URL: nodejs#6322

Make the builtin libraries available for the `--eval` and `--print` CLI options, using the same mechanism that the REPL uses. This renders workarounds like `node -e 'require("fs").doStuff()'` unnecessary. As part of this, the list of builtin modules and the code for adding the corresponding properties to the target context is moved to `internal/module.js`, and the previously missing `repl` entry is added. PR-URL: #6207 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Colin Ihrig <[email protected]>

Make sure that the built-in modules in the repl stay non-enumerable. Previously, they would pop up as enumerable properties of the global object after having been accessed for the first time. PR-URL: #6207 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Colin Ihrig <[email protected]>

Buffer: * Buffer.prototype.compare can now compare sub-ranges of two Buffers (James M Snell) #5880 deps: * update to http-parser 2.7.0 (Fedor Indutny) #6279 * update ESLint to 2.7.0 (silverwind) #6132 net: * adds support for passing DNS lookup hints to createConnection() (Colin Ihrig) #6000 node: * Make the builtin libraries available for the --eval and --print CLI options (Anna Henningsen) #6207 npm: * upgrade npm to 3.8.6 (Kat Marchán) #6153 repl: * Pressing enter in the repl will repeat the last command by default if no input has been received. This behaviour was in node previously and was not removed intentionally. (Rich Trott) #6090 src: * add SIGINFO to supported signals (James Reggio) #6093 streams: * Fix a regression that caused by net streams requesting multiple chunks synchronously when combined with cork/uncork (Matteo Collina) #6164 zlib: * The flushing flag is now configurable allowing for decompression of partial data (Anna Henningsen) #6069 PR-URL: #6322

mscdex added the lib / src Issues and PRs related to general changes in the lib or src directory. label Apr 15, 2016

jasnell added the semver-minor PRs that contain new features and should be released in the next minor version. label Apr 15, 2016

bnoordhuis reviewed Apr 15, 2016
View reviewed changes

Fishrock123 reviewed Apr 15, 2016
View reviewed changes

Fishrock123 mentioned this pull request Apr 15, 2016

Expose up-to-date list of builtin modules #3307

Closed

thefourtheye reviewed Apr 17, 2016
View reviewed changes

bnoordhuis reviewed Apr 18, 2016
View reviewed changes

addaleax added the repl Issues and PRs related to the REPL subsystem. label Apr 18, 2016

addaleax force-pushed the cli-eval-with-builtin-libs branch from ca980db to dafea9d Compare April 18, 2016 12:58

repl: Keep the built-in modules non-enumerable

6015b6a

Make sure that the built-in modules in the repl stay non-enumerable. Previously, they would pop up as enumerable properties of the global object after having been accessed for the first time.

addaleax force-pushed the cli-eval-with-builtin-libs branch from dafea9d to 6015b6a Compare April 18, 2016 13:32

This was referenced Apr 21, 2016

V5.11.0 proposal #6321

Closed

V5.11.0 proposal #6322

Closed

addaleax mentioned this pull request Apr 21, 2016

Compare with REPL, "node -p" & "node -e" has different default imports packages #4069

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

node: Make builtin libs available for `--eval` #6207

node: Make builtin libs available for `--eval` #6207

addaleax commented Apr 15, 2016

jasnell commented Apr 15, 2016

bnoordhuis Apr 15, 2016

Fishrock123 commented Apr 15, 2016

Fishrock123 Apr 15, 2016

addaleax Apr 15, 2016

jasnell Apr 15, 2016

Fishrock123 commented Apr 15, 2016

thefourtheye Apr 17, 2016

addaleax Apr 17, 2016

addaleax commented Apr 17, 2016

bnoordhuis Apr 18, 2016 •

edited

Loading

addaleax Apr 18, 2016

bnoordhuis Apr 18, 2016 •

edited

Loading

addaleax Apr 18, 2016

addaleax commented Apr 18, 2016

evanlucas commented Apr 18, 2016

addaleax commented Apr 18, 2016

jasnell commented Apr 18, 2016

bnoordhuis commented Apr 18, 2016

addaleax commented Apr 18, 2016

cjihrig commented Apr 18, 2016

jasnell commented Apr 18, 2016

cpojer commented Apr 22, 2016

addaleax commented Apr 22, 2016

addaleax commented Apr 22, 2016

cpojer commented Apr 25, 2016

addaleax commented Apr 25, 2016

node: Make builtin libs available for --eval #6207

node: Make builtin libs available for --eval #6207

Conversation

addaleax commented Apr 15, 2016

Checklist

Affected core subsystem(s)

Description of change

jasnell commented Apr 15, 2016

Choose a reason for hiding this comment

Fishrock123 commented Apr 15, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Fishrock123 commented Apr 15, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

addaleax commented Apr 17, 2016

bnoordhuis Apr 18, 2016 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bnoordhuis Apr 18, 2016 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

addaleax commented Apr 18, 2016

evanlucas commented Apr 18, 2016

addaleax commented Apr 18, 2016

jasnell commented Apr 18, 2016

bnoordhuis commented Apr 18, 2016

addaleax commented Apr 18, 2016

cjihrig commented Apr 18, 2016

jasnell commented Apr 18, 2016

cpojer commented Apr 22, 2016

addaleax commented Apr 22, 2016

addaleax commented Apr 22, 2016

cpojer commented Apr 25, 2016

addaleax commented Apr 25, 2016

node: Make builtin libs available for `--eval` #6207

node: Make builtin libs available for `--eval` #6207

bnoordhuis Apr 18, 2016 •

edited

Loading

bnoordhuis Apr 18, 2016 •

edited

Loading