tools: don't use global Buffer and add linter rule for it #1794

silverwind · 2015-05-25T21:29:26Z

This does a few things to the linter configuration:

instead of relying on the predefined node environment, specify all our globals manually and as read-only.
enable no-undef to error on undefined variable access.
add globalReturn which allows to return from the top scope. This was implied by the node option before.

A number of tests intentionally use undefined globals to trigger an error or set globals (in the vm tests). I had to exclude those tests with an eslint-disable comment. If these comments are too noisy, we could alternatively disable that rule for all tests.

I might have missed a few globals from the v8 environment. Object.keys(global) doesn't list many of them, presumably because they are unenumerable. Any better ideas on how to discover globals?

cc: @Fishrock123 @domenic

silverwind · 2015-05-25T21:31:57Z

@Fishrock123 what's the plan exactly with Buffer usage mentioned in #1770? Should it error as soon as Buffer is used inside lib?

silverwind · 2015-05-25T21:33:40Z

test/parallel/test-cluster-worker-kill.js

@@ -106,16 +106,12 @@ function checkResults(expected_results, results) {
    var actual = results[k],
        expected = expected_results[k];

-    if (typeof expected === 'function') {
-      expected(r[k]);


Same as above

Fishrock123 · 2015-05-26T14:20:23Z

@Fishrock123 what's the plan exactly with Buffer usage mentioned in #1770? Should it error as soon as Buffer is used inside lib?

Probably? How else would it work?

Fishrock123 · 2015-05-26T14:22:17Z

test/message/nexttick_throw.js

@@ -1,3 +1,4 @@
+/* eslint-disable no-undef */


Perhaps disable for all tests?

Yeah, I think it's better that way, less noisy comments all over the tests, will work something out.

no-undef is one of the few rules that do something good (as opposed to the most eslint rules that do nothing except for enforcing a ~~religion~~ code style). I don't think it should be disabled for all tests, because it could catch real errors.

What about explicitly declaring globals in every file?

/* globals: undefined_reference_error_maker */

I don't remember the exact syntax, but I think you can add globals on per-file basis. And most linters (jshint, eslint) respect that.

I agree generally to using no-undef, but it is pretty common to use undefined globals in tests to trigger an error. With the rule enabled, one would have to research how to configure the linter when writing a test, which I'd like to avoid.

Using a global comment is pretty equivalent to disabling the rule, both require additional ugly comments. I think we're fine with just disabling no-undef in the test directory.

maybe ask people to trigger an error differently?

my favorite way is []()

silverwind · 2015-05-26T16:08:01Z

@Fishrock123 I'll just comment out the Buffer global in .eslintrc to indicate it's there but not allowed to be used directly. We want to use require('buffer').Buffer in place of it, right?

Fishrock123 · 2015-05-26T16:30:51Z

@silverwind correct.

silverwind · 2015-05-29T11:26:57Z

Rebased and disabled no-undef for tests, also moved the rule to the correct category in .eslintrc.

@Fishrock123 There are 83 occasions where Buffer is used directly in lib. Should I go for it and replace them with a non-global?

silverwind · 2015-05-29T16:08:50Z

@Fishrock123 I'll follow up with a port of nodejs/node-v0.x-archive@523929c in another PR. Does this one LGTY?

Fishrock123 · 2015-05-29T17:00:11Z

@silverwind I'd prefer if you did the commits in this order: fix tests, apply full remove-global-buffer patch, add linting.

That being said, this still doesn't seem to catch global Buffer use with ./iojs tools/eslint/bin/eslint.js src lib test --reset?

silverwind · 2015-05-29T17:03:38Z

Well I suppose I can split these up into multiple commits and add the Buffer thing too in its own commit.

silverwind · 2015-05-29T17:44:06Z

Alright, split up in 3 commits now. The circular dependency described in nodejs/node-v0.x-archive#8603 (comment) didn't happen, so no workaround was needed.

silverwind · 2015-05-29T17:45:18Z

CI: https://jenkins-iojs.nodesource.com/view/iojs/job/iojs+any-pr+multi/728/

silverwind · 2015-06-01T16:28:04Z

Updated and cleaned up the list of globals, which is now sorted in categories. @Fishrock123 PTAL

targos · 2015-06-01T16:34:16Z

.eslintrc

+  __dirname         : false
+  global            : false
+  GLOBAL            : false
+  root              : false


GLOBAL and root are going to be deprecated (#1838). Are they still being used in the code base ? If not they do not need to be here.

But they ll still be there till removal, right?

They're not used and no one sane would use them, so good call on removing them.

silverwind · 2015-06-01T17:20:46Z

Filed sindresorhus/globals#33 for the missing DataView builtin in eslint, until then we probably have to keep it listed for correctness (it's only used in one test).

silverwind · 2015-06-01T17:26:59Z

Also filed eslint/eslint#2657 for a simpler way to undefine globals.

silverwind · 2015-06-01T18:14:22Z

Don't review yet, I'll try creating a custom eslint rule for the Buffer case, which would drastically simplify the current changes to globals and also print a nice custom error message in the linter.

silverwind · 2015-06-07T08:12:13Z

Here's my current custom rule, which @xjamundx graciously provided:

'use strict';

module.exports = function(context) {
  var required = false;
  return {
    CallExpression: function(node) {
      if (node.callee.type === 'Identifier' &&
          node.callee.name === 'require') {
        if (node.arguments.length && node.arguments[0].value === 'buffer') {
          required = true;
        }
      }
    },
    Identifier: function(node) {
      if (!required && node.name === 'Buffer') {
        context.report(node, "Make sure to require('buffer').Buffer before using Buffer");
      }
    }
  }
}

It's almost working, except that it does detect an error on the require line itself:

const Buffer = require('buffer').Buffer;
//    ^--- errors here

Any suggestions?

rlidwka · 2015-06-07T08:43:59Z

I think your suggestion in eslint/eslint#2657 is much better than this headache with custom rules. Maybe re-open it?

silverwind · 2015-06-07T20:12:16Z

Up for review again!

trevnorris · 2015-06-08T02:53:19Z

I'm realizing right now that unless we make the exports Buffer property read only it can still be overridden just as easily by users. So what's the point of forcing it to be required in all files?

silverwind · 2015-06-08T08:49:20Z

I don't see writable Buffer as an problem really. It could allow for a custom Buffer implementation and a lot of things in node can be overridden right now anyways.

As for the reason of this change, there's a series of issues/prs that lead to it:

nodejs/node-v0.x-archive#8588
nodejs/node-v0.x-archive#8603
#1770

trevnorris · 2015-06-08T16:38:02Z

Ah yes. I forgot about the REPL issue. Thanks for the reminder.

I don't see the same change here for assert.js as in the final commit in nodejs/node-v0.x-archive@523929c (using b.Buffer instead). Has that issue been fixed?

silverwind · 2015-06-08T19:01:34Z

Yes, I think so. Tests were passing so I assumed it was fixed.

Fishrock123 · 2015-06-09T03:53:23Z

@trevnorris I looked at that while attempting to port the original patch... I'm not sure why that was done in the first place?

Oh, found it here: nodejs/node-v0.x-archive#8603 (comment) Looks like there was a circular dependancy issue?

silverwind · 2015-06-11T16:49:22Z

@Fishrock123 @trevnorris @vkurchatkin I don't understand why this race condition happened last time, but it seems to be gone now, all tests are passing so I assume it must've been fixed in the meanwhile.

I rebased and did a minor correction to the require-buffer rule (switched useless some for forEach). I also caught two errors through no-undef that were introduced in 5795e83, fix for that in #1951.

Enables the following rules: - no-undef: Valuable rule to error on usage of undefined variables - require-buffer: Custom rule that forbids usage of the global Buffer inside lib/ because of REPL issues.

silverwind · 2015-06-11T17:11:11Z

Did another small correction to the printed message. Please review.

trevnorris · 2015-06-11T17:43:08Z

lib/buffer.js

@@ -1,3 +1,4 @@
+/* eslint-disable require-buffer */
 'use strict';


I thought 'use strict' had to go on the first line of either the file or function. Does that exclude comments?

/cc @domenic

nm. see it doesn't matter.

I'm pretty sure it does exclude comments, considering the amount of files with copyright hats at the very top.

trevnorris · 2015-06-11T17:50:22Z

Have one nit, but LGTM.

thefourtheye · 2015-06-11T17:51:26Z

tools/eslint-rules/require-buffer.js

+
+module.exports = function(context) {
+  return {
+    "Program:exit": function() {


Same as above ^^

I'll fix that.

trevnorris · 2015-06-11T18:09:24Z

Great thanks. I'm cool with this change.

Fixes all cases of undeclared variable access as uncovered by the no-undef rule of eslint. PR-URL: #1794 Reviewed-By: Trevor Norris <[email protected]>

Port of nodejs/node-v0.x-archive#8603 The race condition present in the original PR didn't occur, so no workaround was needed. PR-URL: #1794 Reviewed-By: Trevor Norris <[email protected]>

Enables the following rules: - no-undef: Valuable rule to error on usage of undefined variables - require-buffer: Custom rule that forbids usage of the global Buffer inside lib/ because of REPL issues. PR-URL: #1794 Reviewed-By: Trevor Norris <[email protected]>

silverwind · 2015-06-11T18:29:41Z

Thanks! Landed in ff8202c .. 6e4d302.

silverwind reviewed May 25, 2015
View reviewed changes

Fishrock123 reviewed May 26, 2015
View reviewed changes

ChALkeR mentioned this pull request May 26, 2015

Remove undocumented global object aliases. #1747

Closed

silverwind force-pushed the linter-global branch from 4c2445a to 97c17c7 Compare May 29, 2015 11:20

silverwind force-pushed the linter-global branch 2 times, most recently from 002a770 to c38e3cb Compare May 29, 2015 17:42

silverwind force-pushed the linter-global branch from c38e3cb to 54d3652 Compare June 1, 2015 16:26

targos reviewed Jun 1, 2015
View reviewed changes

Fishrock123 mentioned this pull request Jun 1, 2015

[Converge] repl: Private Buffer object in lib/* files nodejs/node-convergence-archive#21

Closed

bnoordhuis force-pushed the master branch from 311c3e4 to b926718 Compare June 2, 2015 17:25

silverwind force-pushed the linter-global branch from 54d3652 to bd8132d Compare June 7, 2015 11:14

silverwind force-pushed the linter-global branch from 920e34c to 56bb849 Compare June 7, 2015 20:07

silverwind changed the title ~~tools: strictly define globals in eslint~~ tools: don't use global Buffer and add linter rule for it Jun 7, 2015

silverwind force-pushed the linter-global branch from 56bb849 to b154935 Compare June 7, 2015 20:13

silverwind added 2 commits June 11, 2015 18:21

test: fix undeclared variable access

092ffd4

lib: don't use global Buffer

12608af

silverwind force-pushed the linter-global branch from b154935 to ac12c6c Compare June 11, 2015 16:46

tools: enable/add additional eslint rules

06135cd

Enables the following rules: - no-undef: Valuable rule to error on usage of undefined variables - require-buffer: Custom rule that forbids usage of the global Buffer inside lib/ because of REPL issues.

silverwind force-pushed the linter-global branch from ac12c6c to 06135cd Compare June 11, 2015 17:09

trevnorris reviewed Jun 11, 2015
View reviewed changes

thefourtheye reviewed Jun 11, 2015
View reviewed changes

Fix nits

98594d0

silverwind added a commit that referenced this pull request Jun 11, 2015

test: fix undeclared variable access

ff8202c

Fixes all cases of undeclared variable access as uncovered by the no-undef rule of eslint. PR-URL: #1794 Reviewed-By: Trevor Norris <[email protected]>

silverwind closed this Jun 11, 2015

TimothyGu mentioned this pull request Jun 13, 2017

buffer: implement WHATWG Encoding Standard API #13644

Closed

4 tasks

snyk-bot mentioned this pull request Jan 20, 2023

[Snyk] Upgrade commander from 2.11.0 to 9.4.1 Supermajestcd/node#3

Open

		@@ -1,3 +1,4 @@
		/* eslint-disable require-buffer */
		'use strict';

tools: don't use global Buffer and add linter rule for it #1794

tools: don't use global Buffer and add linter rule for it #1794

Conversation

silverwind commented May 25, 2015

silverwind commented May 25, 2015

Choose a reason for hiding this comment

Fishrock123 commented May 26, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

silverwind commented May 26, 2015

Fishrock123 commented May 26, 2015

silverwind commented May 29, 2015

silverwind commented May 29, 2015

Fishrock123 commented May 29, 2015

silverwind commented May 29, 2015

silverwind commented May 29, 2015

silverwind commented May 29, 2015

silverwind commented Jun 1, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

silverwind commented Jun 1, 2015

silverwind commented Jun 1, 2015

silverwind commented Jun 1, 2015

silverwind commented Jun 7, 2015

rlidwka commented Jun 7, 2015

silverwind commented Jun 7, 2015

trevnorris commented Jun 8, 2015

silverwind commented Jun 8, 2015

trevnorris commented Jun 8, 2015

silverwind commented Jun 8, 2015

Fishrock123 commented Jun 9, 2015

silverwind commented Jun 11, 2015

silverwind commented Jun 11, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

trevnorris commented Jun 11, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

trevnorris commented Jun 11, 2015

silverwind commented Jun 11, 2015