[v10.x] buffer: fix crash for invalid index types #23795

addaleax · 2018-10-21T04:50:10Z

2555cb4 introduced a crash
when a non-number value was passed to ParseArrayIndex().

We do not always have JS typechecking for that in place, though.
This returns back to the previous behavior of defaulting to 0,
which is certainly questionable.

Refs: #22129
Fixes: #23668

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
commit message follows commit guidelines

nodejs-github-bot · 2018-10-21T04:50:12Z

@addaleax build started: https://ci.nodejs.org/blue/organizations/jenkins/node-test-pull-request-lite-pipeline/detail/node-test-pull-request-lite-pipeline/1323/pipeline

addaleax · 2018-10-21T04:51:05Z

CI: https://ci.nodejs.org/job/node-test-pull-request/18022/

refack

IMHO this should be only for node10. Masking errors is bad practice, with possible security implications.
Let 11 crash until we implement a clearer error message.

addaleax · 2018-10-21T16:03:41Z

@nodejs/tsc PTAL … if there are objections standing on Wednesday I’ll call for a vote.

mcollina · 2018-10-21T17:25:16Z

I don’t think this is a good one. I would prefer a PR that throwed a JS error instead of crashing. Why is that approach unfeasible?

addaleax · 2018-10-21T17:33:30Z

@mcollina I was going for the existing behavior on v10.9.0 and below to avoid a breaking change… I’d be happy to implement that as a semver-major follow up, or, if the TSC thinks it could land on v10.x and v11.x, we could make an exception and add typechecking in a patch release?

Edit: I’ve also decided to not go for the breaking change because the fact that there’s an issue on this demonstrates that this does affect in-the-wild code. I fully agree with throwing, but I’d prefer to keep breaking changes semver-major…

mcollina · 2018-10-21T19:08:42Z

+1 as a follow-up semver-major.

joyeecheung

LGTM since it just restores the previous behavior. A proper (semver-major?) fix would be nice but we don't have to fix it here.

joyeecheung

LGTM since it just restores the previous behavior. A proper (semver-major?) fix would be nice but we don't have to fix it here - or we can go with reverting 2555cb4 but I don't think it's necessary with this change.

joyeecheung

LGTM since it just restores the previous behavior. A proper (semver-major?) fix would be nice but we don't have to fix it here - or we can go with reverting 2555cb4 but I don't think it's necessary with this change.

joyeecheung

LGTM since it just restores the previous behavior. A proper (semver-major?) fix would be nice but we don't have to fix it here - or we can go with reverting 2555cb4 but I don't think it's necessary with this change.

joyeecheung

LGTM since it just restores the previous behavior. A proper (semver-major?) fix would be nice but we don't have to fix it here - or we can go with reverting 2555cb4 but I don't think it's necessary with this change.

joyeecheung · 2018-10-22T13:31:26Z

(Oops, multiple reviews submitted during the GitHub outage)

ryzokuken

+1 to backporting this, fixing the bug. Let me know what behavior everyone agrees with for the actual semver-major fix and I'll gladly open a PR.

refack · 2018-10-23T20:14:10Z

Edit: I’ve also decided to not go for the breaking change because the fact that there’s an issue on this demonstrates that this does affect in-the-wild code. I fully agree with throwing, but I’d prefer to keep breaking changes semver-major…

BTW the bug (#23668) was opened for v11.0.0-nightly20181007061e09891c.

The actual bug (wyhaya/updns#7) is in https://github.com/wyhaya/updns/blob/4fb62c6fa6863089c8443d737ad1de61031280af/lib/parse.js#L125-L126

    question.qtype.copy(buf, 12 + qname.length, question.qtype, 2)
    question.qclass.copy(buf, 12 + qname.length + 2, question.qclass, 2)

third argument is the buffer, and not a bufferStart number

This module has been failing silently since the module was published, possibly leaking memory content.

So IIUC we have a crash instead of a silent fail in the node10 line, and that is going like that into LTS?
Also node11 just went out with this behaviour.
So IMHO there's an argument to be made that turning a crash into an exception is semver-patch.

refack · 2018-10-23T20:20:04Z

This returns back to the previous behavior of defaulting to 0,
which is certainly questionable.

P.S. The previous behavior is undefined, it's not default to 0.
e.g. on node 10.9.0

> const b = Buffer.allocUnsafe(1024);
> const c = Buffer.allocUnsafe(512);
> b.copy(c, 'not a valid offset');
512
> b.copy(c, '4');
508
> const d = Buffer.alloc(1)
> d[0]='8'.charCodeAt(0)
> b.copy(c, d);
504

So this PR is semver-major

refack · 2018-10-24T01:01:10Z

Summery:

`buf.copy(target[, targetStart[, sourceStart[, sourceEnd]]])`

Added in: v0.1.90

target | A Buffer or Uint8Array to copy into.
targetStart The offset within target at which to begin writing. Default: 0.
sourceStart The offset within buf from which to begin copying. Default: 0.
sourceEnd The offset within buf at which to stop copying (not inclusive). Default: buf.length.
Returns: The number of bytes copied.

node <= 10.9.0:

> const b = Buffer.allocUnsafe(1024);
undefined
> const c = Buffer.allocUnsafe(512);
undefined
> b.copy(c, 8)
504
> b.copy(c, 8, 'gaga')
504
> b.copy(c, '8', '600')
424
> b.copy(c, 'gaga', 600)
424
> b.copy(c, '112', 600)
400

node >= 10.10.0 || 11.0.0

> const b = Buffer.allocUnsafe(1024);
undefined
> const c = Buffer.allocUnsafe(512);
undefined
> b.copy(c, 8)
504
> b.copy(c, 8, 'gaga')
# node11.0.0.exe  [27956]: src\node_buffer.cc:173: Assertion `arg->IsNumber()' failed.

i.e. Crash

this patch

> const b = Buffer.allocUnsafe(1024);
undefined
> const c = Buffer.allocUnsafe(512);
undefined
> b.copy(c, 8)
504
> b.copy(c, 'gaga')
512
> b.copy(c, '8')
512
> b.copy(c, 'gaga', 600)
424
> b.copy(c, '112', 600)
424

my suggestion

> const b = Buffer.allocUnsafe(1024);
undefined
> const c = Buffer.allocUnsafe(512);
undefined
> b.copy(c, 8)
504
> b.copy(c, 'gaga')
RangeError: Index out of range
    at Buffer.copy (buffer.js:637:12)
> b.copy(c, '8')
RangeError: Index out of range
    at Buffer.copy (buffer.js:637:12)
> b.copy(c, 'gaga', 600)
RangeError: Index out of range
    at Buffer.copy (buffer.js:637:12)
> b.copy(c, '112', 600)
RangeError: Index out of range
    at Buffer.copy (buffer.js:637:12)

Trott · 2018-10-24T03:01:17Z

After talking a bunch with @refack and doing a minimal bit of tinkering, it does seem like this does not quite restore the previous behavior. This might duplicate comments above by @refack, but this is a small behavior change:

Node.js 10.9.0 (which is the version with the behavior we wish to restore):

> var b = Buffer.allocUnsafe(1024)
undefined
> var c = Buffer.allocUnsafe(512)
undefined
> b.copy(c, '112', 600)
400
> b.copy(c, 112, 600)
400
>

With this patch:

$ ./node
> var b = Buffer.allocUnsafe(1024)
undefined
> var c = Buffer.allocUnsafe(512)
undefined
> b.copy(c, '112', 600)
424
> b.copy(c, 112, 600)
400
>

Can we patch that up? Or is that not desirable for a reason I don't see?

addaleax · 2018-10-25T14:38:01Z

I’ve re-targeted this PR to v10.x, under the assumption that we can do something like #23840 a) for all affected methods and b) without significant performance impact.

CI: https://ci.nodejs.org/job/node-test-pull-request/18150/

IMHO this is a valid option for 10.x

jasnell · 2018-10-25T14:46:55Z

@nodejs/tsc ... We have two competing PRs addressing the same issue. We need a resolution. This PR is one option, #23875 is the other.

MylesBorins · 2018-10-25T15:47:49Z

I personally defer to the review of folks that know this code but would like to see something land ASAP so we can test 10.13.0 before pushing it out

mcollina

LGTM

ryzokuken · 2018-10-25T15:53:30Z

@MylesBorins for the record (as the person who originally made the change that broke things, sorry for that) I'd prefer this one over #23875 as long as it uses V8 functions that'll be deprecated soon.

richardlau · 2018-10-25T16:04:38Z

With regards to v10.x:

https://github.com/nodejs/node/blob/f14274714d00bb5b58482bef15e84cdda9237910/COLLABORATOR_GUIDE.md#when-breaking-changes-actually-break-things

If a breaking commit does accidentally land in a Current or LTS branch, an attempt to fix the issue will be made before the next release; If no fix is provided then the commit will be reverted.

2555cb4 is the breaking commit that landed in v10.x (v10.10.0, although the changelog has this as c637d41b9d).

#23795 is an attempt to fix the issue while #23875 is a revert.

#23795 has a number of approvals, so IMHO can land as a fix.

The alternative is to revert (#23875) and then immediately re-backport #22129 including the changes from #23795.

addaleax · 2018-10-25T16:06:11Z

@MylesBorins @richardlau I think all we need at this point is a green CI.

We could either rebase out the commit that broke the bootstrap module count for Workers CI, or land #23876 as soon as we can.

Trott · 2018-10-25T16:45:44Z

Resume Build CI: https://ci.nodejs.org/job/node-test-pull-request/18153/

MylesBorins · 2018-10-26T06:30:00Z

Another attempt at CI: https://ci.nodejs.org/job/node-test-pull-request/18166/

Trott · 2018-10-26T18:16:18Z

Resume Build CI: https://ci.nodejs.org/job/node-test-pull-request/18176/

2555cb4 introduced a crash when a non-number value was passed to `ParseArrayIndex()`. We do not always have JS typechecking for that in place, though. This returns back to the previous behavior of coercing values to integers, which is certainly questionable. Refs: nodejs#22129 Fixes: nodejs#23668

addaleax · 2018-10-27T08:15:10Z

I’ve removed #23625 from v10.x-staging.

CI should be good now: https://ci.nodejs.org/job/node-test-pull-request/18181/

addaleax · 2018-10-28T09:22:21Z

Resume CI again: https://ci.nodejs.org/job/node-test-pull-request/18186/

MylesBorins · 2018-10-29T04:40:40Z

Another CI attempt: https://ci.nodejs.org/job/node-test-pull-request/18193/

addaleax · 2018-10-29T16:20:46Z

Sigh, resume CI again: https://ci.nodejs.org/job/node-test-pull-request/18208/
Again: https://ci.nodejs.org/job/node-test-pull-request/18209/
Again: https://ci.nodejs.org/job/node-test-pull-request/18210/
Again: https://ci.nodejs.org/job/node-test-pull-request/18211/

refack · 2018-10-29T19:17:46Z

I've reimplemented the workaround for #22006
https://ci.nodejs.org/job/node-test-pull-request/18213/
The better solution is to backport #23733

MylesBorins · 2018-10-29T19:42:11Z

landed in the v10.13.0-proposal branch in 2ba6010

addaleax added regression Issues related to regressions. lts-watch-v10.x labels Oct 21, 2018

addaleax requested a review from ryzokuken October 21, 2018 04:50

nodejs-github-bot added buffer Issues and PRs related to the buffer subsystem. c++ Issues and PRs that require attention from people who are familiar with C++. labels Oct 21, 2018

cjihrig approved these changes Oct 21, 2018

View reviewed changes

refack previously requested changes Oct 21, 2018

View reviewed changes

addaleax added the tsc-agenda Issues and PRs to discuss during the meetings of the TSC. label Oct 21, 2018

joyeecheung approved these changes Oct 22, 2018

View reviewed changes

mhdawson mentioned this pull request Oct 22, 2018

Node.js Foundation Technical Steering Committee (TSC) Meeting 2018-10-24 nodejs/TSC#615

Closed

Trott approved these changes Oct 22, 2018

View reviewed changes

joyeecheung approved these changes Oct 22, 2018

View reviewed changes

ryzokuken approved these changes Oct 22, 2018

View reviewed changes

refack added the semver-major PRs that contain breaking changes and should be released in the next major version. label Oct 23, 2018

refack mentioned this pull request Oct 24, 2018

buffer: validate arg types #23840

Closed

3 tasks

addaleax force-pushed the fix-buffer-intvalue branch from 27aa8f6 to b31dd6e Compare October 24, 2018 05:44

addaleax force-pushed the fix-buffer-intvalue branch from 1036d6e to 027082d Compare October 25, 2018 14:37

addaleax changed the base branch from master to v10.x-staging October 25, 2018 14:38

mcollina approved these changes Oct 25, 2018

View reviewed changes

addaleax force-pushed the v10.x-staging branch from 8b431ad to 3f63297 Compare October 27, 2018 08:14

addaleax force-pushed the fix-buffer-intvalue branch from 027082d to b617de0 Compare October 27, 2018 08:14

refack mentioned this pull request Oct 29, 2018

build: add -Werror=undefined-inline to clang builds #23961

Merged

3 tasks

refack added the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label Oct 29, 2018

MylesBorins closed this Oct 29, 2018

addaleax deleted the fix-buffer-intvalue branch October 29, 2018 19:42

MylesBorins mentioned this pull request Nov 20, 2018

[v10.x backport] test: move tick.js from test/async-hooks to test/common #23780

Closed

4 tasks

refack mentioned this pull request Nov 25, 2018

Regression v10.x node_file.cc:1713: Assertion `(off) <= (buffer_length)' failed. #24640

Closed

addaleax mentioned this pull request Dec 20, 2018

buffer: fix crash for invalid index types #25154

Closed

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[v10.x] buffer: fix crash for invalid index types #23795

[v10.x] buffer: fix crash for invalid index types #23795

addaleax commented Oct 21, 2018

nodejs-github-bot commented Oct 21, 2018

addaleax commented Oct 21, 2018

refack left a comment •

edited

Loading

addaleax commented Oct 21, 2018

mcollina commented Oct 21, 2018

addaleax commented Oct 21, 2018 •

edited

Loading

mcollina commented Oct 21, 2018

joyeecheung left a comment

joyeecheung left a comment

joyeecheung left a comment

joyeecheung left a comment

joyeecheung left a comment

joyeecheung commented Oct 22, 2018

ryzokuken left a comment

refack commented Oct 23, 2018

refack commented Oct 23, 2018 •

edited

Loading

refack commented Oct 24, 2018 •

edited

Loading

Trott commented Oct 24, 2018 •

edited

Loading

addaleax commented Oct 25, 2018 •

edited

Loading

jasnell commented Oct 25, 2018

MylesBorins commented Oct 25, 2018

mcollina left a comment

ryzokuken commented Oct 25, 2018

richardlau commented Oct 25, 2018

addaleax commented Oct 25, 2018

Trott commented Oct 25, 2018

MylesBorins commented Oct 26, 2018

Trott commented Oct 26, 2018

addaleax commented Oct 27, 2018

addaleax commented Oct 28, 2018

MylesBorins commented Oct 29, 2018

addaleax commented Oct 29, 2018 •

edited

Loading

refack commented Oct 29, 2018

MylesBorins commented Oct 29, 2018

[v10.x] buffer: fix crash for invalid index types #23795

[v10.x] buffer: fix crash for invalid index types #23795

Conversation

addaleax commented Oct 21, 2018

Checklist

nodejs-github-bot commented Oct 21, 2018

addaleax commented Oct 21, 2018

refack left a comment • edited Loading

Choose a reason for hiding this comment

addaleax commented Oct 21, 2018

mcollina commented Oct 21, 2018

addaleax commented Oct 21, 2018 • edited Loading

mcollina commented Oct 21, 2018

joyeecheung left a comment

Choose a reason for hiding this comment

joyeecheung left a comment

Choose a reason for hiding this comment

joyeecheung left a comment

Choose a reason for hiding this comment

joyeecheung left a comment

Choose a reason for hiding this comment

joyeecheung left a comment

Choose a reason for hiding this comment

joyeecheung commented Oct 22, 2018

ryzokuken left a comment

Choose a reason for hiding this comment

refack commented Oct 23, 2018

refack commented Oct 23, 2018 • edited Loading

refack commented Oct 24, 2018 • edited Loading

Summery:

buf.copy(target[, targetStart[, sourceStart[, sourceEnd]]])

node <= 10.9.0:

node >= 10.10.0 || 11.0.0

this patch

my suggestion

Trott commented Oct 24, 2018 • edited Loading

addaleax commented Oct 25, 2018 • edited Loading

jasnell commented Oct 25, 2018

MylesBorins commented Oct 25, 2018

mcollina left a comment

Choose a reason for hiding this comment

ryzokuken commented Oct 25, 2018

richardlau commented Oct 25, 2018

addaleax commented Oct 25, 2018

Trott commented Oct 25, 2018

MylesBorins commented Oct 26, 2018

Trott commented Oct 26, 2018

addaleax commented Oct 27, 2018

addaleax commented Oct 28, 2018

MylesBorins commented Oct 29, 2018

addaleax commented Oct 29, 2018 • edited Loading

refack commented Oct 29, 2018

MylesBorins commented Oct 29, 2018

refack left a comment •

edited

Loading

addaleax commented Oct 21, 2018 •

edited

Loading

refack commented Oct 23, 2018 •

edited

Loading

refack commented Oct 24, 2018 •

edited

Loading

`buf.copy(target[, targetStart[, sourceStart[, sourceEnd]]])`

Trott commented Oct 24, 2018 •

edited

Loading

addaleax commented Oct 25, 2018 •

edited

Loading

addaleax commented Oct 29, 2018 •

edited

Loading