Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

buffer: improve creation performance #6893

Closed
wants to merge 1 commit into from
Closed

buffer: improve creation performance #6893

wants to merge 1 commit into from

Conversation

RReverser
Copy link
Member

Checklist
  • tests and code linting passes
  • the commit message follows commit guidelines
Affected core subsystem(s)

buffer

Description of change

Improves performance of allocating unsafe buffers, creating buffers from
an existing ArrayBuffer and creating .slice(...) from existing Buffer by
avoiding deoptimizing change of prototype after Uint8Array allocation
in favor of ES6 native subclassing.

This is done through an internal ES6 class that extends Uint8Array and
is used for allocations, but the regular Buffer function is exposed, so
calling Buffer(...) with or without new continues to work as usual
and prototype chains are also preserved.

Performance wins for .slice are +120% (2.2x), and, consequently, for
unsafe allocations up to +95% (1.9x) for small buffers, and for safe
allocations (zero-filled) up to +30% (1.3x).

chart

@nodejs-github-bot nodejs-github-bot added buffer Issues and PRs related to the buffer subsystem. c++ Issues and PRs that require attention from people who are familiar with C++. labels May 20, 2016
@RReverser
Copy link
Member Author

cc @trevnorris as per request

@targos
Copy link
Member

targos commented May 20, 2016

@RReverser
Copy link
Member Author

CI finished, I see only two failures in Windows build configs which are network timeouts. I guess unrelated?

@Fishrock123
Copy link
Contributor

Hasn't Trevor already changed this between JS and C++ like 3 times? 😂

@RReverser
Copy link
Member Author

RReverser commented May 20, 2016

@Fishrock123 Well, it's mostly in JS, but the inheritance from Uint8Array was done inefficiently. I just additionally removed CreateFromArrayBuffer in C++ as it's not needed when you have native subclass of Uint8Array which can be already instantiated with an ArrayBuffer as an argument (and do that faster).

TL;DR: the biggest win here is unrelated to the C++ change, but rather to the ES6 native subclassing.

@jasnell
Copy link
Member

jasnell commented May 20, 2016

@nodejs/buffer

@eljefedelrodeodeljefe
Copy link
Contributor

I like the changes very much. Also makes it more readable...

I'll test around .slice since I conducted some experiments and found out that v8 is already really fast there. I am curious whether there is an regression now.

@RReverser
Copy link
Member Author

v8 is already really fast there

Well, now Buffer.slice is very close to Uint8Array.subarray (I was comparing against it in my local benchmarks as a "theoretical maximum" which obviously can't be achieved in a wrapper, but can be very close (and it is now)).

Please do let me know if I missed something / need to change before this can be merged.

@@ -213,7 +214,7 @@ function allocate(size) {
// Even though this is checked above, the conditional is a safety net and
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Side note: this comment is irrelevant now, probably since dd67608, I overlooked it.
/cc @trevnorris

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like it can be removed.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like it has returned after a rebase =).
It's not critical, though, that could be removed later.

@RReverser
Copy link
Member Author

Not directly related, but now that I'm trying to submit changes from my Windows machine (previous one was from Mac), I've found one error: .eslintrc reports every line as invalid due to linebreak-style: [2, "unix"] in .eslintrc, while Git by default matches OS native endlines and checks out with CRLF (unless .gitattributes specifies custom eol for all text files, and it doesn't in this repo).

What would be the best fix for this - a PR that removes .eslintrc rule against that or a PR that adds * text=auto eol=lf or [another option]?

@addaleax
Copy link
Member

I think that discussion would best be taken to #6912 :)

@RReverser
Copy link
Member Author

Oh cool, thanks! That's a new issue I haven't noticed yet :)

@RReverser
Copy link
Member Author

Btw, can someone please explain what

function SlowBuffer(length) {
  if (+length != length)
    length = 0;

is for / supposed to do? Just tried to wrap my head around it, and wasn't sure whether the additional semantics on top of simple typeof length !== 'number' were intended or accidental.

@addaleax
Copy link
Member

There was some discussion on that in #2635… though I can’t seem to find the advantage of using +length != length, either. It behaves differently for inputs like false or '42', and probably not even in a wanted way.

@addaleax
Copy link
Member

Note that it only exists as part of a deprecated API anyway.

@RReverser
Copy link
Member Author

It behaves differently for inputs like false or '42', and probably not even in a wanted way.

Exactly my thoughts.

Note that it only exists as part of a deprecated API anyway.

That's true, just looks pretty weird when trying to read / understand the code.


// Regression test
assert.doesNotThrow(() => {
new Buffer(new ArrayBuffer());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should preferably use Buffer.from instead of new Buffer

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh ok.

@addaleax
Copy link
Member

I’d maybe separate the <= to < change, along with its regresseion test, out into its own commit that can be landed separately too.

LGTM either way.

@RReverser
Copy link
Member Author

@addaleax Isn't 9d480d9 exactly that? (well, it also removes a comment but that's a non-functional change anyway).

@addaleax
Copy link
Member

@RReverser Kinda… you don’t have to move that if you don’t want to, but keep in mind that the commit history ideally still makes sense for someone looking at it in a few years, without having the context of this PR in mind. Happens more often than you think. :)

Also, it would be cool if you could re-format the commit message for that commit so that it adheres to the guidelines (i.e. it starts with buffer:, has an all-lowercase subject line, and is under < 72 columns)

@trevnorris
Copy link
Contributor

Hm. If this PR addresses the comment removal, leave it in its own commit. Yes it's minor, but if this needs to be reverted for some unforeseen reason don't want the comment coming back in.

As far as the regression, I vote we leave that to its own PR (since it'll require it's own regression test, etc.).

@addaleax
Copy link
Member

addaleax commented Jun 3, 2016

@trevnorris Want to go ahead and land this then?

if (size <= 0)
return createBuffer(size);
if (fill !== undefined) {
if (size > 0 && fill !== undefined) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does a separate check for 0 make sense here, i.e. size > 0 && fill !== undefined && fill !== 0?

Buffer.alloc(size, 0) is equivalent to Buffer.alloc(size), so just new FastBuffer(size) should work faster in that case.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, I think that would require benchmarking – createUnsafeBuffer() returns a slice from the pool, so I’d actually expect that to be faster than an extra typed array allocation.

Copy link
Member

@ChALkeR ChALkeR Jun 3, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@addaleax It's not just createUnsafeBuffer, it's createUnsafeBuffer + fill(0).
I believe that has been discussed before, and allocation was proven to be faster — that's why simple Buffer.alloc(size) does not use the pool.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ChALkeR I know there’s the extra fill() in there. But if you say it’s faster, I believe that :)

Copy link
Member

@ChALkeR ChALkeR Jun 3, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@addaleax Btw, nothing in this function uses slices from the pool. Perhaps it should?
Both new FastBuffer and createUnsafeBuffer just directly allocate a new instance.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, right. Maybe, but I’d leave that open for another PR, too, especially as it would introduce the subtle change that the return values of Buffer.alloc() would share their buffer property.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i'm down for that change. The kernel can probably optimize calls to calloc() a hair better. Though not going to consider it a blocking change.

@ChALkeR
Copy link
Member

ChALkeR commented Jun 3, 2016

LGTM

@trevnorris
Copy link
Contributor

@addaleax Going ahead w/ these sounds good to me.

@ChALkeR
Copy link
Member

ChALkeR commented Jun 3, 2016

Curious: would

Buffer.alloc = function(size, fill, encoding) {
  assertSize(size);
  if (size > 0 && fill !== undefined && fill !== 0) {
    if (typeof encoding !== 'string')
      encoding = undefined;
    return allocate(size).fill(fill, encoding);
  }
  return new FastBuffer(size);
};

be faster for short buffers filled with some non-zero argument?

There are two changes here: && fill !== 0 (for Buffer.alloc(size, 0) opt), and createUnsafeBuffer to allocate change to use the pool for short buffers.

@ChALkeR
Copy link
Member

ChALkeR commented Jun 3, 2016

On a second thought, we can do that in a separate PR, those are independent changes.
This PR LGTM.

@addaleax
Copy link
Member

addaleax commented Jun 6, 2016

I’m going to land this later today if nobody beats me to it.

@RReverser
Copy link
Member Author

On a second thought, we can do that in a separate PR, those are independent changes.
This PR LGTM.

Yes, thought about similar further optimizations, but they would be rather backward-incompatible and cases where they give any win are more rare, so decided not to change.

addaleax pushed a commit that referenced this pull request Jun 6, 2016
Improves performance of allocating unsafe buffers, creating buffers from
an existing ArrayBuffer and creating .slice(...) from existing Buffer by
avoiding deoptimizing change of prototype after Uint8Array allocation
in favor of ES6 native subclassing.

This is done through an internal ES6 class that extends Uint8Array and
is used for allocations, but the regular Buffer function is exposed, so
calling Buffer(...) with or without `new` continues to work as usual
and prototype chains are also preserved.

Performance wins for .slice are +120% (2.2x), and, consequently, for
unsafe allocations up to +95% (1.9x) for small buffers, and for safe
allocations (zero-filled) up to +30% (1.3x).

PR-URL: #6893
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Сковорода Никита Андреевич <[email protected]>
@addaleax
Copy link
Member

addaleax commented Jun 6, 2016

Landed in 5292a13. Thanks for the contribution and for your patience with us!

@addaleax addaleax closed this Jun 6, 2016
@RReverser
Copy link
Member Author

@addaleax Thank you! That was quite a trip, but totally fine as for the first PR to the project :)

@addaleax
Copy link
Member

addaleax commented Jun 6, 2016

That was quite a trip

No arguing about that. 😄 If you like, you can also do PRs for some of the issues that popped up as side notes in the discussion here. If not, you don’t have to, of course.

… if I’ve managed to get everything right. ;)

@ChALkeR
Copy link
Member

ChALkeR commented Jun 6, 2016

@addaleax Also .alloc(size, fill) should probably use the pool since it fills manually either way — #6893 (comment).

@RReverser
Copy link
Member Author

@addaleax

The regression for creating buffers from zero-length ArrayBuffers mentioned here: #6893 (comment)

Found that commit: 7454298

Should I just submit it as a new PR?

@addaleax
Copy link
Member

addaleax commented Jun 6, 2016

@RReverser sounds good, yup :)

@trevnorris
Copy link
Contributor

@ChALkeR It was deliberate to not have Buffer.alloc() use the pool, since it was introduced to force the user to safety, and allocating from the pool allows others to read your memory. For example:

var b;
while ((b = Buffer.allocUnsafe(1)).byteOffset > 0);
Buffer.from(b.buffer).fill(0);
setTimeout(() => {
  // See what else has been written to the buffer since
  console.log(b);
}, 3000);

Can collect more information by messing with Buffer.poolSize. The argument is that allowing those allocations to come from the pool undermines the secure aspect they're focused on.

@ChALkeR
Copy link
Member

ChALkeR commented Jun 7, 2016

@trevnorris Ah, understood. I personally don't see how that is a problem, because .buffer properties are accessible only locally (i.e. not saved to the db, not transfered over network, etc), and we don't (and can't) gurantee any safety in presence of local malicious code.

But ok, let's keep it that way if there are concerns about that. Perhaps that should be documented as a small one-line comment in the source code?

@trevnorris
Copy link
Contributor

@ChALkeR That decision was simply my call when it was first implemented to make sure the PR would avoid additional scrutiny. If everyone's alright with using the pool then I won't stand in the way.

@rvagg rvagg mentioned this pull request Jun 8, 2016
5 tasks
@ChALkeR
Copy link
Member

ChALkeR commented Jun 8, 2016

@trevnorris On a second though, I think that you are correct here and that we should keep that as it is now. There could be various code errors on user side which could potentially cause issues if the code somehow uses the .buffer property.

Also, the current behaviour is documented, and changing that would be a semver-major.

So let's not change that =).

@evanlucas
Copy link
Contributor

This depends on #7082 and #7093, both of which have been marked dont-land-on-v6.x. @RReverser interested in opening a backport PR against the v6.x branch?

@RReverser
Copy link
Member Author

#7176 (comment) same question here

RReverser added a commit that referenced this pull request Jun 25, 2016
Improves performance of allocating unsafe buffers, creating buffers from
an existing ArrayBuffer and creating .slice(...) from existing Buffer by
avoiding deoptimizing change of prototype after Uint8Array allocation
in favor of ES6 native subclassing.

This is done through an internal ES6 class that extends Uint8Array and
is used for allocations, but the regular Buffer function is exposed, so
calling Buffer(...) with or without `new` continues to work as usual
and prototype chains are also preserved.

Performance wins for .slice are +120% (2.2x), and, consequently, for
unsafe allocations up to +95% (1.9x) for small buffers, and for safe
allocations (zero-filled) up to +30% (1.3x).

PR-URL: #7349
Ref: #6893
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Сковорода Никита Андреевич <[email protected]>
Reviewed-By: Trevor Norris <[email protected]>
@gibfahn gibfahn mentioned this pull request Jun 15, 2017
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
buffer Issues and PRs related to the buffer subsystem. c++ Issues and PRs that require attention from people who are familiar with C++. performance Issues and PRs related to the performance of Node.js.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants