-
-
Notifications
You must be signed in to change notification settings - Fork 910
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fixed mem issue when generating uuid #267
Conversation
It'd be nice to have a more concrete recipe for reproducing this. E.g. what diagnostic tools are you using? Specific steps to see the result? Also, is this causing problems for in-the-wild code, or is this just an academic issue? That Chromium issue has been open for 4+ years. That nobody has complained about it here before has me questioning how important this is. My apologies for being mulish, but I'm not a fan of patching code for the sake of fixing issues in upstream projects. There are a couple problems with this. One, it increases code complexity. E.g. before accepting your PR, I'll ask that you add a comment explaining why the code is structured the way it is, and referencing the above issue. Two, there are always trade offs. E.g. your PR results in the creation of a [transient] array, thus adding [admittedly trivial] GC perf hits elsewhere. But, really, this just boils down to the fact it's silly for dozens or hundreds of current and future module owners to have to inject this hack when this should be fixed in Chromium. [Leaving this open for the time being, since this is a legitimate issue and change request. 'Just not 100% sure it's worth worrying about.] [Edit: as explained by @jakobkummerow below, this appears to be an expected byproduct of how Chrome and Firefox optimize string concatenation at the expense of cases where many small strings are concatenated. Thus, I'm going to support fixing this here.] |
V8 developer here, just to quickly note that this is unrelated to https://bugs.chromium.org/p/v8/issues/detail?id=2869 (which is about substrings of long strings, not string concatenations created with The reason for the difference in memory consumption is that V8 makes string additions fast by simply creating a "cons string" that refers to the two inputs, rather than copying all the characters. Certain operations later cause such "cons strings" to be "flattened", so depending on what you use the generated UUIDs for, memory consumption may well decrease later. My opinion is that it is fine to land this patch, as the |
Sorry that's the bug for a different issue I'm working on. Here is one open bug filled that is about concatenations (in this case they ran out of memory due to the issue). Our app is using this to create our ids - many at a time, so we are noticing the difference in memory. If you would like to reproduce this I made this test app: http://plnkr.co/edit/gonu837kTr0IS8uG4SLt?p=preview |
lib/bytesToUuid.js
Outdated
bth[buf[i++]] + bth[buf[i++]] + | ||
bth[buf[i++]] + bth[buf[i++]] + | ||
bth[buf[i++]] + bth[buf[i++]]; | ||
return ([bth[buf[i++]], bth[buf[i++]], |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please add a comment here referencing https://bugs.chromium.org/p/v8/issues/detail?id=3175#c4 as the reason for using Array#join() here, then I'll merge this.
@jakobkummerow Thanks for chiming in. For the sake of others that may stumble across this, do you happen to know what other operations will cause cons strings to be flattened? (I.e. is |
For anyone else stumbling across this, it appears that some (but not all) functions that transform a string have the side-effect of flattening the underlying "cons strings". E.g. @sava-smith Knowing this, we could do |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please add that comment (noted above)
. As far as join vs toLowerCase, I'm not sure if it is more efficient or not. based on memory allocation theyre around the same. I think using tolowercase though might be a little more confusing to anyone else looking at the code. |
Thanks! This was actually a fun PR to investigate. TIL: How to kill Chrome with Strings. |
Re operations that cause flattening: I think the
There's nothing special about Strings here, 100 million of just about anything will exceed the heap limit, leading to an out-of-memory crash ;-) |
... except that bit about
|
* fixed mem issues when generating uuid * updated comment on memory issue
Currently, whenever we create an id, we are allocating 600 bytes of memory for each id.
This is because chromium has a bug that does not allow using the '+' operator to allocate memory correctly: https://bugs.chromium.org/p/v8/issues/detail?id=2869
Changing this to a join will fix the size difference:
Concatenate: Shallow Size 40, Retained Size 600
Join: Shallow Size: 64, Retained Size: 64