HTTPParser inhibits garbage collection on keep-alive connections #9668
Comments
|
@nodejs/http |
|
#9440 landed. |
|
It seems that the original comment and repro code were describing a memory leak on the server side, not on the client side. However, it seems that #9440 fixes a leak in the HTTP client's implementation. I would think that this issue is not a duplicate of #9628, and is not fixed by #9440. As a result, reopening it, my apologies if I'm missing something. |
|
I don't know if it can be related, but i'm posting here. We have tested several http libraries (request, superagent, got, node-fetch) pointing to a specific url. After running about ~20k requests in some minutes, we got this in our heapdump comparison:
While we keep sending traffic and using a request library, the memory keeps growing and growing until the app crashes. As the url we're using has a |
|
Should this remain open? |
|
I think this is still relevant. I'll take a look. |
|
I can confirm that it's still an issue but I suspect it cannot be fixed. The request object is stored in the What is wryly amusing is that it's a mitigation for resource leaks when people hold on to the request object but it evidently replaces one kind of leak with another. |
|
Node V8.2.1 |
|
Node v6.10.3 @iagomelanias node-fetch does not support keep-alive. https://github.com/bitinn/node-fetch#class-request. Wondering if this happens to even non keep-alive requets ? I see that http-parser is allocated even after load stops and after manually triggering gc on a SIGINT.
|
|
I used also chrome debug for monitoring the process memory consumption. The
HTTPParser is a memory leak for sure although a minor one. It will have
effect only after long time.
I reproduced the issue with and without keep alive connections.
Michael.
…On Nov 17, 2017 09:27, "Haaris M" ***@***.***> wrote:
Node v6.10.3
@iagomelanias <https://github.com/iagomelanias> node-fetch does not
support keep-alive. https://github.com/bitinn/node-fetch#class-request.
Wondering if this happens to even non keep-alive requets ?
I see that http-parser is allocated even after load stops and after
manually triggering gc on a SIGINT.
[image: http-parser-leak]
<https://user-images.githubusercontent.com/8192892/32935488-7333e672-cb96-11e7-8ec2-e1fa51b283a1.png>
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#9668 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/ADk6eWZ5aEk7APkabEvkYvroe3BWQtZKks5s3TVOgaJpZM4K14tN>
.
|
|
Is there any solution/idea how to solve this issue yet? Also it looks like this issue is present in: |
|
Any updates? |
|
@3axap4eHko 8.1x.x should be giving no issues. Try it |
|
Still present in node 8.11.4 |
|
This is also still a problem in 10.15.3. Does anyone have any direction for where to look to start fixing the issue? |
|
@bnoordhuis Since you looked into this … is there an obvious reason why this doesn’t work (it passes the test suite + resolves OP’s reproduction, at the very least)? --- a/lib/_http_server.js
+++ b/lib/_http_server.js
@@ -601,6 +601,7 @@ function resOnFinish(req, res, socket, state, server) {
assert(state.incoming.length === 0 || state.incoming[0] === req);
state.incoming.shift();
+ if (socket.parser) socket.parser.incoming = null;
// If the user never called req.read(), and didn't pipe() or
// .resume() or .on('data'), then we call req._dump() so that the |
addaleax
added
the
memory
addaleax
added a commit
to addaleax/node
that referenced
this issue
Jul 11, 2019
This resolves a memory leak for keep-alive connections with a naïve approach. Fixes: nodejs#9668
2 tasks
|
I'm able to reproduce HTTPParser leak using a simple express server and a client that terminates the socket with RST. I think it only happens on load. On the client side I used 20 threads with no delay between them, so we get 20 connections every second at the same time, and they all close with RST. I couldn't find a way to terminate with RST with node after receiving the response (#27428), so the client is python (adapted from the example on that issue). Reproduced for me on Windows and Linux. |
|
Now reproduced also with node only. |
targos
pushed a commit
that referenced
this issue
Aug 2, 2019
This resolves a memory leak for keep-alive connections with a naïve approach. Fixes: #9668 PR-URL: #28646 Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Rich Trott <[email protected]>
vikashk1
added a commit
to vikashk1/node
that referenced
this issue
Aug 13, 2019
* report: modify getReport() to return an Object It's likely that anyone using `process.report.getReport()` will be processing the return value thereafter (e.g., filtering fields or redacting secrets). This change eliminates boilerplate by calling `JSON.parse()` on the return value. Also modified the `validateContent()` and `validate()` test helpers in `test/common/report.js` to be somewhat more obvious and helpful. Of note, a report failing validation will now be easier (though still not _easy_) to read when prepended to the stack trace. - Refs: https://github.com/nodejs/diagnostics/issues/315 PR-URL: https://github.com/nodejs/node/pull/28630 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Jiawen Geng <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: Rich Trott <[email protected]> * src: replace already elevated Object, Local v8 namespace PR-URL: https://github.com/nodejs/node/pull/28611 Reviewed-By: Michaël Zasso <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> * src, tools: replace raw ptr with smart ptr NodeMainInstance::Create will now returrn an instance of NodeMainInstance in a unique_ptr. PR-URL: https://github.com/nodejs/node/pull/28577 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Rich Trott <[email protected]> * doc: add example on how to create __filename, __dirname for esm PR-URL: https://github.com/nodejs/node/pull/28282 Fixes: https://github.com/nodejs/node/issues/28114 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Franziska Hinkelmann <[email protected]> Reviewed-By: Michaël Zasso <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> Reviewed-By: Guy Bedford <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Rich Trott <[email protected]> * src: simplify --debug flags Any use of --debug, --debug=, --debug-brk, or --debug-brk= now triggers an error. That means we can eliminate their aliases with --inspect counterparts and simplify the code. PR-URL: https://github.com/nodejs/node/pull/28615 Reviewed-By: Michaël Zasso <[email protected]> Reviewed-By: Jeremiah Senkpiel <[email protected]> * doc: drop 'for more details' in deprecations The deprecations documentation links to the GitHub issue tracker in several places. This commit makes the text around those links consistent. PR-URL: https://github.com/nodejs/node/pull/28617 Reviewed-By: Franziska Hinkelmann <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> Reviewed-By: Jiawen Geng <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> * readline: use named constant for surrogate checks This commit defines a named constant instead of using a mix of 2 ** 16 and 0x10000 throughout the code. PR-URL: https://github.com/nodejs/node/pull/28638 Reviewed-By: Michaël Zasso <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Jeremiah Senkpiel <[email protected]> * readline: remove IIFE in SIGCONT handler This commit removes an IIFE in the readline SIGCONT handler that was previously being used to bind `this`. PR-URL: https://github.com/nodejs/node/pull/28639 Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Michaël Zasso <[email protected]> Reviewed-By: Jeremiah Senkpiel <[email protected]> * readline: simplify isFullWidthCodePoint() The non-ICU-based isFullWidthCodePoint() can be simplified to a single `return` statement. This commit removes the extra branching logic. PR-URL: https://github.com/nodejs/node/pull/28640 Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Michaël Zasso <[email protected]> Reviewed-By: Jeremiah Senkpiel <[email protected]> Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Tobias Nießen <[email protected]> * doc: remove superfluous MDN link in assert.md PR-URL: https://github.com/nodejs/node/pull/28246 Reviewed-By: Michaël Zasso <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Yongsheng Zhang <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> * readline: expose stream API in clearScreenDown() This commit adds an optional callback to clearScreenDown(), which is passed to the stream's write() method. It also exposes the return value of write(). PR-URL: https://github.com/nodejs/node/pull/28641 Reviewed-By: Michaël Zasso <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> * path: using .relative() should not return a trailing slash Resolving a path against root with `path.relative()` should not include a trailing slash. Fixes: https://github.com/nodejs/node/issues/28549 PR-URL: https://github.com/nodejs/node/pull/28556 Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Weijia Wang <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Jiawen Geng <[email protected]> Reviewed-By: Yongsheng Zhang <[email protected]> * path: move branch to the correct location This code branch only makes sense when i === length. Otherwise it'll already be handled. PR-URL: https://github.com/nodejs/node/pull/28556 Fixes: https://github.com/nodejs/node/issues/28549 Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Weijia Wang <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Jiawen Geng <[email protected]> Reviewed-By: Yongsheng Zhang <[email protected]> * doc: mark process.report as experimental Everything under process.report is experimental. This commit adds the missing stability index entries. PR-URL: https://github.com/nodejs/node/pull/28653 Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Gireesh Punathil <[email protected]> Reviewed-By: Jeremiah Senkpiel <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: Tobias Nießen <[email protected]> * doc: mark N-API thread-safe function stable The various TSFN APIs are marked as stable, but the TSFN heading itself is still marked as experimental. PR-URL: https://github.com/nodejs/node/pull/28643 Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> * zlib: do not coalesce multiple `.flush()` calls This is an approach to address the issue linked below. Previously, when `.write()` and `.flush()` calls to a zlib stream were interleaved synchronously (i.e. without waiting for these operations to finish), multiple flush calls would have been coalesced into a single flushing operation. This patch changes behaviour so that each `.flush()` all corresponds to one flushing operation on the underlying zlib resource, and the order of operations is as if the `.flush()` call were a `.write()` call. One test had to be removed because it specifically tested the previous behaviour. As a drive-by fix, this also makes sure that all flush callbacks are called. Previously, that was not the case. Fixes: https://github.com/nodejs/node/issues/28478 PR-URL: https://github.com/nodejs/node/pull/28520 Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> * src: add cleanup hook for ContextifyContext Otherwise there’s a memory leak left by the context when the Isolate tears down without having run the weak callback. PR-URL: https://github.com/nodejs/node/pull/28631 Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Jeremiah Senkpiel <[email protected]> * deps: update acorn to 6.2.0 Includes support for bigint syntax so we can remove the acorn-bigint plugin. PR-URL: https://github.com/nodejs/node/pull/28649 Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Yongsheng Zhang <[email protected]> * http2: report memory allocated by nghttp2 to V8 This helps the JS engine have a better understanding of the memory situation in HTTP/2-heavy applications, and avoids situations that behave like memory leaks due to previous underestimation of memory usage which is tied to JS objects. Refs: https://github.com/nodejs/node/issues/28088#issuecomment-509965105 PR-URL: https://github.com/nodejs/node/pull/28645 Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Ben Noordhuis <[email protected]> * tools: add coverage to ignored files This adds the coverage directory to the .gitignore file. PR-URL: https://github.com/nodejs/node/pull/28626 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Masashi Hirano <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: Сковорода Никита Андреевич <[email protected]> Reviewed-By: Michaël Zasso <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Rich Trott <[email protected]> * build,v8: support IBM i Some libraries do not exist on IBM i (OS400). Commit 417c18e introduces these missing libraries. Need to differentiate `AIX` and `OS400`(IBM i). PR-URL: https://github.com/nodejs/node/pull/28607 Reviewed-By: Beth Griggs <[email protected]> Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Gireesh Punathil <[email protected]> Reviewed-By: Michael Dawson <[email protected]> * test: fix pty test hangs on aix Some pty tests persistently hung on the AIX CI buildbots. Fix that by adding a helper script that properly sets up the pty before spawning the script under test. On investigation I discovered that the test runner hung when it tried to close the slave pty's file descriptor, probably due to a bug in AIX's pty implementation. I could reproduce it with a short C program. The test runner also leaked file descriptors to the child process. I couldn't convince python's `subprocess.Popen()` to do what I wanted it to do so I opted to move the logic to a helper script that can do fork/setsid/etc. without having to worry about stomping on state in tools/test.py. In the process I also uncovered some bugs in the pty module of the python distro that ships with macOS 10.14, leading me to reimplement a sizable chunk of the functionality of that module. And last but not least, of course there are differences between ptys on different platforms and the helper script has to paper over that. Of course. Really, this commit took me longer to put together than I care to admit. Caveat emptor: this commit takes the hacky ^D feeding to the slave out of tools/test.py and puts it in the *.in input files. You can also feed other control characters to tests, like ^C or ^Z, simply by inserting them into the corresponding input file. I think that's nice. Fixes: https://github.com/nodejs/build/issues/1820 Fixes: https://github.com/nodejs/node/issues/28489 PR-URL: https://github.com/nodejs/node/pull/28600 Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Sam Roberts <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> * stream: simplify `.pipe()` and `.unpipe()` in Readable Now we are using `pipes` and `pipesCount` in Readable state and the `pipes` value can be a stream or an array of streams. This change reducing them into one `pipes` value, which is an array of streams. PR-URL: https://github.com/nodejs/node/pull/28583 Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> * src: lint #defines in src/node.h A few #defines in src/node.h had inconsistent spacing and tabbing. This commit changes the spacing to be the same style as the rest of the project. PR-URL: https://github.com/nodejs/node/pull/28547 Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: Michaël Zasso <[email protected]> * build: remove broken intel vtune support Support for VTune profiling was added in commit a881b53 from November 2015 but has since bitrotted. Remove it. Fixes: https://github.com/nodejs/node/issues/28310 Refs: https://github.com/nodejs/node/pull/3785 PR-URL: https://github.com/nodejs/node/pull/28522 Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> * src: add missing option parser template for the DebugOptionsParser This allows embedders to run `node::options_parser::Parse` for a `node::DebugOptions`. PR-URL: https://github.com/nodejs/node/pull/28543 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Joyee Cheung <[email protected]> Reviewed-By: Eugene Ostroukhov <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> * test: increase limit for network space overhead test This test imposes a limit on the average bytes of space per chunk for network traffic. However this number depends on VM implementation details, and upcoming changes to V8's array buffer management require a small bump to this limit in this test. PR-URL: https://github.com/nodejs/node/pull/28492 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> * http: fix test where aborted should not be emitted PR-URL: https://github.com/nodejs/node/pull/20077 Fixes: https://github.com/nodejs/node/issues/20107 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Matteo Collina <[email protected]> * stream: use readableEncoding public api for child_process PR-URL: https://github.com/nodejs/node/pull/28548 Refs: https://github.com/nodejs/node/issues/445 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Rich Trott <[email protected]> * doc: add documentation for createDiffieHellmanGroup PR-URL: https://github.com/nodejs/node/pull/28585 Reviewed-By: Franziska Hinkelmann <[email protected]> Reviewed-By: Rich Trott <[email protected]> * deps: cherry-pick 13a04aba from V8 upstream Original commit message: fix: move V8_EXPORT_PRIVATE marks to prevent unresolvable references This change fixes missing symbol errors in the Windows 10 on ARM build of Node.js. When a whole class is marked for export, all of its members are marked as well. This can be a problem when inline members call undefined yet inline members of other classes: the exported function will contain a reference to the undefined inline function that should be satisfied at link time, but because the other function is inline no symbol will be produced that will satisfy that reference. Clang gets around this by masking inlined class members from export using /Fc:dllexportInlines-. This is why b0a2a567 worked. Node.js' Windows builds use MSVC and so do not have access to this flag. This results in unresolved symbols at link time. Bug: v8:9465 Change-Id: Ief9c7ab6ba35d22f995939eb62a64d6f1992ed85 Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1696771 Reviewed-by: Sigurd Schneider <[email protected]> Reviewed-by: Jakob Gruber <[email protected]> Commit-Queue: Sigurd Schneider <[email protected]> Cr-Commit-Position: refs/heads/master@{#62660} Refs: https://github.com/v8/v8/commit/13a04abacd6a15b0b06c9ad08e237af703a57dec PR-URL: https://github.com/nodejs/node/pull/28602 Reviewed-By: João Reis <[email protected]> Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: Richard Lau <[email protected]> * deps: cherry-pick 721dc7d from node-gyp upstream This change cherry-picks a small set of node-gyp v5.0.0 changes needed to enable Node.js ARM64 Windows builds. Original commit message: Add ARM64 to MSBuild /Platform logic PR-URL: https://github.com/nodejs/node-gyp/pull/1655 Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: João Reis <[email protected]> Refs: https://github.com/nodejs/node-gyp/commit/721dc7d3148ab71ee283d9cb15df84d9b87b7efc PR-URL: https://github.com/nodejs/node/pull/28604 Reviewed-By: Christian Clauss <[email protected]> Reviewed-By: João Reis <[email protected]> * deps: cherry-pick 91744bf from node-gyp upstream This change cherry-picks a small set of node-gyp v5.0.0 changes needed to enable Node.js ARM64 Windows builds. Original commit message: gyp: add support for Windows on Arm Cherry-pick of https://github.com/refack/GYP/pull/33, supersedes https://github.com/nodejs/node-gyp/pull/1678 until GYP3 is merged. `npm test` passes Change-Id: I2b1e1e03e378b4812d34afa527087793864d1576 PR-URL: https://github.com/nodejs/node-gyp/pull/1739 Reviewed-By: Refael Ackermann <[email protected]> Reviewed-By: João Reis <[email protected]> Refs: https://github.com/nodejs/node-gyp/commit/91744bfecc67fda7db58e2a1c7aa72f196d6da4f PR-URL: https://github.com/nodejs/node/pull/28604 Reviewed-By: Christian Clauss <[email protected]> Reviewed-By: João Reis <[email protected]> * Revert "http: fix test where aborted should not be emitted" This reverts commit 461bf36d701f3f7c669e2d916d5a5bc17fc447bf. PR-URL: https://github.com/nodejs/node/pull/28699 Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Matteo Collina <[email protected]> * doc: small grammar correction This commit improves the grammar of one sentence in the ESM documentation. PR-URL: https://github.com/nodejs/node/pull/28669 Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Jiawen Geng <[email protected]> Reviewed-By: Tobias Nießen <[email protected]> Reviewed-By: James M Snell <[email protected]> * inspector: do not change async call stack depth if the worker is done Fixes: https://github.com/nodejs/node/issues/28528 PR-URL: https://github.com/nodejs/node/pull/28613 Reviewed-By: Aleksei Koziatinskii <[email protected]> Reviewed-By: James M Snell <[email protected]> * doc: add missing version metadata for Readable.from Fixes: https://github.com/nodejs/node/issues/28693 PR-URL: https://github.com/nodejs/node/pull/28695 Reviewed-By: Yongsheng Zhang <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> * doc: update js-native-api example Update example that shows how to separate N-API code which is not Node.js-specific from code which defines a Node.js N-API addon. In its existing state the example uses the pattern ```C assert(napi_*() == napi_ok); ``` However, this would result in no N-API calls when building with `-DNDEBUG`. This change moves away from assert and uses a macro `NAPI_CALL()` which throws the string corresponding to the non-`napi_ok` status as a JS exception and short-circuits the binding by returning `NULL`. PR-URL: https://github.com/nodejs/node/pull/28657 Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Rich Trott <[email protected]> * doc: fix minor typo PR-URL: https://github.com/nodejs/node/pull/28148 Reviewed-By: Gireesh Punathil <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Rich Trott <[email protected]> * deps: V8: backport d2ccc59 Original commit message: [snapshot] print reference stack for JSFunctions in the isolate snapshot This helps debugging incorrect usage of the SnapshotCreator API in debug mode. Change-Id: Ibd9db76a5f460cdf7ea6d14e865592ebaf69aeef Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1648240 Reviewed-by: Yang Guo <[email protected]> Commit-Queue: Yang Guo <[email protected]> Cr-Commit-Position: refs/heads/master@{#62095} Refs: https://github.com/v8/v8/commit/d2ccc599c7a31838752350ae927e41bc386df414 PR-URL: https://github.com/nodejs/node/pull/28648 Reviewed-By: Michaël Zasso <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: James M Snell <[email protected]> * src: large pages option: FreeBSD support proposal Enabling on amd64 and as Linux, are 2MB large. The ELF section linkage script is compatible only with GNU ld. PR-URL: https://github.com/nodejs/node/pull/28331 Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Rich Trott <[email protected]> * module: increase code coverage of cjs loader Add test cases to cover uncovered wrap and wrapper getters. Refs: https://coverage.nodejs.org/coverage-99268b1e996d13a0/lib/internal/modules/cjs/loader.js.html#L153 PR-URL: https://github.com/nodejs/node/pull/27898 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Jan Krems <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Rich Trott <[email protected]> * test: use openssl_is_fips instead of hasFipsCrypto Currently, when dynamically linking against a FIPS enabled OpenSSL library test-process-env-allowed-flags-are-documented will fail with the following error: assert.js:89 throw new AssertionError(obj); ^ AssertionError [ERR_ASSERTION]: The following options are not documented as allowed in NODE_OPTIONS in /root/node/doc/api/cli.md: --enable-fips --force-fips at Object.<anonymous> (/test/parallel/test-process-env-allowed-flags-are-documented.js:82:8) at Module._compile (internal/modules/cjs/loader.js:779:30) at Object.Module._extensions..js (internal/modules/cjs/loader.js:790:10) at Module.load (internal/modules/cjs/loader.js:642:32) at Function.Module._load (internal/modules/cjs/loader.js:555:12) at Function.Module.runMain (internal/modules/cjs/loader.js:842:10) at internal/main/run_main_module.js:17:11 { generatedMessage: false, code: 'ERR_ASSERTION', actual: 2, expected: 0, operator: 'strictEqual' } This commit updates the test to use process.config.variables.openssl_is_fips instead of common.hasFipsCrypto as hasFipsCrypto only returns true if the OpenSSL library that is shipped with node was configured with FIPS enabled. PR-URL: https://github.com/nodejs/node/pull/28507 Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> * test: update hasFipsCrypto in test/common/README PR-URL: https://github.com/nodejs/node/pull/28507 Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> * http: expose headers on an http.ClientRequest "information" event 1xx intermediate status responses are allowed to have headers; so expose the "httpVersion", "httpVersionMajor", "httpVersionMinor", "headers", "rawHeaders", and "statusMessage" properties on this event. PR-URL: https://github.com/nodejs/node/pull/28459 Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Rich Trott <[email protected]> * readline: expose stream API in clearLine() This commit adds an optional callback to clearLine(), which is passed to the stream's write() method. It also exposes the return value of write(). PR-URL: https://github.com/nodejs/node/pull/28674 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> * readline: expose stream API in moveCursor() This commit adds an optional callback to moveCursor(), which is passed to the stream's write() method. It also exposes the return value of write(). PR-URL: https://github.com/nodejs/node/pull/28674 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> * readline: expose stream API in cursorTo() This commit adds an optional callback to cursorTo(), which is passed to the stream's write() method. It also exposes the return value of write(). PR-URL: https://github.com/nodejs/node/pull/28674 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> * http: add response.writableFinished response.writableFinished is true if all data has been flushed to the underlying system. PR-URL: https://github.com/nodejs/node/pull/28681 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> Reviewed-By: Rich Trott <[email protected]> * gyp: cherrypick more Python3 changes from node-gyp PR-URL: https://github.com/nodejs/node/pull/28563 Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: Rich Trott <[email protected]> * n-api: correct bug in napi_get_last_error napi_get_last_error returns incorrect napi_status. PR-URL: https://github.com/nodejs/node/pull/28702 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Gabriel Schulhof <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Michael Dawson <[email protected]> * build: specify Python version once for all tests Extracted from #28537 for shorter review cycle. This makes it easier to experiment with new versions of Python as they become available on the Travis CI platform. PR-URL: https://github.com/nodejs/node/pull/28694 Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> * test: improve variable names in pty_helper.py Using names like `parent_fd` and `child_fd` is more accurate here, and doesn’t come with unnecessary negative connotations, even if the previous naming is somewhat common terminology here. PR-URL: https://github.com/nodejs/node/pull/28688 Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> * test: fix race condition in test-worker-process-cwd.js This simplifies the test logic and fixes the race condition that could happen right now. PR-URL: https://github.com/nodejs/node/pull/28609 Refs: https://github.com/nodejs/node/issues/28193 Closes: https://github.com/nodejs/node/pull/28477 Fixes: https://github.com/nodejs/node/issues/27669 Fixes: https://github.com/nodejs/node/issues/28477 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Rich Trott <[email protected]> * test: make repl tests more resilient This refactors two tests to ignore line numbers in stack traces. That way changed line numbers do not have any impact on the test outcome anymore. PR-URL: https://github.com/nodejs/node/pull/28608 Fixes: https://github.com/nodejs/node/issues/28546 Reviewed-By: Jiawen Geng <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Lance Ball <[email protected]> Reviewed-By: Anto Aravinth <[email protected]> * repl: fix autocomplete while using .load This makes sure that complete functions work as expected after using the REPL's `.load` command. It also fixes the corresponding test. So far the assertion where swallowed and the test passed even though it should not have. Fixes: https://github.com/nodejs/node/issues/28546 PR-URL: https://github.com/nodejs/node/pull/28608 Reviewed-By: Jiawen Geng <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Lance Ball <[email protected]> Reviewed-By: Anto Aravinth <[email protected]> * repl: fix some repl context issues This partially fixes contexts like `{} instanceof Object === false` in the REPL. This does not fix all cases, since it's something fundamental from the REPL's design that things like these can happen. Refs: https://github.com/nodejs/node/issues/27859 PR-URL: https://github.com/nodejs/node/pull/28561 Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Anto Aravinth <[email protected]> Reviewed-By: James M Snell <[email protected]> * doc: add examples at assert.strictEqual PR-URL: https://github.com/nodejs/node/pull/28092 Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Rich Trott <[email protected]> * doc: improve os.homedir() docs PR-URL: https://github.com/nodejs/node/pull/28401 Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: James M Snell <[email protected]> * doc: add example for zlib.createGzip() PR-URL: https://github.com/nodejs/node/pull/28136 Reviewed-By: Sam Roberts <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: Masashi Hirano <[email protected]> Reviewed-By: James M Snell <[email protected]> * doc: add example for beforeExit event PR-URL: https://github.com/nodejs/node/pull/28430 Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> * test: use consistent test naming To conform with other test names, move test/async-hooks/test-httparser-reuse.js to test/async-hooks/test-httpparser-reuse.js. PR-URL: https://github.com/nodejs/node/pull/28744 Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> * test: propagate napi_status to JS Re: https://github.com/nodejs/node/pull/27945#discussion_r288833979 This commit regards reporting to the JS level an actual event that happens when using suspected improper null arguments. It is better to report the exact reason from N-API to the JS level. PR-URL: https://github.com/nodejs/node/pull/28505 Reviewed-By: Gabriel Schulhof <[email protected]> Reviewed-By: Michael Dawson <[email protected]> Reviewed-By: Rich Trott <[email protected]> * tty: expose stream API from readline methods This commit exposes the return value and callback of the underlying readline APIs from the tty module. PR-URL: https://github.com/nodejs/node/pull/28721 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> * test,win: cleanup exec-timeout processes When CMD is used to launch a process and CMD is killed too quickly, the process can stay behind running in suspended state, never completing. This only happens in Windows Server 2008R2. Refs: https://github.com/nodejs/build/issues/1829 PR-URL: https://github.com/nodejs/node/pull/28723 Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Beth Griggs <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> * esm: implement "pkg-exports" proposal Refs: https://github.com/jkrems/proposal-pkg-exports/issues/36 PR-URL: https://github.com/nodejs/node/pull/28568 Reviewed-By: Anna Henningsen <[email protected]> * deps: upgrade npm to 6.10.0 PR-URL: https://github.com/nodejs/node/pull/28525 Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Roman Reiss <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> Reviewed-By: Yongsheng Zhang <[email protected]> Reviewed-By: Jiawen Geng <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: Daijiro Wachi <[email protected]> * http: avoid extra listener PR-URL: https://github.com/nodejs/node/pull/28705 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> Reviewed-By: Fedor Indutny <[email protected]> * vm: remove usage of public util module PR-URL: https://github.com/nodejs/node/pull/28460 Reviewed-By: Anatoli Papirovski <[email protected]> Reviewed-By: Masashi Hirano <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Yongsheng Zhang <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> * zlib: remove usage of public util module PR-URL: https://github.com/nodejs/node/pull/28454 Reviewed-By: Anatoli Papirovski <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: James M Snell <[email protected]> * build: fix building with d8 PR-URL: https://github.com/nodejs/node/pull/28733 Reviewed-By: Gus Caplan <[email protected]> Reviewed-By: Joyee Cheung <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Rich Trott <[email protected]> * test: changed function to arrow function Convert callback functions that are anonymous to arrow functions for better readability. Also adjusted the `this` object in places where that was required. PR-URL: https://github.com/nodejs/node/pull/28726 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Masashi Hirano <[email protected]> Reviewed-By: Beth Griggs <[email protected]> Reviewed-By: Gireesh Punathil <[email protected]> Reviewed-By: Rich Trott <[email protected]> * build: update of the large page option error Now large pages is also supported by FreeBSD. PR-URL: https://github.com/nodejs/node/pull/28729 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Rich Trott <[email protected]> * deps: V8: backport b33af60 Original commit message: [api] Get ScriptOrModule from CompileFunctionInContext Adds a new out param which allows accessing the ScriptOrModule of a function, which allows an embedder such as Node.js to use the function's i::Script lifetime. Refs: https://github.com/nodejs/node-v8/issues/111 Change-Id: I34346d94d76e8f9b8377c97d948673f4b95eb9d5 Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1699698 Reviewed-by: Yang Guo <[email protected]> Commit-Queue: Yang Guo <[email protected]> Cr-Commit-Position: refs/heads/master@{#62830} Refs: https://github.com/v8/v8/commit/b33af60dd9e7e5b2557b9fbf3fdb80209f6db844 PR-URL: https://github.com/nodejs/node/pull/28671 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Guy Bedford <[email protected]> * vm: fix gc bug with modules and compiled functions PR-URL: https://github.com/nodejs/node/pull/28671 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Guy Bedford <[email protected]> * build: skip test-ci doc targets if no crypto PR-URL: https://github.com/nodejs/node/pull/28747 Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Rich Trott <[email protected]> * lib: rename lib/internal/readline.js This commit moves lib/internal/readline.js to lib/internal/readline/utils.js. This is in preparation of adding a readline.promises implementation. PR-URL: https://github.com/nodejs/node/pull/28753 Reviewed-By: Michaël Zasso <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Anto Aravinth <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> Reviewed-By: Shingo Inoue <[email protected]> * doc: amplify warning for execute callback Add specific recommendation not to use the to the napi-env parameter in napi_async_execute_callback PR-URL: https://github.com/nodejs/node/pull/28738 Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Rich Trott <[email protected]> * doc: add information for heap snapshot flag It's nice to have usage examples, especially since the flag requires the `SIG` version of the signal name, unlike `kill`. PR-URL: https://github.com/nodejs/node/pull/28754 Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> * doc: add code example to subprocess.stdout PR-URL: https://github.com/nodejs/node/pull/28402 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> Reviewed-By: Rich Trott <[email protected]> * policy: add policy-integrity to mitigate policy tampering PR-URL: https://github.com/nodejs/node/pull/28734 Reviewed-By: Gus Caplan <[email protected]> Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Guy Bedford <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Rich Trott <[email protected]> * inspector: do not spin-wait while waiting for the initial connection Fixes: https://github.com/nodejs/node/issues/28741 PR-URL: https://github.com/nodejs/node/pull/28756 Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: Aleksei Koziatinskii <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Anto Aravinth <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> * src: add public virtual destructor for KVStore As KVStore has derived classes, it is essential to declare a public virtual destructor in the base KVStore class. Otherwise, deleting derived class instances using base class pointers would potentially cause undefined behaviour. Additionally, since we are implementing a non-default destructor, the special member functions have also been implemented in order to abide by the rule of five. PR-URL: https://github.com/nodejs/node/pull/28737 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Rich Trott <[email protected]> * http2: compat req.complete PR-URL: https://github.com/nodejs/node/pull/28627 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Rich Trott <[email protected]> * inspector: add inspector.waitForDebugger() This method blocks current node process until a client sends Runtime.runifWaitingForDebugger. It can be useful when we need to report inspector.url() before waiting for connection: ``` inspector.open(0, undefined, false); fs.writeFileSync(someFileName, inspector.url()); inspector.waitForDebugger(); ``` PR-URL: https://github.com/nodejs/node/pull/28453 Reviewed-By: Eugene Ostroukhov <[email protected]> Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Rich Trott <[email protected]> * stream: add null push transform in async_iterator when the readable side of a transform ends any for await loop on that transform stream should also complete. This fix prevents for await loop on a transform stream from hanging indefinitely. PR-URL: https://github.com/nodejs/node/pull/28566 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Jeremiah Senkpiel <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Rich Trott <[email protected]> * test: fix assertion argument order in test-esm-namespace PR-URL: https://github.com/nodejs/node/pull/28474 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> Reviewed-By: Rich Trott <[email protected]> * src: expose TraceEventHelper with NODE_EXTERN As node requires a tracing controller to be initialized embedders need access to the TraceEventHelper so that we can actually set the tracing controller. Refs: https://github.com/electron/electron/commit/0e5b6f93000e4718c9e35332ddbd0f6b76cdd585/#diff-89b287b2edd0a02dddae60cb26157f47 PR-URL: https://github.com/nodejs/node/pull/28724 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Rich Trott <[email protected]> * deps: update nghttp2 to 1.39.1 PR-URL: https://github.com/nodejs/node/pull/28448 Reviewed-By: Trivikram Kamat <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> * doc: claim NODE_MODULE_VERSION=75 for Electron 7 PR-URL: https://github.com/nodejs/node/pull/28774 Reviewed-By: Michaël Zasso <[email protected]> Reviewed-By: Yongsheng Zhang <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Jiawen Geng <[email protected]> * src: silence compiler warning This commit fixes the following warning: warning: missing field 'exports' initializer PR-URL: https://github.com/nodejs/node/pull/28764 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Yongsheng Zhang <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Jiawen Geng <[email protected]> * doc: update env default on child_process functions PR-URL: https://github.com/nodejs/node/pull/28776 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Rich Trott <[email protected]> * tools: remove unused pkgsrc directory The pkgsrc Makefile target was removed in 2015 Refs: https://github.com/nodejs/node/pull/1938 PR-URL: https://github.com/nodejs/node/pull/28783 Reviewed-By: Jiawen Geng <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: Rich Trott <[email protected]> * src: make `CompiledFnEntry` a `BaseObject` In particular: - Move the class definition to the relevant header file, i.e. `node_contextify.h`. - Make sure that class instances are destroyed on `Environment` teardown. - Make instances of the key object traceable in heap dumps. This is particularly relevant here because our C++ script → map key mapping could introduce memory leaks when the import function metadata refers back to the script in some way. Refs: https://github.com/nodejs/node/pull/28671 PR-URL: https://github.com/nodejs/node/pull/28782 Reviewed-By: Gus Caplan <[email protected]> Reviewed-By: Rich Trott <[email protected]> * src: do not include partial AsyncWrap instances in heap dump Heap dumps can be taken either through the inspector or the public API for it during an async_hooks init() hook, but at that point the AsyncWrap in question is not done initializing yet and virtual methods cannot be called on it. Address this issue (somewhat hackily) by excluding `AsyncWrap` instances which have not yet executed their `init()` hook fully from heap dumps. Fixes: https://github.com/nodejs/node/issues/28786 PR-URL: https://github.com/nodejs/node/pull/28789 Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: Rich Trott <[email protected]> * tools: update ESLint to 6.1.0 Update ESLint to 6.1.0 PR-URL: https://github.com/nodejs/node/pull/28793 Reviewed-By: Yongsheng Zhang <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> Reviewed-By: Jiawen Geng <[email protected]> * dns: fix unsigned record values Fixes: https://github.com/nodejs/node/issues/28790 PR-URL: https://github.com/nodejs/node/pull/28792 Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Ben Noordhuis <[email protected]> * deps: float 15d7e79 from openssl The upstream commit fixes an incorrect initialization of memory in rand_lib.c. This fixes all errors that are reported by valgrind during startup. Origin: https://github.com/openssl/openssl/commit/15d7e7997e219fc PR-URL: https://github.com/nodejs/node/pull/28796 Fixes: https://github.com/nodejs/node/issues/28739 Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Michaël Zasso <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Shigeki Ohtsu <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: Sam Roberts <[email protected]> * src: fix type name in comment The comment refers to an exception type that JS land throws on the C++ code's behalf but apparently I changed the JS name before landing the pull request and forgot to update the comment. Refs: https://github.com/nodejs/node/pull/20816 PR-URL: https://github.com/nodejs/node/pull/28320 Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Daniel Bevenius <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: Anto Aravinth <[email protected]> * 2019-07-23, Version 12.7.0 (Current) Notable changes: * deps: * Updated nghttp2 to 1.39.1. https://github.com/nodejs/node/pull/28448 * Updated npm to 6.10.0. https://github.com/nodejs/node/pull/28525 * esm: * Implemented experimental "pkg-exports" proposal. A new `"exports"` field can be added to a module's `package.json` file to provide custom subpath aliasing. See https://github.com/jkrems/proposal-pkg-exports/ for more information. https://github.com/nodejs/node/pull/28568 * http: * Added `response.writableFinished`. https://github.com/nodejs/node/pull/28681 * Exposed `headers`, `rawHeaders` and other fields on an `http.ClientRequest` `"information"` event. https://github.com/nodejs/node/pull/28459 * inspector: * Added `inspector.waitForDebugger()`. https://github.com/nodejs/node/pull/28453 * policy: * Added `--policy-integrity=sri` CLI option to mitigate policy tampering. If a policy integrity is specified and the policy does not have that integrity, Node.js will error prior to running any code. https://github.com/nodejs/node/pull/28734 * readline,tty: * Exposed stream API from various methods which write characters. https://github.com/nodejs/node/pull/28674 https://github.com/nodejs/node/pull/28721 * src: * Use cgroups to get memory limits. This improves the way we set the memory ceiling for a Node.js process. Previously we would use the physical memory size to estimate the necessary V8 heap sizes. The physical memory size is not necessarily the correct limit, e.g. if the process is running inside a docker container or is otherwise constrained. This change adds the ability to get a memory limit set by linux cgroups, which is used by docker containers to set resource constraints. https://docs.docker.com/config/containers/resource_constraints/ https://github.com/nodejs/node/pull/27508 PR-URL: https://github.com/nodejs/node/pull/28817 * lib: support min/max values in validateInteger() This commit updates validateInteger() in two ways: - Number.isInteger() is used instead of Number.isSafeInteger(). This ensures that all integer values are supported. - Minimum and maximum values are supported. They default to the min and max safe integer values, but can be customized. PR-URL: https://github.com/nodejs/node/pull/28810 Reviewed-By: Tobias Nießen <[email protected]> Reviewed-By: Michaël Zasso <[email protected]> Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> * module: implement "exports" proposal for CommonJS Refs: https://github.com/jkrems/proposal-pkg-exports/issues/36 Refs: https://github.com/nodejs/node/pull/28568 PR-URL: https://github.com/nodejs/node/pull/28759 Reviewed-By: Guy Bedford <[email protected]> Reviewed-By: Bradley Farias <[email protected]> * doc: api/stream.md typo from writeable to writable PR-URL: https://github.com/nodejs/node/pull/28822 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> Reviewed-By: Richard Lau <[email protected]> * crypto: increase maxmem range from 32 to 53 bits Fixes: https://github.com/nodejs/node/issues/28755 PR-URL: https://github.com/nodejs/node/pull/28799 Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Rich Trott <[email protected]> * tools: update certdata.txt This is the certdata.txt[0] from NSS 3.45, released on 2019-07-05. This is the version of NSS that will ship in Firefox 69 on 2019-09-03. [0] https://hg.mozilla.org/projects/nss/raw-file/NSS_3_45_RTM/lib/ckfw/builtins/certdata.txt PR-URL: https://github.com/nodejs/node/pull/28808 Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> * crypto: update root certificates Update the list of root certificates in src/node_root_certs.h with tools/mk-ca-bundle.pl. Certificates added: (none) Certificates removed: - Certinomis - Root CA PR-URL: https://github.com/nodejs/node/pull/28808 Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> * doc: fix type in NSS update instructions The perl script must be fully named, correct so that the command can be copy-pasted-run from the docs. PR-URL: https://github.com/nodejs/node/pull/28808 Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> * build: `uname -m` is amd64 on freebsd, not x86_64 Fixes: https://github.com/nodejs/node/issues/13150 PR-URL: https://github.com/nodejs/node/pull/28804 Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Rich Trott <[email protected]> * src : elevate v8 namespaces Leverage `using` semantics for repeated usage of v8 artifacts. PR-URL: https://github.com/nodejs/node/pull/28801 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Michaël Zasso <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Gireesh Punathil <[email protected]> * doc: add documentation for response.flushHeaders() PR-URL: https://github.com/nodejs/node/pull/28807 Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Rich Trott <[email protected]> * doc: claim NODE_MODULE_VERSION=76 for Electron 8 PR-URL: https://github.com/nodejs/node/pull/28809 Refs: https://github.com/electron/electron/projects/20#card-24099810 Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Michaël Zasso <[email protected]> Reviewed-By: Michael Dawson <[email protected]> * deps: backport b107214 from upstream V8 Original commit message: [code-serializer] Handlify in CodeSerializer::Deserialize This section potentially contains allocations and thus gc, all object references should be handlified. Bug: v8:9333 Change-Id: I5814e66e8b9b75a8bd952afecae7a3a27b42a642 Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1647695 Auto-Submit: Jakob Gruber <[email protected]> Commit-Queue: Simon Zünd <[email protected]> Reviewed-by: Simon Zünd <[email protected]> Cr-Commit-Position: refs/heads/master@{#62012} (This required resolution of a few merge conflicts, so it’s essentially a manual backport.) Refs: https://github.com/v8/v8/commit/b10721426503b87d013ecf314ca139fa5334ebb7 Refs: https://github.com/nodejs/node/pull/28847 PR-URL: https://github.com/nodejs/node/pull/28850 Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Jan Krems <[email protected]> * domain: use strong reference to domain while active When an uncaught exception is thrown inside a domain, the domain is removed from the stack as of 43a51708589ac789ce08beaeb49d6d778dfbdc49. This means that it might not be kept alive as an object anymore, and may be garbage collected before the `after()` hook can run, which tries to exit it as well. Resolve that by making references to the domain strong while it is active. Fixes: https://github.com/nodejs/node/issues/28275 PR-URL: https://github.com/nodejs/node/pull/28313 Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: Vladimir de Turckheim <[email protected]> Reviewed-By: Rich Trott <[email protected]> * build: re-enable openssl arm for arm64 In #23913 it looked like arm64 testing was flaky, and as a result a number of changes were made including disabling openssl asm for that architecture. This PR is limited to re-enabling the one change to turn on openssl asm for arm64, in the hopes that either it works now, or that this specific change introduces a reproducible condition. See: https://github.com/nodejs/node/pull/23913 PR-URL: https://github.com/nodejs/node/pull/28180 Reviewed-By: Rod Vagg <[email protected]> Reviewed-By: Sam Roberts <[email protected]> Reviewed-By: Richard Lau <[email protected]> Reviewed-By: James M Snell <[email protected]> * doc: describe why new Buffer() is problematic Existing docs weren't clear on the actual problem. In addition, the text described 8.0.0 as being a future Node.js release, so adjust language to reflect that 8.0.0 is in the past (while not losing important information about what the pre-8.x behaviour was). PR-URL: https://github.com/nodejs/node/pull/28825 Reviewed-By: Michael Dawson <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Сковорода Никита Андреевич <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Jeremiah Senkpiel <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> * n-api: add APIs for per-instance state management Adds `napi_set_instance_data()` and `napi_get_instance_data()`, which allow native addons to store their data on and retrieve their data from `napi_env`. `napi_set_instance_data()` accepts a finalizer which is called when the `node::Environment()` is destroyed. This entails rendering the `napi_env` local to each add-on. Fixes: https://github.com/nodejs/abi-stable-node/issues/378 PR-URL: https://github.com/nodejs/node/pull/28682 Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: Michael Dawson <[email protected]> * doc: fix incorrect name in report docs In diagnostic reports, the CPUs are listed in a "cpus" field. This commit fixes the docs, which refer to the field as "osCpus" PR-URL: https://github.com/nodejs/node/pull/28830 Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Anto Aravinth <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> * report: loop over uv_cpu_info() results The code currently loops over the results, but only the first result is accessed. PR-URL: https://github.com/nodejs/node/pull/28829 Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Rich Trott <[email protected]> * assert: avoid potentially misleading reference to object identity Often, the word “identical” when referring to JS objects will be read as referring to having the same object identity (which is called “reference equality” here), but what the error message is trying to say here is that the objects are different but yield the same `util.inspect()` output. Since `util.inspect()` output represents the structure rather than the identity of objects, (hopefully) clarify the error message to reflect that. PR-URL: https://github.com/nodejs/node/pull/28824 Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> * crypto: add outputLength option to crypto.createHash This change adds an outputLength option to crypto.createHash which allows users to produce variable-length hash values using XOF hash functons. Fixes: https://github.com/nodejs/node/issues/28757 PR-URL: https://github.com/nodejs/node/pull/28805 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Sam Roberts <[email protected]> Reviewed-By: Rich Trott <[email protected]> * crypto: fix handling of malicious getters (scrypt) It is possible to bypass parameter validation in crypto.scrypt and crypto.scryptSync by crafting option objects with malicious getters as demonstrated in the regression test. After bypassing validation, any value can be passed to the C++ layer, causing an assertion to crash the process. Fixes: https://github.com/nodejs/node/issues/28836 PR-URL: https://github.com/nodejs/node/pull/28838 Reviewed-By: Michaël Zasso <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: Sam Roberts <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> * policy: add dependencies map for resources Adds a "dependencies" field to resources in policy manifest files. In order to ease development and testing while using manifests, wildcard values for both "dependencies" and "integrity" have been added using the boolean value "true" in the policy manifest. PR-URL: https://github.com/nodejs/node/pull/28767 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Jan Krems <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Rich Trott <[email protected]> * doc: add documentation for stream.destroyed PR-URL: https://github.com/nodejs/node/pull/28815 Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Matteo Collina <[email protected]> * n-api: refactor a previous commit This is a refactoring of https://github.com/nodejs/node/issues/27628 following https://github.com/nodejs/node/pull/28505. This change factors out functions `add_last_status()` and `add_returned_status()` so they may be reused in the tests for passing information about the last error status and/or a returned `napi_status` to JavaScript. PR-URL: https://github.com/nodejs/node/pull/28848 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Gabriel Schulhof <[email protected]> * stream: resolve perf regression introduced by V8 7.3 This commit contains two fixes: 1. use instanceof instead of Object.getPrototypeOf, as checking an object prototype with Object.getPrototypeOf is slower than an instanceof check. 2. avoid parseInt(undefined, 10) to get NaN as it regressed. PR-URL: https://github.com/nodejs/node/pull/28842 Fixes: https://github.com/nodejs/node/issues/28586 Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Michaël Zasso <[email protected]> Reviewed-By: Anto Aravinth <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Trivikram Kamat <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Rich Trott <[email protected]> * build,tools: support building with Visual Studio 2019 Add a `vs2019` option to `vcbuild.bat` to use VS 2019 instead of VS 2017 PR-URL: https://github.com/nodejs/node/pull/28781 Reviewed-By: João Reis <[email protected]> Reviewed-By: Refael Ackermann (רפאל פלחי) <[email protected]> * test: specialize OOM check for AIX Assumption that if memory can be malloc()ed it can be used is not true on AIX. Later access of the allocated pages can trigger SIGKILL if there are insufficient VM pages. Use psdanger() to better estimate available memory. Fixes: https://github.com/nodejs/build/issues/1849 More info: - https://www.ibm.com/support/knowledgecenter/en/ssw_aix_71/generalprogramming/sys_mem_alloc.html - https://www.ibm.com/support/knowledgecenter/en/ssw_aix_71/p_bostechref/psdanger.html Related to: - https://github.com/nodejs/build/issues/1820#issuecomment-505998851 - https://github.com/nodejs/node/pull/28469 - https://github.com/nodejs/node/pull/28516 PR-URL: https://github.com/nodejs/node/pull/28857 Reviewed-By: Michael Dawson <[email protected]> Reviewed-By: Beth Griggs <[email protected]> Reviewed-By: Rich Trott <[email protected]> * src: move relative uptime init PR-URL: https://github.com/nodejs/node/pull/28849 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> * http: reset parser.incoming when server response is finished This resolves a memory leak for keep-alive connections with a naïve approach. Fixes: https://github.com/nodejs/node/issues/9668 PR-URL: https://github.com/nodejs/node/pull/28646 Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Rich Trott <[email protected]> * deps: remove backup files PR-URL: https://github.com/nodejs/node/pull/28865 Reviewed-By: Richard Lau <riclau@uk.…
3 tasks
|
Reopening as the fix was reverted. |
addaleax
added a commit
to addaleax/node
that referenced
this issue
Aug 24, 2019
This resolves a memory leak for keep-alive connections and does not regress in the way that 779a05d did by waiting for the incoming request to be finished before releasing the `parser.incoming` object. Refs: nodejs#28646 Refs: nodejs#29263 Fixes: nodejs#9668
3 tasks
targos
pushed a commit
that referenced
this issue
Aug 26, 2019
This resolves a memory leak for keep-alive connections and does not regress in the way that 779a05d did by waiting for the incoming request to be finished before releasing the `parser.incoming` object. Refs: #28646 Refs: #29263 Fixes: #9668 PR-URL: #29297 Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Matteo Collina <[email protected]>
|
Not sure if this fix has been backported to Node 12.x too |
|
@paolomainardi It has been released in v12.9.1. |
|
Thanks @addaleax |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I'm not too familiar with node internals, so please correct any details that I may have gotten wrong.
HTTPParser objects appear to be created on a per-socket basis, and retain an
incomingreference to an IncomingMessage. Theincomingreference is kept around even after the message has been completely parsed and handled, making it un-gc-able for as long as the underlying socket remains open.At Twitter, we've found that this can contribute to high memory usage when:
In this environment, already-handled messages are unable to be garbage collected until either a new request comes in on the existing socket or the connection is closed.
I put together a small demo of this issue at jbellenger/node-message-retention
The text was updated successfully, but these errors were encountered: