Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

http: emit close on socket re-use #28685

Closed
wants to merge 3 commits into from

Conversation

ronag
Copy link
Member

@ronag ronag commented Jul 14, 2019

When using agents we currently don't emit 'close' on req & res.

I will need a little help here... I don't think I quite got it right.

Refs: #28684

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • documentation is changed or added
  • commit message follows commit guidelines

@nodejs-github-bot nodejs-github-bot added the http Issues or PRs related to the http subsystem. label Jul 14, 2019
@ronag
Copy link
Member Author

ronag commented Jul 14, 2019

@benjamingr I think I'm getting close to spamming issues and PR's now... I'll stop here for now until some of the other's get merged or closed. If there anyway I can help speed up the process feel free to let me know. Matteo has my mail if you want to setup any phone chat.

@benjamingr
Copy link
Member

@benjamingr I think I'm getting close to spamming issues and PR's now... I'll stop here for now until some of the other's get merged or closed.

You are not spamming PRs or issues everything here has substance, it is getting slow uptake because there aren't many people who understand this area in the code not because the changes are bad. Please keep making PRs :]

lib/_http_client.js Outdated Show resolved Hide resolved
@jasnell jasnell added the semver-major PRs that contain breaking changes and should be released in the next major version. label Jul 15, 2019
@jasnell
Copy link
Member

jasnell commented Jul 15, 2019

Defensively marking as semver-major as this has a non-zero chance of breaking at least someone.

@ronag
Copy link
Member Author

ronag commented Jul 15, 2019

tests added and bugs fixed

@ronag ronag force-pushed the fix-req-res-agent-close branch 4 times, most recently from 286fc50 to 15f46b5 Compare July 16, 2019 07:48
Copy link
Member

@mcollina mcollina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@nodejs-github-bot
Copy link
Collaborator

@Trott
Copy link
Member

Trott commented Aug 4, 2019

Needs a CITGM run but CITGM is kinda-sorta blocked by standard-things/esm#821 right now (as that issue causes a number of failures in CITGM, or at least that's my understanding).

@ronag ronag mentioned this pull request Aug 6, 2019
9 tasks
@ronag
Copy link
Member Author

ronag commented Sep 4, 2019

@Trott: can we get that CITGM on this one?

@nodejs-github-bot
Copy link
Collaborator

@Trott
Copy link
Member

Trott commented Sep 5, 2019

@ronag
Copy link
Member Author

ronag commented Sep 11, 2019

@Trott can we land this?

@mcollina
Copy link
Member

We can't, this is breaking Ember:

 Command failed: yarn install --non-interactive
 warning You are using Node "13.0.0-pre" which is not supported and may encounter bugs or unexpected behavior. Yarn supports the following semver range: "^4.8.0 || ^5.7.0 || ^6.2.2 || >=8.0.0"
 warning package.json: No license field
 warning foo: No license field
 events.js:484
   const copy = new Array(n);
                ^
 RangeError: Maximum call stack size exceeded
     at arrayClone (events.js:484:16)
     at IncomingMessage.emit (events.js:212:23)
     at responseKeepAlive (_http_client.js:613:13)
     at IncomingMessage.responseOnEnd (_http_client.js:645:5)
     at IncomingMessage.emit (events.js:214:15)
     at IncomingMessage.<anonymous> (/home/iojs/build/workspace/citgm-smoker/nodes/fedora-last-latest-x64/smoker/lib/node_modules/citgm/node_modules/yarn/lib/cli.js:141275:23)
     at IncomingMessage.emit (events.js:214:15)
     at responseKeepAlive (_http_client.js:613:13)
     at IncomingMessage.responseOnEnd (_http_client.js:645:5)
     at IncomingMessage.emit (events.js:214:15)
 yarn install v1.17.3
 [1/4] Resolving packages...
 Error: Command failed: yarn install --non-interactive
 warning You are using Node "13.0.0-pre" which is not supported and may encounter bugs or unexpected behavior. Yarn supports the following semver range: "^4.8.0 || ^5.7.0 || ^6.2.2 || >=8.0.0"
 warning package.json: No license field
 warning foo: No license field

https://ci.nodejs.org/view/Node.js-citgm/job/citgm-smoker/1995/nodes=fedora-last-latest-x64/testReport/junit/(root)/citgm/ember_cli_v3_12_0/

@ronag
Copy link
Member Author

ronag commented Sep 11, 2019

Ember? Interesting. Would you like me to take a look?

@mcollina
Copy link
Member

mcollina commented Sep 11, 2019 via email

@ronag
Copy link
Member Author

ronag commented Sep 11, 2019

It's actually yarn that fails here:

    // Be a good stream and emit end when the response is finished.
    // Hack to emit end on close because of a core bug that never fires end
    response.on('close', function () {
      if (!self._ended) {
        self.response.emit('end')
      }
    })

    response.once('end', function () {
      self._ended = true
    })

Looks like a bug in yarn. It should be checking res.req._ended.

@ronag
Copy link
Member Author

ronag commented Sep 23, 2019

This needed some refactoring to make CITGM pass and resolve conflicts. Will request re-reviews once I get an ok from Travis.

@@ -629,7 +630,7 @@ function responseOnEnd() {
socket.end();
}
assert(!socket.writable);
} else if (req.finished) {
} else if (req.finished && !this.aborted) {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If response is aborted then the socket is closed and there is nothing to keep alive.

@ronag ronag force-pushed the fix-req-res-agent-close branch 4 times, most recently from 41e6890 to 3c7d3f9 Compare September 23, 2019 23:13
req.emit('close');
if (req.res) {
req.res.emit('close');
}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Had to move the close emitters here to ensure they are emitted after all userland 'end' handlers have completed.

@nodejs-github-bot
Copy link
Collaborator

@ronag
Copy link
Member Author

ronag commented Oct 13, 2019

@Trott ready to land?

@nodejs-github-bot
Copy link
Collaborator

@Trott
Copy link
Member

Trott commented Oct 13, 2019

@Trott
Copy link
Member

Trott commented Oct 13, 2019

@mcollina @jasnell A quick re-review might not be a bad idea, given that some stuff has moved/ changed.

@Trott Trott added the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label Oct 13, 2019
@nodejs-github-bot
Copy link
Collaborator

nodejs-github-bot commented Oct 14, 2019

@Trott
Copy link
Member

Trott commented Oct 14, 2019

CITGM: https://ci.nodejs.org/view/Node.js-citgm/job/citgm-smoker/2056/

CITGM results look good.

@Trott Trott added the notable-change PRs with changes that should be highlighted in changelogs. label Oct 14, 2019
@Trott Trott added this to the 13.0.0 milestone Oct 14, 2019
@Trott
Copy link
Member

Trott commented Oct 14, 2019

@nodejs/http @nodejs/collaborators Some final reviews on this one?

Trott pushed a commit that referenced this pull request Oct 15, 2019
PR-URL: #28685
Refs: #28684
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Trivikram Kamat <[email protected]>
Reviewed-By: Rich Trott <[email protected]>
@Trott
Copy link
Member

Trott commented Oct 15, 2019

Landed in d247a8e

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
author ready PRs that have at least one approval, no pending requests for changes, and a CI started. http Issues or PRs related to the http subsystem. notable-change PRs with changes that should be highlighted in changelogs. semver-major PRs that contain breaking changes and should be released in the next major version.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants