stream: abort signal in readable streams #36061

benjamingr · 2020-11-10T08:54:00Z

Mostly opened for discussion about semantics.

Are these semantics correct? Do they make sense?

This is WIP: I want to test this thoroughly before considering merging + docs need to be updated.

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
documentation is changed or added
commit message follows commit guidelines

nodejs-github-bot · 2020-11-10T08:54:04Z

Review requested:

@nodejs/streams

benjamingr · 2020-11-10T14:28:50Z

Idea from Matteo:

Expose an .abortSignal property on stream.Readable with:
- A getter that returns the signal if present or null otherwise.
- A setter that sets the abort signal on the stream.

Question: what if someone sets it multiple times does it unsubscribe from previous calls?

Maybe a static helper for setting an abort signal?

ronag · 2020-11-10T14:38:07Z

I think it’s better without setter.

benjamingr · 2020-11-10T14:50:01Z

@ronag talking to Matteo on FB: I think it's probably better to have a static shedEventTarget (name pending) method and probably not expose a getter/setter to avoid confusion about "what happens if we call a setter on multiple signals" and to avoid versioning issues.

benjamingr · 2020-11-10T15:04:11Z

Naming suggestions welcome, also ccing @bterlson regarding naming suggestions or opinions here

benjamingr · 2020-11-10T15:46:56Z

I've prototyped an alternative approach using a static method here: 6a92d8c - I think the DX is slightly better though still not great?

lib/internal/streams/add-abort-signal.js

benjamingr · 2020-11-20T09:57:16Z

@mcollina @ronag please take a look I've updated this PR based on the meeting consensus namely - streams are destroyed with an error

nodejs-github-bot · 2020-11-20T15:21:04Z

CI: https://ci.nodejs.org/job/node-test-pull-request/34485/

jasnell · 2020-11-20T15:37:22Z

doc/api/stream.md

+);
+// Later, abort the operation closing the stream
+controller.abort();
+```


An example that shows this used with an async iterator would be good.

jasnell · 2020-11-20T15:38:56Z

test/parallel/test-stream-readable-destroy.js

+    for await (const chunk of read) {}
+  })(), /AbortError/);
+  setTimeout(() => controller.abort(), 0);
+}


A test that sets up a pipeline that uses AbortController would be good, as would a test using a single AbortController for two different Readables at the same time.

lib/internal/streams/add-abort-signal.js

doc/api/stream.md

mcollina · 2020-11-20T16:10:32Z

test/parallel/test-stream-readable-destroy.js

+  }));
+  controller.abort();
+  read.on('data', common.mustNotCall());
+}


Please add tests for writable as well.

benjamingr · 2020-11-21T09:51:45Z

@mcollina I've changed the API to work with Readable and Writeable and not just Readable and added a test for that as well.

@jasnell added a test for pipeline and an example with async iterators in the README.

@ronag I'll add the signal as a constructor argument to Readable and Writeable like in the original iteration of this PR in a follow up PR.

PTAL :]

mcollina · 2020-11-21T13:28:32Z

lib/internal/streams/add-abort-signal.js

+  return !!(obj && typeof obj.pipe === 'function');
+}
+
+module.exports = function addAbortSignal(signal, readable) {


this should not be called readable.

lib/internal/streams/add-abort-signal.js

mcollina · 2020-11-21T13:29:28Z

lib/internal/streams/add-abort-signal.js

+
+function isReadable(obj) {
+  return !!(obj && typeof obj.pipe === 'function');
+}


I don't think this works as you think it would. I think this passes for every descendant of Stream.

I think so too and so do the tests - but I just copied isReadable from pipeline

lib/internal/streams/add-abort-signal.js

lib/internal/streams/readable.js

lib/internal/streams/add-abort-signal.js

ronag · 2020-11-21T13:43:53Z

@ronag I'll add the signal as a constructor argument to Readable and Writeable like in the original iteration of this PR in a follow up PR.

Why not in same PR?

benjamingr · 2020-11-21T14:13:38Z

Why not in same PR?

There is already a lot of discussion happening here and I don't feel like making more changes here would make our lives easier basically.

benjamingr · 2020-11-21T14:21:09Z

@mcollina @ronag addressed :]

lib/internal/streams/add-abort-signal.js

nodejs-github-bot · 2020-12-06T21:21:34Z

CI: https://ci.nodejs.org/job/node-test-pull-request/34804/

nodejs-github-bot · 2020-12-07T10:27:24Z

CI: https://ci.nodejs.org/job/node-test-pull-request/34813/

nodejs-github-bot · 2020-12-07T13:08:26Z

CI: https://ci.nodejs.org/job/node-test-pull-request/34817/

github-actions · 2020-12-07T14:39:28Z

Commit Queue failed

- Loading data for nodejs/node/pull/36061
✔  Done loading data for nodejs/node/pull/36061
----------------------------------- PR info ------------------------------------
Title      stream: abort signal in readable streams (#36061)
   ⚠  Could not retrieve the email or name of the PR author's from user's GitHub profile!
Branch     benjamingr:abort-signal-stream -> nodejs:master
Labels     dont-land-on-v10.x, dont-land-on-v12.x, dont-land-on-v14.x, semver-minor, stream
Commits    1
 - stream: support abort signal
Committers 1
 - Benjamin Gruenbaum 
PR-URL: https://github.com/nodejs/node/pull/36061
Reviewed-By: Matteo Collina 
Reviewed-By: Robert Nagy 
------------------------------ Generated metadata ------------------------------
PR-URL: https://github.com/nodejs/node/pull/36061
Reviewed-By: Matteo Collina 
Reviewed-By: Robert Nagy 
--------------------------------------------------------------------------------
   ⚠  Commits were pushed since the last review:
   ⚠  - stream: support abort signal
   ✔  Last GitHub Actions successful
   ℹ  Last Full PR CI on 2020-12-07T13:08:26Z: https://ci.nodejs.org/job/node-test-pull-request/34817/
- Querying data for job/node-test-pull-request/34817/
✔  Build data downloaded
   ✔  Last Jenkins CI successful
   ℹ  This PR was created on Tue, 10 Nov 2020 08:54:00 GMT
   ✔  Approvals: 2
   ✔  - Matteo Collina (@mcollina) (TSC): https://github.com/nodejs/node/pull/36061#pullrequestreview-544382792
   ✔  - Robert Nagy (@ronag): https://github.com/nodejs/node/pull/36061#pullrequestreview-545601618
--------------------------------------------------------------------------------
   ✔  Aborted `git node land` session in /home/runner/work/node/node/.ncu

https://github.com/nodejs/node/actions/runs/406052944

benjamingr · 2020-12-07T15:23:09Z

@mcollina can you LGTM after the last rebase so I can land with the commit queue? I'm hesitant to land with ncu locally if I can help it :]

mcollina

lgtm

github-actions · 2020-12-07T16:08:35Z

Landed in 5122456...5bd1eec

PR-URL: #36061 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Robert Nagy <[email protected]>

benjamingr · 2020-12-07T16:12:40Z

Next up, the constructor arg version.

PR-URL: #36061 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Robert Nagy <[email protected]>

PR-URL: #36435 Notable changes: * child_processes: * add AbortSignal support (Benjamin Gruenbaum) (#36308) * deps: * update ICU to 68.1 (Michaël Zasso) (#36187) * events: * support signal in EventTarget (Benjamin Gruenbaum) (#36258) * graduate Event, EventTarget, AbortController (James M Snell) (#35949) * http: * enable call chaining with setHeader() (pooja d.p) (#35924) * module: * add isPreloading indicator (James M Snell) (#36263) * stream: * support abort signal (Benjamin Gruenbaum) (#36061) * add FileHandle support to Read/WriteStream (Momtchil Momtchev) (#35922) * worker: * add experimental BroadcastChannel (James M Snell) (#36271)

PR-URL: nodejs#36061 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Robert Nagy <[email protected]>

PR-URL: #36435 Notable changes: * child_processes: * add AbortSignal support (Benjamin Gruenbaum) (#36308) * deps: * update ICU to 68.1 (Michaël Zasso) (#36187) * events: * support signal in EventTarget (Benjamin Gruenbaum) (#36258) * graduate Event, EventTarget, AbortController (James M Snell) (#35949) * http: * enable call chaining with setHeader() (pooja d.p) (#35924) * module: * add isPreloading indicator (James M Snell) (#36263) * stream: * support abort signal (Benjamin Gruenbaum) (#36061) * add FileHandle support to Read/WriteStream (Momtchil Momtchev) (#35922) * worker: * add experimental BroadcastChannel (James M Snell) (#36271)

benjamingr added the stream Issues and PRs related to the stream subsystem. label Nov 10, 2020

benjamingr requested review from mcollina and ronag November 10, 2020 08:54

mcollina reviewed Nov 10, 2020

View reviewed changes

lib/internal/streams/add-abort-signal.js Outdated Show resolved Hide resolved

benjamingr marked this pull request as ready for review November 20, 2020 09:56

benjamingr force-pushed the abort-signal-stream branch from 6a92d8c to 52d3dd6 Compare November 20, 2020 09:57

benjamingr added the request-ci Add this label to start a Jenkins CI on a PR. label Nov 20, 2020

github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Nov 20, 2020

jasnell reviewed Nov 20, 2020

View reviewed changes

mcollina requested changes Nov 20, 2020

View reviewed changes

lib/internal/streams/add-abort-signal.js Outdated Show resolved Hide resolved

ronag reviewed Nov 20, 2020

View reviewed changes

doc/api/stream.md Show resolved Hide resolved

mcollina requested changes Nov 20, 2020

View reviewed changes

mcollina requested changes Nov 21, 2020

View reviewed changes

ronag reviewed Nov 21, 2020

View reviewed changes

lib/internal/streams/add-abort-signal.js Outdated Show resolved Hide resolved

ronag reviewed Nov 21, 2020

View reviewed changes

lib/internal/streams/readable.js Outdated Show resolved Hide resolved

ronag reviewed Nov 21, 2020

View reviewed changes

lib/internal/streams/add-abort-signal.js Outdated Show resolved Hide resolved

ronag reviewed Nov 21, 2020

View reviewed changes

lib/internal/streams/add-abort-signal.js Show resolved Hide resolved

stream: support abort signal

04cf1c7

benjamingr force-pushed the abort-signal-stream branch from 4b21891 to 04cf1c7 Compare December 6, 2020 20:35

benjamingr added the request-ci Add this label to start a Jenkins CI on a PR. label Dec 6, 2020

github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Dec 6, 2020

mcollina added the commit-queue Add this label to land a pull request using GitHub Actions. label Dec 7, 2020

github-actions bot added commit-queue-failed An error occurred while landing this pull request using GitHub Actions. and removed commit-queue Add this label to land a pull request using GitHub Actions. labels Dec 7, 2020

mcollina approved these changes Dec 7, 2020

View reviewed changes

benjamingr added commit-queue Add this label to land a pull request using GitHub Actions. and removed commit-queue-failed An error occurred while landing this pull request using GitHub Actions. labels Dec 7, 2020

github-actions bot removed the commit-queue Add this label to land a pull request using GitHub Actions. label Dec 7, 2020

github-actions bot closed this Dec 7, 2020

nodejs-github-bot pushed a commit that referenced this pull request Dec 7, 2020

stream: support abort signal

5bd1eec

PR-URL: #36061 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Robert Nagy <[email protected]>

benjamingr deleted the abort-signal-stream branch December 7, 2020 16:12

danielleadams pushed a commit that referenced this pull request Dec 7, 2020

stream: support abort signal

197ba21

PR-URL: #36061 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Robert Nagy <[email protected]>

danielleadams mentioned this pull request Dec 7, 2020

v15.4.0 proposal #36435

Merged

cjihrig pushed a commit to cjihrig/node that referenced this pull request Dec 8, 2020

stream: support abort signal

0ae0eb5

PR-URL: nodejs#36061 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Robert Nagy <[email protected]>

Linkgoron mentioned this pull request Oct 29, 2021

lib: make validateAbortSignal stricter #40657

Closed

github-actions bot mentioned this pull request Apr 24, 2024

Adding VersionStream for nodejs-22 wolfi-dev/os#17646

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

stream: abort signal in readable streams #36061

stream: abort signal in readable streams #36061

benjamingr commented Nov 10, 2020

nodejs-github-bot commented Nov 10, 2020

benjamingr commented Nov 10, 2020

ronag commented Nov 10, 2020

benjamingr commented Nov 10, 2020

benjamingr commented Nov 10, 2020

benjamingr commented Nov 10, 2020

benjamingr commented Nov 20, 2020

nodejs-github-bot commented Nov 20, 2020

jasnell Nov 20, 2020

jasnell Nov 20, 2020

mcollina Nov 20, 2020

benjamingr commented Nov 21, 2020

mcollina Nov 21, 2020

mcollina Nov 21, 2020

benjamingr Nov 21, 2020

ronag commented Nov 21, 2020 •

edited

Loading

benjamingr commented Nov 21, 2020

benjamingr commented Nov 21, 2020

nodejs-github-bot commented Dec 6, 2020

nodejs-github-bot commented Dec 7, 2020

nodejs-github-bot commented Dec 7, 2020

github-actions bot commented Dec 7, 2020

benjamingr commented Dec 7, 2020

mcollina left a comment

github-actions bot commented Dec 7, 2020

benjamingr commented Dec 7, 2020

stream: abort signal in readable streams #36061

stream: abort signal in readable streams #36061

Conversation

benjamingr commented Nov 10, 2020

Checklist

nodejs-github-bot commented Nov 10, 2020

benjamingr commented Nov 10, 2020

ronag commented Nov 10, 2020

benjamingr commented Nov 10, 2020

benjamingr commented Nov 10, 2020

benjamingr commented Nov 10, 2020

benjamingr commented Nov 20, 2020

nodejs-github-bot commented Nov 20, 2020

jasnell Nov 20, 2020

Choose a reason for hiding this comment

jasnell Nov 20, 2020

Choose a reason for hiding this comment

mcollina Nov 20, 2020

Choose a reason for hiding this comment

benjamingr commented Nov 21, 2020

mcollina Nov 21, 2020

Choose a reason for hiding this comment

mcollina Nov 21, 2020

Choose a reason for hiding this comment

benjamingr Nov 21, 2020

Choose a reason for hiding this comment

ronag commented Nov 21, 2020 • edited Loading

benjamingr commented Nov 21, 2020

benjamingr commented Nov 21, 2020

nodejs-github-bot commented Dec 6, 2020

nodejs-github-bot commented Dec 7, 2020

nodejs-github-bot commented Dec 7, 2020

github-actions bot commented Dec 7, 2020

benjamingr commented Dec 7, 2020

mcollina left a comment

Choose a reason for hiding this comment

github-actions bot commented Dec 7, 2020

benjamingr commented Dec 7, 2020

ronag commented Nov 21, 2020 •

edited

Loading