Editorial: use dictionaries for { value, done } and eliminate "forAuthorCode"?

[domenic]

After #1035 lands, I think it might be possible to go further.

In particular, I'm hoping to clean up the return type of read() to be a dictionary with value and done members.

But while looking into this, I'm noodling on ways to clean up the whole "forAuthorCode" business. Is there a way we can keep the internals of the stream agnostic to the JS object format, and only convert the chunks/end-of-stream signals into dictionaries (and then via Web IDL, JS objects) at the end?

The difficult part of this is that the internals of the streams seem bound up in the promise machinery, and promises are fundamentally resolved with JS values. Maybe we could resolve those promises with JS values for the chunks, and have a special JS value representing end-of-stream, instead of having to create prototype-pollution-prone { value, done } objects and then potentially null out their prototype?

I think it's also worth figuring out what the best interface is for other specifications. If we could avoid them having to deal with raw promises for { value, done } objects, that would be a plus.

[ricea]

JS values for chunks would also be vulnerable to prototype pollution.

When I thought about this before, the only solution I came up with was to have something almost but not quite the same as a Promise to use internally.

[domenic]

Oh, you're totally right. Dang.

Streams is in a peculiar position where it is using promises for internal flow control (throughout the spec ecosystem) as well as externally exposed to web developers. Everything else in the ecosystem only uses promises at the developer boundary, and uses "async algorithms" of some sort internally. Those async algorithms are sometimes informal, and sometimes formal (using mechanisms like "in parallel", "queue a task", etc.). whatwg/infra#181 touches on this more.

So I think if we were to do anything here it would be best to make streams behave more like the rest of the ecosystem. In particular, I think

the only solution I came up with was to have something almost but not quite the same as a Promise to use internally

is right, but I wonder if we'd be able to do that surgically, in only the places that handle chunks. I.e., keep using promises for closed/ready/the return values of the underlying X methods/etc. But for ReadableStreamDefaultReaderRead / [[PullSteps]] / read(), do something a bit different, passing the raw chunk around (not in promise form), until finally wrapping it up right before returning from read().

If this is surgical enough, then perhaps we could just use "spec callbacks". Callbacks are gross and messy compared to promises, but if you only go through one or two algorithms using them, it might not be so bad.

[ricea]

So, concretely, we'd change ReadableStreamDefaultReaderRead() and PullSteps() to take (resolve, reject) callbacks as arguments instead of returning a Promise. And [[readRequests]] would become a queue for (resolve, reject) pairs rather than promises.

Tricky part is: do we call the callbacks synchronously, and risk re-entrancy, or asynchronously, and have to post microtasks somehow.