From 6ad9998e8dbeb31f50f3e40334a1f31170a23107 Mon Sep 17 00:00:00 2001 From: Anne van Kesteren Date: Wed, 24 Feb 2021 13:00:07 +0100 Subject: [PATCH] Formalize reading bodies This refactoring allows specifications building on top of Fetch to avoid having to directly integrate with Streams. It makes these changes: * body's transmitted bytes is removed. body's total bytes will likely be removed in #604. * body's done and wait concepts are removed. * It introduces three primitives to read a body: incrementally read (process each chunk), fully read (get all), and fully reading body as promise (get all as promise). * Removed transmit request body in favor of HTTP-network fetch using incrementally read directly to transmit the request body. * Fetch's processRequestBody and processRequestEndOfBody are no longer passed a request. The former is passed the chunk length for progress reporting. (Needed by XMLHttpRequest.) * Fetch's processResponseEndOfBody when passed will now cause the body to be fully read and will pass the result in the second argument. (This means that callers can no longer read the body themselves.) * Subresource Integrity changed slightly with Fetch handing it the bytes and integrity metadata, and no longer a response. Essentially fetch fully reads the body, does the check, and then creates a new body from the same bytes to hand to the caller. Subresoruce Integrity PR: https://github.com/w3c/webappsec-subresource-integrity/pull/97. XMLHttpRequest PR: https://github.com/whatwg/xhr/pull/313. Closes #661. --- fetch.bs | 437 +++++++++++++++++++++++++++++++------------------------ 1 file changed, 244 insertions(+), 193 deletions(-) diff --git a/fetch.bs b/fetch.bs index 355a59387..425716e36 100644 --- a/fetch.bs +++ b/fetch.bs @@ -182,7 +182,7 @@ lt="authentication entry">authentication entries (for HTTP authentication). steps:
    -

  1. If taskDestination is a parallel queue, then +

  2. If taskDestination is a parallel queue, then enqueue algorithm to taskDestination. @@ -1003,25 +1003,12 @@ worked on in issue #1156

    • A stream (a {{ReadableStream}} object). -

    • A transmitted bytes - (an integer), initially 0. -

    • A total bytes (an integer), initially 0.

    • A source, initially null.

    -

    A body body is said to be -done if body is null or -body's stream is -closed or -errored. - -

    To wait for a -body body, wait for body to be -done. -

    To clone a body body, run these steps: @@ -1036,6 +1023,128 @@ worked on in issue #1156 body.

+
+ +

To incrementally read a body body, given an +algorithm processBodyChunk, an algorithm processEndOfBody, an algorithm +processBodyError, and an optional null, parallel queue, or +global object taskDestination (default null), run these steps. +processBodyChunk must be an algorithm accepting a byte sequence. +processEndOfBody must be an algorithm accepting no arguments. processBodyError +must be an algorithm accepting an exception. + +

    +

  1. If taskDestination is null, then set taskDestination to the result of + starting a new parallel queue. + +

  2. +

    Let reader be the result of getting a reader for + body's stream. + +

    This operation will not throw an exception. + +

  3. Perform the incrementally-read loop given reader, + taskDestination, processBodyChunk, processEndOfBody, and + processBodyError. +

+ +

To perform the incrementally-read loop, given a {{ReadableStreamDefaultReader}} object +reader, parallel queue or global object +taskDestination, algorithm processBodyChunk, algorithm +processEndOfBody, and algorithm processBodyError: + +

    +
  1. +

    Let readRequest be the following read request: + +

    +
    chunk steps, given chunk +
    +
      +

    1. Let continueAlgorithm be null. + +

    2. If chunk is not a {{Uint8Array}} object, then set + continueAlgorithm to this step: run processBodyError given a + {{TypeError}}. + +

    3. +

      Otherwise: + +

        +
      1. +

        Let bytes be a copy of + chunk. + +

        Implementations are strongly encouraged to use an implementation strategy that + avoids this copy where possible. + +

      2. +

        Set continueAlgorithm to these steps: + +

          +

        1. Run processBodyChunk given bytes. + +

        2. Perform the incrementally-read loop given reader, + taskDestination, processBodyChunk, processEndOfBody, and + processBodyError. +

        +
      + +

    4. Queue a fetch task given continueAlgorithm and + taskDestination. +

    + +
    close steps +

    1. Queue a fetch task given processEndOfBody and + taskDestination.

    + +
    error steps, given e +

    1. Queue a fetch task to run processBodyError given e, + with taskDestination.

    +
    + +

  2. Read a chunk from reader given + readRequest. +

+ +

To fully read a body body, given an algorithm +processBody, an algorithm processBodyError, and an optional null, +parallel queue, or global object taskDestination (default +null), run these steps. processBody must be an algorithm accepting a +byte sequence. processBodyError must be an algorithm accepting no arguments. + +

    +

  1. If taskDestination is null, then set taskDestination to the result of + starting a new parallel queue. + +

  2. Let promise be the result of fully reading body as promise given + body. + +

  3. Let fulfilledSteps given a byte sequence bytes be to + queue a fetch task to run processBody given bytes, with + taskDestination. + +

  4. Let rejectedSteps be to queue a fetch task to run + processBodyError, with taskDestination. + +

  5. React to promise with fulfilledSteps and + rejectedSteps. +

+ +

To fully read body as promise, given a +body body, run these steps: + +

    +

  1. Let reader be the result of getting a reader for + body's stream. If that threw an exception, then return + a promise rejected with that exception. + +

  2. Return the result of reading all bytes from + reader. +

+ +
+

To handle content codings given codings and bytes, run these steps: @@ -1202,7 +1311,7 @@ the empty string, * CSP: https://w3c.github.io/webappsec-csp/#effective-directive-for-a-request * Mixed Content: https://w3c.github.io/webappsec-mixed-content/#should-block-fetch * Preload: https://w3c.github.io/preload/#processing - * SRI: https://w3c.github.io/webappsec-subresource-integrity/#apply-algorithm-to-request + * SRI * HTML -->

A request's destination is @@ -1656,126 +1765,6 @@ is to return the result of serializing a request origin with request

-

To transmit request body, given a -fetch params fetchParams, run these steps: - -

    -
  1. Let body be fetchParams's request's - body. - -
  2. -

    If body is null and fetchParams's - process request end-of-body is non-null, then: - -

      -

    1. Let processRequestEndOfBody be this step: run fetchParams's - process request end-of-body given fetchParams's - request. - -

    2. Queue a fetch task given processRequestEndOfBody and - fetchParams's task destination. - -

    3. -

      Otherwise, if body is non-null: - -

        -
      1. -

        Let reader be the result of getting a reader for - body's stream. - -

        This operation cannot throw an exception. - -

      2. Perform the transmit-request-body loop given fetchParams, - body, and reader. -

      -
    - -

    To perform the transmit-request-body loop, given a fetch params -fetchParams, body body, and {{ReadableStreamDefaultReader}} -object reader: - -

      -
    1. -

      Let readRequest be the following read request: - -

      -
      chunk steps, given chunk -
      -
        -

      1. If the ongoing fetch is terminated, then abort these steps. - -

      2. If chunk is not a {{Uint8Array}} object, - terminate the ongoing fetch and abort these steps. - -

      3. Let bs be the byte sequence represented by the {{Uint8Array}} object. - -

      4. Let processRequestBody be null. - -

      5. If fetchParams's process request body is non-null, - then set processRequestBody to this step: run fetchParams's - process request body given fetchParams's - request. - -

      6. -

        In parallel: - -

          -
        1. -

          Transmit bs. Whenever one or more bytes are transmitted, increase - body's transmitted bytes by the number of transmitted bytes and - if processRequestBody is non-null then queue a fetch task given - processRequestBody and fetchParams's - task destination. - -

          This step blocks until bs is fully transmitted. - -

        2. Let continueAlgorithm be this step: perform the - transmit-request-body loop given fetchParams, body, and - reader. - -

        3. If the ongoing fetch is not terminated, then - queue a fetch task given continueAlgorithm and fetchParams's - task destination. -

        -
      - -
      close steps -
      -
        -

      1. If the ongoing fetch is terminated, then abort these steps. - -

      2. -

        If fetchParams's process request end-of-body is - non-null, then: - -

          -

        1. Let processRequestEndOfBody be this step: run fetchParams's - process request end-of-body given fetchParams's - request. - -

        2. Queue a fetch task given processRequestEndOfBody and - fetchParams's task destination. -

        -
      - -
      error steps, given e -
      -
        -

      1. If the ongoing fetch is terminated, then abort these steps. - -

      2. If e is an "AbortError" {{DOMException}}, then - terminate the ongoing fetch with the aborted flag set. - -

      3. Otherwise, terminate the ongoing fetch. -

      -
      - -

    2. Read a chunk from reader given - readRequest. -

    - -
    -

    To add a range header to a request request, with an integer first, and an optional integer last, run these steps: @@ -3243,9 +3232,11 @@ an optional algorithm processResp optional algorithm processResponseEndOfBody, and an optional boolean useParallelQueue (default false), run -the steps below. If given, processRequestBody and processRequestEndOfBody must -be an algorithm accepting a request; processResponse and -processResponseEndOfBody must be an algorithm accepting a response. +the steps below. If given, processRequestBody must be an algorithm accepting an integer +representing the number of bytes transmitted. If given, processRequestEndOfBody must be +an algorithm accepting no arguments. If given, processResponse must be an algorithm +accepting a response. If given, processResponseEndOfBody must be an +algorithm accepting a response and null, failure, or a byte sequence.

    An ongoing fetch can be terminated with flag aborted, @@ -3286,15 +3277,9 @@ the request. process response end-of-body is processResponseEndOfBody, and task destination is taskDestination. -

  3. -

    If request's body is a byte sequence, then: - -

      -

    1. Let body and ignoreType be the result of - safely extracting request's body. - -

    2. Set request's body to body. -

    +

  4. If request's body is a byte sequence, then set + request's body to the first return value of + safely extracting request's body.

  5. If request's window is "client", then set request's window to request's client, @@ -3632,51 +3617,75 @@ steps:

    This standardizes the error handling for servers that violate HTTP.

  6. -

    If response is not a network error and request's - integrity metadata is not the empty string, then: +

    If request's integrity metadata is not the empty string, then:

      -

    1. Wait for response's - body. - -

    2. If response's body's stream has not - errored, and response does not - match - request's integrity metadata, set response and - internalResponse to a network error. - [[!SRI]] -

    +

  7. Let processBodyError be this step: run fetch finale given + fetchParams and a network error. -

    This operates on response as this algorithm is not supposed to observe - internalResponse. That would allow an attacker to use hashes as an oracle. -

  8. -

    If fetchParams's process response is non-null, then: +

  9. If request's response tainting is "opaque", + response is a network error, or response's body + is null, then run processBodyError and abort these steps. -

      -

    1. Let processResponse be this step: run fetchParams's - process response given response. +

    2. +

      Let processBody given bytes be these steps: -

    3. Queue a fetch task given processResponse and fetchParams's - task destination. +

        +

      1. If bytes do not match + request's integrity metadata, then run + processBodyError and abort these steps. [[!SRI]] + +

      2. Set response's body to the first return value of + safely extracting bytes. + +

      3. Run fetch finale given fetchParams and response. +

      + +

    4. Fully read response's body given + processBody and processBodyError.

    -

  10. Wait for internalResponse's - body. +

  11. Otherwise, run fetch finale given fetchParams and response. +

+ +
+ +

The fetch finale, given a fetch params fetchParams and a +response response, run these steps: + +

    +

  1. If fetchParams's process response is non-null, then + queue a fetch task to run fetchParams's + process response given response, with fetchParams's + task destination.

  2. -

    Let doneAlgorithm be these steps: +

    If fetchParams's process response end-of-body is non-null, + then:

      -

    1. Set request's done flag. +

    2. Let processBody given nullOrBytes be this step: run + fetchParams's process response end-of-body given + response and nullOrBytes. + +

    3. Let processBodyError be this step: run fetchParams's + process response end-of-body given response and failure. -

    4. If fetchParams's process response end-of-body is - non-null, then run fetchParams's - process response end-of-body given response. +

    5. If response's body is null, then queue a fetch task + to run processBody given null, with fetchParams's + task destination. + +

    6. Otherwise, fully read response's body given + processBody, processBodyError, and fetchParams's + task destination.

    -

  3. Queue a fetch task given doneAlgorithm and fetchParams's - task destination. +

  4. Wait for either response's body to be null, or + response's body's stream to be + closed or errored, and then set + request's done flag. +

@@ -4610,7 +4619,6 @@ these steps: includeCredentials. -

  • Run these steps, but abort when the ongoing fetch is terminated: @@ -4660,7 +4668,62 @@ these steps:

  • Otherwise, return a network error. -

    Transmit request body given fetchParams. +

    To transmit request's body body, run these steps: + +

      +

    1. If body is null and fetchParams's + process request end-of-body is non-null, then + queue a fetch task given fetchParams's + process request end-of-body and fetchParams's + task destination. + +

    2. +

      Otherwise, if body is non-null: + +

        +
      1. +

        Let processBodyChunk given bytes be these steps: + +

          +

        1. If the ongoing fetch is terminated, then abort these steps. + +

        2. Run this step in parallel: transmit bytes. + +

        3. If fetchParams's process request body is + non-null, then run fetchParams's process request body + given bytes's length. +

        + +
      2. +

        Let processEndOfBody be these steps: + +

          +

        1. If the ongoing fetch is terminated, then abort these steps. + +

        2. If fetchParams's process request end-of-body is + non-null, then run fetchParams's + process request end-of-body. +

        + +
      3. +

        Let processBodyError given e be these steps: + +

          +

        1. If the ongoing fetch is terminated, then abort these steps. + +

        2. If e is an "AbortError" {{DOMException}}, + then terminate the ongoing fetch with the aborted flag set. + +

        3. Otherwise, terminate the ongoing fetch. +

        + +

      4. Incrementally read request's body given + processBodyChunk, processEndOfBody, processBodyError, and + fetchParams's task destination. + +

      +
    @@ -5724,20 +5787,8 @@ the associated steps:

  • Let promise be a promise resolved with an empty byte sequence. -

  • -

    If object's body is non-null, then: - -

      -

    1. Let stream be object's body's - stream. - -

    2. Let reader be the result of getting a reader from - stream. If that threw an exception, then return a promise rejected with that - exception. - -

    3. Set promise to the result of - reading all bytes from reader. -

    +

  • If object's body is non-null, then set promise to the + result of fully reading body as promise given object's body.

  • Let steps be to return the result of package data with the first argument given, type, and object's MIME type.