Skip to content

Commit

Permalink
Clarify that encoding tokens are scalar values
Browse files Browse the repository at this point in the history
This replaces some occurrences of "code point" with "scalar value", since it might not be clear to passing readers that output streams from decoding and input streams to encoding cannot contain tokens which are surrogates.

Fixes #195.
  • Loading branch information
Andreu Botella authored and annevk committed Jan 20, 2020
1 parent b818a7f commit 47c667e
Showing 1 changed file with 27 additions and 14 deletions.
41 changes: 27 additions & 14 deletions encoding.bs
Original file line number Diff line number Diff line change
Expand Up @@ -98,8 +98,8 @@ bitwise AND by "&", and bitwise OR by "|".

<hr>

<p>A <dfn id=concept-token>token</dfn> is a piece of data, such as a <a>byte</a>
or <a>code point</a>.
<p>A <dfn id=concept-token>token</dfn> is a piece of data, such as a <a>byte</a> or
<a>scalar value</a>.

<p>A <dfn id=concept-stream>stream</dfn> represents an ordered sequence of
<a>tokens</a>. <dfn>End-of-stream</dfn> is a special
Expand Down Expand Up @@ -216,15 +216,24 @@ happening.
<var>encoderDecoderInstance</var> is a <a for=/>decoder</a> instance, and "<code>fatal</code>"
otherwise.

<li><p>Assert: if <var>encoderDecoderInstance</var> is an <a for=/>encoder</a> instance,
<var>token</var> is not a <a>surrogate</a>.

<li><p>Let <var>result</var> be the result of running <var>encoderDecoderInstance</var>'s
<a>handler</a> on <var>input</var> and <var>token</var>.

<li><p>If <var>result</var> is <a>continue</a> or <a>finished</a>, return
<var>result</var>.

<li><p>Otherwise, if <var>result</var> is one or more
<a>tokens</a>, <a>push</a>
<var>result</var> to <var>output</var>.
<li>
<p>Otherwise, if <var>result</var> is one or more <a>tokens</a>:

<ol>
<li><p>Assert: if <var>encoderDecoderInstance</var> is a <a for=/>decoder</a> instance,
<var>result</var> does not contain any <a>surrogates</a>.

<li><p><a>Push</a> <var>result</var> to <var>output</var>.
</ol>

<li>
<p>Otherwise, if <var>result</var> is <a>error</a>, switch on <var>mode</var> and
Expand Down Expand Up @@ -858,6 +867,10 @@ different format here, to be able to represent ranges.)

<p>The <a>get an encoding</a> algorithm is to be used to turn a <a>label</a> into an
<a for=/>encoding</a>.

<p>Standards are to ensure that the streams they pass to the <a for=/>encode</a> and
<a>UTF-8 encode</a> algorithms are effectively scalar value streams, i.e., they contain no
<a>surrogates</a>.
</div>

<p>To <dfn export>decode</dfn> a byte stream <var>stream</var> using
Expand Down Expand Up @@ -898,7 +911,7 @@ fallback encoding <var>encoding</var>, run these steps:
<a>prepend</a> the last byte of <var>buffer</var> to
<var>stream</var>.

<li><p>Let <var>output</var> be a code point <a for=/>stream</a>.
<li><p>Let <var>output</var> be a scalar value <a for=/>stream</a>.

<li><p><a>Run</a> <var>encoding</var>'s
<a for=/>decoder</a> with <var>stream</var> and <var>output</var>.
Expand All @@ -918,7 +931,7 @@ these steps:
<li><p>If <var>buffer</var> does not match 0xEF 0xBB 0xBF,
<a>prepend</a> <var>buffer</var> to <var>stream</var>.

<li><p>Let <var>output</var> be a code point <a for=/>stream</a>.
<li><p>Let <var>output</var> be a scalar value <a for=/>stream</a>.

<li><p><a>Run</a> <a>UTF-8</a>'s
<a for=/>decoder</a> with <var>stream</var> and <var>output</var>.
Expand All @@ -930,7 +943,7 @@ these steps:
steps:

<ol>
<li><p>Let <var>output</var> be a code point <a for=/>stream</a>.
<li><p>Let <var>output</var> be a scalar value <a for=/>stream</a>.

<li><p><a>Run</a> <a>UTF-8</a>'s
<a for=/>decoder</a> with <var>stream</var> and <var>output</var>.
Expand All @@ -945,7 +958,7 @@ steps:
-->

<ol>
<li><p>Let <var>output</var> be a code point stream.
<li><p>Let <var>output</var> be a scalar value stream.

<li><p>Let <var>potentialError</var> be the result of <a>running</a>
<a>UTF-8</a>'s <a for=/>decoder</a> with <var>stream</var>, <var>output</var>, and
Expand All @@ -958,8 +971,8 @@ steps:

<hr>

<p>To <dfn export>encode</dfn> a code point stream <var>stream</var> using
encoding <var>encoding</var>, run these steps:
<p>To <dfn export>encode</dfn> a scalar value stream <var>stream</var> using encoding
<var>encoding</var>, run these steps:

<ol>
<li><p>Assert: <var>encoding</var> is not <a>replacement</a>, <a>UTF-16BE</a> or
Expand All @@ -979,9 +992,8 @@ encoding <var>encoding</var>, run these steps:
[[URL]]
[[HTML]]

<p>To <dfn export>UTF-8 encode</dfn> a code point stream <var>stream</var>,
return the result of <a lt=encode for=/>encoding</a>
<var>stream</var> using encoding <a>UTF-8</a>.
<p>To <dfn export>UTF-8 encode</dfn> a scalar value stream <var>stream</var>, return the result of
<a lt=encode for=/>encoding</a> <var>stream</var> using encoding <a>UTF-8</a>.



Expand Down Expand Up @@ -3278,6 +3290,7 @@ Adam Rice,
Alan Chaney,
Alexander Shtuchkin,
Allen Wirfs-Brock,
Andreu Botella,
Aneesh Agrawal,
Arkadiusz Michalski,
Asmus Freytag,
Expand Down

0 comments on commit 47c667e

Please sign in to comment.