Decode without BOM

[annevk]

For whatwg/url#84 I need a new specification hook of sorts. This is similar to what is already exposed through TextDecoder with ignoreBOM.

(I should probably create some tests around this first though.)

[annevk]

The obvious solution here would be to add an "ignore BOM flag" to the algorithm, but since whatwg/html#1077 (comment) potentially requires revamping things anyway, perhaps both should be tackled at once, to minimize the number of changes.

[hsivonen]

The present spec concepts, when generalized to apply to all encodings (currently some only apply to UTF-8) result is:

• Decode (with BOM sniffing)
• Decode with BOM removal (remove the BOM if it's for this encoding but treat BOMs for other encodings as garbage)
• Decode without BOM handling (don't look at the BOM, treat it as garbage)

encoding_rs already supports these generalizations. The mapping onto these from the current spec concepts is documented.

[annevk]

So the problem is that https://url.spec.whatwg.org/#urlencoded-parsing is only for servers. Browsers only ever use it with UTF-8 (in Fetch and in the URL API). So I think the solution here is to not change Encoding, but rather change URL to only support UTF-8 and let servers modify the algorithm if they need more than UTF-8.