[css-typed-om] CSSFontFace value

[flackr]

Currently PaintRenderingContext2D (i.e. PaintWorklet) doesn't implement the CanvasText interface. My understanding is one reason is due to not being able to resolve fonts on the worklet context.

I propose we add an opaque type which would ideally allow capturing the 'font' style and provide a resolved font to the worklet. e.g.

registerPaint('foo', class {
  static get inputProperties() { return ['color', 'font']; }
  paint(ctx, geom, styleMap) {
    ctx.fillStyle = styleMap.get('color').toString();
    ctx.font = styleMap.get('font');
    ctx.fillText('foo', 0, 0);
  }
});

[AmeliaBR]

So, to clarify, the proposal is:

1. Add a new TypedOM type for font.
2. Change the CanvasText font property to accept this type object as an alternative to a string.
3. Implement the CanvasText interface in paint worklets, but throw an error if the font property is set to anything other than this special font object.

(1) seems like it's something that will probably happen anyway. It's a little complicated because font is a shorthand that also has single-keyword values that override all the longhands. But I'm sure @tabatkins has a long-term vague plan for how to make it work.

Once (1) happens, (2) makes sense, as part of a general approach to get Canvas APIs to accept TypedOM values as alternatives to strings that are going to get parsed as CSS values.

(3) is the question. Are there other reasons that text in worklets interferes with isolation & optimization? Does it make sense to restrict it to one TypedOM value that is passed in? What about constructing a CSSFontValue (or whatever it is called) in the worklet: which part would error out?

[css-meeting-bot]

The Houdini Task Force just discussed Opaque font P&V type for PaintWorklet font rendering, and agreed to the following:

• RESOLVED: disable svg text when in a paint worklet, possibly falling back to svg native

The full IRC log of that discussion

<Rossen_> Topic: Opaque font P&V type for PaintWorklet font rendering
<Rossen_> github: https://github.com//issues/899
<emilio> flackr: So while experimenting with PaintWorklet I decided to use fonts and the general idea seems to be that we avoid everything that relies on the document data
<heycam> q+
<emilio> ... I wonder if we could expose a font type so that we could resolve the font on the main thread and pass it down to the paint worklet
<emilio> bkardell_: so like a transferable font?
<AmeliaBR> q+
<emilio> flackr: I'd expect it to be more like an <image> type
<emilio> ... I don't expect to be able to do anything with it other than drawing
<emilio> myles: why not just passing a string?
<emilio> flackr: because that requires going through the platform code
<emilio> myles: should just be thread-safe
<emilio> TabAtkins: we disallow parsers on the worklet
<emilio> ... Chromium's css parser is not thread-safe
<emilio> myles: I'd be hesitant to add more API surface until there's more investigation
<emilio> iank_: you have a side-channel, you don't know if that font is loaded
<emilio> myles: I thought these weren't supposed to have any document data
<heycam> emilio: the fact that Chromium's CSS parser is not thread safe is a Chromium limitation
<emilio> myles: you want @font-face then this problem is really hard
<emilio> flackr: limiting it to system font limits the use cases
<emilio> myles: I think it'd be a good start
<emilio> TabAtkins: if we want to do font-face you could be able to send fontface objects
<emilio> myles: and no fallback/
<emilio> *?
<emilio> TabAtkins: you can do it yourself
<emilio> myles: that's not great
<emilio> TabAtkins: or send an array
<Rossen_> q?
<bkardell_> s/*?bkardell: can you not just transfer the array of resolved fonts?
<emilio> myles: I think we can choose between two worlds, on where you (maybe temporarily) only have access to system fonts, and one where the worklet is associated with a document and they share the same loading context and there's no need to transfer anything
<myles> s/system/installed/
<AmeliaBR> s/*?/bkardell: can you not just transfer the array of resolved fonts?/
<AmeliaBR> s/s\/*?bkardell: can you not just transfer the array of resolved fonts?//
<Rossen_> ack heycam
<emilio> heycam: I was going to ask with the relationship with the font loading api, but if for now there's only going to be installed fonts
<emilio> ... If we decide we want font-face i think we may want to not tie it to the document, and the document would have the same FontFaceSet
<emilio> hober: Wouldn't that require sync access from the worklet?
<hober> s/sync/network/
<emilio> heycam: if you want all the font types in the doc then sure, but we could limit it for blobs and array buffers and such
<emilio> [missed]
<emilio> heycam: you can have svg images with text in it
<emilio> myles: but that's not the same, no subpixel AA, etc.
<emilio> TabAtkins: we cannot avoid putting text on worklets at some point
<dbaron> heycam was talking about <svg:text>
<emilio> AmeliaBR: svg images has similar restrictions where there are no font-faces and such and it's isolated from the main document rendering
<emilio> ... in a way it's not that much different from how paint() is supposed to work
<emilio> ... if we have the same restrictions on the paint worklet so that it it can define a font-face where it doesn't have network access
<TabAtkins> q+
<emilio> ... so it is not as different as <text> in an svg image
<emilio> ... whether that addresses authoring use-cases is another question
<Rossen_> ack AmeliaBR
<emilio> ... saying that it can only use installed fonts doesn't seem aligned with paint()
<emilio> ... paint is for graphical effects so you want to have control over the fonts you're drawing
<bkardell_> AmeliaBR: is better articulating the question I was attempting to ask myles earlier about the use case -- for paint
<emilio> ... if you want textual content it shouldn't be using paint() but be content instead
<emilio> ... the proposal that flackr made was adding a new typed om type for fonts
<emilio> ... which i think it's a sensible thing for typed om to be able to represent
<emilio> ... we could set that from canvas as well
<emilio> ... some other people were talking about FontFace object, but fonts is much more complicated, it's a list of fonts with fallback, etc.
<emilio> ... we should aim for typed om to be able to represent all CSS data types like the font shorthand
<emilio> ... but that's a different question from text in canvas in paint()
<emilio> ... i'm not sure where in the cycle we'd prevent you from using constructed typed om objects constructed in the worklet
<emilio> ... seems like Chromium's parser is a limitation for chrome but others disagreed
<emilio> iank_: that part of the parser may be thread-safe already
<Rossen_> q?
<Rossen_> ack TabAtkins
<emilio> TabAtkins: so a multi-part proposal, I think we can do immediately, given that you can put <svg:text>, I think we could add a canvas function to render svg text
<emilio> ... you can create an svg text in the worklet right?
<emilio> TabAtkins: so proposal is adding back drawText() to canvas
<astearns> s/so proposal/so part 1 of the proposal/
<emilio> ... second part of the proposal is that we should see if parsing font stuff is still a problem for browsers (particularly chrome), and if it is figuring out something for that in the short term
<heycam> q+
<emilio> ... if it's not then probably just expose it and reconsider the decisions
<emilio> ... about exposing parser things in the canvas
<emilio> iank_: I think that's already thread-safe in blink
<emilio> TabAtkins: font stuff in particular?
<emilio> iank_: yes
<emilio> TabAtkins: part three is: in the medium term we investigate how to expose web fonts
<emilio> ... hober's idea about explicitly telling the worklet what fonts to is may work
<emilio> myles_: another idea is making FontFaceSet transferrable
<emilio> TabAtkins: you still need a way to thread-that down, but that could be a way
<emilio> dbaron: I don't recall why we decide text stuff to not be in there?
<AmeliaBR> FYI, working demo of text in a Paint worklet, using WebGL: https://github.com/AdaRoseCannon/three-paint
<emilio> TabAtkins: it was a philosophical reason, we didn't want people to do a bunch of text stuff in paint worklet
<emilio> Rossen_: also we wanted to look how the measuring APIs would work
<emilio> ... before doing that
<emilio> dbaron: I do recall some system libraries not being thread-safe
<emilio> myles_: coretext is not thread-safe, we'd have to rewrite it and that's fine probably
<emilio> dbaron: we're on multiple platforms
<Rossen_> ack dbaron
<TabAtkins> emilio: I think we use mostly harfbuzz these days, and that can be threadsafe
<TabAtkins> myles_: CoreText is threadsafe too
<myles_> s/coretext/webkit's font handling/
<heycam> q-
<emilio> dbaron: this makes probably a harder dependency on the idea that paintworklet is a list of drawing commands and not a raster
<emilio> TabAtkins: yeah it does
<emilio> myles_: why adding drawText if we want to investigate better things in the future?
<emilio> TabAtkins: it'd be in canvas2d right?
<emilio> myles_: I guess what I'm saying is that the eventual solution may not look like drawText at all
<emilio> AmeliaBR: whatever solution should probably be coordinated with canvas api
<emilio> myles_: right and we may add a new api to canvas2d
<emilio> ... I think it's a bit too early to say that we need to support this API
<emilio> TabAtkins: I don't like that there's a hacky way to draw text
<emilio> ... but not a good way
<emilio> emilio: does the svg stuff _actually_ work?
<emilio> TabAtkins: not sure, can somebody confirm quickly that it works?
<emilio> TabAtkins: disabling svg text stuff is also a suitable resolution
<emilio> iank_: we may be able to do the FontFaceSet stuff
<emilio> myles_: I think that's a bad design
<emilio> TabAtkins: it also requires network
<emilio> heycam: wouldn't you be able to use local sources?
<TabAtkins> s/it also requires network/it doesn't give us access to installed fonts/
<heycam> q+
<emilio> myles_: I think not representing the fact that fonts are a list of fallbacks is doing a disservice to authors
<myles_> s/authors/users/
<emilio> ... I think we should have transferrable font objects, but they probably shouldn't be FontFace
<emilio> ... should be some sort of thin wrapper over what the font shorthand represents
<emilio> flackr: I'm hoping that typed om font property can represent that wrapper
<emilio> TabAtkins: let's solve this in the medium term so we can all think about this properly
<emilio> TabAtkins: I want to resolve whether text is allowed at all or not
<emilio> heycam: it's plausible that the svg subsystem doesn't know how to disable svg text in a standalone way
<emilio> myles_: svg native doesn't allow text so it may be a good fit for that
<emilio> heycam: so you said that the original reason to disallow text is to avoid inaccessible text, is that no longer a concern?
<emilio> ... or is it because it works with svg?
<emilio> ... because if it's because of the latter we probably don't want to give up
<emilio> TabAtkins: it was only one of the factors, others include text is hard, and high-quality text needing font metrics
<AmeliaBR> https://ada.is/three-paint/
<emilio> AmeliaBR: you could use webgl too, see ^
<emilio> myles_: no you can't, you'd need to upload the font as a texture
<emilio> TabAtkins: that demo is fake webgl, it's 2d
<emilio> ... we cannot really avoid it
<emilio> TabAtkins: proposed resolution is to disable text when in a paint worklet, possibly falling back to svg native
<emilio> ... and long term we want to lift the text restriction
<emilio> AmeliaBR: we could point to the SVG-in-OT restrictions
<emilio> RESOLVED: disable svg text when in a paint worklet, possibly falling back to svg native
<TabAtkins> s/falling back to/by subsetting to/