Hyperion custom types for annotation assistance

[tomcrane]

Types:

• DisplayAnnotation (this is very similar to RegionHighlight)
  • Annotation - with different pre-canned subclasses of HighlightingAnnotation, LinkingAnnotation, TaggingAnnotation and more
    • Target
    • Body
• TransitionOptions (see Constrain a single image or canvas view to a region #5)

DisplayAnnotation and TransitionOptions are CP-specific classes, they are about rendering things on the canvas surface and/or animation of elements / pan and zoom on the canvas surface. A DisplayAnnotation can have a CSS class, it can be given HTML.

Annotation, Target and Body are Hyperion helper classes for annotations, and don't know about display and HTML concerns.

It might make sense to have Hyperion-specific models for non-IIIF resources.
The framework is opinionated that it doesn't do that for Manifest, Canvas, etc, because Presentation 3 is supposed to be nicely normalised and you have a direct relationship between spec and your code using these objects.
But W3C isn't so normalised, and needs helpers.

Target

Target is a standardised object representing a spatial and/or temporal target on a canvas, from the various forms that could take in JSON.

"target": "http://example.org/canvas-1.json#xywh=1,2,3,4"

or

"target": { 
  "id": http://example.org/canvas-1.json", 
  "refinedBy": "#xywh=1,2,3,4" 
}

=> an object with .x, .y properties, etc, that can be used for all anno targets

.target.spatial.x, .target.spatial.y, .target.spatial.w ...

^ PointSelector has .point = true? w,h=0?

.target.temporal.tStart,.target.temporal.tEnd, target.temporal.point ?

Then this target property is available on a number of other special anno types - for links, tags etc.

You can also construct a target in code and then tell canvas panel to navigate to it, as in #5.

Body

Body is similar, and they are probably subclasses of something else. A body could be very similar to a target (e.g., a canvas in a manifest could be the body of a link annotation). But a body could be a URL, a textual body, and other kinds of resource.

Annotation

Annotation is not just a W3C Web Annotation, although you can create an instance from a W3C Anno in a constructor, and you can call .toW3CAnnotation() on an Annotation to get the W3C JSON.

Annotation is loaded and managed by the Vault.

DisplayAnnotation

For displaying an Annotation, you wrap it in a DisplayAnnotation (this name is still not right).
The Vault doesn't know anything about these. They wrap the Annotation, adding properties that help you style them, react to events on them, manage their visibility on the canvas, and other utility / extensions.

If you want total control of what you draw on the canvas, outside of IIIF and annotations, you can step down into Atlas and access the world directly in terms of Atlas. But for general annotation scenarios - including annotation creation and editing as well as tags, links, descriptions, markers, highlights... the DisplayAnnotation provides common functionality, and why it is tied to (by wrapping) an Annotation.

A DisplayAnnotation can be created from a vault-managed Annotation for use on CP and then given its extra properties for styling. Events can be wired up to it. It can be made interactive, allowing it to be positioned and re-sized by the user.

The point is not to give a complete world-creation toolkit, but to cater for common annotation-centric tasks for both display and capture of information on the canvas.

[stephenwf]

Another consideration - perhaps out of scope for this work - is serialisation. If we do form opinions for annotation formats (such as expanding targets and selectors) we would need a way to undo this on the way out.

[ pres 2 ] -- upgrader --> [ pres 3 ] -- normalizer --> [ vault ] -- serialiser ----> [ pres 3 ]
                                                                                 \--> [ pres 2 ]
^^^ optional ^^^^

In an ideal world, the input presentation 2 or 3 would match the output.

This means the more opinionated in the normalizer, the harder it will be to keep the output true to the original.

[tomcrane]

More on DisplayAnnotation

One major part of CP is simply rendering scenes - passing a canvas in and relying on CP to render all the painting annotations. Occasionally that requires some user interaction - when the painting anno has a Choice body - but usually, the point of CP is to forget about this.

The other major part of CP is working with annotations - displaying

CP could provide a general purpose drawing surface, but that would dilute the particular canvas abstraction we are trying to work with. It's not supposed to be general purpose, it's anno-focussed. If you want general you can step down into Atlas. The how-might-we scenarios are annotation centric.

But the W3C model is too loose to program against in a visual rendered environment, with events.
You can't put a W3C annotation onto the canvas, that doesn't really mean anything, it's just a piece of data.

Naming

This was RegionHighlight, which is the same name as an Atlas class.
Perhaps CP's internal Atlas impl of these would use RegionHighlight.
But not expose the internal RegionHighlight directly as part of API (though still accessible...?)
DisplayAnnotation is the iiif/w3c focused helper

DisplayAnnotation is the class used to wrap the underlying Annotation (which is still itself not a pure W3C anno); it corresponds directly with something renderable on the canvas.

Brainstorming

DisplayAnnotation has properties and events that relate to the visual (and/or audible) canvas - selection, resize, click etc.

Using CP as part of an anno creation tool, you'd make one.
In display modes, you'd turn W3C annos into them to make them visible to the user.

If the canvas allowed it, the user could resize or reposition something on the canvas. The object through which the code does this on the canvas - position, style, state, events - is a DisplayAnnotation

CP manages the set of DisplayAnnotations for you. It has a list of them. An ordered list. You can inspect this and manipulate it.
CP knows what DisplayAnnotations are on it
If you allowed all of the annos that come referenced from a manifest to be wrapped and displayed then they'd all be on the canvas and accessible.
You could add another one.
The developer has access to all the actual annos via Vault, and might want to be selective about which ones become visible on the canvas and when.
They need to become DisplayAnnotations to become visible, or become interactive
DisplayAnnotation is a canvas "entity" - but it's not a free-for-all general drawing tool.
While you can always drop down into Atlas and draw whatever you like, DisplayAnnotation is a way of drawing and managing common scenarios that usually come from annotations. Even if you synthesise an anno just to do the job - e.g., drawing a box on the canvas, #12, could be done in purely drawing/div/html terms at the Atlas level, but via a highlighting annotation, exposed by Vault as the helper Annotation class, then wrapped with a DisplayAnnotation to make it visible.

DisplayAnnotation has properties to show and hide it, you can move its stacking order (z-index) position, you can allow it to be moved by the user, you can allow it to be resized by the user...

See events introduction at #45 (comment)

Vault loads canvas... sees that there's a lot of annos here. You could just pipe them all through onto the canvas to become visible as DisplayAnnotation with defaults. You could pipe them all through and call callbacks to decorate/cancel/whatever. You could query by motivation, or expose the different anno pages they are on (e.g., German captions and English captions in separate anno page links).

You don't know what's there, so how do you find out what's there and then how do you react to it.. and when - with the Vault annos, before they become displayAnnos?
Let some out onto the canvas to become visible.. as wrapped DisplayAnnotations

Many ways this could be done...

Automatically have all annos become displayAnnos known to the canvas but don't make them visible (i.e., don't make them be reflected in DOM) until you say so...
Manually manage what annos become displayAnnos by listening to data loading events and making decisions (and decorating in passing)

cp.displayAnnotations.onPaint({ motivation: 'linking' }, () => { /* do something */ });

Stephen - I think I’ve found a way to do events in an easy, scalable way:

cp.query({
  type: "Annotation",
  motivation: "linking",
}).addEventListener("click", (e) => {
  // Handle the event here.
});

query first, then add event listener for normal events

cp.query({
  type: 'Canvas',
  options: { choice: true }
}).addEvenetListener('load', () => {
 // ...
})

then we can scale out with different matchers (like motivation: 'some value' ) and options, where the logic is maybe more specific

See Annotation Library (gist)

Constructing

Often constructed from a W3C annotation directly (because it's come from some IIIF).
Hyperion will turn a W3C annotation into a strongly typed helper annotation, if it can...

LinkingAnnotation
TaggingAnnotation

There might not be a lot of difference between these classes - they'll have different motivation properties, but many of them will have similar targets and many will have similar bodies.

While Target is probably the same class, bodies can be TextualBody (which can take HTML too), IIIFResourceBody (links to other IIIF resources or parts of), ResourceBody (links to other web resources like text/html pages, application/pdf document, etc).

A DisplayAnnotation can be constructed from one of these.

In the #13 linking example it's shown how the DisplayAnnotation constructor can take additional options to control handling of annotation bodies. For example if the linking annotation has a body that is part of a IIIF resource, you might want the generated hyperlink the end user clicks on to point to a viewer that loads that resource via a content state - you need a helper to transform that, CP won't know.

But can be constructed from Hyperion helper classes

How far do we want to go with default behaviour?

let dispAnno = new DisplayAnnotation("<any HTML>");
// now you can position dispAnno, have the user move it around...

// that's probably too loose. But:
const anno2 = new DescribingAnnotation("describing"); 

// its position and content are determined by the backing annotation:
anno2.target = new Target("xywh=2000,2000,1000,1000");
anno2.body = new HtmlBody("<any HTML>");
vault.load("xxx", anno2);

// but its style is at the DisplayAnnotation level
let dispAnno = new DisplayAnnotation(anno2, { cssClass: "desc" });
cp.displayAnnotations.add("xxx"); // add via vault ref?

A DisplayAnnotation has an annotation, and the annotation has Body and Target properties, where these are helper classes that help with the underlying W3C model variability.

Body and Target are probable subclasses of some other resource.

in #13 there's an example of body.getContentState() but a Target could have that too; there's some base class. A body might be a plain string, or a plain hyperlink.

[tomcrane]

NB DisplayAnnotation is a type that comes with Canvas Panel, it's not part of Hyperion; it depends on Annotation (and XXXAnnotation) types, with Body and Target types, from Hyperion.