Using Annotations for Timed Text

[tomcrane]

Using Annotations for Timed Text

(Provide link to index.md of the issue, if available. To be filled in after issue is created - you need the issue number!)

Use case

You have WebVTT data for audio or video material. How do you transform that into web annotations?

[bencomp]

I think I have a highly similar use case, or perhaps an extension. In my use case, the existing data are multi-layer annotations in the ELAN Annotation Format (PDF). Annotation layers ('tiers') can be hierarchical, and annotations may be references to controlled vocabularies or simply textual.

[stephenwf]

@bencomp Do you think that structure would fit into ranges with each range having annotations on them? Perhaps annotation collections could be used at that point under the ranges to denote layers.

{
  "id": "https://example.org/manifest",
  "type": "Manifest",
  "structures": [
      {
         "id": "https://example.org/manifest/r1",
         "type": "Range",
         ....
         "annotations": [
            {
               "id": "https://example.org/manifest/r1/annoPage1",
               "type": "AnnotationPage",
               "items": [ 
                   ... converted annotations ...
                ],
               "partOf": {
                   "id": "https://example.org/manifest/layer1",
                   "type": "AnnotationCollection"
                }
            }
            ...
         ]
      }
      ...
  ]
}

[bencomp]

@stephenwf I hadn't thought of that, but yes, that might work. The EAF spec does relate annotations on child tier to (parent) annotations in the parent tier, so I'm not fully convinced yet :)

[stephenmcconnachie]

Could be worth broadening this to include other a/v timed text formats as source? Eg there are lots of subtitle formats including STR, TTML, EBU-STL, some structured text based and some XML based... Or is it preferred to have one issue per susbtitle format?

[stephenmcconnachie]

If it's preferred to create issues for other common subtitle formats I can do that, and I would start with EBU-STL and EBU-TT, and TTML maybe...

[mcwhitaker]

I have started creating the WebVTT captions for one of the fixtures (not for the whole duration, but enough to exemplify) so we have something to use in the recipe. I am not opposed broadening the title and mentioning in the Implementation notes that we are using WebVTT in the example but other formats could also be used.

[mcwhitaker]

@stephenmcconnachie - We have a cookbook editors meeting early next week; I can ask there if there is a preference for one recipe or multiple recipes.

[mcwhitaker]

The Cookbook Editors' suggestion is to broaden the scope of this recipe, mentioning in the Implementation notes that we are using WebVTT in the example but other formats could also be used.

[stephenmcconnachie]

The Cookbook Editors' suggestion is to broaden the scope of this recipe, mentioning in the Implementation notes that we are using WebVTT in the example but other formats could also be used.

That's great @mcwhitaker thank you. I have lots of subtitle files in various formats -SRT and TTML, and EBU-STL for that matter - and I also have plenty of WebVTT too. If it's useful, I could request permission to provide a real world a/v file with subtitle files in all formats I have - ie WebVTT, SRT, TTML, EBU-STL for the same a/v

[elynema]

The AV Annotations TSG would like to resurrect this recipe and get it created using the proposed provides property. The provides property will require us to specify closedCaptions or transcript, so we'll either need 2 recipes or simply have a single recipe with 2 sets of annotations.

Is an example manifest needed or a sample set of timed text annotations? We do have manifest creators who are creating timed text annotations containing transcripts/captions in the wild (AVAnnotate, Europeana, Aviary).