Resolve issues locating and presenting URL metadata records

[anjackson]

Working on the video support, we found some issues.

The necessary metadata for a URL like http://this.page/ is stored under a URL like metadata://this.page/. However, when using OutbackCDX, these records are not canonicalised in the same way, and so do not get stored under the same key. Hence, when pywb performs the vi_/http://this.page/ lookup, it doesn't find the record.

I can hack my way around this by telling OutbackCDX that the metadata URL is an alias for the http one. This means the pywb playback finds the metadata and playback work fine! However, it also shows up in the calendar as if it was another instance of the original http URL.

I'm not sure how best to proceed.

My preference is that we change the JavaScript so that it explicitly looks for a record like vi_/metadata://this.page, but I'm not sure if this is breaking the semantics of the vi_ hook?

Alternatively, could the vi_/http://this.page/' hook actually convert this into a lookup for metadata://this.page/` in the back-end?

[ikreymer]

The planned change for this is to use a resource record with target uri of: urn:embeds:<url of page>, so it will perform a lookup for https://<webarchive>/collection/id_/urn:embeds:<url of page>. The vi_/ hook will no longer be needed.

[ikreymer]

pywb 2.4.0 has been updated to use urn:embeds:<url> requests for videos.
warcit will also be updated to generate the same.