Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Asset event type and capture assets #1239

Open
wants to merge 86 commits into
base: master
Choose a base branch
from
Open

Add Asset event type and capture assets #1239

wants to merge 86 commits into from

Conversation

Juice10
Copy link
Contributor

@Juice10 Juice10 commented Jun 8, 2023

Asset events allow for asynchronous events containing (image or potentially other) assets.
See #860 for more info.

These are async events that get emmited after the original dom mutation events and can be used to inject these assets after the fact in the player or via post processing.

Todo:

  • create asset event for assets that failed to load
  • implement asset in player
  • [ ] add flag to log assets being captured
  • add documentation for recording config
  • [docs] explain how its different to inline images
  • decided whether or not to move the capturing and encoding to a web worker (out of scope)
  • use canvas webgl code to do blob encoding
  • move more types from rrweb-snapshot to @rrweb/types to circumvent circular dependencies
  • add meta data to meta event including what asset origins are going to be cached,
    • also filter hooks in Asset Manager listening to those origins
  • add support for all elements
    • video
    • audio
    • embed
    • source
    • track
    • input#type=image
    • img#srcset
    • svg
    • table#background
  • [ ] Remove inlineImages in favour of assetCapture
  • Forward inlineImages config to assetCapture
  • get it working in yarn live-stream
  • hide src attributes while asset is being awaited
  • work with rrdom
  • add tests for svg elements, also update asset manager to work with setAttributeNS setAttributeNS was needed for xlink:href, but all modern browsers rewrite that to href which is already supported out of the box.
  • add implementation for srcset
  • rename replay/asset/index.ts to replay/asset-manager/index.ts

@changeset-bot
Copy link

changeset-bot bot commented Jun 8, 2023

🦋 Changeset detected

Latest commit: c160f50

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 18 packages
Name Type
rrweb-snapshot Major
rrweb Major
rrdom Major
@rrweb/types Major
@rrweb/rrweb-plugin-canvas-webrtc-record Major
@rrweb/rrweb-plugin-canvas-webrtc-replay Major
@rrweb/rrweb-plugin-console-record Major
@rrweb/rrweb-plugin-console-replay Major
@rrweb/rrweb-plugin-sequential-id-record Major
@rrweb/rrweb-plugin-sequential-id-replay Major
rrdom-nodejs Major
rrweb-player Major
@rrweb/all Major
@rrweb/replay Major
@rrweb/record Major
@rrweb/packer Major
@rrweb/web-extension Major
rrvideo Major

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

Copy link
Contributor

@eoghanmurray eoghanmurray left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering whether it's worth taking this commit out and merging in a separate PR?
Fine if not; maybe in future this sort of thing could be merged in earlier while rest of a PR is ongoing.

@Juice10
Copy link
Contributor Author

Juice10 commented Jan 16, 2024

I'm wondering whether it's worth taking this commit out and merging in a separate PR? Fine if not; maybe in future this sort of thing could be merged in earlier while rest of a PR is ongoing.

If we weren't on an alpha release that type of commit would also require us to do a major version change. Not sure if it makes sense now, it's bound to break some tests and those got fixed later in subsequent commits. But in general I do agree with you, this type of thing would be better in a separate PR

return Promise.all(promises);
}

public reset(config?: captureAssetsParam | undefined): void {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we have to reset when a meta event is captured?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is for live-mode/addEvent. Once a meta event is triggered its an indication that a page navigation has happened. That tells us that we don't have to keep waiting for new asset events to show up, hence the reset being called.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. But do we need to store the capture config in the meta event, and reload it as this.config = config when a meta event is emitted?

Or we can think the capture config is the same across a session and also call reset when a meta event is emitted, but without storing a copy of the capture params in the meta event?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict of our conversation:

  • I'll move over the captureAssetsParam for the replay from the meta event to the full snapshot.
  • Copy dimensions from meta to full snapshot.

};

this.capturedURLs.add(url);
this.capturingURLs.delete(url);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how about move this line and the same line in the error block into a finally block?

@Yuyz0112
Copy link
Member

Just reviewed the last two files: the asset manager observer and the replay side manager, really cool implementation!

Leave some small comments, and we should be able to merge this in the following days.

@eoghanmurray
Copy link
Contributor

Not sure if this is a goal or a non-goal, but should we be collecting all assets for the srcset attribute? Shouldn't we be able to tell which one of the images is the one that will need to be rendered?

Copy link
Contributor

@eoghanmurray eoghanmurray left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rename cacheable -> capturable

packages/rrweb-snapshot/src/utils.ts Outdated Show resolved Hide resolved
packages/rrweb-snapshot/src/utils.ts Outdated Show resolved Hide resolved
Copy link
Contributor

@eoghanmurray eoghanmurray left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

more cacheable -> capturable

(This is now a commit)

packages/rrweb-snapshot/src/snapshot.ts Outdated Show resolved Hide resolved
packages/rrweb-snapshot/src/snapshot.ts Outdated Show resolved Hide resolved
packages/rrweb-snapshot/src/utils.ts Outdated Show resolved Hide resolved
packages/rrweb-snapshot/test/utils.test.ts Outdated Show resolved Hide resolved
packages/rrweb-snapshot/test/utils.test.ts Outdated Show resolved Hide resolved
packages/types/src/index.ts Outdated Show resolved Hide resolved
packages/rrdom/src/diff.ts Outdated Show resolved Hide resolved
packages/rrdom/test/diff.test.ts Outdated Show resolved Hide resolved
packages/rrdom/test/diff.test.ts Outdated Show resolved Hide resolved
packages/rrweb/src/replay/asset-manager/index.ts Outdated Show resolved Hide resolved
private async preloadAllAssets(timestamp: number): Promise<void[]> {
const promises: Promise<void>[] = [];
for (const event of this.service.state.context.events) {
if (event.timestamp <= timestamp) continue;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we are ignoring old asset events ... is the idea here to get the 'most recent version' of an asset?
Aside: is there any risk that the timestamp of the asset event will be the same as the fullsnapshot timestamp?

Juice10 and others added 29 commits June 26, 2024 10:39
xlink:href is deprecated
Most modern browsers rewrite `xlink:href` to `href` which is currently supported
Tested in Firefox, Safari & Chrome
"diff algorithm for rrdom › apply virtual style rules to node › should insert rule at index [0,0] and keep existing rules"
AssetManager.manageAttribute
different types of assets (not just img#src)
…the AssetManager needs to interrogate other attributes to determine how to manage, e.g. <link with href depending on rel="stylesheet"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants