Video recorder: optionally record subtitle file from displayed telemetry

[ES-Alexander]

QGC records .ass files that allow overlaying the telemetry data that it was showing when the video was recorded. We should be able to do the same, but potentially more powerful because we could likely record the actual shapes the widgets draw on the screen.

I’m thinking we could have a simple overlay option that just records the mini-widget contents (plus maybe the mission name and timestamp), and then an advanced one that records as much of the interface as possible.

Some links for context:

• relevant Cockpit forum discussion (yes, they wanted the .ass files)
  • includes a comment asking for independent selection of what to include in the file (compared to what the operator sees)
• relevant forum post (about what the SubStation Alpha format can include - would be very cool if we could support including vector-based logos)
• initial QGC proposal (for reasoning)
• current QGC subtitle writer code (it currently includes only/exactly the telemetry values that the operator has chosen to display in the telemetry widget)

[rafaellehmkuhl]

.ass, really? 😆

[rafaellehmkuhl]

PS: we currently have the option to record the entire screen, which helps with that, but is not available on Cockpit as an extension as BlueOS is not in a secure context.

[ES-Alexander]

.ass, really? 😆

Haha, yep, always a bit of a laugh.

… we currently have the option to record the entire screen, which helps with that, …

As I see it there’s significant value in being able to have an optional telemetry overlay, that’s separate from one or more full-resolution unobstructed video stream recordings, so I don’t consider recording the whole screen a viable equivalent here.

[ES-Alexander]

Noting that Carcinus have flagged that this functionality is "vital to most of [their] use-cases".

Relevant forum comment

[rafaellehmkuhl]

@ES-Alexander do you have a sample of a video with the .ass file so I can take a look on?

[patrickelectric]

I just want to raise the point that users not necessary want .ass files, but a video with OSD. I believe that using cockpit widget system for the users to arrange and a create his own video with custom OSD video would be better.
We can have an option on the record video widget that could open a popup where the users could add and arrange widgets around, and the same arrangement would be used to create de video with OSD.

[rafaellehmkuhl]

I just want to raise the point that users not necessary want .ass files, but a video with OSD. I believe that using cockpit widget system for the users to arrange and a create his own video with custom OSD video would be better. We can have an option on the record video widget that could open a popup where the users could add and arrange widgets around, and the same arrangement would be used to create de video with OSD.

Agree. We should discuss this better with the users before implementing.

[rafaellehmkuhl]

Noting here that @ES-Alexander has code for writing images to the subtitle file.

[ES-Alexander]

Noting here that @ES-Alexander has code for writing images to the subtitle file.

Noting that I'm yet to find a software that can actually display non-vector images in subtitles (or at least, VLC doesn't seem to support it, and that's the main one we care about).

For what it's worth, there is an online converter of SVG files to the ass format drawing commands, although it may be worth rolling our own, or at least including options for scaling and a position offset.

It could also be valuable to make/recommend a raster -> vector image converter that's focused on logos, since that's what we're expecting people to want to display, and users may not have a nice vector source already.

[ES-Alexander]

As a couple of separate notes:

• It could be cool to have an optional Cockpit logo in the top right corner, and possibly also logo options for the autopilot firmwares (e.g. ArduSub, ArduPilot, PX4, etc)
• If we want to get fancy we could add some kind of progress bar (or proportion of max dive depth) indicator
  • we shouldn't get overzealous with indicator creation though, because playback with normal indicators should end up possible through Cockpit itself

[ES-Alexander]

I can't remember if I've said this already, but in case I haven't, it would be useful if the configuration thing has a checkbox (that's selected by default) which uses the the values from all widgets* in the selected profile (not sure if toggling views should also toggle logged widgets, or if we want all widgets from all views in the profile to have their values shown at once in the recording, or just those from the first/primary view).

That way the default behaviour will match what people are used to from QGC, and then advanced users can configure things manually if they want. We may even want to leave the manual configuration option for later to determine whether people want it.

*Note: we may want to make an interface / specification for widgets to provide things like their

• name
• latest value
  • string, with the desired precision and units (as appropriate), and a relevant "no value" indication
• max character length?
  • useful for positioning

for logging purposes. That way the logger could just loop through and query each widget, and doesn't have to actually know anything about the values they represent, or how to format them.

Advanced configuration of the logging could then be focused entirely on which widgets to include, and where they're positioned (we could even allow allocating widgets to the 9 screen boxes that the subtitle format is based around).

I personally think it would be ok to only allow logging values that are configured in at least one of the views of the current profile (with the option to choose some widgets not to log), in which case if someone wants to log a value that they don't want to see during operation then they could add it in an extra view that the pilot doesn't use (we could make a "hidden view" option if it seems likely to be annoying to accidentally toggle to the view you don't want to see, although that might be unnecessary complexity).

[rafaellehmkuhl]

I personally think it would be ok to only allow logging values that are configured in at least one of the views of the current profile (with the option to choose some widgets not to log), in which case if someone wants to log a value that they don't want to see during operation then they could add it in an extra view that the pilot doesn't use (we could make a "hidden view" option if it seems likely to be annoying to accidentally toggle to the view you don't want to see, although that might be unnecessary complexity).

I was having the same idea yesterday night haha, but I don't think recording a DOM that is not being displayed is an easy task, although I think this would be the ultimate recording solution (configuring a view with the video, company logo, mission name, needed telemetry, etc, and this being the final video record). It would basically means our users would be able to generate commercial level videos without any editing.

I can't remember if I've said this already, but in case I haven't, it would be useful if the configuration thing has a checkbox (that's selected by default) which uses the the values from all widgets* in the selected profile (not sure if toggling views should also toggle logged widgets, or if we want all widgets from all views in the profile to have their values shown at once in the recording, or just those from the first/primary view).

Didn't though about that, but it seems like a good solution indeed. We can just make a callback register system that allows widgets to send their updated data with a line of code.