feat: Mux new_asset_settings Config Pane, Uploader FC

[evankirkiles]

This draft PR builds off of work @hdoro did on the branch feat/jan-24-features, as the implementation there provides a solid foundation for adding text tracks in the future (my initial goal when setting out to make changes).

Changes

Uploader.tsx

The main body of this PR is the conversion of the Uploader component from a class component to a functional component. This also comes with some organizational improvements to the logic around what state values can be set, and when. The lifecycle of a video upload under this PR is:

1. A user "stages" an upload by pasting a video URL, selecting a video file from the file selector, or dragging+dropping.
2. @hdoro's configuration modal is triggered, allowing user to specify options passed to Mux's new_asset_setting, as well as append subtitles (potentially auto-generated) and caption tracks. You can specify on a plugin-level or on a schema-option-level whether users are presented with this configuration modal. If you disable this feature, then the upload begins immediately with the merged configuration from the defaults, pluginOptions, and schemaOptions.
   1. If the user closes the configuration modal, the upload state is reset.
   2. If the user clicks Upload in the configuration modal, the staged upload (file or URL) is uploaded to Mux with the configuration provided, and the configuration modal is closed.
3. The user now sees the progress bar and cancel button until the asset finishes uploading, at which point the Player is displayed (this functionality has not changed).

If an error occurs anywhere in the above process, the upload is canceled, any staged upload or upload progress is deleted, and the error that occurred is displayed.

UploadConfiguration.tsx

The new UploadConfiguration component receives a staged upload (a file or URL) from Uploader.tsx and displays a modal for building a new_asset_settings object as per Mux typings. The default values for this configuration can be set on both a plugin-wide level (when initializing muxInput({ ... }) in sanity.config.ts or on a schema-specific level (in the options nested object when adding a schema field of type mux.video). A developer can additionally disable each part of the configuration:

1. disableUploadConfig: The base fields for new_asset_settings
2. disableTextTrackConfig: The array of text tracks for adding subtitles / closed captions

If both of these configuration parts are disabled, the configuration modal is never shown—the upload begins immediately with the default values (which potentially includes auto-generated subtitles as specified by defaultAutogeneratedSubtitleLangs) once a file or URL is provided to the upload staging box.

Example Configuration Modal

Example Configuration Output Object

The UploadConfiguration modal calls uploadUrl or uploadFile with a new_asset_settings object directly as typed from Mux, passed on as-is as the body of the POST request to /addons/mux/upload—no intermediary representation. For example:

{
  "input": [
    {
      "type": "video",
      "generated_subtitles": [
        { "name": "English", "language_code": "en" }
      ]
    }
  ],
  "mp4_support": "standard",
  "playback_policy": ["public"],
  "max_resolution_tier": "1440p",
  "encoding_tier": "smart",
  "normalize_audio": false
}

However, there seem to be things going on in that backend proxy that prevent many of these fields from going through.

TextTracksEditor.tsx

TextTracksEditor is delegated responsibility for managing user-added and auto-generated text tracks within the UploadConfiguration modal. There are three types of text tracks:

1. Auto-generated Subtitles, only available when encoding_tier == "smart"
2. Subtitles, which represent translations of dialog
3. Closed Captions, which are like subtitles but provide a transcription of all audio in the video

Currently, auto-generated subtitles are the only type of text track which will work out-of-the-box. The other two require us to somehow provide a URL to Mux of where to download the text track, functionality which has yet to be implemented. Possible solutions include:

• On the frontend, providing a Sanity FileInput (wasn't sure how to get this working) which can use Sanity's standard asset storage and spit out the URL we need to provide Mux with
• On the backend, intake the uploaded text track as a string from the frontend and somehow get it into a shareable URL. This would probably be the same solution as above, just handled entirely on the backend.

Addendum

Caution

For any of this configurable new_asset_settings functionality to work, someone on the Sanity team needs to address the lack of proxy-ability for new_asset_settings in the Sanity API's /addons/mux endpoints. Currently, it only seems to support passthrough for the mp4_support field. Would be happy to discuss how this should be done.

Essentially, all the backend proxy needs to do is sign the request to Mux with the user's API key and secret, remove the Origin header, and then:

1. If the upload is from a URL (/addons/mux/assets/{dataset}), create the Asset directly using the assets endpoint of Mux. The input should already be well-formatted from to just proxy straight through from Sanity's endpoint to Mux's /assets endpoint, adding in the uuid passthrough data.
2. If the upload is a local file (/addons/mux/uploads/{dataset}), generate a direct upload URL using the uploads endpoint of Mux and send that back to the client to use for uploading directly. This also should be formatted correctly to pass straight through from Sanity's endpoint to Mux's /uploads endpoint, adding in the uuid passthrough data—the body will just be provided as new_asset_settings.

Additional QoL improvements

This PR also includes some small QoL things:

1. The events emitted from uploadFile and uploadUrl are now correctly typed with discriminated unions.
2. The height of the Input component is now auto, fixing a visual bug where it would stretch to 100vh.
3. Validation of a user-pasted URL now occurs before calling uploadUrl and shows a toast on failure, similar to how validation of a file MIME type occurs on DragEnter as well as within uploadFile. This makes more logical sense to me, as before we were trying to detect these validation errors by muffling outputted "Invalid URL" errors from the uploadUrl pipeline and resetting it after an arbitrary 2 second delay.

Remaining TODO

• Add schema/plugin option for disabling configuration modal
• Consistent leading / text hierarchy on Configuration modal
• Debug DragEnter MIME type validation not working

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
sanity-plugin-mux-input-codesandbox	❌ Failed (Inspect)			Feb 27, 2024 1:22am

[vercel]

@evankirkiles is attempting to deploy a commit to the Sanity Team on Vercel.

A member of the Team first needs to authorize it.

[socket-security]

New and removed dependencies detected. Learn more about Socket for GitHub ↗︎

Package	New capabilities	Transitives	Size	Publisher

🚮 Removed packages: npm/[email protected]

View full report↗︎

[hdoro]

Thank you so much for the stellar work on this, @evankirkiles!

• the UI suggestions are great
• disabling configuration and text track generation makes sense and the way you approached it is very clear
• we're aware of the limitations of the backend proxy and I'm on it
• happy you've uncovered those QoL issues and fixed them 🙏

In general, we're quite aligned in our approach, but can you clarify why you've decided to split the text track options between "Caption" and "Subtitle"?

In order for me to make proper use of your code, can you change this PR to target feat/jan-24-features instead of main? We'll need to coordinate with the backend proxy before putting this live as a new version on main, so I can't accept this PR as-is.

As I never imagined someone would go through the hoops of using my messy branch as we were figuring out what were the next steps, I had a few commits locally that I hadn't pushed before. Sorry about that, Evan!

[sabrichu]

Just want to say thanks to both you @evankirkiles and @hdoro for working on this 🙌 It's frustrating that any image uploaded via the plugin is forced to opt into the more expensive default "smart" encoding option, so having these asset settings exposed/passed through will be a great help

[evankirkiles]

Thanks for the kind words @hdoro and @sabrichu ^•^

In general, we're quite aligned in our approach, but can you clarify why you've decided to split the text track options between "Caption" and "Subtitle"?

My main motivating factor was the opaqueness of the previous tab names ("Auto-Generated" vs "Custom")—I felt these weren't very descriptive and didn't immediately lend themselves to which type of track was being provided... for example it's not immediately clear that "Auto-Generated" makes subtitles, not closed captions, or that "Custom" starts out as subtitles and is treated as closed captions when that sub-option is checked. Splitting it into explicitly named tabs I felt cut out that extra little bit of thinking on the side of the content editor and made it instantly apparent what's going on, even if those tabs map to very similar Mux input data under the hood.

In order for me to make proper use of your code, can you change this PR to target feat/jan-24-features instead of main? We'll need to coordinate with the backend proxy before putting this live as a new version on main, so I can't accept this PR as-is.

Will do!

Going to keep testing things—one thing that would be super great would be listing tracks in the main Mux pane as well, but that takes a little bit more thought than this PR entails I imagine ^•^

[hdoro]

I felt these weren't very descriptive and didn't immediately lend themselves to which type of track was being provided

Agreed and love your suggestion! I just wanted to be sure this wasn't due to a reason I didn't understand, such as a fundamental difference between the tracks in Mux's APIs. I'm glad to go with your suggestion :)

And sounds good, let me know if you need my help or if you'd rather have me update the branch by adapting your code. I understand rebasing is quite a lot of work and I'm grateful for the work you've already done 🙂

Ah, and as for listing tracks, that's actually quite simple as we're persisting the track information to the Sanity document. I'll include them in the asset details in the final PR, but won't make them editable for now.

[evankirkiles]

@hdoro Rebased onto feat/jan-24-features, but I'm not sure why the deployment is failing... hopefully this at least cleans up the history somewhat. Let me know how you want to proceed from here so we can split up division of labor! Might be out of my hands at this point until the backend catches up, but happy to attend to wherever I can :)

[hdoro]

Great work on this, I didn't even consider we weren't taking per-schema configuration into account ✨

[hdoro]

Looking good! I'll do minor refactors, but overall this is stellar work, @evankirkiles 🚀

I'll take it from here - I'm mainly waiting on unblocking the backend side of this, but you can expect a release soon :)