Trunk lib: Bundle & hash images/other resources referenced in Rust WASM source code

[thedodd]

In ParcelJS, the JS import system is overloaded to allow users to "import" images, css, sass and the like into their JS, and then Parcel will intercept those non-JS imports and build/bundle/hash them. It would be awesome to do something similar for trunk.

option 0

Create a compile-time macro which will take a path to a resource, maybe a few config options as well. Something like trunk::include!("assets/my-image.png"), or trunk::include!("my-component.css", Opt::Hash, Opt::Compress).

• first argument will be a path to the asset, verified at compile time.
• any values passed after the path parameter will be treated as pipeline configuration options for the asset.
• generate a manifest of all assets included during the cargo build of the Rust WASM app. Will use a std::sync::Once var to ensure the manifest is cleared at the beginning of each build (probably).
• after the cargo build is finished, trunk will spawn asset pipelines for each entry in the manifest and add them to the bundle.
• we could also do glob processing on these, something like trunk::include!("src/**/*.css"), to spawn pipelines for, and include, all css files under the src dir. A similar pattern is already planned as part of Support additional asset types #3. Will probably support both.

A variant of the macro will be exported, say trunk::include_ref!(...), which will return a String value which will be the public url of the asset, hashed and all, once the trunk pipeline is complete. This will allow applications to include an image directly in their Rust source code and have a correct ref to the asset after hashing and all.

considerations

• the macros will need to work the same way even when cargo build is executed outside of the context of trunk.
• however, when applications ship, and are being served over the network, they will need to be able to reference assets via their public URL (typically something like /<asset> or /static/<asset> &c).
• as such, the macros should default to / as the base path of the asset, but users will be able to set an env var to overwrite the default. When cargo is invoked by trunk during a trunk build (which is how all of this is intended to be used), trunk will be able to set the env var based on the value of --public-url to coordinate the values.
• trunk should look for a manifest generated in the cargo output dir matching the debug/release mode.

option n

Definitely open to other design options here. Please comment and let me know if you've got ideas you would like to share.

[philip-peterson]

I would love to see something like this supported by Trunk.

One way it could be achieved would be by having Trunk do a pre-build compile step, which only exists to create access to a dynamically linked function trunk_config(), defined by the library or application being compiled. This function would return a struct TrunkConfig that specifies all available assets for this library/application. The only trick would be making sure the version of Trunk matches or is at least compatible with the version defined in the library/application.

I started something similar to this on an old Yew branch that was abandoned. https://github.com/yewstack/yew/pull/1419/files#diff-f7645ea6b210138a842badb6858a7f96809d3f7ae896312a10f4f74e62aeea49R4

[9876691]

One way to do this I think is via build.rs in 2 stages

Say we have an images folder and we add a gif.

images/
       never-gonna-give-you-up.gif

Stage 1

Trunk copies the file to the dist folder and adds the hash.

dist/
       never-gonna-give-you-up-a23f4576.gif

Stage 2

When the image is added or changes in the images folder, build.rs runs and generates a struct that allows us to access the image without looking up the hash.

pub struct Images {
    pub get_never_gonna_give_you_up_gif() -> String {
        String::from("never-gonna-give-you-up-a23f4576.gif")
    }
}

And in my code I do something like

html! {
    <img src=Images::get_never_gonna_give_you_up_gif() />
}

The image will bust the cache on change. Also with this pattern if I remove an image or change the name the compiler will let me know.

[ghost]

I just came up with a barebones implementation. For now, it has nothing to do with Trunk, but I would love to work on this (should I wait for #207?). Here's what it does:

First, the user is supposed to configure the build.rs:

use preprocess_hash::ConfigHashPreprocessorExt;

fn main() {
    preprocess::configure()
        .hash() // adds hash to file name
        .run();
}

Then, the build script processes files in the assets folder (src/assets by default) and writes the results to the dist folder (dist by default). The following code will be generated:

pub const NEVER_GONNA_GIVE_YOU_UP_GIF: &str = "/never_gonna_give_you_up.e55dbac5.gif";

But in order to actually use it, the user has to manually include it:

mod assets {
    include!(concat!(env!("OUT_DIR"), "/assets.rs"));
}

#[function_component(App)]
fn app() -> Html {
    html! {
        <img alt="" src={assets::NEVER_GONNA_GIVE_YOU_UP_GIF}/>
    }
}

This is all good until it comes to using Trunk. The user runs trunk serve, which cleans up the dist folder, deleting all processed files. To work around this, they must add the following to the Trunk.toml (not ideal):

[[hooks]]
stage = "post_build"
command = "sh"
command_arguments = ["-c", "cp -r ./dist/* $TRUNK_STAGING_DIR"]

[9876691]

Hi the approach use by ructe https://github.com/kaj/ructe wouldn't require the hook step.

They don't transform the files they just use the files to calculate a hash.

So for example

images/
       never-gonna-give-you-up.gif

build.rs generates some helper.

So in your code you call

#[function_component(App)]
fn app() -> Html {
    html! {
        <img alt="" src={never_gonna_give_you_up_gif.name}/> // <-- src="never_gonna_give_you_up-1234545.gif"
    }
}

never_gonna_give_you_up_gif.name generates the name + hash.

Then on the server you have another helper that converts hashed names back to the original file name.

In Axum it looks like the following

async fn static_path(Path(path): Path<String>) -> impl IntoResponse {
    let path = path.trim_start_matches('/');

    if let Some(data) = StaticFile::get(path) {
        Response::builder()
            .status(StatusCode::OK)
            .header(
                header::CONTENT_TYPE,
                HeaderValue::from_str(data.mime.as_ref()).unwrap(),
            )
            .body(body::boxed(Body::from(data.content)))
            .unwrap()
    } else {
        Response::builder()
            .status(StatusCode::NOT_FOUND)
            .body(body::boxed(Empty::new()))
            .unwrap()
    }
}

Notice the StaticFile::get(path).

[ghost]

The issue is not about just hashing. An average app will also need compression, minification, compilation (e.g. sass) etc.