Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: update manifest/eTags on asset file content change #6736

Merged
merged 6 commits into from
Sep 19, 2024
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 40 additions & 20 deletions packages/miniflare/src/plugins/assets/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -150,13 +150,15 @@ export const ASSETS_PLUGIN: Plugin<typeof AssetsOptionsSchema> = {
},
};

// The Asset Manifest and Asset Reverse Map are used to map a request path to an asset.
// 1. Hash path of request
// 2. Use this path hash to find the manifest entry
// 3. Get content hash from manifest entry
// 4a. In prod, use content hash to get asset from KV
// 4b. In dev, we fake out the KV store and use the file system instead.
// Use content hash to get file path from asset reverse map.
/**
* The Asset Manifest and Asset Reverse Map are used to map a request path to an asset.
* 1. Hash path of request
* 2. Use this path hash to find the manifest entry
* 3. Get content hash from manifest entry
* 4a. In prod, use content hash to get asset from KV
* 4b. In dev, we fake out the KV store and use the file system instead.
* Use content hash to get file path from asset reverse map.
*/

emily-shen marked this conversation as resolved.
Show resolved Hide resolved
export const buildAssetManifest = async (dir: string) => {
const { manifest, assetsReverseMap } = await walk(dir);
Expand All @@ -176,7 +178,7 @@ export type AssetReverseMap = {

/**
* Traverses the asset directory to create an asset manifest and asset reverse map.
* These are available to asset service worker as a binding.
* These are available to the Asset Worker as a binding.
* NB: This runs every time the dev server restarts.
*/
const walk = async (dir: string) => {
Expand Down Expand Up @@ -212,18 +214,36 @@ const walk = async (dir: string) => {
);
}

// Each manifest entry is [header, pathHash, contentHash].
// In dev we want to avoid reading and hashing every file every time we
// reload the dev server, so in dev the 'content hash' is hash(path + modifiedTime).
// This way the 'content hash' still updates on file change, so the eTag also updates,
// and we don't incorrectly get a 304.
const pathHash = await hashPath(
encodeFilePath(relativeFilepath, path.sep)
);
const modifiedTime = (await fs.stat(filepath)).mtimeMs;
const contentHash = await hashPath(
encodeFilePath(relativeFilepath, path.sep) + modifiedTime.toString()
);
/*
* Each manifest entry is a series of bytes that store data in a [header,
* pathHash, contentHash] format, where:
* - `header` is a fixed ??x?? bytes reserved for ??
emily-shen marked this conversation as resolved.
Show resolved Hide resolved
* - `pathHash` is the hashed file path
* - `contentHash` is the hashed file content
*
* The `contentHash` of a file is determined by reading the contents of
* the file, and applying a hash function on the read data. In local
* development, performing this operation for each asset file would
* become very expensive very quickly, as it would have to be performed
* every time a dev server reload is trigerred. In watch mode, depending
* on the user's setup, this could potentially be on every file change.
*
* To avoid this from becoming a performance bottleneck, we're doing
* things a bit differently for dev, and implementing the `contentHash`
* as a hash function of the file path and the modified timestamp.
* (`hash(filePath + modifiedTime)`).
* This way a file's corresponding 'contentHash' will always update
* if the file changes, and `wrangler dev` will serve the updated asset
* files instead of incorrectly returning 304s.
*/

const [pathHash, contentHash] = await Promise.all([
hashPath(encodeFilePath(relativeFilepath, path.sep)),
hashPath(
encodeFilePath(relativeFilepath, path.sep) +
filestat.mtimeMs.toString()
),
]);
manifest.push({
pathHash,
contentHash,
Expand Down
Loading