WIP: Use go:embed with server-side gzip support

[wxiaoguang]

Replace #22887

Without pre-compressing, the binary size is 20MB larger.

gitea-vfsgen-gzip 102M
gitea-embed 123M

This approach could also support pre-compressing, just wrap the fs.FS to add the gzip support. However, it's not that easy to make Gitea's asset code and go:embed work well together with the pre-compressed files. Maybe it needs a separate temp dir like "temp/public", "temp/templates" and "temp/options" to store the pre-comporessed files, and then let "go:embed" to pack the "temp" directory.

---

This PR is mainly for a demo purpose (PoC), since vfsgen doesn't bother me too much, so at the moment I wouldn't spend time on the "pre-compressing" feature.

Feel free to edit this PR directly or start a new one.

[silverwind]

Suggested change

	//go:embed options public templates modules/migration/schemas
	//go:embed all:options all:public all:templates all:modules/migration/schemas

The all: prefix makes it embed files starting with _ and . too. I ran into this issue in another app where a file named like _.8538ed73.js would not embed. Currently webpack outputs a file -.8538ed73.js so it would still embed, but I think it's better safe to add this now instead of be surprised in the future.

[silverwind]

I think its better to compress before embed, e.g. with a temp directory. That way we don't have to pay the runtime cost of compression and binary size will be optimal.

https://github.com/silverwind/precompress is one tool to do it, but I see it's not ideal because i outputs into the same directory as the source file. To output into a temp directory, a new output directory option will be necessary for it.

[wxiaoguang]

This PR is mainly for a demo purpose (PoC), since vfsgen doesn't bother me too much, so at the moment I wouldn't spend time on the "pre-compressing" feature.

Feel free to edit this PR directly or start a new one.

[silverwind]

I will work on outputting to a separate dir in precompress (leaving source directory structure intact), then I can try integrating it here.

I think we may be able to get rid of all the go generate steps in Makefile with this.

[wxiaoguang]

No problem, then I will finish the "wrap the fs.FS to add the gzip support" part if there are compressed assets.

[silverwind]

I guess the assets will be:

/embed/migration/${path}.gz
/embed/options/${path}.gz
/embed/public/${path}.gz
/embed/assets/${path}.gz

If we decide to switch to brotli with the decoder, it would be .br extension.

[wxiaoguang]

/embed/migration/${path}.gz

To keep the consistent directory layout, IMO it could be /embed/modules/migration/schemas/...

[wxiaoguang]

If we decide to switch to brotli with the decoder, it would be .br extension.

No extension should be used. Because they should have the same file name as the "custom" directory.

Otherwise, the "List" function couldn't get the correct file list.

eg: List expects to see "/templates/head.tmpl" but not "(custom)/templates/head.tmpl" + "(embed)/templates/head.tmpl.gz"

[silverwind]

Okay, so original filename, but compressed bytes within. A bit unconventional, but possible.

[silverwind]

I've released https://github.com/silverwind/precompress v12, so in theory it should be:

npx precompress -t gz -E -o embed/assets -b public/assets public/assets

This should give compressed assets like embed/assets/js/index.js.

[silverwind]

Will check later to contribute it to this PR. One thing we need to consider is that we should serve uncompressed files during development as the cost of compression would be too high during development for no real benefit, especially once it's switched to brotli.

Ideally I would still like to emit assets with a .gz extension, by the way. It's clearer and then you don't need to guess the encoding while serving.

[wxiaoguang]

This PR is mainly for a demo purpose (PoC), since vfsgen doesn't bother me too much, so at the moment I wouldn't spend time on the "pre-compressing" feature.

Feel free to edit this PR directly or start a new one.