space savings in opam switches

[edwintorok]

If your issue concerns a package not building, please report to
https://github.com/ocaml/opam-repository/issues or to the package maintainer
unless you are confident it is an issue in the opam tool itself.

opam config report
# opam config report
# opam-version         2.1.3
# self-upgrade         no
# system               arch=x86_64 os=linux os-distribution=fedora os-version=37
# solver               builtin-mccs+glpk
# install-criteria     -removed,-count[avoid-version,changed],-count[version-lag,request],-count[version-lag,changed],-count[missing-depexts,changed],-changed
# upgrade-criteria     -removed,-count[avoid-version,changed],-count[version-lag,solution],-count[missing-depexts,changed],-new
# jobs                 8
# repositories         1 (http), 3 (version-controlled) (default repo at ffef6370)
# pinned               4 (git)
# current-switch       trunk
[WARNING] Opam packages conf-libcurl.2 and conf-ncurses.1 depend on the following system packages that are no longer installed: libcurl-devel ncurses-devel
# ocaml:native         true
# ocaml:native-tools   true
# ocaml:native-dynlink true
# ocaml:stubsdir       /home/edvint/.opam/trunk/lib/ocaml/stublibs:/home/edvint/.opam/trunk/lib/ocaml
# ocaml:preinstalled   false
# ocaml:compiler       4.13.1

There is some discussion on space savings by compressing files produced by the compiler.
However there might be also some low-hanging fruit in terms of space savings in .opam itself which seems to contain a lot of redundant or unnecessary files.

du -csh ~/.opam/trunk/.opam-switch/sources/
487M	/home/edvint/.opam/trunk/.opam-switch/sources/
487M	total

I'm not sure what the purpose of that is (merlin/LSP jump to source, would it work?), but source files are also (sometimes?) available and installed alongside libraries in '~/.opam/trunk/lib'.

The raw sources like that are sometimes very big because they also contain unit test source code, and unit test source input data. And especially for monorepos that contain multiple packages these sources are duplicated N times... (well in my case I have a monorepo with 56 .opam files, so the sources are duplicated up to 56x times, and each source contains ~30MB of test input data too).

if the files here serve an essential purpose, I 'd recommend:

• keeping only relevant source files (e.g. based on file extension, or based on what files are referenced by .cmt* files/etc. to account for preprocessing and renaming files via copy#)
• hardlink identical files to save space (especially between files here and in lib/)

jdupes -L -r ~/.opam/trunk/.opam-switch/ reduces the size to ~250M

[rjbou]

On a first look, it is indeed possible to automatically remove sources directory after install action. We only need to keep in some cases:

• pinned packages
• dev packages
• packages url is vcs

[lefessan]

By the way, the sources for binary archives created by opam-bin are actually the binary artefacts themselves. Keeping them in sources does not make any sense as it only duplicates space usage.