Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow using uncompressed metadata for proc-macros and dylibs #77602

Open
Aaron1011 opened this issue Oct 6, 2020 · 1 comment
Open

Allow using uncompressed metadata for proc-macros and dylibs #77602

Aaron1011 opened this issue Oct 6, 2020 · 1 comment
Labels
A-metadata Area: Crate metadata C-enhancement Category: An issue proposing an enhancement or a PR with one. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Comments

@Aaron1011
Copy link
Member

Currently, we always compress crate metadata for dylibs and proc-macros:

let metadata_kind = tcx
.sess
.crate_types()
.iter()
.map(|ty| match *ty {
CrateType::Executable | CrateType::Staticlib | CrateType::Cdylib => MetadataKind::None,
CrateType::Rlib => MetadataKind::Uncompressed,
CrateType::Dylib | CrateType::ProcMacro => MetadataKind::Compressed,
})
.max()
.unwrap_or(MetadataKind::None);

This prevents us from mmap-ing the metadata into memory when we decode it, since we need to read the entire file in order to decompress it.

We should allow configuring this setting via a -Z flag (and maybe through the Cargo.toml at some point), to allow users to trade disk space for speed. In addition to avoiding the overhead of compressing/decompressing, gaining the ability to mmap the crate metadata means that we may avoid loading unused portions from disk entirely. This is less important for proc-macros after #76897, but might still have a noticable performance impact for dylibs.

Suggested by @joshtriplett on Zulip.

@Aaron1011 Aaron1011 added C-enhancement Category: An issue proposing an enhancement or a PR with one. A-metadata Area: Crate metadata labels Oct 6, 2020
@jyn514 jyn514 added the T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. label Oct 6, 2020
@bjorn3
Copy link
Member

bjorn3 commented May 16, 2023

I think it would make sense to unconditionally use uncompressed metadata for proc-macros. They aren't used at runtime anyway, so binary size isn't as important as for dylibs. Dylibs aren't as common and even for libstd.so we actually load the metadata from libstd.rlib instead so no decompression happens during builds unless someone builds a crate as rust dylib only and even then I suspect for regular cargo builds rustc will instead load the metadata from the .rmeta file instead.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-metadata Area: Crate metadata C-enhancement Category: An issue proposing an enhancement or a PR with one. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

No branches or pull requests

3 participants