Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Default to zstd compression in v6 format #3312

Closed
pmatilai opened this issue Sep 18, 2024 · 7 comments · Fixed by #3483
Closed

Default to zstd compression in v6 format #3312

pmatilai opened this issue Sep 18, 2024 · 7 comments · Fixed by #3483
Assignees
Labels
fileformat Matters concerning package (file) format v6 Related to rpm v6 (readiness)
Milestone

Comments

@pmatilai
Copy link
Member

Suggested by @dralley in #2919:
gzip is ubiquitous but also getting long in the tooth and lacking threading options entirely. The best option in 2024 is probably zstd - its fast both ways, multithreads, used widely already and not to be overlooked: has an active upstream.

v6 will of course preserve the ability to switch the compressor, but defaulting to zstd makes it look a little more something from this millenium.

@pmatilai pmatilai added the v6 Related to rpm v6 (readiness) label Sep 18, 2024
@pmatilai pmatilai added this to RPM Sep 18, 2024
@github-project-automation github-project-automation bot moved this to Backlog in RPM Sep 18, 2024
@pmatilai pmatilai added the fileformat Matters concerning package (file) format label Sep 18, 2024
@pmatilai pmatilai added this to the 6.0.0 alpha milestone Oct 9, 2024
@ffesti ffesti moved this from Backlog to In Progress in RPM Dec 4, 2024
@ffesti ffesti self-assigned this Dec 4, 2024
@ffesti
Copy link
Contributor

ffesti commented Dec 4, 2024

Guess we should keep the default for source packages at gzip to maintain maximum compatibility. People will want to take srpm back to older versions and will want to at least install them there. Or is zstd old enough that we can assume older but still relevant rpm version would be built with zstd support?

@pmatilai
Copy link
Member Author

pmatilai commented Dec 4, 2024

Oh, good question. Binary payload is what I was thinking of, anyhow. Sources tend to be compressed as-is, so there's not much benefit trying to further squeeze them.

So yeah, lets leave src.rpm to gzdio.

ffesti added a commit to ffesti/rpm that referenced this issue Dec 4, 2024
Keep the compression for source packages untouched to keep maximum
forward compatibility for those.

This also enables multi-threaded compression per default.

Resolves: rpm-software-management#3312
@ffesti ffesti moved this from In Progress to In Review in RPM Dec 4, 2024
@pmatilai
Copy link
Member Author

pmatilai commented Dec 4, 2024

This another example of a seemingly simple thing that on a closer look has all sorts of open questions...

One way to actually "force" our v6 default would be using a different macro for it, eg %_binary_payload_6 and leave the non-versioned to older.

@ffesti
Copy link
Contributor

ffesti commented Dec 4, 2024

I don't like that too much. Splitting the settings to different macros is not something we want to get into. Otherwise we'll have to question whether we need two different setting all over the place.

@pmatilai
Copy link
Member Author

pmatilai commented Dec 4, 2024

That questioning is exactly what we should do actually.

There are other items where the defaults might differ between v4 and v6, including but probably not limited to %_binary_filedigest_algorithm. So it pays to stop to think about it a bit: this is actually a broader problem, so how to handle this in a general manner?

@dralley
Copy link
Contributor

dralley commented Dec 4, 2024

I'm not sure I understand why the default should remain the same. Don't most if not all relevant distros (like Fedora), in practice, override that value with their own distro-default config anyway? In that case, almost all users that care about building such SRPMs would already need to change the config to do so.

edit: or distros can make that decision for them downstream if they deem it worthwhile

Plus zstd support was added in RPM 4.14.0, so it ought to be the case that every distro newer than EL7 ought to support it. And EL7 is completely EOL from an upstream perspective and only receives ELS support. So I would say this is already a fairly minor problem.

ffesti added a commit to ffesti/rpm that referenced this issue Dec 4, 2024
Keep the compression for source packages untouched to keep maximum
forward compatibility for those.

This also enables multi-threaded compression per default.

Resolves: rpm-software-management#3312
@ffesti
Copy link
Contributor

ffesti commented Dec 4, 2024

OK, I pushed the macros expression as an example. For rpmbuild settings this could work in general. For rpm proper not so much. But may be there are not that many use cases for rpm itself.

ffesti added a commit to ffesti/rpm that referenced this issue Dec 4, 2024
Keep the compression for source packages untouched to keep maximum
forward compatibility for those.

Resolves: rpm-software-management#3312
@github-project-automation github-project-automation bot moved this from In Review to Done in RPM Dec 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
fileformat Matters concerning package (file) format v6 Related to rpm v6 (readiness)
Projects
Status: Done
Development

Successfully merging a pull request may close this issue.

3 participants