btrfs-progs: mkfs: add --compress #925
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
maharmstone
changed the title
|
Thanks a lot. I'll close #919 and use this branch instead. Although to avoid previous accidents l'll take a more comprehensive review before merge. |
adam900710
reviewed
Nov 26, 2024
kdave
force-pushed
the
devel
branch
3 times, most recently
from
72c9865 to
164145b
Compare
maharmstone
force-pushed
the
compression
branch
from
934b46c to
e20b421
Compare
The function btrfs_record_file_extent() has extra handling that's specific to convert, like allowing the range to be split by block group boundary and image file extent boundary. All of these split can only lead to corruption for non-converted fs. As the only caller out of btrfs-convert is rootdir, which expects the file extent item insert to respect the reserved data extent, and never to be split. Thankfully this is not going to cause huge problem, as btrfs_record_file_extent() has extra checks if the data extent overlaps with any existing one, and if it doesn't the handling will be the same as the kernel. But to avoid abuse, change btrfs_record_file_extent() by: - Rename it to btrfs_convert_file_extent() And add extra comments on that it is specific to btrfs-convert. - Move it to convert/common.[ch] - Introduce a helper insert_reserved_file_extent() for rootdir.c Signed-off-by: Qu Wenruo <[email protected]>
…extent item Just like insert_reserved_file_extent() from the kernel, we can make btrfs_insert_file_extent() accept an on-stack file extent item directly. This makes btrfs_insert_file_extent() more flex, and it can now handle the converted file extent where it has an non-zero offset. And this makes it much easier to expand for future compressed file extent generation. Signed-off-by: Qu Wenruo <[email protected]>
There were two major problems with add_file_items(): it was writing all files sector-by-sector, making compression impossible, and it was assuming that pread would never do a short read. Fix these problems, and create a new helper add_file_item_extent(). Signed-off-by: Mark Harmstone <[email protected]>
Add an option --compress to mkfs.btrfs, to allow creating files using zlib when using --rootdir. Signed-off-by: Mark Harmstone <[email protected]>
Allow --compress to work with zstd, when compiled in. Signed-off-by: Mark Harmstone <[email protected]>
Allow --compress to work with lzo. Signed-off-by: Mark Harmstone <[email protected]>
maharmstone
force-pushed
the
compression
branch
from
e20b421 to
0a3108f
Compare
|
Qu, can you please give this latest version a go? In addition to the issues mentioned above, I've changed zlib_compress_inline_extent etc. so that like zlib_compress_extent etc. they return the size of the compressed data, rather than putting it in a pointer. I've also removed the confusing 0 / -E2BIG distinction, and put the logic in the caller instead. |
|
Looks very good to me now. I'll merge it after manually verifying the resulted image. You may want to add another series just for the extra test cases. |
|
I manually checked the following points, and hopes it will help your next test cases:
|
|
Thanks Qu |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds a --compress option to mkfs.btrfs when using --rootdir. (Resurrection of #882, closed by mistake.)
@adam900710, this includes your two patches, and all the suggestions that you made. If you're happy with all this, I suggested accepting this PR and abandoning #919.