btrfs-convert aborted

[jonashaag]

This is with btrfs-progs master

$ sudo ./btrfs-convert /dev/sdc1
create btrfs filesystem:
	blocksize: 4096
	nodesize:  16384
	features:  extref, skinny-metadata (default)
	checksum:  crc32c
free space report:
	total:     417914159104
	free:      101515264 (0.02%)
creating ext2 image file
Unable to find block group for 0
Unable to find block group for 0
Unable to find block group for 0
kernel-shared/ctree.c:2375: split_leaf: BUG_ON `1` triggered, value 1
./btrfs-convert(+0x29cac)[0x5578d3e9fcac]
./btrfs-convert(+0x2dce3)[0x5578d3ea3ce3]
./btrfs-convert(btrfs_search_slot+0x11d5)[0x5578d3ea57f4]
./btrfs-convert(btrfs_csum_file_block+0x42b)[0x5578d3ec2564]
./btrfs-convert(+0xdf52)[0x5578d3e83f52]
./btrfs-convert(main+0x1cd4)[0x5578d3e86120]
/usr/lib/libc.so.6(__libc_start_main+0xf2)[0x7f738f4d1152]
./btrfs-convert(_start+0x2e)[0x5578d3e83a8e]
Aborted

[adam900710]

free space report:
	total:     417914159104
	free:      101515264 (0.02%)

Isn't this already showing the problem?

You only have around 100MiB For metadata.

Considering how many data you have already, 400+G, just for csum you will need at least 400+MiB space.

No wonder btrfs_search_slot() fails with -ENOSPC

[jonashaag]

Fair enough, I just reported this because the traceback said BUG. If there isn’t anything to fix here maybe at least the error message could be improved.

[jonashaag]

Also, according to df, I’ve had a lot more space left. Could it be that there was some space used by an earlier conversion attempt (that failed due to a bug in v5.4)?

[adam900710]

For the BUG_ON we definitely need to make it more graceful.

The space reported by ext* is not directly usable for btrfs.
The space btrfs can utilize for its metadata is just a subset of the free space.

One of the main reason of greatly reduced usable space is fragmentation.

Btrfs needs somewhat contig free space for its new chunks, thus if your fs has a lot of free space, but very fragmented, then you can see the result.

Previously convert attempt won't cause anything different.

[cmurf]

Downstream bug with btrfs-progs 5.10.

[tomchiverton]

Kernel bug report with v6.2

https://bugzilla.kernel.org/show_bug.cgi?id=203693

[henfri]

Hello,

(in which kernel) is this bug fixed?
I have just encountered it with v6.1

Is it the btrfs-progs or the kernel, that I need to update?

best regards,
Hendrik

[adam900710]

It's pure btrfs-progs. Kernel is not involved.

Please check the free space first. If there is not enough free space, btrfs-convert will fail no matter which version.

[henfri]

Hello,

thanks.
Indeed, btrfs-convert does report very little space left:

btrfs-convert from btrfs-progs v6.2

Source filesystem:
  Type:           ext2
  Label:          x
  Blocksize:      4096
  UUID:           x
Target filesystem:
  Label:
  Blocksize:      4096
  Nodesize:       16384
  UUID:           x
  Checksum:       crc32c
  Features:       extref, skinny-metadata, no-holes (default)
    Data csum:    yes
    Inline data:  yes
    Copy xattr:   yes
Reported stats:
  Total space:    5000981073920
  Free space:       2107527168 (0.04%)
  Inode count:       152621056
  Free inodes:       151980558
  Block count:      1220942645
Create initial btrfs filesystem
Create ext2 image file
Unable to find block group for 0
Unable to find block group for 0
Unable to find block group for 0
kernel-shared/ctree.c:2455: split_leaf: BUG_ON `1` triggered, value 1
btrfs-convert(+0x19354)[0x561595be2354]
btrfs-convert(+0x194e3)[0x561595be24e3]
btrfs-convert(+0x1beea)[0x561595be4eea]
btrfs-convert(btrfs_search_slot+0xe82)[0x561595be6406]
btrfs-convert(btrfs_csum_file_block+0x2a0)[0x561595bf3399]
btrfs-convert(+0xfd05)[0x561595bd8d05]
btrfs-convert(+0x1101e)[0x561595bda01e]
btrfs-convert(main+0x468)[0x561595bd6ea8]
/lib/x86_64-linux-gnu/libc.so.6(+0x2724a)[0x7f839937b24a]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x85)[0x7f839937b305]
btrfs-convert(_start+0x21)[0x561595bd80e1]

But df reports 510 GB out of 4.5TB free.
I am now filling the disk (with dd) and deleting that file, hoping that that solves the issue.
Do you have any better idea?

Apart from that: Should btrfs-convert not give a warning in cases where the space may be insufficient and abort if it is sure that the space is insufficient (i.e. if there is not enough space for the csums)

[adam900710]

Btrfs-convert reports different number because the original fs is fragmented.

Btrfs needs to reserve contiguous space for its metadata, thus heavily fragmented ext4 is not a good start.
Furthermore, due to the block group design of ext4, each block group will has its beginning utilized by ext4 already, further limiting the usable space.

The BUG_ON() will be removed soon, since callers can handle the error gracefully.

[henfri]

Hello,

so, best to defragment the original fs?

The BUG_ON() will be removed soon, since callers can handle the error gracefully.

Yes, but that error may come after 10 hours. It is preferable to know before, that the conversion may fail due to lack of space.
So it would be better to warn and in extreme cases (like mine) abort, don't you think?

Greetings,
Hendrik

[adam900710]

It's a little hard to estimate. Just shown in your case, although the percentage is very low 0.02%, but the number itself is still pretty high (almost 2G). It may be enough to contain the metadata or may not.

Your case is already at the boundary, in fact if you disable data checksum during conversion, it may even succeed.

The free space management of btrfs is already too complex even for non-converted fs, I'm afraid we can either cause too many false alerts (where the convert can succeed but still warns) or can only know the result until tried.

[henfri]

ok. You are the expert :-)

if I disable data checksum during conversion, (how) can I enable it later?
would e4defrag help?

[adam900710]

Defragging ext4 may help, but I'm not sure how helpful it can really be.
As the ext4 block group design is making it pretty hard already for btrfs to find enough space. It really depends on the details of the ext4 defrag behavior, which I'm not familiar with.

For the csum part, if you disable it (and conversion succeeded), you will hit a very hard situation where you have to delete the ext4 image first (aka, no way to rollback), then delete tons of files to release enough space.
Even after that, you can not enable data checksum for all the existing files, but creating new ones (aka, copying the old files to new location).

So if you care about data checksum, then do not bother to disable that in the first place, and delete as many files possible in the ext4 first.