Skip to content

Commit

Permalink
BACKPORT: erofs: support flattened block device for multi-blob images
Browse files Browse the repository at this point in the history
In order to support mounting multi-blobs container image as a single
block device, add flattened block device feature for EROFS.

In this mode, all meta/data contents will be mapped into one block
space. User could compose a block device(by nbd/ublk/virtio-blk/
vhost-user-blk) from multiple sources and mount the block device by
EROFS directly. It can reduce the number of block devices used, and
it's also benefits in both VM file passthrough and distributed storage
scenarios.

You can test this using the method mentioned by:
dragonflyoss/nydus#1139
1. Compose a (nbd)block device from multi-blobs.
2. Mount EROFS on mntdir/.
3. Compare the md5sum between source dir and mntdir/.

Later, we could also use it to refer original tar blobs.

Signed-off-by: Jia Zhu <[email protected]>
Signed-off-by: Xin Yin <[email protected]>
Reviewed-by: Jingbo Xu <[email protected]>
Acked-by: Chao Yu <[email protected]>
Tested-by: Jiang Liu <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
[ Gao Xiang: refine commit message and use erofs_pos(). ]
Signed-off-by: Gao Xiang <[email protected]>
  • Loading branch information
Jia Zhu authored and asuka-mio committed Apr 29, 2023
1 parent 71edbe8 commit 8bb5a81
Show file tree
Hide file tree
Showing 3 changed files with 17 additions and 9 deletions.
8 changes: 6 additions & 2 deletions fs/erofs/data.c
Original file line number Diff line number Diff line change
Expand Up @@ -192,7 +192,6 @@ int erofs_map_dev(struct super_block *sb, struct erofs_map_dev *map)
struct erofs_device_info *dif;
int id;

/* primary device by default */
map->m_bdev = sb->s_bdev;

if (map->m_deviceid) {
Expand All @@ -202,9 +201,14 @@ int erofs_map_dev(struct super_block *sb, struct erofs_map_dev *map)
up_read(&devs->rwsem);
return -ENODEV;
}
if (devs->flatdev) {
map->m_pa += erofs_pos(sb, dif->mapped_blkaddr);
up_read(&devs->rwsem);
return 0;
}
map->m_bdev = dif->bdev;
up_read(&devs->rwsem);
} else if (devs->extra_devices) {
} else if (devs->extra_devices && !devs->flatdev) {
down_read(&devs->rwsem);
idr_for_each_entry(&devs->tree, dif, id) {
erofs_off_t startoff, length;
Expand Down
1 change: 1 addition & 0 deletions fs/erofs/internal.h
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@ struct erofs_dev_context {
struct rw_semaphore rwsem;

unsigned int extra_devices;
bool flatdev;
};

/* all filesystem-wide lz4 configurations */
Expand Down
17 changes: 10 additions & 7 deletions fs/erofs/super.c
Original file line number Diff line number Diff line change
Expand Up @@ -252,6 +252,9 @@ static int erofs_init_devices(struct super_block *sb,
if (!ondisk_extradevs)
return 0;

if (!sbi->devs->extra_devices)
sbi->devs->flatdev = true;

sbi->device_id_mask = roundup_pow_of_two(ondisk_extradevs + 1) - 1;
pos = le16_to_cpu(dsb->devt_slotoff) * EROFS_DEVT_SLOT_SIZE;
down_read(&sbi->devs->rwsem);
Expand All @@ -265,14 +268,14 @@ static int erofs_init_devices(struct super_block *sb,
}
dis = ptr + erofs_blkoff(sb, pos);

bdev = blkdev_get_by_path(dif->path,
FMODE_READ | FMODE_EXCL,
sb->s_type);
if (IS_ERR(bdev)) {
err = PTR_ERR(bdev);
break;
if (!sbi->devs->flatdev) {
bdev = blkdev_get_by_path(dif->path, FMODE_READ | FMODE_EXCL, sb->s_type);
if (IS_ERR(bdev)) {
err = PTR_ERR(bdev);
break;
}
dif->bdev = bdev;
}
dif->bdev = bdev;
dif->blocks = le32_to_cpu(dis->blocks);
dif->mapped_blkaddr = le32_to_cpu(dis->mapped_blkaddr);
sbi->total_blocks += dif->blocks;
Expand Down

0 comments on commit 8bb5a81

Please sign in to comment.