arm64 image missing ganesha.nfsd binary

[tibbe]

Is this a bug report or feature request?

• Bug Report

Bug Report

What happened:
The ceph-v15.2.10 (and others) arm64 image doesn't contain the ganesha.nfsd binary, causing the NFS container to fail to start. See rook/rook#7671 for the full bug report against rook (where I first experienced this).

What you expected to happen:
arm64 image to contain ganesha.nfsd binary.

How to reproduce it (minimal and precise):

Please see rook/rook#7671.

Environment:
Please see data in rook/rook#7671.

• OS (e.g. from /etc/os-release):
• Kernel (e.g. uname -a):
• Docker version (e.g. docker version):
• Ceph version (e.g. ceph -v):

[dsavineau]

That's because nfs-ganesha packages aren't available on arm64 at the moment.

https://download.ceph.com/nfs-ganesha/rpm-V3.3-stable/octopus/el8/aarch64/

[tibbe]

Filed an issue against ceph: https://tracker.ceph.com/issues/50437

[tibbe]

I'm confused: https://hub.docker.com/layers/rook/nfs/v1.6.0/images/sha256-d83c8fef4ecaf16e18de65ff4c6a767aaad9a84eb648c2ea057f4dd45fac8c23?context=explore is an arm64 ganesha and is used by rook/nfs so there's at least a ganesha build available somewhere that is used by rook, just not through their rook/ceph images.

[dsavineau]

I'm not a rook expert but I'm pretty sure rook/nfs is not the same than rook/ceph via nfs-ganesha.

The one you linked seems like a regular nfs service as a rook storage provider. But nothing related to nfs-ganesha.

The rook/nfs container images aren't based on this project (as opposed to rook/ceph images)

https://github.com/rook/rook#project-status
https://github.com/rook/rook/blob/master/Documentation/nfs.md

[tibbe]

I thought so as well but I took a peak at the rook/nfs image and it's launching the NFS service like so:

nfs server --ganeshaConfigPath=/nfs-ganesha/config/rook-nfs

I realize that these are different containers. I just wanted to point out that there are arm64 packages out there already used by the rook project (which I realize is different from this project).

It's one of these things that seems to fall between the cracks when lots of different projects are involved ending up with a bit of a fragmented support from a (rook/ceph) user's perspective.

[dsavineau]

nfs server --ganeshaConfigPath=/nfs-ganesha/config/rook-nfs

That's not how ganesha should be started so it's maybe a different thing. The binary for ganesha is ganesha.nfsd, like:

https://github.com/ceph/ceph-container/blob/master/src/daemon/demo.sh#L303
https://github.com/ceph/ceph-container/blob/master/src/daemon/start_nfs.sh#L38

I just wanted to point out that there are arm64 packages out there already used by the rook project (which I realize is different from this project).

I'm not saying there's no nfs-ganesha packages for arm64 at all. I'm just saying that the ceph infrastructure (download.ceph.com) doesn't offer nfs-ganesha arm64 builds for both rpm and deb. And the ceph container image only use the ceph infrastructure as source of ceph/ganesha packages.

I know that there are existing arm64 elsewhere like

• in the CentOS storage SIG http://mirror.centos.org/centos-8/8/storage/aarch64/nfsganesha-3/Packages/
• in Ubuntu main repository https://packages.ubuntu.com/search?keywords=nfs-ganesha-ceph
• in Debian main repository https://packages.debian.org/search?keywords=nfs-ganesha-ceph

And probably in other locations.

The rook/nfs container image is based on Ubuntu (whereas the ceph one is based on CentOS) and uses a gluster ppa repository http://ppa.launchpad.net/gluster/nfs-ganesha-2.7/ubuntu/pool/main/n/nfs-ganesha/.

Don't get me wrong, I would love to add nfs-ganesha to the ceph arm64 container image if we have builds on the ceph infra in the future (like we did for ceph-iscsi recently).