Enable arm build for nfs-ganesha-stable

[ustaerk]

Currently arm builds for nfs-ganesha-stable are disabled, resulting in issues on arm-based ceph clusters (e.g. ceph/ceph-container#1878, rook/rook#7671, https://tracker.ceph.com/issues/50437).

Please enable arm builds for nfs-ganesha-stable.

[sfxworks]

Hello,
This is unfortunately still an issue. Just ran into this today.

Error: container create failed: executable file `ganesha.nfsd` not found in $PATH: No such file or directory

[Infinoid]

I ran into this today too. I got this in my cephadm log when trying to export a cephfs to nfs on my aarch64 cluster:

2022-12-12T18:01:30.973994+0000 mgr.ceph00.cjoafg [INF] Saving service nfs.nfs spec with placement ceph08;ceph09
2022-12-12T18:01:31.312370+0000 mgr.ceph00.cjoafg [INF] Creating key for client.nfs.nfs.0.0.ceph08.qiqvns
2022-12-12T18:01:31.372245+0000 mgr.ceph00.cjoafg [INF] Ensuring nfs.nfs.0 is in the ganesha grace table
2022-12-12T18:01:31.612414+0000 mgr.ceph00.cjoafg [ERR] Failed to apply nfs.nfs spec NFSServiceSpec.from_json(yaml.safe_load('''service_type: nfs
service_id: nfs
service_name: nfs.nfs
placement:
  hosts:
  - ceph08
  - ceph09
spec:
  port: 2049
''')): [Errno 2] No such file or directory: 'ganesha-rados-grace': 'ganesha-rados-grace'

After searching around, I see that the container build is waiting for ganesha packages to be built by the ceph build system for arm64. This affects me because I've deployed using cephadm, so everything is running in centos8 containers.

I also see that the centos 8 mirrors have packages for this.

So, what's needed to incorporate this into the ceph build system?

I see that the build config has some logic that limits arm64 builds to only ubuntu 16.04 and centos 7. That line was added by @alimaredia in June 2017, and these were the latest stable versions of the ubuntu and centos distros available at that time. I wonder if it was intended to prevent building on older versions, but is now preventing newer builds too?

Should we simplify or update the logic? I'm not very familiar with ceph-build... is there an easy way I can run it locally to test changes to this?

[mmeier86]

The problem also isn't restricted to the Hosts running the NFS daemon being arm64. You also can't have your MGR daemons running on arm64 hosts. Even when the NFS daemons are going to be placed on amd64 machines, the deploy via ceph orch apply nfs will still fail when the currently active MGR daemon is running on arm64.

[Infinoid]

Yes, I am running MGR daemons on arm64 hosts, as you describe. The above log snippet was taken from one of those hosts.

[mmeier86]

I've just done a bit of testing, and it seems that the NFS Ganesha packages install cleanly in the v16.2.10 Ceph aarch64 image with the following commands:

• dnf install centos-release-nfs-ganesha30
• dnf install nfs-ganesha-ceph nfs-ganesha-rados-grace nfs-ganesha-rados-urls nfs-ganesha-rgw nfs-ganesha

I will try to see what happens when I build an image from that and use it in my (rather small) Ceph cluster, but I have to setup a bit of infrastructure first, because my only current internal Docker Registry uses Ceph S3 as backing storage.

[ganniterix]

Hi, any udpate on this? Trying to get my cluster running on some RPi4's, via k8s and rook. Pod in crash loop backoff.

Error: failed to create containerd task: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "ganesha.nfsd": executable file not found in $PATH: unknown

[slawomirksiazek]

Hello team, Hi, any progress with this ?

Error: failed to create containerd task: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "ganesha.nfsd": executable file not found in $PATH: unknown

i trying to get my cluster running on some rockpi board witk k8s and rook but i have already the same situation pod in crash loop backoff.

[brauhausdc]

Any updates here? Just ran into this as well.

[encure]

ping.

[ganniterix]

https://github.com/ceph/ceph-container/issues/1878#issuecomment-824422577

As higlighted in this comment, what can we do to get the packges in to the correct repos?