Q: What is the expected memory usage of nfs-provisioner?

[nieldw]

I'm running nfs-provisioner:v1.0.8 as a daemonset. There is not a lot of load on this cluster, and without increasing it, I see the memory usage slowly but steadily increasing (running kubectl top pod). Most of the pods started with less than 200Mi usage, but some have been steadily growing over the past two days, now to over 400Mi.

How much memory are nfs-provisioner pods expected to use? Can it be limited?

Storage is quite crucial, I want these pods to run with a Guaranteed QoS.

@wongma7 Do you have any insights?

[wongma7]

To be honest I could not tell you as I have not performed real benchmarking. The app dutifully adds/remove exports using NFS-ganesha's provided methods, I don't know what reason the server would have to keep old exports/remnants of them in memory. It is not impossible there are some memory leak bugs in NFS-ganesha, however diagnosing those would not be easy https://github.com/nfs-ganesha/nfs-ganesha/issues

Generally in a performance critical environment I would not recommend using nfs-provisioner, at best it can serve as a taste of or stepping stone to a more full-featured/supported storage solution. there are no guarantees about NFS-ganesha running in VFS mode in this environment. A more viable alternative imo would be use the nfs-client provisioner + some knfs server, it's a better separation versus complicated all-in-one nfs-provisioner

[hasonhai]

hello @wongma7,
what is the main different between nfs-provisioner and nfs-client-provisioner? I saw there is too little documents on this. I only guess that the former acts as a NFS server and provides dynamic volumes to the pods while the latter is just a linker connecting the pods to an NFS server outside K8S cluster (that's why they have to provide the config for NFS server in ENV in the deployment yaml file). Am I correct?

[wongma7]

@hasonhai yes you are correct. nfs-provisioner is an all-in-one demo type of provisioner, nfs-client is a simple "proxy"

RE: memory usage, we are going to need to overhaul the external provisioner code soon for performance reasons anyway, it is based on outdated 'kubernetes controller' code, so stay tuned.