NFS Provisioner - "Invalid Argument" errors with certain write patterns

[abrenneke]

A little related to #223.

Kubenetes version: 1.8.1
NFS Provisioner version: v1.0.8

Basically, I cannot use the kube-registry with the NFS Provisioner. NFS 4.2, 4.1, 4.0, and 3.0 all error when pushing images to the registry. The exact error in the kube-registry logs:

err.code=unknown err.detail="filesystem: readdirent: invalid argument" err.message="unknown error" go.version=go1.7.6

readdirent: invalid argument is the only info I have to go on.

Reproduction:

• Set up the NFS provisioner, set up the kubernetes image registry backed by a dynamic persistent volume that the provisioner sets up.
• Push an image to your registry (it's through the nginx ingress for us if that matters)

I had other "Invalid Argument" errors, but switching to NFS 3 fixed them. This one still remains. Something about how the registry writes or reads files I guess.

[Paxa]

Having same problem, how did you switch to NFS 3?

[abrenneke]

@Paxa by editing the StorageClass like so:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: etc
parameters:
  mountOptions: vers=3
provisioner: etc

[wongma7]

See #762 (comment) for my comment on these hard-to-debug problems

[Queuecumber]

It might be a good idea to add this to the official storage class yaml (https://github.com/kubernetes-incubator/external-storage/blob/master/nfs/deploy/kubernetes/class.yaml). None of my pods were able to write until I added vers=3 to the mount options

[LoicMahieu]

Maybe related to https://serverfault.com/a/724463

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

[k8s-ci-robot]

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.