problem encountered while using HostNetworkDNSPolicy

[david-enli]

Hi all, first time posting and apologize if doing it wrong. With the new release of k8s 1.16.10 , our pure storage CSI node plugin (pods) for iscsi deployed by DaemonSet start to have issue to talk to K8S service..

We deploy CSI node plugin pod on each master and worker nodes. However, it looks like network routing on worker nodes is problematic.
It looks like master have correct routing to our internal k8s service:

# nslookup pso-db-public.nstk
Server:    10.96.0.10
Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local
Name:      pso-db-public.nstk
Address 1: 10.102.98.125 pso-db-public.nstk.svc.cluster.local

but the worker nodes s the problem:

/ # nslookup pso-db-public.nstk
Server:    10.96.0.10

Here is our Demonset pod network and DNS setting:

hostNetwork: true
dnsPolicy: ClusterFirstWithHostNet

Since first discovered this issue, we've been hitting it in 1.17.1, 1.17.5, 1.17.6. Still trying out more versions but it looks like this wasn't fixed since 1.16.10, thank you very much for your help.

[david-enli]

/sig network

[athenabot]

/triage unresolved

Comment /remove-triage unresolved when the issue is assessed and confirmed.

🤖 I am a bot run by vllry. 👩‍🔬

[wangyira]

/assign @rikatz

[rikatz]

Hey @david-enli let's take a look into that.

I didn't understood the first part of the issue, when you do a nslookup from the nodes they don't resolve correctly the address?

Also, can you please provide some further information:

• Kubernetes version (use kubectl version):
• Cloud provider or hardware configuration:
• OS (e.g: cat /etc/os-release):
• Kernel (e.g. uname -a):
• Install tools:
• Network plugin (CNI) and version (if this is a network-related bug):
• Others:

Thank you

[rikatz]

/triage needs-information

[rikatz]

@david-enli friendly ping

[athenabot]

@rikatz
If this issue has been triaged, please comment /remove-triage unresolved.

If you aren't able to handle this issue, consider unassigning yourself and/or adding the help-wanted label.

🤖 I am a bot run by vllry. 👩‍🔬

[rikatz]

/remove-triage unresolved

[david-enli]

hi @rikatz I sincerely apologize for the delay, this was logged using a different git account and I missed your previous pings. In the meantime, my team member came accross this open issue flannel-io/flannel#1243 and we have adopted one of the solutions provided in the comments: Logging into the node and turn off flannel device check sum by running: ethtool -K flannel.1 tx-checksum-ip-generic off

This issue has present for us since 1.16.10 and it was on both Ubunto and Centos enviornment. The network plugin is flannel. Again, apologize if this not the right place for the issue.

[rikatz]

@david-enli no problem :D

I was trying to figure out if this was also related with #88986 and it seems to be. There's a PR already merged for this here #92035 and a good explanation also why this happens.

As this is a dup and already solved, I'll close this issue but please feel free to re-open if you think there's a need to deal with something else

Tks

/close

[k8s-ci-robot]

@rikatz: Closing this issue.

In response to this:

@david-enli no problem :D

I was trying to figure out if this was also related with #88986 and it seems to be. There's a PR already merged for this here #92035 and a good explanation also why this happens.

As this is a dup and already solved, I'll close this issue but please feel free to re-open if you think there's a need to deal with something else

Tks

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.