Cannot unmarshal DNS message

[raintonr]

Just today I started getting this error on push:

Get https://www.googleapis.com/drive/v2/files?alt=json&maxResults=1&q=%22root%22+in+parents+and+title+%3D+%22Pictures%22+and+trashed%3Dfalse: dial tcp: lookup www.googleapis.com on 10.1.109.1:53: cannot unmarshal DNS message

Note the DNS server on my LAN (10.1.109.1) does properly resolve www.googleapis.com:

$ dig @10.1.109.1 www.googleapis.com

; <<>> DiG 9.10.2-P3-RedHat-9.10.2-4.P3.fc22 <<>> @10.1.109.1 www.googleapis.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 12113
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 13, ADDITIONAL: 13

;; QUESTION SECTION:
;www.googleapis.com.        IN  A

;; ANSWER SECTION:
www.googleapis.com. 2347    IN  CNAME   googleapis.l.google.com.
googleapis.l.google.com. 171    IN  A   74.125.136.95

;; AUTHORITY SECTION:
com.            142080  IN  NS  i.gtld-servers.net.
com.            142080  IN  NS  a.gtld-servers.net.
com.            142080  IN  NS  m.gtld-servers.net.
com.            142080  IN  NS  k.gtld-servers.net.
com.            142080  IN  NS  l.gtld-servers.net.
com.            142080  IN  NS  g.gtld-servers.net.
com.            142080  IN  NS  h.gtld-servers.net.
com.            142080  IN  NS  f.gtld-servers.net.
com.            142080  IN  NS  b.gtld-servers.net.
com.            142080  IN  NS  c.gtld-servers.net.
com.            142080  IN  NS  e.gtld-servers.net.
com.            142080  IN  NS  j.gtld-servers.net.
com.            142080  IN  NS  d.gtld-servers.net.

;; ADDITIONAL SECTION:
i.gtld-servers.net. 142080  IN  A   192.43.172.30
a.gtld-servers.net. 142080  IN  A   192.5.6.30
m.gtld-servers.net. 142080  IN  A   192.55.83.30
k.gtld-servers.net. 142080  IN  A   192.52.178.30
l.gtld-servers.net. 142080  IN  A   192.41.162.30
g.gtld-servers.net. 142080  IN  A   192.42.93.30
h.gtld-servers.net. 142080  IN  A   192.54.112.30
f.gtld-servers.net. 142080  IN  A   192.35.51.30
b.gtld-servers.net. 142080  IN  A   192.33.14.30
c.gtld-servers.net. 142080  IN  A   192.26.92.30
e.gtld-servers.net. 142080  IN  A   192.12.94.30
j.gtld-servers.net. 142080  IN  A   192.48.79.30
d.gtld-servers.net. 142080  IN  A   192.31.80.30

;; Query time: 4 msec
;; SERVER: 10.1.109.1#53(10.1.109.1)
;; WHEN: Sat Aug 29 11:33:09 CEST 2015
;; MSG SIZE  rcvd: 518

Note the message size is 518 bytes.

If we try Google's public DNS we get a much shorter reply:

dig @8.8.8.8 www.googleapis.com

; <<>> DiG 9.10.2-P3-RedHat-9.10.2-4.P3.fc22 <<>> @8.8.8.8 www.googleapis.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 1655
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;www.googleapis.com.        IN  A

;; ANSWER SECTION:
www.googleapis.com. 3599    IN  CNAME   googleapis.l.google.com.
googleapis.l.google.com. 299    IN  A   74.125.136.95

;; Query time: 35 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Sat Aug 29 11:37:28 CEST 2015
;; MSG SIZE  rcvd: 97

Doing a quick search I find some errors regarding the Go language not being able to handle large DNS replies. See hashicorp/consul#854

I'm logging this here in case anyone else using drive sees similar issue and wants to know how to fix it.

[odeke-em]

Hello again @raintonr thanks for reporting this issue. I haven't yet encountered it but that was a good read. It seems to suggest a non compliance given that the UDP DNS response could be greater than 512 bytes yet doesn't cause the truncate flag to be set. An issue was filed against the Go repo golang/go#11070 and then in a different repo weaveworks/weave you can see a fix weaveworks/weave@7fb597a.

[odeke-em]

Hello @raintonr, please get the latest from master. I assume this error is retryable, if so PR #395 will help mitigate it by retrying as much until all the tries are exhausted. To get the latest from master, please do

$ go get -v -u github.com/odeke-em/drive/drive-gen && drive-gen

[raintonr]

Well, interestingly I now get:

$ dig @10.1.109.1 www.googleapis.com

; <<>> DiG 9.10.2-P4-RedHat-9.10.2-5.P4.fc22 <<>> @10.1.109.1 www.googleapis.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 42833
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;www.googleapis.com.        IN  A

;; ANSWER SECTION:
www.googleapis.com. 1095    IN  CNAME   googleapis.l.google.com.
googleapis.l.google.com. 95 IN  A   216.58.208.234

;; Query time: 2 msec
;; SERVER: 10.1.109.1#53(10.1.109.1)
;; WHEN: Wed Sep 30 13:01:29 CEST 2015
;; MSG SIZE  rcvd: 86

... so this appears to have 'gone away' on it's own so cannot reproduce it prior to testing the fix. Sorry :(

[odeke-em]

Yeah unfortunately this is an issue that maybe if it returns, you can open on the golang issues page as drive has no control over such.
Thanks for reporting this nonetheless.

[chandankrishnan]

cannot unmarshal DNS message in heroku...How resolve this problem??

Post https://api.heroku.com/login: dial tcp: lookup api.heroku.com on 127.0.1.1:55: cannot unmarshal DNS message