httplib2.ServerNotFoundError: Unable to find the server at www.googleapis.com

[RaveGun]

Please add info about your configuration here, along with a brief description of what you were doing and what happened. Detail is always helpful for investigating an error. You can enable verbos logging by setting {"verbose": true} in your add-on configuration and including that here. :

Traceback (most recent call last):
File "/app/backup/engine.py", line 92, in doBackupWorkflow
self._checkForBackup()
File "/app/backup/engine.py", line 277, in _checkForBackup
self._syncSnapshots()
File "/app/backup/engine.py", line 214, in _syncSnapshots
self.folder_id = self.drive.getFolderId()
File "/app/backup/drive.py", line 193, in getFolderId
return self._createDriveFolder()
File "/app/backup/drive.py", line 201, in _createDriveFolder
folder = self._retryDriveServiceCall(self._drive().files().create(body=file_metadata, fields='id'))
File "/app/backup/drive.py", line 65, in _drive
return build(DRIVE_SERVICE, DRIVE_VERSION, credentials=self.creds)
File "/usr/lib/python3.6/site-packages/googleapiclient/_helpers.py", line 130, in positional_wrapper
return wrapped(*args, **kwargs)
File "/usr/lib/python3.6/site-packages/googleapiclient/discovery.py", line 224, in build
requested_url, discovery_http, cache_discovery, cache, developerKey)
File "/usr/lib/python3.6/site-packages/googleapiclient/discovery.py", line 274, in _retrieve_discovery_doc
resp, content = http.request(actual_url)
File "/usr/lib/python3.6/site-packages/httplib2/init.py", line 1926, in request
cachekey,
File "/usr/lib/python3.6/site-packages/httplib2/init.py", line 1595, in _request
conn, request_uri, method, body, headers
File "/usr/lib/python3.6/site-packages/httplib2/init.py", line 1508, in _conn_request
raise ServerNotFoundError("Unable to find the server at %s" % conn.host)
httplib2.ServerNotFoundError: Unable to find the server at www.googleapis.com

[RaveGun]

Hi,

It looks like none of the Google Drive add-ons out there are working.

Actually if I try to go to the http://www.googleapis.com/ I am getting:
NotFound

Interesting, it might have to do something with my google account.
I have no idea what it could be.

Thank you.

[sabeechen]

The GSuite Status Page usually has information about drive-wide outtages, but I don't see anything from the time period you posted. googleapis.com isn't a website, its just the address used for the Google Drive api so getting an error from a web browser is expected, I think.

Did the error go away after a while? It could have just been a temporary connectivity problem. If its a problem with your account, you could try the "reauthenticate" button to re-link your drive account.

I'll keep this open, since I should really surface a better error message if this happens.

[RaveGun]

Hi,
I managed to test this several times without success.
I also tried and created a new application and used the second option but it didn't work either. I did this on my phone so the error was thrown as a toast notification but I could see that it was related to oauth2.googleapi.com. This was after I got and entered the authentication code.

I would like to mention that a long time ago I created another application that I am using right now to get the calendar information in the home assistant. This works but I am seeing from time to time in the log that it was not able to query the same Google's API URL.

Maybe it has to do something with my developer account or some strange security settings.

I might try to reuse the above mentioned application and to enable the drive feature.

Thanks for the feedback.

[sabeechen]

I haven't heard of any other users having authentication problems so I also suspect it has to do with your account. It might be worthwhile to make sure "Hass.io Google Drive Backup" shows up under "Third Party Apps with account access" under your google security settings.
You could also try creating another empty @gmail.com account and see if you can hook that one up to the add-on, that would tell for sure if the problem is with your account.

[gdreelin]

I have been getting this exact same error.

[sabeechen]

I posted this in the issue gdreedlin created, but it might help you too:
I randomly stumbled into this issue, which present similarly to what you're seeing. It suggest that giving your Home Assistance instance a static IP might fix the problem, or that your router might be using a stale DNS server.

[gdreelin]

I posted this in the issue gdreedlin created, but it might help you too:
I randomly stumbled into this issue, which present similarly to what you're seeing. It suggest that giving your Home Assistance instance a static IP might fix the problem, or that your router might be using a stale DNS server.

I currently have a static IP on HA. I manually assign all my systems with IPs only guests get DHCP.

[gdreelin]

Strangely enough it seemed to work the issue out, not sure why I just checked it again an all is working good.

[sabeechen]

My only guess is that something is wrong with your DNS configuration, but DNS problems are never easy to sort out. I'm glad it resolved out for you.

[sabeechen]

I intend to keep this open in case other people run into the same trouble and look for help, maybe down the line someone will be able to sort out what the root cause is.

[jmart518]

I too am having this issue. Tried uninstalling, reinstalling and no dice

httplib2.ServerNotFoundError: Unable to find the server at www.googleapis.com

[gdreelin]

After a few of your updates it just stopped. All is good now.

[sabeechen]

I just released v0.94, it gives a link in the big red error dialog to show the IP addresses that Google's API servers resolve to from inside the add-on's docker container. It also shows the result of trying to ping each IP address. If this is some kind of DNS resolution issue on your network, that should help to sort it out.
I'm still unable to reproduce this, even once, on any of the machines I'm using for testing :(

[nikiyao]

I'm having the same issue. It was working now. Today it stopped, I've tried restarting, unistalling, reauthorizing. I can't figure it out.

[sabeechen]

This is starting to keep me up at night :)

The more I dig into this the more it looks to me like we aren't the only ones running into this problem with Google's python libraries. The add-on uses Google's provided python libraries to talk to Google Drive, which use Python's httplib2 under the hood. It seems like like httplib2 might have some strange interaction with how docker handles DNS (the add-on runs inside a docker container Hassio creates for it). The error reports I'm seeing come in with this problem all indicate that google servers are responding to pings just fine, which further lends credibility to that idea, since since the add-on gets those results from a separate command line call avoiding the whole python network stack.

One possible solution I've considered is avoiding Google's libraries altogether and communicating with Drive using regular python3 http requests. That would be a significant rewrite of a big part of the add-on's logic where I'd potentially just end up with the same problem, but I'll try to get sense of how much work it is in reality.

[sabeechen]

Just super weird that it comes and goes seemingly randomly. I can see in the error reports that for most it resolves within an hour, sometimes after a few hours, and for some it never gets through. This is the worst kind of issue to debug.

[sabeechen]

If anyone seeing this error is willing to muck around with their home network's DNS settings, I'd be curious to know if it goes away after configuring your router to use Google's Publis DNS servers (8.8.8.8 and 8.8.4.4) and restarting the machine Home Assistant is running on.

[nikiyao]

So for me at least I may have found the issue. I think I'm having a problem resolving local Host names. Is there anyway to make it call for the local IP address instead?

[dhzl84]

I also have an issue resolving DNS.
I have AdGuard running and can see that a bunch of IPs is returned to the hassio network (172.30.33.7) for oauth2.googleapis.com and www.googleapis.com but the addon doesn't seem to see them.

Strange thing is, as soon as I switch over to Pi-Hole the DNS issue is gone, so maybe AdGuard related?

[nikiyao]

That's weird because I run pihole and that's my issue.

[jmart518]

@sabeechen I went ahead and changed my router DNS to 8.8.8.8 and 8.8.4.4, disabled adguard, and hard coded my hassOS box to 8.8.8.8/8.8.4.4 for good measure.

Still getting the same error

[sabeechen]

@jmart518 Thanks for giving it a shot. Did you restart the Home Assistant machine too? DNS caches can stick around for a while, especially through all the layers Docker puts between a container and host environment.

[jmart518]

@sabeechen I sure did. First tried a simple HA restart and then did a full hassOS reboot

[frigidplatypus]

I am facing the same DNS problems. I've restarted HA, the full system, and the add-on many times.

httplib2.ServerNotFoundError: Unable to find the server at oauth2.googleapis.com

[RaveGun]

Hi,

Sorry it took so long.
So I did some test on the dockers containers.

1. in the console from the Hass.io Google Drive Backup I got the following:
   bash-4.4# ping www.googleapis.com
   ping: bad address 'www.googleapis.com'

bash-4.4# ping oauth2.googleapis.com
ping: bad address 'oauth2.googleapis.com'

bash-4.4# ping 192.168.1.1
PING 192.168.1.1 (192.168.1.1): 56 data bytes
64 bytes from 192.168.1.1: seq=0 ttl=63 time=0.655 ms
64 bytes from 192.168.1.1: seq=1 ttl=63 time=0.407 ms
64 bytes from 192.168.1.1: seq=2 ttl=63 time=0.487 ms
64 bytes from 192.168.1.1: seq=3 ttl=63 time=0.485 ms
^C
--- 192.168.1.1 ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max = 0.407/0.508/0.655 ms
bash-4.4# ping 192.168.1.90
PING 192.168.1.90 (192.168.1.90): 56 data bytes
64 bytes from 192.168.1.90: seq=0 ttl=64 time=0.245 ms
64 bytes from 192.168.1.90: seq=1 ttl=64 time=0.285 ms
64 bytes from 192.168.1.90: seq=2 ttl=64 time=0.221 ms
^C
--- 192.168.1.90 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 0.221/0.250/0.285 ms
So there seems to be no DNS resolver available at all.

2. On the homeassistant container I get the following:
   bash-4.4# ping www.googleapis.com
   PING www.googleapis.com (216.58.208.42): 56 data bytes
   64 bytes from 216.58.208.42: seq=0 ttl=53 time=23.993 ms
   64 bytes from 216.58.208.42: seq=1 ttl=53 time=27.369 ms
   64 bytes from 216.58.208.42: seq=2 ttl=53 time=23.814 ms
   64 bytes from 216.58.208.42: seq=3 ttl=53 time=21.890 ms
   ^C
   --- www.googleapis.com ping statistics ---
   4 packets transmitted, 4 packets received, 0% packet loss
   round-trip min/avg/max = 21.890/24.266/27.369 ms

bash-4.4# ping oauth2.googleapis.com
PING oauth2.googleapis.com (2a00:1450:4001:815::200a): 56 data bytes
ping: sendto: Address not available

bash-4.4# ping 192.168.1.1
PING 192.168.1.1 (192.168.1.1): 56 data bytes
64 bytes from 192.168.1.1: seq=0 ttl=64 time=0.654 ms
64 bytes from 192.168.1.1: seq=1 ttl=64 time=0.389 ms
^C
--- 192.168.1.1 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0.389/0.521/0.654 ms

bash-4.4# ping 192.168.1.90
PING 192.168.1.90 (192.168.1.90): 56 data bytes
64 bytes from 192.168.1.90: seq=0 ttl=64 time=0.249 ms
64 bytes from 192.168.1.90: seq=1 ttl=64 time=0.152 ms
^C
--- 192.168.1.90 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0.152/0.200/0.249 ms
bash-4.4#

So it seems that the oauth2 is getting a IPv6 only and this is not enabled on my network.
Since I am using a router that is know for having IPv6 security issues I cannot enable this for the time being.

I hope this helps with the debugging.

Have a great day.

[dhzl84]

I did some back and forth testing with Pi-Hole and AdGuard and observed the following behavior.
If DNS resolution fails, there is a red box drawn in the AddOns UI showing some DNS debug data.

Strange thing is, that I also find exactly these letters in my DNS query log of AdGuard, so it seems that the AddOn wants to resolve those:

P.S. If anyone knows how to collapse images in Markdown I'd appreciate feedback.

[sabeechen]

@dhzl84 what you're seeing in the error dialog is a parsing error I made getting the DNS info to display there, its actually treating every character in the string "Failed to resolve host" as if it were a separate ip address. I'll be fixing it in a release later today.

@RaveGun Thats getting somewhere! I don't think it would explain everything in this thread but at least I can do something about it. I'll make a release later today with an option to ignore ipv6 addresses.

[sabeechen]

I've made a new release, 0.96, that includes a bunch of stuff to help resolve this issue. Please follow the instructions below if you'd like to give it a shot.

1. Ensure you're running addon verison 0.96 (or higher). You may need to refresh the add-on store page (icon in the top right) for it to see the update.
2. Add these two options to your add-on configuration
   • "drive_experimental": true
   • "ignore_ipv6_addresses": true
     This will enable a new library I wrote for communicating with Google Drive that ignores IPv6 addresses. Eventually I'd like to make this the default option, but I want to ensure actually fixes the problem with you guys before I witch everyone over with an update. Everything should work, but errors won't be as user friendly yet, you'll just see the raw exception text if problems come up. You'll need to set these through the hass.io addon config page, I haven't made support in the settings UI yet. I try to test thoroughly, but because its a lot of new code there could be new issues with it.
3. Restart the add-on. Hopefully, you'll no longer see DNS errors when it tries to sync snapshots. Please let me know if it does work, so I can start the work the enable this for everyone. I'll sleep much better at night if I can put this bug to rest as well.
4. If you're still getting the error, you can try setting "drive_ipv4": "172.217.1.202" in the add-on config and restarting the add-on. This will hard-code the DNS resolution for Google's API servers. "172.217.1.202" is one of the addresses that www.googleapis.com resolves to on my network, you can lookup other ipv4 addresses that should work using a tool like this one.

[jmart518]

Thanks for the update, I added the two new lines and received a new error:

05-05 21:20:25 ERROR Traceback (most recent call last): File "/app/backup/engine.py", line 162, in doBackupWorkflow self._checkForBackup() File "/app/backup/engine.py", line 401, in _checkForBackup self._syncSnapshots() File "/app/backup/engine.py", line 332, in _syncSnapshots self.folder_id = self.drive.getFolderId() File "/app/backup/drive.py", line 129, in getFolderId folder = self._get(folder_id) File "/app/backup/drive.py", line 145, in _get return self.drivebackend.get(id) File "/app/backup/driverequests.py", line 128, in get return self.retryRequest("GET", URL_FILES + id + "/?" + urlencode(q), is_json=True) File "/app/backup/driverequests.py", line 216, in retryRequest response = request(method, url, headers=send_headers, json=json, timeout=(30, 30), data=data, stream=stream) File "/usr/lib/python3.6/site-packages/requests/api.py", line 60, in request return session.request(method=method, url=url, **kwargs) File "/usr/lib/python3.6/site-packages/requests/sessions.py", line 533, in request resp = self.send(prep, **send_kwargs) File "/usr/lib/python3.6/site-packages/requests/sessions.py", line 646, in send r = adapter.send(request, **kwargs) File "/usr/lib/python3.6/site-packages/requests/adapters.py", line 516, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='www.googleapis.com', port=443): Max retries exceeded with url: /drive/v3/files/<redacted>?fields=id%2Cname%2CappProperties%2Csize%2Ctrashed%2CmimeType%2CmodifiedTime%2Ccapabilities (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7fa0b8249828>: Failed to establish a new connection: [Errno -3] Try again',))

However, adding the last line:
"drive_ipv4": "172.217.1.202"

Appears to have resolved everything for me, just don't know how permanent/reliable the hard IP fix will be.

[r3mcos3]

For me to
"drive_ipv4": "172.217.1.202"
Has resolved the problem

[RaveGun]

Hi,

Thank you for the update.

The predefined drive IPv4 worked on my side too.

[dhzl84]

Hi, tested it with 0.96.1 but no success with DNS, only hardcoded IPv4 works for me.
Anything else I could test for you?

[sabeechen]

Bummer, I was really really hoping this was just an ipv6 problem but its encouraging that the hard-coding works. Are you getting the same error @jmart518 posted (copied below)?
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='www.googleapis.com', port=443): Max retries exceeded with url: <long url> (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7fa0b8249828>: Failed to establish a new connection: [Errno -3] Try again',)):

[r3mcos3]

Yes for me it was the same error

[Bluhme1]

Me too. Same error. But hardcoding works

[dhzl84]

I can also confirm the same error message without hardcoded IPv4.

[r3mcos3]

Did a clean install off HA and the addon without the above mentioned fixes and everything works fine.

[BertrumUK]

I was getting the same issue until I added

"drive_ipv4": "172.217.1.202"

Thank you for your work on this excellent addon

[WildcardTech]

I had to add
"drive_experimental": true,
"ignore_ipv6_addresses": true,
"drive_ipv4": "172.217.1.202",

for it to work

[sabeechen]

In the latest version (v0.97) I've made the new Drive library the default, so the "experimental_drive" option no longer has any effect. I've also added failover to known good DNS servers (Google's public DNS) if the add-on runs into problems trying to resolve www.googleapis.com.

The add-on will still respect the drive_ipv4 setting if you have it set, but I suspect that you guys should be able to remove it and just make use of the DNS failover I implemented, which will also keep you up-to-date if google updates their DNS. You might see an error message still show up in the UI for a few minutes if you do this, since it will take at least on failed attempt before using the failover DNS. These options can now also be changed from the settings UI and take effect immediately.

If you still run into problems with this update, I'd be interested in getting a copy of your add-on's debug logs (link in the top right).

[dhzl84]

Nice to hear, I gave it a go right away.
I used the DNS failover option to point to my local DNS resolver instead of google.
Until now everything runs pretty smooth after updating.
Any chance to see when the DNS failover kicks in?

[sabeechen]

Not in the UI, but in the debug logs (link at the top right of the web UI) you'll see a log message that reads

Ran into trouble reaching Google Drive's servers.  We'll use alternate DNS servers on the next attempt.

[dhzl84]

So I made a manual backup and restarted the addon. No DNS failover until now! thumbs up

[sabeechen]

I haven't heard of any problems with the latest version (which I think is a good thing) so I'm going to close this issue for now. If anyone is still running into problem, feel free to reopen this and I'll look into it further.

[Padre653]

05-23 10:41:21 INFO Syncing Snapshots
05-23 10:41:21 DEBUG Making Hassio request: http://hassio/snapshots
05-23 10:41:21 DEBUG Making Google Drive request: https://www.googleapis.com/drive/v3/files/1LV8USkmPT9AkGGJ8xceqS_bDN0lXUMfk/?fields=id%2Cname%2CappProperties%2Csize%2Ctrashed%2CmimeType%2CmodifiedTime%2Ccapabilities
05-23 10:41:26 DEBUG Ran into trouble reaching Google Drive's servers. We'll use alternate DNS servers on the next attempt.
05-23 10:41:26 ERROR Unable to connect to www.googleapis.com
05-23 10:41:26 INFO Another attempt to sync will be made in 3600 seconds

It looks like the application is trying to resolve to DNS even though I have manually specified an IP address.

"max_snapshots_in_hassio": 4,
"max_snapshots_in_google_drive": 3,
"days_between_snapshots": 7,
"use_ssl": false,
"snapshot_time_of_day": "03:30",
"snapshot_password": "!secret snapshot_password",
"send_error_reports": false,
"ignore_ipv6_addresses": true,
"drive_ipv4": "172.217.9.42"

I'm running 0.97.1

[Bluhme1]

Same here. Worked before. Is there a problem with 0.97.1?

[dhzl84]

For me 0.97.1 still works as flawless as 0.97, no indication of failed DNS.
I do not use:
"ignore_ipv6_addresses": true,
"drive_ipv4": "172.217.9.42"

[sabeechen]

Must have broken something in how this is configured, I'll do some digging.

[sabeechen]

Yup, I accidentally made it read the setting incorrectly. I've fixed it in 0.97.3 (just released).

[Padre653]

That fixed it, thanks!

[Meshram-Vibhor]

Go to your router setting and check if the if the DNS is 8.8.8.8 . If it's not add secondary DNS as 8.8.8.8 this has worked for me.