Skip to content
This repository has been archived by the owner on Aug 27, 2023. It is now read-only.

gcs client is not threadsafe #258

Closed
camerondavison opened this issue Sep 3, 2020 · 2 comments
Closed

gcs client is not threadsafe #258

camerondavison opened this issue Sep 3, 2020 · 2 comments

Comments

@camerondavison
Copy link

I am seeing

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 670, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 426, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 421, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/local/lib/python3.8/http/client.py", line 1347, in getresponse
    response.begin()
  File "/usr/local/lib/python3.8/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/local/lib/python3.8/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/local/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
  File "/usr/local/lib/python3.8/ssl.py", line 1241, in recv_into
    return self.read(nbytes, buffer)
  File "/usr/local/lib/python3.8/ssl.py", line 1099, in read
    return self._sslobj.read(len, buffer)
ssl.SSLError: [SSL: DECRYPTION_FAILED_OR_BAD_RECORD_MAC] decryption failed or bad record mac (_ssl.c:2622)

when trying to upload a package.

Looks like it is known that the client is not thread safe googleapis/google-cloud-python#3501

Could be wrong but it looks like there is one shared client for everything.

@stevearc
Copy link
Owner

stevearc commented Sep 8, 2020

Thanks for pointing this out! I've never used GCS myself, so I don't have a good way to test this. I pushed up a change to a branch that might fix the issue. Any chance you could try it out? https://github.com/stevearc/pypicloud/commits/gcs-threadsafe

@camerondavison
Copy link
Author

camerondavison commented Sep 12, 2020

I upgraded my docker image by running

pip install -U git+https://github.com/stevearc/pypicloud.git@gcs-threadsafe

and did not run into any of the problems above. so LGTM

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants