TypeError: string indices must be integers when using bucket.exists() in Google Cloud Code

[Ark-kun]

Thanks for stopping by to let us know something could be better!

PLEASE READ: If you have a support contract with Google, please create an issue in the support console instead of filing on GitHub. This will ensure a timely response.

Please run down the following list and make sure you've tried the usual "quick fixes":

• Search the issues already opened: https://github.com/googleapis/google-auth-library-python/issues

If you are still having issues, please be sure to include as much information as possible:

Environment details

• OS:
• Python version: 3.9
• google-auth version: google-auth==2.9.1

Steps to reproduce

1. Go to Google's official Storage samples: https://github.com/googleapis/python-storage/tree/main/samples
2. Click on the "List buckets" sample: https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/python-storage&page=editor&open_in_editor=samples/snippets/storage_list_buckets.py
3. Try to run sample.

(Cloud Code was in ephemeral mode)

Traceback (most recent call last):
  File "/home/avolkov_ota/cloudshell_open/python-storage/samples/snippets/storage_list_buckets.py", line 35, in <module>
    list_buckets()
  File "/home/avolkov_ota/cloudshell_open/python-storage/samples/snippets/storage_list_buckets.py", line 27, in list_buckets
    for bucket in buckets:
  File "/usr/local/lib/python3.9/dist-packages/google/api_core/page_iterator.py", line 208, in _items_iter
    for page in self._page_iter(increment=False):
  File "/usr/local/lib/python3.9/dist-packages/google/api_core/page_iterator.py", line 244, in _page_iter
    page = self._next_page()
  File "/usr/local/lib/python3.9/dist-packages/google/api_core/page_iterator.py", line 373, in _next_page
    response = self._get_next_page_response()
  File "/usr/local/lib/python3.9/dist-packages/google/api_core/page_iterator.py", line 432, in _get_next_page_response
    return self.api_request(
  File "/home/avolkov_ota/.local/lib/python3.9/site-packages/google/cloud/storage/_http.py", line 73, in api_request
    return call()
  File "/usr/local/lib/python3.9/dist-packages/google/api_core/retry.py", line 283, in retry_wrapped_func
    return retry_target(
  File "/usr/local/lib/python3.9/dist-packages/google/api_core/retry.py", line 190, in retry_target
    return target()
  File "/usr/local/lib/python3.9/dist-packages/google/cloud/_http/__init__.py", line 482, in api_request
    response = self._make_request(
  File "/usr/local/lib/python3.9/dist-packages/google/cloud/_http/__init__.py", line 341, in _make_request
    return self._do_request(
  File "/usr/local/lib/python3.9/dist-packages/google/cloud/_http/__init__.py", line 379, in _do_request
    return self.http.request(
  File "/home/avolkov_ota/.local/lib/python3.9/site-packages/google/auth/transport/requests.py", line 545, in request
    self.credentials.before_request(auth_request, method, url, request_headers)
  File "/home/avolkov_ota/.local/lib/python3.9/site-packages/google/auth/credentials.py", line 133, in before_request
    self.refresh(request)
  File "/home/avolkov_ota/.local/lib/python3.9/site-packages/google/auth/compute_engine/credentials.py", line 111, in refresh
    self._retrieve_info(request)
  File "/home/avolkov_ota/.local/lib/python3.9/site-packages/google/auth/compute_engine/credentials.py", line 91, in _retrieve_info
    self._service_account_email = info["email"]
TypeError: string indices must be integers

Making sure to follow these steps will guarantee the quickest resolution possible.

Thanks!

[clundin25]

I took a look and it seems the issue is that get() may return a JSON or string here and credentials.py does not guard against this here.

I am not sure why your request received a different content-type but I was unable to reproduce it. I only observed a text/plain response if I added &alt=text without the recursive flag.

I think the next steps are to add a check that info is a dictionary and throw an error if it is not. I also want to dig into why the metadata server did not send JSON in this case.

[tswast]

I was investigating the BigQuery client libraries with @chalmerlowe and we observe this on Google Cloud Shell as well. I suspect Cloud Shell may be emulating the metadata server, but not as accurately as needed.

[chalmerlowe]

I had a very similar traceback.
As Tim noted: was running my code on the Google Cloud Shell.

  File "/home/chalmerlowe/.local/lib/python3.9/site-packages/pydata_google_auth/auth.py", line 99, in default
    credentials, default_project = get_application_default_credentials(scopes)
  File "/home/chalmerlowe/.local/lib/python3.9/site-packages/pydata_google_auth/auth.py", line 148, in get_application_default_credentials
    credentials.refresh(request)
  File "/usr/local/lib/python3.9/dist-packages/google/auth/compute_engine/credentials.py", line 111, in refresh
    self._retrieve_info(request)
  File "/usr/local/lib/python3.9/dist-packages/google/auth/compute_engine/credentials.py", line 91, in _retrieve_info
    self._service_account_email = info["email"]
TypeError: string indices must be integers

[daniel-goldstein]

I suspect Cloud Shell may be emulating the metadata server, but not as accurately as needed.

While I don't have a reproduction it looks like this code expects an overly-restrictive Content-Type header. It is expecting there to only be a mime-type, but web servers commonly also add a charset directive. A response that included Content-Type: application/json; charset=utf-8 (the default in aiohttp and I suspect many other web servers) would not be properly deserialized and result in this exact error.

[DanieleQuasimodo]

I suspect Cloud Shell may be emulating the metadata server, but not as accurately as needed.

While I don't have a reproduction it looks like this code expects an overly-restrictive Content-Type header. It is expecting there to only be a mime-type, but web servers commonly also add a charset directive. A response that included Content-Type: application/json; charset=utf-8 (the default in aiohttp and I suspect many other web servers) would not be properly deserialized and result in this exact error.

The problem was exactly that, thanks fo the suggestion.

This should fix it: #1382
It's far from the most elegant solution but should do the trick avoiding regexes and third-party libraries.

[arithmetic1728]

The fix #1382 is merged. Closing the bug. Thank you @DanieleQuasimodo!

[dslans]

I still hit this error on cloud shell when trying to query with the bigquery client

TypeError: string indices must be integers

[arithmetic1728]

@dslans could you file a new bug with the details such as pip freeze output, a simple repro example, and the stack trace etc. Thanks!

[alinh99]

I still hit this error on cloud shell when trying to query with the bigquery client

TypeError: string indices must be integers

Have you fixed this bug yet?

[abhinigam]

I am still hitting this error with bigquery client in google cloud shell.
@alinh99 @dslans have you found a workaround to this issue?

[abhinigam]

$ pip freeze
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
astroid==1.6.6
backports.functools-lru-cache==1.6.6
cachetools==3.1.1
certifi==2021.10.8
chardet==4.0.0
configparser==4.0.2
crcmod==1.7
enum34==1.1.10
futures==3.4.0
google-api-core==1.32.0
google-auth==1.35.0
google-cloud-bigquery==1.28.3
google-cloud-core==1.7.3
google-resumable-media==1.3.3
googleapis-common-protos==1.52.0
idna==2.10
isort==4.3.21
lazy-object-proxy==1.6.0
mccabe==0.6.1
packaging==20.9
protobuf==3.17.3
pyasn1==0.5.1
pyasn1-modules==0.3.0
pylint==1.9.5
pyparsing==2.4.7
pytz==2024.2
requests==2.27.1
rsa==4.5
singledispatch==3.7.0
six==1.16.0
urllib3==1.26.20
wrapt==1.15.0