Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add retry to _metadata.ping() #323

Merged
merged 1 commit into from
Feb 15, 2019
Merged

Add retry to _metadata.ping() #323

merged 1 commit into from
Feb 15, 2019

Conversation

rmbarron
Copy link
Contributor

This is to address #211.

I was able to reproduce the issue in App Engine Flex - my repro is an autoscaling application with minimum 1 instance and max 10 instances that makes calls to google.auth.default() on GET requests. Using apache benchmark to scale it up, there was a brief period of time '[Errno 111] Connection Refused' would be intermittently thrown (during the scale up event). Adding this retry has resolved this for me.

This has been tested manually. Please let me know if there are any integration tests I should run first, though this change should be low risk and largely invisible to all users of _metadata.py.

Many Thanks!

@googlebot googlebot added the cla: yes This human has signed the Contributor License Agreement. label Feb 13, 2019
@theacodes
Copy link
Contributor

The reason we previously did not do this is because we want to prevent the program from hanging for a long time while attempting to contact the metadata service. How long will this hang before raising a fatal error?

@rmbarron
Copy link
Contributor Author

The default timeout is already 3 seconds. So adding a retry default of 3, the worst case scenario of timeout in this lib is 9 seconds.

The average case is much quicker - in my reproduction, the fatal error would be thrown within a second of a successful one, and another would succeed within the same second. If desired, I can gather deeper specifics - stackdriver likes to group stdout logs as an aggregate timestamp in 1 second intervals.

@theacodes
Copy link
Contributor

Yeah this seems fine. Thanks!

@theacodes theacodes merged commit 7a62768 into googleapis:master Feb 15, 2019
bors bot referenced this pull request in mozilla/normandy Feb 26, 2019
@bors @pyup-bot @mythmon
Merge #1760
f47decc 
1760: Scheduled weekly dependency update for week 08 r=peterbe a=pyup-bot






### Update [botocore](https://pypi.org/project/botocore) from **1.12.96** to **1.12.101**.


<details>
  <summary>Changelog</summary>
  
  
   ### 1.12.101
   ```
   ========

* api-change:``athena``: Update athena client to latest version
* api-change:``glue``: Update glue client to latest version
* api-change:``stepfunctions``: Update stepfunctions client to latest version
* api-change:``cloud9``: Update cloud9 client to latest version
   ```
   
  
  
   ### 1.12.100
   ```
   ========

* api-change:``kinesis-video-archived-media``: Update kinesis-video-archived-media client to latest version
* api-change:``workdocs``: Update workdocs client to latest version
* api-change:``codebuild``: Update codebuild client to latest version
* api-change:``cloudwatch``: Update cloudwatch client to latest version
* api-change:``organizations``: Update organizations client to latest version
* api-change:``kinesisvideo``: Update kinesisvideo client to latest version
* api-change:``kinesis-video-media``: Update kinesis-video-media client to latest version
* api-change:``transfer``: Update transfer client to latest version
   ```
   
  
  
   ### 1.12.99
   ```
   =======

* api-change:``codecommit``: Update codecommit client to latest version
* api-change:``directconnect``: Update directconnect client to latest version
* api-change:``medialive``: Update medialive client to latest version
   ```
   
  
  
   ### 1.12.98
   ```
   =======

* api-change:``iot``: Update iot client to latest version
* api-change:``ssm``: Update ssm client to latest version
* api-change:``ds``: Update ds client to latest version
* enhancement:Paginator: Add additional paginators for CloudFormation
* api-change:``efs``: Update efs client to latest version
   ```
   
  
  
   ### 1.12.97
   ```
   =======

* api-change:``athena``: Update athena client to latest version
* api-change:``secretsmanager``: Update secretsmanager client to latest version
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/botocore
  - Changelog: https://pyup.io/changelogs/botocore/
  - Repo: https://github.com/boto/botocore
</details>





### Update [jmespath](https://pypi.org/project/jmespath) from **0.9.3** to **0.9.4**.


<details>
  <summary>Changelog</summary>
  
  
   ### 0.9.4
   ```
   =====

* Fix ``min_by``/``max_by`` with empty lists
  `(`issue 151 &lt;https://github.com/jmespath/jmespath.py/pull/151&gt;`__)
* Fix reverse type for ``null`` type
  (`issue 145 &lt;https://github.com/jmespath/jmespath.py/pull/145&gt;`__)
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/jmespath
  - Changelog: https://pyup.io/changelogs/jmespath/
  - Repo: https://github.com/jmespath/jmespath.py
</details>





### Update [MarkupSafe](https://pypi.org/project/MarkupSafe) from **1.1.0** to **1.1.1**.


*The bot wasn't able to find a changelog for this release. [Got an idea?](https://github.com/pyupio/changelogs/issues/new)*

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/markupsafe
  - Changelog: https://pyup.io/changelogs/markupsafe/
  - Homepage: https://palletsprojects.com/p/markupsafe/
</details>





### Update [pluggy](https://pypi.org/project/pluggy) from **0.8.1** to **0.9.0**.


<details>
  <summary>Changelog</summary>
  
  
   ### 0.9.0
   ```
   =========================

Features
--------

- `189 &lt;https://github.com/pytest-dev/pluggy/issues/189&gt;`_: ``PluginManager.load_setuptools_entrypoints`` now accepts a ``name`` parameter that when given will
  load only entry points with that name.

  ``PluginManager.load_setuptools_entrypoints`` also now returns the number of plugins loaded by the
  call, as opposed to the number of all plugins loaded by all calls to this method.



Bug Fixes
---------

- `187 &lt;https://github.com/pytest-dev/pluggy/issues/187&gt;`_: Fix internal ``varnames`` function for PyPy3.
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/pluggy
  - Changelog: https://pyup.io/changelogs/pluggy/
  - Repo: https://github.com/pytest-dev/pluggy
</details>





### Update [py](https://pypi.org/project/py) from **1.7.0** to **1.8.0**.


<details>
  <summary>Changelog</summary>
  
  
   ### 1.8.0
   ```
   ==================

- add ``&quot;importlib&quot;`` pyimport mode for python3.5+, allowing unimportable test suites
  to contain identically named modules.

- fix ``LocalPath.as_cwd()`` not calling ``os.chdir()`` with ``None``, when
  being invoked from a non-existing directory.
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/py
  - Changelog: https://pyup.io/changelogs/py/
  - Docs: http://py.readthedocs.io/
</details>





### Update [google-auth](https://pypi.org/project/google-auth) from **1.6.2** to **1.6.3**.


<details>
  <summary>Changelog</summary>
  
  
   ### 1.6.3
   ```
   ------

02-15-2019 9:31 PST

Implementation Changes
+++++++++++++

- follow rfc 7515 : strip padding from JWS segments 324 (`324 &lt;https://github.com/googleapis/google-auth-library-python/pull/324&gt;`_)
- Add retry to _metadata.ping() (`323 &lt;https://github.com/googleapis/google-auth-library-python/pull/323&gt;`_)
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/google-auth
  - Changelog: https://pyup.io/changelogs/google-auth/
  - Repo: https://github.com/GoogleCloudPlatform/google-auth-library-python
</details>





### Update [boto3](https://pypi.org/project/boto3) from **1.9.96** to **1.9.101**.


<details>
  <summary>Changelog</summary>
  
  
   ### 1.9.101
   ```
   =======

* api-change:``athena``: [``botocore``] Update athena client to latest version
* api-change:``glue``: [``botocore``] Update glue client to latest version
* api-change:``stepfunctions``: [``botocore``] Update stepfunctions client to latest version
* api-change:``cloud9``: [``botocore``] Update cloud9 client to latest version
   ```
   
  
  
   ### 1.9.100
   ```
   =======

* api-change:``kinesis-video-archived-media``: [``botocore``] Update kinesis-video-archived-media client to latest version
* api-change:``workdocs``: [``botocore``] Update workdocs client to latest version
* api-change:``codebuild``: [``botocore``] Update codebuild client to latest version
* api-change:``cloudwatch``: [``botocore``] Update cloudwatch client to latest version
* api-change:``organizations``: [``botocore``] Update organizations client to latest version
* api-change:``kinesisvideo``: [``botocore``] Update kinesisvideo client to latest version
* api-change:``kinesis-video-media``: [``botocore``] Update kinesis-video-media client to latest version
* api-change:``transfer``: [``botocore``] Update transfer client to latest version
   ```
   
  
  
   ### 1.9.99
   ```
   ======

* api-change:``codecommit``: [``botocore``] Update codecommit client to latest version
* api-change:``directconnect``: [``botocore``] Update directconnect client to latest version
* api-change:``medialive``: [``botocore``] Update medialive client to latest version
   ```
   
  
  
   ### 1.9.98
   ```
   ======

* api-change:``iot``: [``botocore``] Update iot client to latest version
* api-change:``ssm``: [``botocore``] Update ssm client to latest version
* api-change:``ds``: [``botocore``] Update ds client to latest version
* enhancement:Paginator: [``botocore``] Add additional paginators for CloudFormation
* api-change:``efs``: [``botocore``] Update efs client to latest version
   ```
   
  
  
   ### 1.9.97
   ```
   ======

* api-change:``athena``: [``botocore``] Update athena client to latest version
* api-change:``secretsmanager``: [``botocore``] Update secretsmanager client to latest version
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/boto3
  - Changelog: https://pyup.io/changelogs/boto3/
  - Repo: https://github.com/boto/boto3
</details>





### Update [flake8](https://pypi.org/project/flake8) from **3.7.5** to **3.7.7**.


<details>
  <summary>Changelog</summary>
  
  
   ### 3.7.7
   ```
   -------------------

You can view the `3.7.7 milestone`_ on GitLab for more details.

Bugs Fixed
~~~~~~~~~~

- Fix crahes in plugins causing ``flake8`` to hang while unpickling errors (See
  also `GitLab!308`_, `GitLab505`_)


.. all links
.. _3.7.7 milestone:
    https://gitlab.com/pycqa/flake8/milestones/30

.. issue links
.. _GitLab505:
    https://gitlab.com/pycqa/flake8/issues/505

.. merge request links
.. _GitLab!308:
    https://gitlab.com/pycqa/flake8/merge_requests/308
   ```
   
  
  
   ### 3.7.6
   ```
   -------------------

You can view the `3.7.6 milestone`_ on GitLab for more details.

Bugs Fixed
~~~~~~~~~~

- Fix ``--per-file-ignores`` for multi-letter error codes (See also
  `GitLab!303`_, `GitLab507`_)

- Improve flake8 speed when only 1 filename is passed (See also `GitLab!305`_)


.. all links
.. _3.7.6 milestone:
    https://gitlab.com/pycqa/flake8/milestones/29

.. issue links
.. _GitLab507:
    https://gitlab.com/pycqa/flake8/issues/507

.. merge request links
.. _GitLab!303:
    https://gitlab.com/pycqa/flake8/merge_requests/303
.. _GitLab!305:
    https://gitlab.com/pycqa/flake8/merge_requests/305
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/flake8
  - Changelog: https://pyup.io/changelogs/flake8/
  - Repo: https://gitlab.com/pycqa/flake8
</details>





### Update [jsonschema](https://pypi.org/project/jsonschema) from **2.6.0** to **3.0.0**.


<details>
  <summary>Changelog</summary>
  
  
   ### 3.0.0
   ```
   ------

* Support for Draft 6 and Draft 7
* Draft 7 is now the default
* New ``TypeChecker`` object for more complex type definitions (and overrides)
* Falling back to isodate for the date-time format checker is no longer
  attempted, in accordance with the specification
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/jsonschema
  - Changelog: https://pyup.io/changelogs/jsonschema/
  - Repo: https://github.com/Julian/jsonschema
</details>





### Update [pytest](https://pypi.org/project/pytest) from **4.2.1** to **4.3.0**.


<details>
  <summary>Changelog</summary>
  
  
   ### 4.3.0
   ```
   =========================

Deprecations
------------

- `4724 &lt;https://github.com/pytest-dev/pytest/issues/4724&gt;`_: ``pytest.warns()`` now emits a warning when it receives unknown keyword arguments.

  This will be changed into an error in the future.



Features
--------

- `2753 &lt;https://github.com/pytest-dev/pytest/issues/2753&gt;`_: Usage errors from argparse are mapped to pytest&#39;s ``UsageError``.


- `3711 &lt;https://github.com/pytest-dev/pytest/issues/3711&gt;`_: Add the ``--ignore-glob`` parameter to exclude test-modules with Unix shell-style wildcards.
  Add the ``collect_ignore_glob`` for ``conftest.py`` to exclude test-modules with Unix shell-style wildcards.


- `4698 &lt;https://github.com/pytest-dev/pytest/issues/4698&gt;`_: The warning about Python 2.7 and 3.4 not being supported in pytest 5.0 has been removed.

  In the end it was considered to be more
  of a nuisance than actual utility and users of those Python versions shouldn&#39;t have problems as ``pip`` will not
  install pytest 5.0 on those interpreters.


- `4707 &lt;https://github.com/pytest-dev/pytest/issues/4707&gt;`_: With the help of new ``set_log_path()`` method there is a way to set ``log_file`` paths from hooks.



Bug Fixes
---------

- `4651 &lt;https://github.com/pytest-dev/pytest/issues/4651&gt;`_: ``--help`` and ``--version`` are handled with ``UsageError``.


- `4782 &lt;https://github.com/pytest-dev/pytest/issues/4782&gt;`_: Fix ``AssertionError`` with collection of broken symlinks with packages.
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/pytest
  - Changelog: https://pyup.io/changelogs/pytest/
  - Homepage: https://docs.pytest.org/en/latest/
</details>







Co-authored-by: pyup-bot <[email protected]>
Co-authored-by: Mike Cooper <[email protected]>
@mike-seekwell mike-seekwell mentioned this pull request Jun 30, 2019
Closed
DaniilAnichin pushed a commit to DaniilAnichin/google-auth-library-python that referenced this pull request Dec 5, 2019
Add retry to google.auth.compute_engine._metadata.get()
0181bec 
Initial fix of issue googleapis#211 was done in CL googleapis#323, but only for .ping(). This one is adding same behaviour & tests for .get() method, as the problem still occurres
DaniilAnichin pushed a commit to DaniilAnichin/google-auth-library-python that referenced this pull request Dec 5, 2019
Add retry to google.auth.compute_engine._metadata.get()
4b24118 
Initial fix of issue googleapis#211 was done in CL googleapis#323, but only for .ping(). This one is adding same behaviour & tests for .get() method, as the problem still occurres
DaniilAnichin added a commit to DaniilAnichin/google-auth-library-python that referenced this pull request Dec 5, 2019
@DaniilAnichin
Add retry to google.auth.compute_engine._metadata.get()
dffdce2 
Initial fix of issue googleapis#211 was done in CL googleapis#323, but only for .ping(). This one is adding same behaviour & tests for .get() method, as the problem still occurres
DaniilAnichin added a commit to DaniilAnichin/google-auth-library-python that referenced this pull request Dec 5, 2019
@DaniilAnichin
Add retry to google.auth.compute_engine._metadata.get()
832c1c5 
Initial fix of issue googleapis#211 was done in CL googleapis#323, but only for .ping(). This one is adding same behaviour & tests for .get() method, as the problem still occurres
DaniilAnichin added a commit to DaniilAnichin/google-auth-library-python that referenced this pull request Dec 5, 2019
@DaniilAnichin
Add retry to google.auth.compute_engine._metadata.get()
6e2b994 
Initial fix of issue googleapis#211 was done in CL googleapis#323, but only for .ping()n This one is adding same behaviour & tests for .get() method, as the problem still occurres\n\nResolves: googleapis#211
DaniilAnichin added a commit to DaniilAnichin/google-auth-library-python that referenced this pull request Dec 5, 2019
@DaniilAnichin
Add retry to google.auth.compute_engine._metadata.get()
d6b6756 
Initial fix of issue googleapis#211 was done in CL googleapis#323, but only for .ping()n This one is adding same behaviour & tests for .get() method, as the problem still occurres\n\nResolves: googleapis#211
@DaniilAnichin DaniilAnichin mentioned this pull request Dec 5, 2019
Merged
DaniilAnichin added a commit to DaniilAnichin/google-auth-library-python that referenced this pull request Dec 5, 2019
@DaniilAnichin
fix(google.auth.compute_engine.metadata): add retry to google.auth.co…
e0c2dbd 
…mpute_engine._metadata.get()

Initial fix of issue googleapis#211 was done in CL googleapis#323, but only for .ping()
This one is adding same behaviour & tests for .get() method, as the problem still occurres
See the issue for details

Refs: googleapis#323
Resolves: googleapis#211
DaniilAnichin added a commit to DaniilAnichin/google-auth-library-python that referenced this pull request Dec 5, 2019
@DaniilAnichin
fix(google.auth.compute_engine.metadata): add retry to google.auth.co…
917df1a 
…mpute_engine._metadata.get()

Initial fix of issue googleapis#211 was done in CL googleapis#323, but only for .ping()
This one is adding same behaviour & tests for .get() method, as the problem still occurres
See the issue for details

Refs: googleapis#323
Resolves: googleapis#211
DaniilAnichin added a commit to DaniilAnichin/google-auth-library-python that referenced this pull request Dec 19, 2019
@DaniilAnichin
fix(google.auth.compute_engine.metadata): add retry to google.auth.co…
1226931 
…mpute_engine._metadata.get()

Initial fix of issue googleapis#211 was done in CL googleapis#323, but only for .ping()
This one is adding same behaviour & tests for .get() method, as the problem still occurres
See the issue for details

Refs: googleapis#323
Resolves: googleapis#211
DaniilAnichin added a commit to DaniilAnichin/google-auth-library-python that referenced this pull request Dec 26, 2019
@DaniilAnichin
fix(google.auth.compute_engine.metadata): add retry to google.auth.co…
2f75de9 
…mpute_engine._metadata.get()

Initial fix of issue googleapis#211 was done in CL googleapis#323, but only for .ping()
This one is adding same behaviour & tests for .get() method, as the problem still occurres
See the issue for details

Refs: googleapis#323
Resolves: googleapis#211
DaniilAnichin added a commit to DaniilAnichin/google-auth-library-python that referenced this pull request Jan 9, 2020
@DaniilAnichin
fix(google.auth.compute_engine.metadata): add retry to google.auth.co…
3a1187f 
…mpute_engine._metadata.get()

Initial fix of issue googleapis#211 was done in CL googleapis#323, but only for .ping()
This one is adding same behaviour & tests for .get() method, as the problem still occurres
See the issue for details

Refs: googleapis#323
Resolves: googleapis#211
busunkim96 pushed a commit that referenced this pull request Jan 9, 2020
@DaniilAnichin @busunkim96
fix(google.auth.compute_engine.metadata): add retry to google.auth.co…
af29c1a 
…mpute_engine._metadata.get() (#398)

Initial fix of issue #211 was done in CL #323, but only for .ping()
This one is adding same behaviour & tests for .get() method, as the problem still occurres
See the issue for details

Refs: #323
Resolves: #211
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@theacodes theacodes theacodes approved these changes

Assignees
No one assigned
Labels
cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@rmbarron @theacodes @googlebot