Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tdl 681 add retry mechanism #30

Merged
merged 8 commits into from
Apr 19, 2021
Merged

Conversation

hpatel41
Copy link
Contributor

@hpatel41 hpatel41 commented Apr 9, 2021

Description of change

  • TDL-681: HTTP requests that fail are not retrying - Added retry mechanism for requests that are failing
  • Added exception handling ie. KlaviyoError

Manual QA steps

  • Manually raised KlaviyoError and JSONDecoder error and checked retrying of requests
  • Test Cases that checks the retrying of requests on error
  • Test Cases for checking exception handling and error message

Risks

Rollback steps

  • revert this branch


def raise_for_error(response):
try:
json_resp = response.json()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@harshpatel4crest Can you please get the response of response.raise_for_status() here and except the HTTTError in the exception.
Please refer to the tap-Xero code (https://github.com/singer-io/tap-xero/pull/85/files#diff-43d32c99c6992e01f4dbdb49df5e372db029dae0de306b9191283ef21b852438R216) for the same

import singer

class Mockresponse:
def __init__(self, resp, status_code, content=None, headers=None, raise_error=False):
Copy link
Contributor

@karanpanchal-crest karanpanchal-crest Apr 9, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@harshpatel4crest You can make resp an optional argument.

class Mockresponse:
    def __init__(self, status_code, resp={}, content=None, headers=None, raise_error=False):
        self.json_data = resp
        ...


def klaviyo_400_error(*args, **kwargs):
    return Mockresponse(json_str, 400, raise_error=True)

Please follow the above code for the remaining functions

class TestBackoff(unittest.TestCase):

@mock.patch('requests.Session.request', side_effect=klaviyo_400_error)
def test_400_error(self, klaviyo_400_error):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@harshpatel4crest Can you also add a positive scenario as well

Copy link
Contributor

@KAllan357 KAllan357 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please change some of the error message text. Also Pylint is failing so please get that fixed as well.

},
403: {
"raise_exception": KlaviyoForbiddenError,
"message": "Request is missing or has an invalid API key."
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we adjust the message here to say "Invalid authorization credentials or permissions."

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have added the desired string, however I have taken the string from Klaviyo Documentation https://www.klaviyo.com/docs/api/lists.
So, let me know if want to revert the change.

},
500: {
"raise_exception": KlaviyoInternalServiceError,
"message": "Something is wrong on Klaviyo's end."
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we also adjust this to say "Internal Service Error from Klaviyo"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added the string in message

@KAllan357
Copy link
Contributor

Can you resolve the conflicts?

@hpatel41
Copy link
Contributor Author

Can you resolve the conflicts?

@KAllan357
Resolved conflicts

@KAllan357 KAllan357 merged commit 7a09e92 into master Apr 19, 2021
@KAllan357 KAllan357 deleted the TDL-681-add-retry-mechanism branch April 19, 2021 13:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants