Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(webhooks): allow manually retrying delivery of outgoing webhooks #4176

Merged
merged 6 commits into from
Apr 4, 2024
Merged

feat(webhooks): allow manually retrying delivery of outgoing webhooks #4176

merged 6 commits into from
Apr 4, 2024

Conversation

SanchithHegde
Copy link
Member

Type of Change

  • Bugfix
  • New feature
  • Enhancement
  • Refactoring
  • Dependency updates
  • Documentation
  • CI/CD

Description

This PR includes changes in the outgoing webhooks flow to allow manually retrying webhook deliveries, and adds an API endpoint for the same.

  • URL: /events/{merchant_id_or_profile_id}/{event_id}/retry
  • Authentication: Admin API authentication + JWT
  • The endpoint must return the request and response information for the attempted retry.

Additional Changes

  • This PR modifies the API contract
  • This PR modifies the database schema
  • This PR modifies application configuration/environment variables

This PR adds a new endpoint to retry outgoing webhook delivery.

Motivation and Context

This allows users to debug issues in case the delivery of webhooks to their application server fails, without having to wait for the application to automatically retry at set intervals.

How did you test it?

This PR adds a new endpoint and testing it is pretty straightforward. It can be tried out using a request like so:

curl -X POST -H 'api-key: {{admin_api_key}}' '{{baseUrl}}/events/{{merchant_id_or_profile_id}}/{{event_id}}/retry'

  • In case of errors hitting the merchant server (say URL is inaccessible):

    curl -X POST -H 'api-key: test_admin' 'http://localhost:8080/events/pro_H7FbvX2pBuJLjlPhkpRa/evt_018e62ea2215731c915c3a47e4753e4a/retry' 2>/dev/null | jq '.'
{
  "event_id": "evt_018e6581512d7b9bba60ec4ca44f364f",
  "merchant_id": "merchant_1710609960",
  "profile_id": "pro_H7FbvX2pBuJLjlPhkpRa",
  "object_id": "pay_6AMmb6rtlPDarBLlotAH",
  "event_type": "payment_succeeded",
  "event_class": "payments",
  "is_delivery_successful": false,
  "initial_attempt_id": "evt_018e48d5ad6c73d5a289342e6132b224",
  "created": "2024-03-22T09:32:37.550Z",
  "request": {
    "body": "{\"merchant_id\":\"merchant_1710609960\",\"event_id\":\"evt_018e48d5ad6c73d5a289342e6132b224\",\"event_type\":\"payment_succeeded\",\"content\":{\"type\":\"payment_details\",\"object\":{\"payment_id\":\"pay_6AMmb6rtlPDarBLlotAH\",\"merchant_id\":\"merchant_1710609960\",\"status\":\"succeeded\",\"amount\":6540,\"net_amount\":6540,\"amount_capturable\":0,\"amount_received\":6540,\"connector\":\"stripe\",\"client_secret\":\"pay_6AMmb6rtlPDarBLlotAH_secret_1hjI8KVOKf5xhMXBTwRj\",\"created\":\"2024-03-16T19:55:45.335Z\",\"currency\":\"USD\",\"customer_id\":\"StripeCustomer\",\"description\":\"Its my first payment request\",\"refunds\":null,\"disputes\":null,\"mandate_id\":null,\"mandate_data\":null,\"setup_future_usage\":null,\"off_session\":null,\"capture_on\":null,\"capture_method\":\"automatic\",\"payment_method\":\"card\",\"payment_method_data\":{\"card\":{\"last4\":\"4242\",\"card_type\":null,\"card_network\":null,\"card_issuer\":null,\"card_issuing_country\":null,\"card_isin\":\"424242\",\"card_extended_bin\":\"42424242\",\"card_exp_month\":\"10\",\"card_exp_year\":\"25\",\"card_holder_name\":\"joseph Doe\"},\"billing\":{\"address\":{\"city\":\"San Fransico\",\"country\":\"US\",\"line1\":\"1467\",\"line2\":\"Harrison Street\",\"line3\":\"Harrison Street\",\"zip\":\"94122\",\"state\":\"California\",\"first_name\":\"joseph\",\"last_name\":\"Doe\"},\"phone\":{\"number\":\"8056594427\",\"country_code\":\"+91\"},\"email\":null}},\"payment_token\":null,\"shipping\":{\"address\":{\"city\":\"San Fransico\",\"country\":\"US\",\"line1\":\"1467\",\"line2\":\"Harrison Street\",\"line3\":\"Harrison Street\",\"zip\":\"94122\",\"state\":\"California\",\"first_name\":\"joseph\",\"last_name\":\"Doe\"},\"phone\":{\"number\":\"8056594427\",\"country_code\":\"+91\"},\"email\":null},\"billing\":{\"address\":{\"city\":\"San Fransico\",\"country\":\"US\",\"line1\":\"1467\",\"line2\":\"Harrison Street\",\"line3\":\"Harrison Street\",\"zip\":\"94122\",\"state\":\"California\",\"first_name\":\"joseph\",\"last_name\":\"Doe\"},\"phone\":{\"number\":\"8056594427\",\"country_code\":\"+91\"},\"email\":null},\"order_details\":null,\"email\":\"[email protected]\",\"name\":\"John Doe\",\"phone\":\"999999999\",\"return_url\":\"https://google.com/\",\"authentication_type\":\"no_three_ds\",\"statement_descriptor_name\":\"joseph\",\"statement_descriptor_suffix\":\"JS\",\"next_action\":null,\"cancellation_reason\":null,\"error_code\":null,\"error_message\":null,\"unified_code\":null,\"unified_message\":null,\"payment_experience\":null,\"payment_method_type\":\"credit\",\"connector_label\":null,\"business_country\":null,\"business_label\":\"default\",\"business_sub_label\":null,\"allowed_payment_method_types\":null,\"ephemeral_key\":{\"customer_id\":\"StripeCustomer\",\"created_at\":1710618945,\"expires\":1710622545,\"secret\":\"epk_5736317c91e04787838bd300d6f53877\"},\"manual_retry_allowed\":false,\"connector_transaction_id\":\"pi_3Ov3KXD5R7gDAGff1ZbtdptB\",\"frm_message\":null,\"metadata\":{\"udf1\":\"value1\",\"login_date\":\"2019-09-10T10:11:12Z\",\"new_customer\":\"true\"},\"connector_metadata\":null,\"feature_metadata\":null,\"reference_id\":\"pi_3Ov3KXD5R7gDAGff1ZbtdptB\",\"payment_link\":null,\"profile_id\":\"pro_H7FbvX2pBuJLjlPhkpRa\",\"surcharge_details\":null,\"attempt_count\":1,\"merchant_decision\":null,\"merchant_connector_id\":\"mca_qeS5H2fLkr8GLzN9SNWd\",\"incremental_authorization_allowed\":null,\"authorization_count\":null,\"incremental_authorizations\":null,\"external_authentication_details\":null,\"external_3ds_authentication_attempted\":false,\"expires_on\":\"2024-03-16T20:10:45.335Z\",\"fingerprint\":null,\"payment_method_id\":null,\"payment_method_status\":null}},\"timestamp\":\"2024-03-16T19:55:46.924Z\"}",
    "headers": [
      [
        "content-type",
        "application/json"
      ],
      [
        "X-Webhook-Signature-512",
        "68dd7abf747b5c33eaef5d7450990594aff5b260e79a33405539311f037efc2b70f86b062bcab9a231bd8fa8e8141df078e5834d52add7751da48247bf31ece5"
      ]
    ]
  },
  "response": {
    "body": null,
    "headers": null,
    "status_code": null,
    "error_message": "Unable to send request to merchant server"
  },
  "delivery_attempt": "manual_retry"
}

    Note:

    • initial_attempt_id would not contain the event_id of the event being retried, but the initial_attempt_id of the event being retried. This would allow us to list all attempts including manual delivery attempts related to the initial attempt.
    • delivery_attempt is manual_retry in this case
    • response.error_message is non-null

  • Successful delivery:

    curl -X POST -H 'api-key: test_admin' 'http://localhost:8080/events/pro_H7FbvX2pBuJLjlPhkpRa/evt_018e62ea2215731c915c3a47e4753e4a/retry' 2>/dev/null | jq '.'
{
  "event_id": "evt_018e6589a52e75a9bb0dbb72fd6b6434",
  "merchant_id": "merchant_1710609960",
  "profile_id": "pro_H7FbvX2pBuJLjlPhkpRa",
  "object_id": "pay_6AMmb6rtlPDarBLlotAH",
  "event_type": "payment_succeeded",
  "event_class": "payments",
  "is_delivery_successful": true,
  "initial_attempt_id": "evt_018e48d5ad6c73d5a289342e6132b224",
  "created": "2024-03-22T09:41:43.342Z",
  "request": {
    "body": "{\"merchant_id\":\"merchant_1710609960\",\"event_id\":\"evt_018e48d5ad6c73d5a289342e6132b224\",\"event_type\":\"payment_succeeded\",\"content\":{\"type\":\"payment_details\",\"object\":{\"payment_id\":\"pay_6AMmb6rtlPDarBLlotAH\",\"merchant_id\":\"merchant_1710609960\",\"status\":\"succeeded\",\"amount\":6540,\"net_amount\":6540,\"amount_capturable\":0,\"amount_received\":6540,\"connector\":\"stripe\",\"client_secret\":\"pay_6AMmb6rtlPDarBLlotAH_secret_1hjI8KVOKf5xhMXBTwRj\",\"created\":\"2024-03-16T19:55:45.335Z\",\"currency\":\"USD\",\"customer_id\":\"StripeCustomer\",\"description\":\"Its my first payment request\",\"refunds\":null,\"disputes\":null,\"mandate_id\":null,\"mandate_data\":null,\"setup_future_usage\":null,\"off_session\":null,\"capture_on\":null,\"capture_method\":\"automatic\",\"payment_method\":\"card\",\"payment_method_data\":{\"card\":{\"last4\":\"4242\",\"card_type\":null,\"card_network\":null,\"card_issuer\":null,\"card_issuing_country\":null,\"card_isin\":\"424242\",\"card_extended_bin\":\"42424242\",\"card_exp_month\":\"10\",\"card_exp_year\":\"25\",\"card_holder_name\":\"joseph Doe\"},\"billing\":{\"address\":{\"city\":\"San Fransico\",\"country\":\"US\",\"line1\":\"1467\",\"line2\":\"Harrison Street\",\"line3\":\"Harrison Street\",\"zip\":\"94122\",\"state\":\"California\",\"first_name\":\"joseph\",\"last_name\":\"Doe\"},\"phone\":{\"number\":\"8056594427\",\"country_code\":\"+91\"},\"email\":null}},\"payment_token\":null,\"shipping\":{\"address\":{\"city\":\"San Fransico\",\"country\":\"US\",\"line1\":\"1467\",\"line2\":\"Harrison Street\",\"line3\":\"Harrison Street\",\"zip\":\"94122\",\"state\":\"California\",\"first_name\":\"joseph\",\"last_name\":\"Doe\"},\"phone\":{\"number\":\"8056594427\",\"country_code\":\"+91\"},\"email\":null},\"billing\":{\"address\":{\"city\":\"San Fransico\",\"country\":\"US\",\"line1\":\"1467\",\"line2\":\"Harrison Street\",\"line3\":\"Harrison Street\",\"zip\":\"94122\",\"state\":\"California\",\"first_name\":\"joseph\",\"last_name\":\"Doe\"},\"phone\":{\"number\":\"8056594427\",\"country_code\":\"+91\"},\"email\":null},\"order_details\":null,\"email\":\"[email protected]\",\"name\":\"John Doe\",\"phone\":\"999999999\",\"return_url\":\"https://google.com/\",\"authentication_type\":\"no_three_ds\",\"statement_descriptor_name\":\"joseph\",\"statement_descriptor_suffix\":\"JS\",\"next_action\":null,\"cancellation_reason\":null,\"error_code\":null,\"error_message\":null,\"unified_code\":null,\"unified_message\":null,\"payment_experience\":null,\"payment_method_type\":\"credit\",\"connector_label\":null,\"business_country\":null,\"business_label\":\"default\",\"business_sub_label\":null,\"allowed_payment_method_types\":null,\"ephemeral_key\":{\"customer_id\":\"StripeCustomer\",\"created_at\":1710618945,\"expires\":1710622545,\"secret\":\"epk_5736317c91e04787838bd300d6f53877\"},\"manual_retry_allowed\":false,\"connector_transaction_id\":\"pi_3Ov3KXD5R7gDAGff1ZbtdptB\",\"frm_message\":null,\"metadata\":{\"udf1\":\"value1\",\"login_date\":\"2019-09-10T10:11:12Z\",\"new_customer\":\"true\"},\"connector_metadata\":null,\"feature_metadata\":null,\"reference_id\":\"pi_3Ov3KXD5R7gDAGff1ZbtdptB\",\"payment_link\":null,\"profile_id\":\"pro_H7FbvX2pBuJLjlPhkpRa\",\"surcharge_details\":null,\"attempt_count\":1,\"merchant_decision\":null,\"merchant_connector_id\":\"mca_qeS5H2fLkr8GLzN9SNWd\",\"incremental_authorization_allowed\":null,\"authorization_count\":null,\"incremental_authorizations\":null,\"external_authentication_details\":null,\"external_3ds_authentication_attempted\":false,\"expires_on\":\"2024-03-16T20:10:45.335Z\",\"fingerprint\":null,\"payment_method_id\":null,\"payment_method_status\":null}},\"timestamp\":\"2024-03-16T19:55:46.924Z\"}",
    "headers": [
      [
        "content-type",
        "application/json"
      ],
      [
        "X-Webhook-Signature-512",
        "68dd7abf747b5c33eaef5d7450990594aff5b260e79a33405539311f037efc2b70f86b062bcab9a231bd8fa8e8141df078e5834d52add7751da48247bf31ece5"
      ]
    ]
  },
  "response": {
    "body": "",
    "headers": [
      [
        "content-length",
        "0"
      ],
      [
        "date",
        "Fri, 22 Mar 2024 09:41:44 GMT"
      ],
      [
        "access-control-allow-origin",
        "*"
      ],
      [
        "set-cookie",
        "ARRAffinity=669b1a143e31fb4dd62d7cac87cd552f46ae1eaa9e19a1bc7e87014967e06b05;Path=/;HttpOnly;Secure;Domain=smee.io"
      ],
      [
        "set-cookie",
        "ARRAffinitySameSite=669b1a143e31fb4dd62d7cac87cd552f46ae1eaa9e19a1bc7e87014967e06b05;Path=/;HttpOnly;SameSite=None;Secure;Domain=smee.io"
      ],
      [
        "request-context",
        "appId=cid-v1:"
      ],
      [
        "x-powered-by",
        "Express"
      ]
    ],
    "status_code": 200,
    "error_message": null
  },
  "delivery_attempt": "manual_retry"
}

  • The above request should behave the same way by replacing the profile ID with the merchant ID, or replacing the api-key header and value with the JWT token received after user sign in.

Checklist

  • I formatted the code cargo +nightly fmt --all
  • I addressed lints thrown by cargo clippy
  • I reviewed the submitted code
  • I added unit tests for my changes where possible
  • I added a CHANGELOG entry if applicable

@SanchithHegde SanchithHegde added A-core Area: Core flows C-feature Category: Feature request or enhancement S-waiting-on-review Status: This PR has been implemented and needs to be reviewed M-api-contract-changes Metadata: This PR involves API contract changes A-webhooks Area: Webhook flows labels Mar 22, 2024
@SanchithHegde SanchithHegde added this to the March 2024 milestone Mar 22, 2024
@SanchithHegde SanchithHegde self-assigned this Mar 22, 2024
@SanchithHegde SanchithHegde requested review from a team as code owners March 22, 2024 09:49
Base automatically changed from events-list-apis-allow-profile-id-in-path to main March 22, 2024 11:46
@SanchithHegde SanchithHegde requested a review from a team as a code owner March 22, 2024 11:46
ThisIsMani
ThisIsMani previously approved these changes Mar 22, 2024
View reviewed changes
Copy link
Contributor

@ThisIsMani ThisIsMani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Auth specific changes looks fine.

@likhinbopanna likhinbopanna modified the milestones: March 2024, April 2024 Apr 2, 2024
sahkal
sahkal reviewed Apr 3, 2024
View reviewed changes
Copy link
Contributor

@sahkal sahkal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

regenerate open-api specs other than that LGTM!

@SanchithHegde
chore(webhook_events): clarify usage of merchant_id_or_profile_id t…
0c33b96 
…o find merchant account and business profile
sai-harsha-vardhan
sai-harsha-vardhan approved these changes Apr 4, 2024
View reviewed changes
Copy link
Contributor

@sai-harsha-vardhan sai-harsha-vardhan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@likhinbopanna likhinbopanna added this pull request to the merge queue Apr 4, 2024
Merged via the queue into main with commit 63d2b68 Apr 4, 2024
9 of 12 checks passed
@likhinbopanna likhinbopanna deleted the outgoing-webhooks-manual-retry branch April 4, 2024 09:21
@SanchithHegde SanchithHegde removed the S-waiting-on-review Status: This PR has been implemented and needs to be reviewed label Apr 4, 2024
@SanchithHegde SanchithHegde mentioned this pull request Aug 12, 2024
Merged
11 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ThisIsMani ThisIsMani ThisIsMani approved these changes

@hrithikesh026 hrithikesh026 hrithikesh026 approved these changes

@sai-harsha-vardhan sai-harsha-vardhan sai-harsha-vardhan approved these changes

@sahkal sahkal sahkal approved these changes

@lsampras lsampras lsampras approved these changes

@Narayanbhat166 Narayanbhat166 Awaiting requested review from Narayanbhat166

Assignees

@SanchithHegde SanchithHegde

Labels
A-core Area: Core flows A-webhooks Area: Webhook flows C-feature Category: Feature request or enhancement M-api-contract-changes Metadata: This PR involves API contract changes
Projects
None yet
Milestone
April 2024 Release
Development

Successfully merging this pull request may close these issues.
7 participants
@SanchithHegde @lsampras @sahkal @sai-harsha-vardhan @hrithikesh026 @ThisIsMani @likhinbopanna