Fix 503 response not adding confirmation to queue to be retried #5599

LaurenWags · 2019-08-08T15:28:55Z

Description

If you get a 503 response to OnCreateConfirmation the confirmation is not created and therefore is not put in the queue for retries.

Steps to Reproduce

Unsure (can you Charles Proxy to rewrite the CreateConfirmation request to return status response code 503

Actual result:

See in the logs:

[4856:775:0808/110311.405988:INFO:redeem_token.cc(169)] OnCreateConfirmation
[4856:775:0808/110311.406032:INFO:redeem_token.cc(171)] URL Request Response:
[4856:775:0808/110311.406047:INFO:redeem_token.cc(172)]   URL: https://ads-serve.brave.com/v1/confirmation/e1afb398-2be4-44b6-96fd-d55dd6e87ff3/eyJwYXlsb2FkIjoie1wiYmxpbmRlZFBheW1lbnRUb2tlblwiOlwiZEJzU2QyZWpTQ1J0R09VV3UybXd3VHY4RTFDOXdqRmZiYUx6cTBzSTlBQT1cIixcImNyZWF0aXZlSW5zdGFuY2VJZFwiOlwiNTcwMTM4ZmYtMDU1OC00MWRjLTg3ZTQtNzk4M2EzZDk0MDQzXCIsXCJwYXlsb2FkXCI6e30sXCJ0eXBlXCI6XCJjbGlja1wifSIsInNpZ25hdHVyZSI6IktUSkRTeUpJQkE5SVNtSlBWTlByendMdHR3ZXEzZkF6ZGJaUENIazFLZytmZTVrOUFKSmJRMjZvdUY4eU1iMWxEMG9QZHl1TkxuYnl2YjcxNXdJWFdnPT0iLCJ0IjoiL05WYlpSOEVCMURNSnRWWHFiOTVPVGhwdHVqcFpqVnliako3Y3k1b2hXRnUvelpKVkhURzFLYm5WSTE2Z2YrWjM2Wi9mU0RGS1haV3dtOHFJcGZuU0E9PSJ9
[4856:775:0808/110311.406063:INFO:redeem_token.cc(173)]   Response Status Code: 503
[4856:775:0808/110311.406077:INFO:redeem_token.cc(174)]   Response: 
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
 "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html>
  <head>
    <title>503 first byte timeout</title>
  </head>
  <body>
    <h1>Error 503 first byte timeout</h1>
    <p>first byte timeout</p>
    <h3>Guru Mediation:</h3>
    <p>Details: cache-ewr18139-EWR 1565276591 2658566784</p>
    <hr>
    <p>Varnish cache server</p>
  </body>
</html>

[4856:775:0808/110311.406094:INFO:redeem_token.cc(175)]   Headers:
[4856:775:0808/110311.406106:INFO:redeem_token.cc(177)]     accept-ranges: bytes
[4856:775:0808/110311.406119:INFO:redeem_token.cc(177)]     connection: close
[4856:775:0808/110311.406131:INFO:redeem_token.cc(177)]     content-length: 449
[4856:775:0808/110311.406143:INFO:redeem_token.cc(177)]     content-type: text/html; charset=utf-8
[4856:775:0808/110311.406155:INFO:redeem_token.cc(177)]     date: Thu, 08 Aug 2019 15:03:11 GMT
[4856:775:0808/110311.406168:INFO:redeem_token.cc(177)]     retry-after: 0
[4856:775:0808/110311.406180:INFO:redeem_token.cc(177)]     server: Varnish
[4856:775:0808/110311.406192:INFO:redeem_token.cc(177)]     via: 1.1 varnish
[4856:775:0808/110311.406204:INFO:redeem_token.cc(177)]     x-cache: MISS
[4856:775:0808/110311.406216:INFO:redeem_token.cc(177)]     x-cache-hits: 0
[4856:775:0808/110311.406228:INFO:redeem_token.cc(177)]     x-served-by: cache-ewr18139-EWR
[4856:775:0808/110311.406241:ERROR:redeem_token.cc(181)] Failed to create confirmation
[4856:775:0808/110311.406260:WARNING:redeem_token.cc(429)] Failed to redeem e1afb398-2be4-44b6-96fd-d55dd6e87ff3 confirmation id with 570138ff-0558-41dc-87e4-7983a3d94043 creative instance id for click

Expected result:

no 503s, or if 503 does happen, put confirmation in failed queue for retries

Reproduces how often:

unsure

Brave version (brave://version info)

Brave	0.70.48 Chromium: 76.0.3809.100 (Official Build) nightly (64-bit)
Revision	ed9d447d30203dc5069e540f05079e493fc1c132-refs/branch-heads/3809@{#990}
OS	Mac OS X

Version/Channel Information:

Can you reproduce this issue with the current release? probably
Can you reproduce this issue with the beta channel? probably
Can you reproduce this issue with the dev channel? probably
Can you reproduce this issue with the nightly channel? yes

Other Additional Information:

Does the issue resolve itself when disabling Brave Shields?
Does the issue resolve itself when disabling Brave Rewards?
Is the issue reproducible on the latest version of Chrome?

Miscellaneous Information:

cc @tmancey @jsecretan

The text was updated successfully, but these errors were encountered:

Fixes brave/brave-browser#5599

GeetaSarvadnya · 2019-08-19T12:50:16Z

The issue is Reproducible in Windows 10 x64 - 0.68.130

STR:

viewed an ad
There is no ad view count reflect in ads panel also there is no estimated pending rewards

tmancey · 2019-08-19T12:51:44Z

@GeetaSarvadnya This fix did not resolve the 503 error as this is from the server end point. This fix makes sure if an error occurs that we retry. Check further down log for retry.

GeetaSarvadnya · 2019-08-19T13:02:48Z

@tmancey There is no Start retrying... after I get status repose status code 503 in logs. Also, failed confirmation array is empty in confirmation.json file, there are no failed confirmations. The Ad which I viewed is completely lost when I get 503 in logs.

chrome_debug.log

tmancey · 2019-08-19T13:09:26Z

@GeetaSarvadnya This issue is not in 0.68, this issue has only been merged with nightly and dev. I have requested uplift but before this is potentially accepted we need to pass QA for nightly and dev.

Fixes brave/brave-browser#5599

GeetaSarvadnya · 2019-09-04T20:28:20Z

Verification passed on

Brave	0.69.123 Chromium: 76.0.3809.132 (Official Build) beta (64-bit)
Revision	fd1acc410994a7a68ac25bc77513d443f3130860-refs/branch-heads/3809@{#1035}
OS	Windows 10 OS Version 1803 (Build 17134.523)

(Used Charles proxy to simulate 503 Response statuses)

Verified that the failed confirmations generated due to Status_Response code 503 are recorded or queued in the Confirmation.json file
Verified that the failed confirmations View, Click, Dismiss and Landed are queued in Confirmation.json file
Verified that the failed confirmations queue becomes empty once retry is completed
Verified that the Ads panel Ads notifications received count and Estimated pending rewards BAT updated properly ( I have viewed 2 ads and both ads data updated properly after retry)

Verified passed with

Brave	0.69.124 Chromium: 76.0.3809.132 (Official Build) (64-bit)
Revision	fd1acc410994a7a68ac25bc77513d443f3130860-refs/branch-heads/3809@{#1035}
OS	Mac OS X

Utilized Charles Proxy to emulate a 503 response to createConfirmation.

Verified that this failed onCreateConfirmation was added to the failed-confirmations queue in confirmations.json to be retried.
Verified for view, click, landed, and dismiss confirmation types.
Verified that retries continuously retried until queue was empty.
Verified brave://rewards Ads panel updated for applicable successful retries.

Verification passed on

Brave	0.69.124 Chromium: 76.0.3809.132 (Official Build) (64-bit)
Revision	fd1acc410994a7a68ac25bc77513d443f3130860-refs/branch-heads/3809@{#1035}
OS	Ubuntu 18.04 LTS

Utilized Charles Proxy to emulate a 503 response to createConfirmation.

Verified that this failed onCreateConfirmation was added to the failed-confirmations queue in confirmations.json to be retried.
Verified for view, click, landed, and dismiss confirmation types.
Verified that retries continuously retried until queue was empty.
Verified brave://rewards Ads panel updated for applicable successful retries.

LaurenWags added QA/Yes feature/ads labels Aug 8, 2019

tmancey added a commit to brave/brave-core that referenced this issue Aug 8, 2019

Fixes 503 response does not add confirmation to queue

180ae2a

Fixes brave/brave-browser#5599

tmancey mentioned this issue Aug 8, 2019

Fixes 503 response does not add confirmation to queue brave/brave-core#3126

Merged

32 tasks

tmancey closed this as completed in brave/brave-core#3126 Aug 8, 2019

tmancey added a commit to brave/brave-core that referenced this issue Aug 8, 2019

Fixes 503 response does not add confirmation to queue

0303cb4

Fixes brave/brave-browser#5599

tmancey added a commit to brave/brave-core that referenced this issue Aug 8, 2019

Fixes 503 response does not add confirmation to queue

70fcdd8

Fixes brave/brave-browser#5599

tmancey added a commit to brave/brave-core that referenced this issue Aug 8, 2019

Fixes 503 response does not add confirmation to queue

d4ca5e7

Fixes brave/brave-browser#5599

tmancey added a commit to brave/brave-core that referenced this issue Aug 8, 2019

Fixes 503 response does not add confirmation to queue

5c04641

Fixes brave/brave-browser#5599

tmancey added a commit to brave/brave-core that referenced this issue Aug 8, 2019

Fixes 503 response does not add confirmation to queue

8a1cd94

Fixes brave/brave-browser#5599

tmancey added a commit to brave/brave-core that referenced this issue Aug 8, 2019

Fixes 503 response does not add confirmation to queue

23e6e85

Fixes brave/brave-browser#5599

tmancey added bug feature/rewards priority/P2 A bad problem. We might uplift this to the next planned release. labels Aug 8, 2019

tmancey modified the milestones: 0.67.x - Release Hotfix 2, 0.70.x - Nightly Aug 8, 2019

NejcZdovc assigned tmancey Aug 12, 2019

LaurenWags mentioned this issue Aug 12, 2019

Issue with Ads Payments (August 5) #5548

Closed

tmancey modified the milestones: 0.70.x - Dev, 0.68.x - Release Aug 14, 2019

tmancey added a commit to brave/brave-core that referenced this issue Aug 30, 2019

Fixes 503 response does not add confirmation to queue

a67c251

Fixes brave/brave-browser#5599

tmancey added a commit to brave/brave-core that referenced this issue Sep 2, 2019

Fixes 503 response does not add confirmation to queue

cd8a066

Fixes brave/brave-browser#5599

tmancey modified the milestones: 0.70.x - Dev, 0.69.x - Beta Sep 2, 2019

LaurenWags added the release-notes/include label Sep 4, 2019

GeetaSarvadnya added the QA Pass-Win64 label Sep 4, 2019

LaurenWags added the QA Pass-macOS label Sep 5, 2019

btlechowski added the QA Pass-Linux label Sep 5, 2019

bsclifton modified the milestones: 0.69.x-backup, 0.70.x - Beta Sep 12, 2019

GeetaSarvadnya mentioned this issue Oct 2, 2019

Assertion on startup when using old ads history JSON data #5623

Closed

LaurenWags changed the title ~~503 response does not add confirmation to queue~~ Fix 503 response not adding confirmation to queue to be retried Oct 3, 2019

kjozwiak mentioned this issue Oct 3, 2019

Release notes for 0.69.x (Release Channel) #6295

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix 503 response not adding confirmation to queue to be retried #5599

Fix 503 response not adding confirmation to queue to be retried #5599

LaurenWags commented Aug 8, 2019 •

edited by tmancey

Loading

GeetaSarvadnya commented Aug 19, 2019 •

edited

Loading

tmancey commented Aug 19, 2019 •

edited

Loading

GeetaSarvadnya commented Aug 19, 2019 •

edited

Loading

tmancey commented Aug 19, 2019 •

edited

Loading

GeetaSarvadnya commented Sep 4, 2019 •

edited by btlechowski

Loading

Fix 503 response not adding confirmation to queue to be retried #5599

Fix 503 response not adding confirmation to queue to be retried #5599

Comments

LaurenWags commented Aug 8, 2019 • edited by tmancey Loading

Description

Steps to Reproduce

Actual result:

Expected result:

Reproduces how often:

Brave version (brave://version info)

Version/Channel Information:

Other Additional Information:

Miscellaneous Information:

GeetaSarvadnya commented Aug 19, 2019 • edited Loading

tmancey commented Aug 19, 2019 • edited Loading

GeetaSarvadnya commented Aug 19, 2019 • edited Loading

tmancey commented Aug 19, 2019 • edited Loading

GeetaSarvadnya commented Sep 4, 2019 • edited by btlechowski Loading

LaurenWags commented Aug 8, 2019 •

edited by tmancey

Loading

GeetaSarvadnya commented Aug 19, 2019 •

edited

Loading

tmancey commented Aug 19, 2019 •

edited

Loading

GeetaSarvadnya commented Aug 19, 2019 •

edited

Loading

tmancey commented Aug 19, 2019 •

edited

Loading

GeetaSarvadnya commented Sep 4, 2019 •

edited by btlechowski

Loading