Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement Brave Ads Serving Improvements #9736

Merged
merged 1 commit into from
Sep 23, 2021
Merged

Implement Brave Ads Serving Improvements #9736

merged 1 commit into from
Sep 23, 2021

Conversation

moritzhaller
Copy link
Contributor

@moritzhaller moritzhaller commented Aug 12, 2021 •
edited
Loading

Resolves brave/brave-browser#17236

Submitter Checklist:

  • I confirm that no security/privacy review is needed, or that I have requested one
  • There is a ticket for my issue
  • Used Github auto-closing keywords in the PR description above
  • Wrote a good PR/commit description
  • Added appropriate labels (QA/Yes or QA/No; release-notes/include or release-notes/exclude; OS/...) to the associated issue
  • Checked the PR locally: npm run test -- brave_browser_tests, npm run test -- brave_unit_tests, npm run lint, npm run gn_check, npm run tslint
  • Ran git rebase master (if needed)

Reviewer Checklist:

  • A security review is not needed, or a link to one is included in the PR description
  • New files have MPL-2.0 license header
  • Adequate test coverage exists to prevent regressions
  • Major classes, functions and non-trivial code blocks are well-commented
  • Changes in component dependencies are properly reflected in gn
  • Code follows the style guide
  • Test plan is specified in PR before merging

After-merge Checklist:

Test Plan:

Ad Notification Serving Version 1:

  1. Run browser with default args npm start -- --enable-logging=stderr --vmodule="*/variations/*"=1,"*/bat-native-ads/*"=6,"*/brave_ads/*"=6,"*/bat_ads/*"=6
  2. Opt-in to rewards, accrue user activity by visiting brave.com in new tab and trigger ad notification from idle
  3. Verify successful ad delivery and ad serving version 1 in logs
[ad_notification_serving.cc(111)] Ad serving version 1
[eligible_ad_notifications.cc(165)] Get eligible ads for parent-child segments:
[eligible_ad_notifications.cc(167)]   technology & computing-software
[eligible_ad_notifications.cc(167)]   personal finance-personal finance
[eligible_ad_notifications.cc(167)]   food & drink-vegetarian
[eligible_ad_notifications.cc(178)] No eligible ads for parent-child segments
[eligible_ad_notifications.cc(199)] Get eligible ads for parent segments:
[eligible_ad_notifications.cc(201)]   technology & computing
[eligible_ad_notifications.cc(201)]   personal finance
[eligible_ad_notifications.cc(201)]   food & drink
[eligible_ad_notifications.cc(212)] No eligible ads for parent segments
[eligible_ad_notifications.cc(224)] Get eligible ads for untargeted segment
[ad_priority.h(32)] 4 ads with a priority of 1 in bucket 1
[ad_notification_serving.cc(148)] Found 4 eligible ads
[ad_notification_serving.cc(272)] Serving ad notification:
  uuid: 2d5d929f-5792-4da9-a6be-de306657c44a
  creativeInstanceId: 40f9f892-2eee-4b8f-8bde-45915ae2fd88
  creativeSetId: e76b526f-79dd-4111-8778-ca7b194c66fa
  campaignId: dd64fb34-f683-45b3-b2ba-dd6e1715f48f
  advertiserId: dd3d043b-51ca-4c6c-8ddd-16afa165ce1b
  segment: untargeted
  title: Progressive Insurance
  body: Get a quote to see how much you could save on car insurance.
  targetUrl: https://www.brave.com/jobs

Ad Notification Serving Version 2:

  1. Run browser with default args npm start -- --enable-logging=stderr --vmodule="*/variations/*"=1,"*/bat-native-ads/*"=6,"*/brave_ads/*"=6,"*/bat_ads/*"=6 --enable-features=AdServing:ad_serving_version/2
  2. Opt-in to rewards, accrue user activity by visiting brave.com in new tab and trigger ad notification from idle
  3. Verify successful ad delivery and ad serving version 1 in logs
[ad_notification_serving.cc(111)] Ad serving version 2
[eligible_ad_notifications.cc(102)] Get eligible ads
[ad_notification_serving.cc(272)] Serving ad notification:
  uuid: 885121de-bdf4-4f52-b9c8-c841a85f7a41
  creativeInstanceId: 2c658159-e18a-4ba7-b4f0-2c51e4eae9ad
  creativeSetId: e76b526f-79dd-4111-8778-ca7b194c66fa
  campaignId: dd64fb34-f683-45b3-b2ba-dd6e1715f48f
  advertiserId: dd3d043b-51ca-4c6c-8ddd-16afa165ce1b
  segment: untargeted
  title: abcdefghijklmnopqrstuvwxyz
  body: abcdefghijklmnopqrstuvwxyz
  targetUrl: https://www.brave.com/foo

Inline Content Ad Serving Version 1:

  1. Run browser with default args npm start -- --enable-logging=stderr --vmodule="*/variations/*"=1,"*/bat-native-ads/*"=6,"*/brave_ads/*"=6,"*/bat_ads/*"=6
  2. Accrue user activity by visiting brave.com in new tab
  3. Open new tab page, enabled Brave News and scroll below the fold
  4. Verify successful ad delivery and ad serving version 1 in logs
[inline_content_ad_serving.cc(68)] Ad serving version 1
[eligible_inline_content_ads.cc(171)] Get eligible ads for parent-child segments:
[eligible_inline_content_ads.cc(173)]   technology & computing-software
[eligible_inline_content_ads.cc(173)]   personal finance-personal finance
[eligible_inline_content_ads.cc(173)]   food & drink-vegetarian
[eligible_inline_content_ads.cc(185)] No eligible ads for parent-child segments
[eligible_inline_content_ads.cc(207)] Get eligible ads for parent segments:
[eligible_inline_content_ads.cc(209)]   technology & computing
[eligible_inline_content_ads.cc(209)]   personal finance
[eligible_inline_content_ads.cc(209)]   food & drink
[eligible_inline_content_ads.cc(221)] No eligible ads for parent segments
[eligible_inline_content_ads.cc(234)] Get eligible ads for untargeted segment
[ad_priority.h(32)] 1 ads with a priority of 1 in bucket 1
[inline_content_ad_serving.cc(104)] Found 1 eligible ads
[inline_content_ad_serving.cc(129)] Serving inline content ad:
  uuid: 6d5631de-1703-4601-b12e-41a6d627c7fa
  creativeInstanceId: 033a95ca-963b-460c-b203-8181346bd41c
  creativeSetId: 6c875e23-00ea-4f1b-805b-5d52559f96da
  campaignId: 8a5ca6f0-f919-461b-8bf2-4848e34ab616
  advertiserId: aa1d1cdb-ec45-42d4-86d0-ea353e1babd4
  segment: untargeted
  title: Makita LF1000 Flip Over Saw
  description: Makita
  imageUrl: https://pcdn.bravesoftware.com/brave-ads/image-1.jpg.pad
  dimensions: 900x750
  ctaText: Learn more
  targetUrl: https://www.powertools2u.co.uk/makita-lf1000-flip-over-sawo

Inline Content Ad Serving Version 2:

  1. Run browser with default args npm start -- --enable-logging=stderr --vmodule="*/variations/*"=1,"*/bat-native-ads/*"=6,"*/brave_ads/*"=6,"*/bat_ads/*"=6 --enable-features=AdServing:ad_serving_version/2
  2. Accrue user activity by visiting brave.com in new tab
  3. Open new tab page, enabled Brave News and scroll below the fold
  4. Verify successful ad delivery and ad serving version 2 in logs
[inline_content_ad_serving.cc(68)] Ad serving version 2
[text_classification_model.cc(86)] No text classification probabilities found for en-GB locale
[eligible_inline_content_ads.cc(105)] Get eligible ads
[inline_content_ad_serving.cc(176)] Serving inline content ad:
  uuid: e6b48954-3ed3-46c9-8b37-0b823f59ad86
  creativeInstanceId: 033a95ca-963b-460c-b203-8181346bd41c
  creativeSetId: 6c875e23-00ea-4f1b-805b-5d52559f96da
  campaignId: 8a5ca6f0-f919-461b-8bf2-4848e34ab616
  advertiserId: aa1d1cdb-ec45-42d4-86d0-ea353e1babd4
  segment: untargeted
  title: Makita LF1000 Flip Over Saw
  description: Makita
  imageUrl: https://pcdn.bravesoftware.com/brave-ads/image-1.jpg.pad
  dimensions: 900x750
  ctaText: Learn more
  targetUrl: https://www.powertools2u.co.uk/makita-lf1000-flip-over-sawo

@moritzhaller moritzhaller force-pushed the issues/17236 branch 2 times, most recently from efdf641 to 6b43e15 Compare August 14, 2021 13:18
@moritzhaller moritzhaller changed the title Implement Brave Ads serving improvements Implement Brave Ads Serving Improvements Aug 17, 2021
@moritzhaller moritzhaller force-pushed the issues/17236 branch 4 times, most recently from 0124946 to 59436f3 Compare August 18, 2021 11:03
@moritzhaller moritzhaller marked this pull request as ready for review August 18, 2021 11:03
@moritzhaller moritzhaller force-pushed the issues/17236 branch 4 times, most recently from 0284b06 to 3392c5e Compare August 23, 2021 17:41
tmancey
tmancey requested changes Aug 23, 2021
View reviewed changes
vendor/bat-native-ads/src/bat/ads/internal/ad_targeting/ad_targeting.cc Outdated Show resolved Hide resolved
vendor/bat-native-ads/src/bat/ads/internal/ad_targeting/ad_targeting.cc Outdated Show resolved Hide resolved
vendor/bat-native-ads/src/bat/ads/internal/ad_targeting/ad_targeting.h Outdated Show resolved Hide resolved
vendor/bat-native-ads/src/bat/ads/internal/bundle/creative_ad_info.cc Outdated Show resolved Hide resolved
vendor/bat-native-ads/src/bat/ads/internal/container_util.h Outdated Show resolved Hide resolved
vendor/bat-native-ads/src/bat/ads/internal/eligible_ads/eligible_ads_util_unittest.cc Outdated Show resolved Hide resolved
vendor/bat-native-ads/src/bat/ads/internal/eligible_ads/sample_ads.h Outdated Show resolved Hide resolved
vendor/bat-native-ads/src/bat/ads/internal/eligible_ads/sample_ads.h Outdated Show resolved Hide resolved
vendor/bat-native-ads/src/bat/ads/internal/eligible_ads/sample_ads_unittest.cc Outdated Show resolved Hide resolved
vendor/bat-native-ads/src/bat/ads/internal/features/ad_serving/ad_serving_features.h Outdated Show resolved Hide resolved
@aseren aseren self-requested a review September 22, 2021 12:19
@moritzhaller moritzhaller force-pushed the issues/17236 branch 5 times, most recently from 4293dbb to f0cf771 Compare September 22, 2021 20:20
@moritzhaller moritzhaller force-pushed the issues/17236 branch 2 times, most recently from 24e31fb to 994c25c Compare September 22, 2021 21:35
tmancey
tmancey approved these changes Sep 22, 2021
View reviewed changes
Copy link
Collaborator

@tmancey tmancey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM++

@moritzhaller
Copy link
Contributor Author

CI/Android failing with unrelated issues

Implement Brave Ads Serving Improvements
97eb067 
Currently, ad serving is done by randomly picking ads based on matching segments. The approach has several problems like under-delivery of creatives and users not seeing as many ads as they could. This PR aims to solve these short comings by introducing a statistical model for ad selection. We assume that the click probability can be approximated by a linear function of features related to locally observed user behaviour. The function is weighted by a control-parameter called pass-through rate (ptr) to throttle delivery of individual ads.
@moritzhaller moritzhaller merged commit cb055c4 into master Sep 23, 2021
@moritzhaller moritzhaller deleted the issues/17236 branch September 23, 2021 02:42
@moritzhaller moritzhaller added this to the 1.32.x - Nightly milestone Sep 23, 2021
@moritzhaller moritzhaller mentioned this pull request Sep 23, 2021
Closed
@kjozwiak kjozwiak mentioned this pull request Oct 26, 2021
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tmancey tmancey tmancey approved these changes

@aseren aseren aseren left review comments

Assignees

@moritzhaller moritzhaller

Labels
None yet
Projects
None yet
Milestone
1.32.x - Release
Development

Successfully merging this pull request may close these issues.

Implement Brave Ads Serving Improvements
3 participants
@moritzhaller @tmancey @aseren