Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reporting: Track events for View report links on Payment activity widget #8687

Merged
merged 18 commits into from
May 1, 2024

Conversation

nagpai
Copy link
Contributor

@nagpai nagpai commented Apr 22, 2024

Fixes #8686

Changes proposed in this Pull Request

Add track events to track clicks on View report links on the following tiles:

  • Total payment volume
  • Charges
  • Refunds
  • Disputes

Testing instructions

  • Enable feature flag, wp option update _wcpay_feature_payment_overview_widget 1.
  • Use Tracks Vigilante chrome extension to track clicks from your local test site
  • Check out this branch and run npm start or npm run watch to build assets
  • Browse to Payments and on the overview page, click on the View report link that you see over Total payment volume tile. Click the link
  • Check the tracked events on the Tracks Vigilante chrome extension. You should see the event wcpay_overview_total_payment_volume_view_report_clicks . Check the other details shared with the event, including the source that should read as total_payment_volume_tile_view_report
  • Test the links on Charges, Refunds and Disputes tile too and see if they generate relevant track events.
Reminder: test this change will achieve the desired impact/result!

Reviewers please keep in mind the flow and funnel that we want to measure, and check that this event will allow us to measure the desired funnel conversion rate. i.e. test the whole flow, not just that this event fires.

  • Can measure conversion rate of: View payments overview ⇒ click View report ⇒ view relevant report screen

See the project thread on P2 for more details and example funnel.

Notes

Discussion on naming events - p1713786808704209-slack-C06BBFTF6EM


  • Run npm run changelog to add a changelog file, choose patch to leave it empty if the change is not significant. You can add multiple changelog files in one PR by running this command a few times.
  • Covered with tests (or have a good reason not to test in description ☝️)
  • Tested on mobile (or does not apply)

Post merge

@botwoo
Copy link
Collaborator

botwoo commented Apr 22, 2024

Test the build

Option 1. Jetpack Beta

  • Install and activate Jetpack Beta.
  • Use this build by searching for PR number 8687 or branch name add/8686-track-events-payment-activity-widget in your-test.site/wp-admin/admin.php?page=jetpack-beta&plugin=woocommerce-payments

Option 2. Jurassic Ninja - available for logged-in A12s

🚀 Launch a JN site with this branch 🚀

ℹ️ Install this Tampermonkey script to get more options.


Build info:

  • Latest commit: 83a690e
  • Build time: 2024-04-29 15:02:06 UTC

Note: the build is updated when a new commit is pushed to this PR.

Copy link
Contributor

github-actions bot commented Apr 22, 2024

Size Change: +41 B (0%)

Total Size: 1.25 MB

Filename Size Change
release/woocommerce-payments/dist/index.js 292 kB +41 B (0%)
ℹ️ View Unchanged
Filename Size
release/woocommerce-payments/assets/css/admin.css 1.08 kB
release/woocommerce-payments/assets/css/admin.rtl.css 1.08 kB
release/woocommerce-payments/assets/css/success.css 158 B
release/woocommerce-payments/assets/css/success.rtl.css 158 B
release/woocommerce-payments/dist/blocks-checkout-rtl.css 2.06 kB
release/woocommerce-payments/dist/blocks-checkout.css 2.07 kB
release/woocommerce-payments/dist/blocks-checkout.js 56.3 kB
release/woocommerce-payments/dist/bnpl-announcement-rtl.css 530 B
release/woocommerce-payments/dist/bnpl-announcement.css 531 B
release/woocommerce-payments/dist/bnpl-announcement.js 20 kB
release/woocommerce-payments/dist/cart-block.js 21.4 kB
release/woocommerce-payments/dist/cart.js 4.38 kB
release/woocommerce-payments/dist/checkout-rtl.css 599 B
release/woocommerce-payments/dist/checkout.css 599 B
release/woocommerce-payments/dist/checkout.js 37.5 kB
release/woocommerce-payments/dist/index-rtl.css 40.7 kB
release/woocommerce-payments/dist/index.css 40.6 kB
release/woocommerce-payments/dist/multi-currency-analytics.js 1.05 kB
release/woocommerce-payments/dist/multi-currency-rtl.css 3.28 kB
release/woocommerce-payments/dist/multi-currency-switcher-block.js 59.5 kB
release/woocommerce-payments/dist/multi-currency.css 3.29 kB
release/woocommerce-payments/dist/multi-currency.js 54.6 kB
release/woocommerce-payments/dist/order-rtl.css 733 B
release/woocommerce-payments/dist/order.css 735 B
release/woocommerce-payments/dist/order.js 41.8 kB
release/woocommerce-payments/dist/payment-gateways-rtl.css 1.21 kB
release/woocommerce-payments/dist/payment-gateways.css 1.21 kB
release/woocommerce-payments/dist/payment-gateways.js 38.6 kB
release/woocommerce-payments/dist/payment-request-rtl.css 155 B
release/woocommerce-payments/dist/payment-request.css 155 B
release/woocommerce-payments/dist/payment-request.js 12 kB
release/woocommerce-payments/dist/product-details-rtl.css 398 B
release/woocommerce-payments/dist/product-details.css 402 B
release/woocommerce-payments/dist/product-details.js 17.2 kB
release/woocommerce-payments/dist/settings-rtl.css 11.1 kB
release/woocommerce-payments/dist/settings.css 10.9 kB
release/woocommerce-payments/dist/settings.js 201 kB
release/woocommerce-payments/dist/subscription-edit-page.js 669 B
release/woocommerce-payments/dist/subscription-product-onboarding-modal-rtl.css 527 B
release/woocommerce-payments/dist/subscription-product-onboarding-modal.css 527 B
release/woocommerce-payments/dist/subscription-product-onboarding-modal.js 19.4 kB
release/woocommerce-payments/dist/subscription-product-onboarding-toast.js 693 B
release/woocommerce-payments/dist/subscriptions-empty-state-rtl.css 120 B
release/woocommerce-payments/dist/subscriptions-empty-state.css 120 B
release/woocommerce-payments/dist/subscriptions-empty-state.js 18.5 kB
release/woocommerce-payments/dist/tos-rtl.css 235 B
release/woocommerce-payments/dist/tos.css 236 B
release/woocommerce-payments/dist/tos.js 21 kB
release/woocommerce-payments/dist/woopay-direct-checkout.js 4.54 kB
release/woocommerce-payments/dist/woopay-express-button-rtl.css 155 B
release/woocommerce-payments/dist/woopay-express-button.css 155 B
release/woocommerce-payments/dist/woopay-express-button.js 21 kB
release/woocommerce-payments/dist/woopay-rtl.css 4.25 kB
release/woocommerce-payments/dist/woopay.css 4.22 kB
release/woocommerce-payments/dist/woopay.js 69.4 kB
release/woocommerce-payments/includes/subscriptions/assets/css/plugin-page.css 622 B
release/woocommerce-payments/includes/subscriptions/assets/js/plugin-page.js 815 B
release/woocommerce-payments/vendor/automattic/jetpack-assets/build/i18n-loader.js 2.44 kB
release/woocommerce-payments/vendor/automattic/jetpack-assets/src/js/i18n-loader.js 1.01 kB
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/tracks-ajax.js 522 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/tracks-callables.js 581 B
release/woocommerce-payments/vendor/automattic/jetpack-identity-crisis/babel.config.js 160 B
release/woocommerce-payments/vendor/automattic/jetpack-identity-crisis/build/index.css 2.37 kB
release/woocommerce-payments/vendor/automattic/jetpack-identity-crisis/build/index.js 13.5 kB
release/woocommerce-payments/vendor/automattic/jetpack-identity-crisis/build/index.rtl.css 2.37 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/about.css 1.03 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/admin-empty-state.css 291 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/admin-order-statuses.css 403 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/admin.css 3.6 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/checkout.css 299 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/modal.css 742 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/view-subscription.css 572 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/wcs-upgrade.css 411 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/admin-pointers.js 544 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/admin.js 9.4 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/jstz.js 6.8 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/jstz.min.js 3.83 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/meta-boxes-coupon.js 544 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/meta-boxes-subscription.js 2.52 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/moment.js 22.1 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/moment.min.js 11.6 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/payment-method-restrictions.js 1.29 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/wcs-meta-boxes-order.js 502 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/frontend/payment-methods.js 355 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/frontend/single-product.js 429 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/frontend/view-subscription.js 1.38 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/frontend/wcs-cart.js 781 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/modal.js 1.1 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/wcs-upgrade.js 1.27 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/build/index.css 392 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/build/index.js 3.05 kB

compressed-size-action

@nagpai nagpai requested a review from a team April 23, 2024 11:09
@nagpai nagpai marked this pull request as ready for review April 23, 2024 11:09
@nagpai nagpai requested review from a team, jessy-p and shendy-a8c April 24, 2024 07:47
@haszari
Copy link
Contributor

haszari commented Apr 26, 2024

Added note to description – reviewers please test and review these events in context, to ensure they meet the needs of project:

  • Can measure conversion rate of: View payments overview ⇒ click View report ⇒ view relevant report screen

Copy link
Contributor

@haszari haszari left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good from a quick test!

I think a generic "tile clicked" event + source prop would be cleaner and work well (and more flexible).

I don't like passing callbacks without good reason :)

'wcpay_overview_payment_activity_total_payment_volume_click',
{
source:
'total_payment_volume_tile_view_report_link',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need the source parameter (I'm assuming it's optional)? I think this is used to distinguish generic events by the originating high-level component or container. How do other WooPayments events use source?

In this case we have unique events for each click target, so source is redundant.

An alternative design would be to use a generic event name, and distinguish which box using source:

  • en wcpay_overview_payment_activity_click
  • source total_payment_volume

The question is whether we'd want to aggregate any click on the activity widget. If so, a common event is useful. If not, unique event for each thing works well.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea of using a common event for the tile is fantastic, and makes things totally clean! Thanks for opening my eyes to this possibility! Also a great note for me "Is this a good enough reason to pass a callback?" . Clearly no in this case with the alternative you suggested :D .

I have implemented a common click event, and a source that is extrapolated from the existing label prop. ( make it all small case and replace spaces with underscores, where needed)

I hope I have done it appropriately here : 5069046

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

I'd suggest using an explicit source value for each tile, rather than generating from label. This is one more prop (tracksSource?), but it's simple and explicit, and allows future tracks investigators to search the code for the source value to work out where it's getting fired.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have added an explicit tracksSource prop here - 5c5da5b . Thanks!

/**
* Optional click handler for the tile.
*/
handleReportLinkClick?: () => void;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is our PaymentDataTile abstraction helping here? If we expose a prop for every detail of it, we might be better off just using Link directly in the outer markup.

Alternatively, if we are ok using the same event name for all tiles e.g. wcpay_overview_payment_tile_click, the component could abstract away the tracking behaviour nicely. It could use an existing prop or expose a tracksEventSource prop, which would avoid complex callback interface.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Explained in the earlier comment.

#8687 (comment)

Thanks!

@haszari haszari requested a review from a team April 26, 2024 03:59
@haszari
Copy link
Contributor

haszari commented Apr 26, 2024

re-requesting a Helix review as my review was only a quick look/test.

Copy link
Contributor

@shendy-a8c shendy-a8c left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Diff looks good and tested well.
Track events were fired when I clicked on the report links from the tiles in the Overview > Payment Activity widget.
It worked on mobile as well.

By the way, don't forget to register the new track event.

@shendy-a8c
Copy link
Contributor

Answering @haszari's question:

Can measure conversion rate of: View payments overview ⇒ click View report ⇒ view relevant report screen

Is 'conversion' a click to reports from the Payment Activity widget? If so, yes, with the newly added track event from this PR, we can measure that.

If we need more than that, for example, after clicking the link, user will land on the 'report' pages.
For the TPV link, user will land on the transaction list page, unfiltered.
For the charges link, user will land on the transactions list page, filtered to only show charges.
For the refunds link, user will land on the transactions list page, filtered to only show refunds.
For the disputes link, user will land on the disputes list page.

Views to the transactions list page and disputes list page are already tracked but we can't differentiate the filtering!

Looking at the success indicator paJDYF-bWn-p2, the step 'WCPay report views' is looking at wcadmin_page_view with path payments_deposits, payments_disputes, and payments_transactions.

Screenshot 2024-04-29 at 22 31 33

Viewing the reports from the TPV link, charges link, and refunds link will lead to an increase in wcadmin_page_view event with path = payments_transactions but since the filtering isn't included in the wcadmin_page_view event, we will get the total number of views to the transactions list page.

I'm struggling to articulate what I mean but I hope it's clear. Otherwise, I can try to explain more.

In summary:

  • If the click track event added by this PR is sufficient for us, we don't need to do anything more IMO.
  • Otherwise, we need to distinguish the tracking on the transactions list page based on the filter on that page.

@haszari
Copy link
Contributor

haszari commented Apr 29, 2024

Viewing the reports from the TPV link, charges link, and refunds link will lead to an increase in wcadmin_page_view event with path = payments_transactions but since the filtering isn't included in the wcadmin_page_view event, we will get the total number of views to the transactions list page.

I think that's fine, we want the user to click the tile and land on report/list page. As I understand it we don't particularly care what state the list view is in (from a tracking/flows POV, we do care of course that the report totals match the widget!).

We should clarify this detail with stakeholders / @souravdebnath1986 – so everyone is on the same page about how the events work.

@nagpai
Copy link
Contributor Author

nagpai commented May 1, 2024

I am merging this PR based on the discussion here - p1714467303200959-slack-C06BBFTF6EM

tl;dr, the funnel tool supports adding event properties, e.g. source, for any step. That should help us segregate traffic by tile.

@nagpai nagpai added this pull request to the merge queue May 1, 2024
Merged via the queue into develop with commit b4d4a0b May 1, 2024
22 of 23 checks passed
@nagpai nagpai deleted the add/8686-track-events-payment-activity-widget branch May 1, 2024 06:52
@haszari
Copy link
Contributor

haszari commented May 7, 2024

By the way, don't forget to register the new track event.

@nagpai checking – did you register these events as part of this or is that still needed? If not please open an issue & add to board so we remember and all our events are registered correctly.

@nagpai
Copy link
Contributor Author

nagpai commented May 7, 2024

@haszari I have registered the event. Sorry, forgot to reference that here. Here it is!

https://github.com/Automattic/tracks-events-registration/pull/2396

What's remaining is to create the funnels. I will update here once that is done

My apologies the PR seems to be related to an older name I was trying, before we moved to single event + different sources per tile. I will have the correct event carefully registered and update here.

@haszari
Copy link
Contributor

haszari commented May 7, 2024

My apologies the PR seems to be related to an older name I was trying, before we moved to single event + different sources per tile. I will have the correct event carefully registered and update here.

Suggest just adding an issue for the outcome we need, then anyone can pick it up – best to have all tasks logged on the team board so anyone can pick up the next priority, and we have one source of truth for what's left to do.

If there's no issue for registering all new / changed events I'd recommend adding one issue for that and mention the specific todos above so they don't get missed.

@nagpai
Copy link
Contributor Author

nagpai commented May 8, 2024

If there's no issue for registering all new / changed events I'd recommend adding one issue for that and mention the specific todos above so they don't get missed.

@haszari - Thanks for the guidance. I have added an issue to make sure the event is registered before we launch.

Automattic/tracks-events-registration#2466 - Added this to the pre-lauch check Epic too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Task] - Add Track events to measure click on 'View report' of each tile
5 participants