Remove all marketing get parameters to minimize the cache #39099

rogerdz · 2024-08-23T08:05:21Z

Description (*)

Same logic when use varnish (https://github.com/magento/magento2/blob/2.4-develop/app/code/Magento/PageCache/etc/varnish7.vcl#L84) but for build-in fpc

Related Pull Requests

Fixed Issues (if relevant)

Fixes magento/magento2#<issue_number>

Manual testing scenarios (*)

Clear cache: bin/magento cache:clean
Run command: curl -I --location-trusted 'https://magento2.test/gear/bags.html?test=111&utm_source=123&gclid=abc' | grep -i X-Magento-Cache-Debug, it return x-magento-cache-debug: MISS => correct
Run command: curl -I --location-trusted 'https://magento2.test/gear/bags.html?test=111&utm_source=123&gclid=abc' | grep -i X-Magento-Cache-Debug, it return x-magento-cache-debug: HIT => correct
Run command: curl -I --location-trusted 'https://magento2.test/gear/bags.html?test=111' | grep -i X-Magento-Cache-Debug => it should return x-magento-cache-debug: HIT

Questions or comments

Contribution checklist (*)

Pull request has a meaningful description of its purpose
All commits are accompanied by meaningful commit messages
All new or changed code is covered with unit/integration tests (if applicable)
README.md files for modified modules are updated and included in the pull request if any README.md predefined sections require an update
All automated tests passed successfully (all builds are green)

Resolved issues:

resolves [Issue] Remove all marketing get parameters to minimize the cache #39266: Remove all marketing get parameters to minimize the cache

m2-assistant · 2024-08-23T08:05:25Z

Hi @rogerdz. Thank you for your contribution!
Here are some useful tips on how you can test your changes using Magento test environment.

Add the comment under your pull request to deploy test or vanilla Magento instance:

@magento give me test instance - deploy test instance based on PR changes
@magento give me 2.4-develop instance - deploy vanilla Magento instance

❗ Automated tests can be triggered manually with an appropriate comment:

@magento run all tests - run or re-run all required tests against the PR changes
@magento run <test-build(s)> - run or re-run specific test build(s)
For example: @magento run Unit Tests

<test-build(s)> is a comma-separated list of build names.

Allowed build names are:

Database Compare
Functional Tests CE
Functional Tests EE
Functional Tests B2B
Integration Tests
Magento Health Index
Sample Data Tests CE
Sample Data Tests EE
Sample Data Tests B2B
Static Tests
Unit Tests
WebAPI Tests
Semantic Version Checker

You can find more information about the builds here
ℹ️ Run only required test builds during development. Run all test builds before sending your pull request for review.

For more details, review the Code Contributions documentation.
Join Magento Community Engineering Slack and ask your questions in #github channel.

rogerdz · 2024-08-23T08:09:59Z

@magento run all tests

lib/internal/Magento/Framework/App/PageCache/Identifier.php

rogerdz · 2024-08-23T15:11:41Z

@magento run all tests

sprankhub · 2024-10-07T15:01:17Z

Nice. Maybe you want to update the parameters based on https://github.com/magento/magento2/pull/39188/files#diff-40e00b332f8dd95fbf48a312b9b8cfeee0d4e70bd7528c0e794c84c9e00c113fR85.

engcom-Hotel · 2024-10-16T06:43:33Z

@magento create issue

engcom-Hotel · 2024-10-16T06:43:52Z

@magento run all tests

engcom-Hotel · 2024-10-16T09:52:09Z

@magento run Database Compare, Functional Tests B2B, Functional Tests CE

sprankhub · 2024-10-16T11:18:56Z

Please mind that #39188 is also community-picked and is more complete. Specifically, it includes the changes from this PR. Hence, I think the solution from #39188 should be merged.

hostep · 2024-10-16T18:10:36Z

@sprankhub, I think you got confused and meant to comment on #35228
This PR here adds the extra parameters to the non-Varnish based FPC solution from Magento. But I believe the list should be updated with the list from #39188 so they match 100%.

Ideally we should have one place where the entire list is defined and can be used both by the Varnish VCL file generator and the non-Varnish based FPC solution. Maybe using a backoffice configuration field? We have a new request for that since today BTW: #39268 (even though the example given there is a very bad idea)

ihor-sviziev

If I got right, you’re removing cache parameters on Magento (php) side.
for that purpose, Varnish does way better job, since it works way better and uses resources more efficiently.
I don’t see how someone would use built in full page cache in production, so I don’t think we should update it

Example use case:
you added a new service, let’s say Klaviyo, that adds a dynamic value for each client.
Marketing team has sent an email to all subscribers and they starting clicking on link that goes to a single product page.
What we have right now is - the dynamic get parameter will be removed by Varnish, and after the first request - it will put the result in cache. all next requests will not get to Magento/php - so they will be delivered to clients waaaay faster and won’t create load to web servers, database and redis.
if we accept your changes - this would mean that this dynamic get parameter will be removed on php side, which will result in additional slower responses, additional load to web servers, database and redis.

Having all that in mind, I don’t think we can accept this pull request, but instead, we should actualize the list in Varnish config.

Please correct me if I’m wrong

ihor-sviziev · 2024-10-17T12:32:18Z

Sorry, looks like I got confused a bit.
But still, it’s not clear if it worth fixing built in full page cache. Does anyone using it in production? If no - why would we support the list in 2 places?

hostep · 2024-10-17T16:21:18Z

Yes, people use Magento shops without Varnish in production as Varnish requires a big amount of memory and certain shops have soo little traffic it's not worth paying the extra cash for extra resources. In that case the built-in FPC using filesystem as storage is more then adequate enough. Some of our clients use this sort of setup.

ihor-sviziev

Could you please cover your changes with unit tests?

rogerdz · 2024-10-28T06:18:07Z

@magento run all tests

rogerdz · 2024-10-28T08:39:04Z

Could you please cover your changes with unit tests?

I updated unit test + marketing parameters

ilnytskyi · 2024-10-28T13:47:09Z

lib/internal/Magento/Framework/App/PageCache/Identifier.php

+     * Pattern detect marketing parameters
+     */
+    public const PATTERN_MARKETING_PARAMETERS = [
+        '/&?gad_source\=[^&]+/',


I would suggest to extract this list to method that can be later pluginized. Or use dedicated class where those patterns can be injected via DI. Or even better to allow editing this list via admin panel.

If merchant uses some local marketing tools that are not part of this list then - hardcoded public const PATTERN_MARKETING_PARAMETERS cannot be easily overridden to remove custom tracking params.

As someone in CR suggested this service can also be used to generate Varnish config.

ilnytskyi · 2024-10-28T13:50:26Z

app/code/Magento/PageCache/Model/App/Request/Http/IdentifierForSave.php

@@ -38,9 +48,11 @@ public function __construct(
     */
    public function getValue()
    {
+        $pattern = Identifier::PATTERN_MARKETING_PARAMETERS;


This line is not extendable to append custom patterns. Dedicated service or public method visible to plugins would work better.

hostep · 2024-10-28T15:44:34Z

app/code/Magento/PageCache/Model/App/Request/Http/IdentifierForSave.php

+ * property laws, including trade secret and copyright laws.
+ * Dissemination of this information or reproduction of this material
+ * is strictly forbidden unless prior written permission is obtained from
+ * Adobe.


This copyright header is too big.
Also the year should be the year the file was created, not when it was last updated.

So in this case the copyright header should be:

* Copyright 2023 Adobe * All Rights Reserved.

Nothing else is needed. Not sure where you got that other paragraph from?

Same remark goes for the other files.

It's better to be moved to separate file COPYING.txt or LICENSE.txt for better opportunity for update the policies.

It's takes years to remove @author tag (comments block) from files, what will happens if all files requires updated license ...

Please do not use any year dates, just use the default magento header here:

<?php /** * Copyright © Magento, Inc. All rights reserved. * See COPYING.txt for license details. */

new rule from magento coding standard (https://github.com/magento/magento-coding-standard/blob/develop/Magento2Framework/Sniffs/Header/CopyrightValidation.php)
I'm copy from core
Have 94 file same format

It couldn't have been made any longer lol but thanks, I hadn't thought of that either

Remove all marketing get parameters to minimize the cache

c9dd5e2

m2-community-project bot added the Progress: pending review label Aug 23, 2024

Bashev reviewed Aug 23, 2024

View reviewed changes

lib/internal/Magento/Framework/App/PageCache/Identifier.php Show resolved Hide resolved

m2-community-project bot assigned Bashev Aug 23, 2024

m2-community-project bot added Progress: review and removed Progress: pending review labels Aug 23, 2024

fix test

4a7bcca

engcom-Hotel added the Priority: P2 A defect with this priority could have functionality issues which are not to expectations. label Aug 27, 2024

Priyakshic added the Project: Community Picked PRs upvoted by the community label Oct 15, 2024

engcom-Hotel self-requested a review October 16, 2024 06:43

m2-assistant bot mentioned this pull request Oct 16, 2024

[Issue] Remove all marketing get parameters to minimize the cache #39266

Open

5 tasks

Merge branch '2.4-develop' into remove_marketing

c01f149

engcom-Hotel approved these changes Oct 16, 2024

View reviewed changes

m2-community-project bot assigned engcom-Hotel Oct 16, 2024

m2-community-project bot added Progress: ready for testing and removed Progress: review labels Oct 16, 2024

ihor-sviziev requested changes Oct 17, 2024

View reviewed changes

m2-community-project bot assigned ihor-sviziev Oct 17, 2024

m2-community-project bot added Progress: needs update and removed Progress: ready for testing labels Oct 17, 2024

ihor-sviziev requested changes Oct 19, 2024

View reviewed changes

rogerdz added 4 commits October 28, 2024 09:52

Merge branch '2.4-develop' into remove_marketing

a575c33

update params

d3df409

add unit test

a80d026

fix unit test

e4fce5a

ilnytskyi reviewed Oct 28, 2024

View reviewed changes

m2-community-project bot assigned ilnytskyi Oct 28, 2024

m2-community-project bot added Progress: review and removed Progress: needs update labels Oct 28, 2024

ilnytskyi reviewed Oct 28, 2024

View reviewed changes

hostep reviewed Oct 28, 2024

View reviewed changes

m2-community-project bot assigned hostep Oct 28, 2024

hostep removed their assignment Oct 28, 2024

m2-community-project bot assigned in-session Oct 28, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove all marketing get parameters to minimize the cache #39099

Remove all marketing get parameters to minimize the cache #39099

rogerdz commented Aug 23, 2024 •

edited by m2-assistant bot

Loading

m2-assistant bot commented Aug 23, 2024

rogerdz commented Aug 23, 2024

rogerdz commented Aug 23, 2024

sprankhub commented Oct 7, 2024

engcom-Hotel commented Oct 16, 2024

engcom-Hotel commented Oct 16, 2024

engcom-Hotel commented Oct 16, 2024

sprankhub commented Oct 16, 2024

hostep commented Oct 16, 2024 •

edited

Loading

ihor-sviziev left a comment •

edited

Loading

ihor-sviziev commented Oct 17, 2024

hostep commented Oct 17, 2024

ihor-sviziev left a comment

rogerdz commented Oct 28, 2024

rogerdz commented Oct 28, 2024 •

edited

Loading

ilnytskyi Oct 28, 2024

ilnytskyi Oct 28, 2024

hostep Oct 28, 2024

Bashev Oct 28, 2024

in-session Oct 28, 2024

rogerdz Oct 28, 2024 •

edited

Loading

in-session Oct 28, 2024

Remove all marketing get parameters to minimize the cache #39099

Are you sure you want to change the base?

Remove all marketing get parameters to minimize the cache #39099

Conversation

rogerdz commented Aug 23, 2024 • edited by m2-assistant bot Loading

Description (*)

Related Pull Requests

Fixed Issues (if relevant)

Manual testing scenarios (*)

Questions or comments

Contribution checklist (*)

Resolved issues:

m2-assistant bot commented Aug 23, 2024

rogerdz commented Aug 23, 2024

rogerdz commented Aug 23, 2024

sprankhub commented Oct 7, 2024

engcom-Hotel commented Oct 16, 2024

engcom-Hotel commented Oct 16, 2024

engcom-Hotel commented Oct 16, 2024

sprankhub commented Oct 16, 2024

hostep commented Oct 16, 2024 • edited Loading

ihor-sviziev left a comment • edited Loading

Choose a reason for hiding this comment

ihor-sviziev commented Oct 17, 2024

hostep commented Oct 17, 2024

ihor-sviziev left a comment

Choose a reason for hiding this comment

rogerdz commented Oct 28, 2024

rogerdz commented Oct 28, 2024 • edited Loading

ilnytskyi Oct 28, 2024

Choose a reason for hiding this comment

ilnytskyi Oct 28, 2024

Choose a reason for hiding this comment

hostep Oct 28, 2024

Choose a reason for hiding this comment

Bashev Oct 28, 2024

Choose a reason for hiding this comment

in-session Oct 28, 2024

Choose a reason for hiding this comment

rogerdz Oct 28, 2024 • edited Loading

Choose a reason for hiding this comment

in-session Oct 28, 2024

Choose a reason for hiding this comment

rogerdz commented Aug 23, 2024 •

edited by m2-assistant bot

Loading

hostep commented Oct 16, 2024 •

edited

Loading

ihor-sviziev left a comment •

edited

Loading

rogerdz commented Oct 28, 2024 •

edited

Loading

rogerdz Oct 28, 2024 •

edited

Loading