Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(Sims) Internal GTM and external GTAG/GA4 tracking via query parameters #60

Closed
oliver-phet opened this issue Sep 14, 2022 · 30 comments
Closed
Assignees

Comments

@oliver-phet
Copy link

oliver-phet commented Sep 14, 2022

In a meeting with @mattpen we discussed that the current "external" Google analytics code that allows external partners to insert their own tracking code via a query parameter will either need to be abandoned or updated to support Google Tag Manager (GTAG) and/or Google Analytics 4 (GA4).

We did a quick search on page views with UA query parameters and there were over 10,000 page views over the last week.
Big users were macmillan and discovery education

@oliver-phet
Copy link
Author

@mattpen mentioned it may be good to query macmillan and discovery education to find out if they are currently/planning to use GTAG or GA4.

@oliver-phet
Copy link
Author

KP confirmed, YES we will continue supporting external GTAG/GA4 tracking via query parameter.

@oliver-phet oliver-phet changed the title Should we continue to support external GTAG/GA4 tracking via query parameters Support external GTAG/GA4 tracking via query parameters Sep 20, 2022
@oliver-phet
Copy link
Author

We did a bit of digging at it appears MacMillan Learning and Discovery Education.

@oliver-phet
Copy link
Author

@jonathanolson I checked with KP on this need and we decided we need to support partners setting both a UA and GA4 via 2 query parameters. We don't plan to support external Tag Manager tracking at this time.

@jonathanolson
Copy link
Contributor

Current JS patch:

Index: js/analytics/google-analytics.js
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/js/analytics/google-analytics.js b/js/analytics/google-analytics.js
--- a/js/analytics/google-analytics.js	(revision 2e3125f719cec7fa52fbdea894c8cd75c257346c)
+++ b/js/analytics/google-analytics.js	(date 1668044941526)
@@ -67,11 +67,7 @@
     }, true );
 
     // {boolean} - Whether analytics.js successfully loaded, see https://github.com/phetsims/yotta/issues/30
-    let googleAnalyticsLoaded = false;
-
-    function onGoogleAnalyticsLoad() {
-      googleAnalyticsLoaded = true;
-    }
+    const googleAnalyticsLoaded = false;
 
     const pingParams = `${'pingver=3&' +
                        'project='}${encodeURIComponent( phet.chipper.project )}&` +
@@ -99,110 +95,72 @@
                `gaLoaded=${encodeURIComponent( googleAnalyticsLoaded )}` );
     }, false );
 
-    // Google Analytics snippet for loading the API
-    ( function( i, s, o, g, r, a, m ) {
-      i.GoogleAnalyticsObject = r;
-      i[ r ] = i[ r ] || function() {
-        // eslint-disable-next-line prefer-rest-params
-        ( i[ r ].q = i[ r ].q || [] ).push( arguments );
-      }, i[ r ].l = 1 * new Date(); // eslint-disable-line no-sequences
-      a = s.createElement( o ), m = s.getElementsByTagName( o )[ 0 ]; // eslint-disable-line no-sequences
-      a.async = 1;
-      a.src = g;
-      m.parentNode.insertBefore( a, m );
-    } )( window, document, 'script', `${document.location.protocol === 'https:' ? 'https:' : 'http:'}//www.google-analytics.com/analytics.js`, 'googleAnalytics' );
+    if ( phet.chipper.queryParameters.ga ) {
+      // Google Analytics snippet for loading the API
+      ( function( i, s, o, g, r, a, m ) {
+        i.GoogleAnalyticsObject = r;
+        i[ r ] = i[ r ] || function() {
+          // eslint-disable-next-line prefer-rest-params
+          ( i[ r ].q = i[ r ].q || [] ).push( arguments );
+        }, i[ r ].l = 1 * new Date(); // eslint-disable-line no-sequences
+        a = s.createElement( o ), m = s.getElementsByTagName( o )[ 0 ]; // eslint-disable-line no-sequences
+        a.async = 1;
+        a.src = g;
+        m.parentNode.insertBefore( a, m );
+      } )( window, document, 'script', `${document.location.protocol === 'https:' ? 'https:' : 'http:'}//www.google-analytics.com/analytics.js`, 'googleAnalytics' );
 
-    // Applies custom dimensions that are common for our main, third-party, and phet-io tracker
-    const phetPageviewOptions = {};
+      // Applies custom dimensions that are common for our main, third-party, and phet-io tracker
+      const phetPageviewOptions = {};
 
-    if ( phet.chipper.project ) {
-      phetPageviewOptions.dimension1 = phet.chipper.project; // simName custom dimension
-    }
-    if ( phet.chipper.version ) {
-      phetPageviewOptions.dimension2 = phet.chipper.version; // simVersion custom dimension
-    }
-    if ( phet.chipper.locale ) {
-      phetPageviewOptions.dimension3 = phet.chipper.locale; // simLocale custom dimension
-    }
-    if ( phet.chipper.buildTimestamp ) {
-      phetPageviewOptions.dimension4 = phet.chipper.buildTimestamp; // simBuildTimestamp custom dimension
-    }
-    phetPageviewOptions.dimension5 = loadType;
-    phetPageviewOptions.dimension6 = document.referrer;
+      if ( phet.chipper.project ) {
+        phetPageviewOptions.dimension1 = phet.chipper.project; // simName custom dimension
+      }
+      if ( phet.chipper.version ) {
+        phetPageviewOptions.dimension2 = phet.chipper.version; // simVersion custom dimension
+      }
+      if ( phet.chipper.locale ) {
+        phetPageviewOptions.dimension3 = phet.chipper.locale; // simLocale custom dimension
+      }
+      if ( phet.chipper.buildTimestamp ) {
+        phetPageviewOptions.dimension4 = phet.chipper.buildTimestamp; // simBuildTimestamp custom dimension
+      }
+      phetPageviewOptions.dimension5 = loadType;
+      phetPageviewOptions.dimension6 = document.referrer;
 
-    const offlineSimLocation = `offline/html/${phet.chipper.project}_${phet.chipper.locale}`;
-
-    // Put our function in the queue, to be invoked when the analytics.js has fully loaded.
-    // See https://github.com/phetsims/yotta/issues/30
-    window.googleAnalytics( onGoogleAnalyticsLoad );
-
-    // Main PhET tracker
-    window.googleAnalytics( 'create', {
-      trackingId: 'UA-5033201-1',
-      storage: 'none',
-      cookieDomain: 'none'
-    } );
-    if ( window.location.protocol === 'file:' ) {
-      window.googleAnalytics( 'set', 'checkProtocolTask', null );
-      window.googleAnalytics( 'set', 'checkStorageTask', null );
-      window.googleAnalytics( 'set', 'location', offlineSimLocation );
-    }
-    window.googleAnalytics( 'set', 'anonymizeIp', true );
-    window.googleAnalytics( 'send', 'pageview', phetPageviewOptions );
-
-    // PhET iO tracker (see https://github.com/phetsims/phetcommon/issues/26)
-    if ( phet.chipper.brand === 'phet-io' ) {
-      window.googleAnalytics( 'create', {
-        trackingId: 'UA-37615182-3',
-        storage: 'none',
-        cookieDomain: 'none',
-        name: 'io'
-      } );
-      if ( window.location.protocol === 'file:' ) {
-        window.googleAnalytics( 'io.set', 'checkProtocolTask', null );
-        window.googleAnalytics( 'io.set', 'checkStorageTask', null );
-        window.googleAnalytics( 'io.set', 'location', offlineSimLocation );
-      }
-      window.googleAnalytics( 'io.set', 'anonymizeIp', true );
-      window.googleAnalytics( 'io.send', 'pageview', phetPageviewOptions );
-    }
-
-    // Third-party PhET tracker (excludes non-third-party usage, see https://github.com/phetsims/yotta/issues/12)
-    if ( window.location.protocol !== 'file:' &&
-         !document.domain.match( /(.*\.colorado\.edu\.?$)|(^localhost$)|(^127\.0\.0\.1$)/ ) ) {
-      window.googleAnalytics( 'create', {
-        trackingId: 'UA-37615182-2',
-        storage: 'none',
-        cookieDomain: 'none',
-        name: 'thirdParty'
-      } );
-      window.googleAnalytics( 'thirdParty.set', 'anonymizeIp', true );
-      window.googleAnalytics( 'thirdParty.send', 'pageview', phetPageviewOptions );
-    }
-
-    // Hewlett tracker
-    window.googleAnalytics( 'create', {
-      trackingId: 'UA-5033010-35',
-      storage: 'none',
-      cookieDomain: 'phet.colorado.edu',
-      name: 'hewlett'
-    } );
-    window.googleAnalytics( 'hewlett.set', 'anonymizeIp', true );
-    window.googleAnalytics( 'hewlett.send', 'pageview' );
-
-    // External tracker
-    if ( phet.chipper.queryParameters.ga ) {
-      window.googleAnalytics( 'create', {
-        trackingId: phet.chipper.queryParameters.ga,
-        storage: 'none',
-        cookieDomain: 'none', // don't require the tracking from our site
-        name: 'external'
-      } );
-      window.googleAnalytics( 'external.set', 'anonymizeIp', true );
-      window.googleAnalytics( 'external.send', 'pageview', phet.chipper.queryParameters.gaPage || undefined );
-    }
-  }
+      // External tracker
+      if ( phet.chipper.queryParameters.ga ) {
+        window.googleAnalytics( 'create', {
+          trackingId: phet.chipper.queryParameters.ga,
+          storage: 'none',
+          cookieDomain: 'none', // don't require the tracking from our site
+          name: 'external'
+        } );
+        window.googleAnalytics( 'external.set', 'anonymizeIp', true );
+        window.googleAnalytics( 'external.send', 'pageview', phet.chipper.queryParameters.gaPage || undefined );
+      }
+    }
 
+    window.dataLayer = window.dataLayer || [];
+    window.dataLayer.push( {
+      simBrand: phet.chipper.brand,
+      simName: phet.chipper.project,
+      simVersion: phet.chipper.version,
+      simLocale: phet.chipper.locale,
+      simBuildTimestamp: phet.chipper.buildTimestamp,
+      simLoadType: loadType,
+      documentReferrer: document.referrer
+    } );
+
+    if ( phet.chipper.queryParameters.ga4 ) {
+      window.dataLayer.push( [
+        'config', phet.chipper.queryParameters.ga4
+      ] );
+    }
+
+    // Google Tag Manager
+    (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-WLNGBXD'); // eslint-disable-line
+  }
+
   if ( loadType === 'phet-app' ) {
     window.addEventListener( 'load', () => {
       setTimeout( sendMessages, 0 ); // eslint-disable-line bad-sim-text

@jonathanolson
Copy link
Contributor

@oliver-phet, I believe I've restored the bit that would handle UA (?ga= query parameter) and added ?ga4= for GA4 IDs. I'm not sure how to test GA4, and also test to make sure (b) ?ga4 works, and (b) that it doesn't mess with our Google Tag Manager.

They both need to add to the window.dataLayer, and I've spent a half hour looking for ways to do this that might not conflict (but haven't found it). Thoughts on how we should add GA4 support?

Additionally, what steps should I make to progress on this?

@mattpen
Copy link
Contributor

mattpen commented Nov 11, 2022

We could rename the window.dataLayer for each instance. Guide to renaming window.datalayer here: https://developers.google.com/tag-platform/tag-manager/web/datalayer#rename_the_data_layer. So we could have something like window.phetDataLayer for our GTM instance, window.gaDataLayer for the ?ga= instance, and window.ga4DataLayer for the ?ga4= instance. @jonathanolson - would that work for this case?

@oliver-phet
Copy link
Author

oliver-phet commented Nov 11, 2022

@oliver-phet, I believe I've restored the bit that would handle UA (?ga= query parameter) and added ?ga4= for GA4 IDs. I'm not sure how to test GA4, and also test to make sure (b) ?ga4 works, and (b) that it doesn't mess with our Google Tag Manager.

They both need to add to the window.dataLayer, and I've spent a half hour looking for ways to do this that might not conflict (but haven't found it). Thoughts on how we should add GA4 support?

Additionally, what steps should I make to progress on this?

I could test by inserting both a test UA and GA4 property ID (I've already created a UA test property). If you think that could work, could publish a test sim that includes the patch?

Edit: this would also allow me to verify that it doesn't interfere with our main UA/GA4 properties that are collected via GTM.

@kathy-phet
Copy link

Hey all, Just to chime in here re timing on this, I've asked @jonathanolson to proceed with the maintenance release of the ?allowlinks functionality and not keep it tied to this work. Getting the GA4 tag stuff worked out is on the list, but its a bigger project and the other maintenance release needs to get out.

JO has a couple of other short-term priorities that need focusing, so it will be a couple weeks before he can dig into this more - if needed.

@jonathanolson - If its easy to publish the current status and test sim for @oliver-phet to test, please do so. If it needs a lot of time, please proceed after the short-term priorities are completed.

@oliver-phet
Copy link
Author

Hi all, I wanted to check on the status of this.
Here is the Epic Issue for migrating to GA4. I know @jonathanolson has other priorities, but I think it would be best to get this integrated into the next maintenance release!

@jonathanolson
Copy link
Contributor

@jonathanolson
Copy link
Contributor

Current JS patch (needed updates due to linting rules):

---
Index: js/analytics/google-analytics.js
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/js/analytics/google-analytics.js b/js/analytics/google-analytics.js
--- a/js/analytics/google-analytics.js	(revision 6ec61b593351cdb8ae5c1ee25f1857919ca73419)
+++ b/js/analytics/google-analytics.js	(date 1671749818175)
@@ -67,11 +67,7 @@
     }, true );
 
     // {boolean} - Whether analytics.js successfully loaded, see https://github.com/phetsims/yotta/issues/30
-    let googleAnalyticsLoaded = false;
-
-    function onGoogleAnalyticsLoad() {
-      googleAnalyticsLoaded = true;
-    }
+    const googleAnalyticsLoaded = false;
 
     const pingParams = `${'pingver=3&' +
                        'project='}${encodeURIComponent( phet.chipper.project )}&` +
@@ -99,110 +95,72 @@
                `gaLoaded=${encodeURIComponent( googleAnalyticsLoaded )}` );
     }, false );
 
-    // Google Analytics snippet for loading the API
-    ( function( i, s, o, g, r, a, m ) {
-      i.GoogleAnalyticsObject = r;
-      i[ r ] = i[ r ] || function() {
-        // eslint-disable-next-line prefer-rest-params
-        ( i[ r ].q = i[ r ].q || [] ).push( arguments );
-      }, i[ r ].l = 1 * new Date(); // eslint-disable-line no-sequences
-      a = s.createElement( o ), m = s.getElementsByTagName( o )[ 0 ]; // eslint-disable-line no-sequences
-      a.async = 1;
-      a.src = g;
-      m.parentNode.insertBefore( a, m );
-    } )( window, document, 'script', `${document.location.protocol === 'https:' ? 'https:' : 'http:'}//www.google-analytics.com/analytics.js`, 'googleAnalytics' );
+    if ( phet.chipper.queryParameters.ga ) {
+      // Google Analytics snippet for loading the API
+      ( function( i, s, o, g, r, a, m ) {
+        i.GoogleAnalyticsObject = r;
+        i[ r ] = i[ r ] || function() {
+          // eslint-disable-next-line prefer-rest-params
+          ( i[ r ].q = i[ r ].q || [] ).push( arguments );
+        }, i[ r ].l = 1 * new Date(); // eslint-disable-line no-sequences
+        a = s.createElement( o ), m = s.getElementsByTagName( o )[ 0 ]; // eslint-disable-line no-sequences
+        a.async = 1;
+        a.src = g;
+        m.parentNode.insertBefore( a, m );
+      } )( window, document, 'script', `${document.location.protocol === 'https:' ? 'https:' : 'http:'}//www.google-analytics.com/analytics.js`, 'googleAnalytics' );
 
-    // Applies custom dimensions that are common for our main, third-party, and phet-io tracker
-    const phetPageviewOptions = {};
+      // Applies custom dimensions that are common for our main, third-party, and phet-io tracker
+      const phetPageviewOptions = {};
 
-    if ( phet.chipper.project ) {
-      phetPageviewOptions.dimension1 = phet.chipper.project; // simName custom dimension
-    }
-    if ( phet.chipper.version ) {
-      phetPageviewOptions.dimension2 = phet.chipper.version; // simVersion custom dimension
-    }
-    if ( phet.chipper.locale ) {
-      phetPageviewOptions.dimension3 = phet.chipper.locale; // simLocale custom dimension
-    }
-    if ( phet.chipper.buildTimestamp ) {
-      phetPageviewOptions.dimension4 = phet.chipper.buildTimestamp; // simBuildTimestamp custom dimension
-    }
-    phetPageviewOptions.dimension5 = loadType;
-    phetPageviewOptions.dimension6 = document.referrer;
+      if ( phet.chipper.project ) {
+        phetPageviewOptions.dimension1 = phet.chipper.project; // simName custom dimension
+      }
+      if ( phet.chipper.version ) {
+        phetPageviewOptions.dimension2 = phet.chipper.version; // simVersion custom dimension
+      }
+      if ( phet.chipper.locale ) {
+        phetPageviewOptions.dimension3 = phet.chipper.locale; // simLocale custom dimension
+      }
+      if ( phet.chipper.buildTimestamp ) {
+        phetPageviewOptions.dimension4 = phet.chipper.buildTimestamp; // simBuildTimestamp custom dimension
+      }
+      phetPageviewOptions.dimension5 = loadType;
+      phetPageviewOptions.dimension6 = document.referrer;
 
-    const offlineSimLocation = `offline/html/${phet.chipper.project}_${phet.chipper.locale}`;
-
-    // Put our function in the queue, to be invoked when the analytics.js has fully loaded.
-    // See https://github.com/phetsims/yotta/issues/30
-    window.googleAnalytics( onGoogleAnalyticsLoad );
-
-    // Main PhET tracker
-    window.googleAnalytics( 'create', {
-      trackingId: 'UA-5033201-1',
-      storage: 'none',
-      cookieDomain: 'none'
-    } );
-    if ( window.location.protocol === 'file:' ) {
-      window.googleAnalytics( 'set', 'checkProtocolTask', null );
-      window.googleAnalytics( 'set', 'checkStorageTask', null );
-      window.googleAnalytics( 'set', 'location', offlineSimLocation );
-    }
-    window.googleAnalytics( 'set', 'anonymizeIp', true );
-    window.googleAnalytics( 'send', 'pageview', phetPageviewOptions );
-
-    // PhET iO tracker (see https://github.com/phetsims/phetcommon/issues/26)
-    if ( phet.chipper.brand === 'phet-io' ) {
-      window.googleAnalytics( 'create', {
-        trackingId: 'UA-37615182-3',
-        storage: 'none',
-        cookieDomain: 'none',
-        name: 'io'
-      } );
-      if ( window.location.protocol === 'file:' ) {
-        window.googleAnalytics( 'io.set', 'checkProtocolTask', null );
-        window.googleAnalytics( 'io.set', 'checkStorageTask', null );
-        window.googleAnalytics( 'io.set', 'location', offlineSimLocation );
-      }
-      window.googleAnalytics( 'io.set', 'anonymizeIp', true );
-      window.googleAnalytics( 'io.send', 'pageview', phetPageviewOptions );
-    }
-
-    // Third-party PhET tracker (excludes non-third-party usage, see https://github.com/phetsims/yotta/issues/12)
-    if ( window.location.protocol !== 'file:' &&
-         !document.domain.match( /(.*\.colorado\.edu\.?$)|(^localhost$)|(^127\.0\.0\.1$)/ ) ) {
-      window.googleAnalytics( 'create', {
-        trackingId: 'UA-37615182-2',
-        storage: 'none',
-        cookieDomain: 'none',
-        name: 'thirdParty'
-      } );
-      window.googleAnalytics( 'thirdParty.set', 'anonymizeIp', true );
-      window.googleAnalytics( 'thirdParty.send', 'pageview', phetPageviewOptions );
-    }
-
-    // Hewlett tracker
-    window.googleAnalytics( 'create', {
-      trackingId: 'UA-5033010-35',
-      storage: 'none',
-      cookieDomain: 'phet.colorado.edu',
-      name: 'hewlett'
-    } );
-    window.googleAnalytics( 'hewlett.set', 'anonymizeIp', true );
-    window.googleAnalytics( 'hewlett.send', 'pageview' );
-
-    // External tracker
-    if ( phet.chipper.queryParameters.ga ) {
-      window.googleAnalytics( 'create', {
-        trackingId: phet.chipper.queryParameters.ga,
-        storage: 'none',
-        cookieDomain: 'none', // don't require the tracking from our site
-        name: 'external'
-      } );
-      window.googleAnalytics( 'external.set', 'anonymizeIp', true );
-      window.googleAnalytics( 'external.send', 'pageview', phet.chipper.queryParameters.gaPage || undefined );
-    }
-  }
+      // External tracker
+      if ( phet.chipper.queryParameters.ga ) {
+        window.googleAnalytics( 'create', {
+          trackingId: phet.chipper.queryParameters.ga,
+          storage: 'none',
+          cookieDomain: 'none', // don't require the tracking from our site
+          name: 'external'
+        } );
+        window.googleAnalytics( 'external.set', 'anonymizeIp', true );
+        window.googleAnalytics( 'external.send', 'pageview', phet.chipper.queryParameters.gaPage || undefined );
+      }
+    }
 
+    window.dataLayer = window.dataLayer || [];
+    window.dataLayer.push( {
+      simBrand: phet.chipper.brand,
+      simName: phet.chipper.project,
+      simVersion: phet.chipper.version,
+      simLocale: phet.chipper.locale,
+      simBuildTimestamp: phet.chipper.buildTimestamp,
+      simLoadType: loadType,
+      documentReferrer: document.referrer
+    } );
+
+    if ( phet.chipper.queryParameters.ga4 ) {
+      window.dataLayer.push( [
+        'config', phet.chipper.queryParameters.ga4
+      ] );
+    }
+
+    // Google Tag Manager
+    (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-WLNGBXD'); // eslint-disable-line space-infix-ops,space-in-parens,comma-spacing,key-spacing,one-var,semi-spacing,eqeqeq,computed-property-spacing,no-var,one-var-declaration-per-line,object-curly-spacing,space-before-blocks
+  }
+
   if ( loadType === 'phet-app' ) {
     window.addEventListener( 'load', () => {
       setTimeout( sendMessages, 0 ); // eslint-disable-line bad-sim-text

@oliver-phet
Copy link
Author

Testing results from today:
https://bayes.colorado.edu/dev/olsonjb/ga7-test.html?ga=UA-5033201-2

  • Realtime recorded an active hit on the page under the PhET Test UA account (viewing property UA-5033201-2)

https://jonathanolson.net/phet/ga7-test.html?ga=UA-5033201-2

  • Realtime recorded an active hit on the page under the PhET Test UA account (viewing property UA-5033201-2)

https://bayes.colorado.edu/dev/olsonjb/ga7-test.html?ga=UA-5033201-2&ga4=G-NT4RB93Q46

  • Realtime recorded an active hit on the page under the PhET Test UA account (viewing property UA-5033201-2)
  • Realtime did NOT record an active hit on the page under the PhET Test GA4 account (viewing property G-NT4RB93Q46)

https://bayes.colorado.edu/dev/olsonjb/ga7-test.html?ga4=G-NT4RB93Q46

  • Realtime did NOT record an active hit on the page under the PhET Test GA4 account (viewing property G-NT4RB93Q46)

@jonathanolson It seems the latest build works for recording to external UA properties, but not GA4. Do my links look correct? Other ideas?

@oliver-phet
Copy link
Author

I don't know if it's helpful, but here is the code snippet provided for the GA4 test property I created:

<!-- Google tag (gtag.js) -->
<script async src="https://www.googletagmanager.com/gtag/js?id=G-NT4RB93Q46"></script>
<script>
  window.dataLayer = window.dataLayer || [];
  function gtag(){dataLayer.push(arguments);}
  gtag('js', new Date());

  gtag('config', 'G-NT4RB93Q46');
</script>

@jonathanolson
Copy link
Contributor

Patch with a push on both of those items:

Subject: [PATCH] Updated chess data
---
Index: js/analytics/google-analytics.js
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/js/analytics/google-analytics.js b/js/analytics/google-analytics.js
--- a/js/analytics/google-analytics.js	(revision 57823ce926bcd2599010a4bf611d471162227df4)
+++ b/js/analytics/google-analytics.js	(date 1672772746202)
@@ -67,11 +67,7 @@
     }, true );
 
     // {boolean} - Whether analytics.js successfully loaded, see https://github.com/phetsims/yotta/issues/30
-    let googleAnalyticsLoaded = false;
-
-    function onGoogleAnalyticsLoad() {
-      googleAnalyticsLoaded = true;
-    }
+    const googleAnalyticsLoaded = false;
 
     const pingParams = `${'pingver=3&' +
                        'project='}${encodeURIComponent( phet.chipper.project )}&` +
@@ -99,110 +95,74 @@
                `gaLoaded=${encodeURIComponent( googleAnalyticsLoaded )}` );
     }, false );
 
-    // Google Analytics snippet for loading the API
-    ( function( i, s, o, g, r, a, m ) {
-      i.GoogleAnalyticsObject = r;
-      i[ r ] = i[ r ] || function() {
-        // eslint-disable-next-line prefer-rest-params
-        ( i[ r ].q = i[ r ].q || [] ).push( arguments );
-      }, i[ r ].l = 1 * new Date(); // eslint-disable-line no-sequences
-      a = s.createElement( o ), m = s.getElementsByTagName( o )[ 0 ]; // eslint-disable-line no-sequences
-      a.async = 1;
-      a.src = g;
-      m.parentNode.insertBefore( a, m );
-    } )( window, document, 'script', `${document.location.protocol === 'https:' ? 'https:' : 'http:'}//www.google-analytics.com/analytics.js`, 'googleAnalytics' );
+    if ( phet.chipper.queryParameters.ga ) {
+      // Google Analytics snippet for loading the API
+      ( function( i, s, o, g, r, a, m ) {
+        i.GoogleAnalyticsObject = r;
+        i[ r ] = i[ r ] || function() {
+          // eslint-disable-next-line prefer-rest-params
+          ( i[ r ].q = i[ r ].q || [] ).push( arguments );
+        }, i[ r ].l = 1 * new Date(); // eslint-disable-line no-sequences
+        a = s.createElement( o ), m = s.getElementsByTagName( o )[ 0 ]; // eslint-disable-line no-sequences
+        a.async = 1;
+        a.src = g;
+        m.parentNode.insertBefore( a, m );
+      } )( window, document, 'script', `${document.location.protocol === 'https:' ? 'https:' : 'http:'}//www.google-analytics.com/analytics.js`, 'googleAnalytics' );
 
-    // Applies custom dimensions that are common for our main, third-party, and phet-io tracker
-    const phetPageviewOptions = {};
+      // Applies custom dimensions that are common for our main, third-party, and phet-io tracker
+      const phetPageviewOptions = {};
 
-    if ( phet.chipper.project ) {
-      phetPageviewOptions.dimension1 = phet.chipper.project; // simName custom dimension
-    }
-    if ( phet.chipper.version ) {
-      phetPageviewOptions.dimension2 = phet.chipper.version; // simVersion custom dimension
-    }
-    if ( phet.chipper.locale ) {
-      phetPageviewOptions.dimension3 = phet.chipper.locale; // simLocale custom dimension
-    }
-    if ( phet.chipper.buildTimestamp ) {
-      phetPageviewOptions.dimension4 = phet.chipper.buildTimestamp; // simBuildTimestamp custom dimension
-    }
-    phetPageviewOptions.dimension5 = loadType;
-    phetPageviewOptions.dimension6 = document.referrer;
+      if ( phet.chipper.project ) {
+        phetPageviewOptions.dimension1 = phet.chipper.project; // simName custom dimension
+      }
+      if ( phet.chipper.version ) {
+        phetPageviewOptions.dimension2 = phet.chipper.version; // simVersion custom dimension
+      }
+      if ( phet.chipper.locale ) {
+        phetPageviewOptions.dimension3 = phet.chipper.locale; // simLocale custom dimension
+      }
+      if ( phet.chipper.buildTimestamp ) {
+        phetPageviewOptions.dimension4 = phet.chipper.buildTimestamp; // simBuildTimestamp custom dimension
+      }
+      phetPageviewOptions.dimension5 = loadType;
+      phetPageviewOptions.dimension6 = document.referrer;
 
-    const offlineSimLocation = `offline/html/${phet.chipper.project}_${phet.chipper.locale}`;
-
-    // Put our function in the queue, to be invoked when the analytics.js has fully loaded.
-    // See https://github.com/phetsims/yotta/issues/30
-    window.googleAnalytics( onGoogleAnalyticsLoad );
-
-    // Main PhET tracker
-    window.googleAnalytics( 'create', {
-      trackingId: 'UA-5033201-1',
-      storage: 'none',
-      cookieDomain: 'none'
-    } );
-    if ( window.location.protocol === 'file:' ) {
-      window.googleAnalytics( 'set', 'checkProtocolTask', null );
-      window.googleAnalytics( 'set', 'checkStorageTask', null );
-      window.googleAnalytics( 'set', 'location', offlineSimLocation );
-    }
-    window.googleAnalytics( 'set', 'anonymizeIp', true );
-    window.googleAnalytics( 'send', 'pageview', phetPageviewOptions );
-
-    // PhET iO tracker (see https://github.com/phetsims/phetcommon/issues/26)
-    if ( phet.chipper.brand === 'phet-io' ) {
-      window.googleAnalytics( 'create', {
-        trackingId: 'UA-37615182-3',
-        storage: 'none',
-        cookieDomain: 'none',
-        name: 'io'
-      } );
-      if ( window.location.protocol === 'file:' ) {
-        window.googleAnalytics( 'io.set', 'checkProtocolTask', null );
-        window.googleAnalytics( 'io.set', 'checkStorageTask', null );
-        window.googleAnalytics( 'io.set', 'location', offlineSimLocation );
-      }
-      window.googleAnalytics( 'io.set', 'anonymizeIp', true );
-      window.googleAnalytics( 'io.send', 'pageview', phetPageviewOptions );
-    }
-
-    // Third-party PhET tracker (excludes non-third-party usage, see https://github.com/phetsims/yotta/issues/12)
-    if ( window.location.protocol !== 'file:' &&
-         !document.domain.match( /(.*\.colorado\.edu\.?$)|(^localhost$)|(^127\.0\.0\.1$)/ ) ) {
-      window.googleAnalytics( 'create', {
-        trackingId: 'UA-37615182-2',
-        storage: 'none',
-        cookieDomain: 'none',
-        name: 'thirdParty'
-      } );
-      window.googleAnalytics( 'thirdParty.set', 'anonymizeIp', true );
-      window.googleAnalytics( 'thirdParty.send', 'pageview', phetPageviewOptions );
-    }
-
-    // Hewlett tracker
-    window.googleAnalytics( 'create', {
-      trackingId: 'UA-5033010-35',
-      storage: 'none',
-      cookieDomain: 'phet.colorado.edu',
-      name: 'hewlett'
-    } );
-    window.googleAnalytics( 'hewlett.set', 'anonymizeIp', true );
-    window.googleAnalytics( 'hewlett.send', 'pageview' );
-
-    // External tracker
-    if ( phet.chipper.queryParameters.ga ) {
-      window.googleAnalytics( 'create', {
-        trackingId: phet.chipper.queryParameters.ga,
-        storage: 'none',
-        cookieDomain: 'none', // don't require the tracking from our site
-        name: 'external'
-      } );
-      window.googleAnalytics( 'external.set', 'anonymizeIp', true );
-      window.googleAnalytics( 'external.send', 'pageview', phet.chipper.queryParameters.gaPage || undefined );
-    }
-  }
+      // External tracker
+      if ( phet.chipper.queryParameters.ga ) {
+        window.googleAnalytics( 'create', {
+          trackingId: phet.chipper.queryParameters.ga,
+          storage: 'none',
+          cookieDomain: 'none', // don't require the tracking from our site
+          name: 'external'
+        } );
+        window.googleAnalytics( 'external.set', 'anonymizeIp', true );
+        window.googleAnalytics( 'external.send', 'pageview', phet.chipper.queryParameters.gaPage || undefined );
+      }
+    }
 
+    window.dataLayer = window.dataLayer || [];
+    window.dataLayer.push( {
+      simBrand: phet.chipper.brand,
+      simName: phet.chipper.project,
+      simVersion: phet.chipper.version,
+      simLocale: phet.chipper.locale,
+      simBuildTimestamp: phet.chipper.buildTimestamp,
+      simLoadType: loadType,
+      documentReferrer: document.referrer
+    } );
+
+    if ( phet.chipper.queryParameters.ga4 ) {
+      ( () => {
+        window.dataLayer.push( 'js', new Date() );
+
+        window.dataLayer.push( 'config', phet.chipper.queryParameters.ga4 );
+      } )();
+    }
+
+    // Google Tag Manager
+    (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-WLNGBXD'); // eslint-disable-line space-infix-ops,space-in-parens,comma-spacing,key-spacing,one-var,semi-spacing,eqeqeq,computed-property-spacing,no-var,one-var-declaration-per-line,object-curly-spacing,space-before-blocks
+  }
+
   if ( loadType === 'phet-app' ) {
     window.addEventListener( 'load', () => {
       setTimeout( sendMessages, 0 ); // eslint-disable-line bad-sim-text

@jonathanolson
Copy link
Contributor

Can we try a test with https://bayes.colorado.edu/dev/olsonjb/ga8-test.html?

@oliver-phet
Copy link
Author

Tested
@jonathanolson I looked at this latest patch with @mattpen :
It looks like there isn't a call to gtag.js for GA4 (just some pieces being pushed to the data layer)
I think it would be best to set up a call with the 3 of us so we can fix this and test!

@oliver-phet
Copy link
Author

We did some testing today but ran into an issue adding an external property via query parameter. @jonathanolson was able to make it work but had to strip out our GTM tag. It seems having both present via the recommended approach to configure a second analytics property isn't working in concert with google tag manager.

@jonathanolson and @mattpen are going to continue investigating possibilities for supporting both gtm (internal PhET trackers) and gtag scripts (external query parameter tracker).

@jonathanolson
Copy link
Contributor

Current (possibly working) patch:

Subject: [PATCH] Updated chess data
---
Index: chipper/js/initialize-globals.js
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/chipper/js/initialize-globals.js b/chipper/js/initialize-globals.js
--- a/chipper/js/initialize-globals.js	(revision da4b632c9232677f98f2c81b1388d37f89cc63ca)
+++ b/chipper/js/initialize-globals.js	(date 1672857662365)
@@ -206,8 +206,10 @@
     },
 
     /**
-     * Used for providing a external Google Analytics property for tracking, see
-     * https://github.com/phetsims/phetcommon/issues/46 for more information.
+     * Used for providing an external Google Analytics (using the soon-to-be-sunset UA/Universial Analytics) property
+     * for tracking, see https://github.com/phetsims/phetcommon/issues/46 for more information.
+     *
+     * Generally, this string will start with 'UA-' (otherwise use ?ga4)
      *
      * This is useful for various users/clients that want to embed simulations, or direct users to simulations. For
      * example, if a sim is included in an epub, the sim HTML won't have to be modified to include page tracking.
@@ -216,6 +218,20 @@
       type: 'string',
       defaultValue: null
     },
+
+    /**
+     * Used for providing an external Google Analytics 4 (gtag.js) property for tracking, see
+     * https://github.com/phetsims/phetcommon/issues/46 for more information.
+     *
+     * Generally, this string will start with 'G-' for GA4 trackers
+     *
+     * This is useful for various users/clients that want to embed simulations, or direct users to simulations. For
+     * example, if a sim is included in an epub, the sim HTML won't have to be modified to include page tracking.
+     */
+    ga4: {
+      type: 'string',
+      defaultValue: null
+    },
 
     /**
      * Launches the game-up-camera code which delivers images to requests in BrainPOP/Game Up/SnapThought
Index: phetcommon/js/analytics/google-analytics.js
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/phetcommon/js/analytics/google-analytics.js b/phetcommon/js/analytics/google-analytics.js
--- a/phetcommon/js/analytics/google-analytics.js	(revision 57823ce926bcd2599010a4bf611d471162227df4)
+++ b/phetcommon/js/analytics/google-analytics.js	(date 1672857698979)
@@ -66,13 +66,6 @@
       }
     }, true );
 
-    // {boolean} - Whether analytics.js successfully loaded, see https://github.com/phetsims/yotta/issues/30
-    let googleAnalyticsLoaded = false;
-
-    function onGoogleAnalyticsLoad() {
-      googleAnalyticsLoaded = true;
-    }
-
     const pingParams = `${'pingver=3&' +
                        'project='}${encodeURIComponent( phet.chipper.project )}&` +
                        `brand=${encodeURIComponent( phet.chipper.brand )}&` +
@@ -96,102 +89,42 @@
     window.addEventListener( 'load', event => {
       pingURL( `https://phet.colorado.edu/yotta/sanity.gif?${pingParams}&` +
                `gaError=${encodeURIComponent( googleAnalyticsErrored )}&` +
-               `gaLoaded=${encodeURIComponent( googleAnalyticsLoaded )}` );
+               `gaLoaded=${encodeURIComponent( false )}` );
     }, false );
 
-    // Google Analytics snippet for loading the API
-    ( function( i, s, o, g, r, a, m ) {
-      i.GoogleAnalyticsObject = r;
-      i[ r ] = i[ r ] || function() {
-        // eslint-disable-next-line prefer-rest-params
-        ( i[ r ].q = i[ r ].q || [] ).push( arguments );
-      }, i[ r ].l = 1 * new Date(); // eslint-disable-line no-sequences
-      a = s.createElement( o ), m = s.getElementsByTagName( o )[ 0 ]; // eslint-disable-line no-sequences
-      a.async = 1;
-      a.src = g;
-      m.parentNode.insertBefore( a, m );
-    } )( window, document, 'script', `${document.location.protocol === 'https:' ? 'https:' : 'http:'}//www.google-analytics.com/analytics.js`, 'googleAnalytics' );
+    // External UA (Google Analytics) tracker
+    if ( phet.chipper.queryParameters.ga ) {
+      // Google Analytics snippet for loading the API
+      ( function( i, s, o, g, r, a, m ) {
+        i.GoogleAnalyticsObject = r;
+        i[ r ] = i[ r ] || function() {
+          // eslint-disable-next-line prefer-rest-params
+          ( i[ r ].q = i[ r ].q || [] ).push( arguments );
+        }, i[ r ].l = 1 * new Date(); // eslint-disable-line no-sequences
+        a = s.createElement( o ), m = s.getElementsByTagName( o )[ 0 ]; // eslint-disable-line no-sequences
+        a.async = 1;
+        a.src = g;
+        m.parentNode.insertBefore( a, m );
+      } )( window, document, 'script', `${document.location.protocol === 'https:' ? 'https:' : 'http:'}//www.google-analytics.com/analytics.js`, 'googleAnalytics' );
 
-    // Applies custom dimensions that are common for our main, third-party, and phet-io tracker
-    const phetPageviewOptions = {};
+      // Applies custom dimensions that are common for our main, third-party, and phet-io tracker
+      const phetPageviewOptions = {};
 
-    if ( phet.chipper.project ) {
-      phetPageviewOptions.dimension1 = phet.chipper.project; // simName custom dimension
-    }
-    if ( phet.chipper.version ) {
-      phetPageviewOptions.dimension2 = phet.chipper.version; // simVersion custom dimension
-    }
-    if ( phet.chipper.locale ) {
-      phetPageviewOptions.dimension3 = phet.chipper.locale; // simLocale custom dimension
-    }
-    if ( phet.chipper.buildTimestamp ) {
-      phetPageviewOptions.dimension4 = phet.chipper.buildTimestamp; // simBuildTimestamp custom dimension
-    }
-    phetPageviewOptions.dimension5 = loadType;
-    phetPageviewOptions.dimension6 = document.referrer;
+      if ( phet.chipper.project ) {
+        phetPageviewOptions.dimension1 = phet.chipper.project; // simName custom dimension
+      }
+      if ( phet.chipper.version ) {
+        phetPageviewOptions.dimension2 = phet.chipper.version; // simVersion custom dimension
+      }
+      if ( phet.chipper.locale ) {
+        phetPageviewOptions.dimension3 = phet.chipper.locale; // simLocale custom dimension
+      }
+      if ( phet.chipper.buildTimestamp ) {
+        phetPageviewOptions.dimension4 = phet.chipper.buildTimestamp; // simBuildTimestamp custom dimension
+      }
+      phetPageviewOptions.dimension5 = loadType;
+      phetPageviewOptions.dimension6 = document.referrer;
 
-    const offlineSimLocation = `offline/html/${phet.chipper.project}_${phet.chipper.locale}`;
-
-    // Put our function in the queue, to be invoked when the analytics.js has fully loaded.
-    // See https://github.com/phetsims/yotta/issues/30
-    window.googleAnalytics( onGoogleAnalyticsLoad );
-
-    // Main PhET tracker
-    window.googleAnalytics( 'create', {
-      trackingId: 'UA-5033201-1',
-      storage: 'none',
-      cookieDomain: 'none'
-    } );
-    if ( window.location.protocol === 'file:' ) {
-      window.googleAnalytics( 'set', 'checkProtocolTask', null );
-      window.googleAnalytics( 'set', 'checkStorageTask', null );
-      window.googleAnalytics( 'set', 'location', offlineSimLocation );
-    }
-    window.googleAnalytics( 'set', 'anonymizeIp', true );
-    window.googleAnalytics( 'send', 'pageview', phetPageviewOptions );
-
-    // PhET iO tracker (see https://github.com/phetsims/phetcommon/issues/26)
-    if ( phet.chipper.brand === 'phet-io' ) {
-      window.googleAnalytics( 'create', {
-        trackingId: 'UA-37615182-3',
-        storage: 'none',
-        cookieDomain: 'none',
-        name: 'io'
-      } );
-      if ( window.location.protocol === 'file:' ) {
-        window.googleAnalytics( 'io.set', 'checkProtocolTask', null );
-        window.googleAnalytics( 'io.set', 'checkStorageTask', null );
-        window.googleAnalytics( 'io.set', 'location', offlineSimLocation );
-      }
-      window.googleAnalytics( 'io.set', 'anonymizeIp', true );
-      window.googleAnalytics( 'io.send', 'pageview', phetPageviewOptions );
-    }
-
-    // Third-party PhET tracker (excludes non-third-party usage, see https://github.com/phetsims/yotta/issues/12)
-    if ( window.location.protocol !== 'file:' &&
-         !document.domain.match( /(.*\.colorado\.edu\.?$)|(^localhost$)|(^127\.0\.0\.1$)/ ) ) {
-      window.googleAnalytics( 'create', {
-        trackingId: 'UA-37615182-2',
-        storage: 'none',
-        cookieDomain: 'none',
-        name: 'thirdParty'
-      } );
-      window.googleAnalytics( 'thirdParty.set', 'anonymizeIp', true );
-      window.googleAnalytics( 'thirdParty.send', 'pageview', phetPageviewOptions );
-    }
-
-    // Hewlett tracker
-    window.googleAnalytics( 'create', {
-      trackingId: 'UA-5033010-35',
-      storage: 'none',
-      cookieDomain: 'phet.colorado.edu',
-      name: 'hewlett'
-    } );
-    window.googleAnalytics( 'hewlett.set', 'anonymizeIp', true );
-    window.googleAnalytics( 'hewlett.send', 'pageview' );
-
-    // External tracker
-    if ( phet.chipper.queryParameters.ga ) {
       window.googleAnalytics( 'create', {
         trackingId: phet.chipper.queryParameters.ga,
         storage: 'none',
@@ -201,6 +134,44 @@
       window.googleAnalytics( 'external.set', 'anonymizeIp', true );
       window.googleAnalytics( 'external.send', 'pageview', phet.chipper.queryParameters.gaPage || undefined );
     }
+
+    // External GA4 tracker
+    if ( phet.chipper.queryParameters.ga4 ) {
+      // Use a custom data layer to both (a) get gtag.js and gtm to work at the same time, and (b) don't provide the
+      // extra data to third parties by default
+      window.ga4DataLayer = window.ga4DataLayer || [];
+
+      // NOTE: Using the GA-provided function here, to be extra cautious.
+      function gtag() { ga4DataLayer.push( arguments ); } // eslint-disable-line no-inner-declarations,no-undef,prefer-rest-params
+
+      gtag( 'js', new Date() );
+      gtag( 'config', phet.chipper.queryParameters.ga4 );
+
+      // Dynamically load the script
+      const firstScript = document.getElementsByTagName( 'script' )[ 0 ];
+      const script = document.createElement( 'script' );
+      script.async = true;
+
+      // `l` query parameter allows a different data layer name
+      script.src = `https://www.googletagmanager.com/gtag/js?id=${phet.chipper.queryParameters.ga4}&l=ga4DataLayer`;
+      firstScript.parentNode.insertBefore( script, firstScript );
+    }
+
+    // For some reason, having dataLayer declaration here might have fixed the ability to use gtag.js and gtm.js at the
+    // same time. Don't move without testing.
+    window.dataLayer = window.dataLayer || [];
+    window.dataLayer.push( {
+      simBrand: phet.chipper.brand,
+      simName: phet.chipper.project,
+      simVersion: phet.chipper.version,
+      simLocale: phet.chipper.locale,
+      simBuildTimestamp: phet.chipper.buildTimestamp,
+      simLoadType: loadType,
+      documentReferrer: document.referrer
+    } );
+
+    // Google Tag Manager (gtm.js) - Identical to recommended snippet with eslint disables to keep it this way.
+    (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-WLNGBXD'); // eslint-disable-line space-infix-ops,space-in-parens,comma-spacing,key-spacing,one-var,semi-spacing,eqeqeq,computed-property-spacing,no-var,one-var-declaration-per-line,object-curly-spacing,space-before-blocks
   }
 
   if ( loadType === 'phet-app' ) {

@oliver-phet
Copy link
Author

oliver-phet commented Jan 4, 2023

ON to test:
https://jonathanolson.net/phet/ga17-test.html

  • Hits and custom dimensions are properly recorded via GTM for our internal UA
  • Hits and custom dimensions are properly recorded via GTM for our internal GA4

https://jonathanolson.net/phet/ga17-test.html?ga=UA-5033201-2

  • Hits properly recorded to external propertie(s)
  • Hits and custom dimensions are properly recorded via GTM for our internal UA
  • Hits and custom dimensions are properly recorded via GTM for our internal GA4

https://jonathanolson.net/phet/ga17-test.html?ga4=G-NT4RB93Q46

  • Hits properly recorded to external propertie(s)
  • Hits and custom dimensions are properly recorded via GTM for our internal UA
  • Hits and custom dimensions are properly recorded via GTM for our internal GA4

https://jonathanolson.net/phet/ga17-test.html?ga=UA-5033201-2&ga4=G-NT4RB93Q46

  • Hits properly recorded to external propertie(s)
  • Hits and custom dimensions are properly recorded via GTM for our internal UA
  • Hits and custom dimensions are properly recorded via GTM for our internal GA4

https://bayes.colorado.edu/dev/olsonjb/ga17-test.html

  • Hits and custom dimensions are properly recorded via GTM for our internal UA
  • Hits and custom dimensions are properly recorded via GTM for our internal GA4

https://bayes.colorado.edu/dev/olsonjb/ga17-test.html?ga=UA-5033201-2

  • Hits properly recorded to external propertie(s)
  • Hits and custom dimensions are properly recorded via GTM for our internal UA
  • Hits and custom dimensions are properly recorded via GTM for our internal GA4

https://bayes.colorado.edu/dev/olsonjb/ga17-test.html?ga4=G-NT4RB93Q46

  • Hits properly recorded to external propertie(s)
  • Hits and custom dimensions are properly recorded via GTM for our internal UA
  • Hits and custom dimensions are properly recorded via GTM for our internal GA4

https://bayes.colorado.edu/dev/olsonjb/ga17-test.html?ga=UA-5033201-2&ga4=G-NT4RB93Q46

  • Hits properly recorded to external propertie(s)
  • Hits and custom dimensions are properly recorded via GTM for our internal UA
  • Hits and custom dimensions are properly recorded via GTM for our internal GA4

@oliver-phet
Copy link
Author

Updating after some testing today:
Things are looking good in the latest patch! The only test I haven't been able to complete is verifying our UA data is properly feeding over via GTM (because our UA property has thousands of hits per day, it's impossible for me to parse out a single hit to a specific page). But I should be able to search for page views of "ga17-test" tomorrow afternoon once the report data is available.

@oliver-phet
Copy link
Author

I just checked both our internal UA and GA4 to verify hits and custom dimensions looked good from yesterday's tests:
UA:
image

GA4:
image

@jonathanolson @mattpen I think this is looking good unless there are any other test you think we should run for this? Hits were recorded to the test external properties and it didn't seem to interfere with anything in our internal UA/GA4 collection that goes through GTM.

@mattpen
Copy link
Contributor

mattpen commented Jan 6, 2023

I can't think of anything else. This is looking good! Do we need to coordinate deployments of gtm, or can I go ahead and publish the website changes?

@samreid
Copy link
Member

samreid commented Jan 10, 2023

@jonathanolson and @oliver-phet heads up about @mattpen question above.

jonathanolson added a commit to phetsims/number-play that referenced this issue Jan 18, 2023
jonathanolson added a commit to phetsims/ohms-law that referenced this issue Jan 18, 2023
jonathanolson added a commit to phetsims/pendulum-lab that referenced this issue Jan 18, 2023
jonathanolson added a commit to phetsims/ph-scale that referenced this issue Jan 18, 2023
jonathanolson added a commit to phetsims/ph-scale-basics that referenced this issue Jan 18, 2023
jonathanolson added a commit to phetsims/plinko-probability that referenced this issue Jan 18, 2023
jonathanolson added a commit to phetsims/projectile-motion that referenced this issue Jan 18, 2023
jonathanolson added a commit to phetsims/proportion-playground that referenced this issue Jan 18, 2023
jonathanolson added a commit to phetsims/ratio-and-proportion that referenced this issue Jan 18, 2023
jonathanolson added a commit to phetsims/reactants-products-and-leftovers that referenced this issue Jan 18, 2023
jonathanolson added a commit to phetsims/resistance-in-a-wire that referenced this issue Jan 18, 2023
jonathanolson added a commit to phetsims/resistance-in-a-wire that referenced this issue Jan 18, 2023
jonathanolson added a commit to phetsims/rutherford-scattering that referenced this issue Jan 18, 2023
jonathanolson added a commit to phetsims/states-of-matter that referenced this issue Jan 18, 2023
jonathanolson added a commit to phetsims/states-of-matter-basics that referenced this issue Jan 18, 2023
jonathanolson added a commit to phetsims/trig-tour that referenced this issue Jan 18, 2023
jonathanolson added a commit to phetsims/under-pressure that referenced this issue Jan 18, 2023
jonathanolson added a commit to phetsims/unit-rates that referenced this issue Jan 18, 2023
jonathanolson added a commit to phetsims/vector-addition that referenced this issue Jan 18, 2023
jonathanolson added a commit to phetsims/vector-addition-equations that referenced this issue Jan 18, 2023
jonathanolson added a commit to phetsims/wave-interference that referenced this issue Jan 18, 2023
jonathanolson added a commit to phetsims/wave-on-a-string that referenced this issue Jan 18, 2023
jonathanolson added a commit to phetsims/waves-intro that referenced this issue Jan 18, 2023
@jonathanolson
Copy link
Contributor

Deployments going out (8 hours in, and it's up to gases-intro).

acid-base-solutions 1.2 ()

area-builder 1.1 ()

area-model-algebra 1.2 ()

area-model-decimals 1.2 ()

area-model-introduction 1.2 ()

area-model-multiplication 1.2 ()

arithmetic 1.0 ()

atomic-interactions 1.2 ()

balancing-act 1.1 ()

balancing-act 1.2 ()

balancing-chemical-equations 1.2 ()

balloons-and-static-electricity 1.3-phetio ()

balloons-and-static-electricity 1.5 ()

beers-law-lab 1.4 ()

beers-law-lab 1.6-phetio ()

bending-light 1.1 ()

blackbody-spectrum 1.0 ()

build-a-fraction 1.0 ()

build-a-molecule 1.0 ()

build-a-nucleus 1.0 ()

build-an-atom 1.5-phetio ()

build-an-atom 1.6 ()

capacitor-lab-basics 1.6 ()

center-and-variability 1.0 ()

charges-and-fields 1.0 ()

charges-and-fields 1.0-phetio ()

circuit-construction-kit-ac 1.0 ()

circuit-construction-kit-ac-virtual-lab 1.0 ()

circuit-construction-kit-black-box-study 1.1-phetio ()

circuit-construction-kit-dc 1.2 ()

circuit-construction-kit-dc-virtual-lab 1.2 ()

collision-lab 1.1 ()

color-vision 1.1 ()

color-vision 1.2-phetio ()

concentration 1.3 ()

concentration 1.5-phetio ()

coulombs-law 1.0 ()

curve-fitting 1.0 ()

density 1.0 ()

diffusion 1.0 ()

energy-forms-and-changes 1.4 ()

energy-skate-park-basics 1.1 ()

energy-skate-park-basics 1.3-phetio ()

equality-explorer 1.1 ()

equality-explorer-basics 1.0 ()

equality-explorer-two-variables 1.0 ()

expression-exchange 1.1 ()

faradays-law 1.3-phetio ()

faradays-law 1.4 ()

forces-and-motion-basics 2.1-phetio ()

forces-and-motion-basics 2.3 ()

fourier-making-waves 1.0 ()

fraction-matcher 1.2 ()

fractions-equality 1.1 ()

fractions-intro 1.0 ()

fractions-mixed-numbers 1.0 ()

friction 1.5 ()

function-builder 1.2 ()

function-builder-basics 1.2 ()

gas-properties 1.0 ()

gases-intro 1.0 ()

gene-expression-essentials 1.0 ()

geometric-optics 1.1 ()

geometric-optics-basics 1.2 ()

graphing-lines 1.3 ()

graphing-quadratics 1.1 ()

graphing-quadratics 1.2 ()

graphing-slope-intercept 1.1 ()

gravity-and-orbits 1.4 ()

gravity-and-orbits 1.5 ()

gravity-and-orbits 1.6 ()

gravity-force-lab 2.2 ()

gravity-force-lab-basics 1.1 ()

greenhouse-effect 1.0 ()

hookes-law 1.0 ()

isotopes-and-atomic-mass 1.1 ()

john-travoltage 1.4-phetio ()

john-travoltage 1.6 ()

least-squares-regression 1.1 ()

make-a-ten 1.0 ()

masses-and-springs 1.0 ()

masses-and-springs-basics 1.0 ()

mean-share-and-balance 1.0 ()

molarity 1.4 ()

molarity 1.5 ()

molecule-polarity 1.2 ()

molecule-shapes 1.2 ()

molecule-shapes-basics 1.2 ()

molecules-and-light 1.3-phetio ()

molecules-and-light 1.5 ()

my-solar-system 1.0 ()

natural-selection 1.2 ()

natural-selection 1.3 ()

natural-selection 1.4 ()

neuron 1.1 ()

normal-modes 1.0 ()

number-line-distance 1.0 ()

number-line-integers 1.1 ()

number-line-operations 1.0 ()

number-play 1.0 ()

ohms-law 1.4 ()

pendulum-lab 1.0 ()

ph-scale 1.5 ()

ph-scale-basics 1.5 ()

plinko-probability 1.1 ()

projectile-motion 1.0 ()

proportion-playground 1.0 ()

ratio-and-proportion 1.2 ()

reactants-products-and-leftovers 1.2 ()

resistance-in-a-wire 1.3-phetio ()

resistance-in-a-wire 1.6 ()

rutherford-scattering 1.1 ()

states-of-matter 1.2 ()

states-of-matter-basics 1.2 ()

trig-tour 1.0 ()

under-pressure 1.1 ()

unit-rates 1.0 ()

vector-addition 1.0 ()

vector-addition-equations 1.0 ()

wave-interference 2.0 ()

wave-on-a-string 1.1 ()

waves-intro 1.1 ()

@jonathanolson
Copy link
Contributor

Code used to test things below:

const ReleaseBranch = require( './js/common/ReleaseBranch' );
const withServer = require( './js/common/withServer' );
const puppeteerEvaluate = require( './js/common/puppeteerEvaluate' );
const winston = require( 'winston' );

winston.default.transports.console.level = 'error';

( async () => {
  const releaseBranches = await ReleaseBranch.getMaintenanceBranches();
  const filter = async() => true;

  try {
    await withServer( async port => {
      for ( const releaseBranch of releaseBranches ) {
        if ( !filter || await filter( releaseBranch ) ) {
          console.log( releaseBranch.toString() );

          const repo = releaseBranch.repo;

          for ( const brand of releaseBranch.brands ) {
            if ( brand !== 'phet' && brand !== 'phet-io' ) {
              continue;
            }
            const path = brand === 'phet-io'
                         ? `${await releaseBranch.getLocalPhetIOBuiltHTMLPath()}?${await releaseBranch.getPhetioStandaloneQueryParameter()}`
                         : `${await releaseBranch.getLocalPhetBuiltHTMLPath()}?`;

            const urlFragment = `http://localhost:${port}/${repo}-${releaseBranch.branch}/${repo}/${path}`;

            const scan = async ( queryParams, expectGTM, expectGA4, expectGA ) => {

              const url = `${urlFragment}${queryParams}`;

              try {
                const results = await puppeteerEvaluate( url, () => [ window.dataLayer, window.ga4DataLayer, window.googleAnalytics ], {
                  waitAfterLoad: 5000
                } );

                const dataLayerActivated = dataLayer => {
                  return !!( dataLayer && dataLayer.find( element => element.event === 'gtm.load' )?.[ 'gtm.uniqueEventId' ] );
                };

                const hasGTM = dataLayerActivated( results[ 0 ] );
                const hasGA4 = dataLayerActivated( results[ 1 ] );
                const hasGA = !!results[ 2 ];

                // console.log( url );
                // console.log( hasGTM, expectGTM, results[ 0 ] );
                // console.log( hasGA4, expectGA4, results[ 1 ] );
                // console.log( hasGA, expectGA, results[ 2 ] );

                if ( hasGTM !== expectGTM ) {
                  console.log( `GTM failure for ${url} with dataLayer: ${JSON.stringify( results[ 0 ], null, 2 )}` );
                }
                if ( hasGA4 !== expectGA4 ) {
                  console.log( `GA4 failure for ${url} with dataLayer: ${JSON.stringify( results[ 1 ], null, 2 )}` );
                }
                if ( hasGA !== expectGA ) {
                  console.log( `GA failure for ${url}` );
                }
              }
              catch( e ) {
                console.log( `Failure for ${url}: ${e}` );
                return {};
              }
            };

            await scan( '', true, false, false );
            await scan( '&ga=UA-5033201-2&ga4=G-NT4RB93Q46', true, true, true );
          }
        }
      }
    }, {
      path: ReleaseBranch.getMaintenanceDirectory()
    } );
  }
  catch( e ) {
    console.log( `[ERROR] withServer failure?: ${e}` );
  }
} )();

@jonathanolson
Copy link
Contributor

Deployment completed, closing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants