Do Not Track support #4046

davidfischer · 2018-05-01T04:04:31Z

What DNT means for us:

No behavioral ad targeting. We weren't doing that anyway.
When DNT is enabled, do not load Google analytics or send anything to analytics
No long lasting unique cookies
Logs must be deleted after 10 days (we are going to do that anyway, see GDPR Meta Issue #3954 (comment))

New endpoints

/.well-known/dnt/ - this shows the DNT status of a particular user and links to our privacy policy.
/.well-known/dnt-policy.txt - this is the EFF's DNT policy verbatim (as they recommend) including trailing whitespace and all

Cookie changes

(Edit: a previous version of this PR had the CSRF cookie as a session cookie) The CSRF cookie is changed to last 30 days instead of a 1 year. The EFF policy says only a little about CSRF protection and it doesn't seem to match what Django does at all. I think this is a simple change that won't cause problems and will proactively reduce discussion the CSRF cookie being used as a unique identifier.
The longest lived GA cookie is set to 30 days instead of 2 years. This isn't strictly required by DNT (GA will be disabled if DNT is enabled). The biggest drawback to this is that users who visit once every 31 days will be counted as "new" instead of "returning". I don't think we care about that.

This doesn't delete any existing cookies so users who are cookied with GA already will have a long lasting cookie.

Advantages to us

This formalizes our "we don't track you" messaging.
A lot of these changes (or similar enough changes) are required by the GDPR (see GDPR Meta Issue #3954)
This allows us to apply for Acceptable Ads Without Tracking (which we already qualify for since all ads are served by us) even if we expand to additional sites.

More

For more details on Do Not Track, see:

NOTE: This should not be merged until after #3978

- Handle navigator.doNotTrack == 'unspecified' - Make GA actually store persistent cookies

safwanrahman · 2018-05-05T20:39:14Z

readthedocs/settings/base.py

@@ -59,6 +59,8 @@ class CommunityBaseSettings(Settings):
    SESSION_COOKIE_DOMAIN = 'readthedocs.org'
    SESSION_COOKIE_HTTPONLY = True
    CSRF_COOKIE_HTTPONLY = True
+    # See: docs/advertising-details.rst
+    CSRF_COOKIE_AGE = None  # session cookie (expires on browser quit)


This will break user experience

If you have closed the browser and restore the window, the page will be loaded from cache. but the CSRF cookie will not be there. So it may make submittion CSRF Error. maybe you can try using django session CSRF?

This is potentially true and is warned about in the Django docs. I don't think it will be a big issue for us since it only affects form submissions of which the only ones on a non-authed page are the login/signup forms. However, we could make the CSRF cookie age match the logged in cookie age (~2 weeks) to mitigate it. Thoughts?

I set it to 30 days so it matches the GA cookie. I think that's a reasonable balance so it's pretty obvious we aren't using it to track users.

safwanrahman · 2018-05-05T20:51:53Z

media/javascript/readthedocs-analytics.js

@@ -2,33 +2,40 @@
 // https://docs.readthedocs.io/en/latest/advertising-details.html#analytics


-// RTD Analytics Code
+// Skip analytics for users with Do Not Track enabled


you can use something like this script to check if do not track is enabled

That entire script is basically to handle an IE10 bug. I'm not sure it's worth the effort.

I think it should be in a script or a function that we can call from any scipt. Regarding IE, we may still support IE10.

IE10 is not supported by Microsoft with a couple exceptions and it is a tiny fraction of our users (sub-0.1%). I don't think it is unreasonable to not support a privacy feature for users who are using a browser unsupported by the vendor. In addition, the "support" that the linked script offers is mostly just to mark IE10 as "unspecified" for DNT which for our purpose would be off.

I lean toward simplicity here.

Understand. then we can keep window.doNotTrack === '1' || navigator.doNotTrack === '1') in a function and call it from everywhere!

Actually, after testing this I'm reconsidering. It looks like IE11 on Windows 7 and Windows 8 set the DNT default to on.

Considering that to use this script would mean marking IE11 and IE10 as having DNT as "unspecified" I'm leaning toward maybe just checking navigator.doNotTrack === '1' and that's it. This would mean that no versions of IE can opt-out of tracking. It would be supported in MS Edge, however.

we can keep window.doNotTrack === '1' || navigator.doNotTrack === '1') in a function and call it from everywhere!

Not very easily actually. readthedocs-analytics.js is loaded on the docs pages and should not have any outside dependencies apart from READTHEDOCS_DATA.

I updated the PR to just check navigator.doNotTrack === '1'.

ericholscher · 2018-05-01T15:35:15Z

docs/advertising-details.rst

+
+* Users can opt-out of analytics by using the Do Not Track feature of their browser.
+* Read the Docs instructs Google to anonymize IPs sent to them before they are stored.
+* The cookies set by GA expire more rapidly (30 days) than the default.


"We configured the cookies set by GA to only last 30 days, instead of the default of 2 years" reads better.

ericholscher · 2018-05-22T17:16:54Z

docs/advertising-details.rst

@@ -110,13 +110,50 @@ However, we always give advance notice in our issue tracker
 and via email about showing ads where none were shown before.


+.. _do-not-track:
+
+Do Not Track Policy


Not sure this is the right part of the docs for this. It feels like it should probably be it's own thing, since it applies to RTD itself, and not just ads.

I realize there are ad-specific things, so maybe a DNT page in the docs, and then also a section here?

Perhaps having it in the Privacy Policy is enough as an additional section. I guess it depends how heavily we want to promote the fact that we support it.

I think a section in the privacy policy is good and the advertising details will link there.

- For the guide to Google Analytics for doc authors

- It doesn't apply to .com

davidfischer · 2018-05-22T18:41:09Z

A small update to the plan:

DNT will also apply to the .com site. There aren't ads there so really all this means is the logging duration must be 10 days and DNT opts-out of GA.
I added a setting to settings.py to enable DNT. It is off by default so that other people using our code base don't accidentally declare DNT support.

ericholscher · 2018-05-22T18:53:40Z

media/javascript/readthedocs-analytics.js

@@ -2,33 +2,40 @@
 // https://docs.readthedocs.io/en/latest/advertising-details.html#analytics


-// RTD Analytics Code
+// Skip analytics for users with Do Not Track enabled
+if (navigator.doNotTrack === '1') {


Should the JS respect the setting as well? Probably needs to be added to the context manager.

I don't understand this comment. Can you elaborate?

We shouldn't add this JS opt out unless the DO_NOT_TRACK_ENABLED setting is True

Ideally, yes we want to only respect DNT if people running RTD have the setting enabled. However, the use case we are supporting by changing that is to help people taking the readthedocs.org code base and don't want to support DNT.

To do this, we would need to pass the DNT setting through the READTHEDOCS_DATA object. We can do that, but I don't think it's worth it. Do you think it is?

Yea, that makes sense for user docs. I guess I was thinking for the base.html more than the analytics.js.

I think that's the only tidbit I'd add before it's deployable -- just so that all our users dashboards don't get DNT'd automatically.

I'm going to add this to base.html, but user dashboards are going to get DNT'd. The only thing this will change is that people who take the readthedocs.org codebase and use it in their infrastructure won't get DNT'd automatically on their installation except on docs pages where they still will.

davidfischer added 2 commits April 30, 2018 20:24

Initial do not track support

d0cc92e

Handle a few issues with DNT

b038d9d

- Handle navigator.doNotTrack == 'unspecified' - Make GA actually store persistent cookies

davidfischer requested review from ericholscher and agjohnson May 1, 2018 04:04

Fix linting issue

c20e6c2

agjohnson added this to the 2.4 milestone May 3, 2018

safwanrahman reviewed May 5, 2018

View reviewed changes

davidfischer mentioned this pull request May 8, 2018

GDPR Meta Issue #3954

Closed

6 tasks

davidfischer added 3 commits May 18, 2018 12:06

Only check navigator.doNotTrack per the standard

3f74168

Merge branch 'master' into do-not-track-support

a62ff17

Update privacy policy with DNT information

f963cdf

ericholscher approved these changes May 22, 2018

View reviewed changes

davidfischer added 5 commits May 22, 2018 10:31

Do Not Track's official space is in the privacy policy

781ccaa

Link to DNT section of privacy policy

39d1663

- For the guide to Google Analytics for doc authors

Rewording of analytics mitigations

7b7d742

Have a setting for DNT

c642625

- It doesn't apply to .com

DNT applies to .com

fd3a886

ericholscher reviewed May 22, 2018

View reviewed changes

davidfischer added 4 commits May 23, 2018 15:28

Change CSRF cookie to last 30 days

4784272

Add readthedocs-hosted.com to same party hosts

f186dc1

Log retention applies to .com as well

66e5e2f

Check DNT setting on base template

df22053

ericholscher approved these changes May 30, 2018

View reviewed changes

Merge branch 'master' into do-not-track-support

e01d94c

ericholscher merged commit d7fceb5 into readthedocs:master May 30, 2018

davidfischer mentioned this pull request Jun 7, 2018

Use autosectionlabel in the privacy policy #4204

Merged

davidfischer mentioned this pull request Sep 10, 2018

Do not track support readthedocs/ethical-ad-server#2

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Do Not Track support #4046

Do Not Track support #4046

davidfischer commented May 1, 2018 •

edited

Loading

safwanrahman May 5, 2018

davidfischer May 6, 2018

safwanrahman May 18, 2018

davidfischer May 18, 2018

davidfischer May 23, 2018

safwanrahman May 5, 2018

davidfischer May 6, 2018

safwanrahman May 18, 2018

davidfischer May 18, 2018

safwanrahman May 18, 2018

davidfischer May 18, 2018

davidfischer May 18, 2018

davidfischer May 18, 2018

davidfischer May 18, 2018

ericholscher May 1, 2018

ericholscher May 22, 2018

ericholscher May 22, 2018

davidfischer May 22, 2018

ericholscher May 22, 2018

davidfischer commented May 22, 2018

ericholscher May 22, 2018

davidfischer May 22, 2018

ericholscher May 22, 2018

davidfischer May 22, 2018

ericholscher May 30, 2018

ericholscher May 30, 2018

davidfischer May 30, 2018

Do Not Track support #4046

Do Not Track support #4046

Conversation

davidfischer commented May 1, 2018 • edited Loading

What DNT means for us:

New endpoints

Cookie changes

Advantages to us

More

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

davidfischer commented May 22, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

davidfischer commented May 1, 2018 •

edited

Loading