Analytics in Go-IPFS #980
Comments
|
We also need to be careful about users using IPFS over a VPN, Tor, etc. Ideally, the webui would communicate with our analytics backend through IPFS/libp2p but we'll have to figure that out. However, if it's opt-in, we can probably punt on that. |
|
For context for people not following along: the current situation is described in mozilla/addons-frontend#930
Agreed. It is simplest from where we are to just disable by default and ask the user.
Agreed.
This what we have in place so far |
|
Ideally, we'd also do as much event aggregation locally instead of sending events to the server (e.g., "the user uses X/Y/Z pages instead of "the user visited X/Y/Z pages at time T"). We don't need (or want) to know when a user is using the WebUI, we just want to know which features are useful, etc. For context, my syncthing instance reports: |
|
Also, we need to audit errors. If we send back errors, we need to make sure they don't include sensitive information (ideally asking the user if they want to send an error report, telling the user exactly what's included). |
|
Just to check we're on the same page, this is only counting which sections of the webapp are visited, and a handful of actions that tell us you added / removed / moved files or saved a config change in the abstract. It doesn't capure any information about go-ipfs, or your config. it doesn't capture local MFS paths nor does it capture CIDs you browses in the explore page. Only that you used the explore page. I can disabled city level granularity for "where are our users" which is just a best effort geolocation on ipv4 address, and i can disable error reporting as it's hard to verify that an error will always be free of identifiable info. The errors it would capture right now are only js errors in the web app and the http client. It won't capture go-ipfs errors. @Stebalien other than opt-in what are the changes from what we have that must happen before we can release it with go-ipfs? I'm happy to investigate event aggregation, but I'd rather not make that a blocker. |
|
IMHO I think there’s a difference between the metrics requirements about go-ipfs usage vs webUI interaction/usage. If I understand correctly, the metrics that @olizilla has implemented are only for the latter, and general usage of go-ipfs would in no way trigger any metrics collection until the user decided to interact with the webUI property to see stats and use features built into that UI, and even then the metrics would be only about usage of that web UI. I think that interaction path is less sensitive than the whole of go-ipfs (which we’d love more ways to understand, but 100% agree we need to be very careful/opt-in about that). I’d argue therefor that this should be able to proceed with the same metrics collection (for webUI included in go-ipfs) as all other instances of webUI/web properties. @Stebalien - if you still strongly disagree I think the work around is as @olizilla suggests - turning the metrics default to “opt-in” (and all the other suggestions are nice-to-have). Does that sound right? Would that also make sending back errors “opt in” by default, oli? |
From the user's perspective, the WebUI is a part of go-ipfs (it's the GUI). Interaction with the WebUI is just as sensitive as interaction through the CLI. We need to think of this as an offline app, not a website.
Other than the privacy implications, my primary concern is the message we're sending. We need to send the message "we care about your privacy, we've thought through this, and we're not trying to sneak anything past you". This is going to take a lot of careful thought (consider how much time Mozilla has spent on this). However, we can take a short-cut. We can:
That will allow us to iterate on this while clearly indicating that it's not finished. To bring this out of beta, we need to:
Once we've done that, we can go ahead and ask users if they'd be willing to opt-in (we can even present a sample of the information that would be reported). However, we can only ever ask users to opt-in ONCE. We need to get it right the first time. |
- remove doNotTrack detection in favour of always opt-in wip on: #980 License: MIT Signed-off-by: Oli Evans <[email protected]>
|
For shared reference, here are the events that are tracked in current implementation Begin SessionSent once at the start of a new session to provide metrics on
Example RequestQuery paramsbegin_session: 1
metrics: {"_app_version":"2.4.0","_ua":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.119 Safari/537.36","_resolution":"1920x1200","_density":1,"_locale":"en-GB"}
app_key: 8fa213e6049bff23b08e5f5fbac89e7c27397612
device_id: 17de0839-d9b5-4800-9a3c-e07270e0b17b
sdk_name: javascript_native_web
sdk_version: 19.02.1
timestamp: 1551799890469
hour: 15
dow: 2Session durationTells us how long people used the Web UI for. Occurs on first enabling analytics and periodically after that. Example RequestQuery paramssession_duration: 9
app_key: 8fa213e6049bff23b08e5f5fbac89e7c27397612
device_id: 313e37ac-4d11-4280-8e39-7d2abeb0e9ad
sdk_name: javascript_native_web
sdk_version: 18.11
timestamp: 1551277005824
hour: 14
dow: 3Page viewTells us when people load the initial page or navigate to a new section. Sections are defined as the items in the primary nav. The paths that are recorded are the regex patterns used to match the route rather than the full url, see: https://github.com/ipfs-shipyard/ipfs-webui/blob/38743dc9201795bc292e6ef132600622eb461cb2/src/bundles/analytics.js#L46-L55 Example RequestQuery paramsevents: [{"key":"[CLY]_view","count":1,"dur":9,"segmentation":{"name":"/"},"timestamp":1551277005823,"hour":14,"dow":3},{"key":"[CLY]_view","count":1,"segmentation":{"name":"/","visit":1,"domain":"localhost"},"timestamp":1551277006675,"hour":14,"dow":3}]
app_key: 8fa213e6049bff23b08e5f5fbac89e7c27397612
device_id: 313e37ac-4d11-4280-8e39-7d2abeb0e9ad
sdk_name: javascript_native_web
sdk_version: 18.11
timestamp: 1551277006676
hour: 14
dow: 3Custom EventApp specific actions. We record that the action happened as the The recorded actions are:
Example RequestQuery paramsevents: [{"key":"CONFIG_SAVE","count":1,"dur":0.03000499999686144,"timestamp":1551277789854,"hour":14,"dow":3}]
app_key: 8fa213e6049bff23b08e5f5fbac89e7c27397612
device_id: 313e37ac-4d11-4280-8e39-7d2abeb0e9ad
sdk_name: javascript_native_web
sdk_version: 18.11
timestamp: 1551277789855
hour: 14
dow: 3 |
I’m not 100% sure we specifically need this in this project but we will certainly use this kind of data collected from our own websites and would potentially use it if collected from other resources. In the past I’ve used this exact data from nodejs.org to figure out where to have the Node.js Foundation run events. |
|
This issue is a useful reference point mozilla/addons#3145 - firefox added google analytics to the add-ons store which, while a seperate site, appears as part of the app ui when accessed via firefoxes add ons menu. The generaly feeling of the thread was that it should be opt-in, and it was weird that they were using google. Their solution was to make it doNotTrack aware, and plead to a special deal they have with google, which seems unconvincing to an end user who has little insight to the deal and if it is meaningfully enforcable. If we roll out opt-in analytics, and dont send the data to a third party like google, and show clearly what is collected, I think we are doing well. It may not do much for our actual goal of getting useful metrics on numbers of users across ipfs webui and ipfs desktop tho. Having some un-representative numbers may be better than none at all. At least we'd see any significant changes in user numbers that occur. |
see: #980 License: MIT Signed-off-by: Oli Evans <[email protected]>
|
@Stebalien work in progress here #985 (comment) |
|
Those events look fine, we just need to find a way to communicate this to the user. My one concern is that I'd much prefer to aggregate locally and avoid sending time-stamped events. It's the difference between turning in a report at the end of every day and having someone look over your shoulder. However, I understand that this just isn't how people do web analytics so I'm not going to block on this.
That sounds like a great metric to collect from our websites and probably significantly more useful/reliable.
Yeah, Mozilla has made a bunch of questionable decisions. We should emulate their messaging and communication, not necessarily their execution. |
Go-IPFS analytics requirements:
If you'd like a good example of how to do this, please check out how syncthing handles analytics.
The text was updated successfully, but these errors were encountered: