Add telemetry

[valentinsulzer]

Description

Adds very basic telemetry - records when a simulation is solved.

The following section has been added to the user guide to explain the telemetry we are doing:

PyBaMM collects anonymous usage data to help improve the library. This telemetry is enabled by default but can be easily disabled. Here's what you need to know:

• What is collected: Basic usage information like PyBaMM version, Python version, and which functions are run.
• Why: To understand how PyBaMM is used and prioritize development efforts.
• Opt-out: To disable telemetry, set the environment variable PYBAMM_OPTOUT_TELEMETRY=true or use pybamm.telemetry.disable() in your code.
• Privacy: No personal information (name, email, etc) or sensitive information (parameter values, simulation results) is ever collected.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 99.27%. Comparing base (9ac0b5b) to head (1022c6b).
Report is 31 commits behind head on develop.

Additional details and impacted files

@@           Coverage Diff            @@
##           develop    #4441   +/-   ##
========================================
  Coverage    99.27%   99.27%           
========================================
  Files          300      302    +2     
  Lines        22795    22868   +73     
========================================
+ Hits         22630    22703   +73     
  Misses         165      165           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[agriyakhetarpal]

We could also make pybamm.telemetry.disable() a top-level pytest fixture as a cleaner way to disable telemetry when running tests

[valentinsulzer]

We could also make pybamm.telemetry.disable() a top-level pytest fixture as a cleaner way to disable telemetry when running tests

Yes, but we also want to disable it when our dependencies run their tests, so that wouldn't be enough right?

[agriyakhetarpal]

We could also make pybamm.telemetry.disable() a top-level pytest fixture as a cleaner way to disable telemetry when running tests

Yes, but we also want to disable it when our dependencies run their tests, so that wouldn't be enough right?

Oh, yes, forgot about that! In that case, we would need both the logic you added and the fixture to call it.

[kratman]

I understand why we want to know which parts of PyBaMM our users are using, but I am not sure this is the best approach.

My concerns:

• I feel like I would be upset to find out a free research tool was tracking my usage and sending it to a server. I think for a paid tool the expectation that they want to know your usage is a little more reasonable.
• Users installing through pip might not realize we are tracking their usage. It would be better to allow an opt-in instead so that users know we are collecting their data.
• Are we only going to be tracking simulation data so that we can just keep it in the simulation class or are we going to have to add the tracking to a lot of different places and make sure we add it to new functionality?

[martinjrobins]

Thanks @valentinsulzer. I suggest a timeout with default opt-out so we don't break ppl's CI, see below

[valentinsulzer]

Added an input with timeout but can't figure out why the docs are failing as I can't reproduce it locally

[agriyakhetarpal]

The code looks fine, the only thing problematic here, at least to me, seems to be the dependency added on inputimeout: https://pypi.org/project/inputimeout hasn't had a release in the last six years. Could we replace it with a different dependency that can do the same task? I'm just worried on the prospect of it causing issues with constraints or with us upgrading to new Python versions as they come some time down the line.

[agriyakhetarpal]

Thanks, @valentinsulzer! There are many #pragma: no cover statements here, I think they can be cleaned up. We can add this to .github/codecov.yml:

ignore:
  - src/pybamm/config.py
  # and so on

Also, there is another problem here that I feel deserves another look. I tried a notebook from this branch, and there's no option for me to provide a Y/n answer for opting in/out of telemetry. One would have to wait for ten seconds to default to a "no", which isn't a lot, but this doesn't make for good UX. Is there a way to detect if one is in an IPython kernel and reduce the timeout (or accept stdin in a different manner)?

[valentinsulzer]

Updated to work for notebooks. Re: coverage, I don't want to skip the whole file since it's possible to test some parts of the file. I'll look into covering more parts with some pytest magic

[agriyakhetarpal]

Thanks, @valentinsulzer! Two more things:

• I started a PR to include this in the release notes just in case we forget it (please feel free to modify what I wrote and push any changes to it directly): Release notes for next release pybamm.org#214 and I read up a bit on PostHog's data protection policies: https://posthog.com/docs/privacy/gdpr-compliance. Should we switch to their EU-based cloud instance instead of the current US one to better achieve GDPR compliance (not that we collect anything non-anonymised right now, but just in case we start collecting more data points in the future)? We use Plausible for the docs analytics already which is compliant, so that makes an additional case for changing.
• Since we're tracking when a simulation is solved, it might also make sense to capture the host operating system (just once at the time of initial config setup) since our compiled solvers are in some ways supported differently across platforms and that can help with platform-specific issues later on.

[agriyakhetarpal]

Thanks, @valentinsulzer, I've approved the code changes already but my points above still apply

[valentinsulzer]

Thanks @agriyakhetarpal , let's chat on the pybamm.org PR about the announcement. Feel free to open additional PRs with more logging as necessary