M3 Additional o²S²PARC Metrics

[elisabettai]

Deliverable: Update the DRC metrics document with the new metrics that SIM-Core plans to collect. The NIH has asked for metrics on most used services, computational resource usage, issue and resolution metrics, statistics on frequent bugs/support issues. Corresponding metrics will be established and reported on.
Deadline: Y6Q1

Related DAT-Core milestones:
Milestone 4.5.1 Q1 Metrics Report

Wrike M9.2

More details on the metrics in DRC Metrics doc (link points already to the list relevant for o²S²PARC)

[sanderegg]

• review if new metrics are same as old ones (described year 2 or 3 of oSparc)
• go through Katie's process on generating the metrics
• review the dashboard generated on grafana

[elisabettai]

For "review if new metrics are same as old ones (described year 2 or 3 of oSparc)"
I've checked in the DRC Metrics doc and there are no surprises in terms of metrics.
There are only two things missing, but we should be able to manually get them from the Portal ("organs with simulations" and "number of publications per dataset")

[elisabettai]

"go through Katie's process on generating the metrics":
reviewed everything.

As a rule-of-thumb to filter puppetters, we have this: https://git.speag.com/oSparc/osparc-metrics/-/blob/master/FilterPuppets.md
If we continue with this manual way, we need to update the numbers for the e2e and p2e tests.

[elisabettai]

Goal for sprint Katherine Switzer

• Review all the metrics (grafana/prometeus) and prepare for the report.

Still open:

• Total CPU/hours, could also check if AWS has something already metrics: total cpu consumption over time does not make sense osparc-simcore#2831
• We miss metrics for dy-sidecar started services metrics for number of services started is only showing legacy dynamic services osparc-simcore#3336
• New bug found: webserver started services count looks weird Webserver started services metrics looks weird ("simcore_simcore_service_webserver_services_started_total") osparc-simcore#3472
• Jupyter notebook cannot be started with a security token osparc-simcore#727. Almost done. Missing: automatically create issue from emails to support

Fixed:

• Total number of anonymous users who are redirected to o²S²PARC osparc-simcore#3412. We can now filter puppeteers in the webserver metrics. Need release to prod
• metrics: data storage metrics in AWS is missing S3 storage osparc-simcore#2840 S3 Stodage metrics

[elisabettai]

I'm re-opening this.
There is a bug that needs to be fixed soon: ITISFoundation/osparc-ops-environments#108
A feature request (filtering puppeteers from service usage metrics): ITISFoundation/osparc-simcore#3908

fyi @sanderegg