dataverse-metrics aggregates metrics from Dataverse installations around the world and visualizes them on a web page. It makes use of the Dataverse Metrics API that was added in Dataverse 4.9.
If you are looking for visualizations for a single installation of Dataverse, please see https://github.com/gdcc/dv-metrics
- Python 2 (deprecated) or Python 3
- Apache web server or similar
- a web browser
Change to the parent directory for where you will install dataverse-metrics. /var/www/html
is the default "DocumentRoot" for Apache on CentOS (defined in /etc/httpd/conf/httpd.conf
) and is suggested as a place to install dataverse-metrics, but you are welcome to install it wherever you want and use any server you want.
cd /var/www/html
Clone the repo:
git clone https://github.com/IQSS/dataverse-metrics.git
Change to the directory you just created by cloning the repo:
cd dataverse-metrics
Copy config.json.sample
to config.json
and edit the following values:
installations
: An array of Dataverse installation URLs.api_response_cache_dir
: Fully qualified directory where JSON files representing API responses will be stored.aggregate_output_dir
: Fully qualified directory where TSV output files of aggregated metrics will be stored.num_months_to_process
: For monthly metrics, the number of months to go back in time to download metrics from each Dataverse installation.month_filter_enabled
: If you havenum_months_to_process
set high (e.g. 36 months) the dates will disappear from the bottom of the bar charts. Changing this boolean to true will make only even months appear and with fewer bars the dates will be visible.endpoints
: An array of Metrics API endpoints to process. Note that the two types aresingle
(i.e.datasets/bySubject
) andmonthly
(i.e.downloads/toMonth
). (You will notice a third type calledmonthly_itemized
inconfig.json.sample
but it is not yet supported.)blacklists
: Arrays of terms to blacklist. Only thedatasets/bySubject
endpoint can have a blacklist.colors
: A single color for bar charts and a palette of colors for tree maps.github_repos
: An array of GitHub repos such ashttps://github.com/IQSS/dataverse
. A line will be added per repo about the number of contributors.
Now that your config.json
file is ready, run the metrics.py
script to create a TSV file for each of the endpoints
and a contributors.json
file for the github_repos
, all of which will be placed in the aggregate_output_dir
directory:
python3 metrics.py
(Please note that if you don't have Python 3 installed, Python 2 should work fine too but Python 3 is highly recommended because Python 2 will not be maintained past January 1, 2020 according to https://pythonclock.org and PEP 373.)
Using the instructions above, index.html has been placed at /var/www/html/dataverse-metrics/index.html and should be available on your Apache server at http://example.com/dataverse-metrics/index.html
The list of Dataverse installations depends on all-dataverse-installations.json
which can be updated with the following script as new installations are added to the map produced by dataverse-installations:
./update-all-installations-list.sh
To update your metrics periodically, you'll want to queue up a shell script in some flavor of cron.
Here's an example shell script to get you started.
On a Red Hat or CentOS system, you might drop a file like update_metrics.cron into /etc/cron.d/ to update on a specified schedule.
We love contributors! Please see our Contributing Guide for ways you can help and check out the to do list below.
- Drop support for Python 2. See https://python3statement.org