Skip to content

SmartRoadSense database administration tool: a simple data exporter

Notifications You must be signed in to change notification settings

SmartRoadSense/exporter

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

16 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

SmartRoadSense data exporter

This simple Python script is thought to make it easier to export SRS data for debugging and analysis purposes.

The script runs a query against the raw or the aggregated database. As the raw database is concerned, by default it queries the single_data table, but using the --old (or -O) option it can be set to use the single_data_old table instead. The current table is queried by using the --aggregate flag.

It requires the following environmental variables set:

  1. SRS_EXPORTER_HOST (default is 127.0.0.1)
  2. SRS_EXPORTER_DB_RAW (default is srs_raw_db)
  3. SRS_EXPORTER_DB_AGG (default is srs_agg_db)
  4. SRS_EXPORTER_USER (default is postgres)
  5. SRS_EXPORTER_PASS (default is postgres)
  6. SRS_EXPORTER_PORT (default is 5432)

The script's usage is quite straight forward:

Usage:
  exporter.py [OPTIONS]

Usage examples:
  exporter.py -l 10
  exporter.py --longitude 12.92290593 --latitude 43.74830223 -d 1000
  exporter.py -T 56237 --meta
  exporter.py -a -l 10
  exporter.py -O -A 2018-04-01 --before 2018-04-02

Options:
  -d --distance <int>      Distance in meters used for range queries
                           (Default:100).
  -t --latitude <float>    Latitude used for range queries.
  -g --longitude <float>   Longitude used for range queries.
  -A --after <datetime>    Selected rows have to be created after this
                           specific datetime value.
  -B --before <datetime>   Selected rows have to be created before this
                           specific datetime value.
  -T --track <track_id>    Track id of selected rows.
  -m --meta                If specified rows will be exported with their
                           track's metadata (NOTE: cause a JOIN).
  -o --output <filename>   Filename/path where results have to be written.
  -a --aggregate           If specified data will be queried from
                           the "current" table in the aggregate database.
  -O --Old                 If specified data will be queried from
                           the "single_data_old" table in the raw database.
  -l --limit <int>         Limit If specified data will be queried from
                           the "single_data_old" table in the raw database.
  --debug                  Print debug information.
  -h --help                Print this help.

Within Docker

Please consider using this script within a Docker container (see: Docker smartroadsense/dataexporter image)

When used within the Docker container linked above please note that the script will be in the /tmp/ directory and should be executed in this way in order to output results in the shared /data docker volume:

$ python exporter.py -o /data/data.csv [OTHER OPTIONS]

About

SmartRoadSense database administration tool: a simple data exporter

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published