Skip to content

Latest commit

 

History

History
62 lines (51 loc) · 2.91 KB

new-dataset.md

File metadata and controls

62 lines (51 loc) · 2.91 KB
Raw

#How to Import a New Dataset

  1. To begin, go to Django admin, and find Datasets under Gis_Csdt. Click Add Dataset.
  2. Give a name to the dataset. If the data is available via an API, enter the base URL for the API. It's been tested with the SODA API from OpenNY data sets. It may also work with other APIs.
  3. Under the Field Mapping header, map the API's field names to your database's field names. For a description of each field, see below for field descriptions. See below for information on formatting field mappings.
  4. Additional fields can be customized for each dataset. These are under the heading Custom fields. Give the field name on the remote server in Field# name. Field# en is the name for this field in plain English and what will be displayed as the label for this field. The same nesting and concatenation rules apply.
  5. Check Needs geocoding if the existing latitude and longitude fields are inaccurate. This will run the address against Google's geocoding service. A limited number of requests can be made per day, so this will be done in nightly batches as described in #6.
  6. The data points will be loaded during the daily job. If this has been scheduled, it will occur when scheduled. Otherwise, it can be manually triggered on the server using the command python manage.py runjobs daily. This may need to be repeated over several days to geocode a large batch, due to the Google Maps quota.

##Field Descriptions

  • Column name of the key field on the remote server If available, enter the name of the field representing a unique ID on the server. This will be used to recache the data.
  • Name This field gives the name of the location to be displayed.
  • Lat, Lon These fields hold the latidude and longitude locations of the points.
  • Street, City, State, Zipcode, County Basic address fields

##Formatting Field Mappings ###A basic mapping For example, if the API has a field called zip, type "zip" in the "zipcode" field.

###Nested fields For nested fields, use a comma to denote both fields. For example, if the JSON response is formatted as follows:

{
 "location": {
   "latitude": 45.0234,
   "longitude": -73.2984
 },
}

This would be mapped as follows:

  • Lat field: location,latitude
  • Lon field: location,longitude

###Concatenated fields To concatenate two fields into one on importing, use a + to denote the concatenation. For example, if the JSON response is formatted as follows:

{
 "house number": 4,
 "street": "Main St"
}

This would be mapped as follows:

  • Street field: house number+street

###An example using both concatenation and nesting These can be used together, as in the following example:

{
 "address":{
   "house number": 4,
   "street": "Main St"
 }
}

This would be mapped as follows:

  • Street field: address,house number+street