Skip to content

Commit

Permalink
Merge branch 'dev' of https://github.com/CityOfLosAngeles/mobility-da…
Browse files Browse the repository at this point in the history 
…ta-standard into dev

* 'dev' of https://github.com/CityOfLosAngeles/mobility-data-standard: (33 commits)
  Update definition of timestamp to match provider Also replacer "UUIDv4" with "UUID4", as is convention
  Update README.md
  Update README.md
  Clarify that the time-based-filtering query parameters should be stackable.
  fix cloud line.
  add mds-agency-cli project
  Adding Cloud as Provider, mds, and gbfs
  language cleanup, link to GPS articles
  truncate -> round
  switch to round, fix typo.
  Update provider to allow precision based truncate. fixes #209
  Cleaning up service_areas endpoint
  Add versioning requirement to Provider API
  Fixing two missing initial event states.
  Update provider/auth.md
  Authentication Requirements. Fixes #171
  Update README.md
  Update README.md
  generate full schemas
  clean up and fix the schema
  ...
  • Loading branch information
@hunterowens
hunterowens committed Feb 14, 2019
2 parents 1bbdc92 + 1da4150 commit 7352119
Show file tree
Hide file tree
Showing 8 changed files with 406 additions and 340 deletions.
11 changes: 10 additions & 1 deletion README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,12 @@ Cities and regulators can choose best how to implement *Agency* and *Provider* e

The specification will be versioned using Git tags and [semantic versioning](https://semver.org/). See prior [releases](https://github.com/CityOfLosAngeles/mobility-data-specification/releases) and the [Release Guidelines](ReleaseGuidelines.md) for more information.

## Roadmap
## Announcements

The City of Los Angeles is currently looking for feedback and comments on the draft versions. Comments can be made by making an Github Issue, while suggested changes can be made using a pull request. The rules and guidelines for the Los Angeles Dockless Bikeshare Systems / Pilot Program can be found on [Council Clerk Connect](https://cityclerk.lacity.org/lacityclerkconnect/index.cfm?fa=ccfi.viewrecord&cfnumber=17-1125).

*2/12/2019 Update*: City of Los Angeles One Year Permit Application MDS Agency Compliance: LADOT is seeking compliance with MDS Agency as a requirement of the one-year permit. LADOT hosted a webinar on Thursday, February 7th to give an overview of the MDS Agency sandbox, discuss and answer questions about integration and timeline for Agency services. Slides from the presentation can be viewed [here](https://ladot.lacity.org/sites/g/files/wph266/f/MDS%20Developer%20Webinar%20-%20One%20Year%20Permitting%20Overview.pdf).

*12/27/2018 Update*: Applications for the One-Year Dockless On-Demand Personal Mobility Permit are now available on the [LADOT Website](https://ladot.lacity.org/ladot-begins-one-year-dockless-demand-personal-mobility-program)

*10/28/2018 Update*: [LADOT Guidelines for Handling of Data from Mobility Service Providers](http://www.urbanmobilityla.com/s/LADOT-Guidelines-for-Handling-of-Data-from-MSPs-2018-10-25.pdf)
Expand All @@ -35,11 +37,18 @@ The City of Los Angeles is currently looking for feedback and comments on the dr
* [`mds-dev`](https://github.com/cityoflosangeles/mds-dev) - Code to do cap checking, fake data generation and more with provider data.
* [`mds-validator`](https://github.com/cityoflosangeles/mds-validator) - Code to validate MDS APIs using JSONSchema.
* [`aqueduct`](https://github.com/cityoflosangeles/aqueduct) - ETL, Data Warehousing, and Machine Learning Platform for LA City Data Science team. Handles extracting MDS provider APIs and storing in data warehouse.
* [`mds-agency-cli`](https://github.com/cityoflosangeles/mds-agency-cli) - Nodejs-based command-line interface to exercise the Agency API in the LADOT sandbox

### City of Santa Monica
* [`mds-provider`](https://github.com/cityofsantamonica/mds-provider) - Python package implementing the provider API, validation using JSONSchema, data loading to multiple targets, and fake provider data generation.
* [`mds-provider-services`](https://github.com/cityofsantamonica/mds-provider-services) - Python scripts wrapped in Docker containers implementing a MDS provider data ingestion flow, using `mds-provider` and handling the various dependencies.

### City of Austin
* [`transportation-dockless-dataviz`](https://github.com/cityofaustin/transportation-dockless-dataviz) - A hexbin origin/destintation web map of dockless trips using jQuery & Mapbox GL JS. See [http://dockless.austintexas.io/](http://dockless.austintexas.io/).
* [`transportation-dockless-api`](https://github.com/cityofaustin/transportation-dockless-api) - Python Sanic-based API that provides an interface for retrieving anonymized and aggregated trip data. This API supplies data to our interactive [Dockless Mobility Explorer](https://dockless.austintexas.io). The source database for the API is our [Dockless Vehicle Trips](https://data.austintexas.gov/Transportation-and-Mobility/Dockless-Vehicle-Trips/7d8e-dm7r) dataset.
* [`transportation-dockless-processing`](https://github.com/cityofaustin/transportation-dockless-processing) - Python scripts for dockless mobility data ETL.
* [`mds-provider-client`](https://github.com/cityofaustin/mds-provider-client) - A forked Python client from [`CityofSantaMonica/mds-provider`](https://github.com/cityofsantamonica/mds-provider).

### Others

Please open a pull request if you create open source or private MDS tooling.
Expand Down
273 changes: 142 additions & 131 deletions agency/README.md
View file

Large diffs are not rendered by default.

189 changes: 90 additions & 99 deletions generate_schema/provider/status_changes.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -41,94 +41,12 @@
"event_time",
"event_location"
],
"oneOf": [
{
"properties": {
"event_type": {
"type": "string",
"description": "The type of the event",
"enum": [
"available"
]
},
"event_type_reason": {
"type": "string",
"description": "The reason for the event",
"enum": [
"service_start",
"user_drop_off",
"rebalance_drop_off",
"maintenance_drop_off"
]
}
}
},
{
"properties": {
"event_type": {
"type": "string",
"description": "The type of the event",
"enum": [
"reserved"
]
},
"event_type_reason": {
"type": "string",
"description": "The reason for the event",
"enum": [
"user_pick_up"
]
}
}
},
{
"properties": {
"event_type": {
"type": "string",
"description": "The type of the event",
"enum": [
"unavailable"
]
},
"event_type_reason": {
"type": "string",
"description": "The reason for the event",
"enum": [
"low_battery",
"maintenance"
]
}
}
},
{
"properties": {
"event_type": {
"type": "string",
"description": "The type of the event",
"enum": [
"removed"
]
},
"event_type_reason": {
"type": "string",
"description": "The reason for the event",
"enum": [
"service_end",
"rebalance_pick_up",
"maintenance_pick_up"
]
}
}
}
],
"properties": {
"provider_name": {
"$id": "#/properties/data/properties/status_changes/items/properties/provider_name",
"type": "string",
"description": "The public-facing name of the Provider",
"examples": [
"Provider Name"
],
"examples": ["Provider Name"],
"pattern": "^(.*)$"
},
"provider_id": {
Expand All @@ -146,9 +64,7 @@
"type": "string",
"description": "The Vehicle Identification Number visible on the vehicle itself",
"default": "",
"examples": [
"ABC123"
],
"examples": ["ABC123"],
"pattern": "^(.*)$"
},
"vehicle_type": {
Expand All @@ -171,26 +87,101 @@
"description": "The GPS coordinates of where the event occurred",
"$ref": "#/definitions/MDS_Feature_Point"
},
"event_type": {
"type": "string",
"description": "The type of the event"
},
"event_type_reason": {
"type": "string",
"description": "The reason for the event"
},
"battery_pct": {
"$id": "#/properties/data/properties/status_changes/items/properties/battery_pct",
"type": ["number", "null"],
"description": "Percent charge of device battery, expressed between 0 and 1",
"examples": [
0.89
],
"examples": [0.89],
"minimum": 0,
"maximum": 1
},
"associated_trips": {
"$id": "#/properties/data/properties/status_changes/items/properties/associated_trips",
"type": ["array", "null"],
"description": "For 'Reserved' event types, associated trip_ids",
"items": {
"$id": "#/properties/data/properties/status_changes/items/properties/associated_trips/items",
"$ref": "#/definitions/uuid"
}
"associated_trip": {
"$id": "#/properties/data/properties/status_changes/items/properties/associated_trip",
"description": "Trip UUID, required if event_type_reason is user_pick_up or user_drop_off",
"$ref": "#/definitions/uuid"
}
},
"allOf": [
{
"description": "valid event_type and event_type_reason combinations",
"oneOf": [
{
"properties": {
"event_type": {
"enum": ["available"]
},
"event_type_reason": {
"enum": [
"service_start",
"user_drop_off",
"rebalance_drop_off",
"maintenance_drop_off"
]
}
}
},
{
"properties": {
"event_type": {
"enum": ["reserved"]
},
"event_type_reason": {
"enum": ["user_pick_up"]
}
}
},
{
"properties": {
"event_type": {
"enum": ["unavailable"]
},
"event_type_reason": {
"enum": ["low_battery", "maintenance"]
}
}
},
{
"properties": {
"event_type": {
"enum": ["removed"]
},
"event_type_reason": {
"enum": [
"service_end",
"rebalance_pick_up",
"maintenance_pick_up"
]
}
}
}
]
},
{
"description": "conditionally require associated_trip when applicable",
"oneOf": [
{
"not": {
"properties": {
"event_type_reason": {
"enum": ["user_pick_up", "user_drop_off"]
}
}
}
},
{
"required": ["associated_trip"]
}
]
}
}
]
}
}
},
Expand All @@ -202,4 +193,4 @@
}
},
"additionalProperties": false
}
}
41 changes: 37 additions & 4 deletions provider/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,39 @@ This specification contains a data standard for *mobility as a service* provider

The following information applies to all `provider` API endpoints. Details on providing authorization to endpoints is specified in the [auth](auth.md) document.

### Versioning

`provider` APIs must handle requests for specific versions of the specification from clients.

Versioning must be implemented through the use of a custom media-type, `application/vnd.mds.provider+json`, combined with a required `version` parameter.

The version parameter specifies the dot-separated combination of major and minor versions from a published version of the specification. For example, the media-type for version `0.2.1` would be specified as `application/vnd.mds.provider+json;version=0.2`

> Note: Normally breaking changes are covered by different major versions in semver notation. However, as this specification is still pre-1.0.0, changes in minor versions may include breaking changes, and therefore are included in the version string.
Clients must specify the version they are targeting through the `Accept` header. For example:

```http
Accept: application/vnd.mds.provider+json;version=0.3
```

> Since versioning was not added until the 0.3.0 release, if the `Accept` header is `application/json` or not set in the request, the `provider` API must respond as if version `0.2` was requested.
Responses to client requests must indicate the version the response adheres to through the `Content-Type` header. For example:

```http
Content-Type: application/vnd.mds.provider+json;version=0.3
```

> Since versioning was not added until the 0.3.0 release, if the `Content-Type` header is `application/json` or not set in the response, version `0.2` must be assumed.
If an unsupported or invalid version is requested, the API must respond with a status of `406 Not Acceptable`. In which case, the response should include a body specifying a list of supported versions.

### Response Format

The response to a client request must include a status code defined in the [IANA HTTP Status Code Registry](https://www.iana.org/assignments/http-status-codes/http-status-codes.xhtml).
The response to a client request must include a valid HTTP status code defined in the [IANA HTTP Status Code Registry](https://www.iana.org/assignments/http-status-codes/http-status-codes.xhtml). It also must set the `Content-Type` header, as specified in the [Versioning](#Versioning) section.

Responses must be `UTF-8` encoded `application/json` and must minimally include the MDS `version` and a `data` payload:
Response bodies must be a `UTF-8` encoded JSON object and must minimally include the MDS `version` and a `data` payload:

```json
{
Expand Down Expand Up @@ -147,6 +175,8 @@ The trips API should allow querying trips with a combination of query parameters
* `min_end_time`: filters for trips where `end_time` occurs at or after the given time
* `max_end_time`: filters for trips where `end_time` occurs before the given time

When multiple query parameters are specified, they should all apply to the returned trips. For example, a request with `?min_end_time=1549800000000&max_end_time=1549886400000` should only return trips whose end time falls in the range `[1549800000000, 1549886400000)`.

### Vehicle Types

| `vehicle_type` |
Expand All @@ -169,7 +199,7 @@ A device may have one or more values from the `propulsion_type`, depending on th

To represent a route, MDS `provider` APIs must create a GeoJSON [`FeatureCollection`](https://tools.ietf.org/html/rfc7946#section-3.3), which includes every [observed point][geo] in the route.

Routes must include at least 2 points: the start point and end point. Additionally, routes must include all possible GPS samples collected by a provider.
Routes must include at least 2 points: the start point and end point. Routes must include all possible GPS samples collected by a Provider. Providers may round the latitude and longitude to the level of precision representing the maximum accuracy of the specific measurement. For example, [a-GPS](https://en.wikipedia.org/wiki/Assisted_GPS) is accurate to 5 decimal places, [differential GPS](https://en.wikipedia.org/wiki/Differential_GPS) is generally accurate to 6 decimal places. Providers may round those readings to the appropriate number for their systems.

```js
"route": {
Expand Down Expand Up @@ -229,7 +259,7 @@ Schema: [`status_changes` schema][sc-schema]
| `event_time` | [timestamp][ts] | Required | Date/time that event occurred, based on device clock |
| `event_location` | GeoJSON [Point Feature][geo] | Required | |
| `battery_pct` | Float | Required if Applicable | Percent battery charge of device, expressed between 0 and 1 |
| `associated_trips` | UUID[] | Optional based `event_type_reason` | Array of UUID's. For `user`-generated event types, associated trips (foreign key to Trips API) |
| `associated_trip` | UUID | Required if Applicable | Trip UUID (foreign key to Trips API) required if `event_type_reason` is `user_pick_up` or `user_drop_off` |

### Status Changes Query Parameters

Expand All @@ -238,6 +268,8 @@ The status_changes API should allow querying status changes with a combination o
* `start_time`: filters for status changes where `event_time` occurs at or after the given time
* `end_time`: filters for status changes where `event_time` occurs before the given time

When multiple query parameters are specified, they should all apply to the returned status changes. For example, a request with `?start_time=1549800000000&end_time=1549886400000` should only return status changes whose `event_time` falls in the range `[1549800000000, 1549886400000)`.

### Event Types

| `event_type` | Description | `event_type_reason` | Description |
Expand Down Expand Up @@ -295,6 +327,7 @@ Response:

All MDS compatible `provider` APIs must expose a public [GBFS](https://github.com/NABSA/gbfs) feed as well. Given that GBFS hasn't fully [evolved to support dockless mobility](https://github.com/NABSA/gbfs/pull/92) yet, we follow the current guidelines in making bike information avaliable to the public.

- `gbfs.json` is always required and must contain a `feeds` property that lists all published feeds
- `system_information.json` is always required
- `free_bike_status.json` is required for MDS
- `station_information.json` and `station_status.json` don't apply for MDS
Expand Down
6 changes: 6 additions & 0 deletions provider/auth.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -31,3 +31,9 @@ OAuth 2.0's `client_credentials` grant type (outlined in [RFC6749](https://tools
OAuth 2.0 is an industry standard authorization framework with a variety of existing tooling. The `client_credentials` grant type facilitates generation of tokens that can be used for access by agencies and distributed to data partners.

If an MDS `provider` implements this auth scheme, it **MAY** choose to specify token scopes that define access parameters like allowable time ranges. These guidelines **SHOULD** be encoded into the returned token in a parseable way.

## Endpoint Authenication Requirements

The `/trips` and `/status_changes` endpoints may or may not require authentication, it is up to the implementing provider to decide if they want historical information available with or without authentication.

As of MDS 0.3.0, `gbfs.json` is required. The required GBFS endpoints should be made available publicly. See [#realtime-data](https://github.com/CityOfLosAngeles/mobility-data-specification/tree/master/provider#realtime-data) for more information about how to implement GBFS for dockless systems.
Loading

0 comments on commit 7352119

Please sign in to comment.