Skip to content

Community over Code, Apache NiFi, Apache Kafka, Apache Flink, Python, GTFS, Transit, Open Source, Open Data

License

Notifications You must be signed in to change notification settings

tspannhw/FLaNK-Halifax

Repository files navigation

FLaNK-Halifax

Community over Code, Apache NiFi, Apache Kafka, Apache Flink, Python, GTFS, Transit, Open Source, Open Data

Halifax Data

https://data-hrm.hub.arcgis.com/pages/open-data-downloads

https://www.halifax.ca/transportation/halifax-transit

https://www.halifax.ca/transportation/halifax-transit/transit-technology/general-transit-feed-gtfs

https://twitter.com/hfxtransit

Halifax GTFS Feeds

https://gtfs.halifax.ca/realtime/TripUpdate/TripUpdates.pb

https://gtfs.halifax.ca/realtime/Alert/Alerts.pb

https://gtfs.halifax.ca/realtime/Vehicle/VehiclePositions.pb

Apache Iceberg Table


CREATE TABLE default.halifaxalerts
(
  stopsequence string,
  arrivaltime string,
  stopid string,
  tripid string,
  tripstartdate string,
  departuretime string,
  ts string,
  uuid string,
  route_long_name string,
  stop_name string
)
PARTITIONED BY (triprouteid string) STORED BY ICEBERG;

FLaNK Projects

Resources

Halifax

https://github.com/transitland/transitland-atlas/blob/master/feeds/gtfs.halifax.ca.dmfr.json

https://dmfr.transit.land/json-schema/dmfr.schema-v0.5.0.json

Other Feeds

Lookup Table Autogenerated by Apache NiFi


-- public.halifaxlookupstops definition

-- Drop table

-- DROP TABLE public.halifaxlookupstops;

CREATE TABLE public.halifaxlookupstops (
	stop_id int4 NOT NULL,
	stop_code int4 NULL,
	stop_name text NULL,
	stop_desc text NULL,
	stop_lat float8 NULL,
	stop_lon float8 NULL,
	zone_id text NULL,
	stop_url text NULL,
	location_type text NULL,
	parent_station int4 NULL,
	stop_timezone text NULL,
	wheelchair_boarding int4 NULL,
	CONSTRAINT halifaxlookupstops_pkey PRIMARY KEY (stop_id)
);


-- public.halifaxlookuproutes definition

-- Drop table

-- DROP TABLE public.halifaxlookuproutes;

CREATE TABLE public.halifaxlookuproutes (
	route_id int4 NOT NULL,
	agency_id text NULL,
	route_short_name int4 NULL,
	route_long_name text NULL,
	route_desc text NULL,
	route_type int4 NULL,
	route_url text NULL,
	route_color text NULL,
	route_text_color int4 NULL,
	CONSTRAINT halifaxlookuproutes_pkey PRIMARY KEY (route_id)
);

Install

https://github.com/MobilityData/gtfs-realtime-bindings/blob/master/python/README.md

pip3 install --upgrade gtfs-realtime-bindings

Halifax Updates


{"stopsequence":"14","arrivaltime":"1693171354","stopid":"7409","tripid":"19776082","tripstartdate":"20230827","departuretime":"1693171354","triprouteid":"1","ts":"1693170735188","uuid":"f9aabf1b-777d-4ec5-a2b3-c17a10ed6281","route_long_name":"Spring Garden","stop_name":"7409"}


Halifax Alerts


[{"routeid4":"","routeid5":"","descriptionlang":"en","cause":"CONSTRUCTION","stopid":"6179","alertlang":"en","alerttext":"Stop closure St Margarets Bay Rd Before Sheppards Run (6179) is closed for construction","routeid":"21","effect":"STOP_MOVED","stopid5":"","descriptiontext":"Stop Closure Notice:\r\nRoute: 21 Timberlea Outbound\r\nLocation: St Margarets Bay Rd Before Sheppards Run (6179)\r\nCause:  Stop Closed Construction\r\n\r\nAffected Stops:\r\nSt Margarets Bay Rd Before Sheppards Run (6179)","stopid4":"","activeperiodend":"32503694400","stopid3":"","stopid2":"","routeid2":"","routeid3":"","activeperiodstart":"1668085020","ts":"1693172857104","uuid":"359a677f-7f58-41e8-857b-c9257e608756"}]

Halifax Vehicle Positions

[{"route_id":"5","route_long_name":"Portland","bearing":"90.0","directionid":"","latitude":44.66741,"tripid":"19774411","startdate":"20230827","uuid":"5b11bc27-07eb-49c6-b259-79d2da40ca12","speed":"6.7056","vehiclelabel":"1389","vehicleid":3389.0,"longitude":-63.57438,"timestamp":1693108357,"ts":1693108383603}]

Hivemetastore

https://docs.cloudera.com/cdp-private-cloud-base/latest/hive-metastore/topics/hive-configure-hms-uri.html

Videos

https://www.youtube.com/watch?v=uLk4fTh2sXM

Slides

https://www.slideshare.net/bunkertor/coc23utilizing-realtime-transit-data-for-travel-optimization

About

Community over Code, Apache NiFi, Apache Kafka, Apache Flink, Python, GTFS, Transit, Open Source, Open Data

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages