Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add MobilityData GTFS Validator to validation step #512

Merged
merged 17 commits into from
Mar 10, 2023

Conversation

miles-grant-ibigroup
Copy link
Contributor

@miles-grant-ibigroup miles-grant-ibigroup commented Feb 16, 2023

Checklist

  • Appropriate branch selected (all PRs must first be merged to dev before they can be merged to master)
  • Any modified or new methods or classes have helpful JavaDoc and code is thoroughly commented
  • The description lists all applicable issues this PR seeks to resolve
  • The description lists any configuration setting(s) that differ from the default settings
  • All tests and CI builds passing

Description

This PR integrates the MobilityData GTFS Validator into datatools, and adds a MobilityData validation to the GTFS validation step. Validation results are saved to a unique temporary folder, then loaded and inserted into Mongo. Failures are handled via the existing validation failure infrastructure.

Room for Improvement

  • I'd like to perhaps see the MobilityData validation happen in a second thread, possibly in parallel to the existing validation. However, I'm not sure how to do this in the cleanest way, and the new validation isn't that slow, especially compared to the existing one.

Important Notices

The MobilityData validator requires a much newer version of the Java spec. I've updated by JDK to 18, and language level to 11!

Tests are failing due to an unrelated shape file export issue

@miles-grant-ibigroup miles-grant-ibigroup force-pushed the mobility-data-validator branch 3 times, most recently from 2ca035d to 9be044a Compare February 16, 2023 15:34
@br648
Copy link
Contributor

br648 commented Feb 17, 2023

@miles-grant-ibigroup for review: #513. We should discuss at some point next week.

@br648 br648 assigned miles-grant-ibigroup and unassigned br648 Feb 20, 2023
@br648
Copy link
Contributor

br648 commented Feb 20, 2023

@miles-grant-ibigroup I'm going to defer my review until this #513 is tested and merged.

@br648 br648 assigned miles-grant-ibigroup and unassigned br648 Mar 6, 2023
@miles-grant-ibigroup
Copy link
Contributor Author

Thanks Rob! All changes have been addressed

Copy link
Contributor

@br648 br648 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All good. Good luck presenting the new data!

Copy link
Contributor

@philip-cline philip-cline left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good to me, seems to work well.

@miles-grant-ibigroup miles-grant-ibigroup merged commit 3def660 into dev Mar 10, 2023
@miles-grant-ibigroup miles-grant-ibigroup deleted the mobility-data-validator branch March 10, 2023 13:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants