Add dataclass for AV2 MF challenge submissions #41
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
benjaminrwilson
approved these changes
Apr 12, 2022
| NUM_PRED_TIMESTEPS: Final[int] = 60 | ||
|
|
||
|
|
||
| @dataclass |
There was a problem hiding this comment.
Can this be frozen?
| """ | ||
| for scenario_predictions in self.predictions.values(): | ||
| for track_predictions in scenario_predictions.values(): | ||
| if track_predictions.shape[-2:] != (NUM_PRED_TIMESTEPS, 2): |
There was a problem hiding this comment.
I think it'd be nice to assign track_predictions.shape[-2:] to a descriptive variable name.
|
|
||
| TrackPredictions = NDArrayNumber # Track predictions are expected to be of shape (*, NUM_PRED_TIMESTEPS, 2) | ||
| ScenarioPredictions = Dict[str, TrackPredictions] # Mapping from track ID to track predictions | ||
| PredictionRow = Tuple[str, str, int, NDArrayNumber, NDArrayNumber] |
There was a problem hiding this comment.
[nit] A docstring can be useful here as well
| SUBMISSION_COL_NAMES: Final[List[str]] = [ | ||
| "scenario_id", | ||
| "track_id", | ||
| "prediction_rank", |
There was a problem hiding this comment.
Does this mean we don't expect probabilities?
There was a problem hiding this comment.
Updated to add probabilities as an input.
| "predicted_trajectory_x", | ||
| "predicted_trajectory_y", | ||
| ] | ||
| NUM_PRED_TIMESTEPS: Final[int] = 60 |
There was a problem hiding this comment.
[nit] Can we make this more visible? Seems like an important constant. Probably a config shared across the API as well.
wqi
force-pushed
the
feature/av2-mf-challenge
branch
from
ce28356 to
eb6d202
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR Summary
This PR adds a dataclass to help researchers build submission files for the AV2 motion forecasting challenge.
Challenge submissions are initialized using a mapping from scenario and track IDs to a numpy array of predicted trajectories. The data dict is then serialized to a parquet file, which will be submittable to an evaluation server.
Testing
Added unit tests for data validation and (de)serialization.
In order to ensure this PR works as intended, it is:
Compliance with Standards
As the author, I certify that this PR conforms to the following standards: