Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade filter logic #399

Merged
merged 3 commits into from
Dec 5, 2023
Merged

Upgrade filter logic #399

merged 3 commits into from
Dec 5, 2023

Conversation

jeremykubica
Copy link
Contributor

The filtering done in analysis_utils.py only filters out results with < 3 observations (after sigmaG clipping) regardless of the number of observations provided in the config. This tightens the threshold to match the given number of observations. This changes the diff test by dropping out results that had too few observations.

This also combines the two filtering steps for efficiency, so we are only looping through the results once (instead of twice).

Adds two new filters:

  • CombinedStatsFilter filter combines likelihood bounds and minimum number of observations. This is for efficiency
  • DurationFilter filters based on the time duration of the track. This is a potential to explore in the future.

Finally the PR removes the custom _make_trajectory helper function in one of the tests and replaces it with the make_trajectory function from trajectory_utils.py.

So these rows get labeled in the filtered labels
@maxwest-uw
Copy link
Collaborator

sorry I'll get this done by EoD monday, I'm trying to chase down a hipscat-import bug atm 😪

@jeremykubica jeremykubica merged commit 0ff88d0 into main Dec 5, 2023
2 checks passed
@jeremykubica jeremykubica deleted the add_filters branch December 5, 2023 00:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants