Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes for prediction API #89

Merged
merged 6 commits into from
Sep 16, 2024
Merged

Fixes for prediction API #89

merged 6 commits into from
Sep 16, 2024

Conversation

cmelone
Copy link
Collaborator

@cmelone cmelone commented Aug 20, 2024

This PR resolves a couple bugs encountered during testing:

  • package and compiler versions can now be made up of any non-space character. previously, the regex matched semver formatting, but spackages can have versions in other formats
  • clarified that specs should be URL-encoded in calls to /api/v1/allocate
  • fixed issue where expensive variants algo was unable to complete due to some packages storing popular variants as string values rather than boolean. ex: quantum-espresso hdf5=none
  • fixed issue where predictions < 0.5 were being rounded down to zero
  • add database index for commonly executed query, leading to >30x speedup in the prediction algorithm

@github-actions github-actions bot added the docs Improvements or additions to documentation label Aug 20, 2024
@cmelone cmelone self-assigned this Aug 20, 2024
@cmelone cmelone added the bug Something isn't working label Aug 20, 2024
@cmelone cmelone marked this pull request as draft August 20, 2024 01:22
@cmelone cmelone marked this pull request as ready for review August 20, 2024 21:37
@cmelone cmelone requested a review from alecbcs August 20, 2024 21:37
This PR resolves a couple bugs encountered during testing:

- package and compiler versions can now be made up of any non-space character. previously, the regex matched semvar formatting, but spackages can have versions in other formats
- clarified that specs should be URL-encoded in calls to /api/v1/allocate
- fixed issue where expensive variants algo was unable to complete due to some packages storing popular variants string values rather than boolean. ex: quantum-espresso hdf5=none
also fixes a bug which rounded predictions < 0.5 to 0
Previously, requests to the `/v1/allocate` endpoint took around 57ms. After adding a composite index that covers the complete spec and the most common query during the prediction algo, we get a large speedup to around 1.3ms per request. While more indices could be added, the returns were diminished and more risky as we don't want to negatively impact the INSERT speed for the jobs table.
@cmelone cmelone force-pushed the fix/prediction-fix branch from 92da30a to 2c7948a Compare August 21, 2024 22:29
`1500m` (millicores) rather than `1.5` (cores)

and round up all prediction values to nearest integer
Copy link
Member

@alecbcs alecbcs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Thanks @cmelone

@alecbcs alecbcs merged commit 721f995 into develop Sep 16, 2024
5 checks passed
@alecbcs alecbcs deleted the fix/prediction-fix branch September 16, 2024 17:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working docs Improvements or additions to documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants