New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support model downloading/caching #8

Closed

carlosgjs opened this issue Dec 7, 2023 · 0 comments · Fixed by #34

Assignees

Milestone

Collaborator

carlosgjs commented Dec 7, 2023 •

edited

Loading

The models are too big to ship with the package, so a better approach is to download them on first use and cache them locally.

Subtasks:

Create an org in Huggingface
Upload the models to the org (two versions? 7b and 7b quantized?
Create a mapping of a model version+platform to physical model file
Add logic to download on-demand
Support model versioning and checking for newer models

carlosgjs self-assigned this

carlosgjs added this to the pre-data milestone

carlosgjs mentioned this issue

feat: Publish/load pre-quantized models #34

Merged

carlosgjs closed this as completed in #34

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment