Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

send HTTP caching headers for index pages to further reduce bandwidth usage #12257

Draft
wants to merge 5 commits into
base: main
Choose a base branch
from
Draft

send HTTP caching headers for index pages to further reduce bandwidth usage #12257

wants to merge 5 commits into from

Commits on Aug 13, 2024

  1. cache "concrete" dists by Distribution instead of InstallRequirement

    @cosmicexplorer
    cosmicexplorer committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    ad93d54 View commit details
    Browse the repository at this point in the history

  2. refactor much of RequirementPreparer and add lengthy docs

    @cosmicexplorer
    cosmicexplorer committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    52133ba View commit details
    Browse the repository at this point in the history

  3. use .metadata distribution info when possible 

    When performing `install --dry-run` and PEP 658 .metadata files are
available to guide the resolve, do not download the associated wheels.

Rather use the distribution information directly from the .metadata
files when reporting the results on the CLI and in the --report file.

- describe the new --dry-run behavior
- finalize linked requirements immediately after resolve
- introduce is_concrete
- funnel InstalledDistribution through _get_prepared_distribution() too
- add test for new install --dry-run functionality (no downloading)
    @cosmicexplorer
    cosmicexplorer committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    e9849f8 View commit details
    Browse the repository at this point in the history

  4. make LinkMetadataCache 

    - catch an exception when parsing metadata which only occurs in CI
- handle --no-cache-dir
- call os.makedirs() before writing to cache too
- catch InvalidSchema when attempting git urls with BatchDownloader
- fix other test failures
- reuse should_cache(req) logic
- gzip compress link metadata for a slight reduction in disk space
- only cache built sdists
- don't check should_cache() when fetching
- cache lazy wheel dists
- add news
- turn debug logs in fetching from cache into exceptions
- use scandir over listdir when searching normal wheel cache
- handle metadata email parsing errors
- correctly handle mutable cached requirement
- use bz2 over gzip for an extremely slight improvement in disk usage
    @cosmicexplorer
    cosmicexplorer committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    b02915a View commit details
    Browse the repository at this point in the history

  5. make FetchResolveCache 

    - pipe in headers arg
- provide full context in Link.comes_from
- pull in etag and date and cache the outputs
- handle --no-cache-dir
- add NEWS
- remove quotes from etag and use binary checksum to save a few bytes
- parse http modified date to compress the cached representation
    @cosmicexplorer
    cosmicexplorer committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    f28ecfd View commit details
    Browse the repository at this point in the history