Breakout pip._internal.download to a pip._internal.network sub-package

[pradyunsg]

pip._internal.download is almost 1200 LoC.

It'd probably make sense to break it up into smaller modules in a pip._internal.network package.

[cjerdonek]

I think some of the helper functions (like url_to_path), would be better somewhere inside utils, especially ones that don’t have extra dependencies and are needed for non-download tasks.

[pradyunsg]

Yep yep.

I think it may also make sense to move the utilities that deal with the urls, credentials, and related things, to pip/_internal/network/utils.py or a pip/_internal/utils/network.py (I have a slight preference for the former but don't really care enough either way to want to discuss differences between them).

[cjerdonek]

My preference has always been the latter (something in utils) because that way it becomes easier to understand and manage our import dependencies. For example, we can have a rule that modules inside utils should have few dependencies, and we can observe and maintain this easily by making sure that modules in utils don't depend on other sub-packages (so it's an inner-most package). With the former, though, the dependencies between sub-packages can look more like spaghetti (modules in network depending on utils and modules in utils depending on modules in network, etc). Things are easier if the dependencies between sub-packages are "one-way."

[cjerdonek]

FYI, I posted issue #6861 ("Move unpack_file_url() out of download.py"), which is related to this (or perhaps part of it). It's a bit different though because unpack_file_url() is a non-network portion of the functionality.

[pradyunsg]

@chrahunt Could you link your PRs to this issue?

[chrahunt]

#7045 #7062 #7089

[chrahunt]

download.py is now 578 lines and most of its contents have overlapping concerns (checking hashes, resolving between http/file/vcs urls) that can't be as easily broken out. OK to close this?

[pradyunsg]

I think we should move that functionality into the network package as well.

[chrahunt]

What specifically?

[pradyunsg]

Just, like, I think we should not have a download.py module on the top level when we also have a network sub-package.

It's more of preferences than something technical.