Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable Cargo to work without any network access #2111

Closed
brson opened this issue Nov 4, 2015 · 6 comments
Closed

Enable Cargo to work without any network access #2111

brson opened this issue Nov 4, 2015 · 6 comments

Comments

@brson
Copy link
Contributor

brson commented Nov 4, 2015

Build farms need to build Cargo projects without hitting the network. Assuming that distros rewrite their Cargo.tomls to convert dependencies to local paths, then as far as I know the other source of network access is just updating the index.

re https://internals.rust-lang.org/t/perfecting-rust-packaging-the-plan/2767

@aidanhs
Copy link
Member

aidanhs commented Nov 4, 2015

It'd also be nice if cargo didn't blow up if it fails to hit the network for downloading the index and instead tries to continue as far as possible.

@alexcrichton
Copy link
Member

cc #1882

@brson
Copy link
Contributor Author

brson commented Nov 16, 2015

For solving gecko's case, where they primarily don't want the build servers talking to the network, @alexcrichton favors a cargo package --deployment that creates a single tarball that contains the source code of the whole dag.

Even if we can sell that to gecko, I'm skeptical that Linux distros will like it - they want to use their package system to track Rust packages, and they don't want to duplicate the source of dependencies.

@cardoe
Copy link
Contributor

cardoe commented Nov 25, 2015

So for Yocto, I would be comfortable with something similar to the Gecko use case. With Yocto the compilation of a package is broken up into several tasks (similar to Gentoo) and once the do_fetch task has ended there shouldn't be anymore network access. This is because Yocto supports users to build in no network access environments and to provide their own source cache which had previously been fetched. To best support this for Cargo built packages I call cargo fetch with a per-package CARGO_HOME set. This usually works but every now and then the sun/moon alignment upsets Cargo and it wants network access again during cargo build.

Hopefully that helps explain my use case and adds some support behind this item.

@vi
Copy link
Contributor

vi commented Dec 12, 2015

+1
Maven (build tool for Java) does have offline mode, for example.

Failure to update registry (while there is some usable cached version) should suggest an option to switch offline mode on.

Editing dependency version in Cargo.toml just to discover that network is not available, then reverting it to old version number should bring offline-buildable setup back.

In general: Internet access shouldn't always be taken for granted.

@yoshuawuyts
Copy link
Member

Having a node background, this behavior on cargo was unexpected:

  • npm install installs dependencies
  • npm install --cache-min Infinity installs dependencies from cache (slight hack)
  • npm start runs the start script

These are separate steps because there are no guarantees as to where someone might be working from. I think Cargo should be reliable, regardless if people are hacking from the artic circle or enjoying the privilege of a fiber connection.

I think cargo build being build + install is a sane default for the connected parts of the world, but there should be a way to opt-out of this so everyone's use cases can be met.

alexcrichton added a commit to alexcrichton/cargo that referenced this issue Jul 19, 2016
If a lock file is generated and some equivalent of `cargo fetch` is run then
Cargo shouldn't ever touch the network or modify `Cargo.lock` until any
`Cargo.toml` later changes, but this often wants to be asserted in some build
environments where it's a programmer error if Cargo attempts to access the
network.

The `--locked` flag added here will assert that `Cargo.lock` does not need to
change to proceed. That is, if `Cargo.lock` would be modified (as it
automatically is by default) this is turned into a hard error instead.

This `--frozen` will not only assert that `Cargo.lock` doesn't change (the same
behavior as `--locked`), but it will also will manually prevent Cargo from
touching the network by ensuring that all network requests return an error.

These flags can be used in environments where it is *expected* that no network
access happens (or no lockfile changes happen) because it has been pre-arranged
for Cargo to not happen. Examples of this include:

* CI for projects want to pass `--locked` to ensure that `Cargo.lock` is up to
  date before changes are checked in.
* Environments with vendored dependencies want to pass `--frozen` as touching
  the network indicates a programmer error that something wasn't vendored
  correctly.

A crucial property of these two flags is that **they do not change the behavior
of Cargo**. They are simply assertions at a few locations in Cargo to ensure
that actions expected to not happen indeed don't happen. Some documentation has
also been added to this effect.

Closes rust-lang#2111
bors added a commit that referenced this issue Jul 19, 2016
Add a flag to force network access to be an error

If a lock file is generated and some equivalent of `cargo fetch` is run then
Cargo shouldn't ever touch the network or modify `Cargo.lock` until any
`Cargo.toml` later changes, but this often wants to be asserted in some build
environments where it's a programmer error if Cargo attempts to access the
network.

The `--locked` flag added here will assert that `Cargo.lock` does not need to
change to proceed. That is, if `Cargo.lock` would be modified (as it
automatically is by default) this is turned into a hard error instead.

This `--frozen` will not only assert that `Cargo.lock` doesn't change (the same
behavior as `--locked`), but it will also will manually prevent Cargo from
touching the network by ensuring that all network requests return an error.

These flags can be used in environments where it is *expected* that no network
access happens (or no lockfile changes happen) because it has been pre-arranged
for Cargo to not happen. Examples of this include:

* CI for projects want to pass `--locked` to ensure that `Cargo.lock` is up to
  date before changes are checked in.
* Environments with vendored dependencies want to pass `--frozen` as touching
  the network indicates a programmer error that something wasn't vendored
  correctly.

A crucial property of these two flags is that **they do not change the behavior
of Cargo**. They are simply assertions at a few locations in Cargo to ensure
that actions expected to not happen indeed don't happen. Some documentation has
also been added to this effect.

Closes #2111
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants