how to build with Pre-download external dependencies

[lxlmummy]

I just learned to use bazel. I've downloaded glog which gflag dependents to the directory '/home/jamescook/lab/test/new-glog/download/ '(My computer network is blocked)
Now, I run the follow command:
bazel build --distdir=/home/jamescook/lab/test/new-glog/download/ glog
then build error happens，follow is the detail info:

bazel build --distdir=/home/jamescook/lab/test/new-glog/download/ glog
INFO: Call stack for the definition of repository 'com_github_gflags_gflags' which is a http_archive (rule definition at /root/.cache/bazel/_bazel_root/4103e065900cc37ff22791b2ccc18505/external/bazel_tools/tools/build_defs/repo/http.bzl:296:16):

• /home/jamescook/lab/test/new-glog/glog-0.4.0/WORKSPACE:3:1
  WARNING: Download from https://mirror.bazel.build/github.com/gflags/gflags/archive/v2.2.2.tar.gz failed: class java.net.ConnectException Network is unreachable (connect failed)
  WARNING: Download from https://github.com/gflags/gflags/archive/v2.2.2.tar.gz failed: class java.net.ConnectException Network is unreachable (connect failed)
  ERROR: An error occurred during the fetch of repository 'com_github_gflags_gflags':
  java.io.IOException: Error downloading [https://mirror.bazel.build/github.com/gflags/gflags/archive/v2.2.2.tar.gz, https://github.com/gflags/gflags/archive/v2.2.2.tar.gz] to /root/.cache/bazel/_bazel_root/4103e065900cc37ff22791b2ccc18505/external/com_github_gflags_gflags/v2.2.2.tar.gz: Network is unreachable (connect failed)
  ERROR: /home/jamescook/lab/test/new-glog/glog-0.4.0/BUILD:5:1: //:glog depends on @com_github_gflags_gflags//:gflags in repository @com_github_gflags_gflags which failed to fetch. no such package '@com_github_gflags_gflags//': java.io.IOException: Error downloading [https://mirror.bazel.build/github.com/gflags/gflags/archive/v2.2.2.tar.gz, https://github.com/gflags/gflags/archive/v2.2.2.tar.gz] to /root/.cache/bazel/_bazel_root/4103e065900cc37ff22791b2ccc18505/external/com_github_gflags_gflags/v2.2.2.tar.gz: Network is unreachable (connect failed)
  ERROR: Analysis of target '//:glog' failed; build aborted: no such package '@com_github_gflags_gflags//': java.io.IOException: Error downloading [https://mirror.bazel.build/github.com/gflags/gflags/archive/v2.2.2.tar.gz, https://github.com/gflags/gflags/archive/v2.2.2.tar.gz] to /root/.cache/bazel/_bazel_root/4103e065900cc37ff22791b2ccc18505/external/com_github_gflags_gflags/v2.2.2.tar.gz: Network is unreachable (connect failed)
  INFO: Elapsed time: 25.645s
  INFO: 0 processes.
  FAILED: Build did NOT complete successfully (0 packages loaded, 0 targets configured)

My question is whether bazel's distdir option didn't work or whether I used it incorrectly.
If I used it wrong, how should I change it?

[jayconrod]

This seems like a general issue with Bazel, not with Gazelle. I can't say why --distdir would or would not work.

The bazel-discuss mailing list or the Bazel Slack would be better places to ask.

Most repository rules need some kind of network access to download dependencies. Tools like Gazelle require it, too. You may be able to work around a lack if network in many cases, but it's going to be a difficult experience.

[lxlmummy]

@jayconrod , issue move to bazelbuild/bazel#11123.

[sluongng]

So Im running into this issue: trying to setup --distdir on Bazel CI workers so that most of the external dependencies(repo rules download) are pre-cached locally before a CI job is run.

After a bit of digging, I was able to get most of the Bazel external dependencies work, but not the go_repository download.

Turns out, using normal Gazelle generated rule, the go_repository are tracked via version and therefore, downloaded by /cmd/fetch_repo https://github.com/bazelbuild/bazel-gazelle/tree/master/cmd/fetch_repo instead of Bazel's native ctx.download_and_extract() which respects Bazel's --distdir.

To get this to work, either we would need to (1) make fetch_repo respects --distdir, or (2) force the download to go through ctx.download_and_extract() by giving go_repository a non-empty urls https://github.com/bazelbuild/bazel-gazelle/blob/d038863ba2e096792c6bb6afca31f6514f1aeecd/internal/go_repository.bzl#L26-L36.

Out of the 2 options, I suspect (2) would be a lot easier to accomplished with a macro wrap around.