If cache.julialang.org is down, try to download Busybox from the upstream URL

[DilumAluthge]

1. If cache.julialang.org is down, try to download Busybox from the upstream URL
2. Wrap the entire download process inside a bounded retry loop

Co-authored-by: Lilith Hafner [email protected]

[DilumAluthge]

Does anyone know how to make the commit coauthorship show up correctly?

[LilithHafner]

Even with a backup URL and retries, there is still a chance that the download fails. Given that we already have a case for busybox not being available, I think it makes sense to put the download in the try block as well as the integrity test.

[DilumAluthge]

Even with a backup URL and retries, there is still a chance that the download fails.

Yes, but we want that to lead to a CI failure, so that we notice. If downloads from the upstream URL are failing frequently enough for us to notice, then we likely need to identify a different mirror for this download.

[LilithHafner]

Is the same true for the case where success(`$busybox`) fails?

[DilumAluthge]

Is the same true for the case where success(`$busybox`) fails?

Yeah, I was just looking at that. I was thinking that for local use, maybe we allow that to fail, but on CI, we throw an error if it fails.

So e.g. we check haskey(ENV, "CI") or something like that.

[vtjnash]

Did this break CI? I am seeing:

Error During Test at C:\buildkite-agent\builds\win2k22-amdci6-5\julialang\julia-master\julia-2c969ea8c8\share\julia\test\testdefs.jl:21
  Got exception outside of a @test
  LoadError: UndefVarError: cache_output not defined
  Stacktrace:
    [1] _tryonce_download_from_cache(desired_url::String)
      @ Main.Test14Main_spawn C:\buildkite-agent\builds\win2k22-amdci6-5\julialang\julia-master\julia-2c969ea8c8\share\julia\test\spawn.jl:27
    [2] #1
      @ C:\buildkite-agent\builds\win2k22-amdci6-5\julialang\julia-master\julia-2c969ea8c8\share\julia\test\spawn.jl:42 [inlined]
    [3] (::Base.var"#90#92"{Base.var"#90#91#93"{Vector{Float64}, Nothing, Main.Test14Main_spawn.var"#1#2"{String}}})(; kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
      @ Base .\error.jl:309
    [4] #90
      @ .\error.jl:291 [inlined]
    [5] download_from_cache(desired_url::String)
      @ Main.Test14Main_spawn C:\buildkite-agent\builds\win2k22-amdci6-5\julialang\julia-master\julia-2c969ea8c8\share\julia\test\spawn.jl:45

[staticfloat]

Whoops, expected failures strike again; we didn't notice the breakage because windows testers are already failing with the clipboard error. :/