Retry download of a dependency with back off

[stipx]

Hi,

as far as I can see there is a retry mechanism in on downloads of dependencies. The thing is that there seems to be no back off strategy implemented. The retry is just done right away:

java-buildpack/lib/java_buildpack/util/cache/download_cache.rb

Lines 330 to 336 in 9d3bc19

	if (failures += 1) > FAILURE_LIMIT
	InternetAvailability.instance.available false, "Request failed: #{e.message}"
	raise e
	else
	@logger.warn { "Request failure #{failures}, retrying. Failure: #{e.message}" }
	retry
	end

It might make sense to have some kind of exponential back off there to give the download endpoint some time to recover if there are some serverside problems.

Please share your thoughts on this.

Best
stipx

[nebhale]

Seems reasonable.

[nebhale]

Any thoughts about the backoff intervals you'd see as helpful in practice?

[stipx]

I always find the exponential back off pretty useful. It reduces backpressure but also picks up small hiccups pretty fast.

The function to get the wait interval is:
sleep = min(cap, base + 2 ** attempt)

I usually use a pretty small 'base' like 5 seconds and a 'cap' of around 60 to 90 seconds.

[nebhale]

Right, I was wondering if you were thinking of a cap in the single second range or larger. Let me poll around some others and see what the consensus is and I’ll put something in.