Update waitUntilCondition to use a watcher instead of polling.

[bbeaudreault]

There were a couple different implementations of waitUntil*, and this PR unifies all of them on an approach that somewhat mirrors the native go watcher code. I couldn't fully mirror that approach because of the typing we do in the WatchConnectionManager and elsewhere. I think this is a good incremental step towards that, and we can do any necessary refactoring in those manager classes in another PR down the line if necessary.

When you call waitUntilCondition or waitUntilReady, the code will first just try to fetch the item once from the server. If the item matches the condition it returns early. Otherwise it proceeds to create a watcher. Any errors in the watcher will cause a retry, except HTTP_GONE which will propagate. We will retry with backoff until the timeout is met.

I wanted to make the backoff configurable, so I added the plumbing necessary to support a withWaitRetryBackoff method. It defaults to 5ms initial backoff with 2x exponential backoff growth.

Fixes #2204

[centos-ci]

Can one of the admins verify this patch?

[rohanKanojia]

ok to test

[bbeaudreault]

This is ready for review. SonarCloud failed again due to coverage, but I think there is reasonably good coverage for the code I changed. I added a bunch of unit tests for WaitForConditionWatcher, and a bunch of mock tests in kubernetes-test for the new functionality of waitUntilCondition.

[rohanKanojia]

@geoand : Hi, Could you please review this PR whenever you find some time? I remember you contributed initial implementation for waitUntilCondition in #1227

[geoand]

It's been a long time since I looked at this code, but the change seems reasonable to me

[manusa]

Besides the nitpicking comment, looks great to me. Thx for contributing!

[rohanKanojia]

@bbeaudreault : There seems to some compilation error. Could you please address them whenever you get time?

[sonarqubecloud]

SonarCloud Quality Gate failed.

0 Bugs
0 Vulnerabilities (and 0 Security Hotspots to review)
20 Code Smells

17.6% Coverage
0.0% Duplication

[bbeaudreault]

Ok everything is working again. Something had merged into master which turned one of my function calls ambiguous.

[manusa]

[merge]

[bbeaudreault]

@manusa looks like the merge build failed:

Failed to execute goal org.jacoco:jacoco-maven-plugin:0.8.4:report (default-report) on project kubernetes-karaf-itests: An error has occurred in JaCoCo report generation.: Error while creating report: null: EOFException -> [Help 1]

[rohanKanojia]

[merge]

[manusa]

@manusa looks like the merge build failed:

Failed to execute goal org.jacoco:jacoco-maven-plugin:0.8.4:report (default-report) on project kubernetes-karaf-itests: An error has occurred in JaCoCo report generation.: Error while creating report: null: EOFException -> [Help 1]

It happens sometimes, we need to fix it.

Retry has been started ;)

[bbeaudreault]

Great, thanks!