Reduce BWC testing to include only latest patch releases #106143
Conversation
This commit alters our backward compatibility testing logic to restrict testing to the latest patch release for any given minor version. Since our policy is not to do any kind of breaking changes in patches, most of this testing is superfluous. This change reduces the number of versions we need to test by over 70%.
mark-vieira
added
>non-issue
:Delivery/Build
|
I think this makes sense and removing the intermediate patches safes us time and money while only giving very little value. If we want to be 100% sure we do not break compatibility between patches we could probably test compatibility between patch releases when releasing the actual patch. Meaning when releasing 8.10.4 we test it is 100% compatible with 8.10.3. |
|
Tests that rarely, if ever, provide any kind of useful signal should be run less often or never (depending on the cost/risk of missing something they might have caught). I don't have the knowledge or data to say if they can be removed entirely, but it sounds like they could at least be moved to being tested weekly, if not removed entirely. Moving to 1x per week would still cut the infrastructure cost of testing these versions by about 95%. But, again, they should just be removed if they aren't useful |
# Conflicts: # .buildkite/pipelines/periodic-packaging.yml # .buildkite/pipelines/periodic.yml # .ci/bwcVersions
|
Pinging @elastic/es-delivery (Team:Delivery) |
This commit alters our backward compatibility testing logic to restrict testing to the latest patch release for any given minor version. Since our policy is not to do any kind of breaking changes in patches, most of this testing is superfluous. This change reduces the number of versions we need to test by over 70%.
(cherry picked from commit 392a759)
This commit alters our backward compatibility testing logic to restrict testing to the latest patch release for any given minor version. Since our policy is not to do any kind of breaking changes in patches, most of this testing is superfluous. This change reduces the number of versions we need to test by over 70%. (cherry picked from commit b1e4407)
(cherry picked from commit 392a759)
This commit alters our backward compatibility testing logic to restrict testing to the latest patch release for any given minor version. Since our policy is not to do any kind of breaking changes in patches, most of this testing is superfluous. This change reduces the number of versions we need to test by over 70%.