[NFC] Refactor PetstoreConsumerTests to allow multiple versions #157
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
czechboy0
commented
Jul 31, 2023
czechboy0
commented
Jul 31, 2023
|
@swift-server-bot test this please |
czechboy0
changed the title
gjcairo
approved these changes
Aug 1, 2023
simonjbeaumont
approved these changes
Aug 1, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
In upcoming features that contain breaking changes, but we'll stage them in behind a feature flag, we'd like to have multiple variants of the
PetstoreConsumerTeststest target, allowing us to test both the existing behavior and the new behavior once a feature flag is enabled. These additional variants would be temporary, and would be deleted again (or rather, the main test suite would be updated) once the feature flag is enabled unconditionally. So the steady state number ofPetstoreConsumerTestvariants would continue to be 1, just for short periods of time, it might be higher.Modifications
This PR makes that possible by refactoring common functionality into a new internal target
PetstoreConsumerTestCore. Note that all the variants of the test target share the same OpenAPI document, but generate different variants of the reference code.Highlights:
PetstoreConsumerTestCoretarget, depended on by the existingPetstoreConsumerTestsignoredDiagnosticMessages: Set<String>parameter), which allows us to test both existing and new behavior on the same OpenAPI document. Other, non-allowlisted diagnostics, still continue to fail the test, so new ones won't sneak through undetected.featureFlagsandignoredDiagnosticMessages, again allowing us to test both existing and proposed behavior hidden behind a feature flagResult
It will be a lot easier to temporarily introduce other variants of the
PetstoreConsumerTeststest target to allow for thoroughly testing features even before they are enabled by default, giving us more confidence in the accuracy of proposals and their associated implementations.Test Plan
All tests continue to pass, this is an NFC, a pure refactoring, making the next PR much smaller. To see how this all fits together, check out the PR that has all the changes: #146. That said, these partial PRs are easier to review, as they're each focused on one task.