Plugin Framework bridge cross tests #2592
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
This change is part of the following stack: Change managed by git-spice. |
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #2592 +/- ##
==========================================
+ Coverage 62.79% 62.81% +0.01%
==========================================
Files 389 389
Lines 51938 51939 +1
==========================================
+ Hits 32615 32624 +9
+ Misses 17515 17506 -9
- Partials 1808 1809 +1 ☔ View full report in Codecov by Sentry. |
VenelinMartinov
force-pushed
the
vvm/pf_diff_tests
branch
3 times, most recently
from
6a336a1 to
a98d243
Compare
VenelinMartinov
force-pushed
the
vvm/refactor_hclwrite
branch
from
481e2e7 to
0843489
Compare
VenelinMartinov
force-pushed
the
vvm/pf_diff_tests
branch
3 times, most recently
from
20d3be3 to
93314c6
Compare
VenelinMartinov
changed the title
|
I'll follow-up with issues for the TODOs |
VenelinMartinov
force-pushed
the
vvm/pf_diff_tests
branch
from
f9f8225 to
e325bdb
Compare
Closed
iwahbe
reviewed
Nov 8, 2024
There was a problem hiding this comment.
This is a massive PR, with 27 commits. Can you give some advice on how to review?
I'm assuming the Pu stdout is collected for humans to read? Are we supposed to read it as part of review? Does it exist only so we notice if we change it? Has it been verified as correct?
t0yv0
reviewed
Nov 11, 2024
t0yv0
reviewed
Nov 11, 2024
t0yv0
reviewed
Nov 11, 2024
pkg/pf/tests/testdata/TestDetailedDiffList/attribute_no_replace/added.golden
Show resolved
Hide resolved
t0yv0
approved these changes
Nov 11, 2024
This reverts commit f285bcd.
VenelinMartinov
force-pushed
the
vvm/pf_diff_tests
branch
from
ef59d8d to
6d57a22
Compare
VenelinMartinov
added a commit
that referenced
this pull request
Nov 20, 2024
This change adds additional testing for Diff in PF for sets involving Computed. The following schemas are covered: - Set block with a Computed attribute - Set block with a Computed attribute and RequiresReplace on the non-computed attribute - Set block with a Computed attribute and RequiresReplace on the computed attribute - Set block with a Computed attribute and RequiresReplace on the block - Set block with a Computed attribute and no UseStateForUnknown on the Computed attribute For each of these we test both when the Computed attribute is specified in the program and when it is not. For each of these schemas we test the full set of scenarios added in #2592 This should give us some confidence that the changes we are making by adding Detailed Diff for PF are good for Computed properties. I will categorize all the issues here as a follow-up before merging #2629
|
This PR has been shipped in release v3.96.0. |
This was referenced Nov 22, 2024
This was referenced Dec 11, 2024
VenelinMartinov
added a commit
that referenced
this pull request
Dec 13, 2024
This is a refactor of the SDKv2 detailed diff tests for sets. Instead of recording them inline, we now run them as cross-tests and record them in separate files along with the name of the test and the input values. This mimics what we did for the PF detailed diff tests in #2592 I've gone through the recordings and believe all of them are correct.
VenelinMartinov
added a commit
that referenced
this pull request
Dec 13, 2024
This is a refactors the rest of the SDKv2 detailed diff tests. Instead of recording them inline, we now run them as cross-tests and record them in separate files along with the name of the test and the input values. This mimics what we did for the PF detailed diff tests in #2592 I've gone through the recordings and believe all of them are correct.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a test-only PR - it adds cross-tests for Diff for the plugin framework for the existing behaviour.
Each test defines a set of schemas and a set of scenarios and the full matrix of schemas X scenarios is run in cross-tests. For each of these test cases both pulumi and TF are run with the initial input then a preview is run with the second set of inputs and the outcome of the preview is compared.
We also record the stdout for both pulumi and TF for manual comparison of the preview output. The recordings also contain the inputs in order to make spot-checking them easier.
The full test plan is available in #2597
fixes #2297
fixes #2597