Fix unknown collections of obj

[VenelinMartinov]

This changes the bridge to correctly return unknowns for objects and collections in sdkv2.

fixes #1885
fixes #2032

stacked on #2158

Confirmed that TF sdkv2 supports both unknown blocks and unknown collections of blocks, so we should be fine to pass these into TF providers.

The TF sdkv1 does not support unknowns for blocks and collections so we keep the old behaviour of passing empty/ collection of unknown.

provider "google" {
  region       = "us-central1"
}

resource "google_storage_bucket" "bucket" {
  name     = "example-bucket12312322312"
  location = "US"
}

resource "google_storage_bucket" "bucket1" {
  name     = "example-bucket123123223"
  location = "US"
  dynamic "lifecycle_rule" {
    for_each = google_storage_bucket.bucket.effective_labels
    content {
      action {
        type = lifecycle_rule.value
      }
      condition {
        age = 1
      }
    }
  }
}

This returns "lifecycle_rule":cty.UnknownVal(cty.List(cty.Object))

Our handling of collections containing unknowns and unknown collections is significantly better:

Unknown collections:
before:

+ prov:index/test:Test: (create)
  [urn=urn:pulumi:test::test::prov:index/test:Test::mainRes]
  tests     : [
      [0]: {}
  ]

after:

+ prov:index/test:Test: (create)
  [urn=urn:pulumi:test::test::prov:index/test:Test::mainRes]
  tests     : output<string>

Note that the array being output as an output<string> is an engine limitation.

Nested unknown collections:
before:

+ prov:index/nestedTest:NestedTest: (create)
  [urn=urn:pulumi:test::test::prov:index/nestedTest:NestedTest::mainRes]
  tests     : [
      [0]: {
          nestedProps: [
              [0]: {
                  testProps : [
                      [0]: output<string>
                  ]
              }
          ]
      }
  ]

after:

+ prov:index/nestedTest:NestedTest: (create)
    [urn=urn:pulumi:test::test::prov:index/nestedTest:NestedTest::mainRes]
    tests     : [
        [0]: {
            nestedProps: [
                [0]: {
                    testProps : output<string>
                }
            ]
        }
    ]

The unknown was being put one level lower than it should be.

Quite a few other very wrong outputs in #2140, including diff duplications, missing diffs etc.

[codecov]

Codecov Report

Attention: Patch coverage is 33.93939% with 109 lines in your changes missing coverage. Please review.

Project coverage is 59.83%. Comparing base (4ae2ad6) to head (ddc6c06).
Report is 1 commits behind head on master.

Files	Patch %	Lines
...g/tfbridge/internal/schemaconvert/schemaconvert.go	0.00%	86 Missing ⚠️
pkg/tests/internal/pulcheck/pulcheck.go	52.38%	7 Missing and 3 partials ⚠️
pf/internal/schemashim/provider.go	0.00%	2 Missing ⚠️
pf/proto/unsupported.go	0.00%	2 Missing ⚠️
pkg/tfshim/schema/provider.go	0.00%	2 Missing ⚠️
pkg/tfshim/tfplugin5/provider.go	0.00%	2 Missing ⚠️
pkg/tfshim/util/filter.go	0.00%	2 Missing ⚠️
pkg/tfshim/util/util.go	0.00%	2 Missing ⚠️
pkg/tfbridge/schema.go	94.11%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2061      +/-   ##
==========================================
- Coverage   60.74%   59.83%   -0.92%     
==========================================
  Files         353      354       +1     
  Lines       46146    46183      +37     
==========================================
- Hits        28032    27633     -399     
- Misses      16559    17002     +443     
+ Partials     1555     1548       -7     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[VenelinMartinov]

These tests were not failing last week, will take a look.

[VenelinMartinov]

I've adapted this to keep the old behaviour for sdkv1 and make unknowns better in sdkv2

[iwahbe]

The for loop doesn't do anything here. It only has conditional continue statements. If we don't need to check the value of fields, we should remove the for loop and just return res.UpdateContext == nil. Otherwise we need to make sure that resNeedsUpdate is under test for whatever functionality it supports.

[pulumi-bot]

This PR has been shipped in release v3.88.0.