Add in process postgres duchy correctness tests

[YuhongWang-Amazon]

This adds in process postgres duchy correctness tests

[wfa-reviewable]

This change is 

[renjiezh]

Reviewed 22 of 26 files at r1, 5 of 5 files at r2, all commit messages.
Reviewable status: complete! all files reviewed, all discussions resolved (waiting on @YuhongWang-Amazon)

[SanjayVas]

Reviewed 16 of 26 files at r1, all commit messages.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @YuhongWang-Amazon)

---

src/main/kotlin/org/wfanet/measurement/duchy/deploy/common/service/BUILD.bazel line 33 at r2 (raw file):

)

kt_jvm_library(

If it's under deploy/common, it implies it's not specific to a given cloud. Anything specific to gcloud (e.g. Spanner) would go under deploy/gcloud.

As an aside, I've never been a fan of the "data services" interface and figure anything that needs a specific implementation of a service can call its constructor. That said, it's totally fine to keep it.

[YuhongWang-Amazon]

Reviewable status: 23 of 28 files reviewed, 1 unresolved discussion (waiting on @renjiezh and @SanjayVas)

---

src/main/kotlin/org/wfanet/measurement/duchy/deploy/common/service/BUILD.bazel line 33 at r2 (raw file):

Previously, SanjayVas (Sanjay Vasandani) wrote…

If it's under deploy/common, it implies it's not specific to a given cloud. Anything specific to gcloud (e.g. Spanner) would go under deploy/gcloud.

As an aside, I've never been a fan of the "data services" interface and figure anything that needs a specific implementation of a service can call its constructor. That said, it's totally fine to keep it.

Done.

[SanjayVas]

Reviewed 5 of 26 files at r1, 4 of 5 files at r2, 5 of 5 files at r3, all commit messages.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @YuhongWang-Amazon)

---

src/test/kotlin/org/wfanet/measurement/integration/postgres/V2PostgresInProcessLifeOfAReportIntegrationTest.kt line 37 at r3 (raw file):

/** Implementation of [InProcessLifeOfAReportIntegrationTest] for Postgres. */
class V2PostgresInProcessLifeOfAReportIntegrationTest : InProcessLifeOfAReportIntegrationTest() {

What's the V2 about, btw?

Code quote:

V2

[YuhongWang-Amazon]

Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @SanjayVas)

---

src/test/kotlin/org/wfanet/measurement/integration/postgres/V2PostgresInProcessLifeOfAReportIntegrationTest.kt line 37 at r3 (raw file):

Previously, SanjayVas (Sanjay Vasandani) wrote…

What's the V2 about, btw?

I think it means to test the v2 reporting service: https://github.com/world-federation-of-advertisers/cross-media-measurement/tree/main/src/main/kotlin/org/wfanet/measurement/reporting/deploy/v2

[SanjayVas]

Reviewable status: complete! all files reviewed, all discussions resolved (waiting on @YuhongWang-Amazon)

---

src/test/kotlin/org/wfanet/measurement/integration/postgres/V2PostgresInProcessLifeOfAReportIntegrationTest.kt line 37 at r3 (raw file):

Previously, YuhongWang-Amazon wrote…

I think it means to test the v2 reporting service: https://github.com/world-federation-of-advertisers/cross-media-measurement/tree/main/src/main/kotlin/org/wfanet/measurement/reporting/deploy/v2

Sorry, I knew that. I misread this as being a Duchy test.

[SanjayVas]

Reviewable status: all files reviewed, 1 unresolved discussion

---

src/test/kotlin/org/wfanet/measurement/integration/gcloud/GCloudPostgresInProcessLifeOfAMeasurementIntegrationTest.kt line 29 at r3 (raw file):

/**
 * Implementation of [InProcessLifeOfAMeasurementIntegrationTest] for GCloud backends with Postgres

Sorry I had missed this before. The naming and comment are inaccurate since the Kingdom still uses Spanner here. Only the Duchies were swapped to Postgres.

Code quote:

GCloud backends with Postgres

---

src/test/kotlin/org/wfanet/measurement/integration/postgres/V2PostgresInProcessLifeOfAReportIntegrationTest.kt line 46 at r3 (raw file):

    ProviderRule<
      (String, ComputationLogEntriesCoroutineStub) -> InProcessDuchy.DuchyDependencies
    > by lazy { PostgresDuchyDependencyProviderRule(ALL_DUCHY_NAMES) }

@tristanvuong2021 reported that this change made the test significantly slower.

[YuhongWang-Amazon]

Reviewable status: all files reviewed, 1 unresolved discussion

---

src/test/kotlin/org/wfanet/measurement/integration/gcloud/GCloudPostgresInProcessLifeOfAMeasurementIntegrationTest.kt line 29 at r3 (raw file):

Previously, SanjayVas (Sanjay Vasandani) wrote…

Sorry I had missed this before. The naming and comment are inaccurate since the Kingdom still uses Spanner here. Only the Duchies were swapped to Postgres.

Any suggestions for the class name? I can update the comment to point out the kingdom is still spanner.

---

src/test/kotlin/org/wfanet/measurement/integration/postgres/V2PostgresInProcessLifeOfAReportIntegrationTest.kt line 46 at r3 (raw file):

Previously, SanjayVas (Sanjay Vasandani) wrote…

@tristanvuong2021 reported that this change made the test significantly slower.

The Postgres Version of the lifeOfmeasurementTest is also slower than the spanner version. My guess is that in process postgres is slow?
I can update this the reporting integration test to use spanner duchy if needed.

[YuhongWang-Amazon]

Reviewable status: all files reviewed, 1 unresolved discussion

---

src/test/kotlin/org/wfanet/measurement/integration/postgres/V2PostgresInProcessLifeOfAReportIntegrationTest.kt line 46 at r3 (raw file):

Previously, YuhongWang-Amazon wrote…

The Postgres Version of the lifeOfmeasurementTest is also slower than the spanner version. My guess is that in process postgres is slow?
I can update this the reporting integration test to use spanner duchy if needed.

Did some tests locally, when using postgres duchy, the tests took 196s. when using spanner duchy, the tests took 110s.
@tristanvuong2021 does the above numbers match with the original performance?

[SanjayVas]

Reviewable status: all files reviewed, 1 unresolved discussion

---

src/test/kotlin/org/wfanet/measurement/integration/gcloud/GCloudPostgresInProcessLifeOfAMeasurementIntegrationTest.kt line 29 at r3 (raw file):

Previously, YuhongWang-Amazon wrote…

Any suggestions for the class name? I can update the comment to point out the kingdom is still spanner.

I'm thinking instead of having a different class name and a different test, perhaps just go back to a single test but replace one of the Duchies with a Postgres implementation.

---

src/test/kotlin/org/wfanet/measurement/integration/postgres/V2PostgresInProcessLifeOfAReportIntegrationTest.kt line 46 at r3 (raw file):

Previously, YuhongWang-Amazon wrote…

Did some tests locally, when using postgres duchy, the tests took 196s. when using spanner duchy, the tests took 110s.
@tristanvuong2021 does the above numbers match with the original performance?

We know for sure that the embedded postgres implementation we use for testing has more overhead due to being a full Postgres instance running in a Docker container, as opposed to the lightweight in-memory emulator we have for Spanner. I'm not sure what the differences are in performance of the two once actually started up.

Regardless, I don't think we actually need to have made any changes here. It can just be reverted back to using the Spanner Duchy implementation for the time being.