[ledger-api-test-tool] - Merge KVCommandDeduplicationIT and CommandDeduplicationIT #12257

nicu-da · 2022-01-04T15:18:28Z

The only updated and not moved test case is DeduplicationMixedClients

For participant deduplication we do not run assertion for the interval between record times and deduplication duration when participant deduplication is in use. This is done because participant deduplication is based on the submittedAt timestamp and not record time, therefore the delta between record times can be smaller than the deduplication duration

Pull Request Checklist

Read and understand the contribution guidelines
Include appropriate tests
Set a descriptive title and thorough description
Add a reference to the issue this PR will solve, if appropriate
Include changelog additions in one or more commit message bodies between the CHANGELOG_BEGIN and CHANGELOG_END tags
Normal production system change, include purpose of change in description
If you mean to change the status of a component, please make sure you keep the Component Status page up to date.

NOTE: CI is not automatically run on non-members pull-requests for security
reasons. The reviewer will have to comment with /AzurePipelines run to
trigger the build.

ghost

Decided to do an interim review. I have a couple of minor comments.

ghost · 2022-01-04T15:24:28Z

ledger/ledger-api-test-tool/src/main/scala/com/daml/ledger/api/testtool/LedgerApiTestTool.scala

    val defaultTests: Vector[LedgerTestSuite] = Tests.default(
      timeoutScaleFactor = config.timeoutScaleFactor,
-      ledgerClockGranularity = config.ledgerClockGranularity,
-      staticTime = config.staticTime,
-    )
-    val optionalTests: Vector[LedgerTestSuite] = Tests.optional(
-      timeoutScaleFactor = config.timeoutScaleFactor,
-      ledgerClockGranularity = config.ledgerClockGranularity,
      staticTime = config.staticTime,
    )
+    val optionalTests: Vector[LedgerTestSuite] = Tests.optional()


Rather than allowing default parameters for these methods (which only makes sense because of a test, which is my fault), can we remove the parameters and inline the default values into NamesSpec?

ghost · 2022-01-04T15:26:07Z

...pi-test-tool/src/main/scala/com/daml/ledger/api/testtool/suites/CommandDeduplicationIT.scala

+  val deduplicationDuration: FiniteDuration = scaledDuration(3.seconds)
+
+  test(
+    s"SimpleDeduplicationBasic",


No need for string interpolation in test names any more.

ghost · 2022-01-04T15:27:46Z

...pi-test-tool/src/main/scala/com/daml/ledger/api/testtool/suites/CommandDeduplicationIT.scala

+        _.commands.deduplicationPeriod :=
+          DeduplicationPeriod.DeduplicationDuration(deduplicationDuration.asProtobuf)
+      )
+    val acceptedSubmissionId1 = newSubmissionId()


What happened to the second accepted submission?

I simplified the test cases as they had a lot of overlap. We now have one more comprehensive test case DeduplicateMixedClients which also handles the second submission.

nicu-da · 2022-01-07T11:39:09Z

...pi-test-tool/src/main/scala/com/daml/ledger/api/testtool/suites/CommandDeduplicationIT.scala

+  )(implicit ec =>
+    configuredParticipants => { case Participants(Participant(ledger, parties @ _*)) =>
+      def generateVariations(elements: List[List[Boolean]]): List[List[Boolean]] =
+        elements match {
+          case Nil => List(Nil)
+          case currentElement :: tail =>
+            currentElement.flatMap(value => generateVariations(tail).map(value :: _))
+        }
+
+      runWithTimeModel(configuredParticipants) { delay =>
+        val numberOfCalls = 4
+        // cover all the different generated variations of submit and submitAndWait
+        val allGeneratedVariations =
+          generateVariations(List.fill(numberOfCalls)(List(true, false))).zip(parties)
+        forAllParallel(allGeneratedVariations) {
+          case (firstCall :: secondCall :: thirdCall :: fourthCall :: Nil, party) =>
+            mixedClientsCommandDeduplicationTestCase(ledger, party, delay)(
+              firstCall,
+              secondCall,
+              thirdCall,
+              fourthCall,
+            )
+          case _ => throw new IllegalArgumentException("Wrong call list constructed")
+        }


This is the only test case which was actually updated and not just moved.

nicu-da · 2022-01-07T11:40:39Z

...pi-test-tool/src/main/scala/com/daml/ledger/api/testtool/suites/CommandDeduplicationIT.scala

+      response1 <- submitAndAssertAccepted(firstCall)
+      completion <- succeedsUntil(
+        succeedDuration = deduplicationDuration,
+        succeedDeadline = Some(
+          response1.recordTime.plusMillis(deduplicationDuration.toMillis)
+        ), // account for any network latency
+      ) {
+        submitAndAssertDeduplicated(
+          secondCall,
+          LedgerString.assertFromString(response1.completion.submissionId),
+          response1.offset,
+        )
+      }
+      deduplicationDurationFromPeriod = completion
+        .map(_.deduplicationPeriod)
+        .map {
+          case CompletionDeduplicationPeriod.Empty =>
+            throw new IllegalStateException("received empty completion")
+          case CompletionDeduplicationPeriod.DeduplicationOffset(_) =>
+            deduplicationDuration
+          case CompletionDeduplicationPeriod.DeduplicationDuration(value) =>
+            value.asScala
+        }
+        .getOrElse(deduplicationDuration + delay.skews)
+        .asInstanceOf[FiniteDuration]
+      // deduplication duration is respected
+      _ <- optionalAssertion(
+        staticTime || deduplicationDurationFromPeriod <= scaledDuration(15.seconds),
+        "The effective deduplication duration is too high to run the optional assertions",
+      ) {
+        for {
+          response2 <- succeedsEventually(
+            maxRetryDuration = deduplicationDurationFromPeriod,
+            description =
+              s"Deduplication period expires and request is accepted for command ${submitRequest.getCommands}.",
+          ) {
+            submitAndAssertAccepted(thirdCall)
+          }
+          _ <- submitAndAssertDeduplicated(
+            fourthCall,
+            LedgerString.assertFromString(response2.completion.submissionId),
+            response2.offset,
+          )
+        } yield {}
+      }.flatMap { _ =>
+        assertPartyHasActiveContracts(
+          ledger,
+          party = party,
+          noOfActiveContracts = 2,


This changes are the bulk of the test case changes

tudor-da · 2022-01-07T14:11:16Z

I see in the description Use feature descriptors to handle any specific behavior but it doesn't seem to contain any new feature descriptors or am I missing something?

nicu-da · 2022-01-07T14:13:43Z

I see in the description Use feature descriptors to handle any specific behavior but it doesn't seem to contain any new feature descriptors or am I missing something?

You are correct, the feature descriptors were added in other PRs..updated the description

…feature descriptors to handle any specific behavior changelog_begin changelog_end

…ore flexibilty accord to the behavior that a ledger can have.

…nterval between two accepted comands

…ication duration when participant deduplication is in use. This is done because participant deduplication is based on the submittedAt timestamp and not record time, therefore the delta between record times can be smaller than the deduplication duration

fabiotudone-da

For participant deduplication we do not run assertion for the interval between record times and deduplication duration when participant deduplication is in use. This is done because participant deduplication is based on the submittedAt timestamp and not record time, therefore the delta between record times can be smaller than the deduplication duration

Is there another assertion that can be used in that case?

fabiotudone-da · 2022-01-10T15:12:37Z

ledger/sandbox-classic/BUILD.bazel

@@ -444,15 +434,12 @@ server_conformance_test(
        "--concurrent-test-runs=1",  # sandbox classic doesn't scale well with concurrent tests (almost no effect on overall run time)
        "--timeout-scale-factor=2",  # sandbox classic is slow in general
        "--open-world",
+        "--static-time",


How come it did work without this?

Because sandbox-classic uses only participant deduplication and that always uses wall clock

Then why is it needed now?

The DeduplicatingMixedClients test case no longer runs in static time so we would either exclude it or actually be consistent and set the static time flag.

This will be removed shortly as we are adding the static time as a feature descriptor.

fabiotudone-da · 2022-01-10T15:14:15Z

ledger/sandbox-classic/BUILD.bazel

+        # This test case runs multiple concurrent tests, which don't work well with h2
+        "--exclude=CommandDeduplicationIT:DeduplicationMixedClients",


Is this a known issue? Do we know why?

By magic, it now runs without issue.

The issue was that the tests were a lot slower to run, and because we were running them in parallel it was a race condition, sometimes test cases waited during execution for more than the deduplication duration.

Afaik there's an H2 upgrade in progress on main, my suspicion is that somehow triggered that behavior. I will keep an eye on it.

hubert-da

LGTM

hubert-da · 2022-01-10T16:04:35Z

...n/scala/com/daml/ledger/api/testtool/infrastructure/participant/ParticipantTestContext.scala

@@ -798,3 +799,5 @@ private[testtool] final class ParticipantTestContext private[participant] (
  private def reservePartyNames(n: Int): Future[Vector[Party]] =
    Future.successful(Vector.fill(n)(Party(nextPartyHintId())))
 }
+
+case class CompletionResponse(completion: Completion, offset: LedgerOffset, recordTime: Instant)


hubert-da · 2022-01-10T16:11:10Z

...pi-test-tool/src/main/scala/com/daml/ledger/api/testtool/suites/CommandDeduplicationIT.scala

+
+  private[this] val logger: Logger = LoggerFactory.getLogger(getClass.getName)
+  private implicit val loggingContext: LoggingContext = LoggingContext.ForTesting
+  val deduplicationDuration: FiniteDuration = scaledDuration(2.seconds)


Nit: Does it need to be public? If so, I would move it to the companion object and use the constant naming convention.

You are correct, this was kept from the merge. Will make it private. We can't move it because it has to be scaled and that needs the param from the constructor.

@garyverhaegen-da

This PR has been created by a script, which is not very smart and does not have all the context. Please do double-check that the version prefix is correct before merging. @garyverhaegen-da is in charge of this release. Commit log: ``` ba2af23 Drop kvutils export (#12359) c6e5fc8 [Sandbox-on-X] ConflictCheckWithCommittedSpec (#12358) 62b61c0 ledger api auth: polish error messages (#12350) 31bc1a7 Use nanos for deduplication duration comparison [KVL-1218] (#12360) 8bc5ff0 Clarify unhandled exception error message (#12353) e5dcea1 design guidelines for JSON API endpoints (#12340) 606019e local import cleanup in json-api (#12335) ba422bf resources: Add more test coverage for parallel acquisition. (#12357) 30300fd LF: Revert #7742 (#12354) 631aa5c Create a new split release (#12351) 8a2449b TagWithLedgerEndSpec (#12346) b845587 Make create-daml-app setup idempotent (#12320) 98b7ad7 [SoX unit testing] Refactored for testing [DPP-841] (#12337) a7c51ef version-service - Move command deduplication feature descriptors into the experimental features [kvl-1218] (#12318) 4211557 Add string interning benchmark (#12237) fafb86b Add static time feature flag and test tool coverage [DPP-849] (#12336) 7ec55d7 LF: Cache field index inside structural record projection builtin (#12338) 4deee3f Add get & delete user endpoints (#12332) 79428bb Add artifactory & SDK EE artifacts to split-release process (#12345) e4dbde3 [JSON-API] Add list users endpoint (#12323) 50deee3 [ledger-api-test-tool] - Merge KVCommandDeduplicationIT and CommandDeduplicationIT (#12257) c7c86d2 LF: Benchmark for structural record projection (#12329) d653ca9 Release SDK 1.18.1 RC (#12339) c8753b5 Switch to H2 BINARY LARGE OBJECT to work around a limitation with BYTEA (#12334) fdee5cb update NOTICES file (#12342) 50de6e3 [JSON-API] HOCON config json api (#12236) 9f5a2f9 Fix terraform (#12333) face7d3 Remove unused line as reported in #12325 (#12331) 37680be Add sha256sum to dev-env (#12328) 83d2d98 Release SDK v2.0.0-snapshot.20220110.8812.0.3a08380b (#12327) ca33990 Use canton port file in daml start and tests. (#12324) c8bbeed Switch to daml-data gcs bucket for split-releases (#12322) ``` Changelog: ``` - [HTTP-JSON] Added endpoints: - /user/delete that if called with GET will delete the current user & with POST will delete the user specified via the payload - /user that if called with POST will now return user info about the user specified via the payload - [HTTP-JSON] Added an endpoint /users which returns the available users on the ledger. [ledger-api-test-tool] - Merge command deduplication ledger API tests (`KVCommandDeduplicationIT`, `CommandDeduplicationIT`) into a single suite(`CommandDeduplicationIT`) which uses feature descriptors to handle different participant behaviors JSON-API can now be started supplying a HOCON application config file using the `--config` option. All CLI flags except `logging` and `non-repudiation` one's are now deprecated and will be cleaned up in some future releases. ``` CHANGELOG_BEGIN CHANGELOG_END

@garyverhaegen-da

This PR has been created by a script, which is not very smart and does not have all the context. Please do double-check that the version prefix is correct before merging. @garyverhaegen-da is in charge of this release. Commit log: ``` d0813e6 Factor out version check in docs cron (#12459) ca3b11c Upgrade H2 to v2.1.210 (#12461) eb2281d Move more docs post processing to non-sphinx docs (#12463) 83b99ba [ledger-api-test-tool] - Add test cases for invalid deduplication periods [KVL-1222] (#12434) 995c8bd Unpin the `self-signed` resolution (#12446) 01b0b76 Support split releases in the docs cronjob (#12456) 4fe6e53 [User management] Persistence with caching (#12344) f07791e Upgrade typescript and typedoc (#12457) 2640bc6 user management: require appropriate scope in user access tokens (#12428) 0c13a4f Error handling for User Management exposed via daml-script (#12416) 530509a Add sanity check in quickstart-java assistant test (#12455) 2500d4f Make quickstart-java template work with canton sandbox. (#12453) 25ddfcd LF: Add tests for evaluation order on fetches (#12411) 430f86b Make skeleton template work with canton sandbox (#12451) 4af48bb Split channel configuration from LedgerClientConfiguration (#12433) b988a3c Use --sandbox-kv in create-daml-app release tests. (#12450) 27bfd40 Split Daml cron executable into separate modules (#12449) b2a7f9e Change default sandbox to canton (#12438) ab16a54 update NOTICES file (#12447) e6fe718 trigger split-release (#12444) 69a8b67 Fix reported vulnerabilities (#12443) b996e5e Support user tokens in Daml Script over JSON API (#12368) 36a93ef try to fix paths in split-release (#12442) 5922df6 trigger split-release (#12436) 98c9461 Remove usages of `Flowable.fromFuture` and `Single.fromFuture` (#12425) 502b21f split-release: publish temp files to artifactory (#12435) bf39f48 Upgrade vsce and markdown-it (#12431) e5b400d [Sandbox-on-X unit test] PrepareSubmissionSpec (#12383) 5a7b2ba ledger-api - Rename error method to invalidDeduplicationPeriod [KVL-1222] (#12424) 553ce22 ledger-api-test-tool: Add assertions for deduplication period, use them in existing tests [KVL-1220] (#12250) 29ba8e6 Drop sandbox classic from the SDK tarball (#12410) 93aa5ab Upgrade to NodeJS 14 (#12429) a0aee0f Rename daml sandbox to daml sandbox-kv (#12394) 8baaf72 Migrate migration tests to Sandbox on X (#12407) 903f473 kvutils - Handle deduplication periods specified by offsets > ledger end [KVL-1222] (#12423) 6a3ef80 [Sandbox-on-X unit test] SequencerStateSpec [DPP-841] (#12380) 6d6b433 Bump shelljs from 0.8.4 to 0.8.5 in /language-support/ts/packages (#12426) 84d413e Release SDK 1.18.1 (#12415) 952a939 Bump follow-redirects from 1.14.0 to 1.14.7 in /navigator/frontend (#12418) d9806f2 [Sandbox-on-X unit test] SequenceSpec [DPP-841] (#12362) 0eb500d update NOTICES file (#12413) 27b8ca0 Publish Sandbox-on-X app lib jar (#12414) 5716d99 Disable printer sharing (#12408) 33bc6fe LF: Wrongly typed local contract is not a internal error (#12406) ce62491 Fix damlc warnings & compile flags (#12404) b21c960 Make Daml Script user mgmt tests order independent (#12409) 9cc2a6e Support UserManagementService on the Java bindings (#12403) ea2d4bc LF: factorize typeAndCompile in Speedy tests (#12402) 8663b7b kvutils: Remove unused proto import (#12405) 2005068 Drop `platform-version` support from `daml.yaml` (#12397) 82bdfa5 ledger-api-auth: remove SupportedJWTPayload (#12343) d04ea6c Rerelease 1.18.1 RC (#12395) f94e24b Bump follow-redirects from 1.13.3 to 1.14.7 (#12400) c027a61 Increase timeout on an integration-test query. (#12399) 846bc38 Switch ledger-api-test-tool compat tests to sandbox on x (#12393) 8d31e93 Drop unused vars from bazel-haskell-deps (#12387) e34ac20 offboarding Akshay (#12396) bb8f455 Add test coverage for validateTokenParties (#12370) 6aa9409 split-releases: gcs accounts for assembly & canton (#12373) dfe22b8 Release SDK 1.18.1 RC 2 (#12385) e9e673c Drop scenario compat helpers (#12389) b7ded08 Publish sandbox on x as a standalone JAR (#12386) 1193f5d Changes to disable multiplexing for websockets (#12384) 903df7c Bump flyway to version 8.4.1 (#12379) 8e9f993 Convert storage backend tests to AnyFlatSuite (#12278) 9513c64 Fix compat test exclusions (#12378) a90122e Fix crash when stream closed without producing an element [DPP-853] (#12363) ae547ef kvutils/app: Add tests for health checks when starting with the Runner. (#12376) bfdf6e7 kvutils/app: Create a test case for the Runner. [KVL-663] (#12374) 249ba22 Be explicit in the usage of feature descriptors (#12356) 0a8568f Attempt to make ConcurrencyLimiterSpec less flaky (#12371) 9c77a3c kvutils: Remove the DAR upload parameters from the runner. (#12366) da55bd4 Simplify sphinx -Werror (#12372) 09013eb Add eager timeouts to the polling functions used in daml start and assistant integration tests. (#12361) 8bc5804 Add feature descriptor for the support for max deduplication duration [KVL-1218] (#12355) ba2af23 Drop kvutils export (#12359) c6e5fc8 [Sandbox-on-X] ConflictCheckWithCommittedSpec (#12358) 62b61c0 ledger api auth: polish error messages (#12350) 31bc1a7 Use nanos for deduplication duration comparison [KVL-1218] (#12360) 8bc5ff0 Clarify unhandled exception error message (#12353) e5dcea1 design guidelines for JSON API endpoints (#12340) 606019e local import cleanup in json-api (#12335) ba422bf resources: Add more test coverage for parallel acquisition. (#12357) 30300fd LF: Revert #7742 (#12354) 631aa5c Create a new split release (#12351) 8a2449b TagWithLedgerEndSpec (#12346) b845587 Make create-daml-app setup idempotent (#12320) 98b7ad7 [SoX unit testing] Refactored for testing [DPP-841] (#12337) a7c51ef version-service - Move command deduplication feature descriptors into the experimental features [kvl-1218] (#12318) 4211557 Add string interning benchmark (#12237) fafb86b Add static time feature flag and test tool coverage [DPP-849] (#12336) 7ec55d7 LF: Cache field index inside structural record projection builtin (#12338) 4deee3f Add get & delete user endpoints (#12332) 79428bb Add artifactory & SDK EE artifacts to split-release process (#12345) e4dbde3 [JSON-API] Add list users endpoint (#12323) 50deee3 [ledger-api-test-tool] - Merge KVCommandDeduplicationIT and CommandDeduplicationIT (#12257) c7c86d2 LF: Benchmark for structural record projection (#12329) d653ca9 Release SDK 1.18.1 RC (#12339) c8753b5 Switch to H2 BINARY LARGE OBJECT to work around a limitation with BYTEA (#12334) fdee5cb update NOTICES file (#12342) 50de6e3 [JSON-API] HOCON config json api (#12236) 9f5a2f9 Fix terraform (#12333) face7d3 Remove unused line as reported in #12325 (#12331) 37680be Add sha256sum to dev-env (#12328) 83d2d98 Release SDK v2.0.0-snapshot.20220110.8812.0.3a08380b (#12327) ca33990 Use canton port file in daml start and tests. (#12324) c8bbeed Switch to daml-data gcs bucket for split-releases (#12322) ``` Changelog: ``` [Scala bindings] The channel configuration has been split from the LedgerClientConfiguration class. Provide the gRPC channel specific configuration separately or use a builder. The channel configuration no longer overrides the builder. [Java bindings] The Single resulting from calling several of the services exposed by the bindings are no longer blocking. For further details, see #12193. [Java bindings] User management service client added - [HTTP-JSON] Added endpoints: - /user/delete that if called with GET will delete the current user & with POST will delete the user specified via the payload - /user that if called with POST will now return user info about the user specified via the payload - [HTTP-JSON] Added an endpoint /users which returns the available users on the ledger. [ledger-api-test-tool] - Merge command deduplication ledger API tests (`KVCommandDeduplicationIT`, `CommandDeduplicationIT`) into a single suite(`CommandDeduplicationIT`) which uses feature descriptors to handle different participant behaviors JSON-API can now be started supplying a HOCON application config file using the `--config` option. All CLI flags except `logging` and `non-repudiation` one's are now deprecated and will be cleaned up in some future releases. ``` CHANGELOG_BEGIN CHANGELOG_END

@garyverhaegen-da

This PR has been created by a script, which is not very smart and does not have all the context. Please do double-check that the version prefix is correct before merging. @garyverhaegen-da is in charge of this release. Commit log: ``` d0813e6 Factor out version check in docs cron (#12459) ca3b11c Upgrade H2 to v2.1.210 (#12461) eb2281d Move more docs post processing to non-sphinx docs (#12463) 83b99ba [ledger-api-test-tool] - Add test cases for invalid deduplication periods [KVL-1222] (#12434) 995c8bd Unpin the `self-signed` resolution (#12446) 01b0b76 Support split releases in the docs cronjob (#12456) 4fe6e53 [User management] Persistence with caching (#12344) f07791e Upgrade typescript and typedoc (#12457) 2640bc6 user management: require appropriate scope in user access tokens (#12428) 0c13a4f Error handling for User Management exposed via daml-script (#12416) 530509a Add sanity check in quickstart-java assistant test (#12455) 2500d4f Make quickstart-java template work with canton sandbox. (#12453) 25ddfcd LF: Add tests for evaluation order on fetches (#12411) 430f86b Make skeleton template work with canton sandbox (#12451) 4af48bb Split channel configuration from LedgerClientConfiguration (#12433) b988a3c Use --sandbox-kv in create-daml-app release tests. (#12450) 27bfd40 Split Daml cron executable into separate modules (#12449) b2a7f9e Change default sandbox to canton (#12438) ab16a54 update NOTICES file (#12447) e6fe718 trigger split-release (#12444) 69a8b67 Fix reported vulnerabilities (#12443) b996e5e Support user tokens in Daml Script over JSON API (#12368) 36a93ef try to fix paths in split-release (#12442) 5922df6 trigger split-release (#12436) 98c9461 Remove usages of `Flowable.fromFuture` and `Single.fromFuture` (#12425) 502b21f split-release: publish temp files to artifactory (#12435) bf39f48 Upgrade vsce and markdown-it (#12431) e5b400d [Sandbox-on-X unit test] PrepareSubmissionSpec (#12383) 5a7b2ba ledger-api - Rename error method to invalidDeduplicationPeriod [KVL-1222] (#12424) 553ce22 ledger-api-test-tool: Add assertions for deduplication period, use them in existing tests [KVL-1220] (#12250) 29ba8e6 Drop sandbox classic from the SDK tarball (#12410) 93aa5ab Upgrade to NodeJS 14 (#12429) a0aee0f Rename daml sandbox to daml sandbox-kv (#12394) 8baaf72 Migrate migration tests to Sandbox on X (#12407) 903f473 kvutils - Handle deduplication periods specified by offsets > ledger end [KVL-1222] (#12423) 6a3ef80 [Sandbox-on-X unit test] SequencerStateSpec [DPP-841] (#12380) 6d6b433 Bump shelljs from 0.8.4 to 0.8.5 in /language-support/ts/packages (#12426) 84d413e Release SDK 1.18.1 (#12415) 952a939 Bump follow-redirects from 1.14.0 to 1.14.7 in /navigator/frontend (#12418) d9806f2 [Sandbox-on-X unit test] SequenceSpec [DPP-841] (#12362) 0eb500d update NOTICES file (#12413) 27b8ca0 Publish Sandbox-on-X app lib jar (#12414) 5716d99 Disable printer sharing (#12408) 33bc6fe LF: Wrongly typed local contract is not a internal error (#12406) ce62491 Fix damlc warnings & compile flags (#12404) b21c960 Make Daml Script user mgmt tests order independent (#12409) 9cc2a6e Support UserManagementService on the Java bindings (#12403) ea2d4bc LF: factorize typeAndCompile in Speedy tests (#12402) 8663b7b kvutils: Remove unused proto import (#12405) 2005068 Drop `platform-version` support from `daml.yaml` (#12397) 82bdfa5 ledger-api-auth: remove SupportedJWTPayload (#12343) d04ea6c Rerelease 1.18.1 RC (#12395) f94e24b Bump follow-redirects from 1.13.3 to 1.14.7 (#12400) c027a61 Increase timeout on an integration-test query. (#12399) 846bc38 Switch ledger-api-test-tool compat tests to sandbox on x (#12393) 8d31e93 Drop unused vars from bazel-haskell-deps (#12387) e34ac20 offboarding Akshay (#12396) bb8f455 Add test coverage for validateTokenParties (#12370) 6aa9409 split-releases: gcs accounts for assembly & canton (#12373) dfe22b8 Release SDK 1.18.1 RC 2 (#12385) e9e673c Drop scenario compat helpers (#12389) b7ded08 Publish sandbox on x as a standalone JAR (#12386) 1193f5d Changes to disable multiplexing for websockets (#12384) 903df7c Bump flyway to version 8.4.1 (#12379) 8e9f993 Convert storage backend tests to AnyFlatSuite (#12278) 9513c64 Fix compat test exclusions (#12378) a90122e Fix crash when stream closed without producing an element [DPP-853] (#12363) ae547ef kvutils/app: Add tests for health checks when starting with the Runner. (#12376) bfdf6e7 kvutils/app: Create a test case for the Runner. [KVL-663] (#12374) 249ba22 Be explicit in the usage of feature descriptors (#12356) 0a8568f Attempt to make ConcurrencyLimiterSpec less flaky (#12371) 9c77a3c kvutils: Remove the DAR upload parameters from the runner. (#12366) da55bd4 Simplify sphinx -Werror (#12372) 09013eb Add eager timeouts to the polling functions used in daml start and assistant integration tests. (#12361) 8bc5804 Add feature descriptor for the support for max deduplication duration [KVL-1218] (#12355) ba2af23 Drop kvutils export (#12359) c6e5fc8 [Sandbox-on-X] ConflictCheckWithCommittedSpec (#12358) 62b61c0 ledger api auth: polish error messages (#12350) 31bc1a7 Use nanos for deduplication duration comparison [KVL-1218] (#12360) 8bc5ff0 Clarify unhandled exception error message (#12353) e5dcea1 design guidelines for JSON API endpoints (#12340) 606019e local import cleanup in json-api (#12335) ba422bf resources: Add more test coverage for parallel acquisition. (#12357) 30300fd LF: Revert #7742 (#12354) 631aa5c Create a new split release (#12351) 8a2449b TagWithLedgerEndSpec (#12346) b845587 Make create-daml-app setup idempotent (#12320) 98b7ad7 [SoX unit testing] Refactored for testing [DPP-841] (#12337) a7c51ef version-service - Move command deduplication feature descriptors into the experimental features [kvl-1218] (#12318) 4211557 Add string interning benchmark (#12237) fafb86b Add static time feature flag and test tool coverage [DPP-849] (#12336) 7ec55d7 LF: Cache field index inside structural record projection builtin (#12338) 4deee3f Add get & delete user endpoints (#12332) 79428bb Add artifactory & SDK EE artifacts to split-release process (#12345) e4dbde3 [JSON-API] Add list users endpoint (#12323) 50deee3 [ledger-api-test-tool] - Merge KVCommandDeduplicationIT and CommandDeduplicationIT (#12257) c7c86d2 LF: Benchmark for structural record projection (#12329) d653ca9 Release SDK 1.18.1 RC (#12339) c8753b5 Switch to H2 BINARY LARGE OBJECT to work around a limitation with BYTEA (#12334) fdee5cb update NOTICES file (#12342) 50de6e3 [JSON-API] HOCON config json api (#12236) 9f5a2f9 Fix terraform (#12333) face7d3 Remove unused line as reported in #12325 (#12331) 37680be Add sha256sum to dev-env (#12328) 83d2d98 Release SDK v2.0.0-snapshot.20220110.8812.0.3a08380b (#12327) ca33990 Use canton port file in daml start and tests. (#12324) c8bbeed Switch to daml-data gcs bucket for split-releases (#12322) ``` Changelog: ``` [Scala bindings] The channel configuration has been split from the LedgerClientConfiguration class. Provide the gRPC channel specific configuration separately or use a builder. The channel configuration no longer overrides the builder. [Java bindings] The Single resulting from calling several of the services exposed by the bindings are no longer blocking. For further details, see #12193. [Java bindings] User management service client added - [HTTP-JSON] Added endpoints: - /user/delete that if called with GET will delete the current user & with POST will delete the user specified via the payload - /user that if called with POST will now return user info about the user specified via the payload - [HTTP-JSON] Added an endpoint /users which returns the available users on the ledger. [ledger-api-test-tool] - Merge command deduplication ledger API tests (`KVCommandDeduplicationIT`, `CommandDeduplicationIT`) into a single suite(`CommandDeduplicationIT`) which uses feature descriptors to handle different participant behaviors JSON-API can now be started supplying a HOCON application config file using the `--config` option. All CLI flags except `logging` and `non-repudiation` one's are now deprecated and will be cleaned up in some future releases. ``` CHANGELOG_BEGIN CHANGELOG_END Co-authored-by: Azure Pipelines Daml Build <[email protected]>

nicu-da force-pushed the nicuda/kvl-1218/merge_command_dedup_specs branch from 64582fc to ee8824a Compare January 4, 2022 15:22

ghost reviewed Jan 4, 2022

View reviewed changes

nicu-da force-pushed the nicuda/kvl-1218/merge_command_dedup_specs branch from ca05e21 to c5507d8 Compare January 7, 2022 11:36

nicu-da commented Jan 7, 2022

View reviewed changes

nicu-da marked this pull request as ready for review January 7, 2022 11:41

nicu-da requested review from stefanobaghino-da and a team as code owners January 7, 2022 11:41

nicu-da requested review from cocreature and garyverhaegen-da as code owners January 7, 2022 20:54

nicu-da force-pushed the nicuda/kvl-1218/merge_command_dedup_specs branch from 5a4a1b4 to d0be92f Compare January 7, 2022 20:57

nicu-da added 11 commits January 10, 2022 11:26

Merge KVCommandDeduplicationIT and CommandDeduplicationIT, using the …

3e953b3

…feature descriptors to handle any specific behavior changelog_begin changelog_end

Assert contracts only after all tests cases were successful

6ba9bcf

Update build files to remove exclusions

9deee6b

Update command dedup test cases to use time interval assertions for m…

53f8280

…ore flexibilty accord to the behavior that a ledger can have.

Update main

6584209

Simplify test case by comparing the deduplication duration with the i…

8532c1b

…nterval between two accepted comands

Remove deprecated exclusion for canton

a29e713

Update exclusions for new test namings

b30ddee

Adapt exclusions for removal of KVCommandDeduplicationIT

780d392

Exclude tests failing under load

a79cc29

nicu-da force-pushed the nicuda/kvl-1218/merge_command_dedup_specs branch from a658258 to a79cc29 Compare January 10, 2022 10:26

nicu-da added 4 commits January 10, 2022 11:37

Format bazel file

6fb7f37

Exclude test from sandbox classic

7eae611

Remove empty space

9fe0879

Set static time for sandbox classic test tool

6f2125b

fabiotudone-da reviewed Jan 10, 2022

View reviewed changes

hubert-da approved these changes Jan 10, 2022

View reviewed changes

nicu-da added 2 commits January 10, 2022 20:33

Remove exclusions from sandbox-classic

2eacd5b

Private field

ceaa3a0

nicu-da requested a review from fabiotudone-da January 11, 2022 08:29

fabiotudone-da approved these changes Jan 11, 2022

View reviewed changes

nicu-da merged commit 50deee3 into main Jan 11, 2022

nicu-da deleted the nicuda/kvl-1218/merge_command_dedup_specs branch January 11, 2022 09:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ledger-api-test-tool] - Merge KVCommandDeduplicationIT and CommandDeduplicationIT #12257

[ledger-api-test-tool] - Merge KVCommandDeduplicationIT and CommandDeduplicationIT #12257

nicu-da commented Jan 4, 2022 •

edited

Loading

ghost left a comment

ghost Jan 4, 2022

ghost Jan 4, 2022

ghost Jan 4, 2022

nicu-da Jan 7, 2022

nicu-da Jan 7, 2022

nicu-da Jan 7, 2022

tudor-da commented Jan 7, 2022

nicu-da commented Jan 7, 2022

fabiotudone-da left a comment

fabiotudone-da Jan 10, 2022

nicu-da Jan 10, 2022

fabiotudone-da Jan 11, 2022

nicu-da Jan 11, 2022

fabiotudone-da Jan 10, 2022

nicu-da Jan 10, 2022

nicu-da Jan 10, 2022

hubert-da left a comment

hubert-da Jan 10, 2022

hubert-da Jan 10, 2022

nicu-da Jan 10, 2022

		# This test case runs multiple concurrent tests, which don't work well with h2
		"--exclude=CommandDeduplicationIT:DeduplicationMixedClients",

[ledger-api-test-tool] - Merge KVCommandDeduplicationIT and CommandDeduplicationIT #12257

[ledger-api-test-tool] - Merge KVCommandDeduplicationIT and CommandDeduplicationIT #12257

Conversation

nicu-da commented Jan 4, 2022 • edited Loading

Pull Request Checklist

ghost left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tudor-da commented Jan 7, 2022

nicu-da commented Jan 7, 2022

fabiotudone-da left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hubert-da left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nicu-da commented Jan 4, 2022 •

edited

Loading