Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ledger-api-test-tool] - Merge KVCommandDeduplicationIT and CommandDeduplicationIT #12257

Merged
merged 17 commits into from
Jan 11, 2022
Merged

[ledger-api-test-tool] - Merge KVCommandDeduplicationIT and CommandDeduplicationIT #12257

Show file tree
Hide file tree
Changes from 15 commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
3e953b3
Merge KVCommandDeduplicationIT and CommandDeduplicationIT, using the …
nicu-da Jan 4, 2022
6ba9bcf
Assert contracts only after all tests cases were successful
nicu-da Jan 4, 2022
9deee6b
Update build files to remove exclusions
nicu-da Jan 6, 2022
53f8280
Update command dedup test cases to use time interval assertions for m…
nicu-da Jan 6, 2022
6584209
Update main
nicu-da Jan 7, 2022
8532c1b
Simplify test case by comparing the deduplication duration with the i…
nicu-da Jan 7, 2022
a29e713
Remove deprecated exclusion for canton
nicu-da Jan 7, 2022
7ff106a
Do not run assertion for the interval between record times and dedupl…
nicu-da Jan 7, 2022
b30ddee
Update exclusions for new test namings
nicu-da Jan 7, 2022
780d392
Adapt exclusions for removal of KVCommandDeduplicationIT
nicu-da Jan 7, 2022
a79cc29
Exclude tests failing under load
nicu-da Jan 7, 2022
6fb7f37
Format bazel file
nicu-da Jan 10, 2022
7eae611
Exclude test from sandbox classic
nicu-da Jan 10, 2022
9fe0879
Remove empty space
nicu-da Jan 10, 2022
6f2125b
Set static time for sandbox classic test tool
nicu-da Jan 10, 2022
2eacd5b
Remove exclusions from sandbox-classic
nicu-da Jan 10, 2022
ceaa3a0
Private field
nicu-da Jan 10, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 16 additions & 6 deletions compatibility/bazel_tools/testing.bzl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -533,13 +533,24 @@ excluded_test_tool_tests = [
},
{
"start": "1.18.0",
"platform_ranges": [
{
"end": "1.18.0",
"exclusions": [
"CommandDeduplicationIT", # Latest version of the test is dependent on having the submission id populated
],
},
],
},
{
"start": "1.18.0",
"end": "2.0.0-snapshot.20220105.8777.1", # was removed in 2.0
"platform_ranges": [
{
"end": "1.18.0",
"exclusions": [
# Exclude dedup tests due to large number of changes (removed participant deduplication, switch to append-only schema, changes in deduplication duration)
"KVCommandDeduplicationIT",
"CommandDeduplicationIT", # Latest version of the test is dependent on having the submission id populated
],
},
],
Expand Down Expand Up @@ -575,14 +586,13 @@ excluded_test_tool_tests = [
"end": "1.18.0",
"exclusions": [
# Unexpected failure (StatusRuntimeException) ALREADY_EXISTS: DUPLICATE_COMMAND(10,KVComman):
"KVCommandDeduplicationIT:KVCommandDeduplicationSimpleDeduplicationMixedClients",
"CommandDeduplicationIT:DeduplicationMixedClients",
# Assertion failed: Expecting completion with status code OK but completion has status Some(Status(6,DUPLICATE_COMMAND(10,972fae4b)
"KVCommandDeduplicationIT:KVCommandDeduplicationSimpleDeduplicationBasic",
"CommandDeduplicationIT:SimpleDeduplicationBasic",
# Unexpected failure (StatusRuntimeException) ALREADY_EXISTS: DUPLICATE_COMMAND(10,KVComman):
"KVCommandDeduplicationIT:KVCommandDeduplicationSimpleDeduplicationCommandClient",
"CommandDeduplicationIT:SimpleDeduplicationCommandClient",
# Offsets are not supported for versions < 2.0.0
"KVCommandDeduplicationIT:KVCommandDeduplicationDeduplicateUsingOffsets",
"CommandDeduplicationIT:ParticipantCommandDeduplicationDeduplicateUsingOffsets",
"CommandDeduplicationIT:DeduplicateUsingOffsets",
# Actual error id (INCONSISTENT) does not match expected error id (DUPLICATE_CONTRACT_KEY}
"ExceptionsIT:ExRollbackDuplicateKeyCreated",
"ExceptionsIT:ExRollbackDuplicateKeyArchived",
Expand Down
4 changes: 0 additions & 4 deletions ledger/daml-on-sql/BUILD.bazel
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -119,10 +119,6 @@ conformance_test(
"--additional=CompletionDeduplicationInfoITCommandService",
"--additional=CompletionDeduplicationInfoITCommandSubmissionService",
"--additional=ContractIdIT:Accept",
# Exclude offset command deduplication tests
# daml-on-sql uses the the sandbox server which has support only for participant side deduplication,
# which in turn only has support for durations as deduplication periods
"--exclude=CommandDeduplicationIT:ParticipantCommandDeduplicationDeduplicateUsingOffsets",
# not enforced
"--exclude=CommandDeduplicationPeriodValidationIT:DeduplicationDurationExceedsMaxDeduplicationDuration",
],
Expand Down
8 changes: 3 additions & 5 deletions ledger/ledger-api-test-tool-on-canton/BUILD.bazel
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,16 +88,14 @@ conformance_test(
",ContractKeysIT:CKFetchOrLookup,ContractKeysIT:CKNoFetchUndisclosed,ContractKeysIT:CKMaintainerScoped" +
",ExceptionsIT,ExceptionRaceConditionIT" + # need UCK mode - added below
",DeeplyNestedValueIT" + # FIXME: Too deeply nested values flake with a time out (half of the time)
",CommandDeduplicationIT:ParticipantCommandDeduplicationSimpleDeduplicationBasic" + # sync vs async error (part of canton #6301)
",CommandDeduplicationIT:ParticipantCommandDeduplicationDeduplicateSubmitterBasic" +
",CommandDeduplicationIT:ParticipantCommandDeduplicationSimpleDeduplicationMixedClients" +
",CommandDeduplicationIT:ParticipantCommandDeduplicationDeduplicateUsingOffsets" + # temporary disabled until we adapt it for canton
",CommandDeduplicationIT:SimpleDeduplicationBasic" + # sync vs async error (part of canton #6301)
",CommandDeduplicationIT:DeduplicateSubmitterBasic" +
",CommandDeduplicationIT:DeduplicateUsingOffsets" + # temporary disabled until we adapt it for canton
# Also exclude "optional tests" - which are run separately below
",CompletionDeduplicationInfoITCommandService" +
",CompletionDeduplicationInfoITCommandSubmissionService" +
",CommandDeduplicationParallelIT" +
",ContractIdIT" +
",KVCommandDeduplicationIT" + # only for KV-utils
",MultiPartySubmissionIT" +
",ParticipantPruningIT" +
",TLSOnePointThreeIT" +
Expand Down
9 changes: 5 additions & 4 deletions ledger/ledger-api-test-tool/BUILD.bazel
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -129,17 +129,19 @@ da_scala_binary(
"//ledger/error",
"//ledger/ledger-api-common",
"//ledger/ledger-resources",
"//libs-scala/resources",
"//libs-scala/resources-akka",
"//libs-scala/resources-grpc",
"//ledger/test-common:test-common-%s" % lf_version,
"//ledger/test-common:package_management-tests-%s.scala" % lf_version,
"//ledger/test-common:model-tests-%s.scala" % lf_version,
"//ledger/test-common:performance-tests-%s.scala" % lf_version,
"//ledger/test-common:semantic-tests-%s.scala" % lf_version,
"//ledger/test-common:dar-files-%s-lib" % lf_version,
"//libs-scala/contextualized-logging",
"//libs-scala/grpc-utils",
"//libs-scala/resources",
"//libs-scala/resources-akka",
"//libs-scala/resources-grpc",
"//libs-scala/timer-utils",
"@maven//:com_google_api_grpc_proto_google_common_protos",
"@maven//:io_grpc_grpc_api",
"@maven//:io_grpc_grpc_context",
"@maven//:io_grpc_grpc_netty",
Expand Down Expand Up @@ -256,7 +258,6 @@ conformance_test(
"--additional=LotsOfPartiesIT",
"--additional=TransactionScaleIT",
"--additional=TLSOnePointThreeIT",
"--exclude=CommandDeduplicationIT",
# Makes sure that deprecated CLI options can still be used to make sure existing CI pipelines are not broken.
# This test should fail if any deprecated CLI option has any effect whatsoever -- they are preserved
# exclusively for backwards-compatibility.
Expand Down
15 changes: 0 additions & 15 deletions ledger/ledger-api-test-tool/README.md
View file
Open in desktop

This file was deleted.

7 changes: 1 addition & 6 deletions .../ledger-api-test-tool/src/main/scala/com/daml/ledger/api/testtool/LedgerApiTestTool.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -92,14 +92,9 @@ object LedgerApiTestTool {

val defaultTests: Vector[LedgerTestSuite] = Tests.default(
timeoutScaleFactor = config.timeoutScaleFactor,
ledgerClockGranularity = config.ledgerClockGranularity,
staticTime = config.staticTime,
)
val optionalTests: Vector[LedgerTestSuite] = Tests.optional(
timeoutScaleFactor = config.timeoutScaleFactor,
ledgerClockGranularity = config.ledgerClockGranularity,
staticTime = config.staticTime,
)
val optionalTests: Vector[LedgerTestSuite] = Tests.optional()
Comment on lines 93 to +97
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than allowing default parameters for these methods (which only makes sense because of a test, which is my fault), can we remove the parameters and inline the default values into NamesSpec?

val visibleTests: Vector[LedgerTestSuite] = defaultTests ++ optionalTests
val allTests: Vector[LedgerTestSuite] = visibleTests ++ Tests.retired
val allTestCaseNames: Set[String] = allTests.flatMap(_.tests).map(_.name).toSet
Expand Down
10 changes: 7 additions & 3 deletions ...st-tool/src/main/scala/com/daml/ledger/api/testtool/infrastructure/FutureAssertions.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -123,22 +123,26 @@ object FutureAssertions {

def forAllParallel[T](
data: Seq[T]
)(testCase: T => Future[Unit])(implicit ec: ExecutionContext): Future[Seq[Unit]] = Future
)(
testCase: T => Future[Unit]
)(implicit ec: ExecutionContext, loggingContext: LoggingContext): Future[Seq[Unit]] = Future
.traverse(data)(input =>
testCase(input).map(Right(_)).recover { case NonFatal(ex) =>
Left(input -> ex)
}
)
.map { results =>
val (failures, successes) = results.partitionMap(identity)
if (failures.nonEmpty)
if (failures.nonEmpty) {
failures
.foreach(res => logger.error(s"Failed parallel test case for input ${res._1}", res._2))
throw ParallelTestFailureException(
s"Failed parallel test case. Failures: ${failures.length}. Success: ${successes.length}\nFailed inputs: ${failures
.map(_._1)
.mkString("[", ",", "]")}",
failures.last._2,
)
else successes
} else successes
}

def optionalAssertion(runs: Boolean, description: String)(
Expand Down
Loading