Add command submission to the ledger-api-bench-tool (basic support) [DPP-658] #11296

kamil-da · 2021-10-19T16:34:30Z

Motivation

This PR is an MVP for Command Submission support in the ledger-api-bench-tool. It is intended to make stream performance testing easier by generating contracts as defined in an input file.

Current features

generating a configurable number of contracts

Limitations

fixed command submission rate of 100cmd/s
single template available
single party available
no contract payload

CHANGELOG_BEGIN
CHANGELOG_END

Pull Request Checklist

Read and understand the contribution guidelines
Include appropriate tests
Set a descriptive title and thorough description
Add a reference to the issue this PR will solve, if appropriate
Include changelog additions in one or more commit message bodies between the CHANGELOG_BEGIN and CHANGELOG_END tags
Normal production system change, include purpose of change in description
If you mean to change the status of a component, please make sure you keep the Component Status page up to date.

NOTE: CI is not automatically run on non-members pull-requests for security
reasons. The reviewer will have to comment with /AzurePipelines run to
trigger the build.

kamil-da · 2021-10-19T16:37:02Z

ledger/ledger-api-bench-tool/src/main/scala/com/daml/ledger/api/benchtool/Benchmark.scala

+  def run(
+      config: Config,
+      apiServices: LedgerApiServices,
+  )(implicit ec: ExecutionContext, resourceContext: ResourceContext): Future[Unit] = {


Just a refactor. The logic in this class was taken directly from

daml/ledger/ledger-api-bench-tool/src/main/scala/com/daml/ledger/api/benchtool/LedgerApiBenchTool.scala

Line 52 in 9b00a1a

private def runBenchmark(config: Config)(implicit ec: ExecutionContext): Future[Unit] = {

pbatko-da · 2021-10-20T14:48:55Z

...er-api-bench-tool/src/main/scala/com/daml/ledger/api/benchtool/services/CommandService.scala

+}
+
+object CommandService {
+  def submitAndWaitRequest(


Suggested change

def submitAndWaitRequest(

def createSubmitAndWaitRequest(

?

I like to use nouns for naming pure functions. In this case, for example, it's just a request, the function doesn't do anything else (I'm not very good at explaining I think :D)
If you have a strong preference towards your suggested option, I'll change it.

submit is a verb

...dger-api-bench-tool/src/main/scala/com/daml/ledger/api/benchtool/util/SimpleFileReader.scala

...bench-tool/src/main/scala/com/daml/ledger/api/benchtool/services/LedgerIdentityService.scala

...ench-tool/src/main/scala/com/daml/ledger/api/benchtool/services/PartyManagementService.scala

...ch-tool/src/main/scala/com/daml/ledger/api/benchtool/services/PackageManagementService.scala

...api-bench-tool/src/main/scala/com/daml/ledger/api/benchtool/services/LedgerApiServices.scala

...er-api-bench-tool/src/main/scala/com/daml/ledger/api/benchtool/services/CommandService.scala

CHANGELOG_BEGIN - [Integration Kit] - The ledger-api-bench-tool is now capable of generating test contracts for testing purposes. CHANGELOG_END

mziolekda · 2021-10-22T09:21:00Z

...pi-bench-tool/src/main/scala/com/daml/ledger/api/benchtool/generating/ContractProducer.scala

+      ec: ExecutionContext
+  ): Future[Unit] = {
+    def logProgress(index: Int): Unit =
+      if (index % 100 == 0) {


We should log on a timer rather than count. But that's something for another PR.

mziolekda · 2021-10-22T09:21:54Z

...bench-tool/src/main/scala/com/daml/ledger/api/benchtool/services/LedgerIdentityService.scala

+        case Failure(exception) =>
+          Future.failed {
+            logger.error(
+              s"Error during fetching the ledger id. Details: ${exception.getLocalizedMessage}",


Suggested change

s"Error during fetching the ledger id. Details: ${exception.getLocalizedMessage}",

s"Error during fetching of the ledger id. Details: ${exception.getLocalizedMessage}",

mziolekda

Looks good, just two little nits. Feel free to address them in another PR.

@akshayshirahatti-da

This PR has been created by a script, which is not very smart and does not have all the context. Please do double-check that the version prefix is correct before merging. @akshayshirahatti-da is in charge of this release. Commit log: ``` e474b2d [JSON-API] Websockets fix for matchedQueries (#11361) 4a34b68 KV: port V2 errors to self-service errors framework [KVL-1143] (#11326) 811a6d3 Fixed AuthorizationInterceptorSpec again (#11418) c8006b8 ScenarioRunner: enrich incomplete transactions (#11384) d9c7031 ACS testing - payload support [DPP-661] (#11308) d87d3d4 deal with deadlocks while fetching contracts in json-api Oracle (#11391) 8212c0b Make submission ID optional [KVL-1107] (#11011) 3587eb8 Use Timestamp instead of Instant (#11356) ea5f09e sandbox: Deprecate the `--eager-package-loading` flag. (#11404) 9f882f2 remove search index on json fields that harm insert and pruning performance (#11041) 70b90f4 optimize max event_sequential_id query for oracle (#11297) b1fed31 Fix missing script results (#11395) 03db0aa Auto run/check security evidence generation in ./fmt.sh (#11407) c928f0e [Short] Typo (#11400) ba6c2be Add missing TransactionId to com.daml.error.ErrorResource (#11396) a2a1571 Generate security evidence by documenting security testcases (#11306) 8d17882 Allocate parties sequentially in script export tests (#11389) 1309c2f DPP-587 Use Timestamp instead of Instant (#11183) 82f9873 Rotate release rotation (#11394) b14077a Fix AuthorizationInterceptorSpec flake (#11387) 7090f2d update NOTICES file (#11367) ad42dfa Update gRPC to the latest (1.41.0) and Protobuf (#11380) 54c400a Update wording in Deploying to a generic Daml ledger (#11327) 4461ed1 Fix log output (#11374) 613aac3 Add support for non-star-kinded type synonyms in data-dependencies (#11293) f89ecc6 interfaces: add an experimental `toTypeRep` builtin. (#11378) 5654d5c fix es ingest for missing files (#11375) 03cfd12 Configurable assertions in Ledger API test tool by feature descriptors (#11328) 96b7b58 [DPP-648][Self-service error codes] Adopt ApiPartyManagementService (#11338) 9e94ae0 LF: move repl exception-auth test from dev to stable (#11369) 5365d68 LF: Remove PartialTransaction out from ScenarioRunner/IdeLedgerClient (#11368) 79037c8 [DPP-646][Self-service error codes] Adopt ApiPackageManagementService (#11314) 0ee59f5 Command submission in the ledger-api-bench-tool. (#11296) 8d5cab5 LF: Simplify seeds generation in scenario runnner (#11353) 9e5b788 Speedup daml repl integration tests (#11335) 3bc0db3 fix contract_tpid_fkey-related race condition (#11330) ab8a863 [docs] Add Daml Driver for VMBC to the commercial integrations section (#11360) c95db72 Fix Bazel cache download retry (#11238) e8d0ccb [DPP-611][Self-service error codes] Adapt ApiCommandService (#11325) a89079b [DPP-647][Self-service error codes] Adopt ApiParticipantPruningService (#11324) cc8ec28 [Self-service error codes] Adapt GrpcHealthService (#11354) c60c94b [DPP-645][Self-service error codes] Adapt ApiConfigManagementService (#11312) e6da1f7 Add step in ghc-lib guide for getting submodules to work (#11351) f3057ea Increase timeout for non-repudation tests on Postgres (#11340) 176f470 interface: adding interfaces to the TS codegen (#11280) 355352f DPP-650 Remove the mutating schema (#11211) 443b64d [DPP-621][Self-service error codes] Adopt error codes in ApiVersionService (#11302) ed9dbed interfaces: Add fixed choice collision check in typechecker (Haskell) (#11337) c37ecd1 [Short] Pass correct loggingContext to withValidatedPackageId (#11307) 0d305cf [Short] Move field before logging statement (ApiTimeService) (#11313) 73c94b5 Increase timeout for non-repudation test (#11281) 88c607b [Self-service error codes] Adapt ApiTransactionService [DPP-613] (#11094) 07ad3e0 Suport multi-party readAs in triggers (#11299) 76eb165 Interface fixed choices: ghc parser (#11275) da27a1e [DPP-619][Self-service error codes] Adopt error codes in ApiVersionService (#11303) 5f5af30 [DPP-628][Self-service error codes] Adapt error codes in ApiTimeService (#11295) f9e67ad [Self-service error codes] Adapt error responses in ledger-api-auth [DPP-617] (#11223) 7282965 Fix component status for triggers (#11311) 17776f3 Factor out npm install step of create-daml-app tests (#11294) 3a8b685 [Short] Fix docs for Dispatcher#startingAt (#11304) f315a90 release 1.18.0-snapshot.20211019.8113.0.8ff347d8 (#11300) 8a3abce [DPP-618][Self-service error codes] Adapt error codes in ApiPackageService (#11284) 50ea92f Use ApiTypes.Party instead of String in the trigger runner (#11298) 2267429 [DPP-656] Assert on self-service error code details in ErrorFactoriesSpec (#11289) c06faf2 LF: remove imperative environment from Speedy compiler (#11285) d3dad75 [DPP-592] Generate docs for self-service error codes. (#11129) ``` Changelog: ``` [JSON-API] fixes a bug related to the matchedQueries value returned for websocket multiqueries, this only happens for patterns where the multiqueries contain a mixture of queries with and without offsets. - [Integration Kit] - ledger-api-bench-tool can generate test contracts with configurable payload size. - [Integration Kit] - Added multi-template support for command submission in the ledger-api-bench-tool - [Sandbox] The ``--eager-package-loading`` flag has been deprecated. It hasn't actually done anything for many releases; packages are always loaded eagerly. This does not affect Daml on SQL, which does support lazy package loading. - [Daml Studio] Fix a bug where script results in Daml Studio sometimes do not show up. - [Integration Kit] - The ledger-api-bench-tool is now capable of generating test contracts for testing purposes. - [JSON API] Fixed a rare error that manifested as ‘violates foreign key constraint "contract_tpid_fkey" Detail: Key (tpid)=(...) is not present in table’ when attempting to run queries and goes away on JSON API restart. See `issue #11330 <https://github.com/digital-asset/daml/pull/11330>`__. - [Participant] All participants now use the new append-only schema. Existing databases will automatically upgrade to the new schema the first time a participant/ledger is started. - [Daml Triggers] Triggers now support readAs parties. They can be specified via `--ledger-readas a,b,c`. As part of this change ``testRule`` gained an extra argument to specify the `readAs` parties. If you previously used ``` testRule trigger party acsBuilder commandsInFlight s ``` you now need to use ``` testRule trigger party [] acsBuilder commandsInFlight s ``` ``` CHANGELOG_BEGIN CHANGELOG_END

@akshayshirahatti-da

This PR has been created by a script, which is not very smart and does not have all the context. Please do double-check that the version prefix is correct before merging. @akshayshirahatti-da is in charge of this release. Commit log: ``` e474b2d [JSON-API] Websockets fix for matchedQueries (#11361) 4a34b68 KV: port V2 errors to self-service errors framework [KVL-1143] (#11326) 811a6d3 Fixed AuthorizationInterceptorSpec again (#11418) c8006b8 ScenarioRunner: enrich incomplete transactions (#11384) d9c7031 ACS testing - payload support [DPP-661] (#11308) d87d3d4 deal with deadlocks while fetching contracts in json-api Oracle (#11391) 8212c0b Make submission ID optional [KVL-1107] (#11011) 3587eb8 Use Timestamp instead of Instant (#11356) ea5f09e sandbox: Deprecate the `--eager-package-loading` flag. (#11404) 9f882f2 remove search index on json fields that harm insert and pruning performance (#11041) 70b90f4 optimize max event_sequential_id query for oracle (#11297) b1fed31 Fix missing script results (#11395) 03db0aa Auto run/check security evidence generation in ./fmt.sh (#11407) c928f0e [Short] Typo (#11400) ba6c2be Add missing TransactionId to com.daml.error.ErrorResource (#11396) a2a1571 Generate security evidence by documenting security testcases (#11306) 8d17882 Allocate parties sequentially in script export tests (#11389) 1309c2f DPP-587 Use Timestamp instead of Instant (#11183) 82f9873 Rotate release rotation (#11394) b14077a Fix AuthorizationInterceptorSpec flake (#11387) 7090f2d update NOTICES file (#11367) ad42dfa Update gRPC to the latest (1.41.0) and Protobuf (#11380) 54c400a Update wording in Deploying to a generic Daml ledger (#11327) 4461ed1 Fix log output (#11374) 613aac3 Add support for non-star-kinded type synonyms in data-dependencies (#11293) f89ecc6 interfaces: add an experimental `toTypeRep` builtin. (#11378) 5654d5c fix es ingest for missing files (#11375) 03cfd12 Configurable assertions in Ledger API test tool by feature descriptors (#11328) 96b7b58 [DPP-648][Self-service error codes] Adopt ApiPartyManagementService (#11338) 9e94ae0 LF: move repl exception-auth test from dev to stable (#11369) 5365d68 LF: Remove PartialTransaction out from ScenarioRunner/IdeLedgerClient (#11368) 79037c8 [DPP-646][Self-service error codes] Adopt ApiPackageManagementService (#11314) 0ee59f5 Command submission in the ledger-api-bench-tool. (#11296) 8d5cab5 LF: Simplify seeds generation in scenario runnner (#11353) 9e5b788 Speedup daml repl integration tests (#11335) 3bc0db3 fix contract_tpid_fkey-related race condition (#11330) ab8a863 [docs] Add Daml Driver for VMBC to the commercial integrations section (#11360) c95db72 Fix Bazel cache download retry (#11238) e8d0ccb [DPP-611][Self-service error codes] Adapt ApiCommandService (#11325) a89079b [DPP-647][Self-service error codes] Adopt ApiParticipantPruningService (#11324) cc8ec28 [Self-service error codes] Adapt GrpcHealthService (#11354) c60c94b [DPP-645][Self-service error codes] Adapt ApiConfigManagementService (#11312) e6da1f7 Add step in ghc-lib guide for getting submodules to work (#11351) f3057ea Increase timeout for non-repudation tests on Postgres (#11340) 176f470 interface: adding interfaces to the TS codegen (#11280) 355352f DPP-650 Remove the mutating schema (#11211) 443b64d [DPP-621][Self-service error codes] Adopt error codes in ApiVersionService (#11302) ed9dbed interfaces: Add fixed choice collision check in typechecker (Haskell) (#11337) c37ecd1 [Short] Pass correct loggingContext to withValidatedPackageId (#11307) 0d305cf [Short] Move field before logging statement (ApiTimeService) (#11313) 73c94b5 Increase timeout for non-repudation test (#11281) 88c607b [Self-service error codes] Adapt ApiTransactionService [DPP-613] (#11094) 07ad3e0 Suport multi-party readAs in triggers (#11299) 76eb165 Interface fixed choices: ghc parser (#11275) da27a1e [DPP-619][Self-service error codes] Adopt error codes in ApiVersionService (#11303) 5f5af30 [DPP-628][Self-service error codes] Adapt error codes in ApiTimeService (#11295) f9e67ad [Self-service error codes] Adapt error responses in ledger-api-auth [DPP-617] (#11223) 7282965 Fix component status for triggers (#11311) 17776f3 Factor out npm install step of create-daml-app tests (#11294) 3a8b685 [Short] Fix docs for Dispatcher#startingAt (#11304) f315a90 release 1.18.0-snapshot.20211019.8113.0.8ff347d8 (#11300) 8a3abce [DPP-618][Self-service error codes] Adapt error codes in ApiPackageService (#11284) 50ea92f Use ApiTypes.Party instead of String in the trigger runner (#11298) 2267429 [DPP-656] Assert on self-service error code details in ErrorFactoriesSpec (#11289) c06faf2 LF: remove imperative environment from Speedy compiler (#11285) d3dad75 [DPP-592] Generate docs for self-service error codes. (#11129) ``` Changelog: ``` [JSON-API] fixes a bug related to the matchedQueries value returned for websocket multiqueries, this only happens for patterns where the multiqueries contain a mixture of queries with and without offsets. - [Integration Kit] - ledger-api-bench-tool can generate test contracts with configurable payload size. - [Integration Kit] - Added multi-template support for command submission in the ledger-api-bench-tool - [Sandbox] The ``--eager-package-loading`` flag has been deprecated. It hasn't actually done anything for many releases; packages are always loaded eagerly. This does not affect Daml on SQL, which does support lazy package loading. - [Daml Studio] Fix a bug where script results in Daml Studio sometimes do not show up. - [Integration Kit] - The ledger-api-bench-tool is now capable of generating test contracts for testing purposes. - [JSON API] Fixed a rare error that manifested as ‘violates foreign key constraint "contract_tpid_fkey" Detail: Key (tpid)=(...) is not present in table’ when attempting to run queries and goes away on JSON API restart. See `issue #11330 <https://github.com/digital-asset/daml/pull/11330>`__. - [Participant] All participants now use the new append-only schema. Existing databases will automatically upgrade to the new schema the first time a participant/ledger is started. - [Daml Triggers] Triggers now support readAs parties. They can be specified via `--ledger-readas a,b,c`. As part of this change ``testRule`` gained an extra argument to specify the `readAs` parties. If you previously used ``` testRule trigger party acsBuilder commandsInFlight s ``` you now need to use ``` testRule trigger party [] acsBuilder commandsInFlight s ``` ``` CHANGELOG_BEGIN CHANGELOG_END Co-authored-by: Azure Pipelines Daml Build <[email protected]>

kamil-da commented Oct 19, 2021

View reviewed changes

kamil-da marked this pull request as ready for review October 19, 2021 16:41

kamil-da requested a review from a team as a code owner October 19, 2021 16:41

kamil-da force-pushed the kamil-da/DPP-658-bench-tool-command-submission-mvp branch 2 times, most recently from e9ab10c to 5c714b9 Compare October 20, 2021 10:12

pbatko-da reviewed Oct 20, 2021

View reviewed changes

mziolekda reviewed Oct 21, 2021

View reviewed changes

Command submission in the ledger-api-bench-tool.

b8c4044

CHANGELOG_BEGIN - [Integration Kit] - The ledger-api-bench-tool is now capable of generating test contracts for testing purposes. CHANGELOG_END

kamil-da force-pushed the kamil-da/DPP-658-bench-tool-command-submission-mvp branch from 5c714b9 to b8c4044 Compare October 21, 2021 12:25

mziolekda reviewed Oct 22, 2021

View reviewed changes

mziolekda approved these changes Oct 22, 2021

View reviewed changes

kamil-da merged commit 0ee59f5 into main Oct 25, 2021

kamil-da deleted the kamil-da/DPP-658-bench-tool-command-submission-mvp branch October 25, 2021 07:57

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add command submission to the ledger-api-bench-tool (basic support) [DPP-658] #11296

Add command submission to the ledger-api-bench-tool (basic support) [DPP-658] #11296

kamil-da commented Oct 19, 2021 •

edited

Loading

kamil-da Oct 19, 2021 •

edited

Loading

pbatko-da Oct 20, 2021

kamil-da Oct 20, 2021

pbatko-da Oct 20, 2021 •

edited

Loading

mziolekda Oct 22, 2021

mziolekda Oct 22, 2021

mziolekda left a comment

	s"Error during fetching the ledger id. Details: ${exception.getLocalizedMessage}",
	s"Error during fetching of the ledger id. Details: ${exception.getLocalizedMessage}",

Add command submission to the ledger-api-bench-tool (basic support) [DPP-658] #11296

Add command submission to the ledger-api-bench-tool (basic support) [DPP-658] #11296

Conversation

kamil-da commented Oct 19, 2021 • edited Loading

Motivation

Current features

Limitations

Pull Request Checklist

kamil-da Oct 19, 2021 • edited Loading

Choose a reason for hiding this comment

pbatko-da Oct 20, 2021

Choose a reason for hiding this comment

kamil-da Oct 20, 2021

Choose a reason for hiding this comment

pbatko-da Oct 20, 2021 • edited Loading

Choose a reason for hiding this comment

mziolekda Oct 22, 2021

Choose a reason for hiding this comment

mziolekda Oct 22, 2021

Choose a reason for hiding this comment

mziolekda left a comment

Choose a reason for hiding this comment

kamil-da commented Oct 19, 2021 •

edited

Loading

kamil-da Oct 19, 2021 •

edited

Loading

pbatko-da Oct 20, 2021 •

edited

Loading