Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DPP-686][Self-service error codes] Removing default error factories #11403

Merged

Conversation

pbatko-da
Copy link
Contributor

@pbatko-da pbatko-da commented Oct 26, 2021

No description provided.

@pbatko-da pbatko-da self-assigned this Oct 26, 2021
@pbatko-da pbatko-da force-pushed the pbatko/error-codes/dpp-676-config-manag-service-validations branch from 9945c8e to 411d3ab Compare October 27, 2021 10:42
@pbatko-da pbatko-da force-pushed the pbatko/error-codes/dpp-686-remove-default-error-factories branch 3 times, most recently from a8dc765 to 1f17e88 Compare October 27, 2021 11:59
@pbatko-da pbatko-da changed the base branch from pbatko/error-codes/dpp-676-config-manag-service-validations to main October 27, 2021 12:00
@@ -52,7 +52,7 @@ class SynchronousResponse[Input, Entry, AcceptedEntry](
.runWith(Sink.head)
.recoverWith { case _: TimeoutException =>
Future.failed(
ErrorFactories.aborted("Request timed out", definiteAnswer = Some(false))
errorFactories.aborted("Request timed out", definiteAnswer = Some(false))
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fyi @tudor-da

Copy link
Contributor

@tudor-da tudor-da Nov 2, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The aborted method does not return self-service error codes, since I thought it was too generic when I read it. I think the correct error category for this one is DeadlineExceededRequestStateUnknown. Please adapt, e.g.

  • Create a new method errorFactories.isTimeoutUnknown_wasAborted and return the versioned error based on the two cases.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok

val errorLogger = new DamlContextualizedErrorLogger(logger, loggingContext, None)
promises.foreach(promise => {
val statusRuntimeException =
errorFactories.trackerFailure(msg = promiseCancellationDescription)(errorLogger)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fyi @tudor-da

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't forget to keep those comments in the right place if changing this piece.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realized I'm not sure what that comment precisely refers to.
I guess it's that _.get call.

Moving the comments right before it.

@@ -144,7 +145,7 @@ private[platform] final class LedgerBackedIndexService(
case Some(end) if begin > end =>
Source.failed(
// TODO error codes: Replace with LedgerApiErrors.ReadErrors.RequestedOffsetAfterLedgerEnd
ErrorFactories.invalidArgument(None)(
errorFactories.invalidArgument(None)(
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fyi @tudor-da

@@ -48,7 +51,7 @@ case class QueryNonPrunedImpl(storageBackend: ParameterStorageBackend) extends Q
case Some(pruningOffsetUpToInclusive) =>
// TODO error codes: Do not throw
// TODO error codes: Enable logging
throw ErrorFactories.participantPrunedDataAccessed(error(pruningOffsetUpToInclusive))(
throw errorFactories.participantPrunedDataAccessed(error(pruningOffsetUpToInclusive))(
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fyi @tudor-da

@@ -146,7 +123,7 @@ private[backend] object PostgresStorageBackend
.as(int("result").singleOpt)(connection)
.foreach(_ =>
// TODO error codes: Use specialized error and enable logging
throw ErrorFactories.invalidArgument(None)(
throw errorFactories.invalidArgument(None)(
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fyi @tudor-da

@@ -63,7 +64,7 @@ class SandboxResetService(
.cond(
ledgerId == LedgerId(request.ledgerId),
request.ledgerId,
ErrorFactories.ledgerIdMismatch(ledgerId, LedgerId(request.ledgerId), None),
errorFactories.ledgerIdMismatch(ledgerId, LedgerId(request.ledgerId), None),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fyi @tudor-da

@pbatko-da pbatko-da force-pushed the pbatko/error-codes/dpp-686-remove-default-error-factories branch 2 times, most recently from 3b81f98 to d57e451 Compare October 27, 2021 14:43
@pbatko-da pbatko-da changed the title [DPP-686][Self-service error codes] Removing default error factories [WIP] [DPP-686][Self-service error codes] Removing default error factories Oct 27, 2021
@pbatko-da pbatko-da marked this pull request as ready for review October 27, 2021 14:44
@pbatko-da pbatko-da requested review from meiersi-da and a team as code owners October 27, 2021 14:44
@pbatko-da pbatko-da force-pushed the pbatko/error-codes/dpp-686-remove-default-error-factories branch from d57e451 to 21a57ef Compare October 28, 2021 13:13
@@ -579,6 +579,13 @@ object LedgerApiErrors extends LedgerApiErrorGroup {
ErrorCategory.SystemInternalAssumptionViolated,
) {

// TODO error codes: This is an internal error not related to the interpreter
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we remove this comment? This error appears under LedgerApiErrors.InternalError

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem is the what is now called InternalError is really "internal errors related to the interpreter":

@explanation("""This error occurs if there was an unexpected error within the interpreter""")
@resolution("Contact support.")
object InternalError

Copy link
Contributor

@tudor-da tudor-da Nov 2, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah right, the explanation was wrong here. Fine, let's leave as it

Comment on lines -431 to -444
* TODO error codes: Remove default implementation once all Ledger API services
* output versioned error codes.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Finally, one removed! 🍾

@@ -29,6 +29,7 @@ da_scala_library(
deps = [
"//daml-lf/data",
"//language-support/scala/bindings",
"//ledger/ledger-api-common",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it still needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't be anymore. Weird. I thought Bazel would error out any time a dependency is not needed.
Could be some bug at play or some unknown to me behavior related to e.g. transitive dependencies?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removing

@@ -144,7 +145,7 @@ private[platform] final class LedgerBackedIndexService(
case Some(end) if begin > end =>
Source.failed(
// TODO error codes: Replace with LedgerApiErrors.ReadErrors.RequestedOffsetAfterLedgerEnd
Copy link
Contributor

@tudor-da tudor-da Oct 29, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's address this TODO:

  • Use errorFactories.offsetAfterLedgerEnd
  • Rename ErrorFactories#offsetAfterLedgerEnd to ErrorFactories#offsetOutOfRange
  • Rename LedgerApiErrors.ReadErrors.RequestedOffsetAfterLedgerEnd to LedgerApiErrors.ReadErrors.RequestedOffsetOutOfRange since each caller specializes it with the reason anyhow.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What you proposed means changing V1 (from INVALID_ARGUMENT to OUT_OF_RANGE). Is it fine?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changing

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Scratch first comment. Actually using errorFactories.readingOffsetOutOfRange_was_invalidArgument since it's already there.

)
) {
// TODO error codes: Use specialized error and enable logging
throw errorFactories.invalidArgument(None)(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you specialize the error now? See ErrorFactories#offsetOutOfRange mentioned previously.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok

// TODO error codes: Use specialized error and enable logging
throw errorFactories.invalidArgument(None)(
"Pruning offset for all divulged contracts needs to be after the migration offset"
)(NoLogging)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why no logging? We have the logging context here

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, logging it now

@@ -48,7 +51,7 @@ case class QueryNonPrunedImpl(storageBackend: ParameterStorageBackend) extends Q
case Some(pruningOffsetUpToInclusive) =>
// TODO error codes: Do not throw
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// TODO error codes: Do not throw

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So now it's ok to throw? Just clarifying

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not imho, but I don't feel strongly about this, since the change has wider impact. Let's leave it as is.

@@ -48,7 +51,7 @@ case class QueryNonPrunedImpl(storageBackend: ParameterStorageBackend) extends Q
case Some(pruningOffsetUpToInclusive) =>
// TODO error codes: Do not throw
// TODO error codes: Enable logging
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you enable logging as mentioned in the TODO, if we are at it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have no logger and loggingContext here.
Not sure what's the best way to implement it.

Choosing to modify executeSql to accept implicit loggingContext

Copy link
Contributor

@tudor-da tudor-da left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Left a few comments worth addressing

@pbatko-da pbatko-da requested a review from tudor-da November 1, 2021 20:30
errorFactories: ErrorFactories,
) extends QueryNonPruned {

private val logger = ContextualizedLogger.get(classOf[QueryNonPrunedImpl])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
private val logger = ContextualizedLogger.get(classOf[QueryNonPrunedImpl])
private val logger = ContextualizedLogger.get(getClass)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok

@@ -12,12 +12,12 @@ import com.daml.ledger.resources.{Resource, ResourceContext}
import com.daml.logging.{ContextualizedLogger, LoggingContext}
import com.daml.metrics.Metrics
import com.daml.platform.configuration.ServerRole
import com.daml.platform.store.appendonlydao.DbDispatcher
Copy link
Contributor

@tudor-da tudor-da Nov 2, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Try to use the scalafmt provided in the repo in your IntelliJ formatter. It might help reduce the noise in the changes.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, will do.

@pbatko-da pbatko-da merged commit 8a9f15b into main Nov 3, 2021
@pbatko-da pbatko-da deleted the pbatko/error-codes/dpp-686-remove-default-error-factories branch November 3, 2021 08:40
azure-pipelines bot pushed a commit that referenced this pull request Nov 10, 2021
This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.

@remyhaemmerle-da is in charge of this release.

Commit log:
```
9218116 LF: Abstract VersionedValue and VersionedContractInstance (#11533)
b87acab foldLeft, foldRight, other Foldable specializations (#11592)
92dfcde [Self-service error codes] Use error validators in ApiPackageManagementService (#11575)
6372d41 limit contract insertion/deletion batching on backpressure (#11589)
3588284 Fix screenshots in daml intro 2 (#11617)
ef3fc0e Cache type synonym expansion in safeToReexport (#11612)
70b2fe3 Provide byInterface fields for fetch, exercise. (#11607)
830779f LF: drop useless class com.daml.lf.transaction (#11601)
7c92cab Cache type synonym expansion in isDuplicate (#11611)
cfd521f Fix GHC build errors on macOS (#11560)
a7559d0 LF: Rename GenTransaction to Transaction (#11599)
3b61a1b LF: allow exercise command by template for inherited choice. (#11584)
af1bee7 precondition lf conversion (#11538)
f722cf1 Bump extractor init timeout to 60s (#11598)
1e0a221 Set symlink_node_modules = False (#11606)
ac19c8c Synchronize in multi-participant script tests (#11594)
dd1b034 Restart the submission interpretation in case of a race [DPP-737] (#11579)
8d2b1b9 [JSON-API] Port of http-perf-test to sandbox(aka next) (#11543)
56a6db8 codeowners: reflect change in onwership of meiersi-da (#11505)
d433651 fix a few typos in Daml-LF spec (#11583)
5dee88b Refactoring store factories [DPP-709] (#11572)
41fb289 Port LedgerApiServer rate limiter interface from canton (#11577)
04cc5d7 Mark JSON API integration tests as flaky (#11586)
3bed7a9 Drop unused defaultFeatureFlags (#11580)
cbc3c8c Add by_interface field in Create, Fetch, & Exercise transaction nodes. (#11576)
7d68e05 Remove virtual choices (#11482)
9b94fa9 [ledger-api] - metadata for invalid deduplication period [KVL-1170] (#11534)
e7eb60f update compat versions for 1.18.0-snapshot.20211102.8257.0.7391a3cd (#11531)
a3d04ca Migrate matches-docs tests to sandbox on postgres (#11573)
21c9745 [Self-service error codes] ValueSwitch parameterized method (#11567)
6a0b84e Run Script export tests against BR (#11574)
a3195fd fix link to blog (#11570)
0823601 Self-service errors: generate error deprecation docs [KVL-1187] (#11539)
8d48cf6 DPP-664 Create contracts in batches (#11515)
9ef2eaf Remove parameter table join from SQL queries [DPP-705] (#11562)
af7f7f3 Fix LF coedowners (#11563)
4450311 Exclude/fix compatibility tests (#11529)
699c28f rotate release duty after 1.18.0-snapshot.20211102.8257.0.7391a3cd (#11522)
4d4869f interfaces: Improve fixed choice lookup (#11551)
ba96bf7 Add DbDtoToStringsForInterning [DPP-708] (#11557)
692dac6 LF: deprecate remaining type alias in Transaction (#11554)
7bc0f82 [kvutils] - Extract command deduplication steps from the TransactionCommitter [KVL-1174] (#11547)
0e95ccb Wire LedgerEndCache to relevant StorageBackend factories [DPP-704] (#11549)
73cb42e LF: deprecate ensureNoCid and assertNoCid in CidContainer (#11502)
30bfee9 Upgrade canton for conformance tests (#11550)
b84a914 Extractor: replace `sandbox-classic`-based fixtures with `sandbox`-based ones (#11489)
d86fe92 ledger-api-bench-tool - flexible stream filters [DPP-667] (#11458)
214c7ab Self-service errors: generate groups documentation [KVL-1186] (#11526)
60dc286 Avoid unnecessarily decoding DALFs during dependency installation (#11541)
afd79b6 Improve logging for dependency installation (#11540)
fdde535 fix hoogle by pinning nixpkgs (#11548)
5e93df6 Full conformance test with self-service error codes on ledger-on-memory (#11524)
efaf20b update NOTICES file (#11546)
c7bb94b Modify contract key attribute age for createAndExercise to avoid contract key violation on slow running tests (#11530)
541d07c Add string-interning data access [DPP-707] (#11484)
6c94164 Switching to composition at StorageBackend [DPP-704] (#11520)
d006ad0 Add string-interning view primitives [DPP-702] (#11475)
a917520 Save the submission time in the command deduplication value [KVL-1173] (#11509)
669186f Add priority-queue based worker Source [DPP-717] (#11500)
2091f59 Remove cats from kvutils (#11537)
b83af37 Fix recache log statement (#11536)
776790e Add choice collisions in scala name collision checker (#11528)
2efcb21 Extract the setting of the time bounds on the commit context [KVL-1173] (#11516)
9bb1d64 [ledger-api] - Return FAILED_PRECONDITION gRPC status code for invalid deduplication duration [KVL-1170] (#11483)
c098d75 Add a --log-level flag to damlc (#11514)
afee948 Bump rules_nixpkgs (#11517)
d6fb67c release 1.18.0-snapshot.20211102.8257.0.7391a3cd (#11521)
ac49d69 Ensure that -Werror converts parser warnings into errors (#11511)
bda35ca ledger-offset: Add a conversion to ByteString. (#11491)
8a9f15b [DPP-686][Self-service error codes] Removing default error factories (#11403)
bb37eef Switch script & trigger tests to postgres-backed Sadbonx (#11525)
```
Changelog:
```
- [JSON API] While updating the contract table for a query, if the DB appears to be slow,
  JSON API will slow down its own inserts and deletes at some point rather than construct
  ever-larger INSERT and DELETE batch commands.
  See `issue #11589 <https://github.com/digital-asset/daml/pull/11589>`__.
[Ledger API] Retry the interpretation of a command in case of a race
with other transactions. This fix drastically reduces the likelihood of the error
"Could not find a suitable ledger time after 0 retries".
[ledger-api] - Return max_deduplication_duration as part of the metadata sent with the gRPC status for commands rejected because of invalid deduplication periods

- [Integration Kit] - ledger-api-bench-tool - added flexible party and template filters for transactions, transaction trees and active contract streams

[ledger-api] - Return FAILED_PRECONDITION gRPC status code, instead of INVALID_ARGUMENT, for an invalid command deduplication duration
```

CHANGELOG_BEGIN
CHANGELOG_END
remyhaemmerle-da pushed a commit that referenced this pull request Nov 10, 2021
This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.

@remyhaemmerle-da is in charge of this release.

Commit log:
```
9218116 LF: Abstract VersionedValue and VersionedContractInstance (#11533)
b87acab foldLeft, foldRight, other Foldable specializations (#11592)
92dfcde [Self-service error codes] Use error validators in ApiPackageManagementService (#11575)
6372d41 limit contract insertion/deletion batching on backpressure (#11589)
3588284 Fix screenshots in daml intro 2 (#11617)
ef3fc0e Cache type synonym expansion in safeToReexport (#11612)
70b2fe3 Provide byInterface fields for fetch, exercise. (#11607)
830779f LF: drop useless class com.daml.lf.transaction (#11601)
7c92cab Cache type synonym expansion in isDuplicate (#11611)
cfd521f Fix GHC build errors on macOS (#11560)
a7559d0 LF: Rename GenTransaction to Transaction (#11599)
3b61a1b LF: allow exercise command by template for inherited choice. (#11584)
af1bee7 precondition lf conversion (#11538)
f722cf1 Bump extractor init timeout to 60s (#11598)
1e0a221 Set symlink_node_modules = False (#11606)
ac19c8c Synchronize in multi-participant script tests (#11594)
dd1b034 Restart the submission interpretation in case of a race [DPP-737] (#11579)
8d2b1b9 [JSON-API] Port of http-perf-test to sandbox(aka next) (#11543)
56a6db8 codeowners: reflect change in onwership of meiersi-da (#11505)
d433651 fix a few typos in Daml-LF spec (#11583)
5dee88b Refactoring store factories [DPP-709] (#11572)
41fb289 Port LedgerApiServer rate limiter interface from canton (#11577)
04cc5d7 Mark JSON API integration tests as flaky (#11586)
3bed7a9 Drop unused defaultFeatureFlags (#11580)
cbc3c8c Add by_interface field in Create, Fetch, & Exercise transaction nodes. (#11576)
7d68e05 Remove virtual choices (#11482)
9b94fa9 [ledger-api] - metadata for invalid deduplication period [KVL-1170] (#11534)
e7eb60f update compat versions for 1.18.0-snapshot.20211102.8257.0.7391a3cd (#11531)
a3d04ca Migrate matches-docs tests to sandbox on postgres (#11573)
21c9745 [Self-service error codes] ValueSwitch parameterized method (#11567)
6a0b84e Run Script export tests against BR (#11574)
a3195fd fix link to blog (#11570)
0823601 Self-service errors: generate error deprecation docs [KVL-1187] (#11539)
8d48cf6 DPP-664 Create contracts in batches (#11515)
9ef2eaf Remove parameter table join from SQL queries [DPP-705] (#11562)
af7f7f3 Fix LF coedowners (#11563)
4450311 Exclude/fix compatibility tests (#11529)
699c28f rotate release duty after 1.18.0-snapshot.20211102.8257.0.7391a3cd (#11522)
4d4869f interfaces: Improve fixed choice lookup (#11551)
ba96bf7 Add DbDtoToStringsForInterning [DPP-708] (#11557)
692dac6 LF: deprecate remaining type alias in Transaction (#11554)
7bc0f82 [kvutils] - Extract command deduplication steps from the TransactionCommitter [KVL-1174] (#11547)
0e95ccb Wire LedgerEndCache to relevant StorageBackend factories [DPP-704] (#11549)
73cb42e LF: deprecate ensureNoCid and assertNoCid in CidContainer (#11502)
30bfee9 Upgrade canton for conformance tests (#11550)
b84a914 Extractor: replace `sandbox-classic`-based fixtures with `sandbox`-based ones (#11489)
d86fe92 ledger-api-bench-tool - flexible stream filters [DPP-667] (#11458)
214c7ab Self-service errors: generate groups documentation [KVL-1186] (#11526)
60dc286 Avoid unnecessarily decoding DALFs during dependency installation (#11541)
afd79b6 Improve logging for dependency installation (#11540)
fdde535 fix hoogle by pinning nixpkgs (#11548)
5e93df6 Full conformance test with self-service error codes on ledger-on-memory (#11524)
efaf20b update NOTICES file (#11546)
c7bb94b Modify contract key attribute age for createAndExercise to avoid contract key violation on slow running tests (#11530)
541d07c Add string-interning data access [DPP-707] (#11484)
6c94164 Switching to composition at StorageBackend [DPP-704] (#11520)
d006ad0 Add string-interning view primitives [DPP-702] (#11475)
a917520 Save the submission time in the command deduplication value [KVL-1173] (#11509)
669186f Add priority-queue based worker Source [DPP-717] (#11500)
2091f59 Remove cats from kvutils (#11537)
b83af37 Fix recache log statement (#11536)
776790e Add choice collisions in scala name collision checker (#11528)
2efcb21 Extract the setting of the time bounds on the commit context [KVL-1173] (#11516)
9bb1d64 [ledger-api] - Return FAILED_PRECONDITION gRPC status code for invalid deduplication duration [KVL-1170] (#11483)
c098d75 Add a --log-level flag to damlc (#11514)
afee948 Bump rules_nixpkgs (#11517)
d6fb67c release 1.18.0-snapshot.20211102.8257.0.7391a3cd (#11521)
ac49d69 Ensure that -Werror converts parser warnings into errors (#11511)
bda35ca ledger-offset: Add a conversion to ByteString. (#11491)
8a9f15b [DPP-686][Self-service error codes] Removing default error factories (#11403)
bb37eef Switch script & trigger tests to postgres-backed Sadbonx (#11525)
```
Changelog:
```
- [JSON API] While updating the contract table for a query, if the DB appears to be slow,
  JSON API will slow down its own inserts and deletes at some point rather than construct
  ever-larger INSERT and DELETE batch commands.
  See `issue #11589 <https://github.com/digital-asset/daml/pull/11589>`__.
[Ledger API] Retry the interpretation of a command in case of a race
with other transactions. This fix drastically reduces the likelihood of the error
"Could not find a suitable ledger time after 0 retries".
[ledger-api] - Return max_deduplication_duration as part of the metadata sent with the gRPC status for commands rejected because of invalid deduplication periods

- [Integration Kit] - ledger-api-bench-tool - added flexible party and template filters for transactions, transaction trees and active contract streams

[ledger-api] - Return FAILED_PRECONDITION gRPC status code, instead of INVALID_ARGUMENT, for an invalid command deduplication duration
```

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines Daml Build <[email protected]>
stefanobaghino-da added a commit that referenced this pull request Nov 11, 2021
changelog_begin
changelog_end

CHANGELOG ENTRIES

```
[TS-Bindings] Ws multiplexing for stream queries is disabled by default as we investigate issues
of intermittent websocket closures.

* There is now an Early Access Helm chart to start all of the Connect components on a Kubernetes cluster.

[JSON-API] Make certain Hikari cp connection pool properties configurable via jdbc conf string, the properties are listed below
poolSize -- specifies the max pool size for the database connection pool
minIdle -- specifies the min idle connections for database connection pool
connectionTimeout -- long value, specifies the connection timeout for database connection pool
idleTimeout -- long value, specifies the idle timeout for the database connection pool
- [Integration Kit] - ledger-api-bench-tool can run a workflow consisting of the command submission phase and stream benchmark phase.
- [JSON API] While updating the contract table for a query, if the DB appears to be slow,
  JSON API will slow down its own inserts and deletes at some point rather than construct
  ever-larger INSERT and DELETE batch commands.
  See `issue #11589 <https://github.com/digital-asset/daml/pull/11589>`__.
[Ledger API] Retry the interpretation of a command in case of a race
with other transactions. This fix drastically reduces the likelihood of the error
"Could not find a suitable ledger time after 0 retries".
[ledger-api] - Return max_deduplication_duration as part of the metadata sent with the gRPC status for commands rejected because of invalid deduplication periods

- [Integration Kit] - ledger-api-bench-tool - added flexible party and template filters for transactions, transaction trees and active contract streams
```

COMMITS

```
d938a44 ws multiplexing is disabled by default as we investigate issues with intermittent web socket closures (#11657)
0f0a865 Make string interning available at ingestion [DPP-711] (#11645)
7296ba4 Move closureConvert and validate compiler phases into separate files. (#11656)
87f282c interfaces: Preserve/provide by_interface data for create actions. (#11639)
a9de728 Update rules_scala (#11616)
f468969 document helm chart (#11648)
55d4a03 Release 1.11.3 snapshot (#11652)
2dd5289 different types for different speedy phases (#11605)
44d604d Release new 1.18 snapshot (#11642)
e69a871 [JSON-API] Configurable Hikari connection pool props  (#11621)
cb7099b release 1.11.3 snapshot (#11628)
fd61d0b String interning read integration [DPP-706] (#11638)
c5a1f0b Republish error codes generator lib (#11633)
a6f745f Add two phase retrieval merge function [DPP-718] (#11571)
743ee46 ledger-api-bench-tool - command submission and stream reading complete workflow [DPP-668] (#11544)
f8f8807 Activate interning write side [DPP-710] (#11614)
93c07f4 Extend synonym expansion caching in data-dependencies (#11629)
cf445b2 Skip platform independence tests on release PRs (#11631)
b8f384e LF: Drop typeparameter from KeyWithMaintainers (#11625)
d0c313d Share dependencyInfo in data-dependencies (#11620)
4eed1d9 release 1.18.0-snapshot.20211109.8328.0.92181161 (#11623)
9218116 LF: Abstract VersionedValue and VersionedContractInstance (#11533)
b87acab foldLeft, foldRight, other Foldable specializations (#11592)
92dfcde [Self-service error codes] Use error validators in ApiPackageManagementService (#11575)
6372d41 limit contract insertion/deletion batching on backpressure (#11589)
3588284 Fix screenshots in daml intro 2 (#11617)
ef3fc0e Cache type synonym expansion in safeToReexport (#11612)
70b2fe3 Provide byInterface fields for fetch, exercise. (#11607)
830779f LF: drop useless class com.daml.lf.transaction (#11601)
7c92cab Cache type synonym expansion in isDuplicate (#11611)
cfd521f Fix GHC build errors on macOS (#11560)
a7559d0 LF: Rename GenTransaction to Transaction (#11599)
3b61a1b LF: allow exercise command by template for inherited choice. (#11584)
af1bee7 precondition lf conversion (#11538)
f722cf1 Bump extractor init timeout to 60s (#11598)
1e0a221 Set symlink_node_modules = False (#11606)
ac19c8c Synchronize in multi-participant script tests (#11594)
dd1b034 Restart the submission interpretation in case of a race [DPP-737] (#11579)
8d2b1b9 [JSON-API] Port of http-perf-test to sandbox(aka next) (#11543)
56a6db8 codeowners: reflect change in onwership of meiersi-da (#11505)
d433651 fix a few typos in Daml-LF spec (#11583)
5dee88b Refactoring store factories [DPP-709] (#11572)
41fb289 Port LedgerApiServer rate limiter interface from canton (#11577)
04cc5d7 Mark JSON API integration tests as flaky (#11586)
3bed7a9 Drop unused defaultFeatureFlags (#11580)
cbc3c8c Add by_interface field in Create, Fetch, & Exercise transaction nodes. (#11576)
7d68e05 Remove virtual choices (#11482)
9b94fa9 [ledger-api] - metadata for invalid deduplication period [KVL-1170] (#11534)
e7eb60f update compat versions for 1.18.0-snapshot.20211102.8257.0.7391a3cd (#11531)
a3d04ca Migrate matches-docs tests to sandbox on postgres (#11573)
21c9745 [Self-service error codes] ValueSwitch parameterized method (#11567)
6a0b84e Run Script export tests against BR (#11574)
a3195fd fix link to blog (#11570)
0823601 Self-service errors: generate error deprecation docs [KVL-1187] (#11539)
8d48cf6 DPP-664 Create contracts in batches (#11515)
9ef2eaf Remove parameter table join from SQL queries [DPP-705] (#11562)
af7f7f3 Fix LF coedowners (#11563)
4450311 Exclude/fix compatibility tests (#11529)
699c28f rotate release duty after 1.18.0-snapshot.20211102.8257.0.7391a3cd (#11522)
4d4869f interfaces: Improve fixed choice lookup (#11551)
ba96bf7 Add DbDtoToStringsForInterning [DPP-708] (#11557)
692dac6 LF: deprecate remaining type alias in Transaction (#11554)
7bc0f82 [kvutils] - Extract command deduplication steps from the TransactionCommitter [KVL-1174] (#11547)
0e95ccb Wire LedgerEndCache to relevant StorageBackend factories [DPP-704] (#11549)
73cb42e LF: deprecate ensureNoCid and assertNoCid in CidContainer (#11502)
30bfee9 Upgrade canton for conformance tests (#11550)
b84a914 Extractor: replace `sandbox-classic`-based fixtures with `sandbox`-based ones (#11489)
d86fe92 ledger-api-bench-tool - flexible stream filters [DPP-667] (#11458)
214c7ab Self-service errors: generate groups documentation [KVL-1186] (#11526)
60dc286 Avoid unnecessarily decoding DALFs during dependency installation (#11541)
afd79b6 Improve logging for dependency installation (#11540)
fdde535 fix hoogle by pinning nixpkgs (#11548)
5e93df6 Full conformance test with self-service error codes on ledger-on-memory (#11524)
efaf20b update NOTICES file (#11546)
c7bb94b Modify contract key attribute age for createAndExercise to avoid contract key violation on slow running tests (#11530)
541d07c Add string-interning data access [DPP-707] (#11484)
6c94164 Switching to composition at StorageBackend [DPP-704] (#11520)
d006ad0 Add string-interning view primitives [DPP-702] (#11475)
a917520 Save the submission time in the command deduplication value [KVL-1173] (#11509)
669186f Add priority-queue based worker Source [DPP-717] (#11500)
2091f59 Remove cats from kvutils (#11537)
b83af37 Fix recache log statement (#11536)
776790e Add choice collisions in scala name collision checker (#11528)
2efcb21 Extract the setting of the time bounds on the commit context [KVL-1173] (#11516)
9bb1d64 [ledger-api] - Return FAILED_PRECONDITION gRPC status code for invalid deduplication duration [KVL-1170] (#11483)
c098d75 Add a --log-level flag to damlc (#11514)
afee948 Bump rules_nixpkgs (#11517)
d6fb67c release 1.18.0-snapshot.20211102.8257.0.7391a3cd (#11521)
ac49d69 Ensure that -Werror converts parser warnings into errors (#11511)
bda35ca ledger-offset: Add a conversion to ByteString. (#11491)
8a9f15b [DPP-686][Self-service error codes] Removing default error factories (#11403)
bb37eef Switch script & trigger tests to postgres-backed Sadbonx (#11525)
```

Since `v1.18.0-snapshot.20211102.8257.0.7391a3cd`
mergify bot pushed a commit that referenced this pull request Nov 12, 2021
changelog_begin
changelog_end

CHANGELOG ENTRIES

```
[TS-Bindings] Ws multiplexing for stream queries is disabled by default as we investigate issues
of intermittent websocket closures.

* There is now an Early Access Helm chart to start all of the Connect components on a Kubernetes cluster.

[JSON-API] Make certain Hikari cp connection pool properties configurable via jdbc conf string, the properties are listed below
poolSize -- specifies the max pool size for the database connection pool
minIdle -- specifies the min idle connections for database connection pool
connectionTimeout -- long value, specifies the connection timeout for database connection pool
idleTimeout -- long value, specifies the idle timeout for the database connection pool
- [Integration Kit] - ledger-api-bench-tool can run a workflow consisting of the command submission phase and stream benchmark phase.
- [JSON API] While updating the contract table for a query, if the DB appears to be slow,
  JSON API will slow down its own inserts and deletes at some point rather than construct
  ever-larger INSERT and DELETE batch commands.
  See `issue #11589 <https://github.com/digital-asset/daml/pull/11589>`__.
[Ledger API] Retry the interpretation of a command in case of a race
with other transactions. This fix drastically reduces the likelihood of the error
"Could not find a suitable ledger time after 0 retries".
[ledger-api] - Return max_deduplication_duration as part of the metadata sent with the gRPC status for commands rejected because of invalid deduplication periods

- [Integration Kit] - ledger-api-bench-tool - added flexible party and template filters for transactions, transaction trees and active contract streams
```

COMMITS

```
d938a44 ws multiplexing is disabled by default as we investigate issues with intermittent web socket closures (#11657)
0f0a865 Make string interning available at ingestion [DPP-711] (#11645)
7296ba4 Move closureConvert and validate compiler phases into separate files. (#11656)
87f282c interfaces: Preserve/provide by_interface data for create actions. (#11639)
a9de728 Update rules_scala (#11616)
f468969 document helm chart (#11648)
55d4a03 Release 1.11.3 snapshot (#11652)
2dd5289 different types for different speedy phases (#11605)
44d604d Release new 1.18 snapshot (#11642)
e69a871 [JSON-API] Configurable Hikari connection pool props  (#11621)
cb7099b release 1.11.3 snapshot (#11628)
fd61d0b String interning read integration [DPP-706] (#11638)
c5a1f0b Republish error codes generator lib (#11633)
a6f745f Add two phase retrieval merge function [DPP-718] (#11571)
743ee46 ledger-api-bench-tool - command submission and stream reading complete workflow [DPP-668] (#11544)
f8f8807 Activate interning write side [DPP-710] (#11614)
93c07f4 Extend synonym expansion caching in data-dependencies (#11629)
cf445b2 Skip platform independence tests on release PRs (#11631)
b8f384e LF: Drop typeparameter from KeyWithMaintainers (#11625)
d0c313d Share dependencyInfo in data-dependencies (#11620)
4eed1d9 release 1.18.0-snapshot.20211109.8328.0.92181161 (#11623)
9218116 LF: Abstract VersionedValue and VersionedContractInstance (#11533)
b87acab foldLeft, foldRight, other Foldable specializations (#11592)
92dfcde [Self-service error codes] Use error validators in ApiPackageManagementService (#11575)
6372d41 limit contract insertion/deletion batching on backpressure (#11589)
3588284 Fix screenshots in daml intro 2 (#11617)
ef3fc0e Cache type synonym expansion in safeToReexport (#11612)
70b2fe3 Provide byInterface fields for fetch, exercise. (#11607)
830779f LF: drop useless class com.daml.lf.transaction (#11601)
7c92cab Cache type synonym expansion in isDuplicate (#11611)
cfd521f Fix GHC build errors on macOS (#11560)
a7559d0 LF: Rename GenTransaction to Transaction (#11599)
3b61a1b LF: allow exercise command by template for inherited choice. (#11584)
af1bee7 precondition lf conversion (#11538)
f722cf1 Bump extractor init timeout to 60s (#11598)
1e0a221 Set symlink_node_modules = False (#11606)
ac19c8c Synchronize in multi-participant script tests (#11594)
dd1b034 Restart the submission interpretation in case of a race [DPP-737] (#11579)
8d2b1b9 [JSON-API] Port of http-perf-test to sandbox(aka next) (#11543)
56a6db8 codeowners: reflect change in onwership of meiersi-da (#11505)
d433651 fix a few typos in Daml-LF spec (#11583)
5dee88b Refactoring store factories [DPP-709] (#11572)
41fb289 Port LedgerApiServer rate limiter interface from canton (#11577)
04cc5d7 Mark JSON API integration tests as flaky (#11586)
3bed7a9 Drop unused defaultFeatureFlags (#11580)
cbc3c8c Add by_interface field in Create, Fetch, & Exercise transaction nodes. (#11576)
7d68e05 Remove virtual choices (#11482)
9b94fa9 [ledger-api] - metadata for invalid deduplication period [KVL-1170] (#11534)
e7eb60f update compat versions for 1.18.0-snapshot.20211102.8257.0.7391a3cd (#11531)
a3d04ca Migrate matches-docs tests to sandbox on postgres (#11573)
21c9745 [Self-service error codes] ValueSwitch parameterized method (#11567)
6a0b84e Run Script export tests against BR (#11574)
a3195fd fix link to blog (#11570)
0823601 Self-service errors: generate error deprecation docs [KVL-1187] (#11539)
8d48cf6 DPP-664 Create contracts in batches (#11515)
9ef2eaf Remove parameter table join from SQL queries [DPP-705] (#11562)
af7f7f3 Fix LF coedowners (#11563)
4450311 Exclude/fix compatibility tests (#11529)
699c28f rotate release duty after 1.18.0-snapshot.20211102.8257.0.7391a3cd (#11522)
4d4869f interfaces: Improve fixed choice lookup (#11551)
ba96bf7 Add DbDtoToStringsForInterning [DPP-708] (#11557)
692dac6 LF: deprecate remaining type alias in Transaction (#11554)
7bc0f82 [kvutils] - Extract command deduplication steps from the TransactionCommitter [KVL-1174] (#11547)
0e95ccb Wire LedgerEndCache to relevant StorageBackend factories [DPP-704] (#11549)
73cb42e LF: deprecate ensureNoCid and assertNoCid in CidContainer (#11502)
30bfee9 Upgrade canton for conformance tests (#11550)
b84a914 Extractor: replace `sandbox-classic`-based fixtures with `sandbox`-based ones (#11489)
d86fe92 ledger-api-bench-tool - flexible stream filters [DPP-667] (#11458)
214c7ab Self-service errors: generate groups documentation [KVL-1186] (#11526)
60dc286 Avoid unnecessarily decoding DALFs during dependency installation (#11541)
afd79b6 Improve logging for dependency installation (#11540)
fdde535 fix hoogle by pinning nixpkgs (#11548)
5e93df6 Full conformance test with self-service error codes on ledger-on-memory (#11524)
efaf20b update NOTICES file (#11546)
c7bb94b Modify contract key attribute age for createAndExercise to avoid contract key violation on slow running tests (#11530)
541d07c Add string-interning data access [DPP-707] (#11484)
6c94164 Switching to composition at StorageBackend [DPP-704] (#11520)
d006ad0 Add string-interning view primitives [DPP-702] (#11475)
a917520 Save the submission time in the command deduplication value [KVL-1173] (#11509)
669186f Add priority-queue based worker Source [DPP-717] (#11500)
2091f59 Remove cats from kvutils (#11537)
b83af37 Fix recache log statement (#11536)
776790e Add choice collisions in scala name collision checker (#11528)
2efcb21 Extract the setting of the time bounds on the commit context [KVL-1173] (#11516)
9bb1d64 [ledger-api] - Return FAILED_PRECONDITION gRPC status code for invalid deduplication duration [KVL-1170] (#11483)
c098d75 Add a --log-level flag to damlc (#11514)
afee948 Bump rules_nixpkgs (#11517)
d6fb67c release 1.18.0-snapshot.20211102.8257.0.7391a3cd (#11521)
ac49d69 Ensure that -Werror converts parser warnings into errors (#11511)
bda35ca ledger-offset: Add a conversion to ByteString. (#11491)
8a9f15b [DPP-686][Self-service error codes] Removing default error factories (#11403)
bb37eef Switch script & trigger tests to postgres-backed Sadbonx (#11525)
```

Since `v1.18.0-snapshot.20211102.8257.0.7391a3cd`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants