Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Limit supported input versions in damlc to >= LF 1.8 #11905

Merged
merged 7 commits into from
Nov 30, 2021
Merged

Limit supported input versions in damlc to >= LF 1.8 #11905

merged 7 commits into from
Nov 30, 2021

Conversation

cocreature
Copy link
Contributor

1.8 was the version that introduced type synonyms, we really don’t
gain much by dropping more since data-dependencies mainly depends on
that. and this provides for a very natural upgrade path for users
where pretty much everyone should be able to upgrade directly to SDK
2.0 without having to go through intermediate versions.

changelog_begin

  • [Daml Compiler] The supported input LF versions for
    data-dependencies are now limited to LF 1.8 and newer.

changelog_end

Pull Request Checklist

  • Read and understand the contribution guidelines
  • Include appropriate tests
  • Set a descriptive title and thorough description
  • Add a reference to the issue this PR will solve, if appropriate
  • Include changelog additions in one or more commit message bodies between the CHANGELOG_BEGIN and CHANGELOG_END tags
  • Normal production system change, include purpose of change in description
  • If you mean to change the status of a component, please make sure you keep the Component Status page up to date.

NOTE: CI is not automatically run on non-members pull-requests for security
reasons. The reviewer will have to comment with /AzurePipelines run to
trigger the build.

@cocreature cocreature force-pushed the damlc-supported-input-versions branch 4 times, most recently from 8b2ec68 to 06a11a7 Compare November 29, 2021 16:34
@cocreature
Limit supported input versions in damlc to >= LF 1.8
e767949 
1.8 was the version that introduced type synonyms, we really don’t
gain much by dropping more since data-dependencies mainly depends on
that. and this provides for a very natural upgrade path for users
where pretty much everyone should be able to upgrade directly to SDK
2.0 without having to go through intermediate versions.

changelog_begin

- [Daml Compiler] The supported input LF versions for
  data-dependencies are now limited to LF 1.8 and newer.

changelog_end
@cocreature cocreature force-pushed the damlc-supported-input-versions branch from 06a11a7 to e767949 Compare November 29, 2021 16:41
@cocreature cocreature marked this pull request as ready for review November 29, 2021 16:41
@cocreature cocreature requested review from sofiafaro-da and a user November 29, 2021 16:41
sofiafaro-da
sofiafaro-da approved these changes Nov 29, 2021
View reviewed changes
Copy link
Contributor

@sofiafaro-da sofiafaro-da left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice, thank you!

Nice code generation :) I'm tempted to say we should do something similar for the list of stable packages we have in scala (for a similar version check).

@cocreature
Copy link
Contributor Author

Nice code generation :) I'm tempted to say we should do something similar for the list of stable packages we have in scala (for a similar version check).

we definitely should but I’d like to tackle #11371 as part of that.

@cocreature
fix some tests
bd3ba2b 
changelog_begin
changelog_end
@cocreature
Drop export 1.6 tests
987520d 
changelog_begin
changelog_end
@cocreature
Drop daml2js support for LF < 1.8
4ec55da 
changelog_begin

- [Daml2js] DARs with LF version < 1.8 are no longer supported.

changelog_end
@cocreature cocreature force-pushed the damlc-supported-input-versions branch from c2c9634 to 4ec55da Compare November 29, 2021 18:32
cocreature
cocreature commented Nov 29, 2021
View reviewed changes
language-support/ts/codegen/tests/src/DA/Test/Daml2js.hs
@@ -14,31 +14,21 @@ import qualified DA.Daml.LF.Ast.Version as LF
import DA.Test.Daml2jsUtils
import Data.List.Extra
import qualified Data.Text.Extended as T
import qualified Data.ByteString.Lazy as BSL
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@stefanobaghino-da can you please take a look if that looks reasonable to you? Factored it into a separate commit for easier review 4ec55da. I just dropped everything that depends on LF < 1.8.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried to go through it but my Haskell and knowledge of daml2js is insufficient to give you a meaningful answer. Who owned this previously? All I can say is that I would trust whomever actively maintained this previously, as to the best of my understanding there hasn't been any handover for this. If the application runtime need to own it, we'll probably need to rewrite it at some point as I don't think we should spread ourselves too thin into maintaining more and more stuff far away from the core skills of the team.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Martin & Robin wrote it originally. since then I did some of the work and Gary made some changes to support genmaps.

ghost
ghost approved these changes Nov 30, 2021
View reviewed changes
Copy link

@ghost ghost left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks!

@cocreature
.
67a5b89 
changelog_begin
changelog_end
@cocreature
bash is bad, stop using it
6ea5a12 
changelog_begin
changelog_end
@cocreature
.
d24d116 
changelog_begin
changelog_end
@cocreature cocreature merged commit 16135e6 into main Nov 30, 2021
@cocreature cocreature deleted the damlc-supported-input-versions branch November 30, 2021 16:24
azure-pipelines bot pushed a commit that referenced this pull request Dec 1, 2021
release 2.0.0-snapshot.20211130.8536.0.683ab871
ce04414 
This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.

@SamirTalwar-DA is in charge of this release.

Commit log:
```
683ab87 Move ghc-lib{,-parser} to bazel-haskell-deps (#11775)
9350632 Fix releasing of resources in case connection initialization failed (#11915)
e1559af Update `ModelConformanceValidator` comments and prevent them from getting outdated easily (#11924)
16a41f7 Avoid package validation in speedy compilation benchmark. (#11927)
16135e6 Limit supported input versions in damlc to >= LF 1.8 (#11905)
0ee4154 Use Absolute-indexes as keys for the Env-mapping during closure-conversion (#11912)
1d7bca8 Add optional typerep argument in UExerciseInterface. (#11910)
c2c22f8 kvutils: Protos no longer depend on the Daml-LF transaction proto [KVL-1166] (#11909)
5641948 [Docs] Add labels to error codes to support references to them (#11913)
0e77676 Update protobuf docs template to handle oneOf (#11887)
5a9481f unify heading markup according to README.md (#11919)
61334cf kvutils - Add Writer which can handle deduplication periods as offsets [KVL-1172] (#11900)
0b9d57b Add ContractDoesntImplementInterface error. (#11884)
49e5d41 align index.rst files for HTML and PDF (#11907)
dbbb05f Split daml-lf encode/decode Haskell libraries (#11906)
e5d3902 iface: support for fixed choices in TS codegen (#11630)
31cc540 Turn package name & version warnings into an error (#11859)
4e50060 self-service compat: set branch name to not main (#11902)
2f4aa47 refactor to avoid impossible code path (#11901)
a81995c switch dev images to Temurin (#11895)
f3a0e2e Set scalafmt dialect explicitly (#11898)
60e372d Don't run pruning tests on H2, they time-out (#11897)
58e69ad LF: replace "dev" LF version by "1.dev" in bazel files (#11894)
8ef348d Use absolute stack locations in SExpr1 (#11877)
071bcf7 update NOTICES file (#11892)
a1705d6 participant-state - Add an implicit logging context to the write service [kvl-1072] (#11838)
9ff64f7 Change daml script’s sleep to sleep for a minimum amount of time (#11886)
132c277 Add a Canton sandbox to the SDK (#11881)
68a2343 Only run self-service compat job on PRs (#11893)
c27406c [DPP-762][Self-service error codes] Automate generation of inventory of error categories. #11879
1379722 Adapt the compatibility exclusions (#11872)
d66ecc9 LF: Drop Archive Snapshot for LF < 1.14 (#11820)
abc141b Increase pruning tests timeout (#11891)
66b4074 Update protobuf docs plugin (#11880)
b0dda53 LF check stable proto with buf and md5sum. (#11888)
056fc52 Log while processing base64 encoded server key [DPP-761] (#11835)
dbda67b bump JVM in Docker image (#11883)
f69bd68 ledger-api-bench-tool: Fix flaky `MetricsCollectorSpec` (#11750)
cb758e8 Fix call to experimental interface signatory builtin (#11882)
024400b Error when fetching the wrong template id (via fetch by interface). (#11862)
0852c8f Make DA.List.Total return Optional instead (#11878)
df37346 [JSON-API] Add query store metrics (#11809)
2f8f69e Drop DA.Next.Set and DA.Next.Map (#11864)
5f3a4d2 [Self-service error codes] Fix section numbering in pdf for error codes section by moving it a level higher. (#11867)
cf3ac01 [Self-service error codes] Do not return error code id and definite_answer in metadata for security sensitive errors (#11828)
026b92a Add gRPC definitions for participant user management service (#11818)
2fde30d Disable writing volatile bits in Scala statsfile (#11875)
4ed9ded Remove xxd from dev-env (#11876)
eaded41 remove mergify (#11866)
3cd5028 fix a few more things in the daml-lf spec (#11851)
beca0ee Refactor StandaloneApiServer factory (#11842)
6356f13 Properly upgrade gRPC to 1.41.0 (#11858)
f6accd3 Release 1.18 RC2 (#11869)
d858873 fix main (#11868)
da8dd7e rotate release duty after 1.18.0-snapshot.20211123.8463.0.bd2a6852 (#11845)
066da4f [Self-service error codes] Small fixes for docs/scripts/live-preview.sh (#11856)
258fb65 Document how to deal with HTTP JSON API schema changes (#11336)
b8937ad ci: self-service compat test start (#11853)
de8d15f fix Nix install on macOS nodes (#11696)
b3d1d40 Expose submissionId via the Java bindings (#11839) (#11847)
86da6e8 LF: Test scala interface type checking (#11833)
5f52f00 increase linux cluster size (#11860)
5c12d75 Add a guard when exercising by interface. (#11836)
7c3a2a7 Add a new KV submission failure error (#11854)
aebc5a7 All packages must be valid (#11850)
0374843 speedy compilation benchmark (#11852)
393893a LF encoder: make package validation optional (#11849)
25b476f DPP-726 Add string interning unit tests (#11841)
59eb0d2 kvutils - For duplicate command rejections, add the submission id as metadata [KVL-1175] (#11848)
970243d Ensure stack-safety during closure-conversion. (#11778)
e63c80d update LATEST (#11846)
db42521 libs-scala: Change `SourceQueueResourceOwner` to `BoundedSourceQueueResourceOwner` [KVL-1177] (#11832)
109b606 Make the `InstrumentedSource.queue` use the `BoundedSourceQueue` [KVL-1177] (#11807)
```
Changelog:
```

- [Daml Compiler] The supported input LF versions for
  data-dependencies are now limited to LF 1.8 and newer.

- [Daml2js] DARs with LF version < 1.8 are no longer supported.

- [Integration Kit] kvutils protos no longer depend on the Daml-LF transaction proto

- [Daml Standard Library] DA.List.Total functions now return Optional
  instead of being polymorphic in the return type. DA.Optional.Total
  has been removed.

- [JSON-API] added metrics to separately track:
    - time taken to update query-store ACS (from ledger)
    - lookup times for the query store

- [Daml Standard Library] DA.Next.Map and DA.Next.Set have been removed
  after being deprecated since Daml-LF 1.11

- [Ledger API] Introduce gRPC definitions for experimental user
  managament service to manage users and their rights for interacting
  with the Ledger API served by a participant node.

[HTTP JSON API] [Docs] Document lack of data continuity guarantees and how to deal with schema changes
[Java Bindings] submissionId is now exposed via the bindings, see issue #11705
[Integration Kit] Add a new SUBMISSION_FAILED internal error
kvutils - For duplicate command rejections, the submission id of the already accepted transaction is returning as part of the gRPC metadata. The submission id will be included under the key `existing_submission_id`.

- [Integration Kit] `SourceQueueResourceOwner` has been renamed to `BoundedSourceQueueResourceOwner` and takes a `BoundedSourceQueue` from now on

- [Integration Kit] InstrumentedSource.queue.offer no longer returns a Future

```

CHANGELOG_BEGIN
CHANGELOG_END
cocreature added a commit that referenced this pull request Dec 1, 2021
@azure-pipelines @cocreature
release 2.0.0-snapshot.20211130.8536.0.683ab871 (#11934)
18dc856 
* release 2.0.0-snapshot.20211130.8536.0.683ab871

This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.

@SamirTalwar-DA is in charge of this release.

Commit log:
```
683ab87 Move ghc-lib{,-parser} to bazel-haskell-deps (#11775)
9350632 Fix releasing of resources in case connection initialization failed (#11915)
e1559af Update `ModelConformanceValidator` comments and prevent them from getting outdated easily (#11924)
16a41f7 Avoid package validation in speedy compilation benchmark. (#11927)
16135e6 Limit supported input versions in damlc to >= LF 1.8 (#11905)
0ee4154 Use Absolute-indexes as keys for the Env-mapping during closure-conversion (#11912)
1d7bca8 Add optional typerep argument in UExerciseInterface. (#11910)
c2c22f8 kvutils: Protos no longer depend on the Daml-LF transaction proto [KVL-1166] (#11909)
5641948 [Docs] Add labels to error codes to support references to them (#11913)
0e77676 Update protobuf docs template to handle oneOf (#11887)
5a9481f unify heading markup according to README.md (#11919)
61334cf kvutils - Add Writer which can handle deduplication periods as offsets [KVL-1172] (#11900)
0b9d57b Add ContractDoesntImplementInterface error. (#11884)
49e5d41 align index.rst files for HTML and PDF (#11907)
dbbb05f Split daml-lf encode/decode Haskell libraries (#11906)
e5d3902 iface: support for fixed choices in TS codegen (#11630)
31cc540 Turn package name & version warnings into an error (#11859)
4e50060 self-service compat: set branch name to not main (#11902)
2f4aa47 refactor to avoid impossible code path (#11901)
a81995c switch dev images to Temurin (#11895)
f3a0e2e Set scalafmt dialect explicitly (#11898)
60e372d Don't run pruning tests on H2, they time-out (#11897)
58e69ad LF: replace "dev" LF version by "1.dev" in bazel files (#11894)
8ef348d Use absolute stack locations in SExpr1 (#11877)
071bcf7 update NOTICES file (#11892)
a1705d6 participant-state - Add an implicit logging context to the write service [kvl-1072] (#11838)
9ff64f7 Change daml script’s sleep to sleep for a minimum amount of time (#11886)
132c277 Add a Canton sandbox to the SDK (#11881)
68a2343 Only run self-service compat job on PRs (#11893)
c27406c [DPP-762][Self-service error codes] Automate generation of inventory of error categories. #11879
1379722 Adapt the compatibility exclusions (#11872)
d66ecc9 LF: Drop Archive Snapshot for LF < 1.14 (#11820)
abc141b Increase pruning tests timeout (#11891)
66b4074 Update protobuf docs plugin (#11880)
b0dda53 LF check stable proto with buf and md5sum. (#11888)
056fc52 Log while processing base64 encoded server key [DPP-761] (#11835)
dbda67b bump JVM in Docker image (#11883)
f69bd68 ledger-api-bench-tool: Fix flaky `MetricsCollectorSpec` (#11750)
cb758e8 Fix call to experimental interface signatory builtin (#11882)
024400b Error when fetching the wrong template id (via fetch by interface). (#11862)
0852c8f Make DA.List.Total return Optional instead (#11878)
df37346 [JSON-API] Add query store metrics (#11809)
2f8f69e Drop DA.Next.Set and DA.Next.Map (#11864)
5f3a4d2 [Self-service error codes] Fix section numbering in pdf for error codes section by moving it a level higher. (#11867)
cf3ac01 [Self-service error codes] Do not return error code id and definite_answer in metadata for security sensitive errors (#11828)
026b92a Add gRPC definitions for participant user management service (#11818)
2fde30d Disable writing volatile bits in Scala statsfile (#11875)
4ed9ded Remove xxd from dev-env (#11876)
eaded41 remove mergify (#11866)
3cd5028 fix a few more things in the daml-lf spec (#11851)
beca0ee Refactor StandaloneApiServer factory (#11842)
6356f13 Properly upgrade gRPC to 1.41.0 (#11858)
f6accd3 Release 1.18 RC2 (#11869)
d858873 fix main (#11868)
da8dd7e rotate release duty after 1.18.0-snapshot.20211123.8463.0.bd2a6852 (#11845)
066da4f [Self-service error codes] Small fixes for docs/scripts/live-preview.sh (#11856)
258fb65 Document how to deal with HTTP JSON API schema changes (#11336)
b8937ad ci: self-service compat test start (#11853)
de8d15f fix Nix install on macOS nodes (#11696)
b3d1d40 Expose submissionId via the Java bindings (#11839) (#11847)
86da6e8 LF: Test scala interface type checking (#11833)
5f52f00 increase linux cluster size (#11860)
5c12d75 Add a guard when exercising by interface. (#11836)
7c3a2a7 Add a new KV submission failure error (#11854)
aebc5a7 All packages must be valid (#11850)
0374843 speedy compilation benchmark (#11852)
393893a LF encoder: make package validation optional (#11849)
25b476f DPP-726 Add string interning unit tests (#11841)
59eb0d2 kvutils - For duplicate command rejections, add the submission id as metadata [KVL-1175] (#11848)
970243d Ensure stack-safety during closure-conversion. (#11778)
e63c80d update LATEST (#11846)
db42521 libs-scala: Change `SourceQueueResourceOwner` to `BoundedSourceQueueResourceOwner` [KVL-1177] (#11832)
109b606 Make the `InstrumentedSource.queue` use the `BoundedSourceQueue` [KVL-1177] (#11807)
```
Changelog:
```

- [Daml Compiler] The supported input LF versions for
  data-dependencies are now limited to LF 1.8 and newer.

- [Daml2js] DARs with LF version < 1.8 are no longer supported.

- [Integration Kit] kvutils protos no longer depend on the Daml-LF transaction proto

- [Daml Standard Library] DA.List.Total functions now return Optional
  instead of being polymorphic in the return type. DA.Optional.Total
  has been removed.

- [JSON-API] added metrics to separately track:
    - time taken to update query-store ACS (from ledger)
    - lookup times for the query store

- [Daml Standard Library] DA.Next.Map and DA.Next.Set have been removed
  after being deprecated since Daml-LF 1.11

- [Ledger API] Introduce gRPC definitions for experimental user
  managament service to manage users and their rights for interacting
  with the Ledger API served by a participant node.

[HTTP JSON API] [Docs] Document lack of data continuity guarantees and how to deal with schema changes
[Java Bindings] submissionId is now exposed via the bindings, see issue #11705
[Integration Kit] Add a new SUBMISSION_FAILED internal error
kvutils - For duplicate command rejections, the submission id of the already accepted transaction is returning as part of the gRPC metadata. The submission id will be included under the key `existing_submission_id`.

- [Integration Kit] `SourceQueueResourceOwner` has been renamed to `BoundedSourceQueueResourceOwner` and takes a `BoundedSourceQueue` from now on

- [Integration Kit] InstrumentedSource.queue.offer no longer returns a Future

```

CHANGELOG_BEGIN
CHANGELOG_END

* bump to include fix for damlc package validation

changelog_begin
changelog_end

Co-authored-by: Azure Pipelines Daml Build <[email protected]>
Co-authored-by: Moritz Kiefer <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sofiafaro-da sofiafaro-da sofiafaro-da approved these changes

@garyverhaegen-da garyverhaegen-da Awaiting requested review from garyverhaegen-da

@stefanobaghino-da stefanobaghino-da Awaiting requested review from stefanobaghino-da

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@cocreature @stefanobaghino-da @sofiafaro-da