Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Support more services on Java #504

Merged
merged 31 commits into from
Oct 9, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
323b919
Quick fixes to name discovery
robin-aws Jul 12, 2024
8018081
Fix a case that needs pascalCase
robin-aws Jul 14, 2024
19a7bdc
Another case
robin-aws Aug 3, 2024
7c53e2a
Avoid typeforListSetOrMapNoEnum for now
robin-aws Aug 3, 2024
ea422d1
More correct unCapitalize
robin-aws Aug 3, 2024
a04a1bd
Disable Input -> Request rules for now
robin-aws Aug 3, 2024
d1548c7
Enable glue and lakeformation in CI on Java
robin-aws Aug 6, 2024
b997983
operation + Request/Response instead
robin-aws Aug 6, 2024
9a2107f
Try different solution to list of enums
robin-aws Aug 7, 2024
d9c08d2
Fix most unit tests
robin-aws Aug 7, 2024
222e20a
Merge branch 'main-1.x' of github.com:smithy-lang/smithy-dafny into r…
robin-aws Aug 22, 2024
f82dfe8
Merge branch 'main-1.x' into robin-aws/support-more-services-on-java
robin-aws Oct 3, 2024
dd9f308
Makefile tweak
robin-aws Oct 3, 2024
e557380
Filter out operation that uses streaming, typo
robin-aws Oct 4, 2024
ec2b23d
Fix dotnet
robin-aws Oct 4, 2024
b0ae9f9
Rename Model to model (1/2)
robin-aws Oct 4, 2024
0394f53
Rename Model to model (2/2)
robin-aws Oct 4, 2024
23089b5
Fixes
robin-aws Oct 4, 2024
d94e8fd
Formatting
robin-aws Oct 4, 2024
7110728
Fix build file
robin-aws Oct 4, 2024
496df4c
Remove dead code
robin-aws Oct 4, 2024
3819333
Unused imports
robin-aws Oct 4, 2024
3ae96db
Merge branch 'main-1.x' into robin-aws/support-more-services-on-java
robin-aws Oct 8, 2024
0ff9a8d
Removing hooks for AWS service limitations
robin-aws Oct 9, 2024
f87f309
Make Native abstract
robin-aws Oct 9, 2024
8a07215
Formatting
robin-aws Oct 9, 2024
1e3d68f
Merge branch 'main-1.x' into robin-aws/support-more-services-on-java
robin-aws Oct 9, 2024
69c149a
Revert "Make Native abstract"
robin-aws Oct 9, 2024
a8b7347
Add exception message instead
robin-aws Oct 9, 2024
f9ecbd8
Merge branch 'robin-aws/support-more-services-on-java' of github.com:…
robin-aws Oct 9, 2024
6edde6d
Formatting
robin-aws Oct 9, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion SmithyDafnyMakefile.mk
Original file line number Diff line number Diff line change
Expand Up @@ -450,7 +450,7 @@ _polymorph_rust: $(if $(RUST_BENERATED), , _polymorph)

########################## .NET targets

net: polymorph_dafny transpile_net polymorph_net test_net
net: polymorph_dafny transpile_net polymorph_dotnet test_net

transpile_net: $(if $(ENABLE_EXTERN_PROCESSING), _with_extern_pre_transpile, )
transpile_net: | transpile_implementation_net transpile_test_net transpile_dependencies_net
Expand Down Expand Up @@ -525,6 +525,7 @@ transpile_test_java: _transpile_test_all _mv_test_java
# To avoid `java/implementation-java` the code is generated and then moved.
_mv_implementation_java:
rm -rf runtimes/java/src/main/dafny-generated
mkdir -p runtimes/java/src/main
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this right?
If you run mkdir against an existing directory, does it succeed?
Or does it fail?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It succeeds - -p makes it recursively create all missing directories, so it's perfectly happy if all already exist

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Notice the same thing is already on the runtimes/java/src/test directory. I actually don't know how we got away without this fix for so long. :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because you did not use to generate the template files,
and we had to manually stub the __default.java files in the Java extern source path.

Thus, we always had a directory.

mv runtimes/java/ImplementationFromDafny-java runtimes/java/src/main/dafny-generated
_mv_test_java:
rm -rf runtimes/java/src/test/dafny-generated
Expand Down
2 changes: 2 additions & 0 deletions TestModels/aws-sdks/glue/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@ AWS_SDK_CMD=--aws-sdk
# This project has no dependencies
# DEPENDENT-MODELS:=

POLYMORPH_OPTIONS=--generate project-files --generate client-constructors

# There is no wrapped target for aws-sdk types
_polymorph_wrapped: ;
_polymorph_wrapped_dafny: ;
Expand Down
13 changes: 13 additions & 0 deletions TestModels/aws-sdks/lakeformation/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -18,12 +18,25 @@ SMITHY_DEPS=dafny-dependencies/Model/traits.smithy

AWS_SDK_CMD=--aws-sdk

SMITHY_MODEL_ROOT := $(LIBRARY_ROOT)/model

# This project has no dependencies
# DEPENDENT-MODELS:=

POLYMORPH_OPTIONS=--generate project-files --generate client-constructors

# Override calling the polymorph CLI to invoke the Smithy build instead
_polymorph_dafny:
$(GRADLEW) polymorphDafny

_polymorph_dotnet:
$(GRADLEW) polymorphDotnet

_polymorph_java:
$(GRADLEW) polymorphJava

# There is no wrapped target for aws-sdk types
_polymorph: ;
_polymorph_wrapped: ;
_polymorph_wrapped_dafny: ;
_polymorph_wrapped_net: ;
Expand Down
89 changes: 89 additions & 0 deletions TestModels/aws-sdks/lakeformation/build.gradle.kts
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
// Copyright Amazon.com Inc. or its affiliates. All Rights Reserved.
// SPDX-License-Identifier: Apache-2.0

plugins {
id("software.amazon.smithy").version("0.6.0")
}

repositories {
mavenLocal()
mavenCentral()
}

dependencies {
implementation("software.amazon.smithy:smithy-model:1.28.0")
implementation("software.amazon.smithy:smithy-aws-traits:1.28.0")
implementation("software.amazon.smithy:smithy-rules-engine:1.28.0")

// Must be built and published to the local Maven repo
implementation("software.amazon.smithy.dafny:smithy-dafny-codegen:0.1.0")
}

tasks.register("polymorphDafny") {
dependsOn("build")
doLast {
// if needed, specify a projection to use instead
// default (no projection) is "source"
val projectionName = "operation-subset"
copy {
from(layout.buildDirectory.dir("smithyprojections/" + project.name + "/" + projectionName + "/dafny-client-codegen/project.properties"))
into(".")
}
copy {
from(layout.buildDirectory.dir("smithyprojections/" + project.name + "/" + projectionName + "/dafny-client-codegen/Model/"))
into("model")
}
exec {
// need to adjust the relative import, since we're copying it away
// the commandLine method does not play nice with sed,
// so we have to execute it through bash :(
commandLine("bash", "-c", "sed '4s|../../../../../../../../dafny-dependencies/StandardLibrary/src/Index.dfy|../../../dafny-dependencies/StandardLibrary/src/Index.dfy|' model/ComAmazonawsLakeformationTypes.dfy > model/tmp && mv model/tmp model/ComAmazonawsLakeformationTypes.dfy")
}
}
}

tasks.register("polymorphDotnet") {
dependsOn("build")
doLast {
// if needed, specify a projection to use instead
// default (no projection) is "source"
val projectionName = "operation-subset"
copy {
from(layout.buildDirectory.dir("smithyprojections/" + project.name + "/" + projectionName + "/dafny-client-codegen/runtimes/net"))
into("runtimes/net")
}
exec {
// need to adjust the relative import, since we're copying it away
// the commandLine method does not play nice with sed,
// so we have to execute it through bash :(
commandLine("bash", "-c", "sed 's|../../../../../../../../../dafny-dependencies/StandardLibrary/runtimes/net/STD.csproj|../../../../dafny-dependencies/StandardLibrary/runtimes/net/STD.csproj|' runtimes/net/LakeFormation.csproj > runtimes/net/tmp && mv runtimes/net/tmp runtimes/net/LakeFormation.csproj")
}
}
}

tasks.register("polymorphJava") {
dependsOn("build")
doLast {
// if needed, specify a projection to use instead
// default (no projection) is "source"
val projectionName = "operation-subset"
// We can't just copy runtimes/java over unfortunately,
// because we need a fresher software.amazon.awssdk:kms version
// than what's in the template.
copy {
from(layout.buildDirectory.dir("smithyprojections/" + project.name + "/" + projectionName + "/dafny-client-codegen/runtimes/java"))
into("runtimes/java")
}
}
}

buildscript {
val smithyVersion: String by project

repositories {
mavenCentral()
}
dependencies {
"classpath"("software.amazon.smithy:smithy-cli:$smithyVersion")
}
}
3 changes: 3 additions & 0 deletions TestModels/aws-sdks/lakeformation/gradle.properties
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
smithyVersion=1.27.2
smithyGradleVersion=0.6.0

28 changes: 28 additions & 0 deletions TestModels/aws-sdks/lakeformation/smithy-build.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
{
"version": "1.0",
"projections": {
"operation-subset": {
"transforms": [
{
"name": "excludeShapesBySelector",
"args": {
"selector": "operation [id|name = GetWorkUnitResults]"
}
},
{
"name": "removeUnusedShapes",
"args": {}
}
],
"plugins": {
"dafny-client-codegen": {
"edition": "2023",
"service": "com.amazonaws.lakeformation#AWSLakeFormation",
"dafnyVersion": "${DAFNY_VERSION}",
"targetLanguages": ["dotnet", "java"],
"includeDafnyFile": "../../dafny-dependencies/StandardLibrary/src/Index.dfy"
}
}
}
}
}
2 changes: 1 addition & 1 deletion TestModels/aws-sdks/lakeformation/src/Index.dfy
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// Copyright Amazon.com Inc. or its affiliates. All Rights Reserved.
// SPDX-License-Identifier: Apache-2.0
include "../Model/ComAmazonawsLakeformationTypes.dfy"
include "../model/ComAmazonawsLakeformationTypes.dfy"

module {:options "--function-syntax:4"}{:extern "software.amazon.cryptography.services.lakeformation.internaldafny"} Com.Amazonaws.LakeFormation refines AbstractComAmazonawsLakeformationService {

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -38,8 +38,6 @@ class JavaTestModels extends TestModelTest {
DISABLED_TESTS.add("SimpleTypes/SimpleString");
DISABLED_TESTS.add("SimpleTypes/SimpleTimestamp");
DISABLED_TESTS.add("Union");
DISABLED_TESTS.add("aws-sdks/glue");
DISABLED_TESTS.add("aws-sdks/lakeformation");
DISABLED_TESTS.add("aws-sdks/kms-lite");
DISABLED_TESTS.add("aws-sdks/sqs");
DISABLED_TESTS.add("aws-sdks/sqs-via-cli");
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@
import com.squareup.javapoet.ClassName;
import com.squareup.javapoet.CodeBlock;
import com.squareup.javapoet.TypeName;
import software.amazon.awssdk.codegen.internal.Utils;
import software.amazon.awssdk.utils.internal.CodegenNamingUtils;
import software.amazon.polymorph.smithydafny.DafnyNameResolver;
import software.amazon.polymorph.smithydafny.DafnyVersion;
import software.amazon.polymorph.smithyjava.generator.CodegenSubject;
Expand Down Expand Up @@ -118,7 +120,7 @@ public CodeBlock methodForGetMember(
return CodeBlock.of(
"$L.$L()",
variableName,
uncapitalize(memberShape.getMemberName())
Utils.unCapitalize(memberShape.getMemberName())
);
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -164,7 +164,7 @@ public ParameterizedTypeName typeForListSetOrMapNoEnum(
};
}

public static ClassName classNameForAwsSdkShape(final Shape shape) {
public ClassName classNameForAwsSdkShape(final Shape shape) {
return ClassName.get(
defaultModelPackageName(packageNameForAwsSdkV1Shape(shape)),
StringUtils.capitalize(shape.getId().getName())
Expand All @@ -187,7 +187,7 @@ public ClassName classNameForStructure(final Shape shape) {
}
// check if this Shape is in AWS SDK for Java V1 package
if (AwsSdkNameResolverHelpers.isInAwsSdkNamespace(shape.getId())) {
AwsSdkNativeV1.classNameForAwsSdkShape(shape);
return classNameForAwsSdkShape(shape);
}
return super.classNameForStructure(shape);
}
Expand Down
Loading
Loading