Java v2: Implementation and tests for synchronous and asynchronous invocations of Amazon Bedrock models

javav2/example_code/bedrockruntime/Readme.md

@@ -0,0 +1,85 @@
+# Bedrock Runtime code examples for the SDK for Java
+
+## Overview
+Shows how to use the AWS SDK for Java 2.x to work with the Amazon Bedrock Runtime client.
+
+Amazon Bedrock is a fully managed service that offers a choice of high-performing foundation models (FMs) from leading AI companies like AI21 Labs, Anthropic, Cohere, Meta, Stability AI, and Amazon with a single API, along with a broad set of capabilities you need to build generative AI applications, simplifying development while maintaining privacy and security.
+
+## ⚠️ Important
+* The SDK for Java examples performs AWS operations for the account and AWS Region for which you've specified credentials. Running these examples might incur charges on your account. For details about the charges you can expect for a given service and API operation, see [AWS Pricing](https://aws.amazon.com/pricing/).
+* Running the tests might result in charges to your AWS account.
+* We recommend that you grant your code least privilege. At most, grant only the minimum permissions required to perform the task. For more information, see [Grant least privilege](https://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html#grant-least-privilege).
+* This code is not tested in every AWS Region. For more information, see [AWS Regional Services](https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services).
+
+## Code examples
+
+### Prerequisites
+
+For prerequisites, see the [README](../../README.md) in the `javav2` folder.
+
+The credential provider used in all code examples is ProfileCredentialsProvider. For more information, see [Using credentials](https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/credentials.html).
+
+### Single actions
+
+The following example uses the **BedrockRuntimeClient**:
+
+- [Invoking a model](./src/main/java/com/example/bedrockruntime/InvokeModel.java) (InvokeModel command)
+
+The following example uses the **BedrockRuntimeAsyncClient**:
+
+- [Invoking a model with response stream](./src/main/java/com/example/bedrockruntime/InvokeModelWithResponseStream.java) (InvokeModelWithResponseStream command)
+
+## Run the examples
+
+To run these examples, set up your development environment. For more information,
+see [Get started with the SDK for Java](https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/setup.html).
+
+## Testing the Amazon Bedrock Java files
+
+You can test the Java code examples for the Amazon Bedrock Runtime client by running the test file [BedrockRuntimeTest.java](./src/test/java/BedrockRuntimeTest.java). This file uses JUnit 5 to run the JUnit tests and is located in the `src/test/java` folder. For more information, see [https://junit.org/junit5/](https://junit.org/junit5/).
+
+You can run the JUnit tests from a Java IDE, such as IntelliJ, or from the command line by using Maven. As each test runs, you can view messages that inform you if the various tests succeed or fail.
+
+**WARNING**: _Running these JUnit tests manipulates real Amazon resources and may incur charges on your account._
+
+### Command line
+
+To run the JUnit tests from the command line, you can use the following command:
+
+		mvn test
+
+You will see output from the JUnit tests, as shown here.
+
+    ...
+    [INFO] --------------------------------------------------
+	[INFO]  T E S T S
+    [INFO] --------------------------------------------------
+	[INFO] Running BedrockRuntimeTest
+    ...
+	Test 1 passed
+    ...
+    Test 2 passed.
+	...
+	[INFO] Results:
+	[INFO]
+	[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0
+	[INFO]
+	[INFO] --------------------------------------------------
+	[INFO] BUILD SUCCESS
+	[INFO] --------------------------------------------------
+	[INFO] Total time:  10.115 s
+	[INFO] Finished at: 2023-11-09T21:01:15+01:00
+	[INFO] --------------------------------------------------
+
+## Additional resources
+
+* [Amazon Bedrock - User Guide](https://docs.aws.amazon.com/bedrock/latest/userguide)
+* [Amazon Bedrock - API Reference](https://docs.aws.amazon.com/bedrock/latest/APIReference)
+* [AWS SDK for Java - Developer Guide](https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/home.html)
+* [AWS SDK for Java - Amazon Bedrock Runtime API reference](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/bedrockruntime/package-summary.html)
+
+---
+
+Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
+
+SPDX-License-Identifier: Apache-2.0

javav2/example_code/bedrockruntime/pom.xml

@@ -0,0 +1,60 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+    <modelVersion>4.0.0</modelVersion>
+    <groupId>BedrockJ2Project</groupId>
+    <artifactId>BedrockJ2Project</artifactId>
+    <version>1.0-SNAPSHOT</version>
+    <properties>
+        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
+        <maven.compiler.target>1.8</maven.compiler.target>
+        <maven.compiler.source>1.8</maven.compiler.source>
+        <java.version>1.8</java.version>
+    </properties>
+    <build>
+        <plugins>
+            <plugin>
+                <groupId>org.apache.maven.plugins</groupId>
+                <artifactId>maven-compiler-plugin</artifactId>
+                <version>3.1</version>
+                <configuration>
+                    <source>${java.version}</source>
+                    <target>${java.version}</target>
+                </configuration>
+            </plugin>
+        </plugins>
+    </build>
+    <dependencyManagement>
+        <dependencies>
+            <dependency>
+                <groupId>software.amazon.awssdk</groupId>
+                <artifactId>bom</artifactId>
+                <version>2.21.17</version>
+                <type>pom</type>
+                <scope>import</scope>
+            </dependency>
+        </dependencies>
+    </dependencyManagement>
+    <dependencies>
+        <dependency>
+            <groupId>software.amazon.awssdk</groupId>
+            <artifactId>bedrockruntime</artifactId>
+        </dependency>
+        <dependency>
+            <groupId>software.amazon.awssdk</groupId>
+            <artifactId>sts</artifactId>
+        </dependency>
+        <dependency>
+            <groupId>org.json</groupId>
+            <artifactId>json</artifactId>
+            <version>20231013</version>
+        </dependency>
+        <dependency>
+            <groupId>org.junit.jupiter</groupId>
+            <artifactId>junit-jupiter-api</artifactId>
+            <version>5.9.2</version>
+            <scope>test</scope>
+        </dependency>
+    </dependencies>
+</project>

javav2/example_code/bedrockruntime/src/main/java/com/example/bedrockruntime/InvokeModel.java

@@ -0,0 +1,86 @@
+// snippet-comment:[These are tags for the AWS doc team's sample catalog. Do not remove.]
+// snippet-sourcedescription:[InvokeModel.java demonstrates how to invoke a model with Amazon Bedrock.]
+// snippet-keyword:[AWS SDK for Java v2]
+// snippet-service:[Amazon Bedrock]
+
+/*
+   Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
+   SPDX-License-Identifier: Apache-2.0
+*/
+
+package com.example.bedrockruntime;
+
+// snippet-start:[bedrockruntime.java2.invoke_model.import]
+import org.json.JSONObject;
+import software.amazon.awssdk.auth.credentials.ProfileCredentialsProvider;
+import software.amazon.awssdk.core.SdkBytes;
+import software.amazon.awssdk.regions.Region;
+import software.amazon.awssdk.services.bedrockruntime.BedrockRuntimeClient;
+import software.amazon.awssdk.services.bedrockruntime.model.BedrockRuntimeException;
+import software.amazon.awssdk.services.bedrockruntime.model.InvokeModelRequest;
+import software.amazon.awssdk.services.bedrockruntime.model.InvokeModelResponse;
+// snippet-end:[bedrockruntime.java2.invoke_model.import]
+
+/**
+ * Before running this Java V2 code example, set up your development environment, including your credentials.
+ *
+ * For more information, see the following documentation topic:
+ *
+ * https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/get-started.html
+ */
+public class InvokeModel {
+
+    public static void main(String[] args) {
+
+        Region region = Region.US_EAST_1;
+        BedrockRuntimeClient bedrockRuntimeClient = BedrockRuntimeClient.builder()
+                .region(region)
+                .credentialsProvider(ProfileCredentialsProvider.create())
+                .build();
+
+        String prompt = "In one sentence, what is a large-language model?";
+
+        invokeModel(bedrockRuntimeClient, prompt);
+    }
+
+    // snippet-start:[bedrockruntime.java2.invoke_model.main]
+    public static String invokeModel(BedrockRuntimeClient bedrockRuntimeClient, String prompt) {
+
+        try {
+
+            double temperature = 0.8;
+            int maxTokensToSample = 300;
+
+            JSONObject payload = new JSONObject()
+                    .put("prompt", "Human: " + prompt + " Assistant:")
+                    .put("temperature", temperature)
+                    .put("max_tokens_to_sample", maxTokensToSample);
+
+            SdkBytes body = SdkBytes.fromUtf8String(payload.toString());
+
+            InvokeModelRequest request = InvokeModelRequest.builder()
+                    .modelId("anthropic.claude-v2")
+                    .contentType("application/json")
+                    .accept("application/json")
+                    .body(body)
+                    .build();
+
+            InvokeModelResponse response = bedrockRuntimeClient.invokeModel(request);
+
+            JSONObject responseBody = new JSONObject(response.body().asUtf8String());
+
+            String completion = responseBody.getString("completion");
+
+            System.out.printf("The model's response to '%s' is:%n%n", prompt);
+            System.out.println(completion);
+
+            return completion;
+
+        } catch (BedrockRuntimeException e) {
+            System.err.println(e.awsErrorDetails().errorMessage());
+            System.exit(1);
+        }
+        return null;
+    }
+    // snippet-end:[bedrockruntime.java2.invoke_model.main]
+}

javav2/example_code/bedrockruntime/src/main/java/com/example/bedrockruntime/InvokeModelWithResponseStream.java

@@ -0,0 +1,99 @@
+// snippet-comment:[These are tags for the AWS doc team's sample catalog. Do not remove.]
+// snippet-sourcedescription:[InvokeModel.java demonstrates how to invoke a model with Amazon Bedrock and process the response stream.]
+// snippet-keyword:[AWS SDK for Java v2]
+// snippet-service:[Amazon Bedrock]
+
+/*
+   Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
+   SPDX-License-Identifier: Apache-2.0
+*/
+
+package com.example.bedrockruntime;
+
+// snippet-start:[bedrockruntime.java2.invoke_model_with_response_stream.import]
+import org.json.JSONObject;
+import software.amazon.awssdk.auth.credentials.ProfileCredentialsProvider;
+import software.amazon.awssdk.core.SdkBytes;
+import software.amazon.awssdk.regions.Region;
+import software.amazon.awssdk.services.bedrockruntime.BedrockRuntimeAsyncClient;
+import software.amazon.awssdk.services.bedrockruntime.model.BedrockRuntimeException;
+import software.amazon.awssdk.services.bedrockruntime.model.InvokeModelWithResponseStreamRequest;
+import software.amazon.awssdk.services.bedrockruntime.model.InvokeModelWithResponseStreamResponseHandler;
+import software.amazon.awssdk.services.bedrockruntime.model.ResponseStream;
+
+import java.util.concurrent.CompletableFuture;
+// snippet-end:[bedrockruntime.java2.invoke_model_with_response_stream.import]
+
+/**
+ * Before running this Java V2 code example, set up your development environment, including your credentials.
+ *
+ * For more information, see the following documentation topic:
+ *
+ * https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/get-started.html
+ */
+public class InvokeModelWithResponseStream {
+
+    public static void main(String[] args) {
+
+        Region region = Region.US_EAST_1;
+        BedrockRuntimeAsyncClient bedrockRuntimeAsyncClient = BedrockRuntimeAsyncClient.builder()
+                .region(region)
+                .credentialsProvider(ProfileCredentialsProvider.create())
+                .build();
+
+        String prompt = "What is a large-language model?";
+
+        invokeModel(bedrockRuntimeAsyncClient, prompt);
+    }
+
+    // snippet-start:[bedrockruntime.java2.invoke_model_with_response_stream.main]
+    public static String invokeModel(BedrockRuntimeAsyncClient bedrockRuntimeAsyncClient, String prompt) {
+
+        try {
+
+            double temperature = 0.8;
+            int maxTokensToSample = 300;
+
+            JSONObject payload = new JSONObject()
+                    .put("prompt", "Human: " + prompt + " Assistant:")
+                    .put("temperature", temperature)
+                    .put("max_tokens_to_sample", maxTokensToSample);
+
+            SdkBytes body = SdkBytes.fromUtf8String(payload.toString());
+
+            InvokeModelWithResponseStreamRequest request = InvokeModelWithResponseStreamRequest.builder()
+                    .modelId("anthropic.claude-v2")
+                    .contentType("application/json")
+                    .accept("application/json")
+                    .body(body)
+                    .build();
+
+            InvokeModelWithResponseStreamResponseHandler.Visitor visitor =
+                    InvokeModelWithResponseStreamResponseHandler.Visitor.builder()
+                            .onDefault((event) -> System.out.println("\n\nDefault: " + event.toString()))
+                            .onChunk((chunk) -> {
+                                JSONObject json = new JSONObject(chunk.bytes().asUtf8String());
+                                System.out.print(json.getString("completion"));
+                            })
+                            .build();
+
+            InvokeModelWithResponseStreamResponseHandler responseHandler =
+                    InvokeModelWithResponseStreamResponseHandler.builder()
+                            .onEventStream((stream) -> stream.subscribe((ResponseStream e) -> e.accept(visitor)))
+                            .onComplete(() -> System.out.println("\n\nCompleted streaming response."))
+                            .onError((e) -> System.out.println("\n\nError: " + e.getMessage()))
+                            .build();
+
+            CompletableFuture<Void> futureResponse = bedrockRuntimeAsyncClient
+                    .invokeModelWithResponseStream(request, responseHandler);
+
+            futureResponse.join();
+
+        } catch (BedrockRuntimeException e) {
+            System.err.println(e.awsErrorDetails().errorMessage());
+            System.exit(1);
+        }
+        return null;
+    }
+    // snippet-end:[bedrockruntime.java2.invoke_model_with_response_stream.main]
+}

javav2/example_code/bedrockruntime/src/test/java/BedrockRuntimeTest.java

@@ -0,0 +1,58 @@
+/*
+   Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
+   SPDX-License-Identifier: Apache-2.0
+*/
+
+import com.example.bedrockruntime.InvokeModel;
+import com.example.bedrockruntime.InvokeModelWithResponseStream;
+import org.junit.jupiter.api.*;
+import software.amazon.awssdk.auth.credentials.ProfileCredentialsProvider;
+import software.amazon.awssdk.regions.Region;
+import software.amazon.awssdk.services.bedrockruntime.BedrockRuntimeAsyncClient;
+import software.amazon.awssdk.services.bedrockruntime.BedrockRuntimeClient;
+
+import static org.junit.jupiter.api.Assertions.*;
+
+@TestInstance(TestInstance.Lifecycle.PER_METHOD)
+@TestMethodOrder(MethodOrderer.OrderAnnotation.class)
+public class BedrockRuntimeTest {
+
+    @Test
+    @Order(1)
+    @Tag("IntegrationTest")
+    public void InvokeModel() {
+
+        try (BedrockRuntimeClient bedrockRuntime = BedrockRuntimeClient.builder()
+                .region(Region.US_EAST_1)
+                .credentialsProvider(ProfileCredentialsProvider.create())
+                .build()) {
+
+            String prompt = "In one sentence, what is a large-language model?";
+
+            String completion = InvokeModel.invokeModel(bedrockRuntime, prompt);
+
+            assertNotNull(completion, "The completion is null");
+            assertFalse(completion.trim().isEmpty(), "The completion is empty");
+
+            System.out.println("Test 1 passed.");
+        }
+    }
+
+    @Test
+    @Order(2)
+    @Tag("IntegrationTest")
+    public void InvokeModelWithResponseStream() {
+
+        try (BedrockRuntimeAsyncClient bedrockRuntime = BedrockRuntimeAsyncClient.builder()
+                .region(Region.US_EAST_1)
+                .credentialsProvider(ProfileCredentialsProvider.create())
+                .build()) {
+
+            String prompt = "In one sentence, what is a large-language model?";
+
+            assertDoesNotThrow(() -> InvokeModelWithResponseStream.invokeModel(bedrockRuntime, prompt));
+
+            System.out.println("Test 2 passed.");
+        }
+    }
+}