-
Notifications
You must be signed in to change notification settings - Fork 23
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Added new seadfarmer module for Question and Answering Gen AI construct #81
Changes from 16 commits
f39af39
5222f23
cfcf045
16f935e
ac753ba
a3165e0
3cd60cd
482fc7f
894260a
269a352
cbcca91
75bb382
4957fb4
5ed92ce
1bf11ab
2422dd7
35858b6
ec7510e
3b8f21b
53737d7
775d7c1
a7a7d48
e21a652
45459f9
ae131d2
e36916a
87c7b65
301206e
a24c5c7
d3ab066
ad0e94d
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,23 @@ | ||
name: qna-rag | ||
path: modules/fmops/qna-rag | ||
parameters: | ||
- name: cognito-pool-id | ||
value: us-east-1-XXXXX | ||
- name: os-domain-endpoint | ||
valueFrom: | ||
moduleMetadata: | ||
group: storage | ||
name: opensearch | ||
key: OpenSearchDomainEndpoint | ||
- name: os-security-group-id | ||
valueFrom: | ||
moduleMetadata: | ||
group: storage | ||
name: opensearch | ||
key: OpenSearchSecurityGroupId | ||
- name: vpc-id | ||
valueFrom: | ||
moduleMetadata: | ||
group: networking | ||
name: networking | ||
key: VpcId |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -51,3 +51,25 @@ parameters: | |
value: 30 | ||
- name: removal-policy | ||
value: DESTROY | ||
--- | ||
name: opensearch | ||
path: git::https://github.com/awslabs/idf-modules.git//modules/storage/opensearch/ | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We should lock this version to a release of IDF:
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. updated the manifest |
||
targetAccount: primary | ||
targetRegion: us-east-1 | ||
parameters: | ||
- name: encryption-type | ||
value: SSE | ||
- name: retention-type | ||
value: RETAIN | ||
- name: vpc-id | ||
valueFrom: | ||
moduleMetadata: | ||
group: networking | ||
name: networking | ||
key: VpcId | ||
- name: private-subnet-ids | ||
valueFrom: | ||
moduleMetadata: | ||
group: networking | ||
name: networking | ||
key: PrivateSubnetIds |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,58 @@ | ||
# AppSync endpoint for Question and Answering using RAG | ||
|
||
## Description | ||
|
||
Deploys an AWS AppSync endpoint for a Question and Answering model using RAG | ||
|
||
The module uses [AWS Generative AI CDK Constructs](https://github.com/awslabs/generative-ai-cdk-constructs/tree/main). | ||
|
||
### Architecture | ||
|
||
![AWS Appsync Question and Answering Endpoint Module Architecture](docs/_static/architecture.png "AWS Appsync Question and Answering RAG module Endpoint Module Architecture") | ||
|
||
## Inputs/Outputs | ||
|
||
### Input Parameters | ||
|
||
#### Required | ||
|
||
- `cognito-pool-id` - ID of the cognito user pool, used to secure GraphQl API | ||
- `os-domain-endpoint` - Open Search doamin url used as knowledge base | ||
- `os-security-group-id` - Security group of open search cluster | ||
- `vpc-id` - VPC id | ||
|
||
### Module Metadata Outputs | ||
|
||
- `GraphqlApiId` - Graphql API ID. | ||
- `GraphqlArn` - Graphql API ARN. | ||
|
||
## Examples | ||
|
||
Example manifest: | ||
|
||
```yaml | ||
name: qna-rag | ||
path: modules/fmops/qna-rag | ||
parameters: | ||
- name: cognito-pool-id | ||
value: us-east-1_XXXXX | ||
- name: os-domain-endpoint | ||
valueFrom: | ||
moduleMetadata: | ||
group: storage | ||
name: opensearch | ||
key: OpenSearchDomainEndpoint | ||
- name: os-security-group-id | ||
valueFrom: | ||
moduleMetadata: | ||
group: storage | ||
name: opensearch | ||
key: OpenSearchSecurityGroupId | ||
- name: vpc-id | ||
valueFrom: | ||
moduleMetadata: | ||
group: networking | ||
name: networking | ||
key: VpcId | ||
|
||
``` |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,63 @@ | ||
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. | ||
# SPDX-License-Identifier: Apache-2.0 | ||
|
||
import os | ||
|
||
import aws_cdk | ||
from aws_cdk import App | ||
from stack import RAGResources | ||
|
||
|
||
def _param(name: str) -> str: | ||
return f"SEEDFARMER_PARAMETER_{name}" | ||
|
||
|
||
project_name = os.getenv("SEEDFARMER_PROJECT_NAME", "") | ||
deployment_name = os.getenv("SEEDFARMER_DEPLOYMENT_NAME", "") | ||
module_name = os.getenv("SEEDFARMER_MODULE_NAME", "") | ||
app_prefix = f"{project_name}-{deployment_name}-{module_name}" | ||
vpc_id = os.getenv(_param("VPC_ID")) | ||
cognito_pool_id = os.getenv(_param("COGNITO_POOL_ID")) | ||
os_domain_endpoint = os.getenv(_param("OS_DOMAIN_ENDPOINT")) | ||
os_security_group_id = os.getenv(_param("OS_SECURITY_GROUP_ID")) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Minor query - can you please add ability to provide input asset bucket & enable observability (ref params here)? |
||
|
||
|
||
if not vpc_id: | ||
raise ValueError("Missing input parameter vpc-id") | ||
|
||
if not cognito_pool_id: | ||
raise ValueError("Missing input parameter cognito-pool-id") | ||
|
||
if not os_domain_endpoint: | ||
raise ValueError("Missing input parameter os-domain-endpoint") | ||
|
||
if not os_security_group_id: | ||
raise ValueError("Missing input parameter os-security-group-id") | ||
|
||
app = App() | ||
|
||
stack = RAGResources( | ||
scope=app, | ||
id=app_prefix, | ||
vpc_id=vpc_id, | ||
cognito_pool_id=cognito_pool_id, | ||
os_domain_endpoint=os_domain_endpoint, | ||
os_security_group_id=os_security_group_id, | ||
env=aws_cdk.Environment( | ||
account=os.environ["CDK_DEFAULT_ACCOUNT"], | ||
region=os.environ["CDK_DEFAULT_REGION"], | ||
), | ||
) | ||
|
||
aws_cdk.CfnOutput( | ||
scope=stack, | ||
id="metadata", | ||
value=stack.to_json_string( | ||
{ | ||
"GraphqlApiId": stack.rag_resource.graphql_api.api_id, | ||
saikatak marked this conversation as resolved.
Show resolved
Hide resolved
|
||
"GraphqlArn": stack.rag_resource.graphql_api.arn, | ||
} | ||
), | ||
) | ||
|
||
app.synth(force=True) |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
publishGenericEnvVariables: true | ||
deploy: | ||
phases: | ||
install: | ||
commands: | ||
- npm install -g [email protected] | ||
- pip install -r requirements.txt | ||
build: | ||
commands: | ||
- cdk deploy --require-approval never --progress events --app "python app.py" --outputs-file ./cdk-exports.json | ||
# Export metadata | ||
- seedfarmer metadata convert -f cdk-exports.json || true | ||
destroy: | ||
phases: | ||
install: | ||
commands: | ||
- npm install -g [email protected] | ||
- pip install -r requirements.txt | ||
build: | ||
commands: | ||
- cdk destroy --force --app "python app.py" |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
aws-cdk-lib==2.137.0 | ||
cdk-nag==2.28.91 | ||
boto3~=1.34.84 | ||
attrs==23.2.0 | ||
cdklabs-generative-ai-cdk-constructs==0.1.119 |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,88 @@ | ||
# | ||
# This file is autogenerated by pip-compile with Python 3.12 | ||
# by the following command: | ||
# | ||
# pip-compile --output-file=requirements.txt requirements.in | ||
# | ||
attrs==23.2.0 | ||
# via | ||
# -r requirements.in | ||
# cattrs | ||
# jsii | ||
aws-cdk-asset-awscli-v1==2.2.202 | ||
# via aws-cdk-lib | ||
aws-cdk-asset-kubectl-v20==2.1.2 | ||
# via aws-cdk-lib | ||
aws-cdk-asset-node-proxy-agent-v6==2.0.3 | ||
# via aws-cdk-lib | ||
aws-cdk-lib==2.137.0 | ||
# via | ||
# -r requirements.in | ||
# cdk-nag | ||
# cdklabs-generative-ai-cdk-constructs | ||
boto3==1.34.86 | ||
# via -r requirements.in | ||
botocore==1.34.86 | ||
# via | ||
# boto3 | ||
# s3transfer | ||
cattrs==23.2.3 | ||
# via jsii | ||
cdk-nag==2.28.91 | ||
# via | ||
# -r requirements.in | ||
# cdklabs-generative-ai-cdk-constructs | ||
cdklabs-generative-ai-cdk-constructs==0.1.119 | ||
# via -r requirements.in | ||
constructs==10.3.0 | ||
# via | ||
# aws-cdk-lib | ||
# cdk-nag | ||
# cdklabs-generative-ai-cdk-constructs | ||
importlib-resources==6.4.0 | ||
# via jsii | ||
jmespath==1.0.1 | ||
# via | ||
# boto3 | ||
# botocore | ||
jsii==1.97.0 | ||
# via | ||
# aws-cdk-asset-awscli-v1 | ||
# aws-cdk-asset-kubectl-v20 | ||
# aws-cdk-asset-node-proxy-agent-v6 | ||
# aws-cdk-lib | ||
# cdk-nag | ||
# cdklabs-generative-ai-cdk-constructs | ||
# constructs | ||
publication==0.0.3 | ||
# via | ||
# aws-cdk-asset-awscli-v1 | ||
# aws-cdk-asset-kubectl-v20 | ||
# aws-cdk-asset-node-proxy-agent-v6 | ||
# aws-cdk-lib | ||
# cdk-nag | ||
# cdklabs-generative-ai-cdk-constructs | ||
# constructs | ||
# jsii | ||
python-dateutil==2.9.0.post0 | ||
# via | ||
# botocore | ||
# jsii | ||
s3transfer==0.10.1 | ||
# via boto3 | ||
six==1.16.0 | ||
# via python-dateutil | ||
typeguard==2.13.3 | ||
# via | ||
# aws-cdk-asset-awscli-v1 | ||
# aws-cdk-asset-kubectl-v20 | ||
# aws-cdk-asset-node-proxy-agent-v6 | ||
# aws-cdk-lib | ||
# cdk-nag | ||
# cdklabs-generative-ai-cdk-constructs | ||
# constructs | ||
# jsii | ||
typing-extensions==4.11.0 | ||
# via jsii | ||
urllib3==1.25.4 | ||
# via botocore |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,73 @@ | ||
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. | ||
# SPDX-License-Identifier: Apache-2.0 | ||
|
||
from constructs import Construct | ||
from cdklabs.generative_ai_cdk_constructs import QaAppsyncOpensearch | ||
from aws_cdk import Stack | ||
from aws_cdk import aws_ec2 as ec2 | ||
from aws_cdk import ( | ||
aws_opensearchservice as os, | ||
aws_cognito as cognito, | ||
) | ||
|
||
|
||
class RAGResources(Stack): | ||
def __init__( | ||
self, | ||
scope: Construct, | ||
id: str, | ||
vpc_id: str, | ||
cognito_pool_id: str, | ||
os_domain_endpoint: str, | ||
os_security_group_id: str, | ||
**kwargs, | ||
) -> None: | ||
super().__init__( | ||
scope, | ||
id, | ||
description=" This stack creates resources for the LLM - QA RAG ", | ||
**kwargs, | ||
) | ||
# get an existing OpenSearch provisioned cluster | ||
os_domain = os.Domain.from_domain_endpoint( | ||
self, | ||
"osdomain", | ||
domain_endpoint="https://" + os_domain_endpoint, | ||
) | ||
self.os_domain = os_domain | ||
# get vpc from vpc id | ||
vpc = ec2.Vpc.from_lookup( | ||
self, | ||
"VPC", | ||
vpc_id=vpc_id, | ||
) | ||
|
||
# get an existing userpool | ||
cognito_pool_id = cognito_pool_id | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is Congnito mandatory? Can we add a parameter to support both IAM and Cognito? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. As of today cognito is mandatory to be used with this construct. Created a feature request on the construct to make cognito user pool id as optional parameter There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Thank you! |
||
user_pool_loaded = cognito.UserPool.from_user_pool_id( | ||
self, | ||
"myuserpool", | ||
user_pool_id=cognito_pool_id, | ||
) | ||
|
||
rag_source = QaAppsyncOpensearch( | ||
self, | ||
"QaAppsyncOpensearch", | ||
existing_vpc=vpc, | ||
existing_opensearch_domain=os_domain, | ||
open_search_index_name="qa-appsync-index", | ||
cognito_user_pool=user_pool_loaded, | ||
) | ||
|
||
security_group = rag_source.security_group | ||
|
||
os_security_group = ec2.SecurityGroup.from_security_group_id( | ||
self, "OSSecurityGroup", os_security_group_id | ||
) | ||
os_security_group.add_ingress_rule( | ||
peer=security_group, | ||
connection=ec2.Port.tcp(443), | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Where is this port number coming from? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is the port on which open search is listening, Thus allowing traffic to Open search from lambda on port 443. Can make this configurable if that makes sense. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Made the port configurable parameter with default value as 443 |
||
description="Allow inbound HTTPS to open search from question answering lambda", | ||
) | ||
|
||
self.rag_resource = rag_source |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks like we need an IDF module for Cognito. Can you move the manifest in
examples/manifests/
since this requires user input/pre-setup Conginto user pool? We can move it back once we have congnito module/or IAM-only auth mode.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Moved the manifest to examples
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks!