Skip to content

Commit 536721b

Browse files
sandykumar93sandykumar93
committed
feat: Streaming DDB, ChatGPT, and Bedrock
1 parent 99ed52d commit 536721b

File tree

89 files changed

+23634
-0
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

89 files changed

+23634
-0
lines changed

‎README copy.md

+49
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
# All About Lambda Response Streaming
2+
This repo consists of 3 use cases for response streaming
3+
1. Streaming data from DynamoDB.
4+
2. Streaming data from ChatGPT and comparing it to a regular API response.
5+
3. Streaming data from Bedrock on a custom python lambda runtime and comparing it to ChatGPT Streaming.
6+
7+
# Pre Requisites
8+
- AWS SAM CLI
9+
- ShelbySAM - `npm i -g @antstackio/shelbysam`
10+
- Docker
11+
- Python3
12+
- You need to add openapi secrets in AWS secrete manager service:
13+
- `CHAT_GPT` as secret name.
14+
- chat gpt api key as `API_KEY` secret.
15+
- chat gpt org id as `ORG_ID` secret.
16+
17+
18+
# Initialization
19+
20+
## samconfig.toml
21+
The path to the file is `/backend/samconfig.toml`. This file contains the stack name, the s3 prefix, and the region of deployment. Note that Amazon Bedrock is supported in only 5 regions, so choose your region carefully
22+
23+
## parameters
24+
The path to parameter file is `/backend/infra_resources/Parameters.yaml`. Change the parameter values as required.
25+
26+
# Deployment
27+
28+
## Backend
29+
1. Navigate to the `backend` folder.
30+
2. Run `shelbysam construct`. After making changes to the paramters and other infra resources, run this command to generate the template.yaml file.
31+
3. Run `sam build`. Note: When running this command docker must be running as well. The first execution will take a long time as the docker build time is quite large.
32+
4. Run `sam deploy --resolve-image-repos`. The additional parameter `resolve-image-repos` to sam deploy will automatically fetch the docker images for the lambda.
33+
34+
35+
## Backend Post Deployment
36+
1. Navigate to the root of the repository
37+
2. Modify the `scan_db_table` and `query_db_table` variable to match the deployment. You can fetch this from the cloudformation output as well.
38+
3. Run `pip3 install -r requirements.txt` to install the required packages.
39+
4. Run `python3 uploadToDynamoDb.py`. This will update the base data required for the DynamoDB streaming use case.
40+
41+
42+
## Frontend
43+
1. `cd frontend`. Navigate to the frontend directory.
44+
2. `npm install`. Install the required packages.
45+
3. `npm run build`. Build the frontend code.
46+
4. `aws s3 sync dist s3://{BucketName}`. Replace `BucketName` with the bucket name used for deployment. It is listed as `HostedBucketName` in the output section of the cloudformation.
47+
48+
49+
The cloudfront url of the deployed application is available in the output section as `CloudFrontDomainName`. Once the above mentioned steps are completed, you can access the link to use the application.

‎backend/.gitignore

+25
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
# Ignore build artifacts
2+
build/
3+
dist/
4+
5+
# Ignore SAM generated artifacts
6+
.sam/
7+
.samconfig.toml
8+
9+
# Ignore dependencies
10+
node_modules/
11+
npm-debug.log
12+
13+
# Ignore .DS_Store files (Mac specific)
14+
.DS_Store
15+
16+
# Ignore AWS credentials
17+
.aws-sam/
18+
.sam.local/
19+
aws-sam-cli.yaml
20+
21+
# Ignore local environment variables
22+
.env
23+
24+
#Shelbysam
25+
.shelbysam

‎backend/README.md

Whitespace-only changes.

‎backend/infra_resources/Outputs.yaml

+37
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
ResponseLambdaUrlDDB:
2+
Value:
3+
Fn::GetAtt:
4+
- DDBStreamingLambdaUrl
5+
- FunctionUrl
6+
ResponseLambdaUrlChatGPT:
7+
Value:
8+
Fn::GetAtt:
9+
- ChatGPTStreamingFunctionUrl
10+
- FunctionUrl
11+
ResponseLambdaUrlBedrock:
12+
Value:
13+
Fn::GetAtt:
14+
- BedrockStreamingFunctionUrl
15+
- FunctionUrl
16+
ApiResourceId:
17+
Value:
18+
Fn::GetAtt:
19+
- ResponseStreamingFunctionApi
20+
- RestApiId
21+
CloudFrontDomainName:
22+
Value:
23+
Fn::GetAtt:
24+
- CloudFrontDistribution
25+
- DomainName
26+
HostedBucketName:
27+
Value:
28+
Ref:
29+
S3Bucket
30+
DDBQueryTable:
31+
Value:
32+
Ref:
33+
QueryTable
34+
DDBScanTable:
35+
Value:
36+
Ref:
37+
ScanTable

‎backend/infra_resources/Parameters.yaml

+9
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
S3BucketName:
2+
Type: String
3+
Default: bucket
4+
DynamoDBScanName:
5+
Type: String
6+
Default: scan-db
7+
DynamoDBQueryName:
8+
Type: String
9+
Default: query-db

‎backend/infra_resources/Resources/BedrockRegularLambda.yaml

+15
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
Type: AWS::Serverless::Function
2+
Properties:
3+
CodeUri: src/BedrockRegularLambda
4+
Handler: index.handler
5+
Runtime: python3.9
6+
Timeout: 60
7+
MemorySize: 1536
8+
Events:
9+
ApiEvent:
10+
Type: Api
11+
Properties:
12+
Path: /bedrock_regular
13+
Method: post
14+
RestApiId:
15+
Ref: ResponseStreamingFunctionApi

‎backend/infra_resources/Resources/BedrockStreamingFunction.yaml

+16
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
Type: AWS::Serverless::Function
2+
Metadata:
3+
Dockerfile: Dockerfile
4+
DockerContext: src/BedrockStreamingLambda
5+
Properties:
6+
PackageType: Image
7+
Timeout: 60
8+
MemorySize: 2048
9+
Policies:
10+
- Version: '2012-10-17'
11+
Statement:
12+
- Sid: FunctionURLAllowPublicAccess
13+
Effect: Allow
14+
Action:
15+
- bedrock:InvokeModelWithResponseStream
16+
Resource: '*'

‎backend/infra_resources/Resources/BedrockStreamingFunctionPublicAccess.yaml

+7
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
Type: AWS::Lambda::Permission
2+
Properties:
3+
Action: lambda:InvokeFunctionUrl
4+
FunctionName:
5+
Ref: BedrockStreamingFunction
6+
FunctionUrlAuthType: NONE
7+
Principal: "*"

‎backend/infra_resources/Resources/BedrockStreamingFunctionUrl.yaml

+9
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
Type: AWS::Lambda::Url
2+
Properties:
3+
AuthType: NONE
4+
InvokeMode: RESPONSE_STREAM
5+
TargetFunctionArn:
6+
Ref: BedrockStreamingFunction
7+
Cors:
8+
AllowOrigins:
9+
- "*"

‎backend/infra_resources/Resources/ChatGPTRegularLambda.yaml

+21
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
Type: AWS::Serverless::Function
2+
Properties:
3+
CodeUri: src/ChatGPTRegularLambda
4+
Handler: index.handler
5+
Runtime: nodejs18.x
6+
Timeout: 60
7+
MemorySize: 1536
8+
Events:
9+
ApiEvent:
10+
Type: Api
11+
Properties:
12+
Path: /chatgpt_regular
13+
Method: post
14+
RestApiId:
15+
Ref: ResponseStreamingFunctionApi
16+
Layers:
17+
- Ref: StreamingLambdaLayer
18+
Environment:
19+
Variables:
20+
API_KEY: "{{resolve:secretsmanager:CHAT_GPT:SecretString:API_KEY}}"
21+
ORG_ID: "{{resolve:secretsmanager:CHAT_GPT:SecretString:ORG_ID}}"

‎backend/infra_resources/Resources/ChatGPTStreamingFunction.yaml

+14
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
Type: AWS::Serverless::Function
2+
Properties:
3+
CodeUri: src/ChatGPTStreamingLambda
4+
Handler: index.handler
5+
Runtime: nodejs18.x
6+
Timeout: 60
7+
MemorySize: 2048
8+
AutoPublishAlias: live
9+
Layers:
10+
- Ref: StreamingLambdaLayer
11+
Environment:
12+
Variables:
13+
API_KEY: "{{resolve:secretsmanager:CHAT_GPT:SecretString:API_KEY}}"
14+
ORG_ID: "{{resolve:secretsmanager:CHAT_GPT:SecretString:ORG_ID}}"

‎backend/infra_resources/Resources/ChatGPTStreamingFunctionPublicAccess.yaml

+7
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
Type: AWS::Lambda::Permission
2+
Properties:
3+
Action: lambda:InvokeFunctionUrl
4+
FunctionName:
5+
Ref: ChatGPTStreamingFunction
6+
FunctionUrlAuthType: NONE
7+
Principal: "*"

‎backend/infra_resources/Resources/ChatGPTStreamingFunctionUrl.yaml

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
Type: AWS::Lambda::Url
2+
Properties:
3+
AuthType: NONE
4+
InvokeMode: RESPONSE_STREAM
5+
TargetFunctionArn:
6+
Ref: ChatGPTStreamingFunction

‎backend/infra_resources/Resources/CloudFrontDistribution.yaml

+41
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
Type: AWS::CloudFront::Distribution
2+
Properties:
3+
DistributionConfig:
4+
CustomErrorResponses:
5+
- ErrorCachingMinTTL: 300
6+
ErrorCode: 403
7+
ResponseCode: 200
8+
ResponsePagePath: /index.html
9+
DefaultCacheBehavior:
10+
AllowedMethods:
11+
- GET
12+
- HEAD
13+
- OPTIONS
14+
CachedMethods:
15+
- GET
16+
- HEAD
17+
- OPTIONS
18+
Compress: true
19+
DefaultTTL: 3600
20+
ForwardedValues:
21+
Cookies:
22+
Forward: none
23+
QueryString: false
24+
MaxTTL: 86400
25+
MinTTL: 60
26+
TargetOriginId: s3origin
27+
ViewerProtocolPolicy: allow-all
28+
DefaultRootObject: index.html
29+
Enabled: true
30+
HttpVersion: http2
31+
Origins:
32+
- DomainName:
33+
Fn::GetAtt:
34+
- S3Bucket
35+
- DomainName
36+
Id: s3origin
37+
S3OriginConfig:
38+
OriginAccessIdentity:
39+
Fn::Sub: >-
40+
origin-access-identity/cloudfront/${CloudFrontOriginAccessIdentity}
41+
PriceClass: PriceClass_All

‎backend/infra_resources/Resources/CloudFrontOriginAccessIdentity.yaml

+5
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
Type: AWS::CloudFront::CloudFrontOriginAccessIdentity
2+
Properties:
3+
CloudFrontOriginAccessIdentityConfig:
4+
Comment:
5+
Ref: S3Bucket

‎backend/infra_resources/Resources/DDBRegularLambda.yaml

+34
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
Type: AWS::Serverless::Function
2+
Properties:
3+
CodeUri: src/DDBRegularLambda
4+
Handler: index.handler
5+
Runtime: nodejs16.x
6+
Timeout: 60
7+
MemorySize: 1536
8+
Environment:
9+
Variables:
10+
SCAN_DB_NAME:
11+
Fn::Sub: ${AWS::StackName}-${DynamoDBScanName}
12+
Events:
13+
ApiEvent:
14+
Type: Api
15+
Properties:
16+
Path: /ddb_regular
17+
Method: get
18+
RestApiId:
19+
Ref: ResponseStreamingFunctionApi
20+
Connectors:
21+
ScanTableConnector:
22+
Properties:
23+
Destination:
24+
Id: ScanTable
25+
Permissions:
26+
- Write
27+
- Read
28+
QueryTableConnector:
29+
Properties:
30+
Destination:
31+
Id: QueryTable
32+
Permissions:
33+
- Write
34+
- Read

‎backend/infra_resources/Resources/DDBStreamingLambda.yaml

+27
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
Type: AWS::Serverless::Function
2+
Properties:
3+
CodeUri: src/DDBStreamingLambda
4+
Handler: index.handler
5+
Runtime: nodejs16.x
6+
Timeout: 60
7+
MemorySize: 2048
8+
AutoPublishAlias: live
9+
Environment:
10+
Variables:
11+
SCAN_DB_NAME:
12+
Fn::Sub: ${AWS::StackName}-${DynamoDBScanName}
13+
Connectors:
14+
ScanTableConnector:
15+
Properties:
16+
Destination:
17+
Id: ScanTable
18+
Permissions:
19+
- Write
20+
- Read
21+
QueryTableConnector:
22+
Properties:
23+
Destination:
24+
Id: QueryTable
25+
Permissions:
26+
- Write
27+
- Read

‎backend/infra_resources/Resources/DDBStreamingLambdaPublicAccess.yaml

+7
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
Type: AWS::Lambda::Permission
2+
Properties:
3+
Action: lambda:InvokeFunctionUrl
4+
FunctionName:
5+
Ref: DDBStreamingLambda
6+
FunctionUrlAuthType: NONE
7+
Principal: "*"

‎backend/infra_resources/Resources/DDBStreamingLambdaUrl.yaml

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
Type: AWS::Lambda::Url
2+
Properties:
3+
AuthType: NONE
4+
InvokeMode: RESPONSE_STREAM
5+
TargetFunctionArn:
6+
Ref: DDBStreamingLambda

‎backend/infra_resources/Resources/QueryTable.yaml

+15
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
Type: AWS::DynamoDB::Table
2+
Properties:
3+
TableName:
4+
Fn::Sub: ${AWS::StackName}-${DynamoDBQueryName}
5+
AttributeDefinitions:
6+
- AttributeName: id
7+
AttributeType: S
8+
- AttributeName: category
9+
AttributeType: S
10+
KeySchema:
11+
- AttributeName: category
12+
KeyType: HASH
13+
- AttributeName: id
14+
KeyType: RANGE
15+
BillingMode: PAY_PER_REQUEST

‎backend/infra_resources/Resources/ReadPolicy.yaml

+15
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
Type: AWS::S3::BucketPolicy
2+
Properties:
3+
Bucket:
4+
Ref: S3Bucket
5+
PolicyDocument:
6+
Statement:
7+
- Action: s3:GetObject
8+
Effect: Allow
9+
Resource:
10+
Fn::Sub: arn:aws:s3:::${S3Bucket}/*
11+
Principal:
12+
CanonicalUser:
13+
Fn::GetAtt:
14+
- CloudFrontOriginAccessIdentity
15+
- S3CanonicalUserId

‎backend/infra_resources/Resources/ResponseStreamingFunctionApi.yaml

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
Type: AWS::Serverless::Api
2+
Properties:
3+
StageName: prod

‎backend/infra_resources/Resources/S3Bucket.yaml

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
Type: AWS::S3::Bucket
2+
Properties:
3+
BucketName:
4+
Fn::Sub: ${AWS::AccountId}-${AWS::StackName}-${S3BucketName}
5+
WebsiteConfiguration:
6+
IndexDocument: index.html

0 commit comments

Comments
 (0)