Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: allow listing keys in kv stores #592

Merged
merged 24 commits into from
Feb 29, 2024
Merged

feat: allow listing keys in kv stores #592

Show file tree
Hide file tree
Changes from 22 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
71abc1d
add keys method to kv contract
jyecusch Feb 27, 2024
d918605
generate kv client with keys method
jyecusch Feb 27, 2024
7eca5c0
implement kv keys method for Azure storage tables
jyecusch Feb 27, 2024
8c21a95
ensure store name is provided
jyecusch Feb 27, 2024
146c81b
fix(azure): return permission denied errors from kv requests
jyecusch Feb 27, 2024
c392083
bump aws packages to ensure inter-compatibility
jyecusch Feb 27, 2024
7554b5a
implement keys method for aws
jyecusch Feb 27, 2024
93534f7
remove comment
jyecusch Feb 27, 2024
eda49f9
add scan permission for kv read
jyecusch Feb 27, 2024
5fc1deb
implement kv keys method for gcp
jyecusch Feb 28, 2024
7794b4d
add kvstore replacement contracts.
tjholm Feb 28, 2024
dcb8fbd
use unique method names.
tjholm Feb 28, 2024
f880751
Implement keyvalue compatibility layer
tjholm Feb 28, 2024
6d9c36a
remove keys request/response models.
tjholm Feb 28, 2024
0cebb80
finalise wrappers.
tjholm Feb 28, 2024
09e0184
add proto linting and regenerate sources.
tjholm Feb 28, 2024
dcaccb1
address lint issues.
tjholm Feb 28, 2024
27139bc
add log warning to compat layer for keyvaluestore
tjholm Feb 28, 2024
70d8649
update log warning
tjholm Feb 28, 2024
b2986f4
update kvstore message names
tjholm Feb 29, 2024
f9dc746
update message and method names
jyecusch Feb 29, 2024
871ebaf
format contracts
tjholm Feb 29, 2024
5b0fafd
add TODO and old keyvalue server registration.
tjholm Feb 29, 2024
da1e896
fix missing content in compat wrapper.
tjholm Feb 29, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 0 additions & 31 deletions cloud/aws/deploy/keyvalue.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,34 +60,3 @@ func (n *NitricAwsPulumiProvider) KeyValueStore(ctx *pulumi.Context, parent pulu

return err
}

// func NewDynamodbKeyValueStore(ctx *pulumi.Context, name string, args *DynamodbKeyValueStoreArgs, opts ...pulumi.ResourceOption) (*DynamodbKeyValueStore, error) {
// res := &DynamodbKeyValueStore{Name: name}

// err := ctx.RegisterComponentResource("nitric:keyvalue:Dynamodb", name, res, opts...)
// if err != nil {
// return nil, err
// }

// res.Table, err = dynamodb.NewTable(ctx, name, &dynamodb.TableArgs{
// Attributes: dynamodb.TableAttributeArray{
// &dynamodb.TableAttributeArgs{
// Name: pulumi.String("_pk"),
// Type: pulumi.String("S"),
// },
// &dynamodb.TableAttributeArgs{
// Name: pulumi.String("_sk"),
// Type: pulumi.String("S"),
// },
// },
// HashKey: pulumi.String("_pk"),
// RangeKey: pulumi.String("_sk"),
// BillingMode: pulumi.String("PAY_PER_REQUEST"),
// Tags: pulumi.ToStringMap(tags.Tags(args.StackID, name, resources.Collection)),
// })
// if err != nil {
// return nil, err
// }

// return res, nil
// }
1 change: 1 addition & 0 deletions cloud/aws/deploy/policy.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@ var awsActionsMap map[resourcespb.Action][]string = map[resourcespb.Action][]str
resourcespb.Action_KeyValueStoreRead: {
"dynamodb:GetItem",
"dynamodb:BatchGetItem",
"dynamodb:Scan", // required to scan keys
},
resourcespb.Action_KeyValueStoreWrite: {
"dynamodb:UpdateItem",
Expand Down
70 changes: 31 additions & 39 deletions cloud/aws/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,22 +6,22 @@ require (
github.com/avast/retry-go v3.0.0+incompatible
github.com/aws/aws-lambda-go v1.34.1
github.com/aws/aws-sdk-go v1.44.298
github.com/aws/aws-sdk-go-v2 v1.18.0
github.com/aws/aws-sdk-go-v2/config v1.18.4
github.com/aws/aws-sdk-go-v2/feature/dynamodb/attributevalue v1.10.6
github.com/aws/aws-sdk-go-v2/service/apigatewaymanagementapi v1.11.10
github.com/aws/aws-sdk-go-v2/service/apigatewayv2 v1.12.20
github.com/aws/aws-sdk-go-v2/service/dynamodb v1.17.8
github.com/aws/aws-sdk-go-v2/service/resourcegroupstaggingapi v1.13.21
github.com/aws/aws-sdk-go-v2/service/s3 v1.29.1
github.com/aws/aws-sdk-go-v2/service/secretsmanager v1.16.4
github.com/aws/aws-sdk-go-v2/service/sfn v1.14.3
github.com/aws/aws-sdk-go-v2/service/sns v1.18.3
github.com/aws/aws-sdk-go-v2/service/sqs v1.19.12
github.com/aws/smithy-go v1.13.5
github.com/aws/aws-sdk-go-v2 v1.25.2
github.com/aws/aws-sdk-go-v2/config v1.27.4
github.com/aws/aws-sdk-go-v2/feature/dynamodb/attributevalue v1.13.6
github.com/aws/aws-sdk-go-v2/feature/dynamodb/expression v1.7.6
github.com/aws/aws-sdk-go-v2/service/apigatewaymanagementapi v1.19.1
github.com/aws/aws-sdk-go-v2/service/apigatewayv2 v1.20.1
github.com/aws/aws-sdk-go-v2/service/dynamodb v1.30.1
github.com/aws/aws-sdk-go-v2/service/resourcegroupstaggingapi v1.21.1
github.com/aws/aws-sdk-go-v2/service/s3 v1.51.1
github.com/aws/aws-sdk-go-v2/service/secretsmanager v1.28.1
github.com/aws/aws-sdk-go-v2/service/sfn v1.26.1
github.com/aws/aws-sdk-go-v2/service/sns v1.29.1
github.com/aws/aws-sdk-go-v2/service/sqs v1.31.1
github.com/aws/smithy-go v1.20.1
github.com/getkin/kin-openapi v0.113.0
github.com/golang/mock v1.6.0
github.com/golang/protobuf v1.5.3
github.com/golangci/golangci-lint v1.56.1
github.com/google/addlicense v1.1.1
github.com/google/uuid v1.5.0
Expand All @@ -39,11 +39,9 @@ require (
github.com/samber/lo v1.38.1
github.com/uw-labs/lichen v0.1.7
github.com/valyala/fasthttp v1.45.0
go.opentelemetry.io/contrib/detectors/aws/lambda v0.36.3
go.opentelemetry.io/contrib/instrumentation/github.com/aws/aws-sdk-go-v2/otelaws v0.36.4
go.opentelemetry.io/contrib/propagators/aws v1.11.0
go.opentelemetry.io/otel v1.11.2
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.11.2
go.opentelemetry.io/otel/sdk v1.11.2
google.golang.org/grpc v1.59.0
google.golang.org/protobuf v1.31.0
Expand Down Expand Up @@ -76,22 +74,22 @@ require (
github.com/ashanbrown/forbidigo v1.6.0 // indirect
github.com/ashanbrown/makezero v1.1.1 // indirect
github.com/atotto/clipboard v0.1.4 // indirect
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.4.9 // indirect
github.com/aws/aws-sdk-go-v2/credentials v1.13.4 // indirect
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.12.20 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.33 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.4.27 // indirect
github.com/aws/aws-sdk-go-v2/internal/ini v1.3.27 // indirect
github.com/aws/aws-sdk-go-v2/internal/v4a v1.0.16 // indirect
github.com/aws/aws-sdk-go-v2/service/dynamodbstreams v1.13.26 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.9.11 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.1.20 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/endpoint-discovery v1.7.20 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.9.20 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.13.19 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.11.26 // indirect
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.13.9 // indirect
github.com/aws/aws-sdk-go-v2/service/sts v1.17.6 // indirect
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.6.1 // indirect
github.com/aws/aws-sdk-go-v2/credentials v1.17.4 // indirect
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.15.2 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.2 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.2 // indirect
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.0 // indirect
github.com/aws/aws-sdk-go-v2/internal/v4a v1.3.2 // indirect
github.com/aws/aws-sdk-go-v2/service/dynamodbstreams v1.20.1 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.11.1 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.3.2 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/endpoint-discovery v1.9.2 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.11.2 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.17.2 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.20.1 // indirect
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.23.1 // indirect
github.com/aws/aws-sdk-go-v2/service/sts v1.28.1 // indirect
github.com/aymanbagabas/go-osc52/v2 v2.0.1 // indirect
github.com/beorn7/perks v1.0.1 // indirect
github.com/bkielbasa/cyclop v1.2.1 // indirect
Expand All @@ -105,7 +103,6 @@ require (
github.com/butuzov/mirror v1.1.0 // indirect
github.com/catenacyber/perfsprint v0.6.0 // indirect
github.com/ccojocar/zxcvbn-go v1.0.1 // indirect
github.com/cenkalti/backoff/v4 v4.2.0 // indirect
github.com/cespare/xxhash/v2 v2.2.0 // indirect
github.com/charithe/durationcheck v0.0.10 // indirect
github.com/charmbracelet/bubbles v0.16.1 // indirect
Expand Down Expand Up @@ -160,6 +157,7 @@ require (
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang/glog v1.1.2 // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/golang/protobuf v1.5.3 // indirect
github.com/golangci/check v0.0.0-20180506172741-cfe4005ccda2 // indirect
github.com/golangci/dupl v0.0.0-20180902072040-3e9179ac440a // indirect
github.com/golangci/go-misc v0.0.0-20220329215616-d24fe342adfe // indirect
Expand All @@ -176,7 +174,6 @@ require (
github.com/gostaticanalysis/comment v1.4.2 // indirect
github.com/gostaticanalysis/forcetypeassert v0.1.0 // indirect
github.com/gostaticanalysis/nilerr v0.1.1 // indirect
github.com/grpc-ecosystem/grpc-gateway/v2 v2.11.3 // indirect
github.com/grpc-ecosystem/grpc-opentracing v0.0.0-20180507213350-8e809c8a8645 // indirect
github.com/hashicorp/errwrap v1.1.0 // indirect
github.com/hashicorp/go-multierror v1.1.1 // indirect
Expand Down Expand Up @@ -317,10 +314,7 @@ require (
go-simpler.org/musttag v0.8.0 // indirect
go-simpler.org/sloglint v0.4.0 // indirect
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.36.4 // indirect
go.opentelemetry.io/otel/exporters/otlp/internal/retry v1.11.2 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.11.2 // indirect
go.opentelemetry.io/otel/trace v1.11.2 // indirect
go.opentelemetry.io/proto/otlp v0.19.0 // indirect
go.uber.org/atomic v1.10.0 // indirect
go.uber.org/multierr v1.8.0 // indirect
go.uber.org/zap v1.24.0 // indirect
Expand All @@ -334,8 +328,6 @@ require (
golang.org/x/term v0.16.0 // indirect
golang.org/x/text v0.14.0 // indirect
golang.org/x/tools v0.17.0 // indirect
google.golang.org/genproto v0.0.0-20231106174013-bbf56f31fb17 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20231106174013-bbf56f31fb17 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20231120223509-83a465c0220f // indirect
gopkg.in/ini.v1 v1.67.0 // indirect
gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 // indirect
Expand Down
Loading
Loading