Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add localstack module #800

Merged
merged 60 commits into from
Feb 14, 2023
Merged
Show file tree
Hide file tree
Changes from 49 commits
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
3fab103
chore: add localstack basic example
mdelapenya Jan 30, 2023
6074b03
chore: add wait for log
mdelapenya Jan 30, 2023
21493cb
chore: add Docker socket binding
mdelapenya Jan 30, 2023
5461f69
chore: support defining legacy mode
mdelapenya Jan 30, 2023
a477156
chore: support passing services and configuring them
mdelapenya Feb 1, 2023
c8b67e5
chore: simplify the creation of a localstack container request
mdelapenya Feb 1, 2023
a0eaf50
chore: support configuring the AWS region
mdelapenya Feb 1, 2023
59b8cd7
chore: expose localstack container
mdelapenya Feb 1, 2023
5013670
chore: support for retrieving the endpoint of a given service
mdelapenya Feb 1, 2023
595f73c
chore: store enabled services in the localstack container
mdelapenya Feb 1, 2023
6eab52d
chore: store the region in the localstack container
mdelapenya Feb 1, 2023
3b59457
chore: apply default region when needed
mdelapenya Feb 1, 2023
de6fea7
chore: support passing version and legacyMode as functional options
mdelapenya Feb 1, 2023
4c863d9
chore: support for overriding the container request with a functional…
mdelapenya Feb 2, 2023
902869b
chore: remove useless method for retrieving the internal endpoint of …
mdelapenya Feb 2, 2023
f3c1754
chore: simplify unit tests
mdelapenya Feb 2, 2023
5ed9cc5
chore: create aws session using v1
mdelapenya Feb 2, 2023
bffe24a
chore: add S3 tests
mdelapenya Feb 2, 2023
c94594f
chore: rename start localstack function
mdelapenya Feb 2, 2023
6c83776
chore: define default functions as vars
mdelapenya Feb 2, 2023
fa939b9
chore: add tests for legacy mode
mdelapenya Feb 2, 2023
6e131ba
chore: move the Session code to the tests
mdelapenya Feb 2, 2023
900b825
docs: use S3 in the docs
mdelapenya Feb 2, 2023
1adcfa2
Merge branch 'main' into localstack-module
mdelapenya Feb 7, 2023
5bd4138
fix: read the expected daemon host from the provider
mdelapenya Feb 7, 2023
ddb3f83
chore: move v1 tests to its own test package
mdelapenya Feb 7, 2023
0f90131
feat: add example test for S3 using v2
mdelapenya Feb 7, 2023
178b9e9
chore: store localstack credentials in a struct
mdelapenya Feb 7, 2023
23d553c
feat: create a functional opt for passing the AWS credentials
mdelapenya Feb 7, 2023
2db880d
chore: define defaults for the AWS credentials
mdelapenya Feb 7, 2023
c9c551b
chore: include a test for overriding the image
mdelapenya Feb 8, 2023
87ec202
chore: remove withCredentials
mdelapenya Feb 8, 2023
11a6da9
chore: remove withRegion
mdelapenya Feb 8, 2023
767a073
chore: remove withVersion
mdelapenya Feb 8, 2023
4d70f27
chore: rename configure method
mdelapenya Feb 8, 2023
6d51794
chore: reorder container configuration
mdelapenya Feb 8, 2023
a79b3d2
chore: move to types
mdelapenya Feb 8, 2023
2639aa5
chor: export structs
mdelapenya Feb 8, 2023
8cbf21b
docs: document exported functions
mdelapenya Feb 8, 2023
7be6c70
docs: document how to create localstack container
mdelapenya Feb 8, 2023
91139cf
docs: reorder
mdelapenya Feb 8, 2023
55ce21c
docs: document containerRequest override
mdelapenya Feb 8, 2023
97abda8
docs: document the container functional opts
mdelapenya Feb 8, 2023
dfee950
chore: be more explicit in comment
mdelapenya Feb 9, 2023
8f1a256
chore: convert into real module
mdelapenya Feb 10, 2023
9f01c71
chore: remove support for legacy mode
mdelapenya Feb 10, 2023
4a435a3
chore: move tests
mdelapenya Feb 10, 2023
40e4f65
fix: update module paths
mdelapenya Feb 10, 2023
5ce1ce6
fix: update tests after moving an example to modules
mdelapenya Feb 10, 2023
bb4e2a8
chore: bump default version to 1.3.1
mdelapenya Feb 13, 2023
f2026e9
chore: remove Port from the Service struct
mdelapenya Feb 13, 2023
f3da875
fix: wrong relative path to the docs
mdelapenya Feb 14, 2023
cca2a3b
chore: remove any code to Services
mdelapenya Feb 14, 2023
87b2021
chore: adjust unit tests
mdelapenya Feb 14, 2023
c7a7896
docs: include tests
mdelapenya Feb 14, 2023
a6b26dd
chore: use waitFor HTTP health
mdelapenya Feb 14, 2023
6ccfb56
Merge branch 'main' into localstack-module
mdelapenya Feb 14, 2023
8bba30b
chore: bump to latest version
mdelapenya Feb 14, 2023
4894746
chore: go mod examples
mdelapenya Feb 14, 2023
b325ffc
chore: rename GH action workflow file
mdelapenya Feb 14, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,12 @@ updates:
interval: weekly
open-pull-requests-limit: 3
rebase-strategy: disabled
- package-ecosystem: gomod
directory: /modules/localstack
schedule:
interval: weekly
open-pull-requests-limit: 3
rebase-strategy: disabled
- package-ecosystem: gomod
directory: /examples/bigtable
schedule:
Expand Down
46 changes: 46 additions & 0 deletions .github/workflows/localstack-example.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
name: LocalStack example pipeline
mdelapenya marked this conversation as resolved.
Show resolved Hide resolved

on: [push, pull_request]

concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.sha }}
cancel-in-progress: true

jobs:
test-localstack:
strategy:
matrix:
go-version: [1.18.x, 1.x]
runs-on: "ubuntu-latest"
steps:

- name: Set up Go
uses: actions/setup-go@v3
with:
go-version: ${{ matrix.go-version }}
id: go

- name: Check out code into the Go module directory
uses: actions/checkout@v3

- name: modVerify
working-directory: ./modules/localstack
run: go mod verify

- name: modTidy
working-directory: ./modules/localstack
run: make tools-tidy

- name: gotestsum
working-directory: ./modules/localstack
run: make test-unit

- name: Run checker
run: |
./scripts/check_environment.sh

- name: Test Summary
uses: test-summary/action@4ee9ece4bca777a38f05c8fc578ac2007fe266f7
with:
paths: "**/TEST-localstack*.xml"
if: always()
134 changes: 134 additions & 0 deletions docs/examples/localstack.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
# LocalStack
mdelapenya marked this conversation as resolved.
Show resolved Hide resolved

The Testcontainers module for [LocalStack](http://localstack.cloud/) is _"a fully functional local AWS cloud stack"_, to develop and test your cloud and serverless apps without actually using the cloud.

## Adding this module to your project dependencies

Please run the following command to add the LocalStack module to your Go dependencies:

```
go get github.com/testcontainers/testcontainers-go/modules/localstack
```

## Usage example

Running LocalStack as a stand-in for multiple AWS services during a test:

<!--codeinclude-->
[Creating a LocalStack container](../../modules/localstack/v1/s3_test.go) inside_block:localStackCreateContainer
<!--/codeinclude-->

Environment variables listed in [Localstack's README](https://github.com/localstack/localstack#configurations) may be used to customize Localstack's configuration.
Use the `OverrideContainerRequest` option when creating the `LocalStackContainer` to apply configuration settings.

## Creating a client using the AWS SDK for Go

### Version 1

<!--codeinclude-->
[Test for a LocalStack container, usinv AWS SDK v1](../../modules/localstack/v1/s3_test.go) inside_block:awsSDKClientV1
<!--/codeinclude-->

For further reference on the SDK v1, please check out the AWS docs [here](https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/setting-up.html).

### Version 2

<!--codeinclude-->
[Test for a LocalStack container, usinv AWS SDK v2](../../modules/localstack/v2/s3_test.go) inside_block:awsSDKClientV2
<!--/codeinclude-->

For further reference on the SDK v2, please check out the AWS docs [here](https://aws.github.io/aws-sdk-go-v2/docs/getting-started)

## `HOSTNAME_EXTERNAL` and hostname-sensitive services

Some Localstack APIs, such as SQS, require the container to be aware of the hostname that it is accessible on - for example, for construction of queue URLs in responses.

Testcontainers will inform Localstack of the best hostname automatically, using the `HOSTNAME_EXTERNAL` environment variable:

* when running the Localstack container directly without a custom network defined, it is expected that all calls to the container will be from the test host. As such, the container address will be used (typically localhost or the address where the Docker daemon is running).

<!--codeinclude-->
[Localstack container running without a custom network](../../modules/localstack/localstack_legacy_mode_test.go) inside_block:withoutNetwork
<!--/codeinclude-->

* when running the Localstack container [with a custom network defined](/features/networking/#advanced-networking), it is expected that all calls to the container will be **from other containers on that network**. `HOSTNAME_EXTERNAL` will be set to the *last* network alias that has been configured for the Localstack container.

<!--codeinclude-->
[Localstack container running with a custom network](../../modules/localstack/localstack_test.go) inside_block:withNetwork
<!--/codeinclude-->

* Other usage scenarios, such as where the Localstack container is used from both the test host and containers on a custom network are not automatically supported. If you have this use case, you should set `HOSTNAME_EXTERNAL` manually.

## Module reference

The LocalStack module exposes one single function to create containers, and this function receives three parameters:

- `context.Context`
- `OverrideContainerRequestOption`
- a variadic argument of `LocalStackContainerOption`

### OverrideContainerRequestOption

The `OverrideContainerRequestOption` functional option represents a way to override the default LocalStack container request:

<!--codeinclude-->
[Default container request](../../modules/localstack/localstack.go) inside_block:defaultContainerRequest
<!--/codeinclude-->

With simply passing your own instance of an `OverrideContainerRequestOption` type to the `StartContainer` function, you'll be able to configure the LocalStack container with your own needs.

In the following example you check how it's possible to set certain environment variables that are needed by the tests. Besides, the container runs in a separate Docker network with an alias:

<!--codeinclude-->
[Overriding the default container request](../../modules/localstack/localstack_test.go) inside_block:withNetwork
<!--/codeinclude-->

If you do not need to override the container request, you can pass `nil` or the `NoopOverrideContainerRequest` instance, which is exposed as a helper for this reason.

<!--codeinclude-->
[Skip overriding the default container request](../../modules/localstack/localstack_test.go) inside_block:noopOverrideContainerRequest
<!--/codeinclude-->

### LocalStackContainerOption, variadic argument

#### WithServices

The `WithServices` functional option represents a way to pass as many AWS services as needed, in a variadic manner:

<!--codeinclude-->
[Passing AWS services to LocalStack](../../modules/localstack/v1/s3_test.go) inside_block:localStackCreateContainer
<!--/codeinclude-->

Using this method will internally populate the `SERVICES` environment variable, alongside the exposed ports of each service.

### Available AWS Services

The LocalStack module supports the following AWS services:

- APIGateway
- CloudFormation
- CloudWatch
- CloudWatchLogs
- DynamoDB
- DynamoDBStreams
- EC2
- Firehose
- IAM
- KMS
- Kinesis
- Lambda
- Redshift
- Route53
- S3
- SES
- SNS
- SQS
- SSM
- STS
- SecretsManager
- StepFunctions
mdelapenya marked this conversation as resolved.
Show resolved Hide resolved

!!!info

At this moment, the tests for the module include tests for the S3 service, only. They live in two different Go packages of the LocalStack module,
`v1` and `v2`, where it'll be easier to add more examples for the rest of services.
2 changes: 1 addition & 1 deletion examples/dependabot_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ func TestExamplesHasDependabotEntry(t *testing.T) {
require.NoError(t, err)

// we have to exclude the main and compose modules from the examples updates
assert.Equal(t, len(exampleUpdates)-2, len(examples))
assert.Equal(t, len(exampleUpdates)-3, len(examples))

// all example modules exist in the dependabot updates
for _, example := range examples {
Expand Down
2 changes: 1 addition & 1 deletion examples/mkdocs_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ func TestExamples(t *testing.T) {
require.NoError(t, err)

// we have to remove the index.md file from the examples docs
assert.Equal(t, len(examplesDocs)-1, len(examples))
assert.Equal(t, len(examplesDocs)-2, len(examples))

// all example modules exist in the documentation
for _, example := range examples {
Expand Down
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ nav:
- examples/consul.md
- examples/datastore.md
- examples/firestore.md
- modules/localstack.md
- examples/mongodb.md
- examples/mysql.md
- examples/nginx.md
Expand Down
5 changes: 5 additions & 0 deletions modules/localstack/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
include ../../commons-test.mk

.PHONY: test
test:
$(MAKE) test-localstack
79 changes: 79 additions & 0 deletions modules/localstack/go.mod
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
module github.com/testcontainers/testcontainers-go/modules/localstack

go 1.18

require (
github.com/aws/aws-sdk-go v1.44.192
github.com/aws/aws-sdk-go-v2 v1.17.4
github.com/aws/aws-sdk-go-v2/config v1.18.12
github.com/aws/aws-sdk-go-v2/credentials v1.13.12
github.com/aws/aws-sdk-go-v2/service/s3 v1.30.2
github.com/docker/go-connections v0.4.0
github.com/imdario/mergo v0.3.12
github.com/stretchr/testify v1.8.1
github.com/testcontainers/testcontainers-go v0.17.0
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4
gotest.tools/gotestsum v1.9.0
)

require (
github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 // indirect
github.com/Microsoft/go-winio v0.5.2 // indirect
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.4.10 // indirect
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.12.22 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.28 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.4.22 // indirect
github.com/aws/aws-sdk-go-v2/internal/ini v1.3.29 // indirect
github.com/aws/aws-sdk-go-v2/internal/v4a v1.0.19 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.9.11 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.1.23 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.9.22 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.13.22 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.12.1 // indirect
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.14.1 // indirect
github.com/aws/aws-sdk-go-v2/service/sts v1.18.3 // indirect
github.com/aws/smithy-go v1.13.5 // indirect
github.com/cenkalti/backoff/v4 v4.2.0 // indirect
github.com/containerd/containerd v1.6.16 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/dnephin/pflag v1.0.7 // indirect
github.com/docker/distribution v2.8.1+incompatible // indirect
github.com/docker/docker v20.10.20+incompatible // indirect
github.com/docker/go-units v0.5.0 // indirect
github.com/fatih/color v1.13.0 // indirect
github.com/fsnotify/fsnotify v1.5.4 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang/protobuf v1.5.2 // indirect
github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 // indirect
github.com/google/uuid v1.3.0 // indirect
github.com/jmespath/go-jmespath v0.4.0 // indirect
github.com/klauspost/compress v1.15.9 // indirect
github.com/magiconair/properties v1.8.7 // indirect
github.com/mattn/go-colorable v0.1.12 // indirect
github.com/mattn/go-isatty v0.0.16 // indirect
github.com/moby/patternmatcher v0.5.0 // indirect
github.com/moby/sys/sequential v0.5.0 // indirect
github.com/moby/term v0.0.0-20221128092401-c43b287e0e0f // indirect
github.com/morikuni/aec v1.0.0 // indirect
github.com/opencontainers/go-digest v1.0.0 // indirect
github.com/opencontainers/image-spec v1.1.0-rc2 // indirect
github.com/opencontainers/runc v1.1.3 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/sirupsen/logrus v1.9.0 // indirect
golang.org/x/net v0.1.0 // indirect
golang.org/x/sync v0.1.0 // indirect
golang.org/x/sys v0.4.0 // indirect
golang.org/x/term v0.1.0 // indirect
golang.org/x/tools v0.1.12 // indirect
google.golang.org/genproto v0.0.0-20220617124728-180714bec0ad // indirect
google.golang.org/grpc v1.47.0 // indirect
google.golang.org/protobuf v1.28.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
gotest.tools/v3 v3.4.0 // indirect
)

replace (
github.com/docker/docker => github.com/docker/docker v20.10.3-0.20221013203545-33ab36d6b304+incompatible // 22.06 branch
github.com/testcontainers/testcontainers-go => ../..
)
Loading