Add Redpanda module (#1058)

Co-authored-by: Manuel de la Peña <[email protected]>

.github/dependabot.yml

@@ -97,13 +97,13 @@ updates:
       open-pull-requests-limit: 3
       rebase-strategy: disabled
     - package-ecosystem: gomod
-      directory: /modules/postgres
+      directory: /modules/neo4j
       schedule:
         interval: monthly
       open-pull-requests-limit: 3
       rebase-strategy: disabled
     - package-ecosystem: gomod
-      directory: /modules/neo4j
+      directory: /modules/postgres
       schedule:
         interval: monthly
       open-pull-requests-limit: 3
@@ -120,6 +120,12 @@ updates:
         interval: monthly
       open-pull-requests-limit: 3
       rebase-strategy: disabled
+    - package-ecosystem: gomod
+      directory: /modules/redpanda
+      schedule:
+        interval: monthly
+      open-pull-requests-limit: 3
+      rebase-strategy: disabled
     - package-ecosystem: gomod
       directory: /modules/vault
       schedule:

.github/workflows/module-redpanda.yml

@@ -0,0 +1,56 @@
+name: Redpanda module pipeline
+
+on:
+  push:
+    paths-ignore:
+    - 'mkdocs.yml'
+    - 'docs/**'
+    - 'README.md'
+  pull_request:
+    paths-ignore:
+    - 'mkdocs.yml'
+    - 'docs/**'
+    - 'README.md'
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.head_ref || github.sha }}
+  cancel-in-progress: true
+
+jobs:
+  test-redpanda:
+    strategy:
+      matrix:
+        go-version: [1.19.x, 1.x]
+    runs-on: "ubuntu-latest"
+    steps:
+
+      - name: Set up Go
+        uses: actions/setup-go@v3
+        with:
+          go-version: ${{ matrix.go-version }}
+        id: go
+
+      - name: Check out code into the Go module directory
+        uses: actions/checkout@v3
+
+      - name: modVerify
+        working-directory: ./modules/redpanda
+        run: go mod verify
+
+      - name: modTidy
+        working-directory: ./modules/redpanda
+        run: make tools-tidy
+
+      - name: gotestsum
+        working-directory: ./modules/redpanda
+        run: make test-unit
+
+      - name: Run checker
+        run: |
+          ./scripts/check_environment.sh
+
+      - name: Test Summary
+        uses: test-summary/action@4ee9ece4bca777a38f05c8fc578ac2007fe266f7
+        with:
+          paths: "**/TEST-redpanda*.xml"
+        if: always()

docs/modules/redpanda.md

@@ -0,0 +1,99 @@
+# Redpanda
+
+Not available until the next release of testcontainers-go <a href="https://github.com/testcontainers/testcontainers-go"><span class="tc-version">:material-tag: main</span></a>
+
+## Introduction
+
+Redpanda is a streaming data platform for developers. Kafka API compatible. 10x faster. No ZooKeeper. No JVM!
+This Testcontainers module provides three APIs:
+
+- Kafka API
+- Schema Registry API
+- Redpanda Admin API
+
+## Adding this module to your project dependencies
+
+Please run the following command to add the Redpanda module to your Go dependencies:
+
+```
+go get github.com/testcontainers/testcontainers-go/modules/redpanda
+```
+
+## Usage example
+
+<!--codeinclude-->
+[Creating a Redpanda container](../../modules/redpanda/redpanda_test.go) inside_block:redpandaCreateContainer
+<!--/codeinclude-->
+
+## Module reference
+
+The Redpanda module exposes one entrypoint function to create the Redpanda container, and this function receives two parameters:
+
+```golang
+func RunContainer(ctx context.Context, opts ...testcontainers.ContainerCustomizer) (*RedpandaContainer, error)
+```
+
+- `context.Context`, the Go context.
+- `testcontainers.ContainerCustomizer`, a variadic argument for passing options.
+
+### Container Options
+
+When starting the Redpanda container, you can pass options in a variadic way to configure it.
+
+#### Image
+
+If you need to set a different Redpanda Docker image, you can use `testcontainers.WithImage` with a valid Docker image
+for Redpanda. E.g. `testcontainers.WithImage("docker.redpanda.com/redpandadata/redpanda:v23.1.7")`.
+
+#### Wait Strategies
+
+If you need to set a different wait strategy for Redpanda, you can use `testcontainers.WithWaitStrategy` with a valid wait strategy
+for Redpanda.
+
+!!!info
+    The default deadline for the wait strategy is 60 seconds.
+
+At the same time, it's possible to set a wait strategy and a custom deadline with `testcontainers.WithWaitStrategyAndDeadline`.
+
+#### Docker type modifiers
+
+If you need an advanced configuration for Redpanda, you can leverage the following Docker type modifiers:
+
+- `testcontainers.WithConfigModifier`
+- `testcontainers.WithHostConfigModifier`
+- `testcontainers.WithEndpointSettingsModifier`
+
+Please read the [Create containers: Advanced Settings](../features/creating_container.md#advanced-settings) documentation for more information.
+
+### Container Methods
+
+The Redpanda container exposes the following methods:
+
+#### KafkaSeedBroker
+
+KafkaSeedBroker returns the seed broker that should be used for connecting
+to the Kafka API with your Kafka client. It'll be returned in the format:
+"host:port" - for example: "localhost:55687".
+
+<!--codeinclude-->
+[Get Kafka seed broker](../../modules/redpanda/redpanda_test.go) inside_block:kafkaSeedBroker
+<!--/codeinclude-->
+
+#### SchemaRegistryAddress
+
+SchemaRegistryAddress returns the address to the schema registry API. This
+is an HTTP-based API and thus the returned format will be: http://host:port.
+
+<!--codeinclude-->
+[Get schema registry address](../../modules/redpanda/redpanda_test.go) inside_block:schemaRegistryAddress
+<!--/codeinclude-->
+
+
+#### AdminAPIAddress
+
+AdminAPIAddress returns the address to the Redpanda Admin API. This
+is an HTTP-based API and thus the returned format will be: http://host:port.
+
+<!--codeinclude-->
+[Get admin API address](../../modules/redpanda/redpanda_test.go) inside_block:adminAPIAddress
+<!--/codeinclude-->

mkdocs.yml

@@ -60,10 +60,11 @@ nav:
         - modules/couchbase.md
         - modules/localstack.md
         - modules/mysql.md
-        - modules/postgres.md
         - modules/neo4j.md
+        - modules/postgres.md
         - modules/pulsar.md
         - modules/redis.md
+        - modules/redpanda.md
         - modules/vault.md
     - Examples:
         - examples/index.md

modules/redpanda/Makefile

@@ -0,0 +1,5 @@
+include ../../commons-test.mk
+
+.PHONY: test
+test:
+	$(MAKE) test-redpanda

modules/redpanda/admin_api.go

@@ -0,0 +1,63 @@
+package redpanda
+
+import (
+	"bytes"
+	"context"
+	"encoding/json"
+	"fmt"
+	"io"
+	"net/http"
+	"net/url"
+)
+
+type AdminAPIClient struct {
+	BaseURL string
+}
+
+func NewAdminAPIClient(baseURL string) *AdminAPIClient {
+	return &AdminAPIClient{BaseURL: baseURL}
+}
+
+type createUserRequest struct {
+	User      string `json:"username,omitempty"`
+	Password  string `json:"password"`
+	Algorithm string `json:"algorithm"`
+}
+
+func (cl *AdminAPIClient) CreateUser(ctx context.Context, username, password string) error {
+	userReq := createUserRequest{
+		User:      username,
+		Password:  password,
+		Algorithm: "SCRAM-SHA-256",
+	}
+	jsonReq, err := json.Marshal(userReq)
+	if err != nil {
+		return fmt.Errorf("failed to marshal create user request: %w", err)
+	}
+
+	endpoint, err := url.JoinPath(cl.BaseURL, "/v1/security/users")
+	if err != nil {
+		return fmt.Errorf("failed to join url path: %w", err)
+	}
+
+	req, err := http.NewRequestWithContext(ctx, http.MethodPost, endpoint, bytes.NewBuffer(jsonReq))
+	if err != nil {
+		return fmt.Errorf("failed to build http request: %w", err)
+	}
+	req.Header.Set("Content-Type", "application/json")
+
+	resp, err := http.DefaultClient.Do(req)
+	if err != nil {
+		return fmt.Errorf("request failed: %w", err)
+	}
+	if resp.StatusCode != http.StatusOK {
+		body, readErr := io.ReadAll(resp.Body)
+		if readErr != nil {
+			return fmt.Errorf("unexpected status code: %d", resp.StatusCode)
+		}
+
+		return fmt.Errorf("unexpected status code in response: %d. Response body is: %q", resp.StatusCode, body)
+	}
+
+	return nil
+}

modules/redpanda/go.mod

@@ -0,0 +1,63 @@
+module github.com/testcontainers/testcontainers-go/modules/redpanda
+
+go 1.19
+
+require (
+	github.com/docker/go-connections v0.4.0
+	github.com/stretchr/testify v1.8.2
+	github.com/testcontainers/testcontainers-go v0.19.0
+	github.com/twmb/franz-go v1.13.2
+	github.com/twmb/franz-go/pkg/kadm v1.7.0
+	gotest.tools/gotestsum v1.9.0
+)
+
+require (
+	github.com/Azure/go-ansiterm v0.0.0-20230124172434-306776ec8161 // indirect
+	github.com/Microsoft/go-winio v0.6.0 // indirect
+	github.com/cenkalti/backoff/v4 v4.2.0 // indirect
+	github.com/containerd/containerd v1.6.19 // indirect
+	github.com/cpuguy83/dockercfg v0.3.1 // indirect
+	github.com/davecgh/go-spew v1.1.1 // indirect
+	github.com/dnephin/pflag v1.0.7 // indirect
+	github.com/docker/distribution v2.8.1+incompatible // indirect
+	github.com/docker/docker v23.0.3+incompatible // indirect
+	github.com/docker/go-units v0.5.0 // indirect
+	github.com/fatih/color v1.14.1 // indirect
+	github.com/fsnotify/fsnotify v1.5.4 // indirect
+	github.com/gogo/protobuf v1.3.2 // indirect
+	github.com/golang/protobuf v1.5.2 // indirect
+	github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 // indirect
+	github.com/google/uuid v1.3.0 // indirect
+	github.com/imdario/mergo v0.3.12 // indirect
+	github.com/klauspost/compress v1.16.4 // indirect
+	github.com/magiconair/properties v1.8.7 // indirect
+	github.com/mattn/go-colorable v0.1.13 // indirect
+	github.com/mattn/go-isatty v0.0.18 // indirect
+	github.com/moby/patternmatcher v0.5.0 // indirect
+	github.com/moby/sys/sequential v0.5.0 // indirect
+	github.com/moby/term v0.0.0-20221205130635-1aeaba878587 // indirect
+	github.com/morikuni/aec v1.0.0 // indirect
+	github.com/opencontainers/go-digest v1.0.0 // indirect
+	github.com/opencontainers/image-spec v1.1.0-rc2 // indirect
+	github.com/opencontainers/runc v1.1.5 // indirect
+	github.com/pierrec/lz4/v4 v4.1.17 // indirect
+	github.com/pkg/errors v0.9.1 // indirect
+	github.com/pmezard/go-difflib v1.0.0 // indirect
+	github.com/sirupsen/logrus v1.9.0 // indirect
+	github.com/twmb/franz-go/pkg/kmsg v1.5.0 // indirect
+	golang.org/x/crypto v0.7.0 // indirect
+	golang.org/x/mod v0.8.0 // indirect
+	golang.org/x/net v0.8.0 // indirect
+	golang.org/x/sync v0.1.0 // indirect
+	golang.org/x/sys v0.7.0 // indirect
+	golang.org/x/term v0.6.0 // indirect
+	golang.org/x/time v0.3.0 // indirect
+	golang.org/x/tools v0.6.0 // indirect
+	google.golang.org/genproto v0.0.0-20220617124728-180714bec0ad // indirect
+	google.golang.org/grpc v1.47.0 // indirect
+	google.golang.org/protobuf v1.28.1 // indirect
+	gopkg.in/yaml.v3 v3.0.1 // indirect
+	gotest.tools/v3 v3.4.0 // indirect
+)
+
+replace github.com/testcontainers/testcontainers-go => ../..