Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add yugabytedb module #2825

Merged
merged 23 commits into from
Oct 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
7436768
feat: add yugabytedb module
henripqt Oct 14, 2024
6f9a7ef
Merge branch 'main' into feat/add-yugabytedb-module
henripqt Oct 15, 2024
07674f5
fix: documentation links
henripqt Oct 15, 2024
084cde9
fix: codeinclude target
henripqt Oct 15, 2024
83a407f
feat: add test example for custom methods
henripqt Oct 15, 2024
6b09e71
Update modules/yugabytedb/yugabytedb_test.go
henripqt Oct 15, 2024
a51e4dd
Update modules/yugabytedb/yugabytedb.go
henripqt Oct 15, 2024
13cac89
Update modules/yugabytedb/yugabytedb_test.go
henripqt Oct 15, 2024
cad9e6a
fix: attended review issues
henripqt Oct 15, 2024
139f4d9
Update modules/yugabytedb/examples_test.go
henripqt Oct 16, 2024
2644453
fix: cleanup and review adjustements
henripqt Oct 16, 2024
4dea5ea
fix: remove test in comments and renamed struct
henripqt Oct 16, 2024
c556bd9
feat: add documentation for options
henripqt Oct 17, 2024
dcd9146
Merge branch 'main' into feat/add-yugabytedb-module
henripqt Oct 17, 2024
29ce547
Update modules/yugabytedb/yugabytedb_test.go
henripqt Oct 17, 2024
cb34dee
Update modules/yugabytedb/yugabytedb_test.go
henripqt Oct 17, 2024
fc9fc9c
fix: add additional wait strategy for data constraint log
henripqt Oct 17, 2024
d9d7aef
fix: remove customer method for ycql config and update test code
henripqt Oct 17, 2024
cd26ece
chore: run mod tidy
mdelapenya Oct 18, 2024
661dfec
chore: rename testable example to fix lint
mdelapenya Oct 18, 2024
d253ff8
fix: wrong cluster addr in example test
henripqt Oct 18, 2024
f31f993
Merge branch 'main' into feat/add-yugabytedb-module
henripqt Oct 18, 2024
2d7a7c7
fix: update code snippet run example
mdelapenya Oct 18, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@ jobs:
matrix:
go-version: [1.22.x, 1.x]
platform: [ubuntu-latest]
module: [artemis, azurite, cassandra, chroma, clickhouse, cockroachdb, compose, consul, couchbase, databend, dolt, dynamodb, elasticsearch, etcd, gcloud, grafana-lgtm, inbucket, influxdb, k3s, k6, kafka, localstack, mariadb, milvus, minio, mockserver, mongodb, mssql, mysql, nats, neo4j, ollama, openfga, openldap, opensearch, postgres, pulsar, qdrant, rabbitmq, redis, redpanda, registry, surrealdb, valkey, vault, vearch, weaviate]
module: [artemis, azurite, cassandra, chroma, clickhouse, cockroachdb, compose, consul, couchbase, databend, dolt, dynamodb, elasticsearch, etcd, gcloud, grafana-lgtm, inbucket, influxdb, k3s, k6, kafka, localstack, mariadb, milvus, minio, mockserver, mongodb, mssql, mysql, nats, neo4j, ollama, openfga, openldap, opensearch, postgres, pulsar, qdrant, rabbitmq, redis, redpanda, registry, surrealdb, valkey, vault, vearch, weaviate, yugabytedb]
uses: ./.github/workflows/ci-test-go.yml
with:
go-version: ${{ matrix.go-version }}
Expand Down
4 changes: 4 additions & 0 deletions .vscode/.testcontainers-go.code-workspace
Original file line number Diff line number Diff line change
Expand Up @@ -201,6 +201,10 @@
"name": "module / weaviate",
"path": "../modules/weaviate"
},
{
"name": "module / yugabytedb",
"path": "../modules/yugabytedb"
},
{
"name": "modulegen",
"path": "../modulegen"
Expand Down
94 changes: 94 additions & 0 deletions docs/modules/yugabytedb.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
# YugabyteDB

Not available until the next release of testcontainers-go <a href="https://github.com/testcontainers/testcontainers-go"><span class="tc-version">:material-tag: main</span></a>

## Introduction

The Testcontainers module for yugabyteDB.

## Adding this module to your project dependencies

Please run the following command to add the yugabyteDB module to your Go dependencies:

```
go get github.com/testcontainers/testcontainers-go/modules/yugabytedb
```

## Usage example

<!--codeinclude-->
[Creating a yugabyteDB container](../../modules/yugabytedb/examples_test.go) inside_block:runyugabyteDBContainer
<!--/codeinclude-->

## Module Reference

### Run function

The yugabyteDB module exposes one entrypoint function to create the yugabyteDB container, and this function receives three parameters:

```golang
func Run(
ctx context.Context,
img string,
opts ...testcontainers.ContainerCustomizer,
) (*Container, error)
```

- `context.Context`, the Go context.
- `string`, the Docker image to use.
- `testcontainers.ContainerCustomizer`, a variadic argument for passing options.

### Container Options

When starting the yugabyteDB container, you can pass options in a variadic way to configure it.

#### Image

If you need to set a different yugabyteDB Docker image, you can set a valid Docker image as the second argument in the `Run` function.
E.g. `Run(context.Background(), "yugabytedb/yugabyte")`.

{% include "../features/common_functional_options.md" %}

henripqt marked this conversation as resolved.
Show resolved Hide resolved
#### Initial Database

By default the yugabyteDB container will start with a database named `yugabyte` and the default credentials `yugabyte` and `yugabyte`.

If you need to set a different database, and its credentials, you can use the `WithDatabaseName(dbName string)`, `WithDatabaseUser(dbUser string)` and `WithDatabasePassword(dbPassword string)` options.
mdelapenya marked this conversation as resolved.
Show resolved Hide resolved

#### Initial Cluster Configuration

By default the yugabyteDB container will start with a cluster keyspace named `yugabyte` and the default credentials `yugabyte` and `yugabyte`.

If you need to set a different cluster keyspace, and its credentials, you can use the `WithKeyspace(keyspace string)`, `WithUser(user string)` and `WithPassword(password string)` options.

### Container Methods

The yugabyteDB container exposes the following methods:

#### YSQLConnectionString

This method returns the connection string for the yugabyteDB container when using
the YSQL query language.
The connection string can then be used to connect to the yugabyteDB container using
a standard PostgreSQL client.

<!--codeinclude-->
[Create a postgres client using the connection string](../../modules/yugabytedb/examples_test.go) block:ExampleContainer_YSQLConnectionString
<!--/codeinclude-->

### Usage examples

#### Usage with YSQL and gocql

To use the YCQL query language, you need to configure the cluster
with the keyspace, user, and password.

By default, the yugabyteDB container will start with a cluster keyspace named `yugabyte` and the default credentials `yugabyte` and `yugabyte` but you can change it using the `WithKeyspace`, `WithUser` and `WithPassword` options.

In order to get the appropriate host and port to connect to the yugabyteDB container,
you can use the `GetHost` and `GetMappedPort` methods on the Container struct.
See the examples below:

<!--codeinclude-->
[Create a yugabyteDB client using the cluster configuration](../../modules/yugabytedb/yugabytedb_test.go) block:TestYugabyteDB_YCQL
<!--/codeinclude-->
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,7 @@ nav:
- modules/vault.md
- modules/vearch.md
- modules/weaviate.md
- modules/yugabytedb.md
- Examples:
- examples/index.md
- examples/nginx.md
Expand Down
5 changes: 5 additions & 0 deletions modules/yugabytedb/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
include ../../commons-test.mk

.PHONY: test
test:
$(MAKE) test-yugabytedb
155 changes: 155 additions & 0 deletions modules/yugabytedb/examples_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,155 @@
package yugabytedb_test

import (
"context"
"database/sql"
"fmt"
"log"
"net"

_ "github.com/lib/pq"
"github.com/yugabyte/gocql"

"github.com/testcontainers/testcontainers-go"
"github.com/testcontainers/testcontainers-go/modules/yugabytedb"
)

func ExampleRun() {
// runyugabyteDBContainer {
ctx := context.Background()

yugabytedbContainer, err := yugabytedb.Run(
ctx,
"yugabytedb/yugabyte:2024.1.3.0-b105",
yugabytedb.WithKeyspace("custom-keyspace"),
yugabytedb.WithUser("custom-user"),
yugabytedb.WithDatabaseName("custom-db"),
yugabytedb.WithDatabaseUser("custom-user"),
yugabytedb.WithDatabasePassword("custom-password"),
)
if err != nil {
log.Printf("failed to start container: %s", err)
return
}

defer func() {
if err := testcontainers.TerminateContainer(yugabytedbContainer); err != nil {
log.Printf("failed to terminate container: %s", err)
}
}()
// }

state, err := yugabytedbContainer.State(ctx)
if err != nil {
log.Printf("failed to get container state: %s", err)
return
}

fmt.Println(state.Running)

// Output: true
}

func ExampleContainer_YSQLConnectionString() {
ctx := context.Background()

yugabytedbContainer, err := yugabytedb.Run(
ctx,
"yugabytedb/yugabyte:2024.1.3.0-b105",
)
if err != nil {
log.Printf("failed to start container: %s", err)
return
}

defer func() {
if err := testcontainers.TerminateContainer(yugabytedbContainer); err != nil {
log.Printf("failed to terminate container: %s", err)
}
}()

connStr, err := yugabytedbContainer.YSQLConnectionString(ctx, "sslmode=disable")
if err != nil {
log.Printf("failed to get connection string: %s", err)
return
}

db, err := sql.Open("postgres", connStr)
if err != nil {
log.Printf("failed to open connection: %s", err)
return
}

defer db.Close()
stevenh marked this conversation as resolved.
Show resolved Hide resolved

var i int
row := db.QueryRowContext(ctx, "SELECT 1")
if err := row.Scan(&i); err != nil {
log.Printf("failed to scan row: %s", err)
return
}

fmt.Println(i)

// Output: 1
}

func ExampleContainer_newCluster() {
henripqt marked this conversation as resolved.
Show resolved Hide resolved
ctx := context.Background()

yugabytedbContainer, err := yugabytedb.Run(
ctx,
"yugabytedb/yugabyte:2024.1.3.0-b105",
)
if err != nil {
log.Printf("failed to start container: %s", err)
return
}

defer func() {
if err := testcontainers.TerminateContainer(yugabytedbContainer); err != nil {
log.Printf("failed to terminate container: %s", err)
}
}()

yugabytedbContainerHost, err := yugabytedbContainer.Host(ctx)
if err != nil {
log.Printf("failed to get container host: %s", err)
return
}

yugabyteContainerPort, err := yugabytedbContainer.MappedPort(ctx, "9042/tcp")
if err != nil {
log.Printf("failed to get container port: %s", err)
return
}

cluster := gocql.NewCluster(net.JoinHostPort(yugabytedbContainerHost, yugabyteContainerPort.Port()))
cluster.Keyspace = "yugabyte"
cluster.Authenticator = gocql.PasswordAuthenticator{
Username: "yugabyte",
Password: "yugabyte",
}

session, err := cluster.CreateSession()
if err != nil {
log.Printf("failed to create session: %s", err)
return
}

defer session.Close()
stevenh marked this conversation as resolved.
Show resolved Hide resolved

var i int
if err := session.Query(`
SELECT COUNT(*)
FROM system_schema.keyspaces
WHERE keyspace_name = 'yugabyte'
`).Scan(&i); err != nil {
log.Printf("failed to scan row: %s", err)
return
}

fmt.Println(i)

// Output: 1
henripqt marked this conversation as resolved.
Show resolved Hide resolved
}
65 changes: 65 additions & 0 deletions modules/yugabytedb/go.mod
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
module github.com/testcontainers/testcontainers-go/modules/yugabytedb

go 1.22

require (
github.com/lib/pq v1.10.9
github.com/stretchr/testify v1.9.0
github.com/testcontainers/testcontainers-go v0.33.0
github.com/yugabyte/gocql v1.6.0-yb-1
)

require (
dario.cat/mergo v1.0.0 // indirect
github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 // indirect
github.com/Microsoft/go-winio v0.6.2 // indirect
github.com/cenkalti/backoff/v4 v4.2.1 // indirect
github.com/containerd/containerd v1.7.18 // indirect
github.com/containerd/log v0.1.0 // indirect
github.com/containerd/platforms v0.2.1 // indirect
github.com/cpuguy83/dockercfg v0.3.2 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/distribution/reference v0.6.0 // indirect
github.com/docker/docker v27.1.1+incompatible // indirect
github.com/docker/go-connections v0.5.0 // indirect
github.com/docker/go-units v0.5.0 // indirect
github.com/felixge/httpsnoop v1.0.4 // indirect
github.com/go-logr/logr v1.4.1 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-ole/go-ole v1.2.6 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang/snappy v0.0.3 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/hailocab/go-hostpool v0.0.0-20160125115350-e80d13ce29ed // indirect
github.com/klauspost/compress v1.17.4 // indirect
github.com/kr/text v0.2.0 // indirect
github.com/lufia/plan9stats v0.0.0-20211012122336-39d0f177ccd0 // indirect
github.com/magiconair/properties v1.8.7 // indirect
github.com/moby/docker-image-spec v1.3.1 // indirect
github.com/moby/patternmatcher v0.6.0 // indirect
github.com/moby/sys/sequential v0.5.0 // indirect
github.com/moby/sys/user v0.1.0 // indirect
github.com/moby/term v0.5.0 // indirect
github.com/morikuni/aec v1.0.0 // indirect
github.com/opencontainers/go-digest v1.0.0 // indirect
github.com/opencontainers/image-spec v1.1.0 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/power-devops/perfstat v0.0.0-20210106213030-5aafc221ea8c // indirect
github.com/shirou/gopsutil/v3 v3.23.12 // indirect
github.com/shoenig/go-m1cpu v0.1.6 // indirect
github.com/sirupsen/logrus v1.9.3 // indirect
github.com/tklauser/go-sysconf v0.3.12 // indirect
github.com/tklauser/numcpus v0.6.1 // indirect
github.com/yusufpapurcu/wmi v1.2.3 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.49.0 // indirect
go.opentelemetry.io/otel v1.24.0 // indirect
go.opentelemetry.io/otel/metric v1.24.0 // indirect
go.opentelemetry.io/otel/trace v1.24.0 // indirect
golang.org/x/crypto v0.22.0 // indirect
golang.org/x/sys v0.21.0 // indirect
gopkg.in/inf.v0 v0.9.1 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
)

replace github.com/testcontainers/testcontainers-go => ../..
Loading
Loading