feat(ryuk): make listen address of exposed port configurable #2809

romainlaurent · 2024-10-02T08:28:02Z

What does this PR do?

Configurable listen address for the ExposedPorts on Ryuk container.

Why is it important?

We run our containers from a gitlab runner where there is a network policy which forbid containers exposing port to listen on 0.0.0.0 .

Related issues

The unit tests should cover the use cases

netlify · 2024-10-02T08:28:19Z

✅ Deploy Preview for testcontainers-go ready!

Name	Link
🔨 Latest commit	`5b08390`
🔍 Latest deploy log	https://app.netlify.com/sites/testcontainers-go/deploys/66fec379bfca4600086a326e
😎 Deploy Preview	https://deploy-preview-2809--testcontainers-go.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

stevenh

Nice addition, thanks for the PR.

I've made some initial suggestions but we should wait for #2728 to be merged before completing as there's going to be some conflicts and it will be easier to fix here.

We should also look to test that the reaper works as expected using a custom port so a test for that should be added, best to wait for the above PR to land first as that changes quite a bit in that area.

internal/config/config.go

internal/config/config_test.go

reaper.go

romainlaurent · 2024-10-02T12:56:12Z

Thanks for your comments!
Waiting patiently for #2728 to be merged

stevenh

Thanks for the quick changes, second round...

stevenh · 2024-10-02T13:45:41Z

internal/config/config.go

+		ryukAddress := os.Getenv("TESTCONTAINERS_RYUK_ADDRESS")
+		if ryukAddress != "" {


nit: we could fold this to be a little more idiomatic

Suggested change

ryukAddress := os.Getenv("TESTCONTAINERS_RYUK_ADDRESS")

if ryukAddress != "" {

if ryukAddress := os.Getenv("TESTCONTAINERS_RYUK_ADDRESS"); ryukAddress != "" {

stevenh · 2024-10-02T13:47:06Z

internal/config/config_test.go

@@ -140,6 +142,15 @@ func TestReadTCConfig(t *testing.T) {
 		assert.Equal(t, expected, config)
 	})

+	t.Run("Invalid IP address", func(t *testing.T) {


suggestion: to my previous set of comments, were working towards simple names with no spaces:

Suggested change

t.Run("Invalid IP address", func(t *testing.T) {

t.Run("invalid-address", func(t *testing.T) {

stevenh · 2024-10-02T13:48:29Z

internal/config/config_test.go

+		t.Cleanup(func() {
+			os.Unsetenv("TESTCONTAINERS_RYUK_ADDRESS")
+		})


bug: this isn't needed t.Setenv already adds a clean up function.

Suggested change

t.Cleanup(func() {

os.Unsetenv("TESTCONTAINERS_RYUK_ADDRESS")

})

stevenh · 2024-10-02T13:55:44Z

reaper.go

+	exposedPort := string(listeningPort)
+	if tcConfig.RyukAddress != "" {
+		exposedPort = net.JoinHostPort(tcConfig.RyukAddress, "") + ":" + exposedPort


bug: pretty sure this would result in a double colon, I would expect the following, did I miss something?

Suggested change

exposedPort := string(listeningPort)

if tcConfig.RyukAddress != "" {

exposedPort = net.JoinHostPort(tcConfig.RyukAddress, "") + ":" + exposedPort

exposedPort := config.ReaperDefaultPort

if tcConfig.RyukAddress != "" {

exposedPort = net.JoinHostPort(tcConfig.RyukAddress, exposedPort)

Also note that the other PR has port configurable using the env var RYUK_PORT, not sure if that will land, but just making you aware.

stevenh · 2024-10-02T13:57:16Z

reaper_test.go

@@ -408,6 +408,22 @@ func Test_NewReaper(t *testing.T) {
 				"TESTCONTAINERS_HUB_IMAGE_NAME_PREFIX": "registry.mycompany.com/mirror",
 			},
 		},
+		{
+			name: "Reaper with a custom listen address",


suggestion: simple test names

Suggested change

name: "Reaper with a custom listen address",

name: "custom-address",

stevenh · 2024-10-02T13:58:50Z

internal/config/config.go

+func parseIP(input string) string {
+	ip := net.ParseIP(input)
+	if ip == nil {
+		panic("invalid IP address: " + input)


suggestion: quote so users can see issues like trailing white space

Suggested change

panic("invalid IP address: " + input)

panic(fmt.Sprintf("invalid IP address: %q", input))

stevenh · 2024-10-02T14:00:39Z

@romainlaurent not sure you wanted to delete this branch?

romainlaurent · 2024-10-03T07:54:06Z

Indeed, a branch rename had more impact than expected. Unfortunately, this approach hasn't fixed my issue.

The problem I have is that my tests are running in GitLab CI with the Docker executor.
This setup causes my Ryuk container to run alongside the one launching the tests.

I will propose something different where I override the Reaper.Endpoint field here.

We can override the Reaper.Endpoint field by using an environment variable, such as TESTCONTAINERS_RYUK_USE_NAME_ENDPOINT=false.

What do you think about it?

mdelapenya · 2024-10-03T08:19:50Z

@romainlaurent You probably read this about Gitlab CI, right? https://golang.testcontainers.org/system_requirements/ci/gitlab_ci/

Isn't it working for you?

romainlaurent · 2024-10-03T08:41:33Z

@mdelapenya thanks for your answer!
Yes I have but it's not enough, there is strict network policies on the runner which forbid traffic on container listening on 0.0.0.0
I got this log error
failed waiting for reaper container 8fdba937 port tcp/8080 to be ready: dial tcp 172.17.0.1:33074: connect: network is unreachable: creating reaper failed
Maybe by changing only here
wait.ForListeningPort(listeningPort) by wait.ForLog("Started")

stevenh

I don't think this is a workable fix I'm afraid as if ForListeningPort doesn't work then the container wont be accessible.

stevenh · 2024-10-03T09:00:05Z

reaper.go

@@ -249,7 +249,7 @@ func newReaper(ctx context.Context, sessionID string, provider ReaperProvider) (
 		ExposedPorts: []string{string(listeningPort)},
 		Labels:       core.DefaultLabels(sessionID),
 		Privileged:   tcConfig.RyukPrivileged,
-		WaitingFor:   wait.ForListeningPort(listeningPort),
+		WaitingFor:   wait.ForLog("Started"),


bug: If the listening for check doesn't work then the reaper wont be accessible so this is not a good fix.

It could work by adding TESTCONTAINERS_RYUK_ENDPOINT_OVERRIDE_BY_NAME=true, allowing us to contact the container directly through its name.
I'm not sure it's the most elegant solution, so I'm open to any alternatives you think would be better.

stevenh · 2024-10-03T09:01:03Z

reaper.go

-					reaperContainer.ID[:8], port.Proto(), port.Port(), err)
-			}
-		}
+	err = wait.ForLog("Started").WaitUntilReady(ctx, reaperContainer)


bug: If the listening for doesn't work the container isn't accessible so wont work.

romainlaurent · 2024-10-04T13:05:36Z

After a lot of try, I couldn't find a quick and elegant way to fix my problem. As it's specific at our gitlab runner configuration, I'll close this PR.
Thanks to @stevenh @mdelapenya for your time and the tool you've made, it's really great!

stevenh · 2024-10-04T14:20:12Z

@romainlaurent would you mind creating a issue to track this as I think its something we should look to fix.

romainlaurent requested a review from a team as a code owner October 2, 2024 08:28

romainlaurent force-pushed the chore/ryuk_listen_address branch from c59809c to d7e0d34 Compare October 2, 2024 08:28

romainlaurent changed the title ~~chore(ryuk): make listen address of exposed port configurable~~ feat(ryuk): make listen address of exposed port configurable Oct 2, 2024

romainlaurent force-pushed the chore/ryuk_listen_address branch from d7e0d34 to 3cb18ac Compare October 2, 2024 08:35

stevenh requested changes Oct 2, 2024

View reviewed changes

romainlaurent force-pushed the chore/ryuk_listen_address branch from 3cb18ac to c5bec9c Compare October 2, 2024 12:51

romainlaurent closed this Oct 2, 2024

romainlaurent deleted the chore/ryuk_listen_address branch October 2, 2024 13:59

stevenh requested changes Oct 2, 2024

View reviewed changes

romainlaurent restored the chore/ryuk_listen_address branch October 3, 2024 07:36

romainlaurent reopened this Oct 3, 2024

romainlaurent force-pushed the chore/ryuk_listen_address branch from c5bec9c to 5cf6c3f Compare October 3, 2024 08:47

feat(ryuk): wait on Started log

937f3b5

romainlaurent force-pushed the chore/ryuk_listen_address branch from 5cf6c3f to 937f3b5 Compare October 3, 2024 09:00

stevenh requested changes Oct 3, 2024

View reviewed changes

romainlaurent marked this pull request as draft October 3, 2024 13:40

romainlaurent force-pushed the chore/ryuk_listen_address branch 3 times, most recently from b7ecbee to 61e3e59 Compare October 3, 2024 14:12

fix(ryul): test by overriding reaper endpoint

755f116

romainlaurent force-pushed the chore/ryuk_listen_address branch from 61e3e59 to 755f116 Compare October 3, 2024 14:26

feat(ryuk): use an external ryuk

5b08390

romainlaurent closed this Oct 4, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(ryuk): make listen address of exposed port configurable #2809

feat(ryuk): make listen address of exposed port configurable #2809

romainlaurent commented Oct 2, 2024

netlify bot commented Oct 2, 2024 •

edited

Loading

stevenh left a comment

romainlaurent commented Oct 2, 2024

stevenh left a comment

stevenh Oct 2, 2024

stevenh Oct 2, 2024

stevenh Oct 2, 2024

stevenh Oct 2, 2024

stevenh Oct 2, 2024

stevenh Oct 2, 2024

stevenh commented Oct 2, 2024

romainlaurent commented Oct 3, 2024 •

edited

Loading

mdelapenya commented Oct 3, 2024

romainlaurent commented Oct 3, 2024 •

edited

Loading

stevenh left a comment

stevenh Oct 3, 2024

romainlaurent Oct 3, 2024

stevenh Oct 3, 2024

romainlaurent commented Oct 4, 2024

stevenh commented Oct 4, 2024

		ryukAddress := os.Getenv("TESTCONTAINERS_RYUK_ADDRESS")
		if ryukAddress != "" {

	ryukAddress := os.Getenv("TESTCONTAINERS_RYUK_ADDRESS")
	if ryukAddress != "" {
	if ryukAddress := os.Getenv("TESTCONTAINERS_RYUK_ADDRESS"); ryukAddress != "" {

	t.Run("Invalid IP address", func(t *testing.T) {
	t.Run("invalid-address", func(t *testing.T) {

	t.Cleanup(func() {
	os.Unsetenv("TESTCONTAINERS_RYUK_ADDRESS")
	})

	name: "Reaper with a custom listen address",
	name: "custom-address",

	panic("invalid IP address: " + input)
	panic(fmt.Sprintf("invalid IP address: %q", input))

feat(ryuk): make listen address of exposed port configurable #2809

feat(ryuk): make listen address of exposed port configurable #2809

Conversation

romainlaurent commented Oct 2, 2024

What does this PR do?

Why is it important?

Related issues

netlify bot commented Oct 2, 2024 • edited Loading

✅ Deploy Preview for testcontainers-go ready!

stevenh left a comment

Choose a reason for hiding this comment

romainlaurent commented Oct 2, 2024

stevenh left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stevenh commented Oct 2, 2024

romainlaurent commented Oct 3, 2024 • edited Loading

mdelapenya commented Oct 3, 2024

romainlaurent commented Oct 3, 2024 • edited Loading

stevenh left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

romainlaurent commented Oct 4, 2024

stevenh commented Oct 4, 2024

netlify bot commented Oct 2, 2024 •

edited

Loading

romainlaurent commented Oct 3, 2024 •

edited

Loading

romainlaurent commented Oct 3, 2024 •

edited

Loading