Got panic when specifying namespace name for service connect

[itkq]

Let's say ecs-service-def.jsonnet is like:

{
  desiredCount: 1,
  networkConfiguration: {
    awsvpcConfiguration: {
      assignPublicIp: 'DISABLED',
      securityGroups: [
        'sg-xxx',
      ],
      subnets: [
        'subnet-yyy', 
      ],
    },
  },
  serviceConnectConfiguration: {
    enabled: true,
    namespace: 'itkq-test.local',
  },
  launchType: "FARGATE",
  platformVersion: '1.4.0',
}

Initial ecspresso deploy failed with SIGSEGV as shown below.

ecspresso deploy --debug
2024/08/30 00:55:58 [INFO] ecspresso version: v2.4.0
2024/08/30 00:56:00 SERVICE_NAME/CLUSTER [DEBUG] config file path: ecspresso.jsonnet
2024/08/30 00:56:00 SERVICE_NAME/CLUSTER [DEBUG] timeout: 10m0s
2024/08/30 00:56:00 SERVICE_NAME/CLUSTER [DEBUG] dispatching subcommand: deploy
2024/08/30 00:56:00 SERVICE_NAME/CLUSTER [DEBUG] deploy
2024/08/30 00:56:00 [DEBUG] {"DryRun":false,"DesiredCount":-1,"SkipTaskDefinition":false,"Revision":0,"ForceNewDeployment":false,"Wait":true,"SuspendAutoScaling":null,"ResumeAutoScaling":null,"AutoScalingMin":null,"AutoScalingMax":null,"RollbackEvents":"","UpdateService":true,"LatestTaskDefinition":false}
2024/08/30 00:56:00 SERVICE_NAME/CLUSTER Starting deploy
2024/08/30 00:56:01 SERVICE_NAME/CLUSTER Service SERVICE_NAME not found. Creating a new service
2024/08/30 00:56:01 SERVICE_NAME/CLUSTER Starting create service
2024/08/30 00:56:01 SERVICE_NAME/CLUSTER [DEBUG] Loaded DesiredCount: 1
2024/08/30 00:56:01 SERVICE_NAME/CLUSTER Registering a new task definition...
2024/08/30 00:56:01 SERVICE_NAME/CLUSTER Task definition is registered SERVICE_NAME:1
2024/08/30 00:56:02 SERVICE_NAME/CLUSTER Service is created
2024/08/30 00:56:05 SERVICE_NAME/CLUSTER [DEBUG] service SERVICE_NAME is ACTIVE
2024/08/30 00:56:05 SERVICE_NAME/CLUSTER [DEBUG] deployment: ecs-svc/XXX PRIMARY
2024/08/30 00:56:05 SERVICE_NAME/CLUSTER [WARNING] failed to get namespace: operation error ServiceDiscovery: GetNamespace, https response error StatusCode: 400, RequestID: 25aa7632-e6cc-463b-a1e0-a3d72595e4f3, NamespaceNotFound:
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x2 addr=0x0 pc=0x105fa5a34]

goroutine 1 [running]:
github.com/kayac/ecspresso/v2.(*App).newServiceFromTypes(_, {_, _}, {{0x0, 0x0, 0x0}, 0x140007606e0, 0x1400075a348, 0x14000760800, 0x1400075ef30, ...})
        /home/runner/work/ecspresso/ecspresso/ecspresso.go:113 +0x3f4
github.com/kayac/ecspresso/v2.(*App).DescribeService(0x14000274a80, {0x10678f6f0, 0x140004540e0})
        /home/runner/work/ecspresso/ecspresso/ecspresso.go:285 +0x40c
github.com/kayac/ecspresso/v2.(*App).createService(0x14000274a80, {0x10678f6f0, 0x140004540e0}, {0x0, 0x140003cec6c, 0x0, 0x0, 0x0, 0x1, 0x0, ...})
        /home/runner/work/ecspresso/ecspresso/create.go:91 +0x6a0
github.com/kayac/ecspresso/v2.(*App).Deploy(0x14000274a80, {0x10678f7d0, 0x1400034d700}, {0x0, 0x140003cec6c, 0x0, 0x0, 0x0, 0x1, 0x0, ...})
        /home/runner/work/ecspresso/ecspresso/deploy.go:90 +0x220
github.com/kayac/ecspresso/v2.dispatchCLI({0x10678f7d0, 0x1400034d700}, {0x14000339830, 0x6}, 0x14000321130, 0x14000344120)
        /home/runner/work/ecspresso/ecspresso/cli.go:124 +0x758
github.com/kayac/ecspresso/v2.CLI({0x10678f7d0, 0x1400034d700}, 0x1073e45a0?)
        /home/runner/work/ecspresso/ecspresso/cli.go:174 +0x90
main.main()
        /home/runner/work/ecspresso/ecspresso/cmd/ecspresso/main.go:19 +0xa4

However, the namespace does indeed exist.

aws servicediscovery list-namespaces | jq -r '.Namespaces[] | select(.Name == "itkq-test.local") | "\(.Id) \(.Name)"'
ns-ZZZ itkq-test.local

The reason whey this nil pointer dereference happens is .services[].deployments[].serviceConnectConfiguration.namespace of DescribeServices response is not the namespace ARN but the namespace name.

aws ecs describe-services --cluster CLUSTER --services SERVICE_NAME | jq -r '.services[].deployments[] | select(.status == "PRIMARY") | .serviceConnectConfiguration.namespace'
itkq-test.local

ecspresso/ecspresso.go

Lines 100 to 115 in 7d2c5da

	// resolve sd namespace arn to name
	id := arnToName(aws.ToString(scc.Namespace))
	res, err := d.sd.GetNamespace(ctx, &servicediscovery.GetNamespaceInput{
	Id: &id,
	})
	if err != nil {
	var oe *smithy.OperationError
	if errors.As(err, &oe) {
	d.Log("[WARNING] failed to get namespace: %s", oe)
	} else {
	return nil, fmt.Errorf("failed to get namespace: %w", err)
	}
	}
	if res.Namespace != nil {
	scc.Namespace = res.Namespace.Name
	}

ecspresso/util.go

Lines 15 to 18 in 7d2c5da

	func arnToName(s string) string {
	ns := strings.Split(s, "/")
	return ns[len(ns)-1]
	}

The strange thing is that previously, specifying the name for the namespace was not an issue, but at some point, it started failing. Can you reproduce this issue?

[fujiwara]

@itkq Thank you for the reporting!

I reproduced this.

• ECS's ServiceConnectConfiguration.Namespace says 'The namespace name or full Amazon Resource Name (ARN) of the Cloud Map namespace for use with Service Connect.'
• ecspresso gets the namespace by servicediscovery#GetNamespaceInput. The input parameter takes only Id (The ID of the namespace), but ecspresso is likely to passe the namespace name as Id value.

The strange thing is that previously, specifying the name for the namespace was not an issue, but at some point, it started failing.

In the past, ecs.DescribeServices probably returns a ServiceConnectConfiguration contains the namespace ID, but it seems to return the namespace name now.

I'll fix this. Wait a minutes!

[itkq]

Thank you for the quick response!

As a workaround, specifying the ARN allows the ECS service creation and update to succeed, but it causes a diff to occur every time as a side effect. It would be great if this also could be resolved on ecspresso side!

$ ecspresso diff
(snip)
+++ ecs-service-def.jsonnet
@@ -69,7 +69,7 @@
         "use_put_object": "On"
       }
     },
-    "namespace": "itkq-test.local",
+    "namespace": "[NAMESPACE_ARN]",
     "services": [
       {
         "clientAliases": [

[fujiwara]

@itkq https://github.com/kayac/ecspresso/releases/tag/v2.4.1 released!

[itkq]

I have confirmed that no panic occurs with both the existing service and the new service! Thanks!