(aws-elasticsearch): Execute code after Elasticsearch domain is created

[jakubgruber]

❓ General Issue

Execute custom code after elasticsearch.CfDomain is created

I am trying to install EFK stack through CDK according to the eksworkshop.

During one step, they are waiting for ES cluster to be fully created to curl-it and register IAM role for fluent-bit.

# We need to retrieve the Fluent Bit Role ARN
export FLUENTBIT_ROLE=$(eksctl get iamserviceaccount --cluster eksworkshop-eksctl --namespace logging -o json | jq '.[].status.roleARN' -r) 

# Get the Elasticsearch Endpoint
export ES_ENDPOINT=$(aws es describe-elasticsearch-domain --domain-name ${ES_DOMAIN_NAME} --output text --query "DomainStatus.Endpoint")

# Update the Elasticsearch internal database
curl -sS -u "${ES_DOMAIN_USER}:${ES_DOMAIN_PASSWORD}" \
    -X PATCH \
    https://${ES_ENDPOINT}/_opendistro/_security/api/rolesmapping/all_access?pretty \
    -H 'Content-Type: application/json' \
    -d'
[
  {
    "op": "add", "path": "/backend_roles", "value": ["'${FLUENTBIT_ROLE}'"]
  }
]
'

Is there any workaround to wait in CDK for ES cluster to run and then automatically execute custom code? My IaaC codebase is in Typescript, but I assume it is rather a general question.

[iliapolo]

@jakubgruber

Is there any workaround to wait in CDK for ES cluster to run and then automatically execute custom code? My IaaC codebase is in Typescript, but I assume it is rather a general question.

Yeap, you can use custom-resources. In fact, we have this exact example as part of our own implementation for the L2 Domain construct.

See https://github.com/aws/aws-cdk/blob/master/packages/%40aws-cdk/aws-elasticsearch/lib/elasticsearch-access-policy.ts

This code configures access policies after the domain has been created, since it runs an aws-sdk call, it uses a predefined version of a custom resource that handles AWS SDK Calls, but you can run any code you like.

In addition, we have an RFC in progress to specifically address this exact use case, called Deployment Triggers, you are welcome to join the discussion :)

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.