Skip to content
This repository has been archived by the owner on Apr 19, 2022. It is now read-only.

Vlaaaaaaad/terraform-aws-fargate-refinery

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

44 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Refinery

GitHub License: MIT Gitpod: ready-to-code Maintenance status: best-effort

Introduction

Refinery is a proxy from Honeycomb which offers trace-aware sampling.

This module contains the Terraform infrastructure code that creates the required AWS resources to run Refinery in AWS, including the following:

  • A Virtual Private Cloud (VPC)
  • A SSL certificate using Amazon Certificate Manager (ACM)
  • An Application Load Balancer (ALB)
  • A DNS Record using AWS Route53 which points to ALB
  • An AWS Elastic Cloud Service (ECS) Cluster leveraging Spot AWS Fargate to run the Refinery Docker image
  • Two Parameters in AWS Systems Manager Parameter Store to store the Refinery configuration and rules and access them natively in Fargate
  • A single-node Redis (cluster mode disabled) Cluster in AWS ElastiCache to be used by Refinery for high-availability and peer discovery

Diagram showing the architecture. The Honeycomb-instrumented apps use Route53 to connect to the ALB. The ALB routes traffic to Refinery containers running in Fargate, in different AZs and public subnets. The Refinery containers connect to a single-AZ Redis and communicate between them.

Gotchas

Due to Fargate on ECS having no support for configuration files, the configuration is Base64-encoded, stored in AWS Systems Manager Parameter Store, and then put in the Container Secrets. When Refinery starts, it decodes the secret and creates the config file on disk. This leads to two limitations:

  • configuration files cannot be bigger than 8Kb
  • a custom image has to be used as the upstream image does not have sh or base64 included

Usage

As a standalone project

Using this module as a standalone project is only recommended for testing.

  1. Clone this github repository:
$ git clone [email protected]:vlaaaaaaad/terraform-aws-fargate-refinery.git

$ cd terraform-aws-fargate-refinery
  1. Copy the sample terraform.tfvars.sample into terraform.tfvars and specify the required variables there.

  2. Create a myrules.toml file with your Refinery rules.

  3. Run terraform init to download required providers and modules.

  4. Run terraform apply to apply the Terraform configuration and create the required infrastructure.

  5. Run terraform output refinery_url to get URL where Refinery is reachable. (Note: It may take a minute or two for the URL to become reachable the first time)

As a Terraform module

Using this as a Terraform module allows integration with your existing Terraform configurations and pipelines.

module "refinery" {
  # Use git to pull the module from GitHub, the latest version
  source = "[email protected]:vlaaaaaaad/terraform-aws-fargate-refinery.git?ref=main"
  # or
  # Pull a specific version from Terraform Module Registry
  # source  = "Vlaaaaaaad/fargate-refinery/aws"
  # version = "1.2.3-replace-me"

  # REQUIRED: DNS (without trailing dot)
  route53_zone_name = "example.com"

  # REQUIRED: Refinery config file
  refinery_rules_file_path = "${path.module}/myrules.toml"

  # Optional: override the name
  name = "refinery"

  # Optional: customize the VPC
  azs                = ["eu-west-1a", "eu-west-1b", "eu-west-1c"]
  vpc_cidr           = "10.20.0.0/16"
  vpc_public_subnets = ["10.20.1.0/24", "10.20.2.0/24", "10.20.3.0/24"]

  # Optional: use pre-exiting ACM Certificate instead of creating and validating a new certificate
  certificate_arn = "arn:aws:acm:eu-west-1:135367859851:certificate/70e008e1-c0e1-4c7e-9670-7bb5bd4f5a84"

  # Optional: Send Refinery logs&metrics to Honeycomb, instead of ECS&nowhere
  refinery_logger_option   = "honeycomb"
  refinery_logger_api_key  = "00000000000000000000000000000000"
  refinery_metrics_option  = "honeycomb"
  refinery_metrics_api_key = "00000000000000000000000000000000"
}

As a Terraform module, as part of existing infrastructure

Using this module also allows integration with existing AWS resources -- VPC, Subnets, IAM Roles. Specify the required arguments.

WARNING: This was not tested.

Inputs

Name Description Type Default Required
refinery_rules_file_path The path to a toml files with the Refinery rules. Must be less than 8Kb any n/a yes
acm_certificate_arn The ARN of a certificate issued by AWS ACM. If empty, a new ACM certificate will be created and validated using Route53 DNS string "" no
acm_certificate_domain_name The Route53 domain name to use for ACM certificate. Route53 zone for this domain should be created in advance. Specify if it is different from value in route53_zone_name string "" no
alb_additional_sgs A list of additional Security Groups to attach to the ALB list(string) [] no
alb_internal Whether the load balancer is internal or external bool false no
alb_log_bucket_name The name of the S3 bucket (externally created) for storing load balancer access logs. Required if alb_logging_enabled is true string "" no
alb_log_location_prefix The S3 prefix within the log_bucket_name under which logs are stored string "" no
alb_logging_enabled Whether if the ALB will log requests to S3 bool false no
aws_cloudwatch_log_group_kms_key_id The ID of the KMS key to use when encrypting log data any null no
azs A list of availability zones that you want to use from the Region list(string) [] no
create_route53_record Whether to create Route53 record for Refinery bool true no
ecs_capacity_providers A list of short names or full Amazon Resource Names (ARNs) of one or more capacity providers to associate with the cluster. Valid values also include FARGATE and FARGATE_SPOT list(string)
[
"FARGATE_SPOT"
]
no
ecs_cloudwatch_log_retention_in_days The retention time for CloudWatch Logs number 30 no
ecs_container_memory_reservation The amount of memory (in MiB) to reserve for Refinery number 4096 no
ecs_default_capacity_provider_strategy The capacity provider strategy to use by default for the cluster. Can be one or more. List of map with corresponding items in docs. See Terraform Docs list(any)
[
{
"capacity_provider": "FARGATE_SPOT"
}
]
no
ecs_execution_role The ARN of an existing IAM Role that will be used ECS to start the Tasks string "" no
ecs_service_additional_sgs A list of additional Security Groups to attach to the ECS Service list(string) [] no
ecs_service_assign_public_ip Whether the ECS Tasks should be assigned a public IP. Should be true, if ECS service is using public subnets. See AWS Docs bool true no
ecs_service_deployment_maximum_percent The upper limit (as a percentage of the service's desiredCount) of the number of running tasks that can be running in a service during a deployment number 300 no
ecs_service_deployment_minimum_healthy_percent The lower limit (as a percentage of the service's desiredCount) of the number of running tasks that must remain running and healthy in a service during a deployment number 100 no
ecs_service_desired_count The number of instances of the task definition to place and keep running number 2 no
ecs_service_subnets If using a pre-existing VPC, subnet IDs to be used for the ECS Service list(string) [] no
ecs_settings A list of maps with cluster settings. For example, this can be used to enable CloudWatch Container Insights for a cluster. See Terraform Docs list(any)
[
{
"name": "containerInsights",
"value": "enabled"
}
]
no
ecs_task_cpu The number of CPU units to be used by Refinery number 2048 no
ecs_task_memory The amount of memory (in MiB) to be used by Samprixy number 4096 no
ecs_task_role The ARN of an existin IAM Role that will be used by the Refinery Task string "" no
ecs_use_new_arn_format Whether the AWS Account has opted in to the new longer ARN format which allows tagging ECS bool false no
execution_policies_arn A list of ARN of the policies to attach to the execution role list(string)
[
"arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy",
"arn:aws:iam::aws:policy/AmazonSSMReadOnlyAccess"
]
no
firelens_configuration The FireLens configuration for the Refinery container. This is used to specify and configure a log router for container logs. See AWS Docs
object({
type = string
options = map(string)
})
null no
image_repository The Refinery image repository string "public.ecr.aws/vlaaaaaaad/refinery-fargate-image" no
image_repository_credentials The container repository credentials; required when using a private repo. This map currently supports a single key; "credentialsParameter", which should be the ARN of a Secrets Manager's secret holding the credentials map(string) null no
image_tag The Refinery image tag to use string "1.4.1" no
name The name to use on all resources created (VPC, ALB, etc) string "refinery" no
redis_node_type The instance type used for the Redis cache cluster. See all available values on the AWS website string "cache.t2.micro" no
redis_port The Redis port string "6379" no
redis_subnets If using a pre-exiting VPC, subnet IDs to be used for Redis list(string) [] no
redis_version The Redis version string "6.x" no
refinery_accepted_api_keys The list of Honeycomb API keys that the proxy will accept list(string)
[
"*"
]
no
refinery_cache_capacity The number of spans to cache number 1000 no
refinery_compress_peer_communication The flag to enable or disable compressing span data when forwarded to peers bool true no
refinery_log_level The Refinery log level string "debug" no
refinery_logger_api_key The API key to use to send Refinery logs to Honeycomb string "" no
refinery_logger_dataset_name The dataset to which to send Refinery logs to string "Refinery Logs" no
refinery_logger_option The loger option for refinery string "logrus" no
refinery_logger_sampler_enabled The flag to enable or disable sampling Refinery logs bool false no
refinery_logger_sampler_throughput The per key per second throughput for the log message dynamic sampler number 10 no
refinery_max_alloc The maximum memory to use number 0 no
refinery_metrics_api_key The API key used to send Refinery metrics to Honeycomb string "" no
refinery_metrics_dataset The dataset to which to send Refinery metrics to string "Refinery Metrics" no
refinery_metrics_option The metrics option for refinery string "prometheus" no
refinery_metrics_reporting_interval The interval (in seconds) to wait between sending metrics to Honeycomb number 3 no
refinery_peer_buffer_size The number of events to buffer before seding to peers number 10000 no
refinery_send_delay The delay to wait after a trace is complete, before sending string "2s" no
refinery_send_ticker The duration to use to check for traces to send string "100ms" no
refinery_trace_timeout The amount of time to wait for a trace to be completed before sending string "60s" no
refinery_upstream_buffer_size The number of events to buffer before sending to Honeycomb number 10000 no
route53_record_name The name of Route53 record to create ACM certificate in and main A-record. If null is specified, var.name is used instead. Provide empty string to point root domain name to ALB string null no
route53_zone_name The Route53 zone name to create ACM certificate in and main A-record, without trailing dot string "" no
tags A mapping of tags to assign to all resources map(string) {} no
vpc_alb_subnets If using a pre-exiting VPC, subnet IDs to be used for the ALBs list(string) [] no
vpc_cidr The CIDR block for the VPC which will be created if vpc_id is not specified string "172.16.0.0/16" no
vpc_id The ID of an existing VPC where resources will be created string "" no
vpc_public_subnets A list of public subnets inside the VPC list(string)
[
"172.16.0.0/18",
"172.16.64.0/18",
"172.16.128.0/18"
]
no

Outputs

Name Description
alb_dns_name The DNS name of the ALB
alb_sg The ID of the Security Group attached to the ALB
alb_zone_id The ID of the Route53 zone containing the ALB record
ecs_cluster_id The ARN of the ECS cluster hosting Refinery
refinery_ecs_security_group The ID of the Security group assigned to the Refinery ECS Service
refinery_ecs_task_definition The task definition for the Refinery ECS service
refinery_execution_role_arn The IAM Role used to create the Refinery tasks
refinery_task_role_arn The Atlantis ECS task role name
refinery_url The URL to use for Refinery
vpc_id The ID of the VPC that was created or passed in

Authors

This module is created and maintained by Vlad Ionescu, in his free time. This is a best-effort allocation of time.

This module was inspired by Anton Babenko's terraform-aws-atlantis, which was, in turn, inspired by Seth Vargo's atlantis-on-gke. Yay, open-source!

License

MIT licensed. See LICENSE details.