Handle multiple security groups and security group rules across regions

[ronaldtse]

Here's an enhancement we need for the module.

For each security group, we want to handle rules (and ports) separately. These security groups may belong to different regions (different providers)

• use providers as argument, don't pass aws_account_id and aws_region into module anymore
• for each security group, we allow defining multiple rules and conditions inside

This is how the new structure should be used:

provider "aws" {
  allowed_account_ids = [ "${var.eu_aws_account_id}" ]
  region              = "eu-west-1"
  alias               = "eu"
}

provider "aws" {
  allowed_account_ids = [ "${var.us_aws_account_id}" ]
  region              = "us-east-1"
  alias               = "us"
}

module "dynamic-secgroup" {
  source = "riboseinc/authenticating-secgroup/aws"

  # prefix string used in named resources
  name_prefix             = "getting-started-"

  # Description of this secgroup
  description             = "${var.description}"

  security_group {
    id = "${aws_security_group.instance_group_1_ssh.id}"

    providers = {
      "aws" = "aws.us"
    }

    security_group_rule {
      secgroup_rule_type      = "ingress"
      secgroup_rule_from_port = 22
      secgroup_rule_to_port   = 22
      secgroup_rule_protocol  = "tcp"
    }

    security_group_rule {
      secgroup_rule_type      = "ingress"
      secgroup_rule_from_port = 443
      secgroup_rule_to_port   = 443
      secgroup_rule_protocol  = "tcp"
    }

    time_to_expire = 600

  }

  security_group {
    id = "${aws_security_group.instance_group_2_http.id}"

    providers = {
      "aws" = "aws.eu"
    }

    security_group_rule {
      secgroup_rule_type      = "ingress"
      secgroup_rule_from_port = 80
      secgroup_rule_to_port   = 80
      secgroup_rule_protocol  = "tcp"
    }

    time_to_expire = 300
  }
}

Let me know what you think!

[phuonghuynh]

I think its OK. We can enhance the module for this. We can use type list for "security_group_rule", like:

security_group_rules =[ 
    {
      secgroup_rule_type      = "ingress"
      secgroup_rule_from_port = 22
      secgroup_rule_to_port   = 22
      secgroup_rule_protocol  = "tcp"
    }, 
    {
      secgroup_rule_type      = "ingress"
      secgroup_rule_from_port = 443
      secgroup_rule_to_port   = 443
      secgroup_rule_protocol  = "tcp"
    }
]

[ronaldtse]

Is type list better, or would a command be cleaner?

[phuonghuynh]

The "command" is better, but i dont know it. Do you mean terraform has a way to define data driven variable?

[ronaldtse]

Now I think about it — it’s not possible to do it as a module. That would have to be a custom resource to accept such syntax.

Your approach is good enough indeed; let’s do it. Thanks!

[ronaldtse]

@phuonghuynh any updates so far? Thanks!

[phuonghuynh]

I still working on it, the syntax might be different. Terraform not support dynamic create module using "count" as its resources. Also, i dont know how to get provider attribute to generate arn for lambda permission, so i keep "aws_account_id" & "aws_region" for now.

[ronaldtse]

Got it.

Provider attribute account arn can be retrieved via https://www.terraform.io/docs/providers/aws/d/caller_identity.html which already includes account ID and region code.

[ronaldtse]

Hi @phuonghuynh , any updates on the syntax? We are kind of stuck here without a method to specify multiple ports. Thanks!

[phuonghuynh]

@ronaldtse the terraform v11 not support syntax to allow configure provider by variable, hashicorp/terraform#11578

This is the syntax that implementing,

// used to deploy
provider "aws" {
  access_key = "${var.aws_access_key}"
  secret_key = "${var.aws_secret_key}"
  region     = "${var.aws_region}"
}

provider "aws" {
  allowed_account_ids = [
    "${var.eu_aws_account_id}"
  ]

  region              = "eu-west-1"
  alias               = "eu"
}

provider "aws" {
  allowed_account_ids = [
    "${var.us_aws_account_id}"
  ]

  region              = "us-east-1"
  alias               = "us"
}

module "dynamic-secgroup" {
  source          = "../.."
  name_prefix     = "getting-started-"

  # Description of this secgroup
  description     = "${var.description}"


  security_groups = [
    {
      group_ids      = [
        "sg-df7a88a3",
        "sg-c9c72eb5"
      ]

      provider_alias = "aws.eu"

      rules          = [
        {
          secgroup_rule_type      = "ingress"
          secgroup_rule_from_port = 22
          secgroup_rule_to_port   = 22
          secgroup_rule_protocol  = "tcp"
        },
        {
          secgroup_rule_type      = "ingress"
          secgroup_rule_from_port = 443
          secgroup_rule_to_port   = 443
          secgroup_rule_protocol  = "tcp"
        }
      ]
    },
  ]

  //  time_to_expire = 600
}

Sample resource using provider alias

resource "aws_api_gateway_rest_api" "this" {
  //not supported in Terraform v11 now, https://github.com/hashicorp/terraform/issues/11578
  provider = "aws.${lookup(element(var.security_groups, count.index), "provider_alias") }"
  count = "${local.count}"
  name        = "${var.name_prefix}terraform-aws-authenticating-secgroup-${count.index}"
  description = "${var.description}"
}

Or we might try with this, hashicorp/terraform#11578 (comment) . However, i not sure how could we use "assume_role" here.

[phuonghuynh]

We are kind of stuck here without a method to specify multiple ports

@ronaldtse Its easier to do support multiple ports with the old config (not support multiple account_id)

[ronaldtse]

@phuonghuynh but this kind of provider support is good enough, right?

module "dynamic-secgroup" {
  source          = "../.."
  name_prefix     = "getting-started-"

  # Description of this secgroup
  description     = "${var.description}"

  security_groups = [
    {
      group_ids      = [
        "sg-df7a88a3",
        "sg-c9c72eb5"
      ]

# SEE THIS >>>
      providers {
        "aws" = "aws.eu"
      }
# <<<

      rules          = [
        {
          secgroup_rule_type      = "ingress"
          secgroup_rule_from_port = 22
          secgroup_rule_to_port   = 22
          secgroup_rule_protocol  = "tcp"
        },
        {
          secgroup_rule_type      = "ingress"
          secgroup_rule_from_port = 443
          secgroup_rule_to_port   = 443
          secgroup_rule_protocol  = "tcp"
        }
      ]
    },
  ]
  //  time_to_expire = 600
}

[ronaldtse]

@phuonghuynh we don't need to use multiple account-ids, but if possible we want to use one dynamic secgroup module to handle multiple regions. But if it can't, we can separate per-region, too.

[phuonghuynh]

Rest API Region is an enhancement now hashicorp/terraform-provider-aws#2167

So i will make it work with multi-ports first.

[erikbor]

Done and closed.