terrascan consider source = "terraform-aws-modules/vpc/aws" as local path #418

ismailyenigul · 2020-12-02T21:21:19Z

terrascan version:v1.2.0
Operating System: MacOS

Description

We can call terraform registry modules iin source with just name of the module.
As described at https://github.com/terraform-aws-modules/terraform-aws-vpc#usage
terrascan consider them as local path and fails to load.

module "vpc" {
  source = "terraform-aws-modules/vpc/aws"

  name = "my-vpc"
  cidr = "10.0.0.0/16"

  azs             = ["eu-west-1a", "eu-west-1b", "eu-west-1c"]
  private_subnets = ["10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24"]
  public_subnets  = ["10.0.101.0/24", "10.0.102.0/24", "10.0.103.0/24"]

  enable_nat_gateway = true
  enable_vpn_gateway = true

  tags = {
    Terraform = "true"
    Environment = "dev"
  }
}

What I Did

2020-12-03T00:16:34.934+0300	debug	v12/module-download.go:87	failed to download remote module. error: 'error downloading 'file:///var/folders/p6/73m8k28j6hsfrpkt8g8yzzjr0000gn/T/9gtozg/terraform-aws-modules/vpc/aws': source path error: stat /var/folders/p6/73m8k28j6hsfrpkt8g8yzzjr0000gn/T/9gtozg/terraform-aws-modules/vpc/aws: no such file or directory'
2020-12-03T00:16:34.935+0300	error	v12/load-dir.go:101	failed to download remote module "terraform-aws-modules/vpc/aws". error: 'error downloading 'file:///var/folders/p6/73m8k28j6hsfrpkt8g8yzzjr0000gn/T/9gtozg/terraform-aws-modules/vpc/aws': source path error: stat /var/folders/p6/73m8k28j6hsfrpkt8g8yzzjr0000gn/T/9gtozg/terraform-aws-modules/vpc/aws: no such file or directory'
2020-12-03T00:16:34.935+0300	error	v12/load-dir.go:113	failed to build unified config. errors:
<nil>: Failed to read module directory; Module directory  does not exist or cannot be read.

If I set source as source = "git::https://github.com/terraform-aws-modules/terraform-aws-vpc.git?ref=v2.64.0"
then I got panic error.

ptions_domain_name_servers}" not present in parent module call
2020-12-03T00:22:23.591+0300	debug	v12/variable-references.go:57	extracted variable name "dhcp_options_domain_name_servers" from reference "${var.dhcp_options_domain_name_servers}"
2020-12-03T00:22:23.592+0300	debug	v12/module-download.go:118	deleting "git::https://github.com/terraform-aws-modules/terraform-aws-vpc.git?ref=v2.64.0" installed at "/var/folders/p6/73m8k28j6hsfrpkt8g8yzzjr0000gn/T/ix1aua"
panic: not a string

goroutine 1 [running]:
github.com/zclconf/go-cty/cty.Value.AsString(0x61dd120, 0xc0006824f0, 0x5e56d00, 0xc0016d4780, 0xc0016d4780, 0x61dd120)
	github.com/zclconf/[email protected]/cty/value_ops.go:1173 +0x1a5
github.com/zclconf/go-cty/cty.Value.AsValueMap(0x61dd1a0, 0xc003fdd8b0, 0x5a94500, 0xc0050b1c60, 0x1099e910)
	github.com/zclconf/[email protected]/cty/value_ops.go:1245 +0x1a6
github.com/accurics/terrascan/pkg/iac-providers/terraform/v12.ctyToMap(0x61dd1a0, 0xc003fdd8b0, 0x5a94500, 0xc0050b1c60, 0x5a94500, 0x7086dc0, 0x6184300, 0xc0016d45a0)
	github.com/accurics/terrascan/pkg/iac-providers/terraform/v12/cty-converters.go:67 +0x6a
github.com/accurics/terrascan/pkg/iac-providers/terraform/v12.(*RefResolver).ResolveVarRef(0xc005e1f698, 0xc002ee3590, 0x27, 0x5ad8ca0, 0xc0034aeaf0)
	github.com/accurics/terrascan/pkg/iac-providers/terraform/v12/variable-references.go:83 +0x1c5
github.com/accurics/terrascan/pkg/iac-providers/terraform/v12.(*RefResolver).ResolveStrRef(0xc005e1f698, 0xc002ee3590, 0x27, 0xb, 0xc000bdf868)
	github.com/accurics/terrascan/pkg/iac-providers/terraform/v12/references.go:166 +0x2a5
github.com/accurics/terrascan/pkg/iac-providers/terraform/v12.(*RefResolver).ResolveRefs(0xc005e1f698, 0xc0016f3c80, 0x19)
	github.com/accurics/terrascan/pkg/iac-providers/terraform/v12/references.go:70 +0x1b2
github.com/accurics/terrascan/pkg/iac-providers/terraform/v12.(*TfV12).LoadIacDir(0x7084b40, 0xc00057bd10, 0x24, 0xc0007fe4e0, 0x0, 0x0)
	github.com/accurics/terrascan/pkg/iac-providers/terraform/v12/load-dir.go:155 +0x65f
github.com/accurics/terrascan/pkg/runtime.(*Executor).Execute(0xc0002f0600, 0x0, 0x0, 0x0, 0xc0006c4c70, 0x1, 0x1, 0x0, 0x0)
	github.com/accurics/terrascan/pkg/runtime/executor.go:107 +0x9a7
github.com/accurics/terrascan/pkg/cli.Run(0x0, 0x0, 0x0, 0x0, 0xc0006c4c70, 0x1, 0x1, 0x0, 0x0, 0x5e62355, ...)
	github.com/accurics/terrascan/pkg/cli/run.go:62 +0x2bc
github.com/accurics/terrascan/pkg/cli.scan(0x702f3a0, 0xc000808000, 0x0, 0x2)
	github.com/accurics/terrascan/pkg/cli/scan.go:102 +0x1df
github.com/spf13/cobra.(*Command).execute(0x702f3a0, 0xc0006f1fe0, 0x2, 0x2, 0x702f3a0, 0xc0006f1fe0)
	github.com/spf13/[email protected]/command.go:846 +0x2c2
github.com/spf13/cobra.(*Command).ExecuteC(0x702f100, 0xc00013a010, 0x3, 0x3)
	github.com/spf13/[email protected]/command.go:950 +0x375
github.com/spf13/cobra.(*Command).Execute(...)
	github.com/spf13/[email protected]/command.go:887
github.com/accurics/terrascan/pkg/cli.Execute()
	github.com/accurics/terrascan/pkg/cli/register.go:85 +0x3c7
main.main()
	github.com/accurics/terrascan/cmd/terrascan/main.go:22 +0x25

The text was updated successfully, but these errors were encountered:

patilpankaj212 · 2021-01-19T17:27:22Z

Hey @ismailyenigul, we are working on fix for this issue.

ismailyenigul · 2021-03-02T16:32:11Z

Hi @kanchwala-yusuf

 cat main.tf 
provider "aws" {
  region = "eu-west-1"
}
module "web_server_sg" {
  source = "terraform-aws-modules/security-group/aws//modules/http-80"

  name        = "web-server"
  description = "Security group for web-server with HTTP ports open within VPC"
  vpc_id      = "vpc-12345678"

  ingress_cidr_blocks = ["10.10.0.0/16"]
}

terraform init works fine.

 terraform init
Initializing modules...
Downloading terraform-aws-modules/security-group/aws 3.18.0 for web_server_sg...
- web_server_sg in .terraform/modules/web_server_sg/modules/http-80
- web_server_sg.sg in .terraform/modules/web_server_sg

Initializing the backend...

Initializing provider plugins...
- Checking for available provider plugins...
- Downloading plugin for provider "aws" (hashicorp/aws) 3.30.0...

But terrascan scan't download codes and still consider the source as local path.

terrascan scan .
2021-03-02T19:24:32.025+0300	error	commons/load-dir.go:121	failed to build unified config. errors:
<nil>: Failed to read module directory; Module directory /Users/ismail/dev/vpc/sg/terraform-aws-modules/security-group/aws does not exist or cannot be read.

2021-03-02T19:24:32.069+0300	error	cli/run.go:103	scan run failed{error 26 0  failed to build terraform allResourcesConfig}

but terrascan works fine if source

source = "terraform-aws-modules/vpc/aws"

like below:

provider "aws" {
  region = "eu-west-2"
}

module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "2.70.0"

  name = "my-vpc"
  cidr = "10.0.0.0/16"

  azs             = ["eu-west-2a", "eu-west-2b", "eu-west-2c"]
  private_subnets = ["10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24"]
  public_subnets  = ["10.0.101.0/24", "10.0.102.0/24", "10.0.103.0/24"]

  enable_s3_endpoint              = true
  enable_dns_hostnames            = true
  enable_dns_support              = true
  create_database_subnet_group    = true
  create_elasticache_subnet_group = false
  enable_nat_gateway              = false
  enable_vpn_gateway              = false

  tags = {
    Terraform   = "true"
    Environment = "dev"
  }
}



1-03-02T19:28:35.270+0300	debug	opa/engine.go:92	skipping already loaded rego file{file 15 0 /Users/ismail/.terrascan/pkg/policies/opa/rego/aws/aws_security_group/portOpenToInternet.rego <nil>}
2021-03-02T19:28:35.274+0300	debug	opa/engine.go:208	loaded 159 Rego rules from 113 rego files (161 metadata files).
2021-03-02T19:28:35.493+0300	debug	runtime/executor.go:127	initialized executor
2021-03-02T19:28:36.334+0300	debug	downloader/getter.go:49	downloading "git::https://github.com/terraform-aws-modules/terraform-aws-vpc?ref=v2.70.0" to "/var/folders/p6/73m8k28j6hsfrpkt8g8yzzjr0000gn/T/lg4pv8"
2021-03-02T19:28:36.334+0300	debug	downloader/getter.go:57	remote URL: "git::https://github.com/terraform-aws-modules/terraform-aws-vpc?ref=v2.70.0"; url with type: "git::https://github.com/terraform-aws-modules/terraform-aws-vpc?ref=v2.70.0"
2021-03-02T19:28:36.334+0300	debug	downloader/getter.go:78	download with remote url: "git::https://github.com/terraform-aws-modules/terraform-aws-vpc?ref=v2.70.0", destination dir: "/var/folders/p6/73m8k28j6hsfrpkt8g8yzzjr0000gn/T/lg4pv8"
2021-03-02T19:28:36.335+0300	debug	downloader/getter.go:49	downloading "git::https://github.com/terraform-aws-modules/terraform-aws-vpc?ref=v2.70.0" to "/var/folders/p6/73m8k28j6hsfrpkt8g8yzzjr0000gn/T/lg4pv8"
2021-03-02T19:28:36.335+0300	debug	downloader/getter.go:57	remote URL: "git::https://github.com/terraform-aws-modules/terraform-aws-vpc?ref=v2.70.0"; url with type: "git::https://github.com/terraform-aws-modules/terraform-aws-vpc?ref=v2.70.0"

it seems that terrascan fails because of module path //modules/http-80 in the source = "terraform-aws-modules/security-group/aws//modules/http-80"

Should we re-open this issue or create new one?

kanchwala-yusuf · 2021-04-07T11:58:45Z

Hi @kanchwala-yusuf

 cat main.tf 
provider "aws" {
  region = "eu-west-1"
}
module "web_server_sg" {
  source = "terraform-aws-modules/security-group/aws//modules/http-80"

  name        = "web-server"
  description = "Security group for web-server with HTTP ports open within VPC"
  vpc_id      = "vpc-12345678"

  ingress_cidr_blocks = ["10.10.0.0/16"]
}

terraform init works fine.

 terraform init
Initializing modules...
Downloading terraform-aws-modules/security-group/aws 3.18.0 for web_server_sg...
- web_server_sg in .terraform/modules/web_server_sg/modules/http-80
- web_server_sg.sg in .terraform/modules/web_server_sg

Initializing the backend...

Initializing provider plugins...
- Checking for available provider plugins...
- Downloading plugin for provider "aws" (hashicorp/aws) 3.30.0...

But terrascan scan't download codes and still consider the source as local path.

terrascan scan .
2021-03-02T19:24:32.025+0300	error	commons/load-dir.go:121	failed to build unified config. errors:
<nil>: Failed to read module directory; Module directory /Users/ismail/dev/vpc/sg/terraform-aws-modules/security-group/aws does not exist or cannot be read.

2021-03-02T19:24:32.069+0300	error	cli/run.go:103	scan run failed{error 26 0  failed to build terraform allResourcesConfig}

but terrascan works fine if source

source = "terraform-aws-modules/vpc/aws"

like below:

provider "aws" {
  region = "eu-west-2"
}

module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "2.70.0"

  name = "my-vpc"
  cidr = "10.0.0.0/16"

  azs             = ["eu-west-2a", "eu-west-2b", "eu-west-2c"]
  private_subnets = ["10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24"]
  public_subnets  = ["10.0.101.0/24", "10.0.102.0/24", "10.0.103.0/24"]

  enable_s3_endpoint              = true
  enable_dns_hostnames            = true
  enable_dns_support              = true
  create_database_subnet_group    = true
  create_elasticache_subnet_group = false
  enable_nat_gateway              = false
  enable_vpn_gateway              = false

  tags = {
    Terraform   = "true"
    Environment = "dev"
  }
}



1-03-02T19:28:35.270+0300	debug	opa/engine.go:92	skipping already loaded rego file{file 15 0 /Users/ismail/.terrascan/pkg/policies/opa/rego/aws/aws_security_group/portOpenToInternet.rego <nil>}
2021-03-02T19:28:35.274+0300	debug	opa/engine.go:208	loaded 159 Rego rules from 113 rego files (161 metadata files).
2021-03-02T19:28:35.493+0300	debug	runtime/executor.go:127	initialized executor
2021-03-02T19:28:36.334+0300	debug	downloader/getter.go:49	downloading "git::https://github.com/terraform-aws-modules/terraform-aws-vpc?ref=v2.70.0" to "/var/folders/p6/73m8k28j6hsfrpkt8g8yzzjr0000gn/T/lg4pv8"
2021-03-02T19:28:36.334+0300	debug	downloader/getter.go:57	remote URL: "git::https://github.com/terraform-aws-modules/terraform-aws-vpc?ref=v2.70.0"; url with type: "git::https://github.com/terraform-aws-modules/terraform-aws-vpc?ref=v2.70.0"
2021-03-02T19:28:36.334+0300	debug	downloader/getter.go:78	download with remote url: "git::https://github.com/terraform-aws-modules/terraform-aws-vpc?ref=v2.70.0", destination dir: "/var/folders/p6/73m8k28j6hsfrpkt8g8yzzjr0000gn/T/lg4pv8"
2021-03-02T19:28:36.335+0300	debug	downloader/getter.go:49	downloading "git::https://github.com/terraform-aws-modules/terraform-aws-vpc?ref=v2.70.0" to "/var/folders/p6/73m8k28j6hsfrpkt8g8yzzjr0000gn/T/lg4pv8"
2021-03-02T19:28:36.335+0300	debug	downloader/getter.go:57	remote URL: "git::https://github.com/terraform-aws-modules/terraform-aws-vpc?ref=v2.70.0"; url with type: "git::https://github.com/terraform-aws-modules/terraform-aws-vpc?ref=v2.70.0"

it seems that terrascan fails because of module path //modules/http-80 in the source = "terraform-aws-modules/security-group/aws//modules/http-80"

Should we re-open this issue or create new one?

Hey @ismailyenigul ,
Sorry for the late reply!
A similar issue has been raised #600 and being addressed as part of #641

amirbenv assigned kanchwala-yusuf Jan 8, 2021

amirbenv added the bug label Jan 9, 2021

amirbenv assigned patilpankaj212 Jan 13, 2021

patilpankaj212 added a commit to patilpankaj212/terrascan that referenced this issue Jan 18, 2021

initial commit for tenable#418

b506c8c

devang-gaur mentioned this issue Jan 18, 2021

support for remote terraform modules #418 #498

Closed

patilpankaj212 linked a pull request Jan 20, 2021 that will close this issue

support for terraform registry remote modules #505

Merged

kanchwala-yusuf closed this as completed in #505 Jan 22, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

terrascan consider source = "terraform-aws-modules/vpc/aws" as local path #418

terrascan consider source = "terraform-aws-modules/vpc/aws" as local path #418

ismailyenigul commented Dec 2, 2020 •

edited

Loading

patilpankaj212 commented Jan 19, 2021

ismailyenigul commented Mar 2, 2021

kanchwala-yusuf commented Apr 7, 2021

terrascan consider source = "terraform-aws-modules/vpc/aws" as local path #418

terrascan consider source = "terraform-aws-modules/vpc/aws" as local path #418

Comments

ismailyenigul commented Dec 2, 2020 • edited Loading

Description

What I Did

patilpankaj212 commented Jan 19, 2021

ismailyenigul commented Mar 2, 2021

kanchwala-yusuf commented Apr 7, 2021

ismailyenigul commented Dec 2, 2020 •

edited

Loading