New resource being ignored by tests

[mrcrilly]

Description :
I am adding an aws_subnet to my Terraform code base without tags, producing a plan file that includes the new resource (which has no tags), and then passing this to terraform-compliance, which is not applying the follow Scenario Outline to the new resource:

    Scenario Outline: Ensure we have specific tags in place
        Given I have resource that supports tags defined
        Then it must contain <tags>
        And its value must match the "<value>" regex

        Examples:
        | tags          | value              |
        | Name          | ^mcrilly\-.+$      |
        | Environment   | ^Sandbox$          |

To Reproduce

1. See below HCL

resource "aws_vpc" "test" {
  cidr_block = "10.103.0.0/16"
  tags = {
    Name = "mcrilly-sandbox"
    Environment = "Sandbox"
  }
}

resource "aws_subnet" "test" {
  cidr_block = "10.103.1.0/24"
  vpc_id = aws_vpc.test.id
}

2. terraform-compliance -f compliance/ -p plan.out
3. I'm using the Python package directly (installed via pip)
4. The output can be seen below:

Feature: Correctly defined tags on all applicable resources  # compliance/tags.feature
    I need to ensure all resources that support tags have a bare minimal set in place

    Scenario Outline: Ensure we have specific tags in place
        Given I have resource that supports tags defined
        Then it must contain <tags>
        And its value must match the "<value>" regex

    Examples:
        | tags        | value        |
        | Name        | ^mcrilly-.+$ |
        | Environment | ^Sandbox$    |

1 features (1 passed)
2 scenarios (2 passed)
6 steps (6 passed)
Run 1572323242 finished within a moment

5. tags.feature...

    Scenario Outline: Ensure we have specific tags in place
        Given I have resource that supports tags defined
        Then it must contain <tags>
        And its value must match the "<value>" regex

        Examples:
        | tags          | value              |
        | Name          | ^mcrilly\-.+$      |
        | Environment   | ^Sandbox$          |

Expected behavior :
I expect the tests to fail given aws_subnet.test does not have any tags defined. It is a resource that can be given tags. My Scenario Outline checks for all resources that support tags and tests them for a minimal set. My aws_subnet.test resource is not being included.

I would expect the tests to fail when attempting to add a new resource that does not comply with the requirements defined.

Tested versions :

$ terraform-compliance -v
terraform-compliance v1.0.51 initiated

1.0.51

$ terraform version
Terraform v0.12.10
+ provider.aws v2.33.0

$ python --version
Python 2.7.15

[mrcrilly]

I'm going to assume "Many missing resources requires tagging." (from the README.md file) is the cause of this issue?

Essentially you've not covered all resources yet? :-)

[eerkunt]

Thanks for reporting this @mrcrilly 🎉

All resources that support tags (according to the utilised terraform provider) will be recognised by terraform-compliance as resource that supports tag.

This is quite a standard test that has been used by many, interestingly failing for you. Debugging.

[eerkunt]

Found the problem, it is due to resource mounting capability of terraform-compliance where aws_vpc is mounted within aws_subnet and because the checks are recursive, on aws_subnet test, it drills down on aws_vpc and found tags that are valid by the test.

Unfortunately, this makes the test pass while it is expected to be failed. Will find a good structure to fix this problem and release the new version.

Thanks again!

[eerkunt]

1.0.52 has been released for fixing this issue. Would it be possible to have a quick re-test with the new version ?

By the way, I highly recommend to upgrade your Python environment, since 2.7 will be obsolete soon and is not supported by terraform-compliance officially :)

[eerkunt]

Hi @mrcrilly,

Did you find any time to re-test your problem with the latest version ?

[eerkunt]

Assuming the problem has been resolved, closing the issue. Please do not hesitate to open a new one if the problem still continue.

Thanks 🎉