WAFv2 managed ruleset exclude_rules throwing error #819

pradeepjairamani · 2021-12-02T17:41:52Z

Summary

When I am trying to use the exclude_rules function while creating WAFv2 WebACL it's throwing an error (attached screenshot) not sure what is the reason

Ref: https://docs.ansible.com/ansible/latest/collections/community/aws/wafv2_web_acl_module.html

  - name: AWS-AWSManagedRulesAnonymousIpList
    priority: 1
    override_action:
      none: {}
    visibility_config:
      sampled_requests_enabled: yes
      cloud_watch_metrics_enabled: yes
      metric_name: AWS-AWSManagedRulesAnonymousIpList
    statement:
      managed_rule_group_statement:
        excluded_rules:
        - HostingProviderIPList

Issue Type

Bug Report

Component Name

community.aws.wafv2_web_acl

Ansible Version

ansible [core 2.11.0]
config file = /Users/pradeep/Important/Repos/ops/playbooks/ansible.cfg
configured module search path = ['/Users/pradeep/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /usr/local/Cellar/ansible/4.0.0/libexec/lib/python3.9/site-packages/ansible
ansible collection location = /Users/pradeep/.ansible/collections:/usr/share/ansible/collections
executable location = /usr/local/bin/ansible
python version = 3.9.5 (default, May 4 2021, 03:36:27) [Clang 12.0.0 (clang-1200.0.32.29)]
jinja version = 3.0.1
libyaml = True

Collection Versions

community.aws 1.5.0

AWS SDK versions

Name: boto3
Version: 1.17.59
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email: None
License: Apache License 2.0
Location: /usr/local/lib/python3.9/site-packages
Requires: botocore, jmespath, s3transfer
Required-by: dsnap

Name: botocore
Version: 1.20.59
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email: None
License: Apache License 2.0
Location: /usr/local/lib/python3.9/site-packages
Requires: urllib3, jmespath, python-dateutil
Required-by: s3transfer, boto3, awscli

Configuration

$ ansible-config dump --only-changed

OS / Environment

No response

Steps to Reproduce

name: create web acl
community.aws.wafv2_web_acl:
name: Istio-6Si-Dev-test2
state: present
description: Dev web Istio services
scope: REGIONAL
default_action: Allow
region: us-east-1
sampled_requests: yes
cloudwatch_metrics: yes
metric_name: Istio-6Si-Dev-test2
rules:
- name: AWS-AWSManagedRulesAnonymousIpList
priority: 1
override_action:
none: {}
visibility_config:
sampled_requests_enabled: yes
cloud_watch_metrics_enabled: yes
metric_name: AWS-AWSManagedRulesAnonymousIpList
statement:
managed_rule_group_statement:
excluded_rules:
- HostingProviderIPList

Expected Results

expected the code to exclude the rule as count in AWS WAF managed ruleset

Actual Results

The full traceback is:
Traceback (most recent call last):
File "/Users/pradeep/.ansible/tmp/ansible-tmp-1638465989.656811-19019-71493735332409/AnsiballZ_wafv2_web_acl.py", line 100, in
_ansiballz_main()
File "/Users/pradeep/.ansible/tmp/ansible-tmp-1638465989.656811-19019-71493735332409/AnsiballZ_wafv2_web_acl.py", line 92, in _ansiballz_main
invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
File "/Users/pradeep/.ansible/tmp/ansible-tmp-1638465989.656811-19019-71493735332409/AnsiballZ_wafv2_web_acl.py", line 40, in invoke_module
runpy.run_module(mod_name='ansible_collections.community.aws.plugins.modules.wafv2_web_acl', init_globals=dict(_module_fqn='ansible_collections.community.aws.plugins.modules.wafv2_web_acl', _modlib_path=modlib_path),
File "/usr/local/Cellar/[email protected]/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 210, in run_module
return _run_module_code(code, init_globals, run_name, mod_spec)
File "/usr/local/Cellar/[email protected]/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 97, in _run_module_code
_run_code(code, mod_globals, init_globals,
File "/usr/local/Cellar/[email protected]/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/var/folders/ts/vyjp0h097tq6zfw63l6_q5_00000gn/T/ansible_community.aws.wafv2_web_acl_payload_zof6b2ia/ansible_community.aws.wafv2_web_acl_payload.zip/ansible_collections/community/aws/plugins/modules/wafv2_web_acl.py", line 413, in
File "/var/folders/ts/vyjp0h097tq6zfw63l6_q5_00000gn/T/ansible_community.aws.wafv2_web_acl_payload_zof6b2ia/ansible_community.aws.wafv2_web_acl_payload.zip/ansible_collections/community/aws/plugins/modules/wafv2_web_acl.py", line 349, in main
File "/var/folders/ts/vyjp0h097tq6zfw63l6_q5_00000gn/T/ansible_community.aws.wafv2_web_acl_payload_zof6b2ia/ansible_community.aws.wafv2_web_acl_payload.zip/ansible_collections/community/aws/plugins/module_utils/wafv2.py", line 57, in wafv2_snake_dict_to_camel_dict
File "/var/folders/ts/vyjp0h097tq6zfw63l6_q5_00000gn/T/ansible_community.aws.wafv2_web_acl_payload_zof6b2ia/ansible_community.aws.wafv2_web_acl_payload.zip/ansible_collections/community/aws/plugins/module_utils/wafv2.py", line 57, in wafv2_snake_dict_to_camel_dict
File "/var/folders/ts/vyjp0h097tq6zfw63l6_q5_00000gn/T/ansible_community.aws.wafv2_web_acl_payload_zof6b2ia/ansible_community.aws.wafv2_web_acl_payload.zip/ansible_collections/community/aws/plugins/module_utils/wafv2.py", line 61, in wafv2_snake_dict_to_camel_dict
File "/var/folders/ts/vyjp0h097tq6zfw63l6_q5_00000gn/T/ansible_community.aws.wafv2_web_acl_payload_zof6b2ia/ansible_community.aws.wafv2_web_acl_payload.zip/ansible_collections/community/aws/plugins/module_utils/wafv2.py", line 50, in wafv2_snake_dict_to_camel_dict
AttributeError: 'str' object has no attribute 'keys'

Code of Conduct

I agree to follow the Ansible Code of Conduct

The text was updated successfully, but these errors were encountered:

markuman · 2021-12-02T17:51:17Z

@pradeepjairamani the managed_rule_group_statement must look like this

    statement:
      managed_rule_group_statement:
        vendor_name: AWS
        name: AWSManagedRulesCommonRuleSet
        excluded_rules:
          - name: SizeRestrictions_BODY

pradeepjairamani · 2021-12-02T19:35:03Z

@markuman I am using the same method

markuman · 2021-12-02T20:24:30Z

But you've pasted another

  - name: AWS-AWSManagedRulesAnonymousIpList
    priority: 1
    override_action:
      none: {}
    visibility_config:
      sampled_requests_enabled: yes
      cloud_watch_metrics_enabled: yes
      metric_name: AWS-AWSManagedRulesAnonymousIpList
    statement:
      managed_rule_group_statement:
        excluded_rules:
        - HostingProviderIPList

Are you sure?

  - name: AWS-AWSManagedRulesAnonymousIpList
    priority: 1
    override_action:
      none: {}
    visibility_config:
      sampled_requests_enabled: yes
      cloud_watch_metrics_enabled: yes
      metric_name: AWS-AWSManagedRulesAnonymousIpList
    statement:
      managed_rule_group_statement:
        excluded_rules:
-        - HostingProviderIPList
+        - name: HostingProviderIPList

pradeepjairamani · 2021-12-03T04:13:16Z

Hey sorry there was a little confusion @markuman, I tried using it with

- name: HostingProviderIPList

before but it's showing a parameter validation failed error

markuman · 2021-12-03T05:29:52Z

Works for me

---
- hosts: localhost
  connection: local

  tasks:
    - name: test web acl
      community.aws.wafv2_web_acl:
        name: test_waf
        state: present
        description: ansible test waf
        scope: REGIONAL
        default_action: Allow
        sampled_requests: yes
        cloudwatch_metrics: yes
        region: eu-central-1
        rules:
          - priority: 0
            name: AWS-AWSManagedRulesAnonymousIpList
            override_action:
              none: {}
            visibility_config:
              sampled_requests_enabled: yes
              cloud_watch_metrics_enabled: yes
              metric_name: AWS-AWSManagedRulesAnonymousIpList
            statement:
              managed_rule_group_statement:
                vendor_name: AWS
                name: AWSManagedRulesKnownBadInputsRuleSet
                excluded_rules:
                  - name: HostingProviderIPList
        
    - name: test web acl
      community.aws.wafv2_web_acl_info:
        name: test_waf
        scope: REGIONAL
        region: eu-central-1
      register: out
    - debug:
        var: out.rules

results in

PLAY [localhost] ********************************************************************************************************************************************************************************************************************

TASK [Gathering Facts] **************************************************************************************************************************************************************************************************************
ok: [localhost]

TASK [test web acl] *****************************************************************************************************************************************************************************************************************
ok: [localhost]

TASK [test web acl] *****************************************************************************************************************************************************************************************************************
ok: [localhost]

TASK [debug] ************************************************************************************************************************************************************************************************************************
ok: [localhost] => {
    "out.rules": [
        {
            "name": "AWS-AWSManagedRulesAnonymousIpList",
            "override_action": {
                "none": {}
            },
            "priority": 0,
            "statement": {
                "managed_rule_group_statement": {
                    "excluded_rules": [
                        {
                            "name": "HostingProviderIPList"
                        }
                    ],
                    "name": "AWSManagedRulesKnownBadInputsRuleSet",
                    "vendor_name": "AWS"
                }
            },
            "visibility_config": {
                "cloud_watch_metrics_enabled": true,
                "metric_name": "AWS-AWSManagedRulesAnonymousIpList",
                "sampled_requests_enabled": true
            }
        }
    ]
}

PLAY RECAP **************************************************************************************************************************************************************************************************************************
localhost                  : ok=4    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

When you click your rules using the webconsole, you can read them out via community.aws.wafv2_web_acl_info to see how the rules are build.

You can also throw the rules in to your_role/defaults/main.yml with out.rules | to_nice_yaml nice yaml filter.
Then just change each element in a single object like

WAFv2_module:
  anon_ip:
    priority: 0
    name: AWS-AWSManagedRulesAnonymousIpList
    override_action:
      none: {}
    visibility_config:
      sampled_requests_enabled: yes
      cloud_watch_metrics_enabled: yes
      metric_name: AWS-AWSManagedRulesAnonymousIpList
    statement:
      managed_rule_group_statement:
        vendor_name: AWS
        name: AWSManagedRulesKnownBadInputsRuleSet
        excluded_rules:
          - name: HostingProviderIPList

Then you only write your rules once and can apply them on multiple WAFs if needed.

    - name: php web acl
      community.aws.wafv2_web_acl:
        name: php_waf
        state: present
        description: public php waf
        scope: REGIONAL
        default_action: Allow
        sampled_requests: yes
        cloudwatch_metrics: yes
        rules:
          - "{{ WAFv2_module.anon_ip }}"

You must only take care about the priority. But basically you can also remove the priority from you defaults/main.yml file and add a filter | add_int_priority that will dynamically add the priority, based on the position in the array.

#!/usr/bin/python
class FilterModule(object):
    def filters(self):
        return {
            'add_int_priority': self.add_int_priority
        }

    def add_int_priority(self, input):
        retval = list()
        counter = 1
        for item in input:
            item['priority'] = int(counter)
            counter += 1
            retval.append(item)
        return retval

pradeepjairamani · 2021-12-03T08:48:48Z

I reinstalled the repo and now it's working, thanks for all the help @markuman

wafv2_web_acl: more examples SUMMARY Closes: #819 More examples Ref to official documentation for managed rules ISSUE TYPE Docs Pull Request COMPONENT NAME wafv2_wab_acl Reviewed-by: Alina Buzachis <None> Reviewed-by: None <None>

wafv2_web_acl: more examples SUMMARY Closes: #819 More examples Ref to official documentation for managed rules ISSUE TYPE Docs Pull Request COMPONENT NAME wafv2_wab_acl Reviewed-by: Alina Buzachis <None> Reviewed-by: None <None> (cherry picked from commit 9790a36)

[PR #821/9790a36d backport][stable-2] wafv2_web_acl: more examples This is a backport of PR #821 as merged into main (9790a36). SUMMARY Closes: #819 More examples Ref to official documentation for managed rules ISSUE TYPE Docs Pull Request COMPONENT NAME wafv2_wab_acl Reviewed-by: Markus Bergholz <[email protected]> Reviewed-by: None <None>

…ion requirements in line with collection requirements (ansible-collections#819) Update plugin docs to explicitly list python, boto3 and botocore version requirements SUMMARY Update plugin docs to explicitly list python, boto3 and botocore version requirements in line with collection requirements. ISSUE TYPE Docs Pull Request COMPONENT NAME plugins/inventory/aws_ec2.py plugins/inventory/aws_rds.py plugins/lookup/aws_account_attribute.py plugins/lookup/aws_secret.py plugins/lookup/aws_ssm.py ADDITIONAL INFORMATION This is in line with the collection requirements, just trying to make it easier to find. Reviewed-by: Alina Buzachis <None>

markuman added the docs label Dec 2, 2021

markuman mentioned this issue Dec 3, 2021

wafv2_web_acl: more examples #821

Merged

pradeepjairamani closed this as completed Dec 3, 2021

patchback bot mentioned this issue Dec 10, 2021

[PR #821/9790a36d backport][stable-2] wafv2_web_acl: more examples #832

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WAFv2 managed ruleset exclude_rules throwing error #819

WAFv2 managed ruleset exclude_rules throwing error #819

pradeepjairamani commented Dec 2, 2021

markuman commented Dec 2, 2021

pradeepjairamani commented Dec 2, 2021

markuman commented Dec 2, 2021

pradeepjairamani commented Dec 3, 2021

markuman commented Dec 3, 2021 •

edited

Loading

pradeepjairamani commented Dec 3, 2021

WAFv2 managed ruleset exclude_rules throwing error #819

WAFv2 managed ruleset exclude_rules throwing error #819

Comments

pradeepjairamani commented Dec 2, 2021

Summary

Issue Type

Component Name

Ansible Version

Collection Versions

AWS SDK versions

Name: boto3 Version: 1.17.59 Summary: The AWS SDK for Python Home-page: https://github.com/boto/boto3 Author: Amazon Web Services Author-email: None License: Apache License 2.0 Location: /usr/local/lib/python3.9/site-packages Requires: botocore, jmespath, s3transfer Required-by: dsnap

Configuration

OS / Environment

Steps to Reproduce

Expected Results

Actual Results

Code of Conduct

markuman commented Dec 2, 2021

pradeepjairamani commented Dec 2, 2021

markuman commented Dec 2, 2021

pradeepjairamani commented Dec 3, 2021

markuman commented Dec 3, 2021 • edited Loading

pradeepjairamani commented Dec 3, 2021

Name: boto3
Version: 1.17.59
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email: None
License: Apache License 2.0
Location: /usr/local/lib/python3.9/site-packages
Requires: botocore, jmespath, s3transfer
Required-by: dsnap

markuman commented Dec 3, 2021 •

edited

Loading