Add remediation job to enable ELB access logs

remediation_worker/jobs/elb_enable_access_logs/README.md

@@ -0,0 +1,70 @@
+# Enable Elastic Load Balancer access logs
+
+This job enables access logs for a classic Elastic Load Balancer (if they are not already enabled).
+It will create a new S3 bucket with the required permissions and configure it to receive access logs from the ELB.
+
+### Applicable Rule
+
+##### Rule ID:
+657c46b7-1cd0-4cce-80bb-9d195f49c987
+
+##### Rule Name:
+Elastic Load Balancer access logs are not enabled
+
+## Getting Started
+
+### Prerequisites
+
+The provided AWS credential must have access to `elb:DescribeLoadBalancerAttributes`, `s3:CreateBucket`, `s3:PutBucketPolicy`, and `elb:ModifyLoadBalancerAttributes`.
+
+You may find the latest example policy file [here](minimum_policy.json).
+
+### Running the script
+
+You may run this script using following commands:
+```shell script
+  pip install -r ../../requirements.txt
+  python3 elb_enable_access_logs.py
+```
+
+## Running the tests
+You may run test using following command under vss-remediation-worker-job-code-python directory:
+```shell script
+    pip install -r requirements-dev.txt
+    python3 -m pytest test
+```
+## Deployment
+1. Provision a Virtual Machine
+Create an EC2 instance to use for the worker. The minimum required specifications are 128 MB memory and 1/2 Core CPU.
+2. Setup Docker
+Install Docker on the newly provisioned EC2 instance. You can refer to the [docs here](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/docker-basics.html) for more information.
+3. Deploy the worker image
+SSH into the EC2 instance and run the command below to deploy the worker image:
+```shell script
+  docker run --rm -it --name worker \
+  -e VSS_CLIENT_ID={ENTER CLIENT ID}
+  -e VSS_CLIENT_SECRET={ENTER CLIENT SECRET} \
+  vmware/vss-remediation-worker:latest-python
+```
+
+
+## Contributing
+The Secure State team welcomes contributions from the community. If you wish to contribute code and you have not signed our contributor license agreement (CLA), our bot will update the issue when you open a Pull Request. For any questions about the CLA process, please refer to our [FAQ](https://cla.vmware.com/faq).
+All contributions to this repository must be signed as described on that page. Your signature certifies that you wrote the patch or have the right to pass it on as an open-source patch.
+
+For more detailed information, refer to [CONTRIBUTING.md](../../../CONTRIBUTING.md).
+
+## Versioning
+
+We use [SemVer](http://semver.org/) for versioning. For the versions available, see the [tags on this repository](https://github.com/vmware-samples/secure-state-remediation-jobs/tags).
+
+## Authors
+
+* **VMware Secure State** - *Initial work*
+
+See also the list of [contributors](https://github.com/vmware-samples/secure-state-remediation-jobs/contributors) who
+ participated in this project.
+
+## License
+
+This project is licensed under the Apache License - see the [LICENSE](https://github.com/vmware-samples/secure-state-remediation-jobs/blob/master/LICENSE.txt) file for details

remediation_worker/jobs/elb_enable_access_logs/constraints.txt

@@ -0,0 +1,43 @@
+docutils==0.15.2 \
+    --hash=sha256:6c4f696463b79f1fb8ba0c594b63840ebd41f059e92b31957c46b74a4599b6d0 \
+    --hash=sha256:9e4d7ecfc600058e07ba661411a2b7de2fd0fafa17d1a7f7361cd47b1175c827 \
+    --hash=sha256:a2aeea129088da402665e92e0b25b04b073c04b2dce4ab65caaa38b7ce2e1a99
+jmespath==0.10.0 \
+    --hash=sha256:b85d0567b8666149a93172712e68920734333c0ce7e89b78b3e987f71e5ed4f9 \
+    --hash=sha256:cdf6525904cc597730141d61b36f2e4b8ecc257c420fa2f4549bac2c2d0cb72f
+python-dateutil==2.8.1 \
+    --hash=sha256:73ebfe9dbf22e832286dafa60473e4cd239f8592f699aa5adaf10050e6e1823c \
+    --hash=sha256:75bb3f31ea686f1197762692a9ee6a7550b59fc6ca3a1f4b5d7e32fb98e2da2a
+s3transfer==0.3.3 \
+    --hash=sha256:2482b4259524933a022d59da830f51bd746db62f047d6eb213f2f8855dcb8a13 \
+    --hash=sha256:921a37e2aefc64145e7b73d50c71bb4f26f46e4c9f414dc648c6245ff92cf7db
+urllib3==1.25.9 \
+    --hash=sha256:3018294ebefce6572a474f0604c2021e33b3fd8006ecd11d62107a5d2a963527 \
+    --hash=sha256:88206b0eb87e6d677d424843ac5209e3fb9d0190d0ee169599165ec25e9d9115
+six==1.15.0 \
+    --hash=sha256:30639c035cdb23534cd4aa2dd52c3bf48f06e5f4a941509c8bafd8ce11080259 \
+    --hash=sha256:8b74bedcbbbaca38ff6d7491d76f2b06b3592611af620f8426e82dddb04a5ced
+packaging==20.4 \
+    --hash=sha256:4357f74f47b9c12db93624a82154e9b120fa8293699949152b22065d556079f8 \
+    --hash=sha256:998416ba6962ae7fbd6596850b80e17859a5753ba17c32284f67bfff33784181
+attrs==19.3.0 \
+    --hash=sha256:08a96c641c3a74e44eb59afb61a24f2cb9f4d7188748e76ba4bb5edfa3cb7d1c \
+    --hash=sha256:f7b7ce16570fe9965acd6d30101a28f62fb4a7f9e926b3bbc9b61f8b04247e72
+more-itertools==8.4.0 \
+    --hash=sha256:68c70cc7167bdf5c7c9d8f6954a7837089c6a36bf565383919bb595efb8a17e5 \
+    --hash=sha256:b78134b2063dd214000685165d81c154522c3ee0a1c0d4d113c80361c234c5a2
+pluggy==0.13.1 \
+    --hash=sha256:15b2acde666561e1298d71b523007ed7364de07029219b604cf808bfa1c765b0 \
+    --hash=sha256:966c145cd83c96502c3c3868f50408687b38434af77734af1e9ca461a4081d2d
+py==1.9.0 \
+    --hash=sha256:366389d1db726cd2fcfc79732e75410e5fe4d31db13692115529d34069a043c2 \
+    --hash=sha256:9ca6883ce56b4e8da7e79ac18787889fa5206c79dcc67fb065376cd2fe03f342
+toml==0.10.1 \
+    --hash=sha256:926b612be1e5ce0634a2ca03470f95169cf16f939018233a670519cb4ac58b0f \
+    --hash=sha256:bda89d5935c2eac546d648028b9901107a595863cb36bae0c73ac804a9b4ce88
+iniconfig==1.0.1 \
+    --hash=sha256:80cf40c597eb564e86346103f609d74efce0f6b4d4f30ec8ce9e2c26411ba437 \
+    --hash=sha256:e5f92f89355a67de0595932a6c6c02ab4afddc6fcdc0bfc5becd0d60884d3f69
+pyparsing==2.4.7 \
+    --hash=sha256:c203ec8783bf771a155b207279b9bccb8dea02d8f0c9e5f8ead507bc3246ecc1 \
+    --hash=sha256:ef9d7589ef3c200abe66653d3f1ab1033c3c419ae9b9bdb1240a85b024efc88b

remediation_worker/jobs/elb_enable_access_logs/elb_enable_access_logs.py

@@ -0,0 +1,182 @@
+# Copyright (c) 2020 VMware Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import boto3
+import json
+import sys
+import logging
+logging.basicConfig(level=logging.INFO)
+
+from botocore.exceptions import ClientError
+
+def logcall(f, *args, **kwargs):
+    logging.info('%s(%s)', f.__name__, ', '.join(list(args) + [f'{k}={repr(v)}' for k, v in kwargs.items()]))
+    res = f(*args, **kwargs)
+    logging.info(res)
+    return res
+
+# taken from https://docs.aws.amazon.com/elasticloadbalancing/latest/classic/enable-access-logs.html
+ELB_ACCOUNT_IDS = {
+    'us-east-1': '127311923021',
+    'us-east-2': '033677994240',
+    'us-west-1': '027434742980',
+    'us-west-2': '797873946194',
+    'af-south-1': '098369216593',
+    'ca-central-1': '985666609251',
+    'eu-central-1': '054676820928',
+    'eu-west-1': '156460612806',
+    'eu-west-2': '652711504416',
+    'eu-south-1': '635631232127',
+    'eu-west-3': '009996457667',
+    'eu-north-1': '897822967062',
+    'ap-east-1': '754344448648',
+    'ap-northeast-1': '582318560864',
+    'ap-northeast-2': '600734575887',
+    'ap-northeast-3': '383597477331',
+    'ap-southeast-1': '114774131450',
+    'ap-southeast-2': '783225319266',
+    'ap-south-1': '718504428378',
+    'me-south-1': '076674570225',
+    'sa-east-1': '507241528517',
+    'us-gov-west-1': '048591011584',
+    'us-gov-east-1': '190560391635',
+    'cn-north-1': '638102146993',
+    'cn-northwest-1': '037604701340'
+}
+
+def create_or_update_bucket_policy(s3_client, bucket_name, bucket_prefix, account_id, region):
+    elb_account_id = ELB_ACCOUNT_IDS[region]
+    statement = {
+        'Effect': 'Allow',
+        'Principal': {
+            'AWS': f'arn:aws:iam::{elb_account_id}:root'
+        },
+        'Action': 's3:PutObject',
+        'Resource': f'arn:aws:s3:::{bucket_name}/{bucket_prefix}/AWSLogs/{account_id}/*'
+    }
+    try:
+        policy = json.loads(logcall(s3_client.get_bucket_policy, Bucket=bucket_name)['Policy'])
+        if statement not in policy['Statement']:
+            policy['Statement'].append(statement)
+            logcall(
+                s3_client.put_bucket_policy,
+                Bucket=bucket_name,
+                Policy=json.dumps(policy)
+            )
+    except ClientError as e:
+        if e.response['Error']['Code'] == 'NoSuchBucketPolicy':
+            policy = {
+                'Version': '2012-10-17',
+                'Statement': [
+                    statement
+                ]
+            }
+            logcall(
+                s3_client.put_bucket_policy,
+                Bucket=bucket_name,
+                Policy=json.dumps(policy)
+            )
+        else:
+            raise e
+
+class ELBEnableAccessLogs(object):
+    def parse(self, payload):
+        """Parse payload received from Remediation Service.
+
+        :param payload: JSON string containing parameters received from the remediation service.
+        :type payload: str.
+        :returns: Dictionary of parsed parameters
+        :rtype: dict
+        :raises: KeyError, JSONDecodeError
+        """
+        payload_dict = json.loads(payload)
+        return {
+            'elb_name': payload_dict['notificationInfo']['FindingInfo']['ObjectId'],
+            'region': payload_dict['notificationInfo']['FindingInfo']['Region'],
+            'cloud_account_id': payload_dict['notificationInfo']['CloudAccountId']
+        }
+
+    def ensure_log_target_bucket(self, s3_client, target_bucket, region):
+        try:
+            logcall(s3_client.head_bucket,
+                    Bucket=target_bucket)
+        except ClientError as e:
+            if e.response["Error"]["Code"] == "404":
+                # The bucket does not exist
+                if region == "us-east-1":
+                    logcall(s3_client.create_bucket,
+                            Bucket=target_bucket)
+                else:
+                    logcall(s3_client.create_bucket,
+                        Bucket=target_bucket,
+                        CreateBucketConfiguration={"LocationConstraint": region}
+                    )
+            elif e.response["Error"]["Code"] == "403":
+                # The assumed role does not have the permission
+                logging.error("Not enough permissions to list buckets")
+                raise e
+            else:
+                raise e
+
+    def remediate(self, elb_client, s3_client, elb_name, cloud_account_id, region):
+        """Enables access logs for the given ELB.
+
+        :param elb_client: AWS ELB boto3 client
+        :param s3_client: AWS S3 boto3 client
+        :param elb_name: Name of elastic load balancer
+        :param cloud_account_id: Customer cloud account id
+        :returns: Integer signaling success or failure
+        :rtype: int
+        :raises: botocore.exceptions.ClientError
+        """
+
+        logs_enabled = logcall(elb_client.describe_load_balancer_attributes, LoadBalancerName=elb_name)['LoadBalancerAttributes']['AccessLog']['Enabled']
+
+        if logs_enabled:
+            logging.info('access logs already enabled')
+        else:
+            logging.info('enabling access logs')
+            bucket_name = f'vss-logging-target-{cloud_account_id}-{region}'
+            bucket_prefix = elb_name
+            self.ensure_log_target_bucket(s3_client, bucket_name, region)
+            create_or_update_bucket_policy(s3_client, bucket_name, bucket_prefix, cloud_account_id, region)
+            logcall(
+                elb_client.modify_load_balancer_attributes,
+                LoadBalancerName=elb_name,
+                LoadBalancerAttributes={
+                    'AccessLog': {
+                        'Enabled': True,
+                        'S3BucketName': bucket_name,
+                        'S3BucketPrefix': bucket_prefix
+                    }
+                }
+            )
+
+        return 0
+
+    def run(self, args):
+        """Run the remediation job.
+
+        :param args: List of arguments provided to the job.
+        :type args: list.
+        :returns: int
+        """
+        params = self.parse(args[1])
+        elb_client = boto3.client('elb', region_name=params['region'])
+        s3_client = boto3.client('s3', region_name=params['region'])
+        return self.remediate(elb_client, s3_client, **params)
+
+
+if __name__ == '__main__':
+    sys.exit(ELBEnableAccessLogs().run(sys.argv))

remediation_worker/jobs/elb_enable_access_logs/minimum_policy.json

@@ -0,0 +1,16 @@
+{
+    "Version": "2012-10-17",
+    "Statement": [
+        {
+            "Sid": "ELBEnableAccessLogs",
+            "Effect": "Allow",
+            "Action": [
+                "elb:DescribeLoadBalancerAttributes",
+                "s3:CreateBucket",
+                "s3:PutBucketPolicy",
+                "elb:ModifyLoadBalancerAttributes"
+            ],
+            "Resource": "*"
+        }
+    ]
+}

remediation_worker/jobs/elb_enable_access_logs/requirements-dev.txt

@@ -0,0 +1,6 @@
+-r requirements.txt
+-c constraints.txt
+
+pytest==6.0.1 \
+    --hash=sha256:85228d75db9f45e06e57ef9bf4429267f81ac7c0d742cc9ed63d09886a9fe6f4 \
+    --hash=sha256:8b6007800c53fdacd5a5c192203f4e531eb2a1540ad9c752e052ec0f7143dbad

remediation_worker/jobs/elb_enable_access_logs/requirements.txt

@@ -0,0 +1,5 @@
+boto3==1.14.9 \
+  --hash=sha256:185f7b36c16f76e501d8dfc5cd209113426e078e4968dd13cc355c916bc99597 \
+  --hash=sha256:51243ba0e976343ca0b98bb4a15fc3d588526220f6ba45bfed7ea45472b1e033
+botocore==1.17.9 \
+  --hash=sha256:7dd59bc766d567ca83bc6113aa139d92ba447738ccdfcd40788848553d329a52

remediation_worker/jobs/elb_enable_access_logs/test_payload.json

@@ -0,0 +1,9 @@
+{
+	"notificationInfo": {
+		"CloudAccountId": "650397460025",
+		"FindingInfo": {
+			"ObjectId": "jackson-test-classic-elb",
+			"Region": "us-east-1"
+		}
+	}
+}