Semi Rewrite/ Major Update (#4)

* first pass of rewrite. Added multithreading and more services

* tests, tox and accounts update

* silly yaml

* more services and moved to separate folder

* added efs

* bump version

.github/workflows/test_and_publish.yaml

@@ -8,7 +8,7 @@ jobs:
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        python-version: [3.7, 3.8, 3.9]
+        python-version: ["3.9", "3.10", "3.11"]
 
     steps:
       - uses: actions/checkout@v2
@@ -31,10 +31,10 @@ jobs:
     if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')
     steps:
     - uses: actions/checkout@master
-    - name: Set up Python 3.9
+    - name: Set up Python 3.10
       uses: actions/setup-python@v1
       with:
-        python-version: 3.9
+        python-version: "3.10"
     - name: Install dependencies
       run: |
         python -m pip install --upgrade pip

README.md

@@ -2,14 +2,18 @@
 
 > Xenos, is Greek for stranger.
 
-AWSXenos will list all the trust relationships in all the IAM roles, and S3 buckets, in an AWS account and give you a breakdown of all the accounts that have trust relationships to your account. It will also highlight whether the trusts have an [external ID](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-user_externalid.html) or not.
+AWSXenos will assess the trust relationships in all the IAM roles, and resource policies for [several AWS services](#aws-iam-access-analyzer-comparison) in an AWS account and give you a breakdown of all the accounts that have trust relationships to your account. It will also highlight whether the trusts have an [external ID](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-user_externalid.html) or not.
 
-This tool reports against the [Trusted Relationship Technique](https://attack.mitre.org/techniques/T1199/) of the ATT&CK Framework. 
+https://docs.aws.amazon.com/IAM/latest/UserGuide/what-is-access-analyzer.html
 
-* For the "known" accounts list AWSXenos uses a modified version of [known AWS Account IDs](https://github.com/rupertbg/aws-public-account-ids).
-* For the Org accounts list AWSXenos query AWS Organizations.
+This tool reports against the [Trusted Relationship Technique](https://attack.mitre.org/techniques/T1199/) and parts of the [Valid Accounts: Cloud Accounts](https://attack.mitre.org/techniques/T1078/004/) of the ATT&CK Framework. 
+
+* For the "known" accounts list AWSXenos uses a modified version of [known AWS Accounts](https://github.com/fwdcloudsec/known_aws_accounts).
+* For the Org accounts list, AWSXenos will query AWS Organizations.
 * AWS Services are classified separately.
 * Everything else falls under unknown account
+* For regional services, e.g. KMS, you'll need to run AWSXenos per region.
+* You can configure which services you'd like to assess by providing a [config.yaml](awsxenos/config.yaml) file.
 
 ## Example
 ![HTML Report Screenshot](screenshots/report.png)
@@ -26,6 +30,46 @@ Access Analyzer falls short because:
 
 4. Does not identify AWS Service principals. This is mainly important because of [Wiz's AWSConfig, et al vulnverabilities](http://i.blackhat.com/USA21/Wednesday-Handouts/us-21-Breaking-The-Isolation-Cross-Account-AWS-Vulnerabilities.pdf)
 
+## AWS IAM Access Analyzer comparison 
+
+Comparison based on AWS Documentation [1](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_aws-services-that-work-with-iam.html) and [2](https://docs.aws.amazon.com/IAM/latest/UserGuide/what-is-access-analyzer.html#what-is-access-analyzer-resource-identification), including services or resources outside of docs, e.g. VPC endpoints.
+
+
+| Service | AWSXenos | Access Analyzer |
+| :--: | :--: | :--: |
+| S3 Bucket| :white_check_mark: |  :white_check_mark: |
+| S3 Directory Buckets | :x: |  :white_check_mark: |
+| S3 Access Points | :x: |  :white_check_mark: |
+| S3 Bucket ACLs | :white_check_mark: | :white_check_mark: |
+| S3 Glacier | :white_check_mark: | :x: |
+| IAM |  :white_check_mark: |  :white_check_mark: |
+| KMS |  :white_check_mark: |  :white_check_mark: |
+| Secrets Manager |  :white_check_mark: |  :white_check_mark: |
+| Lambda | :white_check_mark: |  :white_check_mark: |
+| SNS | :x: |  :white_check_mark: |
+| SQS | :white_check_mark: |  :white_check_mark: |
+| RDS Snapshots | :x: |  :white_check_mark: |
+| RDS Cluster Snapshots | :x: |  :white_check_mark: |
+| ECR | :x: |  :white_check_mark: |
+| EFS | :white_check_mark: |  :white_check_mark: |
+| DynamoDB streams | :x: |  :white_check_mark: |
+| DynamoDB tables | :x: |  :white_check_mark: |
+| EBS Snapshots | :x: |  :white_check_mark: |
+| EventBridge | :white_check_mark: | :x: |
+| EventBridge Schema | :x: | :x: |
+| Mediastore | :x: | :x: |
+| Glue | :x: | :x: |
+| Kinesis Data Streams | :x: | :x: |
+| Lex v2 | :x: | :x: |
+| Migration Hub Orchestrator | :x: | :x: |
+| OpenSearch | :x: | :x: |
+| AWS PCA | :x: | :x: |
+| Redshift Serverless | :x: | :x: |
+| Serverless Application Repository | :x: | :x: |
+| SES v2 | :x: | :x: |
+| Incident Manager | :x: | :x: |
+| Incident Manager Contacts | :x: | :x: |
+| VPC endpoints | :x: | :x: |
 
 ## How to run
 
@@ -39,14 +83,27 @@ You will get an HTML and JSON report.
 
 See [example report](example/example.html)
 
+You can configure the services you care about by using [your own config](awsxenos/config.yaml).
+
 ### Library
 
 ```python
-from awsxenos.scan import Scan
+from awsxenos.scan import PreScan
 from awsxenos.report import Report
+from awsxenos.s3 import S3
+#from awsxenos.iam import IAM
+
+# To run everything based on your config.
 
-s = Scan()
-r = Report(s.findings, s.known_accounts_data)
+prescan = PreScan()
+results = load_and_run(config_path, prescan.accounts)
+r = Report(results, prescan.known_accounts)
+
+# Per service
+prescan = PreScan()
+aws_service = S3()
+findings = aws_service.fetch(prescan.accounts)
+r = Report(s.findings, s.known_accounts)
 json_summary = r.JSON_report()
 html_summary = r.HTML_report()
 ```
@@ -61,11 +118,24 @@ Permissions required.
   "Statement": [
     {
       "Action": [
-        "iam:ListRoles"
+        "elasticfilesystem:DescribeFileSystemPolicy",
+        "elasticfilesystem:DescribeFileSystems",
+        "events:ListEventBuses",
+        "glacier:GetVaultAccessPolicy",
+        "glacier:ListVaults",
+        "iam:ListRoles",
         "organizations:ListAccounts",
-        "s3:ListAllMyBuckets",
+        "kms:GetKeyPolicy",
+        "kms:ListKeys",
+        "lambda:GetPolicy",
+        "lambda:ListFunctions",
+        "s3:GetBucketAcl",
         "s3:GetBucketPolicy",
-        "s3:GetBucketAcl"
+        "s3:ListAllMyBuckets",
+        "secretsmanager:GetResourcePolicy",
+        "secretsmanager:ListSecrets",
+        "sqs:GetQueueAttributes",
+        "sqs:ListQueues"
       ],
       "Effect": "Allow",
       "Resource": "*"
@@ -81,17 +151,35 @@ python3 -m env venv
 source /env/bin/activate
 pip install -r requirements.txt
 ```
+
+1. Create a file with the name of the service.
+2. Create a class with the name of the resource that you want from that service
+3. Your class must inherit from `Service` and return `Findings`
+
+Example: 
+
+```python
+class S3(Service):
+
+    def fetch(self, accounts: DefaultDict[str, Set] ) -> Findings:
+        self._buckets = self.list_account_buckets()
+        self.policies = self.get_bucket_policies()
+        return super().collate(accounts, self.policies)
+```
+4. Add your filename and class to the config
+
+---
+
+:warning: AWSXenos currently assesses access based on [https://github.com/Netflix-Skunkworks/policyuniverse](https://github.com/Netflix-Skunkworks/policyuniverse).
+There are cases where IAM `conditions`, will _not_ be taken into account, therefore resulting in false positives. 
+This could be fairly common in KMS Customer Managed Keys created by AWS Services.
+AWSXenos findings are per IAM statement on an IAM policy.
+
 ## I want to add more known accounts
 Create a PR or raise an issue. Contributions are welcome.
 
+
+
 ## Features
-- [x] IAM Roles
-- [x] S3 Bucket Policies and ACLs
 - [x] Use as library
-- [x] HTML and JSON output 
-- [x] Supports AWS Services
-
-## TODO
-- [ ] Add support for more resource policies services, e.g. SecretsManager, KSM, SNS, SQS, Lambda
-- [ ] Add support for Cognito, RAM
-- [ ] Add support for VPCE
+- [x] HTML and JSON output 

acknowledgements.md

@@ -3,7 +3,7 @@
 The MIT License (MIT)
 
 Copyright (c) 2019 Rupert Bryant-Greene
-Copyright (c) 2021 AirWalk Consulting Limited
+Copyright (c) 2021 Airwalk Reply Consulting Limited
 
 Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
 

awsxenos/__init__.py

@@ -1,5 +1,7 @@
 """Find the Xenoi (external account trusts) in your AWS Account."""
+
 from pathlib import Path
-from awsxenos.finding import AccountType, Finding
+
+from awsxenos.finding import Accounts, Finding, Findings, Resources, Service
 
 package_path = Path(__file__)