Merge pull request #56 from binbashar/feature/BBL-445-iam-sts

Feature/BBL-445 | Identities, Network, Organizations, Workflows and Diagrams

Makefile

@@ -2,7 +2,7 @@
 SHELL         := /bin/bash
 MAKEFILE_PATH := ./Makefile
 MAKEFILES_DIR := ./@bin/makefiles
-MAKEFILES_VER := v0.1.6
+MAKEFILES_VER := v0.1.9
 
 help:
 	@echo 'Available Commands:'

docs/how-it-works/cdn/cdn.md

@@ -14,9 +14,21 @@
 ## Load Balancer (ALB | NLB) & S3 Cloudfront Origins 
 
 ![leverage-aws-cloudfront](../../assets/images/diagrams/aws-cloudfront-acm-elb-s3.png "Leverage"){: style="width:950px"}
-<figcaption>**Figure:** [AWS CloudFront with ELB and S3 as origin diagram](https://aws.amazon.com/blogs/security/how-to-help-achieve-mobile-app-transport-security-compliance-by-using-amazon-cloudfront-and-aws-certificate-manager/) (just as reference).</figcaption>
+<figcaption style="font-size:15px">
+<b>Figure:</b> AWS CloudFront with ELB and S3 as origin diagram.
+(Source: Lee Atkinson, 
+<a href="https://aws.amazon.com/blogs/security/how-to-help-achieve-mobile-app-transport-security-compliance-by-using-amazon-cloudfront-and-aws-certificate-manager/">
+"How to Help Achieve Mobile App Transport Security (ATS) Compliance by Using Amazon CloudFront and AWS Certificate Manager"</a>,
+AWS Security Blog, accessed November 17th 2020).
+</figcaption>
 
 ## API Gateway Cloudfront Origins 
 
 ![leverage-aws-cloudfront](../../assets/images/diagrams/aws-cloudfront-api-gw.png "Leverage"){: style="width:950px"}
-<figcaption>**Figure:** [AWS CloudFront with API Gateway as origin diagram](https://aws.amazon.com/solutions/implementations/serverless-image-handler/) (just as reference).</figcaption>
+<figcaption style="font-size:15px">
+<b>Figure:</b> AWS CloudFront with API Gateway as origin diagram.
+(Source: AWS, 
+<a href="https://aws.amazon.com/solutions/implementations/serverless-image-handler/">
+"AWS Solutions Library  AWS Solutions Implementations Serverless Image Handler"</a>,
+AWS Solutions Library Solutions Implementations, accessed November 17th 2020).
+</figcaption>

docs/how-it-works/ci-cd/ci-cd.md

@@ -2,8 +2,22 @@
 
 ## Opt-1: Jenkins + ArgoCD
 ![leverage-ci-cd-argocd](../../assets/images/diagrams/ci-cd-argocd.png "Leverage"){: style="width:750px"}
-<figcaption>**Figure:** CI/CD with Jenkins + [ArgoCD](https://argoproj.github.io/argo-cd/) diagram (just as reference).</figcaption>
+
+<figcaption style="font-size:15px">
+<b>Figure:</b> ACI/CD with Jenkins + ArgoCD architecture diagram.
+(Source: ArgoCD, 
+<a href="https://argoproj.github.io/argo-cd/">
+"Overview - What Is Argo CD"</a>,
+ArgoCD documentation, accessed November 18th 2020).
+</figcaption>
 
 ## Opt-2: [Jenkins + Spinnaker](https://drive.google.com/file/d/1VtKHzBkw5a3zGKFwgI_2rllL9M7ceuCD/view?usp=sharing)
 ![leverage-ci-cd-spinnaker](../../assets/images/diagrams/ci-cd-spinnaker.png "Leverage"){: style="width:950px"}
-<figcaption>**Figure:** [CI/CD with Jenkins + Spinnaker](https://drive.google.com/file/d/1VtKHzBkw5a3zGKFwgI_2rllL9M7ceuCD/view?usp=sharing) diagram (just as reference).</figcaption>
+
+<figcaption style="font-size:15px">
+<b>Figure:</b> CI/CD with Jenkins + Spinnaker diagram.
+(Source: Irshad Buchh, 
+<a href="https://aws.amazon.com/blogs/opensource/continuous-delivery-spinnaker-amazon-eks/">
+"Continuous Delivery using Spinnaker on Amazon EKS"</a>,
+AWS Open Source Blog, accessed November 18th 2020).
+</figcaption>

docs/how-it-works/code-library/modules-library-per-tech.md

@@ -19,6 +19,5 @@
 | [Dockerfiles](https://github.com/orgs/binbashar/teams/leverage-project-docker-dev/repositories)                        | These are Terraform module we created/imported to build reusable resources / stacks.                                      |
 | [Ansible Playbooks & Roles](https://github.com/orgs/binbashar/teams/leverage-project-ansible-dev/repositories)         | Playbooks we use for provisioning servers such as Jenkins, Spinnaker, Vault, and so on.                                   |
 | [Jenkins Modules](https://github.com/orgs/binbashar/teams/leverage-project-jenkins-dev/repositories)                   | Module we use in our Jenkins pipelines to perform repeated tasks such as posting to Slack, interacting with AWS CLI, etc. |
-| [CloudFormation Modules](https://github.com/orgs/binbashar/teams/leverage-project-aws-cloudformation-dev/repositories) | Local development via Docker Compose.                                                                                     |
 | [Helm Charts](https://github.com/orgs/binbashar/teams/leverage-project-helm-dev/repositories)                          | Complementary Jenkins pipelines to clean docker images, unseal Vault, and more. Also SecOps jobs can be found here.       |
 | [Terraform Modules](https://github.com/orgs/binbashar/teams/leverage-project-terraform-dev/repositories)               | Jenkins pipelines, docker images, and other resources used for load testing.                                              |

docs/how-it-works/compute/k8s-eks.md

@@ -13,4 +13,11 @@ to run **Kubernetes** on AWS without needing to install and operate your own Kub
      Kubernetes code base helping you take advantage of AWS services.
 
 ![leverage-aws-eks](../../assets/images/diagrams/aws-k8s-eks.png "Leverage"){: style="width:950px"}
-<figcaption>**Figure:** AWS K8s EKS architecture diagram (just as reference).</figcaption>
+
+<figcaption style="font-size:15px">
+<b>Figure:</b> AWS K8s EKS architecture diagram (just as reference).
+(Source: Jay McConnell, 
+<a href="https://aws.amazon.com/blogs/infrastructure-and-automation/a-tale-from-the-trenches-the-cloudbees-core-on-aws-quick-start/">
+"A tale from the trenches: The CloudBees Core on AWS Quick Start"</a>,
+AWS Infrastructure & Automation Blog post, accessed November 18th 2020).
+</figcaption>

docs/how-it-works/compute/k8s-kops.md

@@ -16,4 +16,11 @@ The project describes itself as kubectl for clusters.
     - [x] Supports heterogeneous clusters by creating multiple instance groups
 
 ![leverage-aws-k8s-kops](../../assets/images/diagrams/aws-k8s-kops.png "Leverage"){: style="width:950px"}
-<figcaption>**Figure:** AWS K8s Kops architecture diagram (just as reference).</figcaption>
+
+<figcaption style="font-size:15px">
+<b>Figure:</b> AWS K8s Kops architecture diagram (just as reference).
+(Source: Carlos Rodriguez, 
+<a href="https://www.nclouds.com/blog/kubernetes-aws-terraform-kops/">
+"How to deploy a Kubernetes cluster on AWS with Terraform & kops"</a>,
+Nclouds.com Blog post, accessed November 18th 2020).
+</figcaption>

docs/how-it-works/compute/overview.md

@@ -16,9 +16,14 @@ Clusters will be provisioned with [**_Kops_**](https://github.com/kubernetes/kop
  compute engine in AWS. Whenever possible the initial version deployed will be the latest stable release.
 
 ![leverage-k8s-architecture](../../assets/images/diagrams/k8s-architecture.png "Leverage"){: style="width:700"}
-<figcaption>**Figure:** [Kubernetes high level components architecture](https://kubernetes.io/blog/2018/07/18/11-ways-not-to-get-hacked/).
-</figcaption>
 
+<figcaption style="font-size:15px">
+<b>Figure:</b> Kubernetes high level components architecture.
+(Source: Andrew Martin, 
+<a href="https://kubernetes.io/blog/2018/07/18/11-ways-not-to-get-hacked">
+"11 Ways (Not) to Get Hacked"</a>,
+Kubernetes.io Blog post, accessed November 18th 2020).
+</figcaption>
 
 ## Kubernetes addons 
 

docs/how-it-works/compute/serverless.md

@@ -17,9 +17,14 @@ As stated by [AWS Serverless definitions](https://aws.amazon.com/serverless/)
     developers reclaim time and energy that can be spent on developing great products which scale and that are reliable.
 
 ![leverage-aws-serverless](../../assets/images/diagrams/aws-serverless.png "Leverage"){: style="width:950px"}
-<figcaption>**Figure:** AWS 
-[serverless architecture](https://medium.com/containers-on-aws/designing-a-modern-serverless-application-with-aws-lambda-and-aws-fargate-83f4c5fac573)
-(just as reference).</figcaption>
+
+<figcaption style="font-size:15px">
+<b>Figure:</b> AWS serverless architecture diagram (just as reference).
+(Source: Nathan Peck, 
+<a href="https://medium.com/containers-on-aws/designing-a-modern-serverless-application-with-aws-lambda-and-aws-fargate-83f4c5fac573">
+"Designing a modern serverless application with AWS Lambda and AWS Fargate"</a>,
+Containers-on-AWS Medium Blog post, accessed November 18th 2020).
+</figcaption>
 
 !!! info "Serverless Compute ![aws-service](../../assets/images/icons/aws-emojipack/General_AWScloud.png){: style="width:30px"} Services"
     * [x] [AWS Lambda](https://aws.amazon.com/lambda/) lets you run code without provisioning or managing servers. 

docs/how-it-works/costs/costs.md

@@ -43,4 +43,6 @@
 !!! info "Reference links"
     Consider the following extra links as reference:
 
-    - :orange_book: [AWS Ramp-Up Guide: Cost Management](https://d1.awsstatic.com/training-and-certification/ramp-up_guides/Ramp-Up_Guide_Cost_Management.pdf)
+    - :orange_book: [AWS Ramp-Up Guide: Cost Management](https://d1.awsstatic.com/training-and-certification/ramp-up_guides/Ramp-Up_Guide_Cost_Management.pdf)
+    - :books: [A Guide to Cloud Cost Optimization with HashiCorp Terraform](https://www.hashicorp.com/blog/a-guide-to-cloud-cost-optimization-with-hashicorp-terraform)
+    - :books: [FinOps: How Cloud Finance Management Can Save Your Cloud Programme From Extinction](https://www.contino.io/insights/finops-cloud-finance-management)

docs/how-it-works/identities/identities.md

@@ -21,7 +21,14 @@ as reference  we've define a security account structure for managing multiple ac
     audit and compliance monitoring services
 
 ![leverage-aws-iam](../../assets/images/diagrams/aws-iam.png "Leverage"){: style="width:600px"}
-<figcaption>**Figure:** AWS Organization Security account structure for managing multiple accounts (just as reference).</figcaption>
+
+<figcaption style="font-size:15px">
+<b>Figure:</b> AWS Organization Security account structure for managing multiple accounts (just as reference).
+(Source: Yoriyasu Yano, 
+<a href="https://blog.gruntwork.io/how-to-build-an-end-to-end-production-grade-architecture-on-aws-part-2-4f6e5dc30100">
+"How to Build an End to End Production-Grade Architecture on AWS Part 2"</a>,
+Gruntwork.io Blog, accessed November 18th 2020).
+</figcaption>
 
 ## IAM Groups & Roles definition 
 

docs/how-it-works/identities/roles.md

@@ -0,0 +1,137 @@
+# IAM roles
+
+!!! info "What are AWS IAM Roles? ![aws-service](../../assets/images/icons/aws-emojipack/General_AWScloud.png){: style="width:30px"} ![aws-service](../../assets/images/icons/aws-emojipack/SecurityIdentityCompliance_IAM.png){: style="width:15px"}"
+    For the Leverage AWS Reference Architecture we heavily depend on **AWS IAM roles**, which is a standalone IAM entity 
+    that:
+
+     * Allows you to attach **IAM policies** to it, 
+     * Specify which other **IAM entities** to trust, and then 
+     * Those other IAM entities can assume the IAM role to be temporarily get access to the permissions in those IAM 
+       policies. 
+
+
+!!! tip "The two most common use cases for IAM roles are"
+    * [x] **Service roles:**
+    Whereas an IAM user allows a human being to access AWS resources, one of the most common use cases for an IAM 
+    role is to allow a service—e.g., one of your applications, a CI server, or an AWS service—to access specific 
+    resources in your AWS account. For example, you could create an IAM role that gives access to a specific S3 bucket 
+    and allow that role to be assumed by one of your EC2 instances or Lambda functions. The code running on that AWS compute
+    service will then be able to access that S3 bucket (or any other service you granted through this IAM roles) without you
+    having to manually copy AWS credentials (i.e., access keys) onto that instance.
+    * [x] **Cross account access:**
+    Allow to grant an IAM entity in one AWS account access to specific resources in another AWS account. For example, if you
+    have an IAM user in AWS account A, then by default, that IAM user cannot access anything in AWS account B. However, you
+    could create an IAM role in account B that gives access to a specific S3 bucket (or any necessary AWS services) in 
+    AWS account B and allow that role to be assumed by an IAM user in account A. That IAM user will then be able to access
+    the contents of the S3 bucket by assuming the IAM role in account B. This ability to assume IAM roles across different 
+    AWS accounts is the critical glue that truly makes a multi AWS account structure possible.
+
+## How IAM roles work?
+
+![leverage-aws-iam-roles](../../assets/images/diagrams/aws-iam-role-cross-account.png "Leverage"){: style="width:600px"}
+
+<figcaption style="font-size:15px">
+<b>Figure:</b> Example of AWS cross-account AWS access.
+(Source: Kai Zhao, 
+<a href="https://aws.amazon.com/blogs/security/aws-cloudtrail-now-tracks-cross-account-activity-to-its-origin/">
+"AWS CloudTrail Now Tracks Cross-Account Activity to Its Origin"</a>,
+AWS Security Blog, accessed November 17th 2020).
+</figcaption>
+
+---
+
+!!! info "Main IAM Roles related entities"
+    ### IAM policies
+    Just as you can attach IAM policies to an IAM user and IAM group, you can attach IAM policies to an IAM role.
+    ### Trust policy
+    You must define a trust policy for each IAM role, which is a JSON document (very similar to an IAM policy) that 
+    specifies who can assume this IAM role. For example, we present below a trust policy that allows this IAM role to be 
+    assumed by an IAM user named John in AWS account 111111111111:
+    ```json
+    {
+      "Version": "2012-10-17",
+      "Statement": [
+        {
+          "Effect": "Allow",
+          "Action": "sts:AssumeRole",
+          "Principal": {"AWS": "arn:aws:iam::111111111111:user/John"}
+        }
+      ]
+    }
+    ``` 
+    Note that a trust policy alone does NOT automatically give John permissions to assume this IAM role. 
+    Cross-account access always requires permissions in both accounts (2 way authorization). So, if John is in AWS account
+    111111111111 and you want him to have access to an IAM role called `DevOps` in account B ID 222222222222, then you need
+    to configure permissions in both accounts: 
+    1. In account 222222222222, the `DevOps` IAM role must have a trust policy that gives `sts:AssumeRole` permissions to 
+    AWS account A ID 111111111111 (as shown above).
+    2. 2nd, in account A 111111111111, you also need to attach an IAM policy to John’s IAM user that allows him to assume 
+    the `DevOps` IAM role, which might look like this:
+
+    ```json
+    {
+      "Version": "2012-10-17",
+      "Statement": [
+        {
+          "Effect": "Allow",
+          "Action": "sts:AssumeRole",
+          "Resource": "arn:aws:iam::222222222222:role/DevOps"
+        }
+      ]
+    }
+    ```
+
+## Assuming an AWS IAM role
+
+!!! summary "How does it work?" 
+    IAM roles do not have a user name, password, or permanent access keys. To use an IAM role, you must assume it by 
+    making an `AssumeRole` API call (vía [SDKs API](https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRole.html), 
+    [CLI](https://docs.aws.amazon.com/cli/latest/reference/sts/assume-role.html) or 
+    [Web Console](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-console.html), which will
+    return temporary access keys you can use in follow-up API calls to authenticate as the IAM role. The temporary 
+    access keys will be valid for 1-12 hours (depending on your current validity expiration config), after which you
+    must call `AssumeRole` again to fetch new temporary keys. Note that to make the `AssumeRole` API call, you must
+    first authenticate to AWS using some other mechanism. 
+
+For example, for an IAM user to assume an IAM role, the workflow looks like this:
+![leverage-aws-iam-roles](../../assets/images/diagrams/aws-iam-role-assume.png "Leverage"){: style="width:900px"}
+
+<figcaption style="font-size:15px">
+<b>Figure:</b> Assuming an AWS IAM role.
+(Source: Gruntwork.io, 
+<a href="https://gruntwork.io/guides/foundations/how-to-configure-production-grade-aws-account-structure/#iam-roles">
+"How to configure a production-grade AWS account structure using Gruntwork AWS Landing Zone"</a>,
+Gruntwork.io Production deployment guides, accessed November 17th 2020).
+</figcaption>
+
+!!! example "Basic AssumeRole workflow"
+    1. Authenticate using the IAM user’s permanent AWS access keys
+    2. Make the AssumeRole API call
+    3. AWS sends back temporary access keys
+    4. You authenticate using those temporary access keys
+    5. Now all of your subsequent API calls will be on behalf of the assumed IAM role, with access to whatever 
+       permissions are attached to that role
+
+!!! info "IAM roles and AWS services"
+    Most AWS services have native support built-in for assuming IAM roles. 
+
+    For example: 
+
+    * You can associate an IAM role directly with an EC2 instance (instance profile), and that instance will 
+    automatically assume the IAM role every few hours, making the temporary credentials available in EC2 instance metadata. 
+    * Just about every AWS CLI and SDK tool knows how to read and periodically update temporary credentials from EC2 
+    instance metadata, so in practice, as soon as you attach an IAM role to an EC2 instance, any code running on that 
+    EC2 instance can automatically make API calls on behalf of that IAM role, with whatever permissions are attached to
+    that role. This allows you to give code on your EC2 instances IAM permissions without having to manually figure out
+    how to copy credentials (access keys) onto that instance. 
+    * The same strategy works with many other AWS services: e.g., you use IAM roles as a secure way to give your Lambda
+     functions, ECS services, Step Functions, and many other AWS services permissions to access specific resources in 
+     your AWS account.
+
+## Read more
+
+!!! info "AWS reference links"
+    Consider the following AWS official links as reference:
+
+    - :orange_book: [**AWS Identities | Roles terms and concepts**](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_terms-and-concepts.html)
+    - :orange_book: [**AWS Identities | Common scenarios**](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_common-scenarios.html)

docs/how-it-works/monitoring/logs.md

@@ -11,7 +11,13 @@
     purpose, on the security account and given the need these can be streamed to Elasticsearch as well if needed.
 
 ![leverage-monitoring](../../assets/images/diagrams/monitoring-metrics-logs.png "Leverage"){: style="width:750px"}
-<figcaption>**Figure:** Monitoring metrics and log architecture diagram (just as reference).</figcaption>
+<figcaption style="font-size:15px">
+<b>Figure:</b> Monitoring metrics and log architecture diagram (just as reference).
+(Source: Binbash Leverage, 
+<a href="https://drive.google.com/file/d/1KYZC-wTXn2PSVIEtikx9PFOwK2SoCxD8/view?usp=sharing">
+"AWS Well Architected Reliability Report example"</a>,
+Binbash Leverage Doc, accessed November 18th 2020).
+</figcaption>
 
 !!! danger "Alerting based on Logs" 
     Certain features that were only available under licence were recently made available by Elastic, and included in the