diff --git a/docs/how-it-works/identities/identities.md b/docs/how-it-works/identities/identities.md index 6e8d5b353..4582e3c41 100644 --- a/docs/how-it-works/identities/identities.md +++ b/docs/how-it-works/identities/identities.md @@ -21,7 +21,14 @@ as reference we've define a security account structure for managing multiple ac audit and compliance monitoring services ![leverage-aws-iam](../../assets/images/diagrams/aws-iam.png "Leverage"){: style="width:600px"} -
**Figure:** AWS Organization Security account structure for managing multiple accounts (just as reference).
+ +
+Figure: AWS Organization Security account structure for managing multiple accounts (just as reference). +(Source: Yoriyasu Yano, + +"How to Build an End to End Production-Grade Architecture on AWS Part 2", +Gruntwork.io Blog, accessed November 18th 2020). +
## IAM Groups & Roles definition diff --git a/docs/how-it-works/identities/roles.md b/docs/how-it-works/identities/roles.md new file mode 100644 index 000000000..500976e62 --- /dev/null +++ b/docs/how-it-works/identities/roles.md @@ -0,0 +1,137 @@ +# IAM roles + +!!! info "What are AWS IAM Roles? ![aws-service](../../assets/images/icons/aws-emojipack/General_AWScloud.png){: style="width:30px"} ![aws-service](../../assets/images/icons/aws-emojipack/SecurityIdentityCompliance_IAM.png){: style="width:15px"}" + For the Leverage AWS Reference Architecture we heavily depend on **AWS IAM roles**, which is a standalone IAM entity + that: + + * Allows you to attach **IAM policies** to it, + * Specify which other **IAM entities** to trust, and then + * Those other IAM entities can assume the IAM role to be temporarily get access to the permissions in those IAM + policies. + + +!!! tip "The two most common use cases for IAM roles are" + * [x] **Service roles:** + Whereas an IAM user allows a human being to access AWS resources, one of the most common use cases for an IAM + role is to allow a service—e.g., one of your applications, a CI server, or an AWS service—to access specific + resources in your AWS account. For example, you could create an IAM role that gives access to a specific S3 bucket + and allow that role to be assumed by one of your EC2 instances or Lambda functions. The code running on that AWS compute + service will then be able to access that S3 bucket (or any other service you granted through this IAM roles) without you + having to manually copy AWS credentials (i.e., access keys) onto that instance. + * [x] **Cross account access:** + Allow to grant an IAM entity in one AWS account access to specific resources in another AWS account. For example, if you + have an IAM user in AWS account A, then by default, that IAM user cannot access anything in AWS account B. However, you + could create an IAM role in account B that gives access to a specific S3 bucket (or any necessary AWS services) in + AWS account B and allow that role to be assumed by an IAM user in account A. That IAM user will then be able to access + the contents of the S3 bucket by assuming the IAM role in account B. This ability to assume IAM roles across different + AWS accounts is the critical glue that truly makes a multi AWS account structure possible. + +## How IAM roles work? + +![leverage-aws-iam-roles](../../assets/images/diagrams/aws-iam-role-cross-account.png "Leverage"){: style="width:600px"} + +
+Figure: Example of AWS cross-account AWS access. +(Source: Kai Zhao, + +"AWS CloudTrail Now Tracks Cross-Account Activity to Its Origin", +AWS Security Blog, accessed November 17th 2020). +
+ +--- + +!!! info "Main IAM Roles related entities" + ### IAM policies + Just as you can attach IAM policies to an IAM user and IAM group, you can attach IAM policies to an IAM role. + ### Trust policy + You must define a trust policy for each IAM role, which is a JSON document (very similar to an IAM policy) that + specifies who can assume this IAM role. For example, we present below a trust policy that allows this IAM role to be + assumed by an IAM user named John in AWS account 111111111111: + ```json + { + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Action": "sts:AssumeRole", + "Principal": {"AWS": "arn:aws:iam::111111111111:user/John"} + } + ] + } + ``` + Note that a trust policy alone does NOT automatically give John permissions to assume this IAM role. + Cross-account access always requires permissions in both accounts (2 way authorization). So, if John is in AWS account + 111111111111 and you want him to have access to an IAM role called `DevOps` in account B ID 222222222222, then you need + to configure permissions in both accounts: + 1. In account 222222222222, the `DevOps` IAM role must have a trust policy that gives `sts:AssumeRole` permissions to + AWS account A ID 111111111111 (as shown above). + 2. 2nd, in account A 111111111111, you also need to attach an IAM policy to John’s IAM user that allows him to assume + the `DevOps` IAM role, which might look like this: + + ```json + { + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Action": "sts:AssumeRole", + "Resource": "arn:aws:iam::222222222222:role/DevOps" + } + ] + } + ``` + +## Assuming an AWS IAM role + +!!! summary "How does it work?" + IAM roles do not have a user name, password, or permanent access keys. To use an IAM role, you must assume it by + making an `AssumeRole` API call (vía [SDKs API](https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRole.html), + [CLI](https://docs.aws.amazon.com/cli/latest/reference/sts/assume-role.html) or + [Web Console](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-console.html), which will + return temporary access keys you can use in follow-up API calls to authenticate as the IAM role. The temporary + access keys will be valid for 1-12 hours (depending on your current validity expiration config), after which you + must call `AssumeRole` again to fetch new temporary keys. Note that to make the `AssumeRole` API call, you must + first authenticate to AWS using some other mechanism. + +For example, for an IAM user to assume an IAM role, the workflow looks like this: +![leverage-aws-iam-roles](../../assets/images/diagrams/aws-iam-role-assume.png "Leverage"){: style="width:900px"} + +
+Figure: Assuming an AWS IAM role. +(Source: Gruntwork.io, + +"How to configure a production-grade AWS account structure using Gruntwork AWS Landing Zone", +Gruntwork.io Production deployment guides, accessed November 17th 2020). +
+ +!!! example "Basic AssumoRole workflow" + 1. Authenticate using the IAM user’s permanent AWS access keys + 2. Make the AssumeRole API call + 3. AWS sends back temporary access keys + 4. You authenticate using those temporary access keys + 5. Now all of your subsequent API calls will be on behalf of the assumed IAM role, with access to whatever + permissions are attached to that role + +!!! info "IAM roles and AWS services" + Most AWS services have native support built-in for assuming IAM roles. + + For example: + + * You can associate an IAM role directly with an EC2 instance (instance profile), and that instance will + automatically assume the IAM role every few hours, making the temporary credentials available in EC2 instance metadata. + * Just about every AWS CLI and SDK tool knows how to read and periodically update temporary credentials from EC2 + instance metadata, so in practice, as soon as you attach an IAM role to an EC2 instance, any code running on that + EC2 instance can automatically make API calls on behalf of that IAM role, with whatever permissions are attached to + that role. This allows you to give code on your EC2 instances IAM permissions without having to manually figure out + how to copy credentials (access keys) onto that instance. + * The same strategy works with many other AWS services: e.g., you use IAM roles as a secure way to give your Lambda + functions, ECS services, Step Functions, and many other AWS services permissions to access specific resources in + your AWS account. + +## Read more + +!!! info "AWS reference links" + Consider the following AWS official links as reference: + + - :orange_book: [**AWS Identities | Roles terms and concepts**](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_terms-and-concepts.html) + - :orange_book: [**AWS Identities | Common scenarios**](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_common-scenarios.html) \ No newline at end of file diff --git a/docs/how-it-works/monitoring/logs.md b/docs/how-it-works/monitoring/logs.md index 4667bee04..e565ddb11 100644 --- a/docs/how-it-works/monitoring/logs.md +++ b/docs/how-it-works/monitoring/logs.md @@ -11,7 +11,13 @@ purpose, on the security account and given the need these can be streamed to Elasticsearch as well if needed. ![leverage-monitoring](../../assets/images/diagrams/monitoring-metrics-logs.png "Leverage"){: style="width:750px"} -
**Figure:** Monitoring metrics and log architecture diagram (just as reference).
+
+Figure: Monitoring metrics and log architecture diagram (just as reference). +(Source: Binbash Leverage, + +"AWS Well Architected Reliability Report example", +Binbash Leverage Doc, accessed November 18th 2020). +
!!! danger "Alerting based on Logs" Certain features that were only available under licence were recently made available by Elastic, and included in the diff --git a/docs/how-it-works/monitoring/metrics.md b/docs/how-it-works/monitoring/metrics.md index 4b13bc5b3..080ea8d93 100644 --- a/docs/how-it-works/monitoring/metrics.md +++ b/docs/how-it-works/monitoring/metrics.md @@ -17,7 +17,13 @@ Prometheus and [AWS CloudWatch metrics](https://docs.aws.amazon.com/AmazonCloudW certain metrics about your own application, that we can graph or alert based on them. ![leverage-monitoring](../../assets/images/diagrams/monitoring-metrics-logs.png "Leverage"){: style="width:750px"} -
**Figure:** Monitoring metrics and log architecture diagram (just as reference).
+
+Figure: Monitoring metrics and log architecture diagram (just as reference). +(Source: Binbash Leverage, + +"AWS Well Architected Reliability Report example", +Binbash Leverage Doc, accessed November 18th 2020). +
!!! check "Graphing metrics" Grafana is the standard open source visualization tool which can be used on top of a variety of different data @@ -28,11 +34,22 @@ Prometheus and [AWS CloudWatch metrics](https://docs.aws.amazon.com/AmazonCloudW from multiple origins. ![leverage-monitoring](../../assets/images/screenshots/monitoring-metrics-k8s-cluster.png){: style="width:750px"} -
**Figure:** Grafana K8s cluster metrics monitoring dashboard [screenshot](https://grafana.com/grafana/plugins/devopsprodigy-kubegraf-app) (just as reference).
+
+Figure: Grafana K8s cluster metrics monitoring dashboard reference screenshot. +(Source: DevOpsProdigy, + +"Grafana DevOpsProdigy KubeGraf Plugin", +Grafana plugins, accessed November 18th 2020). +
![leverage-monitoring](../../assets/images/screenshots/monitoring-metrics-k8s-nodes.png "Leverage"){: style="width:750px"} -
**Figure:** Grafana K8s node metrics monitoring dashboard [screenshot](https://grafana.com/grafana/plugins/devopsprodigy-kubegraf-app) (just as reference).
- +
+Figure: Grafana K8s cluster metrics monitoring dashboard reference screenshot. +(Source: DevOpsProdigy, + +"Grafana DevOpsProdigy KubeGraf Plugin", +Grafana plugins, accessed November 18th 2020). +
!!! attention "Alerting based on metrics" Although Grafana already has alerting capabilities built in, we rather (most of the times) have Prometheus alerting @@ -40,4 +57,7 @@ Prometheus and [AWS CloudWatch metrics](https://docs.aws.amazon.com/AmazonCloudW extremely readable syntax. Example: ![leverage-monitoring](../../assets/images/screenshots/monitoring-metrics-alerts.png "Leverage"){: style="width:750px"} -
**Figure:** Prometheus Alert Manager `CriticalRamUsage` alert screenshot (just as reference).
+
+Figure: Prometheus Alert Manager `CriticalRamUsage` alert screenshot (just as reference). +(Source: Binbash Leverage). +
diff --git a/docs/how-it-works/monitoring/notification_escalation.md b/docs/how-it-works/monitoring/notification_escalation.md index 6f245efe6..f5dad2114 100644 --- a/docs/how-it-works/monitoring/notification_escalation.md +++ b/docs/how-it-works/monitoring/notification_escalation.md @@ -200,4 +200,4 @@ Notify to Slack => engineering-alerts channel * www.domain_3.com **Note:** a personal account has been set up for this. As a recommendation, an new account should be created using - an email account that belongs to Veev project. + an email account that belongs to your project. diff --git a/docs/how-it-works/monitoring/tracing.md b/docs/how-it-works/monitoring/tracing.md index e50dcd845..a2c7a8573 100644 --- a/docs/how-it-works/monitoring/tracing.md +++ b/docs/how-it-works/monitoring/tracing.md @@ -6,8 +6,14 @@ and what causes poor performance. ![leverage-monitoring](../../assets/images/diagrams/monitoring-tracing.png "Leverage"){: style="width:750px"} -
**Figure:** Distributed tracing architecture diagram (just as reference).
- +
+Figure: Figure: Distributed tracing architecture diagram (just as reference). +(Source: Binbash Leverage, + +"AWS Well Architected Reliability Report example", +Binbash Leverage Doc, accessed November 18th 2020). +
+ ## Read more !!! info "Related sources" diff --git a/docs/how-it-works/network/dns.md b/docs/how-it-works/network/dns.md index 5c944be80..277422560 100644 --- a/docs/how-it-works/network/dns.md +++ b/docs/how-it-works/network/dns.md @@ -8,6 +8,10 @@ (active-active hosted zones) is completely possible and achieviable with Leverage terraform code ![leverage-aws-dns](../../assets/images/diagrams/aws-route53.png "Leverage"){: style="width:800px"} -
**Figure:** AWS multi account Organization -[Route53 Association](https://aws.amazon.com/blogs/security/how-to-centralize-dns-management-in-a-multi-account-environment/) -topology (just as reference).
\ No newline at end of file +
+Figure: AWS Organization shared account Route53 DNS diagram. +(Source: Cristian Southall, + +"Using CloudFormation Custom Resources to Configure Route53 Aliases", +Abstractable.io Blog post, accessed November 18th 2020). +
\ No newline at end of file diff --git a/docs/how-it-works/network/vpc.md b/docs/how-it-works/network/vpc-addressing.md similarity index 100% rename from docs/how-it-works/network/vpc.md rename to docs/how-it-works/network/vpc-addressing.md diff --git a/docs/how-it-works/network/vpc-peering.md b/docs/how-it-works/network/vpc-peering.md index 9d4b4c89b..b6fd1bff3 100644 --- a/docs/how-it-works/network/vpc-peering.md +++ b/docs/how-it-works/network/vpc-peering.md @@ -1,9 +1,19 @@ # Diagram: Network Service (cross-account [VPC peering](https://docs.aws.amazon.com/vpc/latest/peering/what-is-vpc-peering.html)) ![leverage-aws-vpc-peering](../../assets/images/diagrams/aws-vpc-peering-1.png "Leverage"){: style="width:300px"} -
**Figure:** AWS multi account Organization [VPC Peering](https://docs.aws.amazon.com/vpc/latest/peering/vpc-pg.pdf) -topology (just as reference).
+
+Figure: AWS multi account Organization VPC peering diagram. +(Source: AWS, + +"Amazon Virtual Private Cloud VPC Peering", +AWS Documentation Amazon VPC User Guide, accessed November 18th 2020). +
![leverage-aws-vpc-peering](../../assets/images/diagrams/aws-vpc-peering-2.png "Leverage"){: style="width:300px"} -
**Figure:** AWS multi account Organization [VPC Peering](https://docs.aws.amazon.com/vpc/latest/peering/vpc-pg.pdf) -routing (just as reference).
\ No newline at end of file +
+Figure: AWS multi account Organization peering detailed diagram. +(Source: AWS, + +"Amazon Virtual Private Cloud VPC Peering", +AWS Documentation Amazon VPC User Guide, accessed November 18th 2020). +
\ No newline at end of file diff --git a/docs/how-it-works/network/vpc-topology.md b/docs/how-it-works/network/vpc-topology.md new file mode 100644 index 000000000..f992eb455 --- /dev/null +++ b/docs/how-it-works/network/vpc-topology.md @@ -0,0 +1,56 @@ +# Network Layer + +## Network Topology + +!!! summary "VPC with public and private subnets (NAT)" + * [x] The configuration for this scenario includes a virtual private cloud (VPC) with public subnets and a private + subnets (it's number will change depending on our specific needs). We recommend this scenario if you want + to run a public-facing web application, while maintaining back-end servers that aren't publicly accessible. + A common example is a multi-tier website, with a Load Balancer (ALB | NLB) in a public subnet, or other public + facing routing service like AWS CloudFront or Api Gateway, and our web servers (Lambda, EKS, ECS, EC2) and + database (RDS, DynamoDB, etc) servers in private subnets. You can set up security (SGs, ACLs, WAF) and routing + so that the web servers can communicate internally (even between VPC accounts or VPN Endpoints) with all necessary + services and components such as databases, cache, queues, among others. + + * [x] The services running in the public subnet, like an ALB or NLB can send outbound traffic directly to the Internet, + whereas the instances in the private subnet can't. Instead, the instances in the private subnet can access the + Internet by using a network address translation (NAT) gateway that resides in the public subnet. The database + servers can connect to the Internet for software updates using the NAT gateway (if using RDS this is transparently + provided by AWS), but the Internet cannot establish connections to the database servers. + + * [x] So, whenever possible all our AWS resources like EC2, EKS, RDS, Lambda, SQS will be deployed in VPC private + subnets and we'll use a NAT device (Nat Gateway) to enable instances in a private subnet to connect to the + internet (for example, for software updates) or other AWS services, but prevent the internet from initiating + connections with the instances. + + * [x] A NAT device forwards traffic from the instances in the private subnet to the internet (via the VPC Internet + Gateway) or other AWS services, and then sends the response back to the instances. When traffic goes to the + internet, the source IPv4 address is replaced with the NAT device’s address and similarly, when the response + traffic goes to those instances, the NAT device translates the address back to those instances’ private IPv4 + addresses. + +![leverage-aws-vpc-ngw](../../assets/images/diagrams/aws-vpc-nat-gateway.png "Leverage"){: style="width:900px"} +
+Figure: VPC topology diagram. +(Source: AWS, + +"VPC with public and private subnets (NAT)", +AWS Documentation Amazon VPC User Guide, accessed November 18th 2020). +
+ +![leverage-aws-vpc-ngw-ha](../../assets/images/diagrams/aws-vpc-nat-gateway-ha.png "Leverage"){: style="width:900px"} +
+Figure: VPC topology diagram with mutiple Nat Gateways for HA. +(Source: Andreas Wittig, + +"Advanced AWS Networking: Pitfalls That You Should Avoid", +Cloudonaut.io Blog, accessed November 18th 2020). +
+ +## Read more + +!!! info "AWS reference links" + Consider the following AWS official links as reference: + + - :orange_book: [**VPC with public and private subnets (NAT)**](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Scenario2.html) + - :orange_book: [**AWS Elastic Load Balancing**](https://aws.amazon.com/elasticloadbalancing)