Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test #3

Draft
wants to merge 91 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
91 commits
Select commit Hold shift + click to select a range
ffd07ac
added docker compose
Mar 27, 2023
941496d
changed gitignore
Mar 27, 2023
05af1cb
added k8s file
Mar 27, 2023
bede7b8
added terraform files
Mar 27, 2023
c1bf8d7
changed compose for current version
Mar 27, 2023
6ed2f4a
added variables to tf
Mar 27, 2023
0493769
Create push-and-update-gke.yml
asensionacher Mar 27, 2023
e7566d2
changed variable format
Mar 27, 2023
e805eb8
comment deployment from terraform on gke kubectl
Mar 27, 2023
b9b9979
Merge branch 'main' of https://github.com/asensionacher/mms-cloud-ske…
Mar 27, 2023
1fe6321
changed variables
Mar 27, 2023
a4b1fca
added repository creation
Mar 27, 2023
f4105de
added continue on error if create artifact fails when exists
Mar 27, 2023
3f7a1b8
error on push to gke
Mar 27, 2023
986ce0e
added terraform action
Mar 27, 2023
78a54b6
integrate actions in one run
Mar 27, 2023
9e3dd0e
removed unused files
Mar 27, 2023
c80bf5b
added services enable
Mar 27, 2023
e6bde5e
added serviceusage enable
Mar 27, 2023
47cb909
removed secrets from tfvars
Mar 27, 2023
446be21
added more vars
Mar 27, 2023
8649f61
added creation tfstates bucket
Mar 27, 2023
e9c4038
error on env
Mar 27, 2023
7241574
error on usage of envs
Mar 27, 2023
f1761cf
replace bucket name for being dynamic
Mar 27, 2023
8e3acce
added backend config in init
Mar 27, 2023
c62de47
removed bucket and unused provider
Mar 27, 2023
e9cbc89
removed unused provider
Mar 27, 2023
aef21f2
continue if bucket creation fails
Mar 27, 2023
22193db
remove unused provider
Mar 27, 2023
76d0438
terraform fmt
Mar 27, 2023
a2de038
added vars to apply
Mar 27, 2023
97eb8f7
added cloudresourcemanager
Mar 27, 2023
f059018
removed services
Mar 27, 2023
51c13d3
test new creds
Mar 27, 2023
eb21572
error on @v1
Mar 27, 2023
1b50f56
added kubectl tests
Mar 27, 2023
c4ad957
new auth method
Mar 27, 2023
2f32a1e
error on param auth
Mar 27, 2023
6fa608b
added services
Mar 27, 2023
e34137f
test var quotes
Mar 27, 2023
421e80f
test
Mar 28, 2023
cf4ad52
rollback on error
Mar 28, 2023
21a3e8f
error on var value
Mar 28, 2023
4660097
added cloudbuild service
Mar 28, 2023
d8d7eac
changed verion of auth
Mar 28, 2023
201bcf6
added artifactregistry service
Mar 28, 2023
7a154a8
error on deployment to gke
Mar 28, 2023
7f02d3c
added cloudbuild
Mar 28, 2023
daa61fa
changed for using cloudbuild
Mar 28, 2023
ea4c421
added sustitutions
Mar 28, 2023
32ec24e
removed github sha substitution
Mar 28, 2023
f2ee28c
error on substitutions
Mar 28, 2023
f6d9a96
testing
Mar 28, 2023
acf6fb0
removed unused var _IMAGE
Mar 28, 2023
df545ae
added github sha
Mar 28, 2023
9e3eb4c
changed value
Mar 28, 2023
aa6aa32
added network to cloudbuild
Mar 28, 2023
2bfdd52
added quiet
Mar 28, 2023
ec529ec
error on var
Mar 28, 2023
6b2177d
testing var names
Mar 28, 2023
93b053c
changed var
Mar 28, 2023
5d0d4d1
changedvar
Mar 28, 2023
9e63b1f
changed var
Mar 28, 2023
6c056da
chagned var
Mar 28, 2023
77bd437
added cloudbuild
Mar 28, 2023
c0376a2
added cloud services
Mar 28, 2023
a9a2df6
removed action
Mar 28, 2023
e56d973
error on first bash
Mar 28, 2023
1653af9
one line bash
Mar 28, 2023
464ead9
changed bash
Mar 28, 2023
777624c
added gke
Mar 28, 2023
3c81f23
cloudbuild and changes
Mar 28, 2023
335418e
errors solved
Mar 28, 2023
1b0912e
terraform to modules
Mar 28, 2023
9dd1a6a
service_account_email removed
Mar 28, 2023
859ea57
project number var
Mar 28, 2023
0e60512
var error
Mar 28, 2023
9c586e9
removed errors
Mar 28, 2023
34deca9
good service_account
Mar 28, 2023
ec5658a
new prefix
Mar 28, 2023
62f9b38
changed google_compute_router_nat name
Mar 28, 2023
5eeb57b
addad ips to nat
Mar 28, 2023
3b227eb
CHANGED CB
Mar 28, 2023
7319e69
error name
Mar 28, 2023
7d50c15
tesr
Mar 28, 2023
bef937e
cc
Mar 28, 2023
56a1d1b
changed location
Mar 28, 2023
f69ff91
error on value
Mar 28, 2023
75ef4e0
added change of file
Mar 28, 2023
058f11c
last push
Mar 28, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -11,4 +11,7 @@ node_modules/
yarn.lock

# NPM
package-lock.json
package-lock.json

# Terraform
/terraform
6 changes: 6 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -26,3 +26,9 @@ coverage
*.njsproj
*.sln
*.sw?

# Terraform
terraform/.terraform
*.tfstate
*.tfstate.backup
terraform-back
74 changes: 74 additions & 0 deletions cloudbuild.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
steps:
- id: 'enable services'
name: 'gcr.io/cloud-builders/gcloud'
entrypoint: 'bash'
args:
- '-c'
- |-
gcloud services enable serviceusage.googleapis.com
gcloud services enable compute.googleapis.com
gcloud services enable container.googleapis.com
gcloud services enable cloudresourcemanager.googleapis.com
gcloud services enable cloudbuild.googleapis.com
gcloud services enable artifactregistry.googleapis.com

- id: 'tf init'
name: 'hashicorp/terraform:latest'
entrypoint: 'sh'
args:
- '-c'
- |-
cd terraform
terraform init --backend-config="bucket=$REPO_NAME-$PROJECT_ID-tfstate"

# [START tf-plan]
- id: 'tf plan'
name: 'hashicorp/terraform:latest'
entrypoint: 'sh'
args:
- '-c'
- |-
cd terraform
terraform plan -var="project_id=$PROJECT_ID"
# [END tf-plan]

# [START tf-apply]
- id: 'tf apply'
name: 'hashicorp/terraform:latest'
entrypoint: 'sh'
args:
- '-c'
- |-
cd terraform
terraform apply -auto-approve -var="project_id=$PROJECT_ID"
# [END tf-apply]
#Build
- name: 'gcr.io/cloud-builders/docker'
args:
- 'build'
- '-t'
- 'europe-west1-docker.pkg.dev/$PROJECT_ID/$REPO_NAME/$REPO_NAME-image:${BUILD_ID}'
- '-t'
- 'europe-west1-docker.pkg.dev/$PROJECT_ID/$REPO_NAME/$REPO_NAME-image:latest'
- '.'
#Push image
- name: 'gcr.io/cloud-builders/docker'
args:
- 'push'
- 'europe-west1-docker.pkg.dev/$PROJECT_ID/$REPO_NAME/$REPO_NAME-image'
- '--all-tags'
- id: 'new file'
name: 'ubuntu:latest'
entrypoint: 'bash'
args:
- '-c'
- |-
cd kubernetes
cat deployment.yaml | sed "s|{{DOCKER_IMAGE}}|europe-west1-docker.pkg.dev/$PROJECT_ID/$REPO_NAME/$REPO_NAME-image:latest|g" > deployment-changed.yaml
#Deploy to GKE
- name: 'gcr.io/cloud-builders/gke-deploy'
args:
- run
- --filename=kubernetes/deployment-changed.yaml
- --location=europe-west1
- --cluster=mms-cloud-skeleton-gke
5 changes: 5 additions & 0 deletions compose.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
services:
mms-cloud-skeleton:
build: .
ports:
- "3000:3000"
Binary file added deliverable.pdf
Binary file not shown.
Binary file added deliverable/.attachments/artifact_registry.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added deliverable/.attachments/devops_role1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added deliverable/.attachments/devops_role2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added deliverable/.attachments/finance_role1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
169 changes: 169 additions & 0 deletions deliverable/deliverable.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,169 @@
# MediaMarkt Cloud Engineering challenge

## Summary

This document resolves the _TASKS-OBJECTIVES_ and explains how to deploy the _mms-cloud-skeleton_ app in a _Google Cloud Platform_ project in a _Google Kubernetes Engine_.

## Pre-requisites

For deploying all, you need the following:

- [Install the gcloud CLI](https://cloud.google.com/sdk/docs/install)
- [Install terraform](https://developer.hashicorp.com/terraform/tutorials/gcp-get-started/install-cli)

## Tasks-objectives

### Cloud Build/Artifact to generate the container

For pushing the Cloud Build/Artifact to generate the container, please execute the following lines (tested on Ubuntu 20.04) on the root directory. Please change the value of _project\_id_ and _location_ with yours

``` bash
location=europe-west1
project_id=aovofaxlg4dh8oi8np5wpwkmted1bl

gcloud artifacts repositories create mms-cloud-skeleton--repository-format=docker --location=$location --description="mms-cloud-skeleton"

gcloud builds submit --region=$location --tag $location-docker.pkg.dev/$project_id/mms-cloud-skeleton/mms-cloud-skeleton-image:latest
```

Once the _gcloud builds submit_ is finished we can check it with the following command:

``` bash
gcloud artifacts repositories list

ARTIFACT_REGISTRY
REPOSITORY FORMAT MODE DESCRIPTION LOCATION LABELS ENCRYPTION CREATE_TIME UPDATE_TIME SIZE (MB)
mms-cloud-skeleton DOCKER STANDARD_REPOSITORY mms-cloud-skeleton europe-west1 Google-managed key 2023-03-27T18:37:29 2023-03-27T18:42:59 444.649
```

Or in the _GCP_ console in _Artifact Registry_ menu:

![artifact_registry](.attachments/artifact_registry.png)

### Generation of the Docker Composer YAML

The Docker Compose YAML is stored in the root of the repository with the name [compose.yaml](..\compose.yaml) and is the following:

``` yaml
services:
mms-cloud-skeleton:
image: "europe-west1-docker.pkg.dev/aovofaxlg4dh8oi8np5wpwkmted1bl/mms-cloud-skeleton/mms-cloud-skeleton-image"
ports:
- "3000:3000"
```

This file is only for local deployment because _GKE_ can't use _Docker-compose_ files.

### Creation of the Terraform Files

In the folder [terraform](https://github.com/asensionacher/mms-cloud-skeleton/blob/main/terraform) are stored all the files for deploying the resources and the kubectl deployment to _GCP_. Please update the [terraform.tfvars](https://github.com/asensionacher/mms-cloud-skeleton/blob/main/terraform/terraform.tfvars) file with your values. The provided file is working fine for my environment.

This terraform deploys a private _GKE_ cluster with the resources needed for having connection to internet and also makes a _kubectl apply_ from a terraform module.

### Commands for the Deployment through TF files (kubectl)

#### First steps

First you have to prepare your environment and create the bucket for tfstates and the artifact registry. This step must be executed for manual and CI deployment.

``` bash
# Environment variable for bucket name
project_id=aovofaxlg4dh8oi8np5wpwkmted1bl

# Log in GCP
gcloud auth login

# Create environment variable needed for executing from Terraform
export GOOGLE_OAUTH_ACCESS_TOKEN=$(gcloud auth print-access-token)

# Enable required services on GCP
gcloud services enable serviceusage.googleapis.com
gcloud services enable compute.googleapis.com
gcloud services enable container.googleapis.com
gcloud services enable cloudresourcemanager.googleapis.com
gcloud services enable cloudbuild.googleapis.com
gcloud services enable artifactregistry.googleapis.com

# Create storage bucket on GCP for storing TFStates
bucket_name="mms-cloud-skeleton-$project_id-tfstate"
gcloud storage buckets create gs://$bucket_name

```

##### Manual deployment

For manual deployment, the Terraform must be executed for the first time. This would create all the resources, including the _Cloud build_ and its trigger for be prepared for CI.

``` bash
bucket_name="mms-cloud-skeleton-$project_id-tfstate"
project_id=aovofaxlg4dh8oi8np5wpwkmted1bl
CLOUDBUILD_SA="$(gcloud projects describe $project_id \
--format 'value(projectNumber)')@cloudbuild.gserviceaccount.com"
terraform init -backend-config="bucket=$bucket_name"
terraform plan -var="project_id=$project_id" -var="service_account_email=$CLOUDBUILD_SA"
terraform apply -var="project_id=$project_id" -var="service_account_email=$CLOUDBUILD_SA"
```

This may take up to 20 minutes. Now, you have to execute the _kubectl_ command for deploying the image and the service to _GKE_.

``` bash
# Environment variables
location=europe-west1
project_id=aovofaxlg4dh8oi8np5wpwkmted1bl
cluster_name=mms-cluster-gke
docker_image=$location-docker.pkg.dev/$project_id/mms-cloud-skeleton/mms-cloud-skeleton-image

# Get credentials from the GKE for executing kubectl
gcloud container clusters get-credentials $cluster_name --region $location --project $project_id

# Deploy the yaml to the GKE
cat ../kubernetes/deployment.yaml | sed "s|{{DOCKER_IMAGE}}|$docker_image|g" | kubectl apply -f

# Get the public IP of the service deployed. It can take some time until EXTERNAL-IP appears
kubectl get service mms-cloud-skeleton-srv
```

Now, if you open a browser to the IP that appears on _EXTERNAL-IP_ the webpage of _mms-cloud-skeleton_ should appear and you are ready for CI.

#### Cloud Build for CI/CD

The CI integration is going to be done with [GCP Cloud Build](https://cloud.google.com/build?hl=es). In order to have everything up for the deployment, we have to give editor role to the Cloud Build service account:

``` bash
CLOUDBUILD_SA="$(gcloud projects describe $PROJECT_ID \
--format 'value(projectNumber)')@cloudbuild.gserviceaccount.com"
gcloud projects add-iam-policy-binding $PROJECT_ID \
--member serviceAccount:$CLOUDBUILD_SA --role roles/editor
```

Then, go to the GCP console and create a _Cloud Build Project_ and a trigger from your repository. This step is not done by Terraform because we are also deploying the resources from there with Terraform. Once the _Cloud Build_ is created, make a push to your repository and all the environment is going to be created and also the _Kubernetes_ yaml will be pushed to _GKE_.

#### Remove all resources

You can remove all the resources executing the following command:

``` bash
terraform destroy
```

### Solution of the IAM Role assignation

#### Devops Team Group

The Devops Team Group must have the possibility to deploy clusters in Kubernetes. As per our example it is done by _Cloud Build_, they only have to have the permissions for manage _Cloud Build_, so their roles would be:

1. Cloud Build Editor - For being able to manage the Cloud Builds

![devops_cloud_build_editor](.attachments/devops_role1.png)

2. Browser - For navigate between the resources without editing them.

![devops_browser](.attachments/devops_role2.png)

#### Finance team

The Finance Team have to be able to manage all related to billing accounts, so their roles would be:

1. Billing Account Administrator - For being authorized to see and manage all aspects of billing account

![finance_billing](.attachments/finance_role1.png)
33 changes: 33 additions & 0 deletions kubernetes/deployment.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: mms-cloud-skeleton
spec:
replicas: 3
selector:
matchLabels:
app: mms-cloud-skeleton
template:
metadata:
labels:
app: mms-cloud-skeleton
spec:
containers:
- image: "{{DOCKER_IMAGE}}"
imagePullPolicy: Always
name: mms-cloud-skeleton
ports:
- containerPort: 3000
---
apiVersion: v1
kind: Service
metadata:
name: mms-cloud-skeleton-srv
spec:
type: LoadBalancer
selector:
app: mms-cloud-skeleton
ports:
- port: 80
targetPort: 3000
---
22 changes: 22 additions & 0 deletions terraform/.terraform.lock.hcl

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 6 additions & 0 deletions terraform/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# mms-cloud-skeleton Terraform files

## Description

This files are used for deploying a GKE with all its resources for mms-cloud-skeleton project. Please see the deliverable for using it.

20 changes: 20 additions & 0 deletions terraform/main.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
data "google_project" "project" {
}

module "networking" {
source = "./modules/networking/"

name = var.project_name
region = var.region
}

module "kubernetes" {
source = "./modules/kubernetes/"

name = var.project_name
region = var.region
subnet_self_link = module.networking.subnet_self_link
vpc_self_link = module.networking.vpc_self_link
project_number = data.google_project.project.number
project_id = var.project_id
}
Loading