Simple Scaleway Terraform aims to abstract terraform in order to simplify the handling.
I'm not sure that everything is good to take, I learn terraform myself while I do this rest. But I think it might help other people understand terraform, and the Scaleway platform.
Here is the nomenclature of the project this one is only an example not structuring for the moment.
You will need to modify the following files: main.tf and terraform.tfvars to suit your needs
├── README.md
├── main.tf
├── modules
│ └── replicator
│ ├── main.tf
│ ├── securityGroups.tf
│ └── variables.tf
├── terraform.tfvars
└── variables.tf
I wanted to start with terraform, I realized that the sources of information about Scaleway are not yet sufficiently accessible. I also think that the subject could motivate some of you to improve this model, or even to invalidate it quickly.
It's a simple module for improve your experience with terraform on scaleway, I just start terraform, maybe later I realise all this project it's a mistake, it's not exclude.
The goal is start quickly your infrastructure and abstract networks rules with a simple configuration.
Generated Terraform Graph:
- Install terraform
- Scaleway Account:
- Setup access key / secret key
- Export access key / secret key
Export SCW terraform configuration
export TF_VAR_scw_api_key="<YOUR_SCW_API_KEY>"
export TF_VAR_scw_secret_key="<YOUR_SCW_SECRET_KEY>"
export TF_VAR_scw_org_id="<YOUR_SCW_API_KEY>" # OptionalUpdate the image id in terraform.tfvars
terraform init
terraform plan # no cost, just show the deploying result
terraform apply # this action deploy your infrastructure on Scaleway, see the scaleway pricing before runWith a backend.tf file you can then store your terraform states remotely, as in Scaleway s3. (free offer for the first 75 GB, storage and bandwidth).
terraform {
backend "s3" {
bucket = "your-bucket-name"
key = "your-infrastructure-name.tfstate"
region = "fr-par" # ["fr-par" or "nl-ams", warsovie in next time]
endpoint = "https://s3.fr-par.scw.cloud" # warning : region is redonded on the endpoint value
access_key = "your_access_key" # generate token on https://console.scaleway.com
secret_key = "your_secret_key"
skip_credentials_validation = true
skip_region_validation = true
}
}You will also be able to integrate terraform directly into your pipeline. Here is an example of an internship for gitlab_ci.yml. These lines allow to run the tests, only if the terraform file changes and if it is submitted on master.
These lines will have to be adapted to your use.
image: golang:latest
variables:
REPO_NAME: github.com/reversTeam/simple-scaleway-terraform
ARTIFACTS_DIR: artifacts
BINARY_FILE: ffs
TERRAFORM_VERSION: "0.12.17"
before_script:
- apt-get update
- apt-get install -y unzip libprotobuf-dev protobuf-compiler
## Install go environment
- go get -u github.com/golang/dep/cmd/dep
- go get -u github.com/golang/protobuf/{proto,protoc-gen-go}
## Install Terraform
- cd /tmp
- wget https://releases.hashicorp.com/terraform/${TERRAFORM_VERSION}/terraform_${TERRAFORM_VERSION}_linux_amd64.zip
- unzip terraform_${TERRAFORM_VERSION}_linux_amd64.zip
- mv terraform /usr/local/bin/
- cd $CI_PROJECT_DIR/terraform
- terraform init
stages:
# [...]
- validate
- plan
- apply
# [...]
validate:
stage: validate
script:
- cd $CI_PROJECT_DIR/terraform
- terraform validate
only:
refs:
- master
changes:
- terraform/**/*
plan:
stage: plan
script:
- cd $CI_PROJECT_DIR/terraform
- terraform plan -out "planfile"
artifacts:
paths:
- $CI_PROJECT_DIR/terraform/planfile
dependencies:
- validate
only:
refs:
- master
changes:
- terraform/**/*
apply:
stage: apply
script:
- cd $CI_PROJECT_DIR/terraform
- terraform apply -input=false "planfile"
dependencies:
- plan
when: manual
only:
refs:
- master
changes:
- terraform/**/*We want to create a scalable wordpress, with secure network database, loadbalancer and scalable wordpress worker.
For that look the main.tf content file, for loading modules.
In this file we see, the provider configuration (other cloud provider, is not implemented yet)
provider "scaleway" {
access_key = var.scw_api_key
secret_key = var.scw_secret_key
organization_id = var.scw_org_id
zone = var.scw_zone
region = var.scw_region
version = "~> 1.11"
}Note: the scaleway token is trasmited by environment variable
module "database" { // Name of pool type
source = "./modules/replicator" // The base module for all pool type
module_name = "database" // Name of module again, if someone knows how to use a module name var
project = var.project // Your project, or client name
cluster = var.cluster // The cluster, for manage more in futur
region = var.scw_region // The region where you want deploy your infrastructure
infra = var.infra // The global configuration for each pool type (cf. terraform.tfvars)
services = var.services // The global configuration for services on the instance, use by infra (cf. terraform.tfvars)
networks = var.networks // The global configuration for each network, use by services (cf. terraform.tfvars)
pools = {
web = module.web.nodes // Transmitted the instance setup by the web module
proxy = module.database.nodes // Transmitted the instance setup by the proxy module
}
}
# [...]Content file terraform.tfvars
# scw region
scw_region = "fr-par"
# scw zone
scw_zone = "fr-par-1"
# project name
project = "default"
# cluster id
cluster = 1
# ssh key path without passphrase
keypath = "~/.ssh/deployment"
# hosted by each node of infra definition
common_networks = ["ssh_private", "dns", "apt"]
# [...]
# infra definition require main.tf declaration
infra = {
proxy = {
scale = 1
public_ip = true
image = "1ec3f179-5b05-408f-a8b3-344e4d8d22d9"
type = "DEV1-S"
services = [ "nginx" ]
},
web = {
scale = 1
public_ip = false
image = "1ec3f179-5b05-408f-a8b3-344e4d8d22d9"
type = "DEV1-S"
services = [ "wordpress" ]
},
database = {
scale = 1
public_ip = true
image = "1ec3f179-5b05-408f-a8b3-344e4d8d22d9"
type = "DEV1-S"
services = [ "mysql" ]
},
}
# services
service = {
"${service_name}" = { # quoted for exposed vars, need remove this "${}"
networks = {
hosted = ["x", "y"]
linked = ["z"]
}
install = [ # apt-get update is not required
"apt-get install -y what-you-want"
]
run = [
"service what-you-want start"
]
}
# [...]
}
# networks
networks = {
x = { # like used network in "${service_name}"
all = [
{
action = "accept|drop"
port = 80
protocol = "UDP"
interface = "address|public|address|or real ip 123.45.67.89/32"
},
# [...]
]
in = []
out = []
},
}Normaly you have your a cluster on Scaleway.
main.tf
provider "scaleway" { # declare your provider (all variables is defined in terraform.tfvars)
access_key = var.scw_api_key
secret_key = var.scw_secret_key
organization_id = var.scw_org_id
zone = var.scw_zone
region = var.scw_region
version = "~> 1.11"
}
module "proxy" { # declare proxy type pool
source = "./modules/replicator" # path to generic module
module_name = "proxy"
project = var.project
cluster = var.cluster
region = var.scw_region
infra = var.infra # need to know infra global definition (cf. terraform.tfvars)
services = var.services # need to know service global definition (cf. terraform.tfvars)
networks = var.networks # need to know networks global definition (cf. terraform.tfvars)
keypath = var.keypath # need to know ssh key path (cf. terraform.tfvars)
common_networks = var.common_networks
pools = {
web = module.web.nodes # require for linked network
database = module.database.nodes # require for linked network
}
}
# [ ... ] other module like web, database
The infrastructure is required for discribe your node type (beacause it's scalable), and what service are installed on them.
infra = {
proxy = { # infra pool type name, it's a segmentation for scale
scale = 1 # Number of node for this type
public_ip = true # Your instance required an external IP?
image = "<IMG_ID>" # The id of your image (look a packer for create your image with scaleway)
type = "DEV1-S" # The commercial instance type, look the scaleway instance catalog
services = [ "nginx" ] # The service you want to deploy on this server
},
# [...]
}The service it's required, for link hardware and network. When you write in infra.*.service["x"], x is a service and it's necessary to describe, what network is used by this service. The distinction is required between linked and hosted network for know if your service is a psql or is a soft who use psql.
services = {
nginx = {
networks = {
# ssh_public and ssh_private it's a tricks
hosted = ["public", "ssh_public"] # this service is not connected to the other
linked = ["web", "ssh_private"] # this service is linked with other services
}
}
# [...]
}You need to describe the network by service, with that you can use different service in single instance, and in the futur, you can move your service to the other pool.
networks = {
public = {
all = [ # use by nginx, it's for internet expose
{
action = "accept" # accept connexion, drop it's possible
port = 80 # the port used for the service
interface = "address" # Interface address == 0.0.0.0/0, public for public ip, private
protocol = "TCP" # protocol do you want use (UDP, TCP, ICMP)
},
{
action = "accept"
port = 443
interface = "address"
protocol = "TCP"
}
]
in = [] # if you want to drop or accept only for the inbounding
out = [] # if you want to drop or accept only for the outbounding
},
web = {
all = [
{
action = "accept"
port = 8080
interface = "private" # address || public || private || IP/range
}
]
in = []
out = []
},
database = {
# [...]
},
ssh_public = {
# [...]
},
ssh_private = {
# [...]
},
}