-
Notifications
You must be signed in to change notification settings - Fork 13
edX AWS Cloudformation Template
The first step is to provision the CloudFormation stack. There are several options for doing this.
- The AWS console
- The AWS CloudFormation CLI
- Via Ansible
If you don't have experience with CloudFormation, the web console is a good place to start because it will use a form wizard to gather configuration parameters, it will give you continuous feedback during the process of building the stack and useful error messages when problems occur.
Before you create the stack you will need to create a key-pair that can
be used to connect to the stack once it's instantiated. To do this
go to the 'EC2' section and create a new key-pair under the 'Key Pairs'
section. Note the name of this key and update
cloudformation_templates/edx-reference-architecture.json file. Under the
'KeyName' section change the value of 'Default' to the name of the key-pair
you just created.
Details on how to build the stack using Ansible are available below.
From the AWS main page that lists all the services you can use. Click on the
CloudFormation link. This will take you to a list of cloud stacks you currently
have active. Here click the 'Create Stack' button. In the wizard you can give a
name for your stack and pass in a template which defines the edX stack. Use the
edx-reference-architecture.json template in the cloudformation_templates directory.
To build from the CloudFormation CLI you will have to first upload the configuration
file to an S3 Bucket. The easiest way to do this is to use s3cmd.
s3cmd put /path/to/edx-reference-architecture.json s3://<bucket_name>
aws cloudformation create-stack --stack-name <stack_name> --template-url https://s3.amazonaws.com/<bucket_name>/edx-reference-architecture.json --capabilities CAPABILITY_IAM
Because the reference architecture makes use of an Amazon VPC, you will not be able to address the hosts in the private subnets directly. However, you can easily set up a transparent "jumpbox" so that for all hosts in your vpc, connections are tunneled.
Add something like the following to your ~/.ssh/config file.
Host *.us-west-1.compute-internal
ProxyCommand ssh -W %h:%p vpc-us-west-1-jumpbox
ForwardAgent yes
Host vpc-us-west-1-jumpbox
HostName 54.236.202.101
ForwardAgent yes
This assumes that you only have one VPC in the us-west-1 region
that you're trying to ssh into. Internal DNS names aren't qualified
any further than that, so to support multiple VPC's you'd have to get
creative with subnets, for example ip-10-1 and ip-10-2...
Test this by typing ssh ip-10-0-10-1.us-west-1.compute.internal,
(of course using a hostname exists in your environment.) If things
are configured correctly you will ssh to 10.0.10.1, jumping
transparently via your basion host.
Getting this working is important because we'll be using Ansible with the SSH transport and it will rely on this configuration being in place in order to configure your servers.
You can automate building these rules for your VPC by using the
vpc-tools.py script found in this repository at configuration/util/vpc-tools/vpc-tools.py
Run the script with no arguments to get its usage information. An example command might look like:
python vpc-tools.py ssh-config stack-name testedx identity-file /path/to/private/key user ubuntu
Updating the stack name and the path to the identity file should produce ssh configurations which should be copied into your ~/.ssh/config file.
Boto is how fabric looks up metadata about your stack, most importantly
finding the names of your machines. It needs your access information.
This should be the contents of your ~/.boto file. Make sure
to customize the region:
[Credentials]
aws_access_key_id = AAAAAAAAAAAAAAAAAAAA
aws_secret_access_key = BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
[Boto]
debug = 1
ec2_region_name = us-west-1
ec2_region_endpoint = ec2.us-west-1.amazonaws.comTagging is the bridge between the provisioning and configuration phases. The servers provisioned in your VPC will be stock Ubuntu 12.0.4 LTS servers. The only difference between them will be the tags that CloudFormation has applied to them. These tags will be used by Ansible to map playbooks to the correct servers. The application of the appropriate playbook, will turn each stock host into an appropriately configured service.
The Group tag is where the magic happens. Every AWS EC2 instance will have a Group tag that corresponds to a group of machines that need to be deployed/targeted to as a group of servers.
Example:
-
Group:edxapp_stage -
Group:edxapp_prod -
Group:edxapp_some_other_environment
Additional tags can be added to AWS resources in the stack but they should not be made necessary deployment or configuration.
This assumes that you have working ssh as described above
cd playbooks
ansible-playbook -vvv cloudformation.yml -i inventory.ini -e 'region=<aws_region> key=<key_name> name=<stack_name> group=<group_name>'
-
aws_region: example:
us-east-1. Which AWS EC2 region to build stack in. -
key_name: example:
deploy. SSH key name configured in AWS for the region -
stack_name: example:
EdxAppCustom. Name of the stack, must not contain underscores or cloudformation will complain. Must be an unused name or otherwise the existing stack will update. -
group_name: example:
edxapp_stage. The group name should correspond to one of the yml files in theplaybooks/. Used for grouping hosts.
While this is running you see the cloudformation events in the AWS console as
the stack is brought up. Loads the playbooks/cloudformation.yml template
which creates a single small EBS backed EC2 instance.
Note: You should read the output from ansible and not necessarily trust the 'ok'; failures in cloudformation provisioning (for example, in creating the security group), may not cause ansible-playbook to fail.
See files/examples for adding other components to the stack.