Beginners Guide to installing the GMS on an Amazon Web Services (AWS) Instance
This tutorial will walk you through an installation of the Genome Modeling System(GMS) on a Community Amazon Machine Image(AMI) of Ubuntu 12.04.
NOTE: We are currently hosting a live demonstration of the GMS, installed on Amazon AWS. The Beginner's Guide to the Demonstration Analysis and Guide to Importing and Analyzing External Data have been completed on that instance. The web viewer for that demonstration instance can be browsed at: http://ec2-52-10-204-88.us-west-2.compute.amazonaws.com/. Please visit the tutorial on Sharing GMS results using an Amazon AWS Instance for more details on how it was set up and how to connect to it.
To create your own AWS install of the GMS, first open the Amazon AWS Console website.
Login to the AWS console by clicking on the yellow button that says "Sign in to the AWS console."
Click "EC2 - Virtual Servers in the Cloud." In the right hand corner of the AWS console, you may also want to select the "AWS region" that makes sense based on your geographical location (or cost perhaps).
Click the blue "Launch Instance" button.
This brings you to a screen where you can choose the AMI you like. Click the "Community AMIs" link on the left.
This brings you to the Community AMI screen.
Find the following AMI in the 'Community AMIs': "SGMS_INSTALL_V3" and hit search. Note this AMI is available currently in the Oregon region.
This should pull up the right AMI, click the AMI.
The next step is to choose an instance type that meets our resource needs. For the installation of the GMS choose EC2 instance type: i2.2xlarge (8 cpus, 61 Gb ram, 2x800 Gb ssd storage). Other instance type that work well with this demo include: 'c3.8xlarge', 'r3.8xlarge', 'i2.4xlarge', 'i2.8xlarge', and 'hs1.8xlarge'. Some of these instance types may require you to submit a ticket to Amazon to be 'unlocked'.
Click "Next" to Configure the Instance Details. The default options are fine.
Click "Next" to Add Storage. The default options for storage work fine.
Click "Next" to Create Tags for the instance, this helps track usage metrics later. For example, you might create key-value pairs like: "instance_type"="i2.2xlarge" and "ami_name"="SGMS_INSTALL_V3".
Click "Next" to create a new security group. The default settings will be sufficient for demonstration purposes. To make the GMS webviewer publicly accessible you can read about Sharing GMS results using an Amazon AWS Instance. However, if you plan to import your own data it is STRONGLY RECOMMENDED that you CONSULT WITH A SECURITY PROFESSIONAL to set up your instance with appropriate security.
Review the details of the instance to be launched and hit 'Launch' once you are ready.
If it is your first time creating this instance then select 'create a new key pair' in the AWS console and give it a name (e.g., sgms-key). Save this 'private key file' somewhere handy/safe with an appropriate file name: e.g. sgms-key.pem. Once you have created the key pair, or if you've created this instance before, you can select 'choose an existing key pair' for authentication and hit launch instance.
This will show some warnings and launch the instance. Click the 'View Instances' button to see the status of your instance(s).
Once the instance has been successfully launched the 'Instance State' changes to 'running'. The public IP address of the instance is also shown on this screen.
Now we are all set to login to the instance using a terminal application. To login you will need the security key and the ip address. Login looks something like the following. You may also need to alter the permissions of the key file.
chmod 600 sgms-key.pem
ssh -i sgms-key.pem [email protected]
If you see an error and fail to login, most likely you forgot the ubuntu@ part of the login command, or you failed to change the permissions on your key file, or you didn't use the correct public IP address.
You are now logged in to the AMI instance.
Get sGMS installation repo from github and change directory into the repo:
git clone https://github.com/genome/gms.git
cd gms
Run the AWS pre-install script:
bash setup/aws/preinstall.sh
Note that this pre-install step performs basic provisioning of storage and it assumes that there are two volumes present. This is the case for the EC2 instance type listed above and several others that will work with this demonstration. However, if you have more or less of these volumes available in the instance you should customize setup/aws/preinstall.sh to match the instance type.
On one particular instance with two EBS backed volumes(200 GB each), running df -h looks as below.
Once the pre-install is complete, change directory to /opt/src/gms and run make to start the install of the GMS:
cd /opt/src/gms
make
Enter the password for the 'ubuntu' user. By default we have set this password to 'sgms'. If you wish to change this password you can use the passwd command and follow the prompts. CHANGING YOUR PASSWORD IS STRONGLY RECOMMENDED.
Wait for the install to complete, it takes ~30 minutes with good network speeds(~10 Mbps).
Logout and log back in once the install is complete. This can be done by issuing an exit or logout command in the terminal and then logging in again (remembering to use your own IP address).
ssh -i sgms-key.pem [email protected]
Congratulations, you have now successfully installed the GMS !
To get this far you need an EC2 instance with 5 Gb for the OS and an additional 10Gb for the sGMS install (in this example the entire sGMS lives on a separate mount).
Information on complete sanity-checks and database priming can be found here.
Now before we can launch builds and start our analysis we need to prime the system with basic database entries, reference sequences, test data, etc. To do this you will need at least 80 GB of storage on the EC2 instance. To do this,
'cd' into the source directory
cd /opt/src/gms
Run the priming script. Make sure to set --memory and --cpus equal to the memory and number of cpus of the AWS instance type you chose.
./setup/prime-system.pl --data=hcc1395_1tenth_percent --sync=tarball --low_resources --memory=32gb --cpus=8 --metadata=setup/metadata/b6e5b2395bab44289123e9114817a92d.dat
Now the system is fully ready to start your analysis! You may proceed to Beginner's Guide to the Demonstration Analysis.
Testing of the GMS on Amazon AWS EC2 and development of this documentation was generously supported by Amazon AWS Education grants.