Upkeep: jira-cloudformation-maven-plugin

Maven plugin for creating and shutting down JIRA Data Centre AWS cloud formation based on a provided template. This plugin will also restore JIRA data using postgre backup and indexes stored on S3 bucket. See https://github.com/hindsightsoftware/upkeep-jira-example project for more detailed use case example.

What does it exactly do?

This plugin will perform the following using start goal:

1. Use the JIRA template and create a AWS cloud formation stack.
2. Wait for the stack to complete.
3. Use the ssh key to log into EC2 instances.
4. Delete JIRA Postgres database if one exists.
5. Restore JIRA Postgres database from a backup downloaded from S3 bucket.
6. Restore JIRA indexes from a backup downloaded from S3 bucket.
7. Start JIRA on all instances.
8. Wait for load balancer health check.

The second goal: stop will stop the cloud formation and will delete all resources that have been generated.

Parameters

• skip - Whether to skip entire cloud formation process
• jira.cloudformation.template.url - Url to the template file. For example: https://s3.amazonaws.com/quickstart-reference/atlassian/jira/latest/templates/JiraDataCenter.template
• jira.cloudformation.conf.file (defaultValue = ${project.build.testOutputDirectory}/cloudformation.conf) - Where to store output configuration file. This file will contain outputs generated by the template once the cloud formation creation finishes. You can also specify what outputs should be exported via exports parameter (see below).
• jira.cloudformation.region - (defaultValue = us-west-2) - AWS region code
• jira.cloudformation.stack.name - (defaultValue = JIRA-Data-Center) - Name of the stack. If a stack with an identical name already exists, it is assumed that the stack has been created and will be used for integration testing.
• jira.cloudformation.credentials - (defaultValue = aws.properties) - Path to credentials. The file needs to contain two key-value pairs: accessKey=.... and secretKey=....
• jira.cloudformation.onfailure - (defaultValue = DELETE) - What to do if the cloud formation creation fails.
• jira.cloudformation.load.balancer.id - (defaultValue = LoadBalancer) - The logical name of the load balancer that will be created. This value is used to retrieve physical ID of the balancer. The logical name needs to match the value specified by template. If you have used JiraDataCenter.template provided by amazonaws, the balancer id is LoadBalancer.
• jira.cloudformation.base.url.id - (defaultValue = JIRAURL) - Name of the output parameter that will be generated by stack creation. The template specified multiple output values and extracting URL is necessary.
• jira.cloudformation.load.balancer - If you have custom template where you use already existing load balancer, you can specify the physical ID here.
• jira.cloudformation.base.url - You can override the base URL here. If a value is provided, no base url is going to be extracted from the template outputs.
• jira.cloudformation.base.url.path - (defaultValue = ${project.build.testOutputDirectory}/baseurl) - Where to export the base url. The generated file will be a simple text file containing only the base url.
• jira.cloudformation.rds.id - (defaultValue = DB) - Similarly to load.balancer.id this specifies the logical id of the RDS that is generated by the template.
• jira.cloudformation.rds - You can override the endpoint of the databse here. If this value is not empty, the endpoint from here is going to be used, otherwise the endpoint is extracted using the physical ID from the template generated resources.
• jira.cloudformation.rds.password - Password for the RDS. This password needs to match the one you put into template as the master password. This password will be used to restore JIRA data using pg_restore
• parameters - Any key-value parameters needed for the template. See sample maven configuration below.
• exports - What output key-value pairs need to be exported?
• jira.cloudformation.ssh.private.key - Relative or absolute path to SSH key-pair that will be used to access EC2 instances via SSH. This needs to match* the key-pair you specify in the template.
• jira.cloudformation.s3.aws.credentails - Credentials that will be used to access S3 bucket and download postgres backup and indexes in order to restore JIRA data. This credentials only need read-only access to S3 bucket. See: http://docs.aws.amazon.com/cli/latest/userguide/cli-config-files.html for more information.
• jira.cloudformation.s3.aws.config - Config file needed by the credentials.
• jira.cloudformation.s3.restore.enabled - (defaultValue = true) - Set to false if you wish not to restore any JIRA data using postgres backup.
• jira.cloudformation.s3.restore.bucket - Name of the bucket that contains the backup files.
• jira.cloudformation.s3.restore.psql - Name of backup file that contains postgres data. You can create your own backup by logging into any EC2 JIRA node and running pg_dump -i -h [RDS_endpoint_url] -p 5432 -U postgres -F c -b -v -f "your_backup_file.psql" jira. Alternatively, there is a sample project exported and provided in this repository, see: upkeep-jira-cloudformation-maven-plugin/sample-project.psql file, just upload it into your S3 bucket.
• jira.cloudformation.s3.restore.indexes - name of the tar file that contains indexes. Re-indexing whole JIRA takes time and this is the simplest and fastest workaround. There is a sample project exported and provided in this repository, see: upkeep-jira-cloudformation-maven-plugin/sample-project-indexes.tar file, just upload it into your S3 bucket.
• jira.cloudformation.max.wait.jira - (defaultValue = 300) - The maximum time in seconds that will be spent waiting for JIRA to boot-up (after restoring backup).
• jira.cloudformation.max.wait.load - (defaultValue = 300) - The maximum time in seconss that will be spent waiting for the load balancer to update its status (health check).

Sample maven configuration

<project>
    <!-- Common project settings -->
    <properties>
        <jira.cloudformation.ssh.private.key>${user.home}/.ssh/jira-keypair.pem</jira.cloudformation.ssh.private.key>
        <jira.cloudformation.template.url>https://s3.amazonaws.com/quickstart-reference/atlassian/jira/latest/templates/JiraDataCenter.template</jira.cloudformation.template.url>
        <jira.cloudformation.conf.file>${project.build.testOutputDirectory}/cloudformation.conf</jira.cloudformation.conf.file>
        <jira.cloudformation.region>us-east-1</jira.cloudformation.region>
        <jira.cloudformation.stack.name>JIRA-Data-Center</jira.cloudformation.stack.name>
        <jira.cloudformation.s3.aws.credentails>${project.basedir}/src/test/resources/s3.aws.user/credentials</jira.cloudformation.s3.aws.credentails>
        <jira.cloudformation.s3.aws.config>${project.basedir}/src/test/resources/s3.aws.user/config</jira.cloudformation.s3.aws.config>
        <jira.cloudformation.s3.restore.bucket>my-jira-backup-s3-bucket</jira.cloudformation.s3.restore.bucket>
        <jira.cloudformation.s3.restore.psql>sample-project.psql</jira.cloudformation.s3.restore.psql>
        <jira.cloudformation.s3.restore.indexes>sample-project-indexes.tar.gz</jira.cloudformation.s3.restore.indexes>
        <jira.cloudformation.rds.password>rdsmasterpassword</jira.cloudformation.rds.password>
        
        <skipCloudFormation>false</skipCloudFormation>
    </properties>
    <!-- dependencies, etc... -->
    <build>
        <plugins>
            <plugin>
                <groupId>com.hindsightsoftware.upkeep</groupId>
                <artifactId>jira-cloudformation-maven-plugin</artifactId>
                <version>1.0</version>
                <executions>
                    <execution>
                        <id>start-cloud-formation</id>
                        <phase>pre-integration-test</phase>
                        <goals>
                            <goal>start</goal>
                        </goals>
                        <configuration>
                            <!-- Parameters needed by the template.
                                 Optional keys need to be explicitly declared with no value -->
                            <parameters>
                                <!-- In this example, we are using the default VPC with public
                                         subnets so we can access database and instances more easily.
                                         You should not do this for production environment! However,
                                         we are doing this just for testing purposes. The JIRA template
                                         allows us to define already existing VPC. -->
                                <VPC>vpc-7fbcd119</VPC>
                                <ExternalSubnets>subnet-b15306ea,subnet-167ceb2a</ExternalSubnets>
                                <InternalSubnets>subnet-b15306ea,subnet-167ceb2a</InternalSubnets>
                                <AssociatePublicIpAddress>true</AssociatePublicIpAddress>
                                <DBMasterUserPassword>rdsmasterpassword</DBMasterUserPassword>
                                <DBPassword>rdsmasterpassword</DBPassword>
                                <KeyName>jira-keypair</KeyName>
                                <CustomDnsName/>
                                <JiraProduct>Software</JiraProduct>
                                <ClusterNodeMin>1</ClusterNodeMin>
                                <ClusterNodeMax>2</ClusterNodeMax>
                            </parameters>
                            <!-- What parameters to export once the stack is built?
                                 All params will be exported to jira.cloudformation.conf.file
                                 as key-value pairs -->
                            <exports>
                                <param>JIRAURL</param>
                                <param>LoadBalancerURL</param>
                            </exports>
                            <skip>${skipCloudFormation}</skip>
                        </configuration>
                    </execution>
                    <execution>
                        <id>stop-cloud-formation</id>
                        <phase>post-integration-test</phase>
                        <goals>
                            <goal>stop</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <!-- More plugins here -->
        </plugins>
    </build>
</project>