Platform Deployment Procedure: bosh deployment

Set up bosh deployment

From your local machine log in to box shown as bastion_ip

    $ ssh -i ~/.ssh/bosh.pem ubuntu@bastion_ip

Make sure that you have latest gems

    $ cd ~/workspace/deployments/cf-boshworkspace/
    $ bundle update

Edit deployment file

• for AWS deployments: $ vi ~/workspace/deployments/cf-boshworkspace/deployments/cf-aws-tiny.yml
• for OpenStack deployments: $ vi ~/workspace/deployments/cf-boshworkspace/deployments/cf-openstack-tiny.yml

Locate path meta and fill the following section with your own values:

  login_smtp:
    host: mail.YOURSMTP.com
    port: 25
    user: YOURUSER
    password: YOURPASS
    senderEmail: [email protected]
    senderName: Domain

Generate a secure password for uaac_admin_client_secret and note it down. Locate path meta and set generated password to secret. Note that this password should not contain special characters due to the limitations of CloudFoundry, please use characters [0-9A-Za-z].

...
meta:
...
  secret: <uaac_admin_client_secret>
...

Deploy changes

    $ cd ~/workspace/deployments/cf-boshworkspace/

• for AWS deployments:

    $ bosh deployment cf-aws-tiny

• for OpenStack deployments:

    $ bosh deployment cf-openstack-tiny

and execute:

    $ bosh prepare deployment
    $ bosh deploy

Set up Cloud Foundry

Login to Cloud Foundry using admin account and create organization and space

    $ cf login -a http://api.<cf_api_id>.xip.io/ -u admin -p <cf_admin_pass> --skip-ssl-validation
    $ cf create-org <org_name>
    $ cf target -o <org_name>
    $ cf create-space <space_name>
    $ cf target -s <space_name>

If your smtp server which you plan to use for sending e-mail messages in applications is located in one of private IP address ranges (A: 10.0.0.0/8, B: 172.16.0.0/12 or C: 192.168.0.0/16) please create an additional security group in Cloudfoundry, in below example we have two mailservers (10.0.0.1 and 192.168.0.2) and both are using TCP port 25

   $ cf create-security-group smtp <(echo '[{"destination":"10.0.0.1/32","ports":"25","protocol":"tcp"},{"destination":"192.168.0.2/32","ports":"25","protocol":"tcp"}]')

If this security group should be available for applications in all spaces in all orgs it should be binded by below command

   $ cf bind-running-security-group smtp

Else if it should be available only for one space (for example seedspace in organisation seedorg)

   $ cf bind-running-security-group smtp seedorg seedspace

Update default quota

Update the maximum amount of memory to 30 GB:

    $ cf update-quota default -m 30G

Add UAA clients

Generate a secure password for atk_client_pass and note it down. This password should not contain special characters due to the limitations of CloudFoundry, please use characters [0-9A-Za-z].

    $ uaac target https://uaa.<cf_api_id>.xip.io/ --skip-ssl-validation
    $ uaac token client get admin -s <uaac_admin_client_secret>
    $ uaac client add atk-client \
      --scope cloud_controller.read,cloud_controller.write,cloud_controller_service_permissions.read,openid \
      --authorized_grant_types authorization_code,password,refresh_token,client_credentials  \
      --redirect_uri http://atkdash.<cf_api_id>.xip.io/managed/auth/cloudfoundry/callback \
      --authorities uaa.none -s <atk_client_pass>
    $ uaac client update developer_console \
      --scope cloud_controller.admin,cloud_controller.read,cloud_controller.write,console.admin,doppler.firehose,openid,password.write,scim.read,scim.userids,scim.write \
      --authorized_grant_types authorization_code,client_credentials,refresh_token \
      --redirect_uri http://console.<cf_api_id>.xip.io/oauth/callback,https://console.<cf_api_id>.xip.io/oauth/callback \
      --authorities scim.read,uaa.admin,cloud_controller.admin,billing.admin,uaa.resource,password.write,scim.write,cloud_controller.write,cloud_controller.read \
      --autoapprove true --access_token_validity 1209600 --refresh_token_validity 1209600
    $ uaac group add console.admin
    $ uaac member add console.admin admin
    $ uaac client update cf \
      --scope cloud_controller.admin,cloud_controller.read,cloud_controller.write,doppler.firehose,openid,password.write,scim.read,scim.write,console.admin

Push apps and services

Before next step make sure that you installed all required components mentioned on the Development Environment Setup page.

If you do not have access to a remote artifacts repository and cannot download artifacts package, please contact one of the TAP developers listed below who will provide it.

Marko Łaboszczak <[email protected]>
Michał Bultrowicz <[email protected]>
Andrzej Dajnowski <[email protected]>
Grzegorz Świrski <[email protected]>
Andrzej Joskowski <[email protected]>

Next, clone two repos:

[email protected]:trustedanalytics/cloudfoundry-mkappstack.git
[email protected]:trustedanalytics/platform-appstack.git

Next step is filling the data in Cloud Foundry platform definition files, which provide necessary data for deployment process.

Please, follow instructions in Platform Appstack

Make sure that you copied necessary files from your platform-appstack directory to cloudfoundry-mkappstack directory as described in Platform Appstack Readme.

Go to cloudfoundry-mkappstack directory and copy templates

cd cloudfoundry-mkappstack
cp appstack.mk.tmpl appstack.mk
cp secret.mk.tmpl secret.mk

Enter your environment information to secret.mk. Edit cloudfoundry api endpoint, user, password, org & space.

When you are using cloudfoundry-mkappstack to deploy platform, the path to the artifacts must be set. Open appstack.mk file and set artifact_pfx:

artifact_pfx = file://<artifacts_directory>

For example if your artifacts are stored in /tmp/PACKAGES directory, your artifact_pfx should be set to:

artifact_pfx = file:///tmp/PACKAGES (remember about "file://" prefix!)

Fields afcturl and stack_mflist depend on whether your zipped artifact file names contain versions or not. Please, check the names format of zipped artifacts in artifacts directory.

If they contain versions and are in the following format: <appname>-<version>.zip (for example: app-launcher-helper-0.4.5.zip)

• in appstack.mk set the following afcturl:

afcturl = $(artifact_pfx)/$(appname)-$(appver).zip

• in appstack.mk set the following stack_mflist:

stack_mflist = versions.yml settings.yml appstack.yml

If they do not contain version and are in the following format: <appname>.zip (For example: app-launcher-helper.zip)

• in appstack.mk set following afcturl:

afcturl = $(artifact_pfx)/$(appname).zip

• in appstack.mk set the following stack_mflist:

stack_mflist = settings.yml appstack.yml

Uncomment proxy and noproxy and put correct proxy values there.

Now you are ready to deploy whole platform:

export GODEBUG=netdns=cgo
cf api --unset
make deploy

Devops remarks

• Set up and assign to AWS user an appropriate account regional policy, i.e. (please remember to change the ec2:Region attribute):

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ec2:*",
                "vpc:*"
            ],
            "Condition": {
                "StringEquals": {
                    "ec2:Region": "us-east-1"
                }
            },
            "Resource": [
                "*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "elasticloadbalancing:*"
            ],
            "Resource": [
                "*"
            ]
        }
    ]
}