Platform Deployment Procedure: bosh deployment

For release TAP-2016-JAN-2-1/TAP-2016-JAN-2-2 please refer to this document

Set up bosh deployment

From your local machine log in to box shown as bastion_ip

    $ ssh -i ~/.ssh/bosh.pem ubuntu@bastion_ip

Make sure that you have latest gems

    $ cd ~/workspace/deployments/cf-boshworkspace/
    $ bundle update

Edit deployment file

• for AWS deployments: $ vi ~/workspace/deployments/cf-boshworkspace/deployments/cf-aws-tiny.yml
• for OpenStack deployments: $ vi ~/workspace/deployments/cf-boshworkspace/deployments/cf-openstack-tiny.yml

Locate path meta and fill the following section with your own values:

  login_smtp:
    host: mail.YOURSMTP.com
    port: 25
    user: YOURUSER
    password: YOURPASS
    senderEmail: [email protected]
    senderName: Domain

NOTE! In order to configure connection to SMTP server without authorization the value null (without quotes) should be used for fields username and password, e.g.:

  login_smtp:
    host: mail.domain.com
    port: 25
    user: null
    password: null
    senderEmail: [email protected]
    senderName: Domain

Deploy changes

    $ cd ~/workspace/deployments/cf-boshworkspace/

• for AWS deployments:

    $ bosh deployment cf-aws-tiny

• for OpenStack deployments:

    $ bosh deployment cf-openstack-tiny

and execute:

    $ bosh prepare deployment
    $ bosh deploy

Set up Cloud Foundry

Login to Cloud Foundry using admin account

    $ cf login -a http://cf-api.<cf_api_id>.xip.io/ -u admin -p <cf_admin_pass> --skip-ssl-validation
    $ cf target -o system
    $ cf create-space system
    $ cf target -s system

If your smtp server which you plan to use for sending e-mail messages in applications is located in one of private IP address ranges (A: 10.0.0.0/8, B: 172.16.0.0/12 or C: 192.168.0.0/16) please create an additional security group in Cloudfoundry, in below example we have two mailservers (10.0.0.1 and 192.168.0.2) and both are using TCP port 25

   $ cf create-security-group smtp <(echo '[{"destination":"10.0.0.1/32","ports":"25","protocol":"tcp"},{"destination":"192.168.0.2/32","ports":"25","protocol":"tcp"}]')

If this security group should be available for applications in all spaces in all orgs it should be binded by below command

   $ cf bind-running-security-group smtp

Else if it should be available only for one space (for example seedspace in organisation seedorg)

   $ cf bind-running-security-group smtp seedorg seedspace

ℹ️ You should consider using similiar procedure for proxy security group, if your cluster requires proxy usage for data ingestion.

Update default quota

Update the maximum amount of memory to 30 GB:

    $ cf update-quota default -m 30G

Add UAA clients

In this step, if you use any no_proxy rules, please remove all http_proxy and https_proxy rules in your env. UAAC doesn't understand no_proxy setting.

Generate a secure password for atk_client_pass and note it down, cf_client_pass was defined at the beginning of deployment in terraform.tfvars file. Passwords should not contain special characters due to the limitations of CloudFoundry, please use characters [0-9A-Za-z].

    $ uaac target https://uaa.<cf_api_id>.xip.io/ --skip-ssl-validation
    $ uaac token client get admin -s <cf_client_pass>
    $ uaac client add atk-client \
      --scope cloud_controller.read,cloud_controller.write,cloud_controller_service_permissions.read,openid \
      --authorized_grant_types authorization_code,password,refresh_token,client_credentials  \
      --redirect_uri http://atkdash.<cf_api_id>.xip.io/managed/auth/cloudfoundry/callback \
      --authorities uaa.none -s <atk_client_pass>
    $ uaac client add tap_console \
      --scope cloud_controller.admin,cloud_controller.read,cloud_controller.write,console.admin,doppler.firehose,openid,password.write,scim.read,scim.userids,scim.write \
      --authorized_grant_types authorization_code,client_credentials,refresh_token \
      --redirect_uri http://console.<cf_api_id>.xip.io/oauth/callback,https://console.<cf_api_id>.xip.io/oauth/callback \
      --authorities scim.read,uaa.admin,cloud_controller.admin,billing.admin,uaa.resource,password.write,scim.write,cloud_controller.write,cloud_controller.read,clients.write \
      --autoapprove true --access_token_validity 1209600 --refresh_token_validity 1209600 \
      -s <cf_client_pass>
    $ uaac group add console.admin
    $ uaac member add console.admin admin
    $ uaac client update cf \
      --scope cloud_controller.admin,cloud_controller.read,cloud_controller.write,doppler.firehose,openid,password.write,scim.read,scim.write,console.admin
    $ uaac group add hue.admin
    $ uaac client add hue \
    --scope hue.admin,scim.read,scim.userids,uaa.user \
    --authorized_grant_types authorization_code,refresh_token \
    --authorities uaa.user,oauth.login \
    --redirect_uri http://hue.<cf_api_id>.xip.io/oauth/callback \
    --autoapprove true \
    -s huesecret

Members of hue.admin group will have admin privileges in hue.

For Kerberos deployments you may have to upload a UAA public key to a KDC. To do that please log into cdh-launcher host, then run:

• for AWS deployments: $ cd ansible-cdh/
• for OpenStack deployments: $ cd ansible-cdh/platform-ansible/

    $ sh kerberos_token.sh <domain_ name>

where domain_name is a domain specifiec in terraform.tfvars at the beginning of the deployment.

Push apps and services

Follow procedure in Platform application layer deployment