Skip to content

ATK deployment procedure using app launching service broker

Jump to bottom
Grzegorz Świrski edited this page Dec 1, 2015 · 4 revisions

Download and configure app-launching-service-broker

Do the following:

  1. Clone the app-launching-service-broker and go to the downloaded directory:

     git clone [email protected]:trustedanalytics/app-launching-service-broker.git <broker-app-name>
 cd <broker-app-name>

  2. Run the fetch command to download the required cf-cli binaries:

    ./bin/fetch_cf_cli.sh

  3. Open manifest.yml then edit the broker name (which must be unique, for example, atk-broker) and add env values:

    CF_API: http://api.<domain>
CF_DEP: postgresql93|free,cdh|shared,zookeeper|shared
CF_SRC: ./apps/atk
CF_USER: <user, eg. admin>
CF_PASS: <password_for_CF_USER>

  4. The manifest after that should look like this:

    ---
applications:
- name: atk-broker
  memory: 256M
  instances: 1
  path: .
  buildpack: go_buildpack
  env:
    CF_API: http://api.domain.com
    CF_CATALOG_PATH: ./catalog.json
    CF_DEP: postgresql93|free,cdh|shared,zookeeper|shared
    CF_SRC: ./apps/atk
    CF_USER: admin
    CF_PASS: pass_for_CF_USER

  5. Open catalog.json and edit the values as follows:

    • services -> id (must be unique in the environment)
    • services -> name (must be unique; for example, atk*)
    • services -> description
    • services -> tags
    • services -> plans -> id (must be unique in the environment)

    Example catalog.json:

    {
  "services": [{
    "id": "c00be1df-89ce-4d42-8b21-644afaf41121",
    "name": "atk",
    "description": "Intel Analytics Toolkit",
    "bindable": true,
    "tags": ["atk", "analytics"],
    "plans": [{
      "id": "04eb29c9-5b17-4b70-9740-9d9e9cf20db1",
      "name": "simple",
      "description": "Simple",
      "free": true
    }]
  }]
}

NATS

The new version of the app-launching-service-broker uses a message bus called NATS to populate information about the service-creation progress. Use of this feature is optional. It will work conditionally, when nats-provider service is bound to the app. To create such a service, do the following:

cft cups nats-provider -p "{\"url\": \"nats://<USER>:<PASSWORD>@<ADDRESS>:<PORT>\",\"service-creation-subject\":\"platform.<CHOSEN-TOPIC-NAME>\"}"

Note that the platform. prefix is crucial. To bind it to the application, you have to add the services section to manifest.yml. It should look like this:

---
applications:
- name: atk-broker
  memory: 256M
  instances: 1
  path: .
  buildpack: go_buildpack
  services:
    - nats-provider
  env:
    CF_API: http://api.domain.com
    CF_CATALOG_PATH: ./catalog.json
    CF_DEP: postgresql93|free,cdh|shared,zookeeper|shared
    CF_SRC: ./apps/atk
    CF_USER: admin
    CF_PASS: pass_for_CF_USER

Download ATK

Download ATK from S3. The last known working version is: https://s3.amazonaws.com/gao-internal-archive/sprint29/atk.tar.gz. Unpack it to the apps/atk directory inside the app-launching-service-broker. (Use the name you specified during cloning.)

Prepare ATK

  1. Add manifest.yml. It should look like this. (You can copy the following without changing anything.)

    ---
applications:
- name: atk # must be the same as services -> name in catalog.json
  command: bin/rest-server.sh
  memory: 1G
  disk_quota: 2G
  timeout: 180
  instances: 1

  2. Remove '.apps' and '.run'. After that, these lines should look like this:

    export CC_URI=$(echo $VCAP_APPLICATION | $jq  '.application_uris[0]' | sed -e "s/$APP_NAME/http:\/\/api/g" | tr -d '"')

export UAA_URI=$(echo $VCAP_APPLICATION | $jq '.application_uris[0]' | sed -e "s/$APP_NAME/http://uaa/g" | tr -d '"')

9. Before the push, you will need to add three variables for spark inside _conf/application.conf_:

    ```
    spark.eventLog.overwrite = "true"
    spark.eventLog.enabled = "true"
    spark.eventLog.dir = "hdfs://ip-10-10-10-236.us-west-2.compute.internal:8020/user/spark/applicationHistory"
    ```
    Where the example value _ip-10-10-10-236.us-west-2.compute.internal_ is the hostname of hdfs namenode.
#### Push broker
Go to the broker's main directory and push the broker with ```cf push```. This command will create a service broker application. If it is the first push of _atk-broker_, you need to create a service broker and enable access to this service. (```<service_broker_name>``` should be unique. For example, atk*, ```<your_service_name>``` should be the same as services -> name in catalog.json or check it using cf service-access):

cf create-service-broker <service_broker_name> admin admin http:// cf enable-service-access <your_service_name>

If you had atk-broker before and it is the only update, run the update command:

cf update-service-broker <service_broker_name> admin admin http://

#### Create an ATK instance
There are two ways to create an ATK instance:
- Use the console.
- Use the command line (like this): `cf cs <service_broker_name> simple <atk-instance-name>`

NOTE: The name of the created app will be different from ```<atk-instance-name>```. You can check the app name in the console.

#### Verify ATK client connection

To verify that the ATK Server works properly, do the following:

9. Install the ATK client and try to connect it to the ATK server. 

9. Open [ATK Client Installation Configuration and Usage](ATK-Overview#client-installation-configuration-and-usage) page.

Home | FAQs

Clone this wiki locally