Run Demo & local development setup with IntelliJ IDEA

As an alternative to the dockerized run, here's a way how to setup your project with Intellij IDEA using external services you may already have installed.

Prerequisites:

• Cassandra is running at cassandraserver (tested with v. 2.1.3) setup keyspace with:
  1. wget https://raw.githubusercontent.com/pulsarIO/realtime-analytics/master/metriccalculator/pulsar.cql -O /tmp/pulsar.cql
  2. cqlsh -f /tmp/pulsar.cql
• MongoDB is running at mongoserver (tested with v. 2.6.7)
• Zookeeper is running at zkserver (tested with v. 3.4.6)
• Kafka is running at kafkaserver (tested with v. 2.10-0.8.1.1)

Local /etc/hosts setup (assuming your services run at 192.168.0.1)

192.168.0.1 zkserver

192.168.0.1 mongoserver

192.168.0.1 kafkaserver

192.168.0.1 cassandraserver

Notes

YOUR_LOCAL_IP = your local machine's ip PATH_TO = full path to the app

Project realtime-analytics##

01 - Replay App

Application Run Configuration

name: 01 - replay

Main class: com.ebay.jetstream.application.JetstreamApplication

VM options: -Dpulsar.runtime.kafka.brokers=kafkaserver:9092 -Dpulsar.runtime.kafka.zk=kafkaserver:2181 -Djetstream.runtime.zkserver.host=zkserver -Djetstream.runtime.zkserver.port=2181 -Djetstream.runtime.netmask=YOUR_LOCAL_IP/24

Program arguments: -p 8001 -n "replay pulsar app" -cv "1.0"

Working directory: $MODULE_DIR$

Environment variables:

JETSTREAM_HOME = PATH_TO/realtime-analytics/replay/buildsrc

MONGO_HOME = mongo://mongoserver:27017/config

Use classpath of module: replay

02 - Collector App

Application Run Configuration

name: 02 - collector

Main class: com.ebay.jetstream.application.JetstreamApplication

VM options: -Dpulsar.runtime.kafka.brokers=kafkaserver:9092 -Djetstream.runtime.zkserver.host=zkserver -Djetstream.runtime.zkserver.port=2181 -Djetstream.runtime.netmask=YOUR_LOCAL_IP/24

Program arguments: -p 8002 -n "collector app" -cv "1.0"

Working directory: $MODULE_DIR$

Environment variables:

JETSTREAM_HOME = PATH_TO/realtime-analytics/collector/buildsrc

Use classpath of module: collector

03 - Sessionizer App

Application Run Configuration

name: 03 - sessionizer

Main class: com.ebay.jetstream.application.JetstreamApplication

VM options: -Dpulsar.runtime.kafka.brokers=kafkaserver:9092 -Djetstream.runtime.zkserver.host=zkserver -Djetstream.runtime.zkserver.port=2181 -Djetstream.runtime.netmask=YOUR_LOCAL_IP/24

Program arguments: -p 8003 -n "sessionizer app" -cv "1.0"

Working directory: $MODULE_DIR$

Environment variables:

JETSTREAM_HOME = PATH_TO/realtime-analytics/sessionizer/buildsrc

MONGO_HOME = mongo://mongoserver:27017/config

Use classpath of module: sessionizer

04 - Distributor App

Application Run Configuration

name: 04 - distributor

Main class: com.ebay.jetstream.application.JetstreamApplication

VM options: -Dpulsar.runtime.kafka.brokers=kafkaserver:9092 -Djetstream.runtime.zkserver.host=zkserver -Djetstream.runtime.zkserver.port=2181 -Djetstream.runtime.netmask=YOUR_LOCAL_IP/24

Program arguments: -p 8004 -n "distributor app" -cv "1.0"

Working directory: $MODULE_DIR$

Environment variables:

JETSTREAM_HOME = PATH_TO/realtime-analytics/distributor/buildsrc

MONGO_HOME = mongo://mongoserver:27017/config

Use classpath of module: distributor

05- Metriccalculator App

Application Run Configuration

name: 05 - metriccalculator

Main class: com.ebay.jetstream.application.JetstreamApplication

VM options: -Dpulsar.runtime.kafka.brokers=kafkaserver:9092 -Dpulsar.runtime.cassandra=cassandraserver -Djetstream.runtime.zkserver.host=zkserver -Djetstream.runtime.zkserver.port=2181 -Djetstream.runtime.netmask=YOUR_LOCAL_IP/24

Program arguments: -p 8005 -n "metriccalculator app" -cv "1.0"

Working directory: $MODULE_DIR$

Environment variables:

JETSTREAM_HOME = PATH_TO/realtime-analytics/metriccalculator/buildsrc

MONGO_HOME = mongo://mongoserver:27017/config

Use classpath of module: metriccalculator

Project Demo

08 - Metriccalculator App

Application Run Configuration

name: 08 - metricservice

Main class: com.ebay.jetstream.application.JetstreamApplication

VM options: -Dpulsar.runtime.cassandra=cassandraserver -Djetstream.runtime.zkserver.host=zkserver -Djetstream.runtime.zkserver.port=2181 -Djetstream.runtime.netmask=YOUR_LOCAL_IP/24

Program arguments: -p 8006 -n "metricservice app" -cv "1.0"

Working directory: $MODULE_DIR$

Environment variables:

JETSTREAM_HOME = PATH_TO/realtime-analytics/Demo/metricservice/buildsrc

MONGO_HOME = mongo://mongoserver:27017/config

Use classpath of module: metricservice

09 - MetricUI App

Application Run Configuration

name: 09 - metricUI

Main class: com.ebay.jetstream.application.JetstreamApplication

VM options: -Dpulsar.runtime.kafka.brokers=kafkaserver:9092 -Dpulsar.runtime.kafka.zk=kafkaserver:2181 -Djetstream.runtime.zkserver.host=zkserver -Djetstream.runtime.zkserver.port=2181 -Djetstream.runtime.netmask=YOUR_LOCAL_IP/24 -Dpulsar.runtime.cassandra=cassandraserver -Dmetriccalculator.port=8005

Program arguments: -p 8007 -n "metricUI" -cv "1.0"

Working directory: $MODULE_DIR$

Environment variables:

JETSTREAM_HOME = PATH_TO/realtime-analytics/Demo/metricUI/buildsrc

MONGO_HOME = mongo://mongoserver:27017/config

Use classpath of module: metricUI

10 - Twitter Sample App

Application Run Configuration

name: 10 - twittersample

Main class: com.ebay.jetstream.application.JetstreamApplication

VM options: -Djetstream.runtime.zkserver.host=zkserver -Djetstream.runtime.zkserver.port=2181 -Djetstream.runtime.netmask=YOUR_LOCAL_IP/24 -Dtwitter4j.oauth.consumerKey="XYZ" -Dtwitter4j.oauth.consumerSecret="XYZ" -Dtwitter4j.oauth.accessToken="XYZ" -Dtwitter4j.oauth.accessTokenSecret="XYZ"

Program arguments: -p 8008 -n "twittersample app" -cv "1.0"

Working directory: $MODULE_DIR$

Environment variables:

JETSTREAM_HOME = PATH_TO/realtime-analytics/Demo/twittersample/buildsrc

MONGO_HOME = mongo://mongoserver:27017/config

Use classpath of module: twittersample