Skip to content

Home

Jump to bottom
syamp edited this page Sep 28, 2016 · 21 revisions

See the Tutorial for an Overview of MQE

Introduction

Metrics Query Engine(MQE) provides SQL-like interface to time series data with powerful functions.

For example, to find which 10 endpoints have the highest HTTP latency on your web application farm:

select connection.http.latency
| aggregate.sum(group by endpoint)
| filter.highest_mean(10)
where application = 'httpd'
from -2hr to now

In MQE terminology connection.http.latency is the metric with tags endpoint and application.

Infrastructure

MQE currently works best with Blueflood (based on cassandra) to store metric data. MQE provides an indexing system which is backed by Cassandra to automatically de-serialize dot-separated names.

MQE theoretically can work with any time series data store with support for tags. Please see demo folder for example implementation. Better documentation coming soon!

Setup

This tutorial will help you set up MQE on a local machine, along with Cassandra and Blueflood instances.

Cassandra
  • cd ~ # store at root; other commands will assume Cassandra is located here
  • wget http://archive.apache.org/dist/cassandra/2.1.13/apache-cassandra-2.1.13-bin.tar.gz
  • tar zxvf apache-cassandra-2.1.13-bin.tar.gz
  • cd apache-cassandra-2.1.13 && ./bin/cassandra # starts cassandra on port 7200 locally

More detailed instructions on http://wiki.apache.org/cassandra/GettingStarted

Blueflood
  • cd ~
  • git clone https://github.com/rackerlabs/blueflood.git
  • cd blueflood
  • mvn package -P all-modules # generates "uber" JAR with all dependencies

Next, set up blueflood.conf with (HTTP) query and ingestion only:

cd ~/blueflood
cat >blueflood.conf <<BLUEFLOODCONF
# default cassandra
CASSANDRA_HOSTS=127.0.0.1:9160
DEFAULT_CASSANDRA_PORT=9160
ROLLUP_KEYSPACE=DATA
# http for ingestion/query
INGESTION_MODULES=com.rackspacecloud.blueflood.service.HttpIngestionService
QUERY_MODULES=com.rackspacecloud.blueflood.service.HttpQueryService
ZOOKEEPER_CLUSTER=NONE
# allow ingestions and queries
INGEST_MODE=TRUE
QUERY_MODE=TRUE
# no rollups yet
ROLLUP_MODE=FALSE
# ingest on port 1776
HTTP_INGESTION_PORT=1776
HTTP_INGESTION_HOST=localhost
# query on port 1777
HTTP_METRIC_DATA_QUERY_PORT=1777
HTTP_QUERY_HOST=localhost
BLUEFLOODCONF
  • Configure logging for blueflood
cd ~/blueflood
cat >blueflood-log4j.properties <<BLUEFLOODLOG
log4j.appender.console=org.apache.log4j.ConsoleAppender
log4j.appender.console.layout=org.apache.log4j.PatternLayout
log4j.appender.console.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %-5p %-20.20c{1}:%-3L - %m%n
log4j.logger.com.rackspacecloud.blueflood=DEBUG
log4j.logger.httpclient.wire.header=WARN
log4j.logger.httpclient.wire.content=WARN
log4j.logger.org.apache.http.client.protocol=INFO
log4j.logger.org.apache.http.wire=INFO
log4j.logger.org.apache.http.impl=INFO
log4j.logger.org.apache.http.headers=INFO
log4j.rootLogger=INFO, console
BLUEFLOODLOG
  • Setup blueflood schema in cassandra
~/apache-cassandra-2.1.13/bin/cqlsh 127.0.0.1 -f ./src/cassandra/cli/load.cdl
  • Start blueflood
/usr/bin/java -Dblueflood.config=file://$HOME/blueflood/blueflood.conf -Dlog4j.configuration=file://$HOME/blueflood/blueflood-log4j.properties -Xms1G -Xmx1G -classpath $HOME/blueflood/blueflood-all/target/blueflood-all-2.0.0-SNAPSHOT-jar-with-dependencies.jar  com.rackspacecloud.blueflood.service.BluefloodServiceStarter
MQE

  • install go

  • go get -u -v github.com/square/metrics/... # fetches and installs dependencies too
  • cd $GOPATH/src/github.com/square/metrics
  • ~/apache-cassandra-2.1.13/bin/cqlsh 127.0.0.1 -f schema/schema.cql
Demo
  • Indexer - that converts blueflood names into tags based on specified rules
cd $GOPATH/src/github.com/square/metrics
go build ./demo/ingest/...
./ingest -cassandra-host 127.0.0.1 -listen-on 7774 -rule-path demo/conversion_rules
  • Emit - emit sample metrics for demo
cd $GOPATH/src/github.com/square/metrics
go build ./demo/emit/...
./emit -blueflood-address "http://localhost:1776/v2.0/example-tenant/ingest" -mqe-ingestion-address "http://127.0.0.1:7774/ingest"
  • Start UI
cd $GOPATH/src/github.com/square/metrics
go run ./main/ui/ui.go  -config-file ./demo/example-config.yaml

Then head to http://localhost:9007 (specified in example-config.yaml) and you'll get the MQE UI.

Run a describe all query to get a list of all (3) tagged metric names.

  • cpu.percentage
  • connection.http.latency
  • connection.http.count

Each of these metrics corresponds to dozens of time-series: one for each app, and for each host.

Run describe connection.http.latency to get a list of all the time-series associated with this metric name.

Now, run select connection.http.latency from -30m to now. You'll see a graph of these time-series for the past 30 minutes.

Clone this wiki locally