Skip to content

Latest commit

 

History

History
125 lines (93 loc) · 4.45 KB

README.md

File metadata and controls

125 lines (93 loc) · 4.45 KB
Raw

Elasticsearch Stress Tool

Elasticsearch Stress Tool is a tool to check Elasticsearch configuration, the underlying hardware.

It can be used to compare:

  • hardware or cloud solutions,
  • sharding or replication settings,
  • mappings and analyzers,
  • HTTP and Transport protocols,
  • different Elasticsearch versions.

It's inspired by Cassandra Stress Tool. It's not a full bleed benchmarking tool, you'd be better user JMeter, Gatling, Loader.io to build real benchmarks.

Features

  • Generate documents/queries from mixing data from a CSV file with document/queries based on Mustache
  • Index many documents in an index or query an index with varying queries
  • Iterate many times with many threads
  • Measure execution times

Indexing documents

esstresstool index -h localhost:9300 -c my_cluster \ 
  -di my_index -dt my_type \
  -dd doc_data.csv -dm doc_template.mustache \
  -t 4 -n 1000000

Querying documents

esstresstool search -h localhost:9300 -c my_cluster \ 
  -di my_index -dt my_type \
  -qd query_data.csv -qm query_template.mustache \
  -t 4 -n 1000000

Options

Common options

Option Description Default
--help Print help
-x, --protocol Protocol, either node, transport, jest, http transport
-h, --host Hosts and ports localhost port depends on transport
-c, --cluster Cluster name
-t, --thread Thread number Number of CPUs
-sp, --start-period-ms Period in ms between each thread start
-ep, --execute-period-ms Period in ms between each execution
-n, --iterations Number of iterations of each thread 10000
-di, --doc-index, --index Default Index name .stresstest
-dt, --doc-type Default Document type stress
-dd, --doc-data, -qd, --query-data Document/Query data CSV file
-dm, --doc-template, -qm, --query-template Document/Query Mustache file
-mc, --metric-console Output Console for metric reporting false
-mo, --metric-output Output File for metric reporting, ending either .csv or .json
-mp, --metric-period-ms Period in second for metric reporting 10000

Index options

Option Description Default
-b, --bulk-size Bulk size, 1 to disable bulk
-did, --doc-index-delete Index delete at startup false
-dis, --doc-index-settings, --index-settings Index settings for creation at startup

Before indexing data, you can delete the index and/or create the index with specified settings.

Search options

Nothing at the moment

Files

Data File: CSV

This file is used to produce data which can be indexed or used as query parameters. It will be read many times if the number of iterations is bigger than the number of lines.

A comma separated (CSV) file with headers is expected. The header are used as field names and can be referenced in Mustache templates. This file will be read multiple times to get the expected iteration number.

There are some special column names:

Name Description
index Target index, overrides command line -di option
type Document type, overrides command line -dt option
id Document Id

There is also a special Data File called names which references Marvel super heroes.

Template File: Mustache

This file is used to convert data in JSON: the document to index or the query to execute. If no template is specified, then the document to index is built from data using CSV headers.

You can use in the Document/Query template any field provided.

There are some special fields:

Name Description
docIndex Target index
docType Document type
docId Document Id, if read from CSV
docNumber Document index between 0 and thread×iterations×bulk size
timestamp Timestamp in milliseconds
random.boolean Random boolean
random.int Random integer
random.long Random long
random.float Random float
random.double Random double

Metrics file: CSV or JSON

Response times are measured using Yammer/Codahale/Dropwizard Metrics library. Then they can be periodically written to logs, to a CSV or JSON file. The JSON file format can easily be imported into Elasticsearch with Logstash and used to build charts with Kibana.