Publish Shard State Metrics

[amathur1893]

Fixes #:opendistro-for-elasticsearch/performance-analyzer#213

Description of changes:
This change will start publishing shard state - Initializing, Unassigned and Relocating for each shard. We have not published Active shards to save space.

Testing

1. ./gradlew test

SUCCESS: Executed 455 tests in 41m 55s (1 skipped)

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.5/userguide/command_line_interface.html#sec:command_line_warnings

BUILD SUCCESSFUL in 42m 22s
11 actionable tasks: 2 executed, 9 up-to-date

2. Tested using docker.

Tmp File -

^shard_state_metrics
{"current_time":1602500493586}
{"IndexName":pmc}
{"ShardID":2,"ShardType":"p","NodeName":"elasticsearch2","Shard_State":"UNASSIGNED"}
{"ShardID":4,"ShardType":"p","NodeName":"elasticsearch2","Shard_State":"INITIALIZING"}
{"IndexName":pmc1}
{"ShardID":1,"ShardType":"p","NodeName":"elasticsearch1","Shard_State":"UNASSIGNED"}
{"ShardID":0,"ShardType":"p","NodeName":"elasticsearch2","Shard_State":"UNASSIGNED"}
$

Table created

sqlite> .tables
Shard_State

Content of the table

sqlite> select * from Shard_State;
pmc|4|p|elasticsearch1|UNASSIGNED|1.0|1.0|1.0|1.0
pmc|2|p|elasticsearch2|INITIALIZING|1.0|1.0|1.0|1.0
pmc1|1|p|elasticsearch2|UNASSIGNED|1.0|1.0|1.0|1.0
pmc1|0|p|elasticsearch1|UNASSIGNED|1.0|1.0|1.0|1.0

Rest API

arpitamt@c4b301d5bc1f ~> curl -XGET "localhost:9600/_opendistro/_performanceanalyzer/metrics?metrics=Shard_State&agg=avg&dim=Shard_State&nodes=all" | python -m json.tool
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   188  100   188    0     0     22      0  0:00:08  0:00:08 --:--:--    45
{
    "id3qQArxRPSA2EHbEfPlew": {
        "timestamp": 1602771645000,
        "data": {
            "fields": [
                {
                    "name": "Shard_State",
                    "type": "VARCHAR"
                },
                {
                    "name": "Shard_State",
                    "type": "DOUBLE"
                }
            ],
            "records": [
                [
                    "UNASSIGNED",
                    0.0
                ]
            ]
        }
    }
}

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[yojs]

can we change this to a metric ?

[khushbr]

+1, it would be good to have a "timetook" metric to indicate the time taken for metric collection.

[amathur1893]

Already made the change in PA PR - opendistro-for-elasticsearch/performance-analyzer#212

[khushbr]

+1, it would be good to have a "timetook" metric to indicate the time taken for metric collection.