Add support for Index Statistics metrics #10784
Conversation
…collect index metrics
ghost
added
Co-authored-by: Andrew Zhang <[email protected]>
| return formatted_tags | ||
|
|
||
| def _submit_index_node_metrics(self, mname, mval, tags): | ||
| index_state_map = {'Active': 0, 'Pause': 1, 'Warmup': 2} |
There was a problem hiding this comment.
this is constant, so let's move it out of the function
There was a problem hiding this comment.
Added it to couchbase_consts.py and importing it.
| # Error handling in case Couchbase changes their versioning format | ||
| if self._version and int(self._version.split(".")[0]) < 7: | ||
| self.log.warning( | ||
| "Index Stats Metrics are only available in Couchbase version 7+. Detected version: %", |
There was a problem hiding this comment.
Since this option is behind a flag that must be enabled by the user and we properly documented the requirement, I don't know if we need this warning log. We could consider just validating the version in check() before we call _collect_index_stats_metrics, and if version is incompatible, avoid collecting completely versus running into a requests error and log a debug line.
Elastic and postgres have examples of checking versions before collecting a subset of metrics
There was a problem hiding this comment.
I think I initially approached it in a way that we should try to get the index stats anyways in case the version logic would fail or if for whatever reason Couchbase decides to change the way they expose their version.
I moved the logic to check for index_stats_url and version and version >= 7 into check(). I left the try should should the int(self._version.split(".")[0]) not return an integer for whatever reason.
| ] | ||
| return formatted_tags | ||
|
|
||
| def _submit_index_node_metrics(self, mname, mval, tags): |
There was a problem hiding this comment.
| def _submit_index_node_metrics(self, mname, mval, tags): | |
| def _submit_index_node_metrics(self, mname, mval): |
this function is only used once and passes self.tags which can be called directly
There was a problem hiding this comment.
Make sense. I added your change and attach tags via self._tags in the submit function.
| data = self._get_stats(url) | ||
| except requests.exceptions.RequestException as e: | ||
| msg = "Error accessing the Index Statistics endpoint: %s: %s" % (url, str(e)) | ||
| self.log.warning(str(e)) |
There was a problem hiding this comment.
| self.log.warning(str(e)) | |
| self.log.warning(msg) |
* added couchbase 7 with index stats endpoint as a test environment * Initial metric parsing for index stats * updated config model and spec to include the new parameter needed to collect index metrics * styling fix * more descriptive service check message * integration test for index stats metric collection * slight logic refactor * added units test and refactor metric parsing logic * smaller sample bucket for testing * add new metrics to metadata.csv * add comments for documentations and touch ups on tests * fix typo * lower timeout on tests * attempt at fixing flaky test * fix metric descriptions and config param description * style fix * removal of log line and minor description change * incorporated changes from reviews * switch unit test to use parameterize version * change tests and small refactor to tag parsing logic * change few metrics to count metrics * styling * add one unit test for tag extraction * styling * add missing unit for metric * small nits and metadata edits * styling * small tweak in test * warn log for index stats requirement not met * typo in comments * grammaticaal fix and error handling for version * add more error handling * remove unused import * logic error fix * Better version check logic in test Co-authored-by: Andrew Zhang <[email protected]> * small refactor * refactor * slight change to log message Co-authored-by: Andrew Zhang <[email protected]>
What does this PR do?
Add support for the collection of metrics from the Couchbase Index Statistics API that was released in version 7 of Couchbase
Motivation
Feature Request
Additional Notes
https://docs.couchbase.com/server/current/rest-api/rest-index-stats.html
Review checklist (to be filled by reviewers)
changelog/andintegration/labels attached