Add MySQL query metricset with lightweight module and SQL helper (#18955

)

CHANGELOG-developer.next.asciidoc

@@ -95,4 +95,5 @@ The list below covers the major changes between 7.0.0-rc2 and master only.
 - Add IP* fields to `fields.yml` generator script in Filebeat. {issue}17998[17998] {pull}18256[18256]
 - Events intended for the Elasticsearch output can now take an `op_type` metadata field of type events.OpType or string to indicate the `op_type` to use for bulk indexing. {pull}12606[12606]
 - Remove vendor folder from repository. {pull}18655[18655]
+- Added SQL helper that can be used from any Metricbeat module {pull}18955[18955]
 - Update Go version to 1.14.4. {pull}19753[19753]

CHANGELOG.next.asciidoc

@@ -270,6 +270,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
 - Fix config example in the perfmon configuration files. {pull}19539[19539]
 - Add missing info about the rest of the azure metricsets in the documentation. {pull}19601[19601]
 - Fix k8s scheduler compatibility issue. {pull}19699[19699]
+- Fix SQL module mapping NULL values as string {pull}18955[18955] {issue}18898[18898
 
 *Packetbeat*
 
@@ -541,6 +542,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
 - Add param `aws_partition` to support aws-cn, aws-us-gov regions. {issue}18850[18850] {pull}19423[19423]
 - Add support for wildcard `*` in dimension value of AWS CloudWatch metrics config. {issue}18050[18050] {pull}19660[19660]
 - The `elasticsearch/index` metricset now collects metrics for hidden indices as well. {issue}18639[18639] {pull}18703[18703]
+- Added `performance` and `query` metricsets to `mysql` module. {pull}18955[18955]
 - The `elasticsearch-xpack/index` metricset now reports hidden indices as such. {issue}18639[18639] {pull}18706[18706]
 
 *Packetbeat*

metricbeat/docs/fields.asciidoc

@@ -32034,6 +32034,123 @@ type: long
 
 --
 
+[float]
+=== performance
+
+`performance` contains metrics related to the performance of a MySQL instance
+
+
+
+[float]
+=== events_statements
+
+Records statement events summarized by schema and digest
+
+
+*`mysql.performance.events_statements.max.timer.wait`*::
++
+--
+Maximum wait time of the summarized events that are timed
+
+type: long
+
+--
+
+*`mysql.performance.events_statements.last.seen`*::
++
+--
+Time at which the digest was most recently seen
+
+type: date
+
+--
+
+*`mysql.performance.events_statements.quantile.95`*::
++
+--
+The 95th percentile of the statement latency, in picoseconds
+
+type: long
+
+--
+
+*`mysql.performance.events_statements.digest`*::
++
+--
+Performance schema digest
+
+type: text
+
+--
+
+*`mysql.performance.events_statements.count.star`*::
++
+--
+Number of summarized events
+
+type: long
+
+--
+
+*`mysql.performance.events_statements.avg.timer.wait`*::
++
+--
+Average wait time of the summarized events that are timed
+
+type: long
+
+--
+
+[float]
+=== table_io_waits
+
+Records table I/O waits by index
+
+
+
+*`mysql.performance.table_io_waits.object.schema`*::
++
+--
+Schema name
+
+type: keyword
+
+--
+
+*`mysql.performance.table_io_waits.object.name`*::
++
+--
+Table name
+
+type: keyword
+
+--
+
+*`mysql.performance.table_io_waits.index.name`*::
++
+--
+Name of the index that was used when the table I/O wait event was recorded. PRIMARY indicates that table I/O used the primary index. NULL means that table I/O used no index. Inserts are counted against INDEX_NAME = NULL
+
+
+type: keyword
+
+--
+
+*`mysql.performance.table_io_waits.count.fetch`*::
++
+--
+Number of all fetch operations > 0
+
+type: long
+
+--
+
+[float]
+=== query
+
+`query` metricset fetches custom queries from the user to a MySQL instance.
+
+
 [float]
 === status
 
@@ -37141,6 +37258,16 @@ type: object
 Non-numeric values collected.
 
 
+type: object
+
+--
+
+*`sql.metrics.boolean.*`*::
++
+--
+Boolean values collected.
+
+
 type: object
 
 --

metricbeat/docs/modules/mysql.asciidoc

@@ -86,9 +86,17 @@ The following metricsets are available:
 
 * <<metricbeat-metricset-mysql-galera_status,galera_status>>
 
+* <<metricbeat-metricset-mysql-performance,performance>>
+
+* <<metricbeat-metricset-mysql-query,query>>
+
 * <<metricbeat-metricset-mysql-status,status>>
 
 include::mysql/galera_status.asciidoc[]
 
+include::mysql/performance.asciidoc[]
+
+include::mysql/query.asciidoc[]
+
 include::mysql/status.asciidoc[]
 

metricbeat/docs/modules/mysql/performance.asciidoc

@@ -0,0 +1,24 @@
+////
+This file is generated! See scripts/mage/docs_collector.go
+////
+
+[[metricbeat-metricset-mysql-performance]]
+=== MySQL performance metricset
+
+beta[]
+
+include::../../../module/mysql/performance/_meta/docs.asciidoc[]
+
+This is a default metricset. If the host module is unconfigured, this metricset is enabled by default.
+
+==== Fields
+
+For a description of each field in the metricset, see the
+<<exported-fields-mysql,exported fields>> section.
+
+Here is an example document generated by this metricset:
+
+[source,json]
+----
+include::../../../module/mysql/performance/_meta/data.json[]
+----

metricbeat/docs/modules/mysql/query.asciidoc

@@ -0,0 +1,18 @@
+////
+This file is generated! See scripts/mage/docs_collector.go
+////
+
+[[metricbeat-metricset-mysql-query]]
+=== MySQL query metricset
+
+beta[]
+
+include::../../../module/mysql/query/_meta/docs.asciidoc[]
+
+This is a default metricset. If the host module is unconfigured, this metricset is enabled by default.
+
+==== Fields
+
+For a description of each field in the metricset, see the
+<<exported-fields-mysql,exported fields>> section.
+