Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENG-3289: Row Operations metrics for mysql based integrations by using performance_schema #89

Merged
merged 5 commits into from
Jul 29, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,9 @@ module github.com/open-telemetry/opentelemetry-collector-contrib
// For the OpenTelemetry Collector Contrib distribution specifically, see
// https://github.com/open-telemetry/opentelemetry-collector-releases/tree/main/distributions/otelcol-contrib

go 1.21.0
go 1.22.2

toolchain go1.22.5

require (
github.com/open-telemetry/opentelemetry-collector-contrib/connector/countconnector v0.102.0
Expand Down Expand Up @@ -511,6 +513,7 @@ require (
github.com/jpillora/backoff v1.0.0 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/julienschmidt/httprouter v1.3.0 // indirect
github.com/k0kubun/pp v3.0.1+incompatible // indirect
github.com/kardianos/osext v0.0.0-20190222173326-2bc1f35cddc0 // indirect
github.com/karrick/godirwalk v1.17.0 // indirect
github.com/klauspost/compress v1.17.8 // indirect
Expand Down Expand Up @@ -539,6 +542,7 @@ require (
github.com/mattn/go-sqlite3 v1.14.22 // indirect
github.com/microsoft/ApplicationInsights-Go v0.4.4 // indirect
github.com/microsoft/go-mssqldb v1.7.2 // indirect
github.com/middleware-labs/innoParser v0.0.0-20240508090457-8c2fa2246395 // indirect
github.com/miekg/dns v1.1.58 // indirect
github.com/mistifyio/go-zfs v2.1.2-0.20190413222219-f784269be439+incompatible // indirect
github.com/mitchellh/copystructure v1.2.0 // indirect
Expand Down
6 changes: 6 additions & 0 deletions go.sum

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

51 changes: 51 additions & 0 deletions receiver/mysqlreceiver/client.go
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ type client interface {
getInnodbStatusStats() (map[string]int64, error, int)
getTotalRows() ([]NRows, error)
getTotalErrors() (int64, error)
getRowOperationStats() (RowOperationStats, error)
Close() error
}

Expand All @@ -40,6 +41,12 @@ type mySQLClient struct {
statementEventsTimeLimit time.Duration
}

type RowOperationStats struct {
rowsRead int64
rowsUpdated int64
rowsDeleted int64
rowsInserted int64
}
type IoWaitsStats struct {
schema string
name string
Expand Down Expand Up @@ -233,6 +240,50 @@ func (c *mySQLClient) getVersion() (string, error) {
return version, nil
}

func (c *mySQLClient) getRowOperationStats() (RowOperationStats, error) {
// TODO: Improve this logic for complex queries. Cases where INSERT/UPDATE/READ/DELETES are a part of a sub-operation.
query := "SELECT SUBSTRING_INDEX(DIGEST_TEXT, ' ', 1) AS statement_type, " +
"SUM(SUM_ROWS_AFFECTED) AS rows_affected, " +
"SUM(SUM_ROWS_SENT) AS rows_sent " +
"FROM performance_schema.events_statements_summary_by_digest " +
"WHERE DIGEST_TEXT LIKE 'SELECT% '" +
"OR DIGEST_TEXT LIKE 'INSERT%' " +
"OR DIGEST_TEXT LIKE 'UPDATE%' " +
"OR DIGEST_TEXT LIKE 'DELETE%' " +
"GROUP BY statement_type; "

rows, err := c.client.Query(query)
rowOpsStats := new(RowOperationStats)

if err != nil {
return *rowOpsStats, err
}

defer rows.Close()

for rows.Next() {
var rowsAffected int64
var rowsSent int64
var statementType string
err := rows.Scan(&statementType, &rowsAffected, &rowsSent)

if err != nil {
return *rowOpsStats, err
}

if statementType == "SELECT" {
rowOpsStats.rowsRead = rowsSent
} else if statementType == "UPDATE" {
rowOpsStats.rowsUpdated = rowsAffected
} else if statementType == "DELETE" {
rowOpsStats.rowsDeleted = rowsAffected
} else if statementType == "INSERT" {
rowOpsStats.rowsInserted = rowsAffected
}
}
return *rowOpsStats, nil
}

// getGlobalStats queries the db for global status metrics.
func (c *mySQLClient) getGlobalStats() (map[string]string, error) {
q := "SHOW GLOBAL STATUS;"
Expand Down
32 changes: 32 additions & 0 deletions receiver/mysqlreceiver/documentation.md
Original file line number Diff line number Diff line change
Expand Up @@ -286,6 +286,38 @@ The number of InnoDB page operations.
| ---- | ----------- | ------ |
| operation | The page operation types. | Str: ``created``, ``read``, ``written`` |

### mysql.performance.rows_deleted

The number of rows deleted in the database as per the performance schema.

| Unit | Metric Type | Value Type | Aggregation Temporality | Monotonic |
| ---- | ----------- | ---------- | ----------------------- | --------- |
| {row} | Sum | Int | Cumulative | true |

### mysql.performance.rows_inserted

The number of rows inserted in the database as per the performance schema.

| Unit | Metric Type | Value Type | Aggregation Temporality | Monotonic |
| ---- | ----------- | ---------- | ----------------------- | --------- |
| {row} | Sum | Int | Cumulative | true |

### mysql.performance.rows_read

The number of rows read in the database as per the performance schema.

| Unit | Metric Type | Value Type | Aggregation Temporality | Monotonic |
| ---- | ----------- | ---------- | ----------------------- | --------- |
| {row} | Sum | Int | Cumulative | true |

### mysql.performance.rows_updated

The number of rows updated in the database as per the performance schema.

| Unit | Metric Type | Value Type | Aggregation Temporality | Monotonic |
| ---- | ----------- | ---------- | ----------------------- | --------- |
| {row} | Sum | Int | Cumulative | true |

### mysql.prepared_statements

The number of times each type of prepared statement command has been issued.
Expand Down
2 changes: 1 addition & 1 deletion receiver/mysqlreceiver/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ toolchain go1.22.4
require (
github.com/go-sql-driver/mysql v1.8.1
github.com/google/go-cmp v0.6.0
github.com/middleware-labs/innoParser v0.0.0-20240508090457-8c2fa2246395
github.com/middleware-labs/innoParser v0.0.0-20240729092319-ddbdd8e42266
github.com/open-telemetry/opentelemetry-collector-contrib/internal/coreinternal v0.102.0
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/golden v0.102.0
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/pdatatest v0.102.0
Expand Down
4 changes: 2 additions & 2 deletions receiver/mysqlreceiver/go.sum

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

16 changes: 16 additions & 0 deletions receiver/mysqlreceiver/internal/metadata/generated_config.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading