Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

api: new export format #436

Closed
wants to merge 10 commits into from

Conversation

DifferentialOrange
Copy link
Member

@DifferentialOrange DifferentialOrange commented Feb 15, 2023

This PR introduces new exhaustive export format on collect. It contains structured info on collectors which may be used on export and aggregation build. (Current metrics.collect() result is not enough on Prometheus export. metrics.collect{extended_format = true} is enough.) After this PR, export plugins will be based on extended format info.

See commit messages for more detailed info.

I didn't forget about

  • Tests
  • Changelog
  • Documentation (README and rst)
  • Rockspec and rpm spec

Part of tarantool/tarantool#7725
Part of tarantool/tarantool#7728

@DifferentialOrange DifferentialOrange changed the title New export format api: new export format Feb 15, 2023
@DifferentialOrange DifferentialOrange force-pushed the DifferentialOrange/new-export-format branch from 2c25cc0 to b2f41ee Compare February 15, 2023 09:14
@github-actions github-actions bot temporarily deployed to translation-new-export-format February 15, 2023 09:16 Inactive
@github-actions github-actions bot temporarily deployed to translation-new-export-format February 15, 2023 09:22 Inactive
@DifferentialOrange DifferentialOrange force-pushed the DifferentialOrange/new-export-format branch from b2f41ee to 97c29d8 Compare February 15, 2023 09:43
@github-actions github-actions bot temporarily deployed to translation-new-export-format February 15, 2023 09:53 Inactive
@github-actions github-actions bot temporarily deployed to translation-new-export-format February 15, 2023 17:03 Inactive
@DifferentialOrange DifferentialOrange force-pushed the DifferentialOrange/new-export-format branch from cfcc861 to 80db8c6 Compare February 15, 2023 17:08
@github-actions github-actions bot temporarily deployed to translation-new-export-format February 15, 2023 17:08 Inactive
@DifferentialOrange DifferentialOrange force-pushed the DifferentialOrange/metrics-cfg branch from 929f457 to d892e7d Compare February 16, 2023 14:06
Base automatically changed from DifferentialOrange/metrics-cfg to master February 16, 2023 15:03
Deprecate using control characters (including tabulation) in collector
kind, name, observation and global labels. Using such symbols may result
in registry collision (some observations may be lost), as well as
corrupted plugin output.

Part of #435, part of tarantool/tarantool#7725
Collector name prefixes may be used to build names for metric
derivatives (average, min, max etc.). For now the rule is as follows:
name prefix is the name stripped from the "_total" and "_current"
suffixes. (If they presents together at the same time, the name is
already messed up, so we don't cover this.)

Part of tarantool/tarantool#7725
Part of tarantool/tarantool#7728
Support exhaustive output format for shared-based, counter and gauge
collectors collect. It could later be used for export and aggregating
without using any other sources of information, (like collectors data,
as it used now in Prometheus export).

Part of tarantool/tarantool#7725
Part of tarantool/tarantool#7728
Support exhaustive output format for histogram collect. Observations are
grouped into sum, count and bucket observations. sum and count
observations use the same set of keys. It could later be used for export
and aggregating without using any other sources of information, (like
collectors data, as it used now in Prometheus export).

Part of tarantool/tarantool#7725
Part of tarantool/tarantool#7728
Support exhaustive output format for summary collect. Observations are
grouped into sum, count and bucket observations. sum and count
observations use the same set of keys. It could later be used for export
and aggregating without using any other sources of information, (like
collectors data, as it used now in Prometheus export).

Part of tarantool/tarantool#7725
Part of tarantool/tarantool#7728
Support exhaustive output format for metrics.collect(). It could later
be used for export and aggregating without using any other sources of
information (like collectors data, as it used now in Prometheus export).

Part of tarantool/tarantool#7725
Part of tarantool/tarantool#7728
After this patch, prometheus export plugin is based on
metrics.collect{invoke_callbacks = true, extended_format = true} result.
Collectors registry info is not required anymore.

Process API is also separated, so now a developer can reuse export
handles if they already have an output (for example, processing
historical data in Flight Recorder).

Older version was preserved for test purposes to ensure that output is
the same in new implementation.

Part of tarantool/tarantool#7725
Part of tarantool/tarantool#7728
After this patch, json export plugin is based on
metrics.collect{invoke_callbacks = true, extended_format = true} result.

Process API is also separated, so now a developer can reuse export
handles if they already have an output (for example, processing
historical data in Flight Recorder).

Older version was preserved for test purposes to ensure that output is
the same in new implementation.

Part of tarantool/tarantool#7725
Part of tarantool/tarantool#7728
After this patch, graphite export plugin is based on
metrics.collect{invoke_callbacks = true, extended_format = true} result.

Process API is also separated, so now a developer can reuse export
handles if he already has an output (for example, processing historical
data in Flight Recorder).

Older version was preserved for test purposes to ensure that output is
the same in new implementation.

Part of tarantool/tarantool#7725
Part of tarantool/tarantool#7728
Advice users to iterate through extensive
metrics.collect{extended_format = true} output instead of collectors.

Part of tarantool/tarantool#7725
Part of tarantool/tarantool#7728
@DifferentialOrange DifferentialOrange force-pushed the DifferentialOrange/new-export-format branch from 80db8c6 to cc08eec Compare February 16, 2023 15:47
@github-actions github-actions bot temporarily deployed to translation-new-export-format February 16, 2023 15:52 Inactive
@github-actions github-actions bot temporarily deployed to translation-new-export-format February 16, 2023 15:56 Inactive
@DifferentialOrange DifferentialOrange force-pushed the DifferentialOrange/new-export-format branch from 545617f to cc08eec Compare February 16, 2023 16:07
@github-actions
Copy link

This pull request is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 30 days

@github-actions github-actions bot added the Stale label Apr 17, 2023
@DifferentialOrange
Copy link
Member Author

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant