[Metricbeat] Add billing metricset into googlecloud module #20812
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
botelastic
bot
added
the
needs_team
kaiyan-sheng
added
in progress
|
Pinging @elastic/integrations-platforms (Team:Platforms) |
botelastic
bot
removed
the
needs_team
💔 Tests Failed
Expand to view the summary
Build stats
Test stats 🧪
Test errorsExpand to view the tests failures
Steps errorsExpand to view the steps failures
Log outputExpand to view the last 100 lines of log output
|
exekias
suggested changes
Sep 1, 2020
| "googlecloud": { | ||
| "billing": { | ||
| "cost_type": "regular", | ||
| "invoice_month": "202008", |
There was a problem hiding this comment.
Would it make sense to convert this to a timestamp?
There was a problem hiding this comment.
Hmm I think we should keep it as a string/keyword to match invoice.month in billing table schema from BigQuery: https://cloud.google.com/billing/docs/how-to/export-data-bigquery-tables#data-schema
exekias
approved these changes
Sep 2, 2020
|
CI failed with |
6 tasks
kaiyan-sheng
added
v7.10.0
and removed
needs_backport
v1v
added a commit
to v1v/beats
that referenced
this pull request
Sep 2, 2020
…ne-2.0 * upstream/master: (87 commits) [packaging] Normalise GCP bucket folder structure (elastic#20903) [Metricbeat] Add billing metricset into googlecloud module (elastic#20812) Include python docs in devguide index (elastic#20917) Avoid generating incomplete configurations in autodiscover (elastic#20898) Improve docs of leaderelection configuration (elastic#20601) Document how to set the ES host and Kibana URLs in Ingest Manager (elastic#20874) docs: Update beats for APM (elastic#20881) Adding cborbeat to community beats (elastic#20884) Bump kibana version to 7.9.0 in x-pack/metricbeat (elastic#20899) Kubernetes state_daemonset metricset for Metricbeat (elastic#20649) [Filebeat][zeek] Add new x509 fields to zeek (elastic#20867) [Filebeat][Gsuite] Add note about admin in gsuite docs (elastic#20855) Ensure kind cluster has RFC1123 compliant name (elastic#20627) Setup python paths in test runner configuration (elastic#20832) docs: Add `processor.event` info to Logstash output (elastic#20721) docs: update cipher suites (elastic#20697) [ECS] Update ecs to 1.6.0 (elastic#20792) Fix path in hits docs (elastic#20447) Update filebeat azure module documentation (elastic#20815) Remove duplicate ListGroupsForUsers in aws/cloudtrail (elastic#20788) ...
kaiyan-sheng
added a commit
that referenced
this pull request
Sep 4, 2020
melchiormoulin
pushed a commit
to melchiormoulin/beats
that referenced
this pull request
Oct 14, 2020
…0812) * Add billing metricset to googlecloud module
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
This PR is to add billing metricset into googlecloud module. For GCP, there is no direct billing API like AWS Cost Explorer to query for billing information. In order to get the billing date, users need to export billing reports into GCP BigQuery database and then this billing metricset can make some queries to get the actual billing data out.
Why is it important?
This will make GCP users' life a lot easier when tracking billing data 😄
Checklist
CHANGELOG.next.asciidocorCHANGELOG-developer.next.asciidoc.How to test this PR locally
It took me a while to get permissions for GCP billing account. Not sure how this will be tested in a short period of time. The config I used for billing metricset is:
Related issues
Screenshots