Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[receiver/googlecloudmonitoringreceiver] Transform GCP Timeseries Data to OTel Metrics Conversion Pipeline #34677

Conversation

abhishek-at-cloudwerx
Copy link
Contributor

Description: Transform GCP Timeseries Data to OTel Metrics Pipeline by establishing a client connection to the Google Cloud Monitoring API. This PR scrapes timeseries data based on configured metrics and converts it into the OpenTelemetry format for pipeline processing.

Link to tracking Issue: #33762

Testing: No additional test cases were added. Existing tests cover the necessary functionality.

Documentation: The README.md file was previously added and already includes relevant details about the component and the conversion process.

- Create client connection for monitoring REST API
- Implement logic for fetch metrics data and convert it into pipeline data
- Auto import
- Fix fail check issues
- Add cases for metrics value in SUM and DELTA metrics conversion functions
- Refactor code based on PR comments
- Fix test-case issue by adding service.json file on testdata folder
- Setup env variable for GCP service file
- Remove retry logic and sync.Once for client connection
- Fix lint issue for googlecloudmonitoring receiver component
- Add function to fetch a list of metric descriptors
- Call function to convert time series metrics data once
- Apply zap logger instead of default log package
- Make changes
- Fix go.sum file changes
- Skip authentication setup test cases
- Separate functions for initializing the monitoring client and descriptor API
@dashpole dashpole added the enhancement New feature or request label Sep 4, 2024
@dashpole
Copy link
Contributor

dashpole commented Sep 4, 2024

@TylerHelmuth this lgtm if you want to take a look

- Resolved issue with metric descriptor retrieval
- Fixed monitoring client connection problem
@dashpole
Copy link
Contributor

dashpole commented Sep 5, 2024

One other piece of feedback from the Google cloud monitoring team: We should make sure we never query for data more than 24h in the past, as that can cause performance issues for the API. We should either validate that the interval configuration option is less than ~23 hours, or just enforce a max of ~23 hours when we calculate the interval.

- Remove fetch_delay from config file
- Set default collection interval value and max value of 23 hours
- Changes based on PR
- Fix config test cases
@TylerHelmuth TylerHelmuth merged commit 7ec6396 into open-telemetry:main Sep 5, 2024
156 checks passed
@github-actions github-actions bot added this to the next release milestone Sep 5, 2024
f7o pushed a commit to f7o/opentelemetry-collector-contrib that referenced this pull request Sep 12, 2024
…a to OTel Metrics Conversion Pipeline (open-telemetry#34677)

**Description:** Transform GCP Timeseries Data to OTel Metrics Pipeline
by establishing a client connection to the Google Cloud Monitoring API.
This PR scrapes timeseries data based on configured metrics and converts
it into the OpenTelemetry format for pipeline processing.

**Link to tracking Issue:** open-telemetry#33762

**Testing:** No additional test cases were added. Existing tests cover
the necessary functionality.

**Documentation:** The **README.md** file was previously added and
already includes relevant details about the component and the conversion
process.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants