fix: batch BQ writes by entire PRW request #334
Conversation
Batch the entire Remote-Write request, rather than per-timeseries. This allows a significantly higher write throughput. Also, move the tagsFromMetric call out of the inner loop.
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #334 +/- ##
===========================================
- Coverage 70.72% 40.00% -30.73%
===========================================
Files 2 1 -1
Lines 263 5 -258
===========================================
- Hits 186 2 -184
+ Misses 66 3 -63
+ Partials 11 0 -11
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
|
Hi @karl-nilsson ! Unfortunately, the tests are failing because of #204 and I have not had time to try to fix our tests so they can be run from a fork. I pulled the code from your PR and ran the tests manually, they all passed and the changes look good. Since I don't have time to fix #204 soon, I am okay with accepting the changes without fixing the pipeline, and I will try to create a release today. Thank you very much for your contribution! |
|
@karl-nilsson this has been included in v0.8.0, thanks again for your contribution! |
|
TYVM! |
Pull Request Template
Description
Batch the entire Remote-Write request, rather than per-timeseries. This allows a significantly higher write throughput.
Move the tagsFromMetric call out of the inner loop, it's only necessary to invoke it once per timeseries.
Type of change
Environment