added newrelic close buffer #577
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
armstrmi
requested review from
dehaansa and
a team
and removed request for
a team
Codecov Report
@@ Coverage Diff @@
## buffer-rework #577 +/- ##
=================================================
+ Coverage 67.50% 72.80% +5.30%
=================================================
Files 61 75 +14
Lines 3886 5585 +1699
=================================================
+ Hits 2623 4066 +1443
- Misses 1060 1279 +219
- Partials 203 240 +37
Continue to review full report at Codecov.
|
|
The windows unit test failure is interesting, golang/go#22553 - may or may not be related to this pr, I'd be a little surprised if it was related, though. |
cpheps
reviewed
Feb 28, 2022
cpheps
approved these changes
Feb 28, 2022
BinaryFissionGames
approved these changes
Feb 28, 2022
cpheps
pushed a commit
that referenced
this pull request
Feb 28, 2022
* added newrelic close buffer * made necessary fixes * made necessary fixes * unexport client from client.go
cpheps
pushed a commit
that referenced
this pull request
Mar 1, 2022
* added newrelic close buffer * made necessary fixes * made necessary fixes * unexport client from client.go
cpheps
pushed a commit
that referenced
this pull request
Mar 1, 2022
* Changed the buffer interface and updated affected operators Moved current buffers to .old files and created stubs of new ones that satisfy the interface Signed-off-by: Corbin Phelps <[email protected]> * Refactored buffer interface to remove drain in favor of a close that returns entries (#498) Signed-off-by: Corbin Phelps <[email protected]> * Memory buffer rework (#497) * Implemented reworked memory buffer and started tests Signed-off-by: Corbin Phelps <[email protected]> * Working on memory buffer tests Signed-off-by: Corbin Phelps <[email protected]> * Fully tested memory buffer Signed-off-by: Corbin Phelps <[email protected]> * Removed memory_buffer*.old files Signed-off-by: Corbin Phelps <[email protected]> * Removed intentional race condition from memory buffer test so race flag wouldn't tirgger Signed-off-by: Corbin Phelps <[email protected]> * Fixed comment in memorybuffer build Signed-off-by: Corbin Phelps <[email protected]> * Renamed buffer closed error Signed-off-by: Corbin Phelps <[email protected]> * Fixed output operators. Most would fail on error equality to a wrapped error Signed-off-by: Corbin Phelps <[email protected]> * Fixed comment for memorybuffer Signed-off-by: Corbin Phelps <[email protected]> * Removed operatorID from buffer builder interface (#499) Signed-off-by: Corbin Phelps <[email protected]> * Corrected documentation of memory buffer (#502) * Corrected documentation of memory buffer Signed-off-by: Corbin Phelps <[email protected]> * Implemented PR feedback Signed-off-by: Corbin Phelps <[email protected]> * Implemented some wording changes Signed-off-by: Corbin Phelps <[email protected]> * Refactored flusher logic (#501) * Refactored flusher to be a bit more readable and to have the context work as expected Signed-off-by: Corbin Phelps <[email protected]> * Cleaned up flusher doc Signed-off-by: Corbin Phelps <[email protected]> * Fixed up some comments and readme verbage Signed-off-by: Corbin Phelps <[email protected]> * Grammer PR feedback Signed-off-by: Corbin Phelps <[email protected]> * Disk buffer rewrite (#539) * Ran go mod tidy Signed-off-by: Corbin Phelps <[email protected]> * Disk buffer rewrite * cleanup some code; bump up test coverage a bit. * Factor ErrEntryTooLarge into its own exported error * fix mutex copy in tests * use require.ErrorIs when asserting error type * tweak doc * Rewrite to use a circular buffer (ring buffer) -- in-progress * add old benchmarks in * Try using previous greedy strategy The old disk buffer uses an "unfair" strategy, where one reader will get all entries up to max chunk size before it times out. * Use 1 mebibyte for disk buffer benchmark * some performance changes * fix broken circular file * cleanup some todos and commented code * multierr in disk_metadata constructor * update path description * don't export things other than the buffer * rb -> cf, unexport some methods on circularFile struct * unexport another method on circularFile * Comment circular_file liberally * remove multierr, minor comments * unexport some more methods/structs * remove redundant seekedRead sets in discard * update docs; change marshalEntry to not take in a byte slice * remove ability to partial write * Capitalize struct methods used in other files * Make Start, End etc in circular file unexported * put cirularFile reads/writes into a for loop * refactor some conditionals with small helper functions Co-authored-by: Corbin Phelps <[email protected]> * Forward buffer (#549) * push up forward output * Refactored flush buffer * Updated buffer close test for forward output * Added closing of test server in forward tests Signed-off-by: Corbin Phelps <[email protected]> Co-authored-by: jmwilliams89 <[email protected]> Co-authored-by: Corbin Phelps <[email protected]> * Elastic buffer (#552) * Changed the buffer interface and updated affected operators Moved current buffers to .old files and created stubs of new ones that satisfy the interface Signed-off-by: Corbin Phelps <[email protected]> * Refactored buffer interface to remove drain in favor of a close that returns entries (#498) Signed-off-by: Corbin Phelps <[email protected]> * Memory buffer rework (#497) * Implemented reworked memory buffer and started tests Signed-off-by: Corbin Phelps <[email protected]> * Working on memory buffer tests Signed-off-by: Corbin Phelps <[email protected]> * Fully tested memory buffer Signed-off-by: Corbin Phelps <[email protected]> * Removed memory_buffer*.old files Signed-off-by: Corbin Phelps <[email protected]> * Removed intentional race condition from memory buffer test so race flag wouldn't tirgger Signed-off-by: Corbin Phelps <[email protected]> * Fixed comment in memorybuffer build Signed-off-by: Corbin Phelps <[email protected]> * Renamed buffer closed error Signed-off-by: Corbin Phelps <[email protected]> * Fixed output operators. Most would fail on error equality to a wrapped error Signed-off-by: Corbin Phelps <[email protected]> * Fixed comment for memorybuffer Signed-off-by: Corbin Phelps <[email protected]> * Removed operatorID from buffer builder interface (#499) Signed-off-by: Corbin Phelps <[email protected]> * Corrected documentation of memory buffer (#502) * Corrected documentation of memory buffer Signed-off-by: Corbin Phelps <[email protected]> * Implemented PR feedback Signed-off-by: Corbin Phelps <[email protected]> * Implemented some wording changes Signed-off-by: Corbin Phelps <[email protected]> * Refactored flusher logic (#501) * Refactored flusher to be a bit more readable and to have the context work as expected Signed-off-by: Corbin Phelps <[email protected]> * Cleaned up flusher doc Signed-off-by: Corbin Phelps <[email protected]> * Fixed up some comments and readme verbage Signed-off-by: Corbin Phelps <[email protected]> * Grammer PR feedback Signed-off-by: Corbin Phelps <[email protected]> * Disk buffer rewrite (#539) * Ran go mod tidy Signed-off-by: Corbin Phelps <[email protected]> * Disk buffer rewrite * cleanup some code; bump up test coverage a bit. * Factor ErrEntryTooLarge into its own exported error * fix mutex copy in tests * use require.ErrorIs when asserting error type * tweak doc * Rewrite to use a circular buffer (ring buffer) -- in-progress * add old benchmarks in * Try using previous greedy strategy The old disk buffer uses an "unfair" strategy, where one reader will get all entries up to max chunk size before it times out. * Use 1 mebibyte for disk buffer benchmark * some performance changes * fix broken circular file * cleanup some todos and commented code * multierr in disk_metadata constructor * update path description * don't export things other than the buffer * rb -> cf, unexport some methods on circularFile struct * unexport another method on circularFile * Comment circular_file liberally * remove multierr, minor comments * unexport some more methods/structs * remove redundant seekedRead sets in discard * update docs; change marshalEntry to not take in a byte slice * remove ability to partial write * Capitalize struct methods used in other files * Make Start, End etc in circular file unexported * put cirularFile reads/writes into a for loop * refactor some conditionals with small helper functions Co-authored-by: Corbin Phelps <[email protected]> * push up forward output * Refactored flush buffer * Updated buffer close test for forward output * push up forward output * Refactored flush buffer * Updated buffer close test for forward output * updated elastic output * Changed the buffer interface and updated affected operators Moved current buffers to .old files and created stubs of new ones that satisfy the interface Signed-off-by: Corbin Phelps <[email protected]> * Memory buffer rework (#497) * Implemented reworked memory buffer and started tests Signed-off-by: Corbin Phelps <[email protected]> * Working on memory buffer tests Signed-off-by: Corbin Phelps <[email protected]> * Fully tested memory buffer Signed-off-by: Corbin Phelps <[email protected]> * Removed memory_buffer*.old files Signed-off-by: Corbin Phelps <[email protected]> * Removed intentional race condition from memory buffer test so race flag wouldn't tirgger Signed-off-by: Corbin Phelps <[email protected]> * Fixed comment in memorybuffer build Signed-off-by: Corbin Phelps <[email protected]> * Renamed buffer closed error Signed-off-by: Corbin Phelps <[email protected]> * Fixed output operators. Most would fail on error equality to a wrapped error Signed-off-by: Corbin Phelps <[email protected]> * Fixed comment for memorybuffer Signed-off-by: Corbin Phelps <[email protected]> * Removed operatorID from buffer builder interface (#499) Signed-off-by: Corbin Phelps <[email protected]> * Disk buffer rewrite (#539) * Ran go mod tidy Signed-off-by: Corbin Phelps <[email protected]> * Disk buffer rewrite * cleanup some code; bump up test coverage a bit. * Factor ErrEntryTooLarge into its own exported error * fix mutex copy in tests * use require.ErrorIs when asserting error type * tweak doc * Rewrite to use a circular buffer (ring buffer) -- in-progress * add old benchmarks in * Try using previous greedy strategy The old disk buffer uses an "unfair" strategy, where one reader will get all entries up to max chunk size before it times out. * Use 1 mebibyte for disk buffer benchmark * some performance changes * fix broken circular file * cleanup some todos and commented code * multierr in disk_metadata constructor * update path description * don't export things other than the buffer * rb -> cf, unexport some methods on circularFile struct * unexport another method on circularFile * Comment circular_file liberally * remove multierr, minor comments * unexport some more methods/structs * remove redundant seekedRead sets in discard * update docs; change marshalEntry to not take in a byte slice * remove ability to partial write * Capitalize struct methods used in other files * Make Start, End etc in circular file unexported * put cirularFile reads/writes into a for loop * refactor some conditionals with small helper functions Co-authored-by: Corbin Phelps <[email protected]> * got rid of errorf in stop() Co-authored-by: Corbin Phelps <[email protected]> Co-authored-by: Corbin Phelps <[email protected]> Co-authored-by: Brandon Johnson <[email protected]> Co-authored-by: jmwilliams89 <[email protected]> * Google buffer (#563) * wip * wip * flushes buffer after Stop() * updated google-output tests * made recommended changes from brandon * Linter fixes (#566) Signed-off-by: Corbin Phelps <[email protected]> * added newrelic close buffer (#577) * added newrelic close buffer * made necessary fixes * made necessary fixes * unexport client from client.go * Fixed reverted timeout in disk test Signed-off-by: Corbin Phelps <[email protected]> * added check for no entries upon stop Co-authored-by: Brandon Johnson <[email protected]> Co-authored-by: Mitchell Armstrong <[email protected]> Co-authored-by: jmwilliams89 <[email protected]> Co-authored-by: armstrmi <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of Changes
Please check that the PR fulfills these requirements