Forward buffer #549
Merged
Forward buffer #549
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@BinaryFissionGames Looks good to you? |
cpheps
force-pushed
the
buffer-rework
branch
from
b16f8f8 to
f377d5e
Compare
cpheps
force-pushed
the
forward-buffer
branch
from
1d1eff4 to
35e6bbb
Compare
cpheps
suggested changes
Feb 15, 2022
cpheps
force-pushed
the
buffer-rework
branch
from
f377d5e to
4b1db96
Compare
cpheps
force-pushed
the
forward-buffer
branch
from
2ba3ea3 to
feff149
Compare
Signed-off-by: Corbin Phelps <[email protected]>
cpheps
force-pushed
the
forward-buffer
branch
from
feff149 to
7a25f65
Compare
|
Test failures are unrelated to code changes. |
cpheps
pushed a commit
that referenced
this pull request
Feb 16, 2022
* push up forward output * Refactored flush buffer * Updated buffer close test for forward output * Added closing of test server in forward tests Signed-off-by: Corbin Phelps <[email protected]> Co-authored-by: jmwilliams89 <[email protected]> Co-authored-by: Corbin Phelps <[email protected]>
cpheps
pushed a commit
that referenced
this pull request
Feb 17, 2022
* push up forward output * Refactored flush buffer * Updated buffer close test for forward output * Added closing of test server in forward tests Signed-off-by: Corbin Phelps <[email protected]> Co-authored-by: jmwilliams89 <[email protected]> Co-authored-by: Corbin Phelps <[email protected]>
cpheps
pushed a commit
that referenced
this pull request
Feb 18, 2022
* push up forward output * Refactored flush buffer * Updated buffer close test for forward output * Added closing of test server in forward tests Signed-off-by: Corbin Phelps <[email protected]> Co-authored-by: jmwilliams89 <[email protected]> Co-authored-by: Corbin Phelps <[email protected]>
cpheps
pushed a commit
that referenced
this pull request
Feb 21, 2022
* push up forward output * Refactored flush buffer * Updated buffer close test for forward output * Added closing of test server in forward tests Signed-off-by: Corbin Phelps <[email protected]> Co-authored-by: jmwilliams89 <[email protected]> Co-authored-by: Corbin Phelps <[email protected]>
cpheps
pushed a commit
that referenced
this pull request
Feb 23, 2022
* push up forward output * Refactored flush buffer * Updated buffer close test for forward output * Added closing of test server in forward tests Signed-off-by: Corbin Phelps <[email protected]> Co-authored-by: jmwilliams89 <[email protected]> Co-authored-by: Corbin Phelps <[email protected]>
cpheps
pushed a commit
that referenced
this pull request
Feb 25, 2022
* push up forward output * Refactored flush buffer * Updated buffer close test for forward output * Added closing of test server in forward tests Signed-off-by: Corbin Phelps <[email protected]> Co-authored-by: jmwilliams89 <[email protected]> Co-authored-by: Corbin Phelps <[email protected]>
cpheps
pushed a commit
that referenced
this pull request
Feb 28, 2022
* push up forward output * Refactored flush buffer * Updated buffer close test for forward output * Added closing of test server in forward tests Signed-off-by: Corbin Phelps <[email protected]> Co-authored-by: jmwilliams89 <[email protected]> Co-authored-by: Corbin Phelps <[email protected]>
cpheps
pushed a commit
that referenced
this pull request
Mar 1, 2022
* push up forward output * Refactored flush buffer * Updated buffer close test for forward output * Added closing of test server in forward tests Signed-off-by: Corbin Phelps <[email protected]> Co-authored-by: jmwilliams89 <[email protected]> Co-authored-by: Corbin Phelps <[email protected]>
cpheps
pushed a commit
that referenced
this pull request
Mar 1, 2022
* Changed the buffer interface and updated affected operators Moved current buffers to .old files and created stubs of new ones that satisfy the interface Signed-off-by: Corbin Phelps <[email protected]> * Refactored buffer interface to remove drain in favor of a close that returns entries (#498) Signed-off-by: Corbin Phelps <[email protected]> * Memory buffer rework (#497) * Implemented reworked memory buffer and started tests Signed-off-by: Corbin Phelps <[email protected]> * Working on memory buffer tests Signed-off-by: Corbin Phelps <[email protected]> * Fully tested memory buffer Signed-off-by: Corbin Phelps <[email protected]> * Removed memory_buffer*.old files Signed-off-by: Corbin Phelps <[email protected]> * Removed intentional race condition from memory buffer test so race flag wouldn't tirgger Signed-off-by: Corbin Phelps <[email protected]> * Fixed comment in memorybuffer build Signed-off-by: Corbin Phelps <[email protected]> * Renamed buffer closed error Signed-off-by: Corbin Phelps <[email protected]> * Fixed output operators. Most would fail on error equality to a wrapped error Signed-off-by: Corbin Phelps <[email protected]> * Fixed comment for memorybuffer Signed-off-by: Corbin Phelps <[email protected]> * Removed operatorID from buffer builder interface (#499) Signed-off-by: Corbin Phelps <[email protected]> * Corrected documentation of memory buffer (#502) * Corrected documentation of memory buffer Signed-off-by: Corbin Phelps <[email protected]> * Implemented PR feedback Signed-off-by: Corbin Phelps <[email protected]> * Implemented some wording changes Signed-off-by: Corbin Phelps <[email protected]> * Refactored flusher logic (#501) * Refactored flusher to be a bit more readable and to have the context work as expected Signed-off-by: Corbin Phelps <[email protected]> * Cleaned up flusher doc Signed-off-by: Corbin Phelps <[email protected]> * Fixed up some comments and readme verbage Signed-off-by: Corbin Phelps <[email protected]> * Grammer PR feedback Signed-off-by: Corbin Phelps <[email protected]> * Disk buffer rewrite (#539) * Ran go mod tidy Signed-off-by: Corbin Phelps <[email protected]> * Disk buffer rewrite * cleanup some code; bump up test coverage a bit. * Factor ErrEntryTooLarge into its own exported error * fix mutex copy in tests * use require.ErrorIs when asserting error type * tweak doc * Rewrite to use a circular buffer (ring buffer) -- in-progress * add old benchmarks in * Try using previous greedy strategy The old disk buffer uses an "unfair" strategy, where one reader will get all entries up to max chunk size before it times out. * Use 1 mebibyte for disk buffer benchmark * some performance changes * fix broken circular file * cleanup some todos and commented code * multierr in disk_metadata constructor * update path description * don't export things other than the buffer * rb -> cf, unexport some methods on circularFile struct * unexport another method on circularFile * Comment circular_file liberally * remove multierr, minor comments * unexport some more methods/structs * remove redundant seekedRead sets in discard * update docs; change marshalEntry to not take in a byte slice * remove ability to partial write * Capitalize struct methods used in other files * Make Start, End etc in circular file unexported * put cirularFile reads/writes into a for loop * refactor some conditionals with small helper functions Co-authored-by: Corbin Phelps <[email protected]> * Forward buffer (#549) * push up forward output * Refactored flush buffer * Updated buffer close test for forward output * Added closing of test server in forward tests Signed-off-by: Corbin Phelps <[email protected]> Co-authored-by: jmwilliams89 <[email protected]> Co-authored-by: Corbin Phelps <[email protected]> * Elastic buffer (#552) * Changed the buffer interface and updated affected operators Moved current buffers to .old files and created stubs of new ones that satisfy the interface Signed-off-by: Corbin Phelps <[email protected]> * Refactored buffer interface to remove drain in favor of a close that returns entries (#498) Signed-off-by: Corbin Phelps <[email protected]> * Memory buffer rework (#497) * Implemented reworked memory buffer and started tests Signed-off-by: Corbin Phelps <[email protected]> * Working on memory buffer tests Signed-off-by: Corbin Phelps <[email protected]> * Fully tested memory buffer Signed-off-by: Corbin Phelps <[email protected]> * Removed memory_buffer*.old files Signed-off-by: Corbin Phelps <[email protected]> * Removed intentional race condition from memory buffer test so race flag wouldn't tirgger Signed-off-by: Corbin Phelps <[email protected]> * Fixed comment in memorybuffer build Signed-off-by: Corbin Phelps <[email protected]> * Renamed buffer closed error Signed-off-by: Corbin Phelps <[email protected]> * Fixed output operators. Most would fail on error equality to a wrapped error Signed-off-by: Corbin Phelps <[email protected]> * Fixed comment for memorybuffer Signed-off-by: Corbin Phelps <[email protected]> * Removed operatorID from buffer builder interface (#499) Signed-off-by: Corbin Phelps <[email protected]> * Corrected documentation of memory buffer (#502) * Corrected documentation of memory buffer Signed-off-by: Corbin Phelps <[email protected]> * Implemented PR feedback Signed-off-by: Corbin Phelps <[email protected]> * Implemented some wording changes Signed-off-by: Corbin Phelps <[email protected]> * Refactored flusher logic (#501) * Refactored flusher to be a bit more readable and to have the context work as expected Signed-off-by: Corbin Phelps <[email protected]> * Cleaned up flusher doc Signed-off-by: Corbin Phelps <[email protected]> * Fixed up some comments and readme verbage Signed-off-by: Corbin Phelps <[email protected]> * Grammer PR feedback Signed-off-by: Corbin Phelps <[email protected]> * Disk buffer rewrite (#539) * Ran go mod tidy Signed-off-by: Corbin Phelps <[email protected]> * Disk buffer rewrite * cleanup some code; bump up test coverage a bit. * Factor ErrEntryTooLarge into its own exported error * fix mutex copy in tests * use require.ErrorIs when asserting error type * tweak doc * Rewrite to use a circular buffer (ring buffer) -- in-progress * add old benchmarks in * Try using previous greedy strategy The old disk buffer uses an "unfair" strategy, where one reader will get all entries up to max chunk size before it times out. * Use 1 mebibyte for disk buffer benchmark * some performance changes * fix broken circular file * cleanup some todos and commented code * multierr in disk_metadata constructor * update path description * don't export things other than the buffer * rb -> cf, unexport some methods on circularFile struct * unexport another method on circularFile * Comment circular_file liberally * remove multierr, minor comments * unexport some more methods/structs * remove redundant seekedRead sets in discard * update docs; change marshalEntry to not take in a byte slice * remove ability to partial write * Capitalize struct methods used in other files * Make Start, End etc in circular file unexported * put cirularFile reads/writes into a for loop * refactor some conditionals with small helper functions Co-authored-by: Corbin Phelps <[email protected]> * push up forward output * Refactored flush buffer * Updated buffer close test for forward output * push up forward output * Refactored flush buffer * Updated buffer close test for forward output * updated elastic output * Changed the buffer interface and updated affected operators Moved current buffers to .old files and created stubs of new ones that satisfy the interface Signed-off-by: Corbin Phelps <[email protected]> * Memory buffer rework (#497) * Implemented reworked memory buffer and started tests Signed-off-by: Corbin Phelps <[email protected]> * Working on memory buffer tests Signed-off-by: Corbin Phelps <[email protected]> * Fully tested memory buffer Signed-off-by: Corbin Phelps <[email protected]> * Removed memory_buffer*.old files Signed-off-by: Corbin Phelps <[email protected]> * Removed intentional race condition from memory buffer test so race flag wouldn't tirgger Signed-off-by: Corbin Phelps <[email protected]> * Fixed comment in memorybuffer build Signed-off-by: Corbin Phelps <[email protected]> * Renamed buffer closed error Signed-off-by: Corbin Phelps <[email protected]> * Fixed output operators. Most would fail on error equality to a wrapped error Signed-off-by: Corbin Phelps <[email protected]> * Fixed comment for memorybuffer Signed-off-by: Corbin Phelps <[email protected]> * Removed operatorID from buffer builder interface (#499) Signed-off-by: Corbin Phelps <[email protected]> * Disk buffer rewrite (#539) * Ran go mod tidy Signed-off-by: Corbin Phelps <[email protected]> * Disk buffer rewrite * cleanup some code; bump up test coverage a bit. * Factor ErrEntryTooLarge into its own exported error * fix mutex copy in tests * use require.ErrorIs when asserting error type * tweak doc * Rewrite to use a circular buffer (ring buffer) -- in-progress * add old benchmarks in * Try using previous greedy strategy The old disk buffer uses an "unfair" strategy, where one reader will get all entries up to max chunk size before it times out. * Use 1 mebibyte for disk buffer benchmark * some performance changes * fix broken circular file * cleanup some todos and commented code * multierr in disk_metadata constructor * update path description * don't export things other than the buffer * rb -> cf, unexport some methods on circularFile struct * unexport another method on circularFile * Comment circular_file liberally * remove multierr, minor comments * unexport some more methods/structs * remove redundant seekedRead sets in discard * update docs; change marshalEntry to not take in a byte slice * remove ability to partial write * Capitalize struct methods used in other files * Make Start, End etc in circular file unexported * put cirularFile reads/writes into a for loop * refactor some conditionals with small helper functions Co-authored-by: Corbin Phelps <[email protected]> * got rid of errorf in stop() Co-authored-by: Corbin Phelps <[email protected]> Co-authored-by: Corbin Phelps <[email protected]> Co-authored-by: Brandon Johnson <[email protected]> Co-authored-by: jmwilliams89 <[email protected]> * Google buffer (#563) * wip * wip * flushes buffer after Stop() * updated google-output tests * made recommended changes from brandon * Linter fixes (#566) Signed-off-by: Corbin Phelps <[email protected]> * added newrelic close buffer (#577) * added newrelic close buffer * made necessary fixes * made necessary fixes * unexport client from client.go * Fixed reverted timeout in disk test Signed-off-by: Corbin Phelps <[email protected]> * added check for no entries upon stop Co-authored-by: Brandon Johnson <[email protected]> Co-authored-by: Mitchell Armstrong <[email protected]> Co-authored-by: jmwilliams89 <[email protected]> Co-authored-by: armstrmi <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of Changes
Please check that the PR fulfills these requirements