Stream iterators account for unordered data #4114
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This fixes a bug where ingester stream iterators assumed their chunks maintained increasing inter chunk order. This is not guaranteed now that unordered writes are available.
This is a followup after closing #4101. Here, we take a much simpler route, adding inter-chunk ordering calculations while iterating through chunks, but importantly do not incur additional slice allocations doing so.
I spent some time attempting to build centralized logic for these calculations in order to reuse it in a few places with duplicate logic, but I was unable to find a working solution that didn't incur slice allocations which I want to avoid on this code path. I do, however, want to revisit these issues post the introduction of generics in go (or if someone sees something I did not).
ref #1544