Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix failure in AppendProcessorTests.testAppendingToListWithDuplicatesDisallowed #62385

Merged
merged 2 commits into from
Sep 16, 2020

Conversation

danhermann
Copy link
Contributor

@danhermann danhermann commented Sep 15, 2020

One of the test cases purported to test the appending of unique values to an existing list but the list of unique values to be appended was randomly generated and would occasionally contain values already in the list (see example test seed below). This fix ensures that the list of unique values never contains any values in the existing list.

Example of a test seed that would fail:
./gradlew ':modules:ingest-common:test' --tests "org.elasticsearch.ingest.common.AppendProcessorTests.testAppendingToListWithDuplicatesDisallowed" -Dtests.seed=AC5D435485DE7219

@danhermann danhermann added >test Issues or PRs that are addressing/adding tests :Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP v8.0.0 v7.10.0 labels Sep 15, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-features (:Core/Features/Ingest)

@elasticmachine elasticmachine added the Team:Data Management Meta label for data/management team label Sep 15, 2020
Copy link
Member

@dakrone dakrone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left a comment about this, it was hard for me to understand exactly what the code was doing without staring at it for a few minutes

List<String> nonexistingValues = new ArrayList<>();
for (int i = 0; i < nonexistingValuesSize; i++) {
boolean valueExists = true;
while (valueExists) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems really confusing to me, I think we could do the same thing with:

int nonexistingValueSize = randomIntBetween(0, 10);
// generate n new values
Set<String> newValues = Stream.generate(() -> randomAlphaOfLengthBetween(1, 10)).limit(nonexistingValueSize).collect(Collectors.toSet());
// create a set using the new values, making sure there are no overlapping values already present in the existing values
Set<String> nonexistingValues = Sets.difference(newValues, new HashSet<>(existingValues));

Using the while loop for this is a bit overkill, I think Sets makes it easier to avoid duplicates

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, @dakrone. I simplified the test as you suggested.

Copy link
Member

@dakrone dakrone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks Dan

@danhermann danhermann merged commit 03d0e19 into elastic:master Sep 16, 2020
@danhermann
Copy link
Contributor Author

Thanks, @dakrone!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP Team:Data Management Meta label for data/management team >test Issues or PRs that are addressing/adding tests v7.10.0 v8.0.0-alpha1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants