Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deduplicate the list of names when deserializing InternalTopMetrics #116298

Merged
merged 7 commits into from
Nov 7, 2024

Conversation

iverase
Copy link
Contributor

@iverase iverase commented Nov 6, 2024

In #112707 we added infrastructure to deduplicate objects where deserailizing internal aggregations. Let' use it to deduplicate the names of metrics in InternalTopMetrics.

@elasticsearchmachine elasticsearchmachine added the Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) label Nov 6, 2024
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-analytical-engine (Team:Analytics)

@iverase
Copy link
Contributor Author

iverase commented Nov 6, 2024

@elasticmachine update branch

@iverase
Copy link
Contributor Author

iverase commented Nov 6, 2024

@elasticmachine update branch

@iverase
Copy link
Contributor Author

iverase commented Nov 6, 2024

@elasticmachine update branch

@iverase
Copy link
Contributor Author

iverase commented Nov 6, 2024

@elasticmachine update branch

Copy link
Member

@not-napoleon not-napoleon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the change to the aggregation looks fine. I have some questions (noted inline) around the lucene test case changes, but I'm also rarely in that file and fine with deferring to your judgment there.

@@ -74,7 +77,6 @@
import static org.hamcrest.Matchers.equalTo;

public class LuceneTests extends ESTestCase {
private static final NamedWriteableRegistry EMPTY_REGISTRY = new NamedWriteableRegistry(Collections.emptyList());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was this just unused and you're cleaning it up? It's unclear to me what this removal has to do with the main change. No worries if you're just cleaning stuff up, just want to make sure I'm not missing something.

Copy link
Contributor Author

@iverase iverase Nov 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was used for serialization / deserialization using the generic method ESTestCase#copyInstance which required a registry. We cannot use that method anymore because it requires to implement Writable now.

try (StreamInput in = new NamedWriteableAwareStreamInput(output.bytes().streamInput(), namedWriteableRegistry)) {
in.setTransportVersion(version);
return reader.read(in);
if (randomBoolean()) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit - Is it worth adding some logging here so we can figure out what branch we're in if a test fails?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I normally rely on the seed to understand why it has failed.

Lucene::writeSortValue,
Lucene::readSortValue,
TransportVersionUtils.randomVersion(random())
);
assertEquals(sortValue, deserialized);
}

private static <T> T copyInstance(T original, Writeable.Writer<T> writer, Writeable.Reader<T> reader, TransportVersion version)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's unclear to me what this method is for. It's private, and I don't see where it's being called? I'm probably just missing something...

Copy link
Contributor Author

@iverase iverase Nov 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This method is to test the serialization / deserialization of SortValues. We cannot use the generic method ESTestCase#copyInstance because Sort values do not implement Writable. Instead we are using here: Lucene::writeSortValue and Lucene::readSortValue.

@iverase iverase added the auto-backport Automatically create backport pull requests when merged label Nov 7, 2024
@iverase iverase merged commit 3438942 into elastic:main Nov 7, 2024
16 checks passed
@iverase iverase deleted the internaltopMetrics branch November 7, 2024 15:06
iverase added a commit to iverase/elasticsearch that referenced this pull request Nov 7, 2024
…lastic#116298)

use deduplication infrastructure to deduplicate the names of metrics in InternalTopMetrics.
@elasticsearchmachine
Copy link
Collaborator

💚 Backport successful

Status Branch Result
8.x

elasticsearchmachine pushed a commit that referenced this pull request Nov 7, 2024
…116298) (#116417)

use deduplication infrastructure to deduplicate the names of metrics in InternalTopMetrics.
kderusso pushed a commit to kderusso/elasticsearch that referenced this pull request Nov 7, 2024
…lastic#116298)

use deduplication infrastructure to deduplicate the names of metrics in InternalTopMetrics.
jozala pushed a commit that referenced this pull request Nov 13, 2024
…116298)

use deduplication infrastructure to deduplicate the names of metrics in InternalTopMetrics.
alexey-ivanov-es pushed a commit to alexey-ivanov-es/elasticsearch that referenced this pull request Nov 28, 2024
…lastic#116298)

use deduplication infrastructure to deduplicate the names of metrics in InternalTopMetrics.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Analytics/Aggregations Aggregations auto-backport Automatically create backport pull requests when merged >non-issue Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) v8.17.0 v9.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants