Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable GeoIP downloader by default #71505

Merged
merged 29 commits into from
Apr 15, 2021
Merged

Conversation

probakowski
Copy link
Contributor

@probakowski probakowski commented Apr 8, 2021

This change enables GeoIP downloader by default.
It removes feature flag but adds flag that is used by tests to disable it again (as we don't want to hammer GeoIP database service with every test cluster we spin up).

Relates to #68920

@probakowski probakowski mentioned this pull request Apr 8, 2021
15 tasks
@probakowski
Copy link
Contributor Author

@elasticmachine run elasticsearch-ci/2

@probakowski probakowski marked this pull request as ready for review April 13, 2021 23:01
@probakowski probakowski requested a review from martijnvg April 13, 2021 23:01
@probakowski probakowski added :Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP v7.13.0 v8.0.0 labels Apr 13, 2021
@elasticmachine elasticmachine added the Team:Data Management Meta label for data/management team label Apr 13, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-features (Team:Core/Features)

Copy link
Member

@martijnvg martijnvg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @probakowski! I left a few questions.

distribution/docker/docker-compose.yml Show resolved Hide resolved
@@ -52,17 +52,11 @@ if (useFixture) {
}

tasks.named("internalClusterTest").configure {
systemProperty "es.geoip_v2_feature_flag_enabled", "true"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the toplevel build es.geoip_v2_disabled_in_tests is set to true, but I don't see it being set to false here.
How do the yaml, qa and java integ tests run with the geoip downloader enabled here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We switch geoip downloader on in tests themselves, at the beginning we assert state with it disabled then we start it and wait until its work is done. Earlier we had to switch flag because it controlled if downloader is registered at all, right now it only controls default value for enabled setting and everything is registered everytime.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, I forgot about that. However I'm wondering whether for the geoip module, we can run all our tests without enabling the geoip downloader. It should by default be enabled and so there should be no need to enable it manually. I fear that that perhaps somewhere down the line we end up with an issue being reported by users that their databases aren't updated. If all our geoip integ tests enable the downloader if it is disabled, then we can never catch such a mistake.

So should we instead change the tests so that before each test is run we assert that the geoip downloader is enabled and unset the system property that toplevel build file is setting? (If that is the case, then we can address that in a followup pr.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, increased confidence in our default is worth changing it. I'd rather do it in followup though

@probakowski
Copy link
Contributor Author

@elasticmachine update branch

@probakowski probakowski requested a review from martijnvg April 15, 2021 09:35
Copy link
Member

@martijnvg martijnvg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@probakowski probakowski merged commit 39eb12a into elastic:master Apr 15, 2021
@probakowski probakowski deleted the geoip-flag branch April 15, 2021 10:28
probakowski added a commit to probakowski/elasticsearch that referenced this pull request Apr 15, 2021
This change enables GeoIP downloader by default.
It removes feature flag but adds flag that is used by tests to disable it again (as we don't want to hammer GeoIP database service with every test cluster we spin up).

Relates to elastic#68920
probakowski added a commit that referenced this pull request Apr 15, 2021
* Enable GeoIP downloader by default (#71505)

This change enables GeoIP downloader by default.
It removes feature flag but adds flag that is used by tests to disable it again (as we don't want to hammer GeoIP database service with every test cluster we spin up).

Relates to #68920

* fix compilation

* spotless

* packaging tests

* disableGeoIpDownloader

* fix packaging
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP >enhancement Team:Data Management Meta label for data/management team v7.13.0 v8.0.0-alpha1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants