Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move ingest-geoip default databases out of config #36949

Merged
merged 2 commits into from
Dec 21, 2018

Conversation

jasontedor
Copy link
Member

@jasontedor jasontedor commented Dec 21, 2018

Today the default databases bundled with ingest-geoip are treated as configuration files that we unbundle into the Elasticsearch configuration directory. This can cause problems for users using our Docker images if they bind mount over the configuration directory. Additionally, it creates complexity when trying to convert ingest-geoip to a module. This commit moves these databases out of the configuration directory and instead loads from the plugin directory. Further, custom databases can still be put into the configuration directory.

Relates #36898

Today the default databases bundled with ingest-geoip are treated as
configuration files that we unbundle into the Elasticsearch
configuration directory. This can cause problems for users using our
Docker images if they bind mount over the configuration
directory. Additionally, it creates complexity when trying to convert
ingest-geoip to a module. This commit moves these databases out of the
configuration directory and instead loads from the plugin
directory. Further, custom databases can still be put into the
configuration directory.
@jasontedor jasontedor added >enhancement :Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP v7.0.0 v6.7.0 labels Dec 21, 2018
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-features

@jasontedor
Copy link
Member Author

@elasticmachine run the default distro tests

Copy link
Contributor

@jakelandis jakelandis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jasontedor jasontedor merged commit 6781a29 into elastic:master Dec 21, 2018
Copy link
Member

@rjernst rjernst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One general question: could we move the default databases into resource files do they are loaded the same between tests and production? Then we don't need to mess with finding the directory they should exist in, but just load the resources as a stream.

@jasontedor
Copy link
Member Author

No, because then we would not be able to memory map the databases as they have to be actual files on disk to do that (we can not memory map from a resource input stream).

jasontedor added a commit that referenced this pull request Dec 21, 2018
Today the default databases bundled with ingest-geoip are treated as
configuration files that we unbundle into the Elasticsearch
configuration directory. This can cause problems for users using our
Docker images if they bind mount over the configuration
directory. Additionally, it creates complexity when trying to convert
ingest-geoip to a module. This commit moves these databases out of the
configuration directory and instead loads from the plugin
directory. Further, custom databases can still be put into the
configuration directory.
@jasontedor jasontedor deleted the ingest-geoip-default-databases branch December 21, 2018 23:10
jasontedor added a commit to jasontedor/elasticsearch that referenced this pull request Dec 21, 2018
* master: (31 commits)
  Move ingest-geoip default databases out of config (elastic#36949)
  [ILM][DOCS] add extra scenario to policy update docs (elastic#36871)
  [Painless] Add String Casting Tests (elastic#36945)
  SQL: documentation improvements and updates (elastic#36918)
  [DOCS] Merges list of discovery and cluster formation settings (elastic#36909)
  Only compress responses if request was compressed (elastic#36867)
  Remove duplicate paragraph (elastic#36942)
  Fix URI to cluster stats endpoint on specific nodes (elastic#36784)
  Fix typo in unitTest task (elastic#36930)
  RecoveryMonitor#lastSeenAccessTime should be volatile (elastic#36781)
  [CCR] Add `ccr.auto_follow_coordinator.wait_for_timeout` setting (elastic#36714)
  Scripting: Remove deprecated params.ctx (elastic#36848)
  Refactor the REST actions to clarify what endpoints are deprecated. (elastic#36869)
  Add JDK 12 to CI rotation (elastic#36915)
  Improve error message for 6.x style realm settings (elastic#36876)
  Send clear session as routable remote request (elastic#36805)
  [DOCS] Remove redundant ILM attributes (elastic#36808)
  SQL: Fix bug regarding histograms usage in scripting (elastic#36866)
  Update index mappings when ccr restore complete (elastic#36879)
  Docs: Bump version to alpha2 after release
  ...
@jimczi jimczi added v7.0.0-beta1 and removed v7.0.0 labels Feb 7, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP >enhancement v6.7.0 v7.0.0-beta1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants