Test command line tools with custom paths #12712

clintongormley · 2015-08-07T08:42:36Z

Test commandline tools with custom paths in qa/

Especially stuff like custom plugin path

colings86 · 2015-08-07T12:46:05Z

I ran the following as a manual test to test the beta and also to start recording tests that we might want to have for this when we write an automated test to cover it. The test was run on a 2.0.0-beta1-snapshot build with no configuration changes at all (literally unzip and run).

Installing a plugin to a custom path

ran:

bin/plugin -Des.path.plugins=/path/to/custom/plugins install scirpt-python -u file:/path/to/elasticsearch-master/plugins/lang-python/target/releases/elasticsearch-lang-python-2.0.0-beta1-SNAPSHOT.zip

output on command line was:

-> Installing scirpt-python...
Trying file:/path/to/elasticsearch-master/plugins/lang-python/target/releases/elasticsearch-lang-python-2.0.0-beta1-SNAPSHOT.zip ...
Downloading ............................................................................................................................................................................................................................................................................................................................................DONE
PluginInfo{name='scirpt-python', description='The Python language plugin allows to have python as the language of scripts to execute.', site=false, jvm=true, classname=org.elasticsearch.plugin.python.PythonPlugin, isolated=true, version='2.0.0-beta1-SNAPSHOT'}
Installed scirpt-python into /path/to/custom/plugins/scirpt-python

ls /path/to/custom/plugins/scirpt-python gives:

elasticsearch-lang-python-2.0.0-beta1-SNAPSHOT.jar jython-standalone-2.7.0.jar plugin-descriptor.properties

SUCCESSFUL

clintongormley · 2015-08-07T12:48:42Z

@colings86 probably need to test custom logs/config/scripts/repo paths as well

colings86 · 2015-08-07T12:49:23Z

yup, getting to that now. Posting one test at a time ;)

colings86 · 2015-08-07T12:55:27Z

Listing plugins in a custom directory

Ran:

bin/plugin -Des.path.plugins=/path/to/custom/plugins list

output:

Installed plugins in /path/to/custom/plugins:
    - scirpt-python

SUCCESSFUL

Removing a plugin using a custom plugins path

ran:

bin/plugin -Des.path.plugins=/path/to/custom/plugins remove scirpt-python

output:

-> Removing scirpt-python...
Removed scirpt-python

/path/to/custom/plugins directory no longer contained scirpt-python directory

SUCCESSFUL

colings86 · 2015-08-07T13:08:00Z

bin/plugin with custom config directory

For this test I created a conf directory at /path/to/custom/config and copied the default elasticsearch.yml file changing it only to add path.plugins: /path/to/custom/plugins

Installing a plugin

ran:

bin/plugin -Des.path.conf=/path/to/custom/config install script-python -u file:/path/to/elasticsearch-master/plugins/lang-python/target/releases/elasticsearch-lang-python-2.0.0-beta1-SNAPSHOT.zip

output:

-> Installing script-python...
Trying file:/path/to/elasticsearch-master/plugins/lang-python/target/releases/elasticsearch-lang-python-2.0.0-beta1-SNAPSHOT.zip ...
Downloading ............................................................................................................................................................................................................................................................................................................................................DONE
PluginInfo{name='script-python', description='The Python language plugin allows to have python as the language of scripts to execute.', site=false, jvm=true, classname=org.elasticsearch.plugin.python.PythonPlugin, isolated=true, version='2.0.0-beta1-SNAPSHOT'}
log4j:WARN No appenders could be found for logger (org.elasticsearch.bootstrap).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Installed script-python into /path/to/custom/plugins/script-python

Confirmed script-python directory added to /path/to/custom/plugins

SUCCESSFUL

Listing plugins

ran:

bin/plugin -Des.path.conf=/path/to/custom/config list

output:

Installed plugins in /path/to/custom/plugins:
    - script-python

SUCCESSFUL

Remove plugins

ran:

bin/plugin -Des.path.conf=/path/to/custom/config remove script-python

output:

-> Removing script-python...
Removed script-python

Confirmed script-python directory removed from /path/to/custom/plugins

SUCCESSFUL

colings86 · 2015-08-07T13:16:05Z

bin/plugin with custom log directory

Installing a plugin

ran:

bin/plugin -Des.path.logs=/path/to/custom/logs install script-python -u file:/path/to/elasticsearch-master/plugins/lang-python/target/releases/elasticsearch-lang-python-2.0.0-beta1-SNAPSHOT.zip

Confirmed script-python directory added to /path/to/custom/plugins and /path/to/custom/logs directory created with the following files:

elasticsearch.log                        elasticsearch_deprecation.log            elasticsearch_index_indexing_slowlog.log elasticsearch_index_search_slowlog.log

Also confirmed default logs directory was not created.

SUCCESSFUL

Listing plugins

ran:

bin/plugin -Des.path.logs=/path/to/custom/logs list

output:

Installed plugins in /path/to/custom/plugins:
    - script-python

Confirmed /path/to/custom/logs directory created with the following files:

elasticsearch.log                        elasticsearch_deprecation.log            elasticsearch_index_indexing_slowlog.log elasticsearch_index_search_slowlog.log

Also confirmed default logs directory was not created.

SUCCESSFUL

Remove plugins

ran:

bin/plugin -Des.path.logs=/path/to/custom/logs remove script-python

output:

-> Removing script-python...
Removed script-python

Confirmed script-python directory removed from /path/to/custom/plugins and /path/to/custom/logs directory created with the following files:

elasticsearch.log                        elasticsearch_deprecation.log            elasticsearch_index_indexing_slowlog.log elasticsearch_index_search_slowlog.log

Also confirmed default logs directory was not created.

SUCCESSFUL

spinscale · 2015-08-11T15:11:05Z

how exactly should this be implemented for automated tests? Would we have another project in qa/ that installs/removes plugins with custom paths using custom ./integration-tests.xml files?

Or do we have more <ant antfile calls in the pom.xml of the smoke-test-plugins that just installs plugins into different paths?

brwe · 2015-08-18T10:07:19Z

I made a pr with the first option (another project in qa/ that installs/removes plugins with custom paths) here: #12954.
I wonder if a bats test for this would be better though. Maybe @nik9000 has an opinion on this?

nik9000 · 2015-09-09T12:34:55Z

I wonder if a bats test for this would be better though. Maybe @nik9000 has an opinion on this?

Sorry I hadn't noticed this! Right now the bats tests are used to test bin/plugin and bin/elasticsearch with a few custom paths already so its not a huge stretch, especially when many path errors come from space escaping issues with bash. So its a reasonable choice.

This one test for each these options -Des.path.plugins -Des.path.conf -Des.path.logs Each of the new tests installs each plugin with the option and tests if elasticsearch is able to start with all plugins In addition, this adds checks that - plugins can be uninstalled - config path is actually used - log path is actually used - plugins can be listed with 'plugins list' closes elastic#12712

When running bin/elasticsearch config dir and config file can be passed as a parameter independently with -Des.config and -Des.path.conf. If the config file is not provided then it is assumed to be {path.conf}/elasticearch.yml This was not so for running as a service. For deb the CONF_FILE still pointed to the default path even if a custom CONF_DIR was defined. For rpm the CONF_FILE parameter was not passed on at all, see elastic#5329 . Custom config path and config file now work as follows for all services except systemd: CONF_DIR and CONF_FILE undefined: CONF_FILE points to default CONF_DIR defined but CONF_FILE undefined: CONF_FILE points to CONF_DIR/elasticsearch.yml CONF_FILE defined: CONF_FILE must point to an absolute path For systemd this commit only fixes that the service could be started if only a CONF_DIR is defined. However, now a custom CONF_FILE cannot be defined anymore which seemed the lesser evil to me. relates to elastic#12712 and elastic#12954 closes elastic#5329

…efault.conf It is rarely used and was not consistently handled by different distributions anyway. This commit also adds a test for specifying CONF_DIR when installing plugins and starting elasticsearch. relates to elastic#12712 and elastic#12954 closes elastic#5329 closes elastic#13715

brwe · 2015-10-07T08:41:33Z

This has been implemented as bats test now #12717, in #13258 , #13772 , #13617 and #13640

…efault.conf It is rarely used and was not consistently handled by different distributions anyway. This commit also adds a test for specifying CONF_DIR when installing plugins and starting elasticsearch. relates to #12712 and #12954 closes #5329 closes #13715 Merge pull request #13772 from brwe/remove-CONF_FILE

clintongormley added >test Issues or PRs that are addressing/adding tests blocker v2.0.0 labels Aug 7, 2015

clintongormley added the help wanted adoptme label Aug 7, 2015

clintongormley mentioned this issue Aug 10, 2015

Testing and packaging changes for 2.0 #12768

Closed

16 tasks

brwe self-assigned this Aug 12, 2015

brwe mentioned this issue Aug 18, 2015

Add integrations tests for installing plugins with custom options #12954

Closed

brwe mentioned this issue Sep 21, 2015

fix inconsitent handling of config dir and path settings and add test #13687

Closed

brwe mentioned this issue Sep 24, 2015

Remove option to configure custom config file via CONF_FILE or -Des.default.conf #13772

Merged

brwe removed the help wanted adoptme label Oct 5, 2015

brwe closed this as completed Oct 7, 2015

tlrx mentioned this issue Nov 2, 2015

Elasticsearch 2.0 fails to start after install Marvel 2.0 #14347

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Test command line tools with custom paths #12712

Test command line tools with custom paths #12712

clintongormley commented Aug 7, 2015

colings86 commented Aug 7, 2015

clintongormley commented Aug 7, 2015

colings86 commented Aug 7, 2015

colings86 commented Aug 7, 2015

colings86 commented Aug 7, 2015

colings86 commented Aug 7, 2015

spinscale commented Aug 11, 2015

brwe commented Aug 18, 2015

nik9000 commented Sep 9, 2015

brwe commented Oct 7, 2015

Test command line tools with custom paths #12712

Test command line tools with custom paths #12712

Comments

clintongormley commented Aug 7, 2015

colings86 commented Aug 7, 2015

Installing a plugin to a custom path

clintongormley commented Aug 7, 2015

colings86 commented Aug 7, 2015

colings86 commented Aug 7, 2015

Listing plugins in a custom directory

Removing a plugin using a custom plugins path

colings86 commented Aug 7, 2015

bin/plugin with custom config directory

Installing a plugin

Listing plugins

Remove plugins

colings86 commented Aug 7, 2015

bin/plugin with custom log directory

Installing a plugin

Listing plugins

Remove plugins

spinscale commented Aug 11, 2015

brwe commented Aug 18, 2015

nik9000 commented Sep 9, 2015

brwe commented Oct 7, 2015