Include ES_CLASSPATH in provided SysV init script?

[bflad]

Hi Folks,

We're currently using the official elasticsearch/elasticsearch-repository-hdfs/2.0.2-light plugin to support HDFS snapshots. In our environment we use Cloudera Distribution Hadoop, so we need to supply these jars on startup into the Elasticsearch Java classpath.
https://github.com/elasticsearch/elasticsearch-hadoop/tree/master/repository-hdfs

As another point of reference, we're also using the official Puppet module in our environment so this problem would apply to both the supplied RPM files here as well as the Puppet file templates.
https://github.com/elasticsearch/puppet-elasticsearch/blob/master/templates/etc/init.d/elasticsearch.RedHat.erb

Currently, I provide ES_CLASSPATH="$ES_CLASSPATH:/path/to/hadoop/jars" in our /etc/sysconfig/elasticsearch and add an additional line in /etc/init.d/elasticsearch export ES_CLASSPATH (since the official Puppet module doesn't support export ES_CLASSPATH="..." directly).

Should the SysV init script include export ES_CLASSPATH like the other export ES_JAVA_OPTS, export ES_HEAP_SIZE, etc. environments variables in there for consistency? I wouldn't mind submitting a pull request to add this if it makes sense.

Thanks!
Brian

[tlrx]

Thanks for reporting @bflad

We are currently updating SysV/system/init.d script to make them more consistent. I label this issue with "packaging" so that we keep it in mind when we reshape the scripts.

[bflad]

Thanks, @tlrx - please let me know if I can help with anything.

[costin]

Hi,

The security and packaging changes in ES (such as Jar hell) don't play well with Hadoop. Or rather expose some issues in Hadoop which are outside our control. However there are solutions.

Currently the plan is to move the plugin in ES core (see #15191). The plugin takes care of loading the Hadoop libraries from a dedicated folder to avoid jar hell (clashes of classes with the same name/package available in different jars) and also use the proper permissions (and do so in a way that makes the code use them).
The best solution in this case, when using a distro is to package/link the necessary jars in the plugin folder so they get picked up. If they are available in the bootstrapping classpath the plugin can't really do much there and it's up to you to handle that. Typically by enabling the needed permissions to the entire JVM (which is more or less the same as disabling the JVM security) and taking care of any duplicate jars ....