Put default options in config/startup.options

[untergeek]

Working on the packaging system has shown me how painful it is to manage defaults in each of the different init styles (each variant of upstart; SYSV, systemd). We should have Logstash pull from a standard location in the LOGSTASH_HOME path of /usr/share/logstash/, I would propose /usr/share/logstash/config/startup.options.

Everything that normally goes in /etc/default/logstash or /etc/sysconfig/logstash, or manually inserted into the init script should be in this startup.options file, which currently looks like this:

###############################
# Default settings for logstash
###############################

# Override Java location
JAVACMD=/usr/bin/java

# Set a home directory
LS_HOME=/usr/share/logstash

# Arguments to pass to logstash
LS_OPTS=""

# Arguments to pass to java
LS_HEAP_SIZE="1g"
LS_JAVA_OPTS="-Djava.io.tmpdir=$HOME"

# pidfiles aren't used for upstart; this is for sysv users.
LS_PIDFILE=/var/run/logstash.pid

# user id to be invoked as; for upstart: edit /etc/init/logstash.conf
LS_USER=logstash

# logstash logging
LS_LOG_FILE=/var/log/logstash/logstash.log
# Set LS_USE_GC_LOGGING to "true" to enable GC logging.  Leave empty to disable.
LS_USE_GC_LOGGING=""
LS_GC_LOG_FILE=/var/log/logstash/gc.log

# logstash configuration directory
LS_CONF_DIR=/etc/logstash/conf.d

# Open file limit; cannot be overridden in upstart
LS_OPEN_FILES=16384

# Nice level
LS_NICE=19

# If this is set to 1, then when `stop` is called, if the process has
# not exited within a reasonable time, SIGKILL will be sent next.
# The default behavior is to simply log a message "program stop failed; still running"
#KILL_ON_STOP_TIMEOUT=0

JVM options should be moved to config/jvm.options as described in #5302, and anything else goes here.

This results in init scripts, regardless of flavor, reading the default config file, or an alternate flagged path. As far as nice levels, LS_USER, LS_GROUP, LS_OPEN_FILES and such go, these are defaults used by the pleaserun wrapper I'm working on to generate the init scripts. If you change these, you should re-run the system-install script in bin. It would read these settings and re-create the init script with the option changes.

[suyograo]

@untergeek as we spoke in the standup today, this idea is being implemented as a conf/settings.yml in #4401. Then, we wouldn't need this startup.options, IMO

[untergeek]

This settings.yml file will be read by the start script and assign the nice level, and the file limit, and other settings? What's going to initialize these?

[suyograo]

hmm, actually I am not sure we need to muddle settings.yml with init script variables. These settings can go in a separate file.

[jsvd]

logstash will by default fetch the LOGSTASH_HOME/conf/logstash.yml (overridable by LS_SETTINGS_DIR) and load it programatically, it is not fed through the cli.

All options passed in the cli will override the ones in the logstash.yml (which in turn override the default settings, encoded in logstash-core/lib/logstash/environment.rb)

[untergeek]

can we make it LOGSTASH_HOME/config, so it matches the directory pattern used by Elasticsearch?

[suyograo]

+1 to make it LOGSTASH_HOME/config. I thought it already was.

[suyograo]

Fixed in #5341