kibana-keystore does not respect path.data in kibana.yml

[jasontedor]

Here's a simple reproduction:

jason-mbp:~ jason$ mkdir -p /tmp/kibana
jason-mbp:~ jason$ cd /tmp/kibana
jason-mbp:kibana jason$ curl -L https://artifacts.elastic.co/downloads/kibana/kibana-7.3.0-darwin-x86_64.tar.gz | tar xz -C .
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  222M  100  222M    0     0  10.2M      0  0:00:21  0:00:21 --:--:-- 13.1M
jason-mbp:kibana jason$ mkdir -p /tmp/kibana/data && echo "path.data: /tmp/kibana/data" >> ./kibana-7.3.0-darwin-x86_64/config/kibana.yml 
jason-mbp:kibana jason$ rmdir /tmp/kibana/kibana-7.3.0-darwin-x86_64/data
jason-mbp:kibana jason$ ./kibana-7.3.0-darwin-x86_64/bin/kibana-keystore create
FATAL CLI ERROR Error: ENOENT: no such file or directory, open '/private/tmp/kibana/kibana-7.3.0-darwin-x86_64/data/kibana.keystore'
    at Object.openSync (fs.js:439:3)
    at writeFileSync (fs.js:1190:35)
    at Keystore.save (/private/tmp/kibana/kibana-7.3.0-darwin-x86_64/src/legacy/server/keystore/keystore.js:63:27)
    at create (/private/tmp/kibana/kibana-7.3.0-darwin-x86_64/src/cli_keystore/create.js:45:12)
    at Command.<anonymous> (/private/tmp/kibana/kibana-7.3.0-darwin-x86_64/src/cli/command.js:111:20)
    at Command.listener (/private/tmp/kibana/kibana-7.3.0-darwin-x86_64/node_modules/commander/index.js:315:8)
    at Command.emit (events.js:189:13)
    at Command.parseArgs (/private/tmp/kibana/kibana-7.3.0-darwin-x86_64/node_modules/commander/index.js:651:12)
    at Command.parse (/private/tmp/kibana/kibana-7.3.0-darwin-x86_64/node_modules/commander/index.js:474:21)
    at Object.<anonymous> (/private/tmp/kibana/kibana-7.3.0-darwin-x86_64/src/cli_keystore/cli_keystore.js:49:9)

This appears to be because keystone-create only uses getData which doesn't chase the path.data setting:

kibana/src/cli_keystore/cli_keystore.js

Line 24 in 83e3db0

import { getData } from '../legacy/server/path';

kibana/src/legacy/server/path/index.js

Line 51 in 83e3db0

export const getData = () => findFile(DATA_PATHS);

kibana/src/legacy/server/path/index.js

Lines 32 to 36 in 83e3db0

	const DATA_PATHS = [
	process.env.DATA_PATH, //deprecated
	fromRoot('data'),
	'/var/lib/kibana'
	].filter(Boolean);

Note that if we use the deprecated environment variable DATA_PATH then everything is fine:

jason-mbp:kibana jason$ DATA_PATH=/tmp/kibana/data ./kibana-7.3.0-darwin-x86_64/bin/kibana-keystore create
Created Kibana keystore in /tmp/kibana/data/kibana.keystore
jason-mbp:kibana Jason$

Relates #32049

[elasticmachine]

Pinging @elastic/kibana-operations

[nreese]

duplicate of #31171

[jbudz]

We've moved the keystore to /etc/kibana/kibana/yml and it should now respect KBN_PATH_CONF. The data path is backwards compatible if a keystore exists in that location but when we get a chance we'll want to update homebrew

~/dev/kibana(master*) » mkdir foo                                                     
~/dev/kibana(master) » KBN_PATH_CONF=/home/jon/dev/kibana/foo node scripts/kibana_keystore.js create
Created Kibana keystore in /home/jon/dev/kibana/foo/kibana.keystore
~/dev/kibana(master) » ls foo                                                         
kibana.keystore

[scratchmex]

Just adding to this issue. If you installed Kibana via .deb or apt install when you run bin/kibana-keystore create without sudo it throws this error:

ivan@server:/usr/share/kibana$ bin/kibana-keystore create
FATAL CLI ERROR Error: ENOENT: no such file or directory, open '/usr/share/kibana/config/kibana.keystore'
    at Object.openSync (fs.js:443:3)
    at writeFileSync (fs.js:1194:35)
    at Keystore.save (/usr/share/kibana/src/legacy/server/keystore/keystore.js:65:27)
    at create (/usr/share/kibana/src/cli_keystore/create.js:43:12)
    at Command.<anonymous> (/usr/share/kibana/src/cli/command.js:113:20)
    at Command.listener (/usr/share/kibana/node_modules/commander/index.js:291:8)
    at Command.emit (events.js:198:13)
    at Command.parseArgs (/usr/share/kibana/node_modules/commander/index.js:672:12)
    at Command.parse (/usr/share/kibana/node_modules/commander/index.js:459:21)
    at Object.<anonymous> (/usr/share/kibana/src/cli_keystore/cli_keystore.js:69:9)

Instead you must run

ivan@server:/usr/share/kibana$ sudo bin/kibana-keystore create --allow-root
Created Kibana keystore in /etc/kibana/kibana.keystore

which succesfully creates the keystore:

ivan@server:/usr/share/kibana$ sudo ls -la /etc/kibana/
total 32
drwxr-s---   2 root kibana  4096 Dec 31 04:03 .
drwxr-xr-x 111 root root   12288 Dec 31 02:08 ..
-rw-r--r--   1 root kibana   130 Dec 31 04:03 kibana.keystore
-rw-rw----   1 root kibana  5211 Dec 31 03:27 kibana.yml
-rw-r--r--   1 root kibana   216 Dec  5 01:47 node.options