Updated Password Tool to add default user and password to the filebeat.yml when changing passwords. #2989

CarlosALgit · 2024-06-06T11:14:11Z

Related issue
#1369

Description

As seen on related issue, the user for Filebeat connection was wazuh before 4.3.x version. When you tried to migrate to a newer version and then change the passwords, the filebeat.yml file remained witht the same username that was wazuh. This breaks the installations as the user by default for Filebeat connection is admin from 4.3.x versions onwards. More information can be seen here: #1369 (comment)

First aim

So, the aim of this PR is to change the Password Tool so that when a password change is requested for all users, it ensures the filebeat.yml file that the username is admin and the password is updated to the new one.

To do that I added the username: admin to this line:

conf="$(awk '{sub("username: .*", "username: admin"); sub("password: .*", "password: '"${adminpass}"'")}1' /etc/filebeat/filebeat.yml)"

More changes

In addition, while doing the tests, I realized that the way the Password Tool works is that when it detects that the password variable exists in the Filebeat Keystore, it assumes that in the filebeat.yml file ${username} is used as username and ${password} as password.
But it could be that there was plain text instead of variables in the filebeat.yml, and this would break the operation. So we decided that, in addition, we would add that when it is detected that the Keystore variables are being used, the filebeat.yml is updated and what is there is replaced with the Filebeat Keystore variables.
We divided the change in two operations. The first one will check if there is a variable on the Filebeat Keystore with the name password and, if so, we update the password with the new one and change the filebeat.yml with the ${password} variable. The second one will be inside the first one and, if there is a variable called password on the Filebeat Keystore, then we check for a variable named username and, if it's found, we update the filebeat.yml username field with the ${username} variable in order to add more sturdiness.
Also a log was added to report the user about these changes.

To do that I added these lines:

if filebeat keystore list | grep -q password ; then
    eval "echo ${adminpass} | filebeat keystore add password --force --stdin ${debug}"
    conf="$(awk '{sub("password: .*", "password: ${password}")}1' /etc/filebeat/filebeat.yml)"
    echo "${conf}" > /etc/filebeat/filebeat.yml
    common_logger "Updated password on Filebeat Keystore. Also updated filebeat.yml file to use the Keystore password."
    if filebeat keystore list | grep -q username ; then
        conf="$(awk '{sub("username: .*", "username: ${username}")}1' /etc/filebeat/filebeat.yml)"
        echo "${conf}" > /etc/filebeat/filebeat.yml
        common_logger "Updated filebeat.yml file to use the Keystore username."
    fi
else

Tests

Tests for the plain text case

I manually changed the username to other than admin to see if it's updated to admin when passwords are changed.

# Wazuh - Filebeat configuration file
output.elasticsearch.hosts:
        - 127.0.0.1:9200
#        - <elasticsearch_ip_node_2>:9200 
#        - <elasticsearch_ip_node_3>:9200

output.elasticsearch:
  protocol: https
  username: asdfasdfsafasdfas
  password: S*LSmDOcugQn80Rdu7Yl?WnH1TDFQRMa
  ssl.certificate_authorities:
    - /etc/filebeat/certs/root-ca.pem
  ssl.certificate: "/etc/filebeat/certs/filebeat.pem"
  ssl.key: "/etc/filebeat/certs/filebeat-key.pem"
setup.template.json.enabled: true
setup.template.json.path: '/etc/filebeat/wazuh-template.json'
setup.template.json.name: 'wazuh'
setup.ilm.overwrite: true
setup.ilm.enabled: false

filebeat.modules:
  - module: wazuh
    alerts:
      enabled: true
    archives:
      enabled: false

And after the passwords are changed with:

root@ubuntu2204:/home/vagrant# bash wazuh-passwords-tool.sh -a -v -au wazuh -ap t5wmO4Wa8?nDi3RTIXlVGTzlmS91*XtS

The username is updated to admin and the passwords is updated to the new one too:

# Wazuh - Filebeat configuration file
output.elasticsearch.hosts:
        - 127.0.0.1:9200
#        - <elasticsearch_ip_node_2>:9200 
#        - <elasticsearch_ip_node_3>:9200

output.elasticsearch:
  protocol: https
  username: admin
  password: ?EIJd4*QcnyMoblnNzmjC5AW*e7XUYA5
  ssl.certificate_authorities:
    - /etc/filebeat/certs/root-ca.pem
  ssl.certificate: "/etc/filebeat/certs/filebeat.pem"
  ssl.key: "/etc/filebeat/certs/filebeat-key.pem"
setup.template.json.enabled: true
setup.template.json.path: '/etc/filebeat/wazuh-template.json'
setup.template.json.name: 'wazuh'
setup.ilm.overwrite: true
setup.ilm.enabled: false

filebeat.modules:
  - module: wazuh
    alerts:
      enabled: true
    archives:
      enabled: false

Tests for the Filebeat Keystore case

We are going to test if with Filebeat Keystore variables configured, the Password Tool changes the plain text on the `filebeat.yml` to the variables used on the Keystore.

Check if the Keystore has the password variable:

root@ubuntu2204:/home/vagrant# filebeat keystore list
username
password

Manually change the username and password values to plain text:

# Wazuh - Filebeat configuration file
output.elasticsearch.hosts:
        - 127.0.0.1:9200
#        - <elasticsearch_ip_node_2>:9200 
#        - <elasticsearch_ip_node_3>:9200

output.elasticsearch:
  protocol: https
  username: asdasd
  password: lkjhlkjh

And after changing the password with:

root@ubuntu2204:/home/vagrant# sudo bash wazuh-passwords-tool.sh -a
11/06/2024 09:52:47 INFO: Wazuh API admin credentials not provided, Wazuh API passwords not changed.
11/06/2024 09:52:51 INFO: Updated password on Filebeat Keystore. Also updated filebeat.yml file to use the Keystore password.
11/06/2024 09:52:51 INFO: Updated filebeat.yml file to use the Keystore username.
11/06/2024 09:52:55 INFO: The password for user admin is 7ewWSXVPKgQe+Df57SOo4QkHwQIRvvyH
11/06/2024 09:52:55 INFO: The password for user kibanaserver is D1*Li20Qjcg2?YyveXNik9xKSyyFsofx
11/06/2024 09:52:55 INFO: The password for user kibanaro is cL9*IiM0Z1jrsUfhfwSk?7yTiADz527z
11/06/2024 09:52:55 INFO: The password for user logstash is ?jHavEo9+n4MAIoxfQcwCd16.S63riET
11/06/2024 09:52:55 INFO: The password for user readall is 03Yurh6LbSKbU2p?YGpPSMRuTrp?O*3n
11/06/2024 09:52:55 INFO: The password for user snapshotrestore is cn7Kc4qGW47fBRV?vVZaHjyZ21Bpmy*f
11/06/2024 09:52:55 WARNING: Wazuh indexer passwords changed. Remember to update the password in the Wazuh dashboard and Filebeat nodes if necessary, and restart the services.
root@ubuntu2204:/home/vagrant# filebeat test output
elasticsearch: https://127.0.0.1:9200...
  parse url... OK
  connection...
    parse host... OK
    dns lookup... OK
    addresses: 127.0.0.1
    dial up... OK
  TLS...
    security: server's certificate chain verification is enabled
    handshake... OK
    TLS version: TLSv1.2
    dial up... OK
  talk to server... OK
  version: 7.10.2

We can see that the log reports the user about the change made and the filebeat.yml file looks like this:

# Wazuh - Filebeat configuration file
output.elasticsearch.hosts:
        - 127.0.0.1:9200
#        - <elasticsearch_ip_node_2>:9200 
#        - <elasticsearch_ip_node_3>:9200

output.elasticsearch:
  protocol: https
  username: ${username}
  password: ${password}
  ssl.certificate_authorities:
    - /etc/filebeat/certs/root-ca.pem
  ssl.certificate: "/etc/filebeat/certs/wazuh-server.pem"
  ssl.key: "/etc/filebeat/certs/wazuh-server-key.pem"
setup.template.json.enabled: true
setup.template.json.path: '/etc/filebeat/wazuh-template.json'
setup.template.json.name: 'wazuh'
setup.ilm.overwrite: true
setup.ilm.enabled: false

filebeat.modules:
  - module: wazuh
    alerts:
      enabled: true
    archives:
      enabled: false

logging.level: info
logging.to_files: true
logging.files:
  path: /var/log/filebeat
  name: filebeat

But, if the user wants to use the user on plain text and the password on the Filbeat Keystore. For that, we did also a test.
The filebeat.yml file:

# Wazuh - Filebeat configuration file
output.elasticsearch.hosts:
        - 127.0.0.1:9200
#        - <elasticsearch_ip_node_2>:9200 
#        - <elasticsearch_ip_node_3>:9200

output.elasticsearch:
  protocol: https
  username: admin
  password: ${password}
...

We first delete the username of the Keystore. And the output when changing passwords:

root@ubuntu2204:/home/vagrant# sudo filebeat keystore remove username
successfully removed key: username
root@ubuntu2204:/home/vagrant# sudo nano /etc/filebeat/filebeat.yml 
root@ubuntu2204:/home/vagrant# sudo bash wazuh-passwords-tool.sh -a
11/06/2024 09:56:39 INFO: Wazuh API admin credentials not provided, Wazuh API passwords not changed.
11/06/2024 09:56:43 INFO: Updated password on Filebeat Keystore. Also updated filebeat.yml file to use the Keystore password.
11/06/2024 09:56:47 INFO: The password for user admin is 4p*VpnkfEM?R8WP09FGR96H1P32TM.N2
11/06/2024 09:56:47 INFO: The password for user kibanaserver is KmDi2zorpKDgIz2Jv?eLlVjXcRd?1fAa
11/06/2024 09:56:47 INFO: The password for user kibanaro is S2EulqX.hbPF379671EIgzd+QfC5fKWd
11/06/2024 09:56:47 INFO: The password for user logstash is x2eEBo0NF2KWACwiDyg9yp+CdeobeFnK
11/06/2024 09:56:47 INFO: The password for user readall is C*mGrMoUi9R0Zc6?h0r92rM8A8Wr.EPr
11/06/2024 09:56:47 INFO: The password for user snapshotrestore is GDCs.G2*tZufd?2LvCY0sLFfvu25kWw1
11/06/2024 09:56:47 WARNING: Wazuh indexer passwords changed. Remember to update the password in the Wazuh dashboard and Filebeat nodes if necessary, and restart the services.
root@ubuntu2204:/home/vagrant# filebeat test output
elasticsearch: https://127.0.0.1:9200...
  parse url... OK
  connection...
    parse host... OK
    dns lookup... OK
    addresses: 127.0.0.1
    dial up... OK
  TLS...
    security: server's certificate chain verification is enabled
    handshake... OK
    TLS version: TLSv1.2
    dial up... OK
  talk to server... OK
  version: 7.10.2

The filebeat.yml file remains like this:

# Wazuh - Filebeat configuration file
output.elasticsearch.hosts:
        - 127.0.0.1:9200
#        - <elasticsearch_ip_node_2>:9200 
#        - <elasticsearch_ip_node_3>:9200

output.elasticsearch:
  protocol: https
  username: admin
  password: ${password}

Final version

As the final improved version of this PR we thought it would be better to always use Filebeat Keystore in order to grant more security. So, if the user had configured the username or password in plain text in the filebeat.yml file, he will be warned that this will be changed when changing the passwords and the default admin user and the new password will be stored in the Filebeat Keystore. To change the user you can refer to the updated documentation with this change.

Tests for the final version

Test with plain text both on username and password

The filebeat.yml:

# Wazuh - Filebeat configuration file
output.elasticsearch.hosts:
        - 127.0.0.1:9200
#        - <elasticsearch_ip_node_2>:9200 
#        - <elasticsearch_ip_node_3>:9200

output.elasticsearch:
  protocol: https
  username: admin
  password: 123123123123132131
  ssl.certificate_authorities:
...

When changing the passwords:

root@ubuntu2204:/home/vagrant# bash wazuh-passwords-tool.sh -a
17/06/2024 10:45:15 INFO: Wazuh API admin credentials not provided, Wazuh API passwords not changed.
17/06/2024 10:45:19 WARNING: The user and password configured in the filebeat.yml file will be updated and stored in Filebeat Keystore.
17/06/2024 10:45:19 INFO: Updated username and password in Filebeat Keystore. Also updated filebeat.yml file to use the Filebeat Keystore username and password.
17/06/2024 10:45:38 INFO: The password for user admin is B7+NbOda+?O.y8EhMqSG06lSfWGmiRH9
17/06/2024 10:45:38 INFO: The password for user kibanaserver is s0PzZ*0Alr5ILjHgXZR1YSdKAq+XEUHp
17/06/2024 10:45:38 INFO: The password for user kibanaro is ?JSp1IU1dOl7wbpkFPypOnCerVbI.Spn
17/06/2024 10:45:38 INFO: The password for user logstash is z?GpHgUv8ha.E22iqOPDzs8785FedJ9v
17/06/2024 10:45:38 INFO: The password for user readall is 8SA4pt?U+dDdFbNl6mRbE6luxlI?DjDb
17/06/2024 10:45:38 INFO: The password for user snapshotrestore is 8T+7JYpgB.ETDCKwv76.7cp9Fno2m*fC
17/06/2024 10:45:38 WARNING: Wazuh indexer passwords changed. Remember to update the password in the Wazuh dashboard and Filebeat nodes if necessary, and restart the services.
root@ubuntu2204:/home/vagrant# nano /etc/filebeat/filebeat.yml 
root@ubuntu2204:/home/vagrant# filebeat test output
elasticsearch: https://127.0.0.1:9200...
  parse url... OK
  connection...
    parse host... OK
    dns lookup... OK
    addresses: 127.0.0.1
    dial up... OK
  TLS...
    security: server's certificate chain verification is enabled
    handshake... OK
    TLS version: TLSv1.2
    dial up... OK
  talk to server... OK
  version: 7.10.2

The final filebeat.yml:

# Wazuh - Filebeat configuration file
output.elasticsearch.hosts:
        - 127.0.0.1:9200
#        - <elasticsearch_ip_node_2>:9200 
#        - <elasticsearch_ip_node_3>:9200

output.elasticsearch:
  protocol: https
  username: ${username}
  password: ${password}
...

Tests with one variable in plain text and the other on the correct variable

The filebeat.yml:

# Wazuh - Filebeat configuration file
output.elasticsearch.hosts:
        - 127.0.0.1:9200
#        - <elasticsearch_ip_node_2>:9200 
#        - <elasticsearch_ip_node_3>:9200

output.elasticsearch:
  protocol: https
  username: ${username}
  password: 1231231321aaa
....

Changing the passwords:

root@ubuntu2204:/home/vagrant# bash wazuh-passwords-tool.sh -a
17/06/2024 10:47:00 INFO: Wazuh API admin credentials not provided, Wazuh API passwords not changed.
17/06/2024 10:47:04 WARNING: The user and password configured in the filebeat.yml file will be updated and stored in Filebeat Keystore.
17/06/2024 10:47:04 INFO: Updated username and password in Filebeat Keystore. Also updated filebeat.yml file to use the Filebeat Keystore username and password.
17/06/2024 10:47:25 INFO: The password for user admin is 4.UJ4j6CGa0oIVHnuf2xPq*JAsM5vh7T
17/06/2024 10:47:25 INFO: The password for user kibanaserver is Va0JC8v.X0MBVCrMjH7Usa31QMHeKd+h
17/06/2024 10:47:25 INFO: The password for user kibanaro is SJEbg7?24ZRA992bH4imXK?jvZ6c.Z9j
17/06/2024 10:47:25 INFO: The password for user logstash is lKLWeW5wUj1LbVMbU+X1XKz?2SN5pQ.m
17/06/2024 10:47:25 INFO: The password for user readall is Hqks?uoU.rkPZM5MRLYKBfJ6GTjThr0c
17/06/2024 10:47:25 INFO: The password for user snapshotrestore is gnJpGv.wzUhiV+iUb6oX6MG?CDlb7Y.V
17/06/2024 10:47:25 WARNING: Wazuh indexer passwords changed. Remember to update the password in the Wazuh dashboard and Filebeat nodes if necessary, and restart the services.
root@ubuntu2204:/home/vagrant# nano /etc/filebeat/filebeat.yml 
root@ubuntu2204:/home/vagrant# filebeat test output
elasticsearch: https://127.0.0.1:9200...
  parse url... OK
  connection...
    parse host... OK
    dns lookup... OK
    addresses: 127.0.0.1
    dial up... OK
  TLS...
    security: server's certificate chain verification is enabled
    handshake... OK
    TLS version: TLSv1.2
    dial up... OK
  talk to server... OK
  version: 7.10.2

Tests with both on correct variables

The filebeat.yml:

# Wazuh - Filebeat configuration file
output.elasticsearch.hosts:
        - 127.0.0.1:9200
#        - <elasticsearch_ip_node_2>:9200 
#        - <elasticsearch_ip_node_3>:9200

output.elasticsearch:
  protocol: https
  username: ${username}
  password: ${password}
...

Changing the passwords:

root@ubuntu2204:/home/vagrant# bash wazuh-passwords-tool.sh -a
17/06/2024 10:43:11 INFO: Wazuh API admin credentials not provided, Wazuh API passwords not changed.
17/06/2024 10:43:16 INFO: Updated username and password in Filebeat Keystore. Also updated filebeat.yml file to use the Filebeat Keystore username and password.
17/06/2024 10:43:36 INFO: The password for user admin is +WeiI?bqhEzj6D*05NP.GDelSH9e*5uV
17/06/2024 10:43:36 INFO: The password for user kibanaserver is dD0*G6AgQvuIac6Vgc9Xw34I9bddOC?2
17/06/2024 10:43:36 INFO: The password for user kibanaro is PidYbvqSTAYnU0wT7?2tVak+MM.fGBcD
17/06/2024 10:43:36 INFO: The password for user logstash is 15.J*6Js6q5rL9t8IRfafY68jmnOAhs1
17/06/2024 10:43:36 INFO: The password for user readall is 7i0TnCmYZw.U14ek*obSrtX76av46k7z
17/06/2024 10:43:36 INFO: The password for user snapshotrestore is CkG5M9l7NtEJd.PfP2Pd41Ab5CXYr+vB
17/06/2024 10:43:36 WARNING: Wazuh indexer passwords changed. Remember to update the password in the Wazuh dashboard and Filebeat nodes if necessary, and restart the services.
root@ubuntu2204:/home/vagrant# filebeat test output
elasticsearch: https://127.0.0.1:9200...
  parse url... OK
  connection...
    parse host... OK
    dns lookup... OK
    addresses: 127.0.0.1
    dial up... OK
  TLS...
    security: server's certificate chain verification is enabled
    handshake... OK
    TLS version: TLSv1.2
    dial up... OK
  talk to server... OK
  version: 7.10.2

update filebeat.yml username and password by default and log message

2d96134

CarlosALgit self-assigned this Jun 6, 2024

CarlosALgit linked an issue Jun 6, 2024 that may be closed by this pull request

Incorrect password change in filebeat.yml when changing the admin password #1369

Closed

CarlosALgit requested a review from a team June 6, 2024 11:14

split changes tolerating user in plain text and password in the keystore

6ee3efc

CarlosALgit mentioned this pull request Jun 11, 2024

Add custom Filebeat user information and delete obsolete information about ossec.conf file on Wazuh server. wazuh/wazuh-documentation#7382

Merged

c-bordon previously approved these changes Jun 13, 2024

View reviewed changes

changed to always use filebeat keystore

c6f3ca0

CarlosALgit dismissed c-bordon’s stale review via c6f3ca0 June 17, 2024 10:51

CarlosALgit requested a review from c-bordon June 17, 2024 10:52

c-bordon approved these changes Jun 18, 2024

View reviewed changes

c-bordon merged commit f4b2431 into 4.9.0 Jun 18, 2024
13 of 16 checks passed

c-bordon deleted the bug/1369-incorrect-password-change-in-filebeatyml branch June 18, 2024 15:57

davidcr01 mentioned this pull request Jul 19, 2024

Post release tasks for 4.8.1 #3045

Closed

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Updated Password Tool to add default user and password to the filebeat.yml when changing passwords. #2989

Updated Password Tool to add default user and password to the filebeat.yml when changing passwords. #2989

CarlosALgit commented Jun 6, 2024 •

edited by davidcr01

Loading

Updated Password Tool to add default user and password to the filebeat.yml when changing passwords. #2989

Updated Password Tool to add default user and password to the filebeat.yml when changing passwords. #2989

Conversation

CarlosALgit commented Jun 6, 2024 • edited by davidcr01 Loading

Description

First aim

More changes

Tests

Final version

Tests for the final version

CarlosALgit commented Jun 6, 2024 •

edited by davidcr01

Loading