Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[deb/rpm] On upgrade, restart kibana service #82049

Merged
merged 12 commits into from
Nov 9, 2020

Conversation

jbudz
Copy link
Member

@jbudz jbudz commented Oct 29, 2020

This adds a new environment variable RESTART_ON_UPGRADE with a default value of true. In an upgrade scenario, Kibana will automatically restart.

Closes #9863

Release note:

Starting in 8.0 deb and rpm packages will restart on upgrade by default. This can be toggled by setting the environment variable RESTART_ON_UPGRADE in /etc/default/kibana

This adds a new environment variable KBN_RESTART_ON_UPGRADE with a
default value of true.  In an upgrade scenario Kibana will automatically
restart.

Closes elastic#9863
@jbudz jbudz added Team:Operations Team label for Operations Team v8.0.0 v7.11.0 labels Oct 29, 2020
@jbudz
Copy link
Member Author

jbudz commented Oct 29, 2020

Targeting 8.0 with a default restart behavior, 7.11 with support for the flag but no change. Needs docs changes and this is just the outline for testing ATM, review pending undraft

@jbudz
Copy link
Member Author

jbudz commented Oct 29, 2020

defaults / KBN_RESTART_ON_UPGRADE="true"

jon@debd1:~$ sudo dpkg -i kibana-7.9.3-amd64.deb
Selecting previously unselected package kibana.
(Reading database ... 98911 files and directories currently installed.)
Preparing to unpack kibana-7.9.3-amd64.deb ...
Unpacking kibana (7.9.3) ...
Setting up kibana (7.9.3) ...
Processing triggers for systemd (245.4-4ubuntu3.2) ...
jon@debd1:~$ sudo dpkg -i kibana/target/kibana-8.0.0-SNAPSHOT-amd64.deb
(Reading database ... 185777 files and directories currently installed.)
Preparing to unpack .../kibana-8.0.0-SNAPSHOT-amd64.deb ...
Stopping kibana service... OK
Unpacking kibana (8.0.0-SNAPSHOT) over (7.9.3) ...
Setting up kibana (8.0.0-SNAPSHOT) ...
Installing new version of config file /etc/default/kibana ...
Installing new version of config file /etc/kibana/kibana.yml ...
Installing new version of config file /etc/systemd/system/kibana.service ...
Restarting kibana service... OK
jon@debd1:~$ service kibana status
● kibana.service - Kibana
     Loaded: loaded (/etc/systemd/system/kibana.service; disabled; vendor preset: enabled)
     Active: active (running) since Thu 2020-10-29 21:20:09 UTC; 9s ago
       Docs: https://www.elastic.co
   Main PID: 8279 (node)
      Tasks: 11 (limit: 9334)
     Memory: 193.8M
     CGroup: /system.slice/kibana.service
             └─8279 /usr/share/kibana/bin/../node/bin/node /usr/share/kibana/bin/../src/cli/dist --loggi>

Oct 29 21:20:09 debd1 systemd[1]: Started Kibana.

KBN_RESTART_ON_UPGRADE="false"

root@debd1:/home/jon#q dpkg -i kibana-7.9.3-amd64.deb
Selecting previously unselected package kibana.
(Reading database ... 98911 files and directories currently installed.)
Preparing to unpack kibana-7.9.3-amd64.deb ...
Unpacking kibana (7.9.3) ...
Setting up kibana (7.9.3) ...
Processing triggers for systemd (245.4-4ubuntu3.2) ...
root@debd1:/home/jon# echo 'KBN_RESTART_ON_UPGRADE="false"' > /etc/default/kibana
root@debd1:/home/jon# sudo dpkg -i kibana/target/kibana-8.0.0-SNAPSHOT-amd64.deb
(Reading database ... 185777 files and directories currently installed.)
Preparing to unpack .../kibana-8.0.0-SNAPSHOT-amd64.deb ...
Stopping kibana service... OK
Unpacking kibana (8.0.0-SNAPSHOT) over (7.9.3) ...
Setting up kibana (8.0.0-SNAPSHOT) ...

Configuration file '/etc/default/kibana'
 ==> Modified (by you or by a script) since installation.
 ==> Package distributor has shipped an updated version.
   What would you like to do about it ?  Your options are:
    Y or I  : install the package maintainer's version
    N or O  : keep your currently-installed version
      D     : show the differences between the versions
      Z     : start a shell to examine the situation
 The default action is to keep your current version.
*** kibana (Y/I/N/O/D/Z) [default=N] ?
Installing new version of config file /etc/kibana/kibana.yml ...
Installing new version of config file /etc/systemd/system/kibana.service ...
root@debd1:/home/jon# service kibana status
● kibana.service - Kibana
     Loaded: loaded (/etc/systemd/system/kibana.service; disabled; vendor preset: enabled)
     Active: inactive (dead)
       Docs: https://www.elastic.co

Oct 29 21:20:09 debd1 systemd[1]: Started Kibana.
Oct 29 21:20:52 debd1 systemd[1]: Stopping Kibana...
Oct 29 21:20:52 debd1 systemd[1]: kibana.service: Succeeded.
Oct 29 21:20:52 debd1 systemd[1]: Stopped Kibana.
root@debd1:/home/jon# ^C
root@debd1:/home/jon#

https://s3-us-west-1.amazonaws.com/kibana.budzenski.com/82049/0b1d1f0/kibana-8.0.0-SNAPSHOT-amd64.deb
https://s3-us-west-1.amazonaws.com/kibana.budzenski.com/82049/0b1d1f0/kibana-8.0.0-SNAPSHOT-x86_64.rpm

@jbudz jbudz marked this pull request as ready for review October 29, 2020 21:35
@jbudz jbudz requested a review from a team as a code owner October 29, 2020 21:35
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-operations (Team:Operations)

@tylersmalley
Copy link
Contributor

I believe in the past we discussed removing any interaction with the service in our scripts. Have you had different thoughts? What does ES currently do?

@jbudz
Copy link
Member Author

jbudz commented Oct 30, 2020

This is modeled off ES, the main difference is UPGRADE_ON_RESTART in ES vs KBN_UPGRADE_ON_RESTART here. We could make them both the same name now that I'm thinking about it, environment variables are isolated in individual services.

This should remove interaction by default on the builds above (but with a restart) but we were referring to not stopping the service right? I'll get some builds together and linky here with it removed to see how it runs. I'm mainly concerned with the unknowns - server routes going stale, not sure if the the public files are cached in memory.

@jbudz
Copy link
Member Author

jbudz commented Nov 2, 2020

@elasticmachine merge upstream

@jbudz
Copy link
Member Author

jbudz commented Nov 2, 2020

I replaced KBN_RESTART_ON_UPGRADE with RESTART_ON_UPGRADE to avoid any confusion and to be consistent with elasticsearch. The variable is scoped to the service so it shouldn't have any conflicts.

For not stopping the service - I did an install where the service isn't stopped and didn't run into any issues. There were minimal code changes however, and another reason I'm trending towards this is migrations won't run. Any thoughts?

@jbudz
Copy link
Member Author

jbudz commented Nov 4, 2020

@elasticmachine merge upstream

@jbudz
Copy link
Member Author

jbudz commented Nov 6, 2020

@elasticmachine merge upstream

Copy link
Contributor

@tylersmalley tylersmalley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - manually tested upgrade from 7.9.3

@jbudz
Copy link
Member Author

jbudz commented Nov 9, 2020

@elasticmachine merge upstream

@kibanamachine
Copy link
Contributor

💚 Build Succeeded

Metrics [docs]

✅ unchanged

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@jbudz jbudz merged commit 0a71f2c into elastic:master Nov 9, 2020
@jbudz jbudz deleted the os/restart-on-upgrade branch November 9, 2020 15:50
jbudz added a commit to jbudz/kibana that referenced this pull request Nov 9, 2020
@kibanamachine kibanamachine added the backport missing Added to PRs automatically when the are determined to be missing a backport. label Nov 10, 2020
@kibanamachine
Copy link
Contributor

Looks like this PR has a backport PR but it still hasn't been merged. Please merge it ASAP to keep the branches relatively in sync.

1 similar comment
@kibanamachine
Copy link
Contributor

Looks like this PR has a backport PR but it still hasn't been merged. Please merge it ASAP to keep the branches relatively in sync.

@kibanamachine kibanamachine removed the backport missing Added to PRs automatically when the are determined to be missing a backport. label Nov 11, 2020
jbudz added a commit that referenced this pull request Nov 11, 2020
#82971)

* [deb/rpm] On upgrade, restart kibana service (#82049)

Co-authored-by: Kibana Machine <[email protected]>

* default restart behavior to off in 7.x

Co-authored-by: Kibana Machine <[email protected]>
@jbudz
Copy link
Member Author

jbudz commented Nov 11, 2020

7.11: 558ca4b

< 8.0 will have a default behavior of not restarting.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Upgrade stops running service
4 participants