Ubuntu OS version upgrade

Review changes

CHANGELOG-0.10.md

@@ -36,6 +36,7 @@
 - [#1901](https://github.com/epiphany-platform/epiphany/issues/1901) - Make Epiphany upgrades selective (Kafka). Added new parameter for epicli (--upgrade-components)
 - [#2080](https://github.com/epiphany-platform/epiphany/issues/2080) - Update RHEL images in CI pipelines and documentation to the current latest 7.9 version
 - [#1859](https://github.com/epiphany-platform/epiphany/issues/1859) - Upgrade Open Distro for Elasticsearch to v1.13.x and elasticsearch-oss to v7.10.2
+- [#2142](https://github.com/epiphany-platform/epiphany/issues/2142) - Update Ubuntu 18.04-LTS images to the latest version
 
 ### Breaking changes
 - Feature name: `elasticsearch` removed in favor of feature name: `opendistro-for-elasticsearch`.

core/src/epicli/cli/helpers/os_images.py

@@ -15,7 +15,7 @@ def get_os_distro_normalized(vm_doc):
                 return expected_indicators[indicator]
     if vm_doc.provider == "aws":
         # Example public/official AMI names:
-        # - ubuntu/images/hvm-ssd/ubuntu-bionic-18.04-amd64-server-20200611
+        # - ubuntu/images/hvm-ssd/ubuntu-bionic-18.04-amd64-server-20210323
         # - RHEL-7.8_HVM_GA-20200225-x86_64-1-Hourly2-GP2
         # - CentOS 7.8.2003 x86_64
         for indicator in expected_indicators:

core/src/epicli/data/aws/defaults/infrastructure/virtual-machine.yml

@@ -14,7 +14,7 @@ specification:
   tags:
     - version: 0.4.2
   size: t2.micro
-  os_full_name: "ubuntu/images/hvm-ssd/ubuntu-bionic-18.04-amd64-server-20200611"
+  os_full_name: "ubuntu/images/hvm-ssd/ubuntu-bionic-18.04-amd64-server-20210323"
   os_type: linux
   ebs_optimized: false
   disks:

core/src/epicli/data/azure/defaults/infrastructure/virtual-machine.yml

@@ -17,7 +17,7 @@ specification:
     publisher: Canonical
     offer: UbuntuServer
     sku: 18.04-LTS
-    version: "18.04.202006101" # Never put latest on anything! Need to always pin the version number but testing we can get away with it
+    version: "18.04.202103151" # Never put latest on anything! Need to always pin the version number but testing we can get away with it
   storage_os_disk:
     delete_on_termination: false
     managed: false

docs/home/howto/OS_PATCHING.md

@@ -29,7 +29,7 @@ This document will help you decide how you should patch your OS. This is not a s
 For Epiphany v0.10 we recommend the following image (AMI):  
 
 - RHEL: `RHEL-7.9_HVM-20210208-x86_64-0-Hourly2-GP2` (kernel 3.10.0-1160.15.2.el7.x86_64),
-- Ubuntu: `ubuntu-bionic-18.04-amd64-server-20200611` (kernel 5.3.0-1028-aws).
+- Ubuntu: `ubuntu-bionic-18.04-amd64-server-20210323` (kernel 5.4.0-1041-aws).
 
 Note: For different supported OS versions this guide may be useful as well.
 
@@ -57,7 +57,7 @@ For more information, refer to [AWS Systems Manager User Guide](https://docs.aws
 For Epiphany v0.10 we recommend the following image (urn):  
 
 - RHEL: `RHEL:7-LVM:7.9.2020111202` (kernel 3.10.0-1160.el7.x86_64),
-- Ubuntu: `UbuntuServer:18.04-LTS:18.04.202006101` (kernel 5.3.0-1028-azure).
+- Ubuntu: `UbuntuServer:18.04-LTS:18.04.202103151` (kernel 5.4.0-1041-azure).
 
 Note: For different supported OS versions this guide may be useful as well.
 

docs/home/howto/UPGRADE.md

@@ -20,7 +20,38 @@ but upgrade for v0.3.1 is not possible due to the open [issue](https://github.co
 - repository: Adds the repository role needed for component installation in current Epiphany version
 - image_registry: Adds the image_registry role needed for offline installation in current Epiphany version
 
-*Note: The component upgrade takes the existing Ansible build output and based on that performs the upgrade of the currently supported components. If you need to upgrade your entire Epiphany cluster a **manual** upgrade of the input yaml is needed to the latest specification which then should be applied with `epicli apply...` after the offline upgrade which is described here.*
+*Note: The component upgrade takes the existing Ansible build output and based on that performs the upgrade of the currently supported components. If you need to re-apply your entire Epiphany cluster a **manual** adjustment of the input yaml is needed to the latest specification which then should be applied with `epicli apply...`. Please see [Run apply after upgrade](./UPGRADE.md#run-apply-after-upgrade) chapter for more details.
+*
+
+*Note about images' versions: Currently Epiphany does not support image's version upgrade during the process described above. Before you run `epicli apply...` on already upgraded environment, please make sure you set up versions of images used on machines according to versions used in pre-upgrade cluster (this needs to be set up manually according your old configuration yaml, and applied to configuration section for every defined machine in yaml generated from newer version). Please see [Run apply after upgrade](./UPGRADE.md#run-apply-after-upgrade) chapter for more details.*
+
+
+###  Run *apply* after *upgrade*
+
+There is possibility that you will have to reconfigure already upgraded cluster eg. add more Kubernetes nodes.
+
+#### Starting state:
+
+Cluster on which `epicli upgrade` command already ran. Changes described in [introduction](./UPGRADE.md#introduction) are already done.
+
+#### Expected state:
+
+Changed cluster configuration (eg. more kubernetes nodes added)
+
+#### The path to follow
+
+Here are steps to follow to apply changes on already upgraded cluster:
+
+1. Back up important data, we hope that you already have some backup in case of failure, or if not some of components might be backed up by `epicli backup` command described in [Backup](./BACKUP.md) section. And don't forget to make backups periodically.
+2. Manually patch all machines in cluster, see [os patching](./OS_PATCHING.md) section.
+3. Generate configuration yaml by using `epicli init` with `--full` option running from newer version of Epiphany. It's described in [prerequisites](./PREREQUISITES.md) section. 
+4. Apply configuration changes you have performed in pre-upgrade config yaml into later one generated above (key path, subscription and any other performed).
+5. Modify versions of images used for deployment: set versions copied from manifest.yml generated in older version of cluster (pre-upgrade), and attach them to config.yml generated in step 3. This (and also some other changes) could be done by using `sed`:
+```bash
+sed -i 's/"version_from_newer_file"/"version_from_older_file"/g' config.yml
+```
+6. Adjust other options you need to change (eg. increase number of kubernetes nodes).
+7. Run `epicli apply -f config.yml` and apply changes.
 
 ### Online upgrade