Ubuntu OS version upgrade

Upgrade documentation

CHANGELOG-0.10.md

@@ -36,6 +36,7 @@
 - [#1901](https://github.com/epiphany-platform/epiphany/issues/1901) - Make Epiphany upgrades selective (Kafka). Added new parameter for epicli (--upgrade-components)
 - [#2080](https://github.com/epiphany-platform/epiphany/issues/2080) - Update RHEL images in CI pipelines and documentation to the current latest 7.9 version
 - [#1859](https://github.com/epiphany-platform/epiphany/issues/1859) - Upgrade Open Distro for Elasticsearch to v1.13.x and elasticsearch-oss to v7.10.2
+- [#2142](https://github.com/epiphany-platform/epiphany/issues/2142) - Update Ubuntu 18.04-LTS images to the latest version
 
 ### Breaking changes
 - Feature name: `elasticsearch` removed in favor of feature name: `opendistro-for-elasticsearch`.

core/src/epicli/cli/helpers/os_images.py

@@ -15,7 +15,7 @@ def get_os_distro_normalized(vm_doc):
                 return expected_indicators[indicator]
     if vm_doc.provider == "aws":
         # Example public/official AMI names:
-        # - ubuntu/images/hvm-ssd/ubuntu-bionic-18.04-amd64-server-20200611
+        # - ubuntu/images/hvm-ssd/ubuntu-bionic-18.04-amd64-server-20210323
         # - RHEL-7.8_HVM_GA-20200225-x86_64-1-Hourly2-GP2
         # - CentOS 7.8.2003 x86_64
         for indicator in expected_indicators:

core/src/epicli/data/aws/defaults/infrastructure/virtual-machine.yml

@@ -14,7 +14,7 @@ specification:
   tags:
     - version: 0.4.2
   size: t2.micro
-  os_full_name: "ubuntu/images/hvm-ssd/ubuntu-bionic-18.04-amd64-server-20200611"
+  os_full_name: "ubuntu/images/hvm-ssd/ubuntu-bionic-18.04-amd64-server-20210323"
   os_type: linux
   ebs_optimized: false
   disks:

core/src/epicli/data/azure/defaults/infrastructure/virtual-machine.yml

@@ -17,7 +17,7 @@ specification:
     publisher: Canonical
     offer: UbuntuServer
     sku: 18.04-LTS
-    version: "18.04.202006101" # Never put latest on anything! Need to always pin the version number but testing we can get away with it
+    version: "18.04.202103151" # Never put latest on anything! Need to always pin the version number but testing we can get away with it
   storage_os_disk:
     delete_on_termination: false
     managed: false

docs/home/howto/OS_PATCHING.md

@@ -29,7 +29,7 @@ This document will help you decide how you should patch your OS. This is not a s
 For Epiphany v0.10 we recommend the following image (AMI):  
 
 - RHEL: `RHEL-7.9_HVM-20210208-x86_64-0-Hourly2-GP2` (kernel 3.10.0-1160.15.2.el7.x86_64),
-- Ubuntu: `ubuntu-bionic-18.04-amd64-server-20200611` (kernel 5.3.0-1028-aws).
+- Ubuntu: `ubuntu-bionic-18.04-amd64-server-20210323` (kernel 5.4.0-1041-aws).
 
 Note: For different supported OS versions this guide may be useful as well.
 
@@ -57,7 +57,7 @@ For more information, refer to [AWS Systems Manager User Guide](https://docs.aws
 For Epiphany v0.10 we recommend the following image (urn):  
 
 - RHEL: `RHEL:7-LVM:7.9.2020111202` (kernel 3.10.0-1160.el7.x86_64),
-- Ubuntu: `UbuntuServer:18.04-LTS:18.04.202006101` (kernel 5.3.0-1028-azure).
+- Ubuntu: `UbuntuServer:18.04-LTS:18.04.202103151` (kernel 5.4.0-1041-azure).
 
 Note: For different supported OS versions this guide may be useful as well.
 

docs/home/howto/UPGRADE.md

@@ -20,7 +20,10 @@ but upgrade for v0.3.1 is not possible due to the open [issue](https://github.co
 - repository: Adds the repository role needed for component installation in current Epiphany version
 - image_registry: Adds the image_registry role needed for offline installation in current Epiphany version
 
-*Note: The component upgrade takes the existing Ansible build output and based on that performs the upgrade of the currently supported components. If you need to upgrade your entire Epiphany cluster a **manual** upgrade of the input yaml is needed to the latest specification which then should be applied with `epicli apply...` after the offline upgrade which is described here.*
+*Note: The component upgrade takes the existing Ansible build output and based on that performs the upgrade of the currently supported components. If you need to re-apply your entire Epiphany cluster a **manual** adjustment of the input yaml is needed to the latest specification which then should be applied with `epicli apply...`. Please see [Run apply after upgrade](./UPGRADE.md#run-apply-after-upgrade) chapter for more details.
+*
+
+*Note about upgrade from pre-0.8 Epiphany: If you run upgrade from version older than 0.8 you should make sure that you've got enough disk space on master (which is used as repository host) host. If you didn't extend OS disk on master during deployment process you probably have only 32GB disk which is not enough to properly upgrade cluster (at least 40GB is needed). Before you run upgrade please extend os disks on master machine according to cloud provider documentation: [AWS](https://aws.amazon.com/premiumsupport/knowledge-center/expand-root-ebs-linux/), [Azure](https://docs.microsoft.com/en-us/azure/virtual-machines/linux/expand-disks).*
 
 ### Online upgrade
 
@@ -114,6 +117,67 @@ The `epicli upgrade` command has additional flags:
    epicli upgrade -b /buildoutput/ --upgrade-components "kafka,filebeat"
    ```
 
+###  Run *apply* after *upgrade*
+
+Currently Epiphany does not fully support apply after upgrade. There is a possibility to re-apply configuration from newer version of Epicli but this needs some manual work from Administrator. Re-apply on already upgraded cluster needs to be called with `--no-infra` option to skip Terraform part of configuration.
+If you plan modify any infrastructure unit (eg. add Kubernetes Node) you need to create machine by yourself and attach it into configuration yaml. While running `epicli apply...` on already upgraded cluster you should use config yamls generated in newer version of Epiphany and apply changes you had in older one.
+If the cluster is upgraded to version 0.8 or newer you need also add additional feature mapping for repository role as shown on example below:
+
+```yaml
+---
+kind: epiphany-cluster
+name: clustername
+provider: azure
+specification:
+  admin_user:
+    key_path: /path/to/id_rsa
+    name: operations
+  components:
+    repository:
+      count: 0  # Set repository to 0 since it's introduced in v0.8
+    kafka:
+      count: 1
+    kubernetes_master:
+      count: 1
+    kubernetes_node:
+      count: 2
+    load_balancer:
+      count: 1
+    logging:
+      count: 1
+    monitoring:
+      count: 1
+    postgresql:
+      count: 1
+    rabbitmq:
+      count: 0
+    ignite:
+      count: 0
+    opendistro_for_elasticsearch:
+      count: 0
+  name: clustername
+  prefix: 'prefix'
+title: Epiphany cluster Config
+---
+kind: configuration/feature-mapping
+title: Feature mapping to roles
+provider: azure
+name: default
+specification:
+  roles_mapping:
+    kubernetes_master:
+    - kubernetes-master
+    - helm
+    - applications
+    - node-exporter
+    - filebeat
+    - firewall
+    - vault
+    - repository      # add repository here
+    - image-registry  # add image-registry here
+...
+```
+
 ## How to upgrade Kafka
 
 ### Kafka upgrade