draft of early 4.0 architecture updates

_topic_map.yml

@@ -57,6 +57,13 @@ Topics:
 - Name: OpenShift CCS modular docs conventions
   File: mod-docs-conventions-ocp
 ---
+Name: Architecture
+Dir: architecture
+Distros: openshift-*
+Topics:
+- Name: Architecture
+  File: architecture
+---  
 Name: Authentication
 Dir: authentication
 Distros: openshift-*

architecture/architecture.adoc

@@ -0,0 +1,42 @@
+// This assembly is included in the following assemblies:
+//
+// * n/a
+
+
+[id='architecture']
+= {product-title} Architecture
+{product-author}
+{product-version}
+:data-uri:
+:icons:
+:experimental:
+:toc: macro
+:toc-title:
+:prewrap!:
+:context: architecture
+
+toc::[]
+
+
+[IMPORTANT]
+====
+This assembly is a temporary placeholder to port the valid information from
+the 3.11 collection and include specific changes for 4.0 as that information
+becomes available.
+====
+
+include::modules/architecture-overview.adoc[leveloffset=+1]
+
+include::modules/installation-options.adoc[leveloffset=+2]
+
+include::modules/node-management.adoc[leveloffset=+2]
+
+include::modules/node-types.adoc[leveloffset=+2]
+
+include::modules/operators-overview.adoc[leveloffset=+2]
+
+include::modules/abstraction-layers.adoc[leveloffset=+2]
+
+include::modules/security-overview.adoc[leveloffset=+2]
+
+include::modules/machine-api-overview.adoc[leveloffset=+1]

modules/abstraction-layers.adoc

@@ -0,0 +1,23 @@
+// Module included in the following assemblies:
+//
+// * architecture/architecture.adoc
+
+[id='abstraction-layers-{context}']
+= {product-title} abstraction layers
+
+The Docker service provides the abstraction for packaging and creating
+Linux-based, lightweight container images. Kubernetes provides the
+cluster management and orchestrates containers on multiple hosts.
+
+{product-title} adds:
+
+- Source code management, builds, and deployments for developers
+- Managing and promoting images at scale as they flow through your system
+- Application management at scale
+- Team and user tracking for organizing a large developer organization
+- Networking infrastructure that supports the cluster
+
+.{product-title} Architecture Overview
+image::../images/architecture_overview.png[{product-title} Architecture Overview]
+
+The cluster uses a combination of master and worker nodes.

modules/architecture-overview.adoc

@@ -0,0 +1,70 @@
+// Module included in the following assemblies:
+//
+// * architecture/architecture.adoc
+
+[id='architecture-overview-{context}']
+= Architecture overview
+
+Like {product-title} v3, {product-title} v4 is a layered system designed to
+expose underlying container images and Kubernetes concepts as accurately as
+possible, with a focus on easy composition of applications by a developer.
+For example, after your cluster is running, install Ruby, push code, and add
+MySQL to the app.
+
+A major difference between {product-title} v3 and v4 is that v4 uses Operators
+as both the fundamental unit of the product and an option for easily deploying
+and managing utilities that your apps use.
+
+{product-title} has an Operator-based architecture of smaller, decoupled units
+that work together. It runs on top of a Kubernetes cluster, with data about the
+objects stored in etcd, a reliable clustered key-value store. Those services are
+broken down by function:
+
+* Operators, which run the core {product-title} services.
+* REST APIs, which expose each of the core objects:
+** Containers and images, which are the building blocks for deploying your
+applications.
+** Pods and services, which containers use to communicate with each other and
+proxy connections.
+** Projects and users, which provide the space and means for communities to
+organize and manage their content together.
+** Builds and image streams] allow you to
+build working images and react to new images.
+** Deployments, which expand support for the software development and deployment
+lifecycle.
+** Routes, which announce your service to the world.
+** Templates, which allow you to simultaneously create many objects that are
+based on customized parameters.
+* Controllers, which read those REST APIs, apply changes to other objects, and
+report status or write back to the object.
+
+Users make calls to the REST API to change the state of the system. Controllers
+use the REST API to read the user's desired state and then try to bring the
+other parts of the system into sync. For example, when you request a build, the
+REST APIs create a `build` object. The build controller sees that a new build has been created, and
+runs a process on the cluster to perform that build. When the build completes,
+the controller updates the build object via the REST API and the user sees that
+their build is complete.
+
+The controller pattern means that much of the functionality in {product-title}
+is extensible. The way that builds are run and launched can be customized
+independently of how images are managed, or how deployments happen. The controllers
+perform the "business logic" of the system, taking user actions and
+transforming them into reality. By customizing those controllers or replacing
+them with your own logic, you can implement different behaviors. From a system
+administration perspective, this also means that you can use the API to script common
+administrative actions on a repeating schedule. Those scripts are also
+controllers that watch for changes and take action. {product-title} makes the
+ability to customize the cluster in this way a first-class behavior.
+
+To make this possible, controllers use a reliable stream of changes to the
+system to sync their view of the system with what users are doing. This event
+stream pushes changes from etcd to the REST API and then to the controllers as
+soon as changes occur so changes can efficiently ripple through the system.
+However, because failures can occur at any time, the controllers
+must also be able to get the latest state of the system at start up and confirm
+that everything is in the right state. This resynchronization is important
+because it means that even if something goes wrong, you can
+restart the affected components, and the system confirms its status before it
+continues. Because the controllers can always bring the system into sync, the
+system eventually converges to your intent.

modules/cloud_installations.adoc

@@ -19,8 +19,45 @@ cluster. These details include ...
 With a customized cluster, you can specify more details of the installation, 
 such as ...
 
+When you install a fully-managed {product-title} cluster, you download the
+installer from link:try.openshift.com. This site manages:
+* Rest API for accounts
+* Registry tokens, which are the pull secrets that you use to obtain the required
+components
+* Cluster registration, which associates the cluster identity to your Red Hat
+account to facilitate usage metrics gathering
+
+In {product-title} version 4.0, the installer is a go binary that performs a
+series of file transformations on a set of assets. With the fully-managed option,
+you delegate the infrastructure bootstrapping and provisioning to the installer
+instead of doing it yourself. Because you do not use the installer to upgrade or
+update your cluster, if you do not highly customize your cluster, you run the
+installer only once.
+
+You use three sets of files during installation: an installation configuration
+file, Kubernetes manifests, and Ingition configurations for your machine types.
+
+The installation configuration file is transformed into Kubernetes manifests, and
+then the manifests are wrapped into Ignition configurations. The installer uses
+these Ingition configurations to create the cluster.
+
+The install configuration files are all pruned when you run the installer,
+so be sure to back up all configuration files that you want to use again.
+
 [IMPORTANT]
 ====
 You cannot modify the parameters that you set during installation. You have to
 ...
 ====
+
+////
+There are individual commands to perform the different actions in cluster creation
+if you want to try to make customizations, but you can run openshift-install
+create cluster to get the default cluster done quick.
+
+$ openshift-install --help
+$ openshift-install create install-config
+$ openshift-install create manifests
+$ openshift-install create ignition-configs
+$ openshift-install create cluster
+////

modules/installation-options.adoc

@@ -0,0 +1,31 @@
+// Module included in the following assemblies:
+//
+// * architecture/architecture.adoc
+
+[id='installation-options-{context}']
+= Installation options
+
+{product-title} offers two installation options: fully-managed infrastructure and bring your own
+infrastructure. In version 4.0, the fully-managed option allows you to install a
+cluster in Amazon Web Services (AWS) that runs on Red Hat CoreOS nodes. If you want to
+use any other cloud or install your cluster on-premise, use the bring your own
+infrastructure option to install your cluster on existing Red Hat Enterprise 
+Linux (RHEL) hosts.
+
+The fully-managed option offers full-stack automation to:
+
+* Manage compute
+* Manage operating system (RHCOS)
+* Manage control plane
+* Manage nodes
+
+The bring your own infrastructure option, you have more responsibilities.
+You must provide the hosts and update RHEL on them. {product-title} provides:
+
+* Managed control plane
+* Ansible to manage kubelet and runtime
+
+
+With both installation types, installation and upgrade both use a controller
+that constantly reconciles component versions as if it were any other Kubernetes
+controller.

modules/machine-api-overview.adoc

@@ -0,0 +1,56 @@
+// Module included in the following assemblies:
+//
+// * architecture/architecture.adoc
+
+[id='machine-api-overview-{context}']
+= Machine API overview
+
+On fully-managed {product-title} clusters, the Machine API performs all node
+management actions after the cluster installation finishes. Because of this
+system, {product-title} version 4.0 offers an elastic, dynamic provisioning
+method.
+
+The Machine API is a combination of primary resources that are based on the upstream
+link:https://github.com/kubernetes-sigs/cluster-api[Cluster API] project and
+custom {product-title} resources.
+
+The three primary resources are:
+
+`Machines`:: A fundamental unit that describes a node host. A `machine` has a
+class, which describes the types of compute nodes that are offered for different
+cloud platforms. For example, a `machine` type for a worker node on Amazon Web
+Services (AWS) might define a specific machine type and required metadata.
+`MachineClasses`:: A unit that defines a class of `machines` and facilitates
+configuration reuse across `machines` of the same class. This unit functions
+like a `StorageClass` for PersistentVolumeClaims.
+`MachineSets`:: Groups of machines. `MachineSets` are to `machines` as
+`ReplicaSets` are to `Pods`. If you need more `machines` or need to scale them down,
+you change the *replicas* field on the `MachineSet` to meet your compute need.
+
+
+The following custom resources add more capabilities to your cluster:
+
+`MachineAutoscaler`:: This resource automatically scales `machines` in
+a cloud. You can set the minimum and maximum scaling boundaries for nodes in a
+specified `MachineSet`, and the `MachineAutoscaler` maintains that range of nodes.
+`MachineHealthChecker`:: This resource detects when a machine is unhealthy,
+deletes it, and, on supported platforms, makes a new machine.
+`ClusterAutoscaler`:: This resource is based on the upstream
+link:https://github.com/kubernetes/autoscaler/tree/master/cluster-autoscaler[Cluster Autoscaler]
+project. In the {product-title} implementation, it is integrated with the
+Cluster API by extending the `MachineSet` API.
+`ClusterAutoscalerOperator`:: Instead of interacting with the `ClusterAutoscaler`
+itself, you use its Operator. The `ClusterAutoscalerOperator` manages
+the `ClusterAutoscaler` deployment. With this Operator, you can set cluster-wide
+scaling limits for resources such as cores, nodes, memory, and GPU.
+and so on. You can set the priority so that the cluster prioritizes pods so that
+new nodes are not brought online for less important pods. You can also set the
+ScalingPolicy so you can scale up nodes but not scale them down.
+
+
+In 3.x, you couldn't roll out a multi-az architecture easily because the cluster
+did not manage machine provisioning. It's easier in 4.0. Each machinset is scoped
+to a single az so the installer will actually send out machinesets across AZs
+on your behalf. And then because your compute is actually dynamic here, and in 
+the face of a zone failure, you always have a zone for when you need to rebalance
+your machines. the autoscaler proviedes best-effort balancing over the life of a cluster.

modules/node-management.adoc

@@ -0,0 +1,46 @@
+// Module included in the following assemblies:
+//
+// * architecture/architecture.adoc
+
+[id='node-management-{context}']
+= Node management in fully-managed {product-title}
+
+The fully-managed version of {product-title} version 4.0 integrates management of
+the container operating system and cluster management. Because the cluster manages
+both its updates and updates to RHCOS, {product-title} provides an opinionated
+lifecycle management experience that simplifies the orchestration of node upgrades.
+
+{product-title} employs three DaemonSets and controllers to simplify node management:
+
+* The `machine-config-controller` coordinates machine upgrades.
+* The `machine-config-daemon` DaemonSet is a subset of the Ingition configuration that
+applies the specified machine configuration and controls kubelet configuration.
+* The `machine-config-server` DaemonSet provides the Ignition config to new hosts.
+
+These tools orchestrate operating system updates and configuration changes to
+the hosts by using standard Kuberentes-style constructs. A `machine-config-daemon`
+DaemonSet runs on each compute node in the cluster and watches for changes in
+the machine configuration for it to apply. The machine configuration is a subset
+of the Ignition configuration. The `machine-config-daemon` reads the machine configuration to see
+if it needs to do an `os tree` update, if it should apply a series of `systemd`
+kubelet file changes, configuration changes, or other changes to the
+operating system or {product-title} configuration.
+
+The masters also run the `machine-config-controller` process that monitors all of the cluster nodes
+and orchestrates their configuration updates. So if you try to apply
+an update or configuration change to a node on the cluster, the `machine-config-controller`
+directs a node to update. The node sees that it needs to change, drains off its
+pods, applies the update, and reboots. This process is key to the success of
+managing {product-title} and RHCOS updates together.
+
+The `machine-config-server` applies configurations to nodes as they join the
+cluster. It orchestrates configuration to nodes and changes to the operating system
+and is used in both cluster installation and node maintenance. The
+`machine-config-server` components upgrade the operating system and controls the Ignition
+configuration for nodes.
+
+The `bootkube` process calls the `machine-config-server` component when the 
+{product-title} installer bootstraps the initial master node. After installation,
+the `machine-config-server` runs in the cluster.  It reads the `machine-config`
+custom resource definitions (CRDs) and serves the required Ignition configurations
+to new nodes when they join the cluster.

modules/node-types.adoc

@@ -0,0 +1,14 @@
+// Module included in the following assemblies:
+//
+// * architecture/architecture.adoc
+
+[id='node-types-{context}']
+= Node types in {product-title}
+
+{product-title} uses bootstrap, master, and worker nodes. The bootstrap node
+provides the initial configuration to clusters. Master nodes run the cluster
+infrastructure and required components. Instead of being grouped into a `MachineSet`,
+they are a series of standalone machine API resources. Extra controls apply to
+master nodes to prevent you from deleting all master nodes and breaking your
+cluster. The worker nodes drive compute workloads. Each type of worker node is
+governed by a specific machine pool that autoscales them.