Skip to content

Commit

Permalink
Merge pull request #2025 from OctopusDeploy/samj/workers-cloud-improv…
Browse files Browse the repository at this point in the history
…ements

Improve Workers docs for Octopus Cloud
  • Loading branch information
steve-fenton-octopus authored Dec 7, 2023
2 parents f225029 + 4dc96bc commit 4552538
Show file tree
Hide file tree
Showing 5 changed files with 90 additions and 32 deletions.
2 changes: 2 additions & 0 deletions dictionary-octopus.txt
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ bootstrap
bootstrapped
bootstrapper
cicd
comms
COMPUTERNAME
createartifact
combinational
Expand Down Expand Up @@ -96,6 +97,7 @@ WCAG
webfonts
WEBSVR
WIXUI
workerpool
workertools
xlarge
xmark
Expand Down
6 changes: 5 additions & 1 deletion src/pages/docs/infrastructure/workers/built-in-worker.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
layout: src/layouts/Default.astro
pubDate: 2023-01-01
modDate: 2023-01-01
modDate: 2023-11-17
title: Built-in Worker
description: Octopus Server comes with a built-in worker which enables you to conveniently run parts of your deployment process on the Octopus Server without the need to install a Tentacle or other deployment target. This page describes how to configure the built-in worker for a variety of scenarios.
navOrder: 2
Expand All @@ -11,6 +11,10 @@ Octopus Server comes with a built-in worker which enables you to conveniently ru

This page describes how to configure the built-in worker for a variety of scenarios.

:::div{.hint}
The built-in worker is only available on [self-hosted Octopus](/docs/getting-started#self-hosted-octopus) instances. [Octopus Cloud](/docs/octopus-cloud) customers have access to [dynamic worker pools](/docs/infrastructure/workers/dynamic-worker-pools), which provides a pre-configured worker on-demand.
:::

## Built-in Worker

When the built-in Worker is executed, the Octopus Server spawns a new process for Calamari. This conveniently allows a default Octopus set up to enable features like running script steps on the server and Azure deployments. However, this convenience comes at a cost: **security**.
Expand Down
38 changes: 29 additions & 9 deletions src/pages/docs/infrastructure/workers/dynamic-worker-pools.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,13 @@ description: Dynamic Worker pools are used in our cloud product to dynamically c
navOrder: 50
---

Dynamic Worker Pools are a special type of [worker pool](/docs/infrastructure/workers/worker-pools/) used by Octopus Cloud. They use [workers](/docs/infrastructure/workers) provided by Octopus, and they don't require users to configure any infrastructure.
Dynamic Worker Pools provide a quick and easy way to use [workers](/docs/infrastructure/workers) for your deployments. They are a special type of [worker pool](/docs/infrastructure/workers/worker-pools) available on [Octopus Cloud](/docs/octopus-cloud).

Dynamic workers are isolated virtual machines, created on-demand to run your deployments and runbook steps. They run on Azure and are created and managed by Octopus Cloud, which means you don't need to configure or maintain additional infrastructure.

## On demand

Dynamic workers are created on demand and leased to an Octopus Cloud instance for a limited time before being destroyed.
A dynamic worker is created on demand and leased to an Octopus Cloud instance for a limited time before being destroyed.

:::div{.info}
Octopus Cloud will automatically destroy dynamic workers as soon as one of these conditions is met:
Expand All @@ -24,24 +26,42 @@ Please reach out to [[email protected]](mailto:[email protected]) if you nee

## Isolated

Each worker is provisioned exclusively to a specific customer, and is completely isolated from other customers.
Each worker VM is provisioned exclusively to a specific customer, and is completely isolated from other customers.

## Dynamic Worker Images

Each dynamic worker pool can specify the worker image used. Ubuntu Linux 22.04 is the default. Windows Server Core 2019 (end-of-life, see [below](#windows-server-core-2019)) and 2022 worker images are also available.
Each dynamic worker pool uses a specific worker image. This is a VM image which determines the operating system (and OS version) running on the worker. You can edit a dynamic worker pool to change which image is used.

When you sign up to [Octopus Cloud](/docs/octopus-cloud) (or create a new [space](/docs/administration/spaces)), you automatically receive a worker pool for the `Ubuntu (default)` image, and a worker pool for the `Windows (default)` image.

The full list of available worker images includes both specific operating system versions (e.g., `Ubuntu Linux 22.04`), and also generic "default" options such as `Ubuntu (default)`. Choosing the default option means that your worker will use the latest stable worker image when it is released.

The current default images are:

- `Ubuntu (default)``Ubuntu Linux 22.04`
- `Windows (default)``Windows Server Core 2019`

:::div{.warning}
The `Windows (default)` image will change to Windows 2022 on 4 December 2023, and Windows 2019 images will be deprecated on 9 January 2024. You are advised to test your deployment processes with our Windows 2022 images. Please refer to [Windows 2019 end-of-life](/docs/infrastructure/workers/dynamic-worker-pools/windows-2019-end-of-life) for further details.
:::

### Choosing an Image

The default image is a good option to choose if you are:

Editing a dynamic worker pool allows you to modify the image used.
- Running a simple script that doesn't require specific tools or operating system versions
- Running a step [inside a container](/docs/projects/steps/execution-containers-for-workers)

The available worker images list specific operating system versions (e.g., `Ubuntu Linux 22.04`) but also generic "default" options such as `Ubuntu (default)`. Choosing the default option means that your worker will get the latest stable worker image released. This is a good option to choose if you're running a basic script that doesn't have any dependencies on specific tool or operating system versions.
If you're writing a script that relies on a specific version of tooling (e.g., helm), then we recommend using [execution containers for workers](/docs/projects/steps/execution-containers-for-workers) to run the script in a Docker container with the tool versions you need.

If you're writing a script that relies on a specific version of tooling (e.g., helm), then we recommend choosing a specific worker image, instead of the "default" options, to prevent worker image upgrades from impacting your deployments.
Alternatively, you can choose a specific worker image, instead of the "default" options, to prevent worker image upgrades from impacting your deployments.

|Type | Pros | Cons |
|-----|------|------|
| Default (eg `Ubuntu (default)`) | Automatically uses the latest image. Deployments will continue to work even when a worker image is marked as deprecated or decommissioned.| The versions of dependencies (e.g., helm) are not fixed. Deployments that rely on specific versions of dependencies or operating system specific features may break during upgrades. |
| Default (eg `Ubuntu (default)`) | Automatically uses the latest image. Deployments will continue to work even when a worker image is marked as deprecated or decommissioned.| The versions of dependencies (e.g. helm) are not fixed. Deployments that rely on specific versions of dependencies or operating system specific features may break during upgrades. |
| Specific (e.g., `Ubuntu Linux 22.04`) | The version of the operating system and dependencies are fixed and can be relied upon. | When a worker image is marked as deprecated, warnings will start to appear in your deployment logs. When a worker image is decommissioned, you will need to take action to update your worker pool or deployments will fail. |

## Deprecation
### Deprecation

When an image is marked as deprecated, you will see warnings in the Octopus UI, and in the deployment log. After a suitable deprecation period, deployments will start to fail if they target an image that has hit end-of-life.

Expand Down
45 changes: 33 additions & 12 deletions src/pages/docs/infrastructure/workers/index.mdx
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
layout: src/layouts/Default.astro
pubDate: 2023-01-01
modDate: 2023-01-01
modDate: 2023-11-17
title: Workers
description: External workers are machines that can execute steps that don't need to be performed on the Octopus Server or deployment targets.
navOrder: 30
Expand All @@ -13,7 +13,7 @@ import ConfigureSshConnectionWorker from 'src/shared-content/tentacle/configure-
import ConfigureWindowsListeningWorker from 'src/shared-content/tentacle/configure-windows-listening-worker.include.md';
import ConfigureWindowsPollingWorker from 'src/shared-content/tentacle/configure-windows-polling-worker.include.md';
import InstallTentacleManager from 'src/shared-content/tentacle/install-tentacle-manager.include.md';
import PoweShellNamedMutex from 'src/shared-content/tentacle/powershell-named-mutex.include.md';
import PowerShellNamedMutex from 'src/shared-content/tentacle/powershell-named-mutex.include.md';
import Workers from 'src/shared-content/infrastructure/workers.include.md';
import WorkersIndividualProcessExclusively from 'src/shared-content/tentacle/workers-individual-process-exclusively.include.md';
import WorkersMultipleSimultaneousProcesses from 'src/shared-content/tentacle/workers-multiple-simultaneous-processes.include.md';
Expand All @@ -38,15 +38,20 @@ The following steps always run inside the Octopus Server process (and do not run

A worker receives instruction from the Octopus Server to execute a step, it executes the step using Calamari and returns the logs and any collected artifacts to the Octopus Server.

There are two kinds of workers you can use in Octopus:

1. [The built-in worker](#built-in-worker) (default)
1. [External workers](#external-workers)

:::div{.hint}
Workers are assigned at the start of a deployment or runbook, not at the time the individual step executes.
:::

There are three kinds of workers you can use in Octopus:

1. [The built-in worker](#built-in-worker) - available on self-hosted Octopus
1. [Dynamic workers](#dynamic-workers) - available on Octopus Cloud
1. [External workers](#external-workers) - manually configured

:::div{.success}
[Octopus Cloud](/docs/octopus-cloud) uses [dynamic workers](/docs/infrastructure/workers/dynamic-worker-pools) by default, which provides an on-demand worker running on an Ubuntu or Windows VM. Dynamic workers are managed by Octopus Cloud, and are included with your Octopus Cloud subscription.
:::

## Ignoring Workers \{#ignoring-workers}

Octopus works out-of-the-box without setting up workers. You can run all deployment processes, run script steps on the built-in worker, deploy to Azure and run AWS and Terraform steps, without further setup. The built-in worker is available in a default Octopus set up, and Octopus workers are designed so that, if you aren't using external workers, none of your deployment processes need to be worker aware.
Expand All @@ -55,7 +60,7 @@ The choices of built-in worker, built-in worker running in a separate account, a

## Migrating to Workers \{#migrating-to-workers}

Octopus workers provide a way to move work off the built-in worker. This lets you move the work away from the Octopus Server and onto external workers without the need to update the deployment process. Learn [how to use the default worker pool to move steps off the Octopus Server](/docs/infrastructure/workers/worker-pools/#Using-the-default-pool-to-stop-running-scripts-on-the-server).
Octopus workers provide a way to move work off the built-in worker. This lets you move the work away from the Octopus Server and onto external workers without the need to update the deployment process. Learn [how to use the default worker pool to move steps off the Octopus Server](/docs/infrastructure/workers/worker-pools/#default-worker-pool).

## Built-in Worker \{#built-in-worker}

Expand All @@ -67,18 +72,34 @@ Adding a worker to the default worker pool will disable the built-in worker, and

Learn about the security implications and how to configure the [built-in worker](/docs/infrastructure/workers/built-in-worker).

:::div{.hint}
The built-in worker is only available on [self-hosted Octopus](/docs/getting-started#self-hosted-octopus) instances. [Octopus Cloud](/docs/octopus-cloud) customers have access to [dynamic worker pools](/docs/infrastructure/workers/dynamic-worker-pools), which provides a pre-configured worker on-demand.
:::

## Dynamic Workers \{#dynamic-workers}

**Dynamic workers** are on-demand workers managed by [Octopus Cloud](/docs/octopus-cloud), which means you don't need to configure or maintain additional infrastructure. Dynamic workers provides an Ubuntu or Windows VM running as a pre-configured tentacle worker.

Dynamic worker pools are included with all Octopus Cloud instances, and are the default option when creating new worker steps in your deployments and runbooks.

Learn more about configuring and using [dynamic worker pools](/docs/infrastructure/workers/dynamic-worker-pools) and selecting an OS image for your worker tasks.

## External Workers \{#external-workers}

An **External Worker** is either:
- A [Windows](/docs/infrastructure/deployment-targets/tentacle/windows/) or [Linux](/docs/infrastructure/deployment-targets/tentacle/linux) Tentacle.
- An [SSH machine](/docs/infrastructure/deployment-targets/linux/ssh-target) that has been registered with the Octopus Server as a worker.

The setup of a worker is the same as setting up a deployment target as a [Windows Tentacle target](/docs/infrastructure/deployment-targets/tentacle/windows/) or an [SSH target](/docs/infrastructure/deployment-targets/linux/ssh-target), except that instead of being added to an environment, a worker is added to a worker pool.
The setup of a worker is the same as setting up a deployment target as a [Windows Tentacle target](/docs/infrastructure/deployment-targets/tentacle/windows/) or an [SSH target](/docs/infrastructure/deployment-targets/linux/ssh-target), except that instead of being added to an environment, a worker is added to a [worker pool](/docs/infrastructure/workers/worker-pools/).

Using external workers allows delegating work to a machine other than the Octopus Server. This can make the server more secure and allow scaling. When Octopus executes a step on an external worker, it's the external worker that executes Calamari; no user-provided script executes on the Octopus Server itself.

Workers have machine policies, are health checked, and run Calamari, just like deployment targets.

:::div{.success}
[Octopus Cloud](/docs/octopus-cloud) customers can choose to use the included [dynamic worker pools](/docs/infrastructure/workers/dynamic-worker-pools) (enabled by default), and/or register their own external workers.
:::

## Registering an External Worker \{#registering-an-external-worker}

Once the Tentacle or SSH machine has been configured, workers can be added using the [Web Portal](#registering-workers-in-the-web-portal), the [Octopus Deploy REST API](/docs/octopus-rest-api/), the [Octopus.Clients library](/docs/octopus-rest-api/octopus.client) or with the Tentacle executable. Only a user with the `ConfigureServer` permission can add or edit workers.
Expand Down Expand Up @@ -132,14 +153,14 @@ The new Polling Tentacle will automatically show up in the Workers list.
Tentacle workers can also register with the server using the Tentacle executable (version 3.22.0 or later), for example:

```
.\Tentacle.exe register-worker --instance MyInstance --server "https://example.com/" --comms-style TentaclePassive --apikey "API-CS0SW5SQJNLUBQCUBPK8LZY3KYO" --workerpool "Default Worker Pool"
.\Tentacle.exe register-worker --instance MyInstance --server "https://example.com/" --comms-style TentaclePassive --apikey "API-YOUR-KEY" --workerpool "Default Worker Pool"
```

Use `TentacleActive` instead of `TentaclePassive` to register a polling Tentacle worker.

The Tentacle executable can also be used to deregister workers, for example:
```
.\Tentacle.exe deregister-worker --instance MyInstance --server "https://example.com/" --apikey "API-CS0SW5SQJNLUBQCUBPK8LZY3KYO"
.\Tentacle.exe deregister-worker --instance MyInstance --server "https://example.com/" --apikey "API-YOUR-KEY"
```

:::div{.hint}
Expand All @@ -164,7 +185,7 @@ Default pools attached to cloud targets allow co-location of workers and targets

<WorkersIndividualProcessExclusively />

<PoweShellNamedMutex />
<PowerShellNamedMutex />

### Workers in HA setups

Expand Down
Loading

0 comments on commit 4552538

Please sign in to comment.