Skip to content

Commit

Permalink
Merge branch 'main' into test-db-benchmarks
Browse files Browse the repository at this point in the history
  • Loading branch information
MasonM authored Oct 16, 2024
2 parents 8391702 + 5be8cbf commit 0ba27e0
Show file tree
Hide file tree
Showing 11 changed files with 57 additions and 141 deletions.
8 changes: 8 additions & 0 deletions docs/fields.md
Original file line number Diff line number Diff line change
Expand Up @@ -2399,7 +2399,11 @@ Mutex holds Mutex configuration

- [`synchronization-mutex-tmpl-level-legacy.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/synchronization-mutex-tmpl-level-legacy.yaml)

- [`synchronization-mutex-tmpl-level.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/synchronization-mutex-tmpl-level.yaml)

- [`synchronization-mutex-wf-level-legacy.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/synchronization-mutex-wf-level-legacy.yaml)

- [`synchronization-mutex-wf-level.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/synchronization-mutex-wf-level.yaml)
</details>

### Fields
Expand Down Expand Up @@ -3291,7 +3295,11 @@ MutexStatus contains which objects hold mutex locks, and which objects this work

- [`synchronization-mutex-tmpl-level-legacy.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/synchronization-mutex-tmpl-level-legacy.yaml)

- [`synchronization-mutex-tmpl-level.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/synchronization-mutex-tmpl-level.yaml)

- [`synchronization-mutex-wf-level-legacy.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/synchronization-mutex-wf-level-legacy.yaml)

- [`synchronization-mutex-wf-level.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/synchronization-mutex-wf-level.yaml)
</details>

### Fields
Expand Down
135 changes: 8 additions & 127 deletions docs/synchronization.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,10 +24,6 @@ data:
template: "2" # Two instances of template can run at a given time in particular namespace
```
!!! Warning
Each synchronization block may only refer to either a semaphore or a mutex.
If you specify both only the semaphore will be locked.
## Workflow-level Synchronization
You can limit parallel execution of workflows by using the same synchronization reference.
Expand All @@ -36,44 +32,14 @@ In this example the synchronization key `workflow` is configured as limit `"1"`,

Using a semaphore configured by a `ConfigMap`:

```yaml
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: synchronization-wf-level-
spec:
entrypoint: hello-world
synchronization:
semaphores:
- configMapKeyRef:
name: my-config
key: workflow
templates:
- name: hello-world
container:
image: busybox
command: [echo]
args: ["hello world"]
```yaml title="examples/synchronization-wf-level.yaml"
--8<-- "examples/synchronization-wf-level.yaml:12"
```

Using a mutex is equivalent to a limit `"1"` semaphore:

```yaml
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: synchronization-wf-level-
spec:
entrypoint: hello-world
synchronization:
mutexes:
- name: workflow
templates:
- name: hello-world
container:
image: busybox
command: [echo]
args: ["hello world"]
```yaml title="examples/synchronization-mutex-wf-level.yaml"
--8<-- "examples/synchronization-mutex-wf-level.yaml:3"
```

## Template-level Synchronization
Expand All @@ -85,77 +51,16 @@ This applies even when multiple steps or tasks within a workflow or different wo

Using a semaphore configured by a `ConfigMap`:

```yaml
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: synchronization-tmpl-level-
spec:
entrypoint: synchronization-tmpl-level-example
templates:
- name: synchronization-tmpl-level-example
steps:
- - name: synchronization-acquire-lock
template: acquire-lock
arguments:
parameters:
- name: seconds
value: "{{item}}"
withParam: '["1","2","3","4","5"]'
- name: acquire-lock
synchronization:
semaphores:
- configMapKeyRef:
name: my-config
key: template
container:
image: alpine:latest
command: [sh, -c]
args: ["sleep 10; echo acquired lock"]
```yaml title="examples/synchronization-tmpl-level.yaml"
--8<-- "examples/synchronization-tmpl-level.yaml:11"
```

Using a mutex will limit to a single concurrent execution of the template:

```yaml
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: synchronization-tmpl-level-
spec:
entrypoint: synchronization-tmpl-level-example
templates:
- name: synchronization-tmpl-level-example
steps:
- - name: synchronization-acquire-lock
template: acquire-lock
arguments:
parameters:
- name: seconds
value: "{{item}}"
withParam: '["1","2","3","4","5"]'
- name: acquire-lock
synchronization:
mutexes:
- name: template
container:
image: alpine:latest
command: [sh, -c]
args: ["sleep 10; echo acquired lock"]
```yaml title="examples/synchronization-mutex-tmpl-level.yaml"
--8<-- "examples/synchronization-mutex-tmpl-level.yaml:3"
```

Examples:

1. [Workflow level semaphore](https://github.com/argoproj/argo-workflows/blob/main/examples/synchronization-wf-level.yaml)
1. [Workflow level mutex](https://github.com/argoproj/argo-workflows/blob/main/examples/synchronization-mutex-wf-level.yaml)
1. [Step level semaphore](https://github.com/argoproj/argo-workflows/blob/main/examples/synchronization-tmpl-level.yaml)
1. [Step level mutex](https://github.com/argoproj/argo-workflows/blob/main/examples/synchronization-mutex-tmpl-level.yaml)
1. [Legacy workflow level semaphore](https://github.com/argoproj/argo-workflows/blob/main/examples/synchronization-wf-level-legacy.yaml)
1. [Legacy workflow level mutex](https://github.com/argoproj/argo-workflows/blob/main/examples/synchronization-mutex-wf-level-legacy.yaml)
1. [Legacy step level semaphore](https://github.com/argoproj/argo-workflows/blob/main/examples/synchronization-tmpl-level-legacy.yaml)
1. [Legacy step level mutex](https://github.com/argoproj/argo-workflows/blob/main/examples/synchronization-mutex-tmpl-level-legacy.yaml)

## Queuing

When a workflow cannot acquire a lock it will be placed into a ordered queue.
Expand Down Expand Up @@ -204,30 +109,6 @@ Examples:
!!! Warning
If a Workflow is at the front of the queue and it needs to acquire multiple locks, all other Workflows that also need those same locks will wait. This applies even if the other Workflows only wish to acquire a subset of those locks.

## Legacy

In workflows prior to 3.6 you can only specify one lock in any one workflow or template using either a mutex:

```yaml
synchronization:
mutex:
...
```

or a semaphore:

```yaml
synchronizaion:
semamphore:
...
```

Specifying both would not work in <3.6, only the semaphore would be used.

The single `mutex` and `semaphore` syntax still works in version 3.6 but is considered deprecated.
Both the `mutex` and the `semaphore` will be taken in version 3.6 with this syntax.
This syntax can be mixed with `mutexes` and `semaphores`, all locks will be required.

## Other Parallelism support

You can also [restrict parallelism at the Controller-level](parallelism.md).
11 changes: 7 additions & 4 deletions examples/synchronization-mutex-tmpl-level-legacy.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
# This example demonstrates the use of a Synchronization Mutex lock on template execution. Mutex lock limits
# only one of the template execution across the workflows in the namespace which has same Mutex lock.

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
Expand Down Expand Up @@ -28,18 +27,22 @@ spec:

- name: acquire-lock
synchronization:
mutex:
mutex: # deprecated: v3.5 and before
name: welcome
# mutexes: # v3.6 and after
# - name: welcome
container:
image: alpine:latest
command: [sh, -c]
args: ["sleep 20; echo acquired lock"]

- name: acquire-lock-1
synchronization:
mutex:
mutex: # deprecated: v3.5 and before
name: test
# mutexes: # v3.6 and after
# - name: test
container:
image: alpine:latest
command: [sh, -c]
args: ["sleep 50; echo acquired lock"]
args: ["sleep 50; echo acquired lock"]
9 changes: 6 additions & 3 deletions examples/synchronization-mutex-tmpl-level.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
# This example demonstrates the use of a Synchronization Mutex lock on template execution. Mutex lock limits
# only one of the template execution across the workflows in the namespace which has same Mutex lock.

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
Expand Down Expand Up @@ -28,17 +27,21 @@ spec:

- name: acquire-lock
synchronization:
mutexes:
mutexes: # v3.6 and after
- name: welcome
# mutex: # deprecated: v3.5 and before
# name: welcome
container:
image: alpine:latest
command: [sh, -c]
args: ["sleep 20; echo acquired lock"]

- name: acquire-lock-1
synchronization:
mutexes:
mutexes: # v3.6 and after
- name: test
# mutex: # deprecated: v3.5 and before
# name: test
container:
image: alpine:latest
command: [sh, -c]
Expand Down
6 changes: 4 additions & 2 deletions examples/synchronization-mutex-wf-level-legacy.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,10 @@ metadata:
spec:
entrypoint: hello-world
synchronization:
mutex:
name: test
mutex: # deprecated: v3.5 and before
name: test
# mutexes: # v3.6 and after
# - name: test
templates:
- name: hello-world
container:
Expand Down
4 changes: 3 additions & 1 deletion examples/synchronization-mutex-wf-level.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,10 @@ metadata:
spec:
entrypoint: hello-world
synchronization:
mutexes:
mutexes: # v3.6 and after
- name: test
# mutex: # deprecated: v3.5 and before
# name: test
templates:
- name: hello-world
container:
Expand Down
6 changes: 5 additions & 1 deletion examples/synchronization-tmpl-level-legacy.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -27,10 +27,14 @@ spec:

- name: acquire-lock
synchronization:
semaphore:
semaphore: # deprecated: v3.5 and before
configMapKeyRef:
name: my-config
key: template
# semaphores: # v3.6 and after
# - configMapKeyRef:
# name: my-config
# key: template
container:
image: alpine:latest
command: [sh, -c]
Expand Down
6 changes: 5 additions & 1 deletion examples/synchronization-tmpl-level.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -27,10 +27,14 @@ spec:

- name: acquire-lock
synchronization:
semaphores:
semaphores: # v3.6 and after
- configMapKeyRef:
name: my-config
key: template
# semaphore: # deprecated: v3.5 and before
# configMapKeyRef:
# name: my-config
# key: template
container:
image: alpine:latest
command: [sh, -c]
Expand Down
6 changes: 5 additions & 1 deletion examples/synchronization-wf-level-legacy.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,14 @@ metadata:
spec:
entrypoint: hello-world
synchronization:
semaphore:
semaphore: # deprecated: v3.5 and before
configMapKeyRef:
name: my-config
key: workflow
# semaphores: # v3.6 and after
# - configMapKeyRef:
# name: my-config
# key: workflow
templates:
- name: hello-world
container:
Expand Down
6 changes: 5 additions & 1 deletion examples/synchronization-wf-level.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,14 @@ metadata:
spec:
entrypoint: hello-world
synchronization:
semaphores:
semaphores: # v3.6 and after
- configMapKeyRef:
name: my-config
key: workflow
# semaphore: # deprecated: v3.5 and before
# configMapKeyRef:
# name: my-config
# key: workflow
templates:
- name: hello-world
container:
Expand Down
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ markdown_extensions:
- admonition
- md_in_html
- pymdownx.details
- pymdownx.snippets
- pymdownx.superfences:
custom_fences:
# support mermaid diagrams per https://squidfunk.github.io/mkdocs-material/reference/diagrams/#configuration
Expand Down

0 comments on commit 0ba27e0

Please sign in to comment.