Skip to content

Commit

Permalink
Add metrics from V4 and single sidecar support (#619)
Browse files Browse the repository at this point in the history 
<!-- Thank you for submitting a PR to Hera! 🚀 -->

**Pull Request Checklist**
- [x] Fixes #610
- [ ] Tests added
- [ ] Documentation/examples added
- [ ] [Good commit messages](https://cbea.ms/git-commit/) and/or PR
title
<!-- Also remember to sign off commits or the DCO check will fail on
your PR! -->

**Description of PR**
<!-- If not linked to an issue, please describe your changes here -->

Currently, custom metrics objects are not available as they were in V4.
This PR adds those objects.

CC: @abschm

<!-- Piece of cake! ✨🍰✨ -->

---------

Signed-off-by: Flaviu Vadan <[email protected]>
  • Loading branch information
@flaviuvadan
flaviuvadan authored May 9, 2023
1 parent 814e39c commit a1d704e
Show file tree
Hide file tree
Showing 34 changed files with 1,507 additions and 24 deletions.
177 changes: 177 additions & 0 deletions docs/examples/workflows/upstream/custom_metrics.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,177 @@
# Custom Metrics

> Note: This example is a replication of an Argo Workflow example in Hera. The upstream example can be [found here](https://github.com/argoproj/argo-workflows/blob/master/examples/custom-metrics.yaml).



=== "Hera"

```python linenums="1"
from hera.workflows import (
Container,
Counter,
Gauge,
Histogram,
Label,
Parameter,
Steps,
Workflow,
models as m,
)

random_int = Container(
name="random-int",
image="alpine:latest",
command=["sh", "-c"],
args=["RAND_INT=$((1 + RANDOM % 10)); echo $RAND_INT; echo $RAND_INT > /tmp/rand_int.txt"],
metrics=[
Histogram(
name="random_int_step_histogram",
help="Value of the int emitted by random-int at step level",
when="{{status}} == Succeeded",
buckets=[2.01, 4.01, 6.01, 8.01, 10.01],
value="{{outputs.parameters.rand-int-value}}",
),
Gauge(
name="duration_gauge",
labels=Label(key="name", value="random-int"),
help="Duration gauge by name",
realtime=True,
value="{{duration}}",
),
],
outputs=Parameter(
name="rand-int-value", global_name="rand-int-value", value_from=m.ValueFrom(path="/tmp/rand_int.txt")
),
)
flakey = Container(
name="flakey",
image="python:alpine3.6",
command=["python", "-c"],
args=["import random; import sys; exit_code = random.choice([0, 1, 1]); sys.exit(exit_code)"],
metrics=Counter(
name="result_counter",
labels=[Label(key="name", value="flakey"), Label(key="status", value="{{status}}")],
help="Count of step execution by result status",
value="1",
),
)

with Workflow(
generate_name="hello-world-",
entrypoint="steps",
metrics=Gauge(
name="duration_gauge",
labels=Label(key="name", value="workflow"),
help="Duration gauge by name",
realtime=True,
value="{{workflow.duration}}",
),
) as w:
with Steps(
name="steps",
metrics=Gauge(
name="duration_gauge",
labels=Label(key="name", value="steps"),
help="Duration gauge by name",
realtime=True,
value="{{duration}}",
),
):
random_int()
flakey()
```

=== "YAML"

```yaml linenums="1"
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: hello-world-
spec:
entrypoint: steps
metrics:
prometheus:
- gauge:
realtime: true
value: '{{workflow.duration}}'
help: Duration gauge by name
labels:
- key: name
value: workflow
name: duration_gauge
templates:
- metrics:
prometheus:
- gauge:
realtime: true
value: '{{duration}}'
help: Duration gauge by name
labels:
- key: name
value: steps
name: duration_gauge
name: steps
steps:
- - name: random-int
template: random-int
- - name: flakey
template: flakey
- container:
args:
- RAND_INT=$((1 + RANDOM % 10)); echo $RAND_INT; echo $RAND_INT > /tmp/rand_int.txt
command:
- sh
- -c
image: alpine:latest
metrics:
prometheus:
- help: Value of the int emitted by random-int at step level
histogram:
buckets:
- 2.01
- 4.01
- 6.01
- 8.01
- 10.01
value: '{{outputs.parameters.rand-int-value}}'
name: random_int_step_histogram
when: '{{status}} == Succeeded'
- gauge:
realtime: true
value: '{{duration}}'
help: Duration gauge by name
labels:
- key: name
value: random-int
name: duration_gauge
name: random-int
outputs:
parameters:
- globalName: rand-int-value
name: rand-int-value
valueFrom:
path: /tmp/rand_int.txt
- container:
args:
- import random; import sys; exit_code = random.choice([0, 1, 1]); sys.exit(exit_code)
command:
- python
- -c
image: python:alpine3.6
metrics:
prometheus:
- counter:
value: '1'
help: Count of step execution by result status
labels:
- key: name
value: flakey
- key: status
value: '{{status}}'
name: result_counter
name: flakey
```

191 changes: 191 additions & 0 deletions docs/examples/workflows/upstream/dag_custom_metrics.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,191 @@
# Dag Custom Metrics

> Note: This example is a replication of an Argo Workflow example in Hera. The upstream example can be [found here](https://github.com/argoproj/argo-workflows/blob/master/examples/dag-custom-metrics.yaml).



=== "Hera"

```python linenums="1"
from hera.workflows import (
DAG,
Container,
Counter,
Gauge,
Label,
Parameter,
Workflow,
)

echo = Container(
name="echo",
image="alpine:3.7",
command=["echo", "{{inputs.parameters.message}}"],
inputs=[Parameter(name="message"), Parameter(name="tag")],
metrics=[
Gauge(
name="playground_workflow_duration_task_seconds",
help="Duration gauge by task name in seconds - task level",
labels=[
Label(key="playground_task_name", value="{{inputs.parameters.tag}}"),
Label(key="status", value="{{status}}"),
],
realtime=False,
value="{{duration}}",
),
Counter(
name="playground_workflow_result_task_counter",
help="Count of task execution by result status - task level",
labels=[
Label(key="playground_task_name_counter", value="{{inputs.parameters.tag}}"),
Label(key="status", value="{{status}}"),
],
value="1",
),
],
)

with Workflow(
generate_name="dag-task-",
entrypoint="dag-task",
metrics=[
Gauge(
name="playground_workflow_duration",
labels=[
Label(key="playground_id_workflow", value="test"),
Label(key="status", value="{{workflow.status}}"),
],
realtime=False,
value="{{workflow.duration}}",
help="Duration gauge by workflow level",
),
Counter(
name="playground_workflow_result_counter",
labels=[
Label(key="playground_id_workflow_counter", value="test"),
Label(key="status", value="{{workflow.status}}"),
],
value="1",
help="Count of workflow execution by result status - workflow level",
),
],
) as w:
with DAG(name="dag-task"):
echo(
name="TEST-ONE",
arguments=[
Parameter(name="message", value="console output-->TEST-{{item.command}}"),
Parameter(name="tag", value="{{item.tag}}"),
],
with_items=[
{"tag": "TEST-ONE-A", "command": "ONE-A"},
{"tag": "TEST-ONE-B", "command": "ONE-B"},
],
)
echo(
name="TEST-TWO",
arguments=[
Parameter(name="message", value="console output-->TEST-{{item.command}}"),
Parameter(name="tag", value="{{item.tag}}"),
],
with_items=[
{"tag": "TEST-TWO-A", "command": "TWO-A"},
{"tag": "TEST-TWO-B", "command": "TWO-B"},
],
)
```

=== "YAML"

```yaml linenums="1"
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: dag-task-
spec:
entrypoint: dag-task
metrics:
prometheus:
- gauge:
realtime: false
value: '{{workflow.duration}}'
help: Duration gauge by workflow level
labels:
- key: playground_id_workflow
value: test
- key: status
value: '{{workflow.status}}'
name: playground_workflow_duration
- counter:
value: '1'
help: Count of workflow execution by result status - workflow level
labels:
- key: playground_id_workflow_counter
value: test
- key: status
value: '{{workflow.status}}'
name: playground_workflow_result_counter
templates:
- dag:
tasks:
- arguments:
parameters:
- name: message
value: console output-->TEST-{{item.command}}
- name: tag
value: '{{item.tag}}'
name: TEST-ONE
template: echo
withItems:
- tag: TEST-ONE-A
command: ONE-A
- tag: TEST-ONE-B
command: ONE-B
- arguments:
parameters:
- name: message
value: console output-->TEST-{{item.command}}
- name: tag
value: '{{item.tag}}'
name: TEST-TWO
template: echo
withItems:
- tag: TEST-TWO-A
command: TWO-A
- tag: TEST-TWO-B
command: TWO-B
name: dag-task
- container:
command:
- echo
- '{{inputs.parameters.message}}'
image: alpine:3.7
inputs:
parameters:
- name: message
- name: tag
metrics:
prometheus:
- gauge:
realtime: false
value: '{{duration}}'
help: Duration gauge by task name in seconds - task level
labels:
- key: playground_task_name
value: '{{inputs.parameters.tag}}'
- key: status
value: '{{status}}'
name: playground_workflow_duration_task_seconds
- counter:
value: '1'
help: Count of task execution by result status - task level
labels:
- key: playground_task_name_counter
value: '{{inputs.parameters.tag}}'
- key: status
value: '{{status}}'
name: playground_workflow_result_task_counter
name: echo
```

Loading

0 comments on commit a1d704e

Please sign in to comment.