Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pipeline recovery: add test case #1877

Merged
merged 2 commits into from
Oct 9, 2024
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
143 changes: 84 additions & 59 deletions docs/test-cases/pipeline-recovery.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,56 @@ pipelines:

---

## Test Case 02: Recovery not triggered for fatal error - processor
## Test Case 02: Recovery not triggered for fatal error - DLQ threshold exceeded

**Priority** (low/medium/high):

**Description**:
Recovery is not triggered when the DLQ threshold is exceeded.

**Automated** (yes/no)

**Setup**:

**Pipeline configuration file**:

```yaml
version: "2.2"
pipelines:
- id: pipeline1
status: running
name: pipeline1
description: chaos destination with write errors, DLQ threshold specified
connectors:
- id: generator-src
type: source
plugin: builtin:generator
name: generator-src
settings:
format.type: structured
format.options.id: int
format.options.name: string
rate: "1"
- id: chaos-destination-1
type: destination
plugin: standalone:chaos
name: chaos-destination-1
settings:
writeMode: error
dead-letter-queue:
window-size: 2
window-nack-threshold: 1
```

**Steps**:

**Expected Result**:

**Additional comments**:

---

## Test Case 03: Recovery not triggered for fatal error - processor

**Priority** (low/medium/high):

Expand Down Expand Up @@ -93,7 +142,7 @@ pipelines:

---

## Test Case 03: Recovery not triggered - graceful shutdown
## Test Case 04: Recovery not triggered - graceful shutdown

**Priority** (low/medium/high):

Expand All @@ -118,7 +167,7 @@ typing Ctrl+C in the terminal where Conduit is running, or sending a SIGINT).

---

## Test Case 04: Recovery not triggered - user stopped pipeline
## Test Case 05: Recovery not triggered - user stopped pipeline

**Priority** (low/medium/high):

Expand All @@ -143,7 +192,32 @@ Recovery is not triggered if a user stops a pipeline (via the HTTP API's

---

## Test Case 05: Recovery is configured by default
## Test Case 06: Recovery not triggered - user force-stopped pipeline

**Priority** (low/medium/high):

**Description**:
Recovery is not triggered if a user stops a pipeline forcefully(e.g.
`curl -s localhost:8080/v1/pipelines/pipeline1/stop -d '{"force": true}'`).

**Automated** (yes/no)

**Setup**:

**Pipeline configuration file**:

```yaml
```

**Steps**:

**Expected Result**:

**Additional comments**:

---

## Test Case 07: Recovery is configured by default

**Priority** (low/medium/high):

Expand Down Expand Up @@ -185,7 +259,7 @@ pipelines:

---

## Test Case 06: Recovery not triggered on malformed pipeline
## Test Case 08: Recovery not triggered on malformed pipeline

**Priority** (low/medium/high):

Expand Down Expand Up @@ -221,7 +295,7 @@ pipelines:

---

## Test Case 07: Conduit exits with --pipelines.exit-on-degraded=true and a pipeline failing after recovery
## Test Case 09: Conduit exits with --pipelines.exit-on-degraded=true and a pipeline failing after recovery

**Priority** (low/medium/high):

Expand All @@ -246,7 +320,7 @@ maximum number of retries configured, Conduit should shut down gracefully.

---

## Test Case 08: Conduit doesn't exit with --pipelines.exit-on-degraded=true and a pipeline that recovers after a few retries
## Test Case 10: Conduit doesn't exit with --pipelines.exit-on-degraded=true and a pipeline that recovers after a few retries

**Priority** (low/medium/high):

Expand All @@ -271,7 +345,7 @@ pipeline that recovers after a few retries, Conduit should still be running.

---

## Test Case 09: Conduit exits with --pipelines.exit-on-degraded=true, --pipelines.error-recovery.max-retries=0, and a degraded pipeline
## Test Case 11: Conduit exits with --pipelines.exit-on-degraded=true, --pipelines.error-recovery.max-retries=0, and a degraded pipeline

**Priority** (low/medium/high):

Expand All @@ -298,56 +372,7 @@ gracefully shut down. This is due `max-retries=0` disabling the recovery.

---

## Test Case 10: Recovery not triggered for fatal error - DLQ threshold exceeded

**Priority** (low/medium/high):

**Description**:
Recovery is not triggered when the DLQ threshold is exceeded.

**Automated** (yes/no)

**Setup**:

**Pipeline configuration file**:

```yaml
version: "2.2"
pipelines:
- id: pipeline1
status: running
name: pipeline1
description: chaos destination with write errors, DLQ threshold specified
connectors:
- id: generator-src
type: source
plugin: builtin:generator
name: generator-src
settings:
format.type: structured
format.options.id: int
format.options.name: string
rate: "1"
- id: chaos-destination-1
type: destination
plugin: standalone:chaos
name: chaos-destination-1
settings:
writeMode: error
dead-letter-queue:
window-size: 2
window-nack-threshold: 1
```

**Steps**:

**Expected Result**:

**Additional comments**:

---

## Test Case 11: Recovery triggered during a specific max-retries-window, after that pipeline is degraded
## Test Case 12: Recovery triggered during a specific max-retries-window, after that pipeline is degraded

**Priority** (low/medium/high):

Expand Down Expand Up @@ -398,7 +423,7 @@ After 20 seconds the pipeline should be degraded.

---

## Test Case 12: Recovery triggered during a specific max-retries-window, pipeline is resilient during a specific time window
## Test Case 13: Recovery triggered during a specific max-retries-window, pipeline is resilient during a specific time window

**Priority** (low/medium/high):

Expand Down