Skip to content

Commit

Permalink
Adds: Recovery not triggered - user force-stopped pipeline. (#1877)
Browse files Browse the repository at this point in the history
  • Loading branch information
hariso authored Oct 9, 2024
1 parent b429365 commit 6b3a2ad
Showing 1 changed file with 84 additions and 59 deletions.
143 changes: 84 additions & 59 deletions docs/test-cases/pipeline-recovery.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,56 @@ pipelines:
---
## Test Case 02: Recovery not triggered for fatal error - processor
## Test Case 02: Recovery not triggered for fatal error - DLQ threshold exceeded
**Priority** (low/medium/high):
**Description**:
Recovery is not triggered when the DLQ threshold is exceeded.
**Automated** (yes/no)
**Setup**:
**Pipeline configuration file**:
```yaml
version: "2.2"
pipelines:
- id: pipeline1
status: running
name: pipeline1
description: chaos destination with write errors, DLQ threshold specified
connectors:
- id: generator-src
type: source
plugin: builtin:generator
name: generator-src
settings:
format.type: structured
format.options.id: int
format.options.name: string
rate: "1"
- id: chaos-destination-1
type: destination
plugin: standalone:chaos
name: chaos-destination-1
settings:
writeMode: error
dead-letter-queue:
window-size: 2
window-nack-threshold: 1
```
**Steps**:
**Expected Result**:
**Additional comments**:
---
## Test Case 03: Recovery not triggered for fatal error - processor
**Priority** (low/medium/high):
Expand Down Expand Up @@ -93,7 +142,7 @@ pipelines:
---
## Test Case 03: Recovery not triggered - graceful shutdown
## Test Case 04: Recovery not triggered - graceful shutdown
**Priority** (low/medium/high):
Expand All @@ -118,7 +167,7 @@ typing Ctrl+C in the terminal where Conduit is running, or sending a SIGINT).

---

## Test Case 04: Recovery not triggered - user stopped pipeline
## Test Case 05: Recovery not triggered - user stopped pipeline

**Priority** (low/medium/high):

Expand All @@ -143,7 +192,32 @@ Recovery is not triggered if a user stops a pipeline (via the HTTP API's

---

## Test Case 05: Recovery is configured by default
## Test Case 06: Recovery not triggered - user force-stopped pipeline

**Priority** (low/medium/high):

**Description**:
Recovery is not triggered if a user stops a pipeline forcefully(e.g.
`curl -s localhost:8080/v1/pipelines/pipeline1/stop -d '{"force": true}'`).

**Automated** (yes/no)

**Setup**:

**Pipeline configuration file**:

```yaml
```

**Steps**:

**Expected Result**:

**Additional comments**:

---

## Test Case 07: Recovery is configured by default

**Priority** (low/medium/high):

Expand Down Expand Up @@ -185,7 +259,7 @@ pipelines:
---
## Test Case 06: Recovery not triggered on malformed pipeline
## Test Case 08: Recovery not triggered on malformed pipeline
**Priority** (low/medium/high):
Expand Down Expand Up @@ -221,7 +295,7 @@ pipelines:
---
## Test Case 07: Conduit exits with --pipelines.exit-on-degraded=true and a pipeline failing after recovery
## Test Case 09: Conduit exits with --pipelines.exit-on-degraded=true and a pipeline failing after recovery
**Priority** (low/medium/high):
Expand All @@ -246,7 +320,7 @@ maximum number of retries configured, Conduit should shut down gracefully.

---

## Test Case 08: Conduit doesn't exit with --pipelines.exit-on-degraded=true and a pipeline that recovers after a few retries
## Test Case 10: Conduit doesn't exit with --pipelines.exit-on-degraded=true and a pipeline that recovers after a few retries

**Priority** (low/medium/high):

Expand All @@ -271,7 +345,7 @@ pipeline that recovers after a few retries, Conduit should still be running.

---

## Test Case 09: Conduit exits with --pipelines.exit-on-degraded=true, --pipelines.error-recovery.max-retries=0, and a degraded pipeline
## Test Case 11: Conduit exits with --pipelines.exit-on-degraded=true, --pipelines.error-recovery.max-retries=0, and a degraded pipeline

**Priority** (low/medium/high):

Expand All @@ -298,56 +372,7 @@ gracefully shut down. This is due `max-retries=0` disabling the recovery.

---

## Test Case 10: Recovery not triggered for fatal error - DLQ threshold exceeded

**Priority** (low/medium/high):

**Description**:
Recovery is not triggered when the DLQ threshold is exceeded.

**Automated** (yes/no)

**Setup**:

**Pipeline configuration file**:

```yaml
version: "2.2"
pipelines:
- id: pipeline1
status: running
name: pipeline1
description: chaos destination with write errors, DLQ threshold specified
connectors:
- id: generator-src
type: source
plugin: builtin:generator
name: generator-src
settings:
format.type: structured
format.options.id: int
format.options.name: string
rate: "1"
- id: chaos-destination-1
type: destination
plugin: standalone:chaos
name: chaos-destination-1
settings:
writeMode: error
dead-letter-queue:
window-size: 2
window-nack-threshold: 1
```

**Steps**:

**Expected Result**:

**Additional comments**:

---

## Test Case 11: Recovery triggered during a specific max-retries-window, after that pipeline is degraded
## Test Case 12: Recovery triggered during a specific max-retries-window, after that pipeline is degraded

**Priority** (low/medium/high):

Expand Down Expand Up @@ -398,7 +423,7 @@ After 20 seconds the pipeline should be degraded.

---

## Test Case 12: Recovery triggered during a specific max-retries-window, pipeline is resilient during a specific time window
## Test Case 13: Recovery triggered during a specific max-retries-window, pipeline is resilient during a specific time window

**Priority** (low/medium/high):

Expand Down

0 comments on commit 6b3a2ad

Please sign in to comment.