Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Docs] Minor fixes to docs pages around transactions #23926

Merged
merged 2 commits into from
Sep 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -329,12 +329,12 @@ After a transaction T1 (that was waiting for other transactions) unblocks, it co

### Examples

The following examples describe different use cases detailing the Wait-on-Conflict behavior.
The following examples describe different use cases detailing the Wait-on-Conflict behavior. To run the examples, you need to do the following:

1. Note that the examples require you to set the YB-TServer flag `enable_wait_queues=true`.
1. Also, set the YB-TServer flag `ysql_max_write_restart_attempts=0` to disable internal query layer retries on conflict. This is done to illustrate the `Wait-on-Conflict` concurrency control semantics separately without query layer retries. It is not recommended to disable these retries in production.
1. Set the YB-TServer flag `enable_wait_queues=true`.
1. Set the per-session `yb_max_query_layer_retries=0` YSQL configuration parameter to disable internal query layer retries on conflict. This is done to illustrate the `Wait-on-Conflict` concurrency control semantics separately without query layer retries. It is not recommended to disable these retries in production. To set it at the cluster level, use the `ysql_pg_conf_csv` YB-TServer flag.

A restart is necessary for these flags to take effect.
A restart is necessary for the flags to take effect.

Start by setting up the table you'll use in all of the examples in this section.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -82,13 +82,25 @@ The client can reconnect to the server and retry the transaction.
This error occurs when a transaction cannot be applied or progress further because of other conflicting transactions. For example, when multiple transactions are modifying the same key.

```output
ERROR: 40001: Operation expired: Transaction XXXX expired or aborted by a conflict
ERROR: could not serialize access due to concurrent update (...)
```

{{<lead link="../transactions-retries-ysql/#client-side-retry">}}
Serialization failure errors can be retried by the client. See [Client-side retry](../transactions-retries-ysql/#client-side-retry).
{{</lead>}}

## 40P01: Deadlock detected

This error occurs when two or more transactions wait on each other to form a deadlock cycle. One or more of the transactions in the cycle are aborted and they fail with the following error.

```output
ERROR: deadlock detected (...)
```

{{<lead link="../transactions-retries-ysql/#client-side-retry">}}
Deadlock detected errors can be retried by the client. See [Client-side retry](../transactions-retries-ysql/#client-side-retry).
{{</lead>}}

## 2D000: Invalid transaction termination

This error occurs when a transaction is terminated either by a `COMMIT` or a `ROLLBACK` in an invalid location. For example, when a `COMMIT` is issued inside a stored procedure that is called from inside a transaction.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ Follow the [setup instructions](../../../../explore#tabs-00-00) to start a singl

## Automatic retries

YugabyteDB retries failed transactions automatically on the server side whenever possible without client intervention as per the [concurrency control policies](../../../../architecture/transactions/concurrency-control/#best-effort-internal-retries-for-first-statement-in-a-transaction). This is the case even for single statements, which are implicitly considered transactions. In [Read Committed](../../../../explore/transactions/isolation-levels/#read-committed-isolation) isolation mode, the server retries indefinitely.
YugabyteDB retries failed transactions automatically on the server side whenever possible without client intervention as per the [concurrency control policies](../../../../architecture/transactions/concurrency-control/#best-effort-internal-retries-for-first-statement-in-a-transaction). This is the case even for single statements, which are implicitly considered transactions.

In some scenarios, a server-side retry is not suitable. For example, the retry limit has been reached or the transaction is not in a valid state. In these cases, it is the client's responsibility to retry the transaction at the application layer.

Expand Down Expand Up @@ -76,19 +76,11 @@ If the `COMMIT` is successful, the program exits the loop. `attempt < max_attemp

##### 40001 - SerializationFailure

SerializationFailure errors happen when multiple transactions are updating the same set of keys (conflict) or when transactions are waiting on each other (deadlock). The error messages could be one of the following types:
SerializationFailure errors happen when multiple transactions are updating the same set of keys (conflict). During a conflict, certain transactions are retried.

- During a conflict, certain transactions are retried. However, after the retry limit is reached, an error occurs as follows:

```output
ERROR: 40001: All transparent retries exhausted.
```

- All transactions are given a dynamic priority. When a deadlock is detected, the transaction with lower priority is automatically killed. For this scenario, the client might receive a message similar to the following:

```output
ERROR: 40001: Operation expired: Heartbeat: Transaction XXXX expired or aborted by a conflict
```
```output
ERROR: could not serialize access due to concurrent update (...)
```

The correct way to handle this error is with a retry loop with exponential backoff, as described in [Client-side retry](#client-side-retry). When the [UPDATE](../../../../api/ysql/the-sql-language/statements/dml_update/) or [COMMIT](../../../../api/ysql/the-sql-language/statements/txn_commit/) fails because of `SerializationFailure`, the code retries after waiting for `sleep_time` seconds, up to `max_attempts`.

Expand All @@ -98,6 +90,17 @@ In read committed isolation level, as the server retries internally, the client

Another way to handle these failures is would be to rollback to a checkpoint before the failed statement and proceed further as described in [Savepoints](#savepoints).

##### 40001 - Deadlock detected

This error occurs when two or more transactions wait on each other to form a deadlock cycle. One or more of the transactions in the cycle are aborted
and they fail with the following error:

```output
ERROR: deadlock detected (...)
```

Retries to handle this error are similar to serialization errors (40001).

## Savepoints

[Savepoints](../../../../api/ysql/the-sql-language/statements/savepoint_create/) are named checkpoints that can be used to rollback just a few statements, and then proceed with the transaction, rather than aborting the entire transaction when there is an error.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -156,9 +156,9 @@ Each update performed as a part of the transaction is replicated across multiple

### Concurrency control

[Concurrency control](../../../architecture/transactions/concurrency-control/) in databases ensures that multiple transactions can execute concurrently while preserving data integrity. Concurrency control is essential for correctness in environments where two or more transactions can access the same data at the same time. The two primary mechanisms to achieve concurrency control are optimistic and pessimistic.
[Concurrency control](../../../architecture/transactions/concurrency-control/) in databases ensures that multiple transactions can execute concurrently while preserving data integrity. Concurrency control is essential for correctness in environments where two or more transactions can access the same data at the same time. YugabyteDB currently supports two concurrency control mechanisms: Wait-on-Conflict and Fail-on-Conflict.

YugabyteDB currently supports optimistic concurrency control, with pessimistic concurrency control being worked on actively.
These are orthogonal to isolation levels (except for Read Committed isolation which has the same behaviour in both modes barring some limitations and performance penalty in Fail-on-Conflict mode). Also note that the Wait-on-Conflict concurrency control mode exactly matches PostgreSQL semantics and also gives better performance.

## Transaction options

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -329,12 +329,12 @@ After a transaction T1 (that was waiting for other transactions) unblocks, it co

### Examples

The following examples describe different use cases detailing the Wait-on-Conflict behavior.
The following examples describe different use cases detailing the Wait-on-Conflict behavior. To run the examples, you need to do the following:

1. Note that the examples require you to set the YB-TServer flag `enable_wait_queues=true`.
1. Also, set the YB-TServer flag `ysql_max_write_restart_attempts=0` to disable internal query layer retries on conflict. This is done to illustrate the `Wait-on-Conflict` concurrency control semantics separately without query layer retries. It is not recommended to disable these retries in production.
1. Set the YB-TServer flag `enable_wait_queues=true`.
1. Set the per-session `yb_max_query_layer_retries=0` YSQL configuration parameter to disable internal query layer retries on conflict. This is done to illustrate the `Wait-on-Conflict` concurrency control semantics separately without query layer retries. It is not recommended to disable these retries in production. To set it at the cluster level, use the `ysql_pg_conf_csv` YB-TServer flag.

A restart is necessary for these flags to take effect.
A restart is necessary for the flags to take effect.

Start by setting up the table you'll use in all of the examples in this section.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -80,13 +80,25 @@ The client can reconnect to the server and retry the transaction.
This error occurs when a transaction cannot be applied or progress further because of other conflicting transactions. For example, when multiple transactions are modifying the same key.

```output
ERROR: 40001: Operation expired: Transaction XXXX expired or aborted by a conflict
ERROR: could not serialize access due to concurrent update (...)
```

{{<lead link="../transactions-retries-ysql/#client-side-retry">}}
Serialization failure errors can be retried by the client. See [Client-side retry](../transactions-retries-ysql/#client-side-retry).
{{</lead>}}

## 40P01: Deadlock detected

This error occurs when two or more transactions wait on each other to form a deadlock cycle. One or more of the transactions in the cycle are aborted and they fail with the following error.

```output
ERROR: deadlock detected (...)
```

{{<lead link="../transactions-retries-ysql/#client-side-retry">}}
Deadlock detected errors can be retried by the client. See [Client-side retry](../transactions-retries-ysql/#client-side-retry).
{{</lead>}}

## 2D000: Invalid transaction termination

This error occurs when a transaction is terminated either by a `COMMIT` or a `ROLLBACK` in an invalid location. For example, when a `COMMIT` is issued inside a stored procedure that is called from inside a transaction.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ Follow the [setup instructions](../../../../explore#tabs-00-00) to start a singl

## Automatic retries

YugabyteDB retries failed transactions automatically on the server side whenever possible without client intervention as per the [concurrency control policies](../../../../architecture/transactions/concurrency-control/#best-effort-internal-retries-for-first-statement-in-a-transaction). This is the case even for single statements, which are implicitly considered transactions. In [Read Committed](../../../../explore/transactions/isolation-levels/#read-committed-isolation) isolation mode, the server retries indefinitely.
YugabyteDB retries failed transactions automatically on the server side whenever possible without client intervention as per the [concurrency control policies](../../../../architecture/transactions/concurrency-control/#best-effort-internal-retries-for-first-statement-in-a-transaction). This is the case even for single statements, which are implicitly considered transactions.

In some scenarios, a server-side retry is not suitable. For example, the retry limit has been reached or the transaction is not in a valid state. In these cases, it is the client's responsibility to retry the transaction at the application layer.

Expand Down Expand Up @@ -74,19 +74,11 @@ If the `COMMIT` is successful, the program exits the loop. `attempt < max_attemp

##### 40001 - SerializationFailure

SerializationFailure errors happen when multiple transactions are updating the same set of keys (conflict) or when transactions are waiting on each other (deadlock). The error messages could be one of the following types:
SerializationFailure errors happen when multiple transactions are updating the same set of keys (conflict). During a conflict, certain transactions are retried.

- During a conflict, certain transactions are retried. However, after the retry limit is reached, an error occurs as follows:

```output
ERROR: 40001: All transparent retries exhausted.
```

- All transactions are given a dynamic priority. When a deadlock is detected, the transaction with lower priority is automatically killed. For this scenario, the client might receive a message similar to the following:

```output
ERROR: 40001: Operation expired: Heartbeat: Transaction XXXX expired or aborted by a conflict
```
```output
ERROR: could not serialize access due to concurrent update (...)
```

The correct way to handle this error is with a retry loop with exponential backoff, as described in [Client-side retry](#client-side-retry). When the [UPDATE](../../../../api/ysql/the-sql-language/statements/dml_update/) or [COMMIT](../../../../api/ysql/the-sql-language/statements/txn_commit/) fails because of `SerializationFailure`, the code retries after waiting for `sleep_time` seconds, up to `max_attempts`.

Expand All @@ -96,6 +88,17 @@ In read committed isolation level, as the server retries internally, the client

Another way to handle these failures is would be to rollback to a checkpoint before the failed statement and proceed further as described in [Savepoints](#savepoints).

##### 40001 - Deadlock detected

This error occurs when two or more transactions wait on each other to form a deadlock cycle. One or more of the transactions in the cycle are aborted
and they fail with the following error:

```output
ERROR: deadlock detected (...)
```

Retries to handle this error are similar to serialization errors (40001).

## Savepoints

[Savepoints](../../../../api/ysql/the-sql-language/statements/savepoint_create/) are named checkpoints that can be used to rollback just a few statements, and then proceed with the transaction, rather than aborting the entire transaction when there is an error.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -156,9 +156,9 @@ Each update performed as a part of the transaction is replicated across multiple

### Concurrency control

[Concurrency control](../../../architecture/transactions/concurrency-control/) in databases ensures that multiple transactions can execute concurrently while preserving data integrity. Concurrency control is essential for correctness in environments where two or more transactions can access the same data at the same time. The two primary mechanisms to achieve concurrency control are optimistic and pessimistic.
[Concurrency control](../../../architecture/transactions/concurrency-control/) in databases ensures that multiple transactions can execute concurrently while preserving data integrity. Concurrency control is essential for correctness in environments where two or more transactions can access the same data at the same time. YugabyteDB currently supports two concurrency control mechanisms: Wait-on-Conflict and Fail-on-Conflict.

YugabyteDB currently supports optimistic concurrency control, with pessimistic concurrency control being worked on actively.
These are orthogonal to isolation levels (except for Read Committed isolation which has the same behaviour in both modes barring some limitations and performance penalty in Fail-on-Conflict mode). Also note that the Wait-on-Conflict concurrency control mode exactly matches PostgreSQL semantics and also gives better performance.

## Transaction options

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ Follow the [setup instructions](../../../../explore#tabs-00-00) to start a singl

## Automatic retries

YugabyteDB retries failed transactions automatically on the server side whenever possible without client intervention as per the [concurrency control policies](../../../../architecture/transactions/concurrency-control/#best-effort-internal-retries-for-first-statement-in-a-transaction). This is the case even for single statements, which are implicitly considered transactions. In [Read Committed](../../../../explore/transactions/isolation-levels/#read-committed-isolation) isolation mode, the server retries indefinitely.
YugabyteDB retries failed transactions automatically on the server side whenever possible without client intervention as per the [concurrency control policies](../../../../architecture/transactions/concurrency-control/#best-effort-internal-retries-for-first-statement-in-a-transaction). This is the case even for single statements, which are implicitly considered transactions.

In some scenarios, a server-side retry is not suitable. For example, the retry limit has been reached or the transaction is not in a valid state. In these cases, it is the client's responsibility to retry the transaction at the application layer.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ Follow the [setup instructions](../../../../explore#tabs-00-00) to start a singl

## Automatic retries

YugabyteDB retries failed transactions automatically on the server side whenever possible without client intervention as per the [concurrency control policies](../../../../architecture/transactions/concurrency-control/#best-effort-internal-retries-for-first-statement-in-a-transaction). This is the case even for single statements, which are implicitly considered transactions. In [Read Committed](../../../../explore/transactions/isolation-levels/#read-committed-isolation) isolation mode, the server retries indefinitely.
YugabyteDB retries failed transactions automatically on the server side whenever possible without client intervention as per the [concurrency control policies](../../../../architecture/transactions/concurrency-control/#best-effort-internal-retries-for-first-statement-in-a-transaction). This is the case even for single statements, which are implicitly considered transactions.

In some scenarios, a server-side retry is not suitable. For example, the retry limit has been reached or the transaction is not in a valid state. In these cases, it is the client's responsibility to retry the transaction at the application layer.

Expand Down