Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

reconnect option is now deprecated #530

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

damian-tomczak
Copy link

@damian-tomczak damian-tomczak commented Sep 21, 2023

WARNING: MYSQL_OPT_RECONNECT is deprecated and will be removed in a future version.

@rbock
Copy link
Owner

rbock commented Sep 23, 2023

Since which version? It might make sense to have a pre-processor switch.

@damian-tomczak
Copy link
Author

Since which version? It might make sense to have a pre-processor switch.

Since 8.0.34

Yeah, it makes sense, I fixed it.

@MeanSquaredError
Copy link
Contributor

@damian-tomczak
It seems that there is an unwanted #endif around line 91

#if MYSQL_VERSION_ID < 80034
          if (config->auto_reconnect)
          {
            my_bool my_true{true};
@@ -87,6 +88,10 @@ namespace sqlpp
              throw sqlpp::exception{"MySQL: could not set option MYSQL_OPT_RECONNECT"};
            }
          }
#endif <--- HERE
#else
          mysql->reconnect = config->auto_reconnect
#endif

@damian-tomczak
Copy link
Author

@damian-tomczak It seems that there is an unwanted #endif around line 91

#if MYSQL_VERSION_ID < 80034
          if (config->auto_reconnect)
          {
            my_bool my_true{true};
@@ -87,6 +88,10 @@ namespace sqlpp
              throw sqlpp::exception{"MySQL: could not set option MYSQL_OPT_RECONNECT"};
            }
          }
#endif <--- HERE
#else
          mysql->reconnect = config->auto_reconnect
#endif

sry, fixed

@MeanSquaredError
Copy link
Contributor

MeanSquaredError commented Sep 26, 2023

@damian-tomczak
If I understand correctly, they are deprecating not just the reconnect option, but the reconnect functionality as a whole. Here
https://dev.mysql.com/doc/c-api/8.0/en/c-api-auto-reconnect.html
they say the following

Beginning with MySQL 8.0.34, the automatic reconnection feature is deprecated. The related MYSQL_OPT_RECONNECT option is still available but now returns a deprecation warning to the standard error output if your application calls the mysql_get_option() or mysql_options() function with the option.

Expect automatic reconnection functionality to be removed in a future version of MySQL.

Regarding the idea to set MYSQL::reconnect to true, is it a recommendation from the MySQL documentation or from their support? The MYSQL documentation treats the MYSQL structure (which represents a MYSQL handle) as an opaque object and I did not see them providing explicit information or recommendations on setting individual fields in the MYSQL structure.

Moreover if they deprecate the reconnect functionality, I think that the MYSQL client library will just remove the MYSQL::reconnect field or will keep it just for binary compatibility but will ignore its value.

So if my understanding of the issue is correct, then this PR will not help with the reconnect functionality. Once the functionality is removed from the MYSQL client, it will no longer work and if we need it in sqlpp11, we will have to implement it ourselves.

@MeanSquaredError
Copy link
Contributor

On a side note, I always considered MYSQL's reconnect a useless and even harmful feature because it does not preserve transactions. Any serious application uses transactions and thus cannot use MYSQL's automatic reconnect. Correctly implemented automatic reconnects require support from the database access layer and from the application itself so they cannot be implemented purely in the database library, like MYSQL's automatic reconnect feature.

I guess MYSQL's developers reached the same conclusion and finally decided to get rid of this evil feature which causes mostly harm.

@rbock
Copy link
Owner

rbock commented Sep 26, 2023

Quick note: My week is crazy. I will need a few days before I can look into this. But I am glad to see the discussion!

Thanks!

@damian-tomczak
Copy link
Author

@damian-tomczak If I understand correctly, they are deprecating not just the reconnect option, but the reconnect functionality as a whole. Here https://dev.mysql.com/doc/c-api/8.0/en/c-api-auto-reconnect.html they say the following

Beginning with MySQL 8.0.34, the automatic reconnection feature is deprecated. The related MYSQL_OPT_RECONNECT option is still available but now returns a deprecation warning to the standard error output if your application calls the mysql_get_option() or mysql_options() function with the option.
Expect automatic reconnection functionality to be removed in a future version of MySQL.

Regarding the idea to set MYSQL::reconnect to true, is it a recommendation from the MySQL documentation or from their support? The MYSQL documentation treats the MYSQL structure (which represents a MYSQL handle) as an opaque object and I did not see them providing explicit information or recommendations on setting individual fields in the MYSQL structure.

Moreover if they deprecate the reconnect functionality, I think that the MYSQL client library will just remove the MYSQL::reconnect field or will keep it just for binary compatibility but will ignore its value.

So if my understanding of the issue is correct, then this PR will not help with the reconnect functionality. Once the functionality is removed from the MYSQL client, it will no longer work and if we need it in sqlpp11, we will have to implement it ourselves.

On a side note, I always considered MYSQL's reconnect a useless and even harmful feature because it does not preserve transactions. Any serious application uses transactions and thus cannot use MYSQL's automatic reconnect. Correctly implemented automatic reconnects require support from the database access layer and from the application itself so they cannot be implemented purely in the database library, like MYSQL's automatic reconnect feature.

I guess MYSQL's developers reached the same conclusion and finally decided to get rid of this evil feature which causes mostly harm.

Based on what you have said, I would leave it how it now looks. Wdut?

@damian-tomczak
Copy link
Author

damian-tomczak commented Sep 26, 2023

Honestly, I don't like really my proposition.
But already on windows there are plenty of warnings:

Severity	Code	Description	Project	File	Line	Suppression State
Warning	C4244	'initializing': conversion from '[confidential]::Node::IdT' to 'int', possible loss of data	[confidential]	C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.37.32822\include\xutility	240	
Warning	C4267	'=': conversion from 'size_t' to 'unsigned long', possible loss of data	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\bind_result.h	278	
Warning	C4267	'=': conversion from 'size_t' to 'unsigned long', possible loss of data	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\bind_result.h	301	
Warning	C4267	'=': conversion from 'size_t' to 'unsigned long', possible loss of data	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\bind_result.h	324	
Warning	C4267	'=': conversion from 'size_t' to 'unsigned long', possible loss of data	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\bind_result.h	347	
Warning	C4018	'>': signed/unsigned mismatch	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\bind_result.h	379	
Warning	C4018	'>': signed/unsigned mismatch	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\bind_result.h	396	
Warning	C4267	'=': conversion from 'size_t' to 'unsigned long', possible loss of data	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\bind_result.h	451	
Warning	C4996	'sqlpp::mysql::connection_config::auto_reconnect': was declared deprecated	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\connection_config.h	55	
Warning	C4267	'=': conversion from 'size_t' to 'unsigned long', possible loss of data	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\prepared_statement.h	145	
Warning	C4996	'sqlpp::mysql::connection_config::auto_reconnect': was declared deprecated	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\connection.h	128	
Warning	C4267	'argument': conversion from 'size_t' to 'unsigned long', possible loss of data	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\connection.h	202	
Warning	C4267	'argument': conversion from 'size_t' to 'unsigned long', possible loss of data	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\connection.h	490	
Warning	C4267	'=': conversion from 'size_t' to 'unsigned long', possible loss of data	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\bind_result.h	278	
Warning	C4267	'=': conversion from 'size_t' to 'unsigned long', possible loss of data	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\bind_result.h	301	
Warning	C4267	'=': conversion from 'size_t' to 'unsigned long', possible loss of data	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\bind_result.h	324	
Warning	C4267	'=': conversion from 'size_t' to 'unsigned long', possible loss of data	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\bind_result.h	347	
Warning	C4018	'>': signed/unsigned mismatch	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\bind_result.h	379	
Warning	C4018	'>': signed/unsigned mismatch	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\bind_result.h	396	
Warning	C4267	'=': conversion from 'size_t' to 'unsigned long', possible loss of data	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\bind_result.h	451	
Warning	C4996	'sqlpp::mysql::connection_config::auto_reconnect': was declared deprecated	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\connection_config.h	55	
Warning	C4267	'=': conversion from 'size_t' to 'unsigned long', possible loss of data	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\prepared_statement.h	145	
Warning	C4996	'sqlpp::mysql::connection_config::auto_reconnect': was declared deprecated	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\connection.h	128	
Warning	C4267	'argument': conversion from 'size_t' to 'unsigned long', possible loss of data	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\connection.h	202	
Warning	C4267	'argument': conversion from 'size_t' to 'unsigned long', possible loss of data	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\connection.h	490	
Warning	C4244	'=': conversion from '_Rep' to 'unsigned int', possible loss of data	[confidential]	[confidential]src\database.cpp	60	
Warning	C4244	'initializing': conversion from '[confidential]::Node::IdT' to 'int', possible loss of data	[confidential]	C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.37.32822\include\xutility	240	
Warning	C4244	'initializing': conversion from '[confidential]::Node::IdT' to 'int', possible loss of data	[confidential]	C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.37.32822\include\xutility	254	
Warning	C4244	'initializing': conversion from '[confidential]::Node::IdT' to 'int', possible loss of data	[confidential]	C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.37.32822\include\xutility	241	
Warning	C4267	'=': conversion from 'size_t' to 'unsigned long', possible loss of data	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\bind_result.h	278	
Warning	C4267	'=': conversion from 'size_t' to 'unsigned long', possible loss of data	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\bind_result.h	301	
Warning	C4267	'=': conversion from 'size_t' to 'unsigned long', possible loss of data	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\bind_result.h	324	
Warning	C4267	'=': conversion from 'size_t' to 'unsigned long', possible loss of data	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\bind_result.h	347	
Warning	C4018	'>': signed/unsigned mismatch	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\bind_result.h	379	
Warning	C4018	'>': signed/unsigned mismatch	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\bind_result.h	396	
Warning	C4267	'=': conversion from 'size_t' to 'unsigned long', possible loss of data	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\bind_result.h	451	
Warning	C4996	'sqlpp::mysql::connection_config::auto_reconnect': was declared deprecated	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\connection_config.h	55	
Warning	C4267	'=': conversion from 'size_t' to 'unsigned long', possible loss of data	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\prepared_statement.h	145	
Warning	C4996	'sqlpp::mysql::connection_config::auto_reconnect': was declared deprecated	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\connection.h	128	
Warning	C4267	'argument': conversion from 'size_t' to 'unsigned long', possible loss of data	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\connection.h	202	
Warning	C4267	'argument': conversion from 'size_t' to 'unsigned long', possible loss of data	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\connection.h	490	
Warning	C4267	'=': conversion from 'size_t' to 'unsigned long', possible loss of data	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\bind_result.h	278	
Warning	C4267	'=': conversion from 'size_t' to 'unsigned long', possible loss of data	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\bind_result.h	301	
Warning	C4267	'=': conversion from 'size_t' to 'unsigned long', possible loss of data	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\bind_result.h	324	
Warning	C4267	'=': conversion from 'size_t' to 'unsigned long', possible loss of data	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\bind_result.h	347	
Warning	C4018	'>': signed/unsigned mismatch	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\bind_result.h	379	
Warning	C4018	'>': signed/unsigned mismatch	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\bind_result.h	396	
Warning	C4267	'=': conversion from 'size_t' to 'unsigned long', possible loss of data	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\bind_result.h	451	
Warning	C4996	'sqlpp::mysql::connection_config::auto_reconnect': was declared deprecated	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\connection_config.h	55	
Warning	C4267	'=': conversion from 'size_t' to 'unsigned long', possible loss of data	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\prepared_statement.h	145	
Warning	C4996	'sqlpp::mysql::connection_config::auto_reconnect': was declared deprecated	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\connection.h	128	
Warning	C4267	'argument': conversion from 'size_t' to 'unsigned long', possible loss of data	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\connection.h	202	
Warning	C4267	'argument': conversion from 'size_t' to 'unsigned long', possible loss of data	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\connection.h	490	
Warning	C4267	'=': conversion from 'size_t' to 'unsigned long', possible loss of data	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\bind_result.h	278	
Warning	C4267	'=': conversion from 'size_t' to 'unsigned long', possible loss of data	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\bind_result.h	301	
Warning	C4267	'=': conversion from 'size_t' to 'unsigned long', possible loss of data	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\bind_result.h	324	
Warning	C4267	'=': conversion from 'size_t' to 'unsigned long', possible loss of data	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\bind_result.h	347	
Warning	C4018	'>': signed/unsigned mismatch	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\bind_result.h	379	
Warning	C4018	'>': signed/unsigned mismatch	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\bind_result.h	396	
Warning	C4267	'=': conversion from 'size_t' to 'unsigned long', possible loss of data	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\bind_result.h	451	
Warning	C4996	'sqlpp::mysql::connection_config::auto_reconnect': was declared deprecated	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\connection_config.h	55	
Warning	C4267	'=': conversion from 'size_t' to 'unsigned long', possible loss of data	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\prepared_statement.h	145	
Warning	C4996	'sqlpp::mysql::connection_config::auto_reconnect': was declared deprecated	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\connection.h	128	
Warning	C4267	'argument': conversion from 'size_t' to 'unsigned long', possible loss of data	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\connection.h	202	
Warning	C4267	'argument': conversion from 'size_t' to 'unsigned long', possible loss of data	[confidential]	[confidential]external\sqlpp11\include\sqlpp11\mysql\connection.h	490	
Warning	C4267	'argument': conversion from 'size_t' to 'int', possible loss of data	[confidential]	[confidential]src\nodes.cpp	42	
Warning	C4267	'argument': conversion from 'size_t' to 'int', possible loss of data	[confidential]	[confidential]src\nodes.cpp	46	
Warning	C4244	'initializing': conversion from '[confidential]::Node::IdT' to 'int', possible loss of data	[confidential]	C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.37.32822\include\xutility	254	
Warning	C4244	'initializing': conversion from '[confidential]::Node::IdT' to 'int', possible loss of data	[confidential]	C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.37.32822\include\xutility	241	

I have no idea how it looks on another platforms.

@MeanSquaredError
Copy link
Contributor

@damian-tomczak

The question is what is the purpose of the PR? Is it to silence the warnings or something else? It seems that the purpose of the patch is to silence the deprecation warnings and in order to achieve that the PR changes the mysql code in sqlpp so that it works directly with the value in MYSQL::reconnect.

I consider the MYSQL structure opaque because its internals are not really documented and users are expected to work just with pointers to MYSQL and not touch its internal fields.

IMHO working with the internals of such an opaque structure just to silence a deprecation warning means begging for trouble.

Regarding the correct approach to handling the deprecation warning, I am not sure, but I think one possible way handle this issue is to mark sqlpp:mysql::connection_config::auto_reconnect as deprecated (you already did that) and then just use a few pragmas to silence the deprecation warnings in places where sqlpp calls the MySQL API to set the autoreconnect flag of MYSQL structure. The pragmas need only to handle the 3 major compilers (GCC, Clang, MSVC), users of other compilers can probably live with a few extra deprecation warnings.

But let's also see what will rbock say about this issue.

Regarding the other warnings when building the MySQL connector with MSVC, it seems that most of these are about loss of data when doing implicit casts from size_t to unsigned long. The code probably needs a few static_casts from size_t to the internal types used by the MySQL library. Judging from the messages that you posted above, there are not that many (less than 10) places where the problem happens

@rbock
Copy link
Owner

rbock commented Oct 1, 2023

Note that the deprecated attribute is C++14, only, so we should not use it here.

@MeanSquaredError has a good point that that whole reconnect idea is a bit wonky to begin with. Maybe the best path forward is to simply remove both the auto_reconnect field and the reconnect method?

Honestly, I don't like really my proposition. But already on windows there are plenty of warnings:

@damian-tomczak
Those look annoying, indeed. Which version/commit are you using? The line numbers do not seem to match with current HEAD.

Can you file separate issue for warnings?

@MeanSquaredError
Copy link
Contributor

Currently I see several possible approaches:

  1. Don't apply this PR, leave everything as it is now. Once the MySQL client library stops supporting reconnects, use ifdefs to disable reconnects in sqlpp code if the MySQL library is newer that the cut-off version.
  2. Change the PR to silence the MySQL client warnings about reconnect. Optionally we can still deprecate the reconnect flag in sqlpp. Indeed the deprecated attribute is supported in C++14 or newer, but for practical reasons the major compilers were always ignoring the unknown attributes, so we probably we can get away with the deprecated attribute even in C++11 mode.
  3. Remove the reconnect option in sqlpp right away. That may probably cause problems for those who use the library with mysql and enable reconnect when creating sqlpp connections.
  4. Change the sqlpp code so that it no longer uses the MySQL native support for reconnect, but instead implement that same functionality in sqlpp. Viable approach but I am not sure if it makes sense just to silence a warning.
  5. Remove the reconnect option and instead add proper support for retrying transactions until they succeed.

For 5. I am thinking something along the lines of

template <typename Db>
using user_block_of_work = std::function<bool (Db& db)>;

using user_retry_handler = std::function<bool (std::runtime_error&)>;

void atomic_block_of_work (user_block_of_work& work, user_retry_handler& retry);

So when the user needs to run an atomic block of SQL queries, he will call atomic_block_of_work passing two user functions (normally lambdas created on the fly).

The first lambda will contain the SQL queries that the user wants to make atomically. The first lambda will return true for commit, or false for rollback.

The second lambda will contain a custom handler that will be called when an exception occurs (e.g. a network error, or disconnect from the database), allowing the user to choose whether the block should be retried after a reconnect, or should be aborted.

I am using a similar approach in my code and it works well for me. Moreover I think that it is the only reliable way to handle support for reconnects when you use transactions. What MySQL has/had only works with autocommit mode (i.e. no transactions), but for serious applications with transactions, the database access layer (i.e. sqlpp) needs a bit of help from the application that uses the database layer.

Additionally I think that whichever approach sqlpp chooses, it should not work with the internals of the MYSQL object and should not directly change the reconnect flag there. That is more trouble than it is worth. If we just want to silence the deprecation warnings then we can just use compiler pragmas for that in the sqlpp code.

@rbock
Copy link
Owner

rbock commented Oct 3, 2023

Thanks for your detailed thoughts on this!

should not work with the internals of the MYSQL object
+1

Proposal:

  1. Tag a release
  2. Remove the reconnect code (your option 3)
  3. Implement something along the lines of your option 5, or document that this as an idea.

WDYT?

@MeanSquaredError
Copy link
Contributor

MeanSquaredError commented Oct 3, 2023

@rbock
I think it is a good approach. It is probably also worth mentioning in the documentation that if the old behavior is really needed and the user doesn't
want to use the new functions, then the old behavior can be achieved by a few extra lines of code, as long as the native MySQL client library has not completely dropped the support for automatic reconnects.

// Assuming that db is of type sqlpp::mysql::connection, enable the 
// deprecated  MySQL reconnect option. This will generate deprecation 
// warnings  with newer versions of the MySQL client library, so you may
// want to wrap it in compiler-specific pragmas to silence the warnings.
bool reconnect_flag{true};
if (mysql_options(db.native_handle(), MYSQL_OPT_RECONNECT, &reconnect_flag))
{
  throw sqlpp::exception{"MySQL: could not set option MYSQL_OPT_RECONNECT"};
}

rbock added a commit that referenced this pull request Oct 8, 2023
MySQL 8.0.34 deprecates `MYSQL_OPT_RECONNECT`. As discussed in #530,
this commit is removing library support for (auto-)reconnect.

It is of course still possible to reconnect directly using the native
handle.
@rbock
Copy link
Owner

rbock commented Oct 8, 2023

  1. Tag a release [done]
  2. Remove the reconnect code (your option 3) [done]
  3. Implement something along the lines of your option 5, or document that this as an idea.

@MeanSquaredError
Copy link
Contributor

MeanSquaredError commented Dec 31, 2023

Actually the same pattern (atomic_block_of_work) is useful for handling all kinds of transaction failures, not just network errors.

For example using pessimistic locking (LOCK tablename, ...) gets really complicated when there are multiple tables that are intertwined via multiple foreign keys.

Luckily PostgreSQL offers the Repeatable read and Serializable isolation levels that effectively provide a kind of optimistic locking, you just run the transaction without any explicit locks, but if there were conflict with some other parallel transaction, then upon commit, you will get a serialization failure. In that case you repeat the whole transaction again, until it succeeds. In my opinion for complex database schemas it is much easier to use that instead of the explicit pessimistic locking.

The only problem that we have with SQLPP11 is that if we use this interface

template <typename Db>
using user_block_of_work = std::function<bool (Db& db)>;

using user_retry_handler = std::function<bool (std::runtime_error&)>;

void atomic_block_of_work (user_block_of_work& work, user_retry_handler& retry);

then in the retry handler we don't have an easy way to extract the exact SQL error. We don't want to repeat the transaction for all SQL error, just for some of the errors (e.g. serialization failures or network errors, etc). However std::runtime_error does not give us an easy way to extract the error code. We really don't want to extract the error code by calling std::runtim_error::what() because this is error-prone.

So we probably need an overhaul of the exception hierarchy used by the different connectors and by sqlpp11 as a whole because currently it more or less in a state of anarchy. There is the really basic sqlpp::exception which is a wrapper for std::runtime_error and only provides the error message as a string. The sqlite and mysql connectors use sqlpp::exception. The PostgreSQL connector is better but still can be improved further. It has multiple exception classes that form a ramified hierarchy, but all of them with the exception of one, don't provide the SQLSTATE result code, so handling multiple exceptions codes in a single code branch is difficult, given that the C++'s catch is unable to catch multiple exception types in a single branch.

Additionally I am not sure what is the best way to pass the error to the user_retry_handler, maybe we should just cast the derived exception to the base sqlpp::exception and then user_retry_handler will fetch the SQLSTATE code from the sqlpp::exception object and will return the retry/fail flag for the transaction.

Any comments or suggestions are welcome.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants