[Backport 2.12] [#13659] DocDB: Fix Prepare for the failed transaction

Summary:
YBTransaction starts with looking up for status tablet.
When this lookup is failed the transaction is marked as failed.

We lack failure state check at Prepare, so calling Prepare after transaction failure will result in adding callback to waiters list.
But no more action will be taken, so callback will never be notified.

The side effect is master leader restarts can cause prepared statement connections to hang.

This diff fixed the issue by checking failure status in Prepare.

Original commit: 11cdcc6/D18994

Test Plan:
Launch AWS cluster.
Run the following workload against it:
java  -jar yb-sample-apps.jar --workload SqlDataLoad  --num_writes -1 --num_threads_write 5 --num_threads_read 0 --num_reads 0 --num_unique_keys 100000000000 --batch_size 774 --num_value_columns 8 --nodes $NODES
Wait 10 minutes.
Stop node, that runs master leader, using AWS console.

W/o the fix there will be stuck queries.

Reviewers: rthallam, esheng

Reviewed By: esheng

Subscribers: ybase, rthallam

Differential Revision: https://phabricator.dev.yugabyte.com/D19032

build-support/post_install.sh

@@ -152,7 +152,11 @@ if [[ $install_mode == "true" ]]; then
    exit 1
   fi
 
-  ln -sfT "$linuxbrew_dir" "$BREW_HOME"
+  if [[ "$linuxbrew_dir" != "$BREW_HOME" ]]; then
+    ln -sfT "$linuxbrew_dir" "$BREW_HOME"
+  else
+    echo "Skipping linuxbrew symlink, since it already has necessary length: $linuxbrew_dir"
+  fi
 
   # We are relying on the fact that $distribution_dir is not a symlink. We don't want to add symlink
   # resolution to the find command because someone may accidentally add a symlink pointing to a

java/yb-cql/src/test/java/org/yb/cql/TestTransaction.java

@@ -624,11 +624,11 @@ public void testTimeout() throws Exception {
       LOG.info("Initial expired transactions = {}", initialExpiredTransactions);
 
       try {
-        thrown.expect(com.datastax.driver.core.exceptions.OperationTimedOutException.class);
+        thrown.expect(com.datastax.driver.core.exceptions.DriverException.class);
         session.execute("begin transaction" +
                         "  insert into test_timeout (k, v) values (1, 1);" +
                         "end transaction;");
-      } catch (com.datastax.driver.core.exceptions.OperationTimedOutException e) {
+      } catch (com.datastax.driver.core.exceptions.DriverException e) {
         int currentExpiredTransactions = getExpiredTransactionsCount();
         LOG.info("Current expired transactions = {}", currentExpiredTransactions);
         assertTrue(currentExpiredTransactions > initialExpiredTransactions);

src/yb/client/client.cc

@@ -471,6 +471,7 @@ Status YBClientBuilder::DoBuild(rpc::Messenger* messenger, std::unique_ptr<YBCli
   // is clearer to see "mastercb" instead.
   ThreadPoolBuilder tpb(data_->client_name_ + "cb");
   tpb.set_max_threads(callback_threadpool_size);
+  tpb.set_min_threads(1);
   std::unique_ptr<ThreadPool> tp;
   RETURN_NOT_OK_PREPEND(
       tpb.Build(&tp),

src/yb/client/transaction.cc

@@ -306,6 +306,16 @@ class YBTransaction::Impl final : public internal::TxnBatcherIf {
       UNIQUE_LOCK(lock, mutex_);
       const bool defer = !ready_;
 
+      if (!status_.ok()) {
+        auto status = status_;
+        lock.unlock();
+        VLOG_WITH_PREFIX(2) << "Prepare, transaction already failed: " << status;
+        if (waiter) {
+          waiter(status);
+        }
+        return false;
+      }
+
       if (!defer || initial) {
         Status status = CheckTransactionLocality(ops_info);
         if (!status.ok()) {