txn_test_gen_plugin thread pool

[baegjae]

Change Description

• When using txn_test_gen_plugin, the node's main thread will create (sign) and process (accept) transactions.
• However, transaction creation is a client operation, not a node.
• Furthermore, the transaction creation overhead is high, affecting performance results.
• This PR adds a thread pool, which creates transactions and posts them to the main thread.
• The following experiment shows the results of applying txn_test_gen thread pool.

Environments

• Node: 1 producing node (start_generation here)
• HW: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz (44 cores), 128GB
• Options: 8 producer threads, 8 chain threads, 2 txn_test_gen threads, wavm
• Benchmark: generate 20 transactions every 2 ms (intended 10,000 TPS) for 30 seconds

Note that this configuration is intended to focus on the performance of processing transactions except network overhead.

Before

• 4,547 TPS, 97% CPU usage (main thread)

After

• 9,992 TPS, 98% CPU usage (main thread), 74% CPU usage (per txn_test_gen thread)

Note that the number of txn_test_gen threads does not affect the intended input parameter.
That is, if a value that generates 20 transactions every 2 ms is input, it tries to generate transactions at this cycle even if the number of threads increases.
However, depending on the CPU performance, it is necessary to use multiple threads to match the input transaction generation cycle.
In above case (20 transactions every 2 ms), two threads were needed in our environment.

Consensus Changes

API Changes

Documentation Additions

• txn_test_gen_plugin has new option:
  --txn-test-gen-threads arg (=2) Number of worker threads in txn_test_gen thread pool

[heifner]

Thanks