reactor, linux-aio: recommend CPU count reduction, too

When the system can satisfy the AIO requirements by reducing the network IOCBs count, don't only recommend increasing "aio-max-nr" for meeting the network IOCBs count, but also advise a suitable reduction of CPUs. The warning remains unchanged with "aio-max-nr=32768" and "--smp=8", for example -- even with a single CPU, the ideal AIO count is 51026: > DEBUG seastar - Intended AIO control block usage: > DEBUG seastar - > DEBUG seastar - purpose per cpu all 8 cpus > DEBUG seastar - ------- ------- ---------- > DEBUG seastar - reserve 0 > DEBUG seastar - storage 1024 8192 > DEBUG seastar - preempt 2 16 > DEBUG seastar - network 50000 400000 > DEBUG seastar - ------- ------- ---------- > DEBUG seastar - total 51026 408208 > DEBUG seastar - > DEBUG seastar - Available AIO control blocks = aio-max-nr - aio-nr = > 32768 - 0 = 32768 > WARN seastar - Your system does not have enough AIO capacity for > optimal network performance; reducing > `max-networking-io-control-blocks'. > WARN seastar - Resultant AIO control block usage: > WARN seastar - > WARN seastar - purpose per cpu all 8 cpus > WARN seastar - ------- ------- ---------- > WARN seastar - reserve 0 > WARN seastar - storage 1024 8192 > WARN seastar - preempt 2 16 > WARN seastar - network 3070 24560 > WARN seastar - ------- ------- ---------- > WARN seastar - total 4096 32768 > WARN seastar - > WARN seastar - For optimal network performance, set > /proc/sys/fs/aio-max-nr to at least 408208. The new hint is included with "aio-max-nr=200000" and "--smp=8", for example: > DEBUG seastar - Intended AIO control block usage: > DEBUG seastar - > DEBUG seastar - purpose per cpu all 8 cpus > DEBUG seastar - ------- ------- ---------- > DEBUG seastar - reserve 0 > DEBUG seastar - storage 1024 8192 > DEBUG seastar - preempt 2 16 > DEBUG seastar - network 50000 400000 > DEBUG seastar - ------- ------- ---------- > DEBUG seastar - total 51026 408208 > DEBUG seastar - > DEBUG seastar - Available AIO control blocks = aio-max-nr - aio-nr = > 200000 - 0 = 200000 > WARN seastar - Your system does not have enough AIO capacity for > optimal network performance; reducing > `max-networking-io-control-blocks'. > WARN seastar - Resultant AIO control block usage: > WARN seastar - > WARN seastar - purpose per cpu all 8 cpus > WARN seastar - ------- ------- ---------- > WARN seastar - reserve 0 > WARN seastar - storage 1024 8192 > WARN seastar - preempt 2 16 > WARN seastar - network 23974 191792 > WARN seastar - ------- ------- ---------- > WARN seastar - total 25000 200000 > WARN seastar - > WARN seastar - For optimal network performance, set > /proc/sys/fs/aio-max-nr to at least 408208, or > decrease the logical CPU count of the application to > 3. And then with "aio-max-nr=200000" and "--smp=3", the whole warning disappears: > DEBUG seastar - Intended AIO control block usage: > DEBUG seastar - > DEBUG seastar - purpose per cpu all 3 cpus > DEBUG seastar - ------- ------- ---------- > DEBUG seastar - reserve 0 > DEBUG seastar - storage 1024 3072 > DEBUG seastar - preempt 2 6 > DEBUG seastar - network 50000 150000 > DEBUG seastar - ------- ------- ---------- > DEBUG seastar - total 51026 153078 > DEBUG seastar - > DEBUG seastar - Available AIO control blocks = aio-max-nr - aio-nr = > 200000 - 0 = 200000 Signed-off-by: Laszlo Ersek <[email protected]>
scylladb · Nov 26, 2024 · 92e29ab · 92e29ab
1 parent f9d7468
commit 92e29ab
Showing 1 changed file with 7 additions and 1 deletion.
diff --git a/src/core/reactor.cc b/src/core/reactor.cc
@@ -4190,13 +4190,19 @@ unsigned smp::adjust_max_networking_aio_io_control_blocks(unsigned network_iocbs
         };
 
         if (available_aio >= requested_aio_other + smp::count) { // at least one queue for each shard
+            unsigned smp_count_recommended = get_smp_count(network_iocbs);
             network_iocbs = (available_aio - requested_aio_other) / smp::count;
             seastar_logger.warn("Your system does not have enough AIO capacity for optimal network performance; reducing `max-networking-io-control-blocks'.");
             seastar_logger.warn("Resultant AIO control block usage:");
             seastar_logger.warn("");
             log_aiocbs(log_level::warn, storage_iocbs, preempt_iocbs, network_iocbs, reserve_iocbs);
             seastar_logger.warn("");
-            seastar_logger.warn("For optimal network performance, set /proc/sys/fs/aio-max-nr to at least {}.", aio_nr + requested_aio);
+            std::string advice = format("For optimal network performance, set /proc/sys/fs/aio-max-nr to at least {}", aio_nr + requested_aio);
+            if (smp_count_recommended > 0) {
+                advice.append(format(", or decrease the logical CPU count of the application to {}", smp_count_recommended));
+            }
+            advice.append(".");
+            seastar_logger.warn("{}", advice);
         } else {
             std::string err = format("Your system does not satisfy minimum AIO requirements. "
                                      "Set /proc/sys/fs/aio-max-nr to at least {} (minimum) or {} (recommended for networking performance)",