Auto merge of rust-lang#14859 - lunacookies:qos, r=lunacookies

Specify thread types using Quality of Service API

<details>
<summary>Some background (in case you haven’t heard of QoS before)</summary>

Heterogenous multi-core CPUs are increasingly found in laptops and desktops (e.g. Alder Lake, Snapdragon 8cx Gen 3, M1). To maximize efficiency on this kind of hardware, it is important to provide the operating system with more information so threads can be scheduled on different core types appropriately.

The approach that XNU (the kernel of macOS, iOS, etc) and Windows have taken is to provide a high-level semantic API – quality of service, or QoS – which informs the OS of the program’s intent. For instance, you might specify that a thread is running a render loop for a game. This makes the OS provide this thread with as large a share of the system’s resources as possible. Specifying a thread is running an unimportant background task, on the other hand, is cause for it to be scheduled exclusively on high-efficiency cores instead of high-performance cores.

QoS APIs allows for easy configuration of many different parameters at once; for instance, setting QoS on XNU affects scheduling, timer latency, I/O priorities, and of course what core type the thread in question should run on. I don’t know any details on how QoS works on Windows, but I would guess it’s similar.

Hypothetically, taking advantage of these APIs would improve power consumption, thermals, battery life if applicable, etc.

</details>

# Relevance to rust-analyzer

From what I can tell the philosophy behind both the XNU and Windows QoS APIs is that _user interfaces should never stutter under any circumstances._ You can see this in the array of QoS classes which are available: the highest QoS class in both APIs is one intended explicitly for UI render loops.

Imagine rust-analyzer is performing CPU-intensive background work – maybe you just invoked Find Usages on `usize` or opened a large project – in this scenario the editor’s render loop should absolutely get higher priority than rust-analyzer, no matter what. You could view it in terms of “realtime-ness”: flight control software is hard realtime, audio software is soft realtime, GUIs are softer realtime, and rust-analyzer is not realtime at all. Of course, maximizing responsiveness is important, but respecting the rest of the system is more important.

# Implementation

I’ve tried my best to unify thread creation in `stdx`, where the new API I’ve introduced _requires_ specifying a QoS class. Different points along the performance/efficiency curve can make a great difference; the M1’s e-cores use around three times less power than the p-cores, so putting in this effort is worthwhile IMO.

It’s worth mentioning that Linux does not [yet](https://youtu.be/RfgPWpTwTQo) have a QoS API. Maybe translating QoS into regular thread priorities would be acceptable? From what I can tell the only scheduling-related code in rust-analyzer is Windows-specific, so ignoring QoS entirely on Linux shouldn’t cause any new issues. Also, I haven’t implemented support for the Windows QoS APIs because I don’t have a Windows machine to test on, and because I’m completely unfamiliar with Windows APIs :)

I noticed that rust-analyzer handles some requests on the main thread (using `.on_sync()`) and others on a threadpool (using `.on()`). I think it would make sense to run the main thread at the User Initiated QoS and the threadpool at Utility, but only if all requests that are caused by typing use `.on_sync()` and all that don’t use `.on()`. I don’t understand how the `.on_sync()`/`.on()` split that’s currently present was chosen, so I’ve let this code be for the moment. Let me know if changing this to what I proposed makes any sense.

To avoid having to change everything back in case I’ve misunderstood something, I’ve left all threads at the Utility QoS for now. Of course, this isn’t what I hope the code will look like in the end, but I figured I have to start somewhere :P

# References

<ul>

<li><a href="https://developer.apple.com/library/archive/documentation/Performance/Conceptual/power_efficiency_guidelines_osx/PrioritizeWorkAtTheTaskLevel.html">Apple documentation related to QoS</a></li>
<li><a href="https://github.com/apple-oss-distributions/libpthread/blob/67e155c94093be9a204b69637d198eceff2c7c46/include/pthread/qos.h">pthread API for setting QoS on XNU</a></li>
<li><a href="https://learn.microsoft.com/en-us/windows/win32/procthread/quality-of-service">Windows’s QoS classes</a></li>
<li>
<details>
<summary>Full documentation of XNU QoS classes. This documentation is only available as a huge not-very-readable comment in a header file, so I’ve reformatted it and put it here for reference.</summary>
<ul>
<li><p><strong><code>QOS_CLASS_USER_INTERACTIVE</code>: A QOS class which indicates work performed by this thread is interactive with the user.</strong></p><p>Such work is requested to run at high priority relative to other work on the system. Specifying this QOS class is a request to run with nearly all available system CPU and I/O bandwidth even under contention. This is not an energy-efficient QOS class to use for large tasks. The use of this QOS class should be limited to critical interaction with the user such as handling events on the main event loop, view drawing, animation, etc.</p></li>
<li><p><strong><code>QOS_CLASS_USER_INITIATED</code>: A QOS class which indicates work performed by this thread was initiated by the user and that the user is likely waiting for the results.</strong></p><p>Such work is requested to run at a priority below critical user-interactive work, but relatively higher than other work on the system. This is not an energy-efficient QOS class to use for large tasks. Its use should be limited to operations of short enough duration that the user is unlikely to switch tasks while waiting for the results. Typical user-initiated work will have progress indicated by the display of placeholder content or modal user interface.</p></li>
<li><p><strong><code>QOS_CLASS_DEFAULT</code>: A default QOS class used by the system in cases where more specific QOS class information is not available.</strong></p><p>Such work is requested to run at a priority below critical user-interactive and user-initiated work, but relatively higher than utility and background tasks. Threads created by <code>pthread_create()</code> without an attribute specifying a QOS class will default to <code>QOS_CLASS_DEFAULT</code>. This QOS class value is not intended to be used as a work classification, it should only be set when propagating or restoring QOS class values provided by the system.</p></li>
<li><p><strong><code>QOS_CLASS_UTILITY</code>: A QOS class which indicates work performed by this thread may or may not be initiated by the user and that the user is unlikely to be immediately waiting for the results.</strong></p><p>Such work is requested to run at a priority below critical user-interactive and user-initiated work, but relatively higher than low-level system maintenance tasks. The use of this QOS class indicates the work should be run in an energy and thermally-efficient manner. The progress of utility work may or may not be indicated to the user, but the effect of such work is user-visible.</p></li>
<li><p><strong><code>QOS_CLASS_BACKGROUND</code>: A QOS class which indicates work performed by this thread was not initiated by the user and that the user may be unaware of the results.</strong></p><p>Such work is requested to run at a priority below other work. The use of this QOS class indicates the work should be run in the most energy and thermally-efficient manner.</p></li>
<li><p><strong><code>QOS_CLASS_UNSPECIFIED</code>: A QOS class value which indicates the absence or removal of QOS class information.</strong></p><p>As an API return value, may indicate that threads or pthread attributes were configured with legacy API incompatible or in conflict with the QOS class system.</p></li>
</ul>
</details>
</li>

</ul>

crates/flycheck/Cargo.toml

@@ -18,7 +18,6 @@ cargo_metadata = "0.15.0"
 rustc-hash = "1.1.0"
 serde_json.workspace = true
 serde.workspace = true
-jod-thread = "0.1.2"
 command-group = "2.0.1"
 
 # local deps

crates/flycheck/src/lib.rs

@@ -77,7 +77,7 @@ impl fmt::Display for FlycheckConfig {
 pub struct FlycheckHandle {
     // XXX: drop order is significant
     sender: Sender<StateChange>,
-    _thread: jod_thread::JoinHandle,
+    _thread: stdx::thread::JoinHandle,
     id: usize,
 }
 
@@ -90,7 +90,7 @@ impl FlycheckHandle {
     ) -> FlycheckHandle {
         let actor = FlycheckActor::new(id, sender, config, workspace_root);
         let (sender, receiver) = unbounded::<StateChange>();
-        let thread = jod_thread::Builder::new()
+        let thread = stdx::thread::Builder::new(stdx::thread::QoSClass::Utility)
             .name("Flycheck".to_owned())
             .spawn(move || actor.run(receiver))
             .expect("failed to spawn thread");
@@ -395,7 +395,7 @@ struct CargoHandle {
     /// The handle to the actual cargo process. As we cannot cancel directly from with
     /// a read syscall dropping and therefore terminating the process is our best option.
     child: JodGroupChild,
-    thread: jod_thread::JoinHandle<io::Result<(bool, String)>>,
+    thread: stdx::thread::JoinHandle<io::Result<(bool, String)>>,
     receiver: Receiver<CargoMessage>,
 }
 
@@ -409,7 +409,7 @@ impl CargoHandle {
 
         let (sender, receiver) = unbounded();
         let actor = CargoActor::new(sender, stdout, stderr);
-        let thread = jod_thread::Builder::new()
+        let thread = stdx::thread::Builder::new(stdx::thread::QoSClass::Utility)
             .name("CargoHandle".to_owned())
             .spawn(move || actor.run())
             .expect("failed to spawn thread");

crates/ide/src/prime_caches.rs

@@ -80,7 +80,11 @@ pub(crate) fn parallel_prime_caches(
         for _ in 0..num_worker_threads {
             let worker = prime_caches_worker.clone();
             let db = db.snapshot();
-            std::thread::spawn(move || Cancelled::catch(|| worker(db)));
+
+            stdx::thread::Builder::new(stdx::thread::QoSClass::Utility)
+                .allow_leak(true)
+                .spawn(move || Cancelled::catch(|| worker(db)))
+                .expect("failed to spawn thread");
         }
 
         (work_sender, progress_receiver)

crates/proc-macro-srv/Cargo.toml

@@ -22,6 +22,7 @@ object = { version = "0.30.2", default-features = false, features = [
 libloading = "0.7.3"
 memmap2 = "0.5.4"
 
+stdx.workspace = true
 tt.workspace = true
 mbe.workspace = true
 paths.workspace = true

crates/rust-analyzer/Cargo.toml

@@ -86,7 +86,6 @@ jemallocator = { version = "0.5.0", package = "tikv-jemallocator", optional = tr
 
 [dev-dependencies]
 expect-test = "1.4.0"
-jod-thread = "0.1.2"
 xshell = "0.2.2"
 
 test-utils.workspace = true

crates/rust-analyzer/src/bin/main.rs

@@ -78,7 +78,14 @@ fn try_main(flags: flags::RustAnalyzer) -> Result<()> {
                 println!("rust-analyzer {}", rust_analyzer::version());
                 return Ok(());
             }
-            with_extra_thread("LspServer", run_server)?;
+
+            // rust-analyzer’s “main thread” is actually a secondary thread
+            // with an increased stack size at the User Initiated QoS class.
+            // We use this QoS class because any delay in the main loop
+            // will make actions like hitting enter in the editor slow.
+            // rust-analyzer does not block the editor’s render loop,
+            // so we don’t use User Interactive.
+            with_extra_thread("LspServer", stdx::thread::QoSClass::UserInitiated, run_server)?;
         }
         flags::RustAnalyzerCmd::Parse(cmd) => cmd.run()?,
         flags::RustAnalyzerCmd::Symbols(cmd) => cmd.run()?,
@@ -136,14 +143,17 @@ const STACK_SIZE: usize = 1024 * 1024 * 8;
 /// space.
 fn with_extra_thread(
     thread_name: impl Into<String>,
+    qos_class: stdx::thread::QoSClass,
     f: impl FnOnce() -> Result<()> + Send + 'static,
 ) -> Result<()> {
-    let handle =
-        std::thread::Builder::new().name(thread_name.into()).stack_size(STACK_SIZE).spawn(f)?;
-    match handle.join() {
-        Ok(res) => res,
-        Err(panic) => std::panic::resume_unwind(panic),
-    }
+    let handle = stdx::thread::Builder::new(qos_class)
+        .name(thread_name.into())
+        .stack_size(STACK_SIZE)
+        .spawn(f)?;
+
+    handle.join()?;
+
+    Ok(())
 }
 
 fn run_server() -> Result<()> {

crates/rust-analyzer/src/main_loop.rs

@@ -665,14 +665,20 @@ impl GlobalState {
         use crate::handlers::request as handlers;
 
         dispatcher
+            // Request handlers that must run on the main thread
+            // because they mutate GlobalState:
             .on_sync_mut::<lsp_ext::ReloadWorkspace>(handlers::handle_workspace_reload)
             .on_sync_mut::<lsp_ext::RebuildProcMacros>(handlers::handle_proc_macros_rebuild)
             .on_sync_mut::<lsp_ext::MemoryUsage>(handlers::handle_memory_usage)
             .on_sync_mut::<lsp_ext::ShuffleCrateGraph>(handlers::handle_shuffle_crate_graph)
+            // Request handlers which are related to the user typing
+            // are run on the main thread to reduce latency:
             .on_sync::<lsp_ext::JoinLines>(handlers::handle_join_lines)
             .on_sync::<lsp_ext::OnEnter>(handlers::handle_on_enter)
             .on_sync::<lsp_types::request::SelectionRangeRequest>(handlers::handle_selection_range)
             .on_sync::<lsp_ext::MatchingBrace>(handlers::handle_matching_brace)
+            .on_sync::<lsp_ext::OnTypeFormatting>(handlers::handle_on_type_formatting)
+            // All other request handlers:
             .on::<lsp_ext::FetchDependencyList>(handlers::fetch_dependency_list)
             .on::<lsp_ext::AnalyzerStatus>(handlers::handle_analyzer_status)
             .on::<lsp_ext::SyntaxTree>(handlers::handle_syntax_tree)
@@ -693,7 +699,6 @@ impl GlobalState {
             .on::<lsp_ext::OpenCargoToml>(handlers::handle_open_cargo_toml)
             .on::<lsp_ext::MoveItem>(handlers::handle_move_item)
             .on::<lsp_ext::WorkspaceSymbol>(handlers::handle_workspace_symbol)
-            .on::<lsp_ext::OnTypeFormatting>(handlers::handle_on_type_formatting)
             .on::<lsp_types::request::DocumentSymbolRequest>(handlers::handle_document_symbol)
             .on::<lsp_types::request::GotoDefinition>(handlers::handle_goto_definition)
             .on::<lsp_types::request::GotoDeclaration>(handlers::handle_goto_declaration)

crates/rust-analyzer/src/task_pool.rs

@@ -1,5 +1,7 @@
 //! A thin wrapper around `ThreadPool` to make sure that we join all things
 //! properly.
+use std::sync::{Arc, Barrier};
+
 use crossbeam_channel::Sender;
 
 pub(crate) struct TaskPool<T> {
@@ -16,6 +18,18 @@ impl<T> TaskPool<T> {
             .thread_stack_size(STACK_SIZE)
             .num_threads(threads)
             .build();
+
+        // Set QoS of all threads in threadpool.
+        let barrier = Arc::new(Barrier::new(threads + 1));
+        for _ in 0..threads {
+            let barrier = barrier.clone();
+            inner.execute(move || {
+                stdx::thread::set_current_thread_qos_class(stdx::thread::QoSClass::Utility);
+                barrier.wait();
+            });
+        }
+        barrier.wait();
+
         TaskPool { sender, inner }
     }
 
@@ -26,7 +40,16 @@ impl<T> TaskPool<T> {
     {
         self.inner.execute({
             let sender = self.sender.clone();
-            move || sender.send(task()).unwrap()
+            move || {
+                if stdx::thread::IS_QOS_AVAILABLE {
+                    debug_assert_eq!(
+                        stdx::thread::get_current_thread_qos_class(),
+                        Some(stdx::thread::QoSClass::Utility)
+                    );
+                }
+
+                sender.send(task()).unwrap()
+            }
         })
     }
 

crates/rust-analyzer/tests/slow-tests/support.rs

@@ -155,7 +155,7 @@ pub(crate) fn project(fixture: &str) -> Server {
 pub(crate) struct Server {
     req_id: Cell<i32>,
     messages: RefCell<Vec<Message>>,
-    _thread: jod_thread::JoinHandle<()>,
+    _thread: stdx::thread::JoinHandle,
     client: Connection,
     /// XXX: remove the tempdir last
     dir: TestDir,
@@ -165,7 +165,7 @@ impl Server {
     fn new(dir: TestDir, config: Config) -> Server {
         let (connection, client) = Connection::memory();
 
-        let _thread = jod_thread::Builder::new()
+        let _thread = stdx::thread::Builder::new(stdx::thread::QoSClass::Utility)
             .name("test server".to_string())
             .spawn(move || main_loop(config, connection).unwrap())
             .expect("failed to spawn a thread");

crates/stdx/Cargo.toml

@@ -15,6 +15,7 @@ doctest = false
 libc = "0.2.135"
 backtrace = { version = "0.3.65", optional = true }
 always-assert = { version = "0.1.2", features = ["log"] }
+jod-thread = "0.1.2"
 # Think twice before adding anything here
 
 [target.'cfg(windows)'.dependencies]

crates/stdx/src/lib.rs

@@ -11,6 +11,7 @@ pub mod process;
 pub mod panic_context;
 pub mod non_empty_vec;
 pub mod rand;
+pub mod thread;
 
 pub use always_assert::{always, never};