Skip to content

Commit

Permalink
Auto merge of #82754 - rylev:rusage-windows, r=pnkfelix
Browse files Browse the repository at this point in the history
Attempt to gather similar stats as rusage on Windows

A follow up to #82532. This is a bit hacked in because I think we need to discuss this before merging, but this is an attempt to gather similar metrics as `libc::rusage` on Windows.

Some comments on differences:
* Currently, we're passing `RUSAGE_CHILDREN` to `rusage` which collects statistics on all children that have been waited on and terminated. I believe this is currently just the invocation of the real `rustc` that the shim is wrapping. Does `rustc` itself spawn children processes? The windows version gets the child processes handle when spawning it, and uses that to collect the statistics. For maxrss, `rusage` will return "the resident set size of the largest child, not the maximum resident set size of the process tree.", the Windows version will only collect statistics on the wrapped `rustc` child process directly even if some theoretical sub process has a larger memory footprint.
* There might be subtle differences between `rusage`'s "resident set" and Window's "working set". The "working set" and "resident set" should both be the number of pages that are in memory and which would not cause a page fault when accessed.
* I'm not yet sure how best to get the same information that `ru_minflt`, `ru_inblock`, `ru_oublock`, `ru_nivcsw ` and `ru_nvcsw` provide.

r? `@pnkfelix`
  • Loading branch information
bors committed Mar 19, 2021
2 parents eb95ace + 302867c commit b97fd3e
Show file tree
Hide file tree
Showing 2 changed files with 76 additions and 10 deletions.
2 changes: 1 addition & 1 deletion src/bootstrap/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ merge = "0.1.0"

[target.'cfg(windows)'.dependencies.winapi]
version = "0.3"
features = ["fileapi", "ioapiset", "jobapi2", "handleapi", "winioctl"]
features = ["fileapi", "ioapiset", "jobapi2", "handleapi", "winioctl", "psapi", "impl-default"]

[dev-dependencies]
pretty_assertions = "0.6"
84 changes: 75 additions & 9 deletions src/bootstrap/bin/rustc.rs
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@

use std::env;
use std::path::PathBuf;
use std::process::Command;
use std::process::{Child, Command};
use std::str::FromStr;
use std::time::Instant;

Expand Down Expand Up @@ -163,9 +163,11 @@ fn main() {
}

let start = Instant::now();
let status = {
let (child, status) = {
let errmsg = format!("\nFailed to run:\n{:?}\n-------------", cmd);
cmd.status().expect(&errmsg)
let mut child = cmd.spawn().expect(&errmsg);
let status = child.wait().expect(&errmsg);
(child, status)
};

if env::var_os("RUSTC_PRINT_STEP_TIMINGS").is_some()
Expand All @@ -176,7 +178,7 @@ fn main() {
// If the user requested resource usage data, then
// include that in addition to the timing output.
let rusage_data =
env::var_os("RUSTC_PRINT_STEP_RUSAGE").and_then(|_| format_rusage_data());
env::var_os("RUSTC_PRINT_STEP_RUSAGE").and_then(|_| format_rusage_data(child));
eprintln!(
"[RUSTC-TIMING] {} test:{} {}.{:03}{}{}",
crate_name,
Expand Down Expand Up @@ -213,19 +215,83 @@ fn main() {
}
}

#[cfg(not(unix))]
/// getrusage is not available on non-unix platforms. So for now, we do not
/// bother trying to make a shim for it.
fn format_rusage_data() -> Option<String> {
#[cfg(all(not(unix), not(windows)))]
// In the future we can add this for more platforms
fn format_rusage_data(_child: Child) -> Option<String> {
None
}

#[cfg(windows)]
fn format_rusage_data(child: Child) -> Option<String> {
use std::os::windows::io::AsRawHandle;
use winapi::um::{processthreadsapi, psapi, timezoneapi};
let handle = child.as_raw_handle();
macro_rules! try_bool {
($e:expr) => {
if $e != 1 {
return None;
}
};
}

let mut user_filetime = Default::default();
let mut user_time = Default::default();
let mut kernel_filetime = Default::default();
let mut kernel_time = Default::default();
let mut memory_counters = psapi::PROCESS_MEMORY_COUNTERS::default();

unsafe {
try_bool!(processthreadsapi::GetProcessTimes(
handle,
&mut Default::default(),
&mut Default::default(),
&mut kernel_filetime,
&mut user_filetime,
));
try_bool!(timezoneapi::FileTimeToSystemTime(&user_filetime, &mut user_time));
try_bool!(timezoneapi::FileTimeToSystemTime(&kernel_filetime, &mut kernel_time));

// Unlike on Linux with RUSAGE_CHILDREN, this will only return memory information for the process
// with the given handle and none of that process's children.
try_bool!(psapi::GetProcessMemoryInfo(
handle as _,
&mut memory_counters as *mut _ as _,
std::mem::size_of::<psapi::PROCESS_MEMORY_COUNTERS_EX>() as u32,
));
}

// Guide on interpreting these numbers:
// https://docs.microsoft.com/en-us/windows/win32/psapi/process-memory-usage-information
let peak_working_set = memory_counters.PeakWorkingSetSize / 1024;
let peak_page_file = memory_counters.PeakPagefileUsage / 1024;
let peak_paged_pool = memory_counters.QuotaPeakPagedPoolUsage / 1024;
let peak_nonpaged_pool = memory_counters.QuotaPeakNonPagedPoolUsage / 1024;
Some(format!(
"user: {USER_SEC}.{USER_USEC:03} \
sys: {SYS_SEC}.{SYS_USEC:03} \
peak working set (kb): {PEAK_WORKING_SET} \
peak page file usage (kb): {PEAK_PAGE_FILE} \
peak paged pool usage (kb): {PEAK_PAGED_POOL} \
peak non-paged pool usage (kb): {PEAK_NONPAGED_POOL} \
page faults: {PAGE_FAULTS}",
USER_SEC = user_time.wSecond + (user_time.wMinute * 60),
USER_USEC = user_time.wMilliseconds,
SYS_SEC = kernel_time.wSecond + (kernel_time.wMinute * 60),
SYS_USEC = kernel_time.wMilliseconds,
PEAK_WORKING_SET = peak_working_set,
PEAK_PAGE_FILE = peak_page_file,
PEAK_PAGED_POOL = peak_paged_pool,
PEAK_NONPAGED_POOL = peak_nonpaged_pool,
PAGE_FAULTS = memory_counters.PageFaultCount,
))
}

#[cfg(unix)]
/// Tries to build a string with human readable data for several of the rusage
/// fields. Note that we are focusing mainly on data that we believe to be
/// supplied on Linux (the `rusage` struct has other fields in it but they are
/// currently unsupported by Linux).
fn format_rusage_data() -> Option<String> {
fn format_rusage_data(_child: Child) -> Option<String> {
let rusage: libc::rusage = unsafe {
let mut recv = std::mem::zeroed();
// -1 is RUSAGE_CHILDREN, which means to get the rusage for all children
Expand Down

0 comments on commit b97fd3e

Please sign in to comment.