Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Process memory #16

Merged
merged 3 commits into from
Jan 28, 2016
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,6 @@ authors = ["Thijs Cadier <[email protected]>",
"Robert Beekman <[email protected]>",
"Dax Huiberts <[email protected]>",
"Timon Vonk <[email protected]>"]

[dependencies]
libc = "*"
1 change: 1 addition & 0 deletions fixtures/linux/process_memory/proc_self_statm
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
6373 1138 428 214 0 755 0
1 change: 1 addition & 0 deletions fixtures/linux/process_memory/proc_self_statm_garbage
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
eceveveev
1 change: 1 addition & 0 deletions fixtures/linux/process_memory/proc_self_statm_incomplete
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
1086
3 changes: 3 additions & 0 deletions src/lib.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
extern crate libc;

mod error;
pub mod load;
pub mod memory;
pub mod cpu_stat;
pub mod process_memory;

use std::fs;
use std::io;
Expand Down
140 changes: 140 additions & 0 deletions src/process_memory.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,140 @@
// These methods are described in:
// http://nadeausoftware.com/articles/2012/07/c_c_tip_how_get_process_resident_set_size_physical_memory_use

use libc;
use super::Result;

/// Get the current RSS memory of this process in KB
#[cfg(target_os = "linux")]
pub fn current_rss() -> Result<u64> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this a different method instead of calling current_rss_of(mypid)? Is reading the memory of this process really the responsibillity of the probe?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought it might be nice to offer this Linux provides a shortcut for it. It is indeed also possible to call current_rss_for with the current pid. We use this function quite a bit, so for us its convenient to have this in place.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok!

os::current_rss()
}

/// Get the current RSS memory of a process with given pid in KB
#[cfg(target_os = "linux")]
pub fn current_rss_of(pid: libc::pid_t) -> Result<u64> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it's an idea to call it memory_usage instead of RSS? I had never heard of it. Also, I was under the impression that all probes would have the read() pub api.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's no such thing as a single memory usage measurement of a process in Linux. There's multiple ways to do it of which measuring RSS seems to be the most useful. Therefore I think it's better to give it a specific name so people are aware what this does. Also if we want to add alternative ways to measure we can add those later.

See http://stackoverflow.com/questions/63166/how-to-determine-cpu-and-memory-consumption-from-inside-a-process for more information.

I split these two out into two separate functions instead of a read because this mirrors functions we already have in our agent and the underlying way of fetching the information is quite different. Also you can only do max_rss for the current process.

I do feel for your consistency argument, not sure yet if this actually fits the pattern though. How about we plan an evening to go over the whole code base together after we complete the basic probes?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's no such thing as a single memory usage measurement of a process in Linux. There's multiple ways to do it of which measuring RSS seems to be the most useful. Therefore I think it's better to give it a specific name so people are aware what this does. Also if we want to add alternative ways to measure we can add those later.

I didn't know that. Ok, makes sense!

I do feel for your consistency argument, not sure yet if this actually fits the pattern though. How about we plan an evening to go over the whole code base together after we complete the basic probes?

Wit holding from beers until we agree 8)

Sure. Since I'm doing this all in my spare time, unpaid besides other projects, bit hesitant of calling a deadline though. And December is already a crazy busy month :o)

os::current_rss_of(pid)
}

/// Get the max RSS memory of this process in KB
#[cfg(target_os = "linux")]
pub fn max_rss() -> u64 {
os::max_rss()
}

#[cfg(target_os = "linux")]
mod os {
use std::mem;
use libc;
use std::path::Path;
use super::super::Result;
use super::super::ProbeError;
use super::super::file_to_string;

#[inline]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why the inlines here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll only call these from the public function, so we're sure there won't be duplicate code if they're inlined.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So not so zero cost abstraction

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's just a hint to the compiler :-)

pub fn current_rss() -> Result<u64> {
read_and_get_current_rss(&Path::new("/proc/self/statm"))
}

#[inline]
pub fn current_rss_of(pid: libc::pid_t) -> Result<u64> {
read_and_get_current_rss(&Path::new(&format!("/proc/{}/statm", pid)))
}

#[inline]
pub fn read_and_get_current_rss(path: &Path) -> Result<u64> {
let raw_data = try!(file_to_string(path));
let segments: Vec<&str> = raw_data.split_whitespace().collect();

if segments.len() < 2 {
return Err(ProbeError::UnexpectedContent("Incorrect number of segments".to_owned()))
}

let pages: u64 = try!(segments[1].parse().map_err(|_| {
ProbeError::UnexpectedContent("Could not parse segment".to_owned())
}));

// Value is in pages, needs to be multiplied by the page size to get a value in KB. We ask the OS
// for this information using sysconf.
let pagesize = unsafe { libc::sysconf(libc::_SC_PAGESIZE) } as u64;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, but why is it unsafe? Or is just whole libc unsafe?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using libc is almost always unsafe since its a C library that doesn't offer the guarantees Rust needs. Therefore we have to check things manually and make sure they are correct.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The compiler will emit a warning if you use an unnecessary unsafe block by the way, so we'd know about it if this was unnecessary.


Ok(pages * pagesize)
}

#[inline]
pub fn max_rss() -> u64 {
let mut rusage: libc::rusage = unsafe { mem::uninitialized() };
unsafe { libc::getrusage(libc::RUSAGE_SELF, &mut rusage) };
rusage.ru_maxrss as u64
}
}

#[cfg(test)]
mod tests {
use libc;
use std::path::Path;
use super::super::ProbeError;

#[test]
fn test_current_rss() {
assert!(super::current_rss().is_ok());
// See if it's a sort of sane value, between 1 and 10 mb
assert!(super::current_rss().unwrap() > 1_000_000);
assert!(super::current_rss().unwrap() < 10_000_000);
}

#[test]
fn test_read_and_get_current_rss() {
let path = Path::new("fixtures/linux/process_memory/proc_self_statm");
let value = super::os::read_and_get_current_rss(&path).unwrap();
assert_eq!(4_661_248, value);
}

#[test]
fn test_read_and_get_current_rss_wrong_path() {
let path = Path::new("/nonsense");
match super::os::read_and_get_current_rss(&path) {
Err(ProbeError::IO(_)) => (),
r => panic!("Unexpected result: {:?}", r)
}
}

#[test]
fn test_read_and_get_current_rss_incomplete() {
let path = Path::new("fixtures/linux/process_memory/proc_self_statm_incomplete");
match super::os::read_and_get_current_rss(&path) {
Err(ProbeError::UnexpectedContent(_)) => (),
r => panic!("Unexpected result: {:?}", r)
}
}

#[test]
fn test_read_and_get_current_rss_garbage() {
let path = Path::new("fixtures/linux/process_memory/proc_self_statm_garbage");
match super::os::read_and_get_current_rss(&path) {
Err(ProbeError::UnexpectedContent(_)) => (),
r => panic!("Unexpected result: {:?}", r)
}
}

#[test]
fn test_current_rss_of() {
let pid = unsafe { libc::getpid() };
assert!(super::current_rss_of(pid).is_ok());
// See if it's a sort of sane value, between 1 and 10 mb
assert!(super::current_rss_of(pid).unwrap() > 1_000_000);
assert!(super::current_rss_of(pid).unwrap() < 10_000_000);
}

#[test]
fn test_current_rss_of_invalid_pid() {
assert!(super::current_rss_of(0).is_err());
}

#[test]
fn test_max_rss() {
// See if it's a sort of sane value, between 1 and 10 mb
assert!(super::current_rss().unwrap() > 1_000_000);
assert!(super::current_rss().unwrap() < 10_000_000);
}
}