Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rollup of 10 pull requests #83279

Closed
wants to merge 29 commits into from

Conversation

Dylan-DPC-zz
Copy link

Successful merges:

Failed merges:

r? @ghost
@rustbot modify labels: rollup

Create a similar rollup

GuillaumeGomez and others added 29 commits December 14, 2020 11:57
Background:

Over the last year, pidfd support was added to the Linux kernel. This
allows interacting with other processes. In particular, this allows
waiting on a child process with a timeout in a race-free way, bypassing
all of the awful signal-handler tricks that are usually required.

Pidfds can be obtained for a child process (as well as any other
process) via the `pidfd_open` syscall. Unfortunately, this requires
several conditions to hold in order to be race-free (i.e. the pid is not
reused).
Per `man pidfd_open`:

```
· the disposition of SIGCHLD has not been explicitly set to SIG_IGN
 (see sigaction(2));

· the SA_NOCLDWAIT flag was not specified while establishing a han‐
 dler for SIGCHLD or while setting the disposition of that signal to
 SIG_DFL (see sigaction(2)); and

· the zombie process was not reaped elsewhere in the program (e.g.,
 either by an asynchronously executed signal handler or by wait(2)
 or similar in another thread).

If any of these conditions does not hold, then the child process
(along with a PID file descriptor that refers to it) should instead
be created using clone(2) with the CLONE_PIDFD flag.
```

Sadly, these conditions are impossible to guarantee once any libraries
are used. For example, C code runnng in a different thread could call
`wait()`, which is impossible to detect from Rust code trying to open a
pidfd.

While pid reuse issues should (hopefully) be rare in practice, we can do
better. By passing the `CLONE_PIDFD` flag to `clone()` or `clone3()`, we
can obtain a pidfd for the child process in a guaranteed race-free
manner.

This PR:

This PR adds Linux-specific process extension methods to allow obtaining
pidfds for processes spawned via the standard `Command` API. Other than
being made available to user code, the standard library does not make
use of these pidfds in any way. In particular, the implementation of
`Child::wait` is completely unchanged.

Two Linux-specific helper methods are added: `CommandExt::create_pidfd`
and `ChildExt::pidfd`. These methods are intended to serve as a building
block for libraries to build higher-level abstractions - in particular,
waiting on a process with a timeout.

I've included a basic test, which verifies that pidfds are created iff
the `create_pidfd` method is used. This test is somewhat special - it
should always succeed on systems with the `clone3` system call
available, and always fail on systems without `clone3` available. I'm
not sure how to best ensure this programatically.

This PR relies on the newer `clone3` system call to pass the `CLONE_FD`,
rather than the older `clone` system call. `clone3` was added to Linux
in the same release as pidfds, so this shouldn't unnecessarily limit the
kernel versions that this code supports.

Unresolved questions:
* What should the name of the feature gate be for these newly added
  methods?
* Should the `pidfd` method distinguish between an error occurring
  and `create_pidfd` not being called?
Co-authored-by: bjorn3 <[email protected]>
This also relaxes the bounds on some structs and moves them to the impl
block instead.
Unlike the other cases of this lint, there's no simple way to detect if
an old version of the relevant crate (`syn`) is in use. The `actix-web`
crate only depends on `pin-project` v1.0.0, so checking the version of
`actix-web` does not guarantee that a new enough version of
`pin-project` (and therefore `syn`) is in use.

Instead, we rely on the fact that virtually all of the regressed crates
are pinned to a pre-1.0 version of `pin-project`. When this is the case,
bumping the `actix-web` dependency will pull in the *latest* version of
`pin-project`, which has an explicit dependency on a newer v dependency
on a newer version of `syn`.

The lint message tells users to update `actix-web`, since that's what
they're most likely to have control over. We could potentially tell them
to run `cargo update -p syn`, but I think it's more straightforward to
suggest an explicit change to the `Cargo.toml`

The `actori-web` fork had its last commit over a year ago, and appears
to just be a renamed fork of `actix-web`. Therefore, I've removed the
`actori-web` check entirely - any crates that actually get broken can
simply update `syn` themselves.
One of the examples used to say “this leads to a possibly confusing situation,
where the type of the closure is a double reference” while _actually_ referring to
the type of the closure _argument_.
…joshtriplett

Implement String::remove_matches

Closes rust-lang#50206.

I lifted the function help from `@frewsxcv's` original PR (rust-lang#50015), hope they don't mind.

I'm also wondering whether it would be useful for `remove_matches` to collect up the removed substrings into a `Vec` and return them, right now they're just overwritten by the copy and lost.
…ed, r=Nemo157

Only build help popup when it's really needed

When working on rust-lang#79985, I realized that the help popup was built even when it wasn't needed. This PR only makes the help popup to be built when required.

r? `@jyn514`
Add Linux-specific pidfd process extensions (take 2)

Continuation of rust-lang#77168.
I addressed the following concerns from the original PR:

- make `CommandExt` and `ChildExt` sealed traits
- wrap file descriptors in `PidFd` struct representing ownership over the fd
- add `take_pidfd` to take the fd out of `Child`
- close fd when dropped

Tracking Issue: rust-lang#82971
Reuse `std::sys::unsupported::pipe` on `hermit`

Pipes are not supported on `hermit` and `hermit/pipe.rs` is identical to `unsupported/pipe.rs`. This PR reduces duplication between the two by doing the following on `hermit`:

```rust
#[path = "../unsupported/pipe.rs"]
pub mod pipe;
```
Attempt to gather similar stats as rusage on Windows

A follow up to rust-lang#82532. This is a bit hacked in because I think we need to discuss this before merging, but this is an attempt to gather similar metrics as `libc::rusage` on Windows.

Some comments on differences:
* Currently, we're passing `RUSAGE_CHILDREN` to `rusage` which collects statistics on all children that have been waited on and terminated. I believe this is currently just the invocation of the real `rustc` that the shim is wrapping. Does `rustc` itself spawn children processes? The windows version gets the child processes handle when spawning it, and uses that to collect the statistics. For maxrss, `rusage` will return "the resident set size of the largest child, not the maximum resident set size of the process tree.", the Windows version will only collect statistics on the wrapped `rustc` child process directly even if some theoretical sub process has a larger memory footprint.
* There might be subtle differences between `rusage`'s "resident set" and Window's "working set". The "working set" and "resident set" should both be the number of pages that are in memory and which would not cause a page fault when accessed.
* I'm not yet sure how best to get the same information that `ru_minflt`, `ru_inblock`, `ru_oublock`, `ru_nivcsw ` and `ru_nvcsw` provide.

r? `@pnkfelix`
…chenkov

Remove unwrap_none/expect_none from compiler/.

We're not going to stabilize `Option::{unwrap_none, expect_none}`. (See rust-lang#62633.) This removes the usage of those unstable methods from `compiler/`.
…enkov

Extend `proc_macro_back_compat` lint to `actix-web`

Unlike the other cases of this lint, there's no simple way to detect if
an old version of the relevant crate (`syn`) is in use. The `actix-web`
crate only depends on `pin-project` v1.0.0, so checking the version of
`actix-web` does not guarantee that a new enough version of
`pin-project` (and therefore `syn`) is in use.

Instead, we rely on the fact that virtually all of the regressed crates
are pinned to a pre-1.0 version of `pin-project`. When this is the case,
bumping the `actix-web` dependency will pull in the *latest* version of
`pin-project`, which has an explicit dependency on a newer v dependency
on a newer version of `syn`.

The lint message tells users to update `actix-web`, since that's what
they're most likely to have control over. We could potentially tell them
to run `cargo update -p syn`, but I think it's more straightforward to
suggest an explicit change to the `Cargo.toml`

The `actori-web` fork had its last commit over a year ago, and appears
to just be a renamed fork of `actix-web`. Therefore, I've removed the
`actori-web` check entirely - any crates that actually get broken can
simply update `syn` themselves.
…iplett

Move some test-only code to test files

Split out from rust-lang#83185.
…_doc, r=joshtriplett

Fix typo/inaccuracy in the documentation of Iterator::skip_while

One of the examples used to say “this leads to a possibly confusing situation, where the type of the closure is a double reference” while _actually_ referring to the type of the closure _argument_.

This PR just changes a single word in documentation.

`@rustbot` modify labels: A-iterators, T-doc, T-lang
@rustbot rustbot added the rollup A PR which is a rollup label Mar 18, 2021
@Dylan-DPC-zz
Copy link
Author

@bors r+ rollup=never p=5

@bors
Copy link
Contributor

bors commented Mar 18, 2021

📌 Commit 6f61327 has been approved by Dylan-DPC

@bors bors added the S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. label Mar 18, 2021
@rust-log-analyzer
Copy link
Collaborator

The job mingw-check failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)
configure: rust.channel         := nightly
configure: rust.debug-assertions := True
configure: llvm.assertions      := True
configure: dist.missing-tools   := True
configure: build.configure-args := ['--enable-sccache', '--disable-manage-submodu ...
configure: writing `config.toml` in current directory
configure: 
configure: run `python /checkout/x.py --help`
configure: 
---
Checking which error codes lack tests...
Found 435 error codes
Found 0 error codes with no tests
Done!
tidy error: /checkout/src/librustdoc/html/static/main.js: too many lines (3001) (add `// ignore-tidy-filelength` to the file to suppress this error)
some tidy checks failed


command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/rust-tidy" "/checkout" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "/checkout/obj/build"


failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test --stage 2 src/tools/tidy
Build completed unsuccessfully in 0:00:16

@Aaron1011
Copy link
Member

@bors r-

@bors bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Mar 18, 2021
@Dylan-DPC-zz
Copy link
Author

oops thought i'd done it already :) thanks

@Dylan-DPC-zz Dylan-DPC-zz deleted the rollup-vcgnc7h branch March 18, 2021 23:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
rollup A PR which is a rollup S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author.
Projects
None yet
Development

Successfully merging this pull request may close these issues.