Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bootstrap cannot remove host symlink on Windows if created from WSL2 #112544

Open
ChrisDenton opened this issue Jun 12, 2023 · 1 comment · Fixed by #129187
Open

Bootstrap cannot remove host symlink on Windows if created from WSL2 #112544

ChrisDenton opened this issue Jun 12, 2023 · 1 comment · Fixed by #129187
Labels
C-bug Category: This is a bug. O-windows Operating system: Windows T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)

Comments

@ChrisDenton
Copy link
Member

If I use ./x in WSL2 it will create a host symlink in the build directory. If I then switch to the Windows host using the same build directory I get this error:

thread 'main' panicked at 'fs::remove_dir(&host) failed with The directory name is invalid. (os error 267)', lib.rs:516:13
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

The simplest solution here would be to detect ERROR_DIRECTORY (aka os error 267) and fallback to fs::remove_file.

@ChrisDenton ChrisDenton added O-windows Operating system: Windows T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) C-bug Category: This is a bug. labels Jun 12, 2023
bors added a commit to rust-lang-ci/rust that referenced this issue Aug 17, 2024
…nks, r=<try>

bootstrap: fix clean's remove_dir_all implementation

It turns out bootstrap's `clean.rs`'s hand-rolled `rm_rf` (which probably comes before `std::fs::remove_dir_all` was stable) is very broken on native Windows around both read-only files/directories and especially symbolic links. So instead of rolling our own, just use `std::fs::remove_dir_all`. Because I wasn't super sure about `std::fs::remove_dir_all`'s behavior and to catch `std::fs::remove_dir_all`'s behavioral changes here on forward, I added a collection of tests that checks if our expectation of the behavior of `std::fs::remove_dir_all` and its underlying Unix syscalls and Win32 APIs matches with its actual behavior.

This is a blocker for compiletest's own `rm_rf` implementation rust-lang#129155 which happens to be also buggy, which in turn is a blocker for the rmake.rs test port rust-lang#128562 that heavily exercises symlinks (I was reviewing rust-lang#128562 and testing it on native Windows which is how I found out).

I also left a FIXME for `detect_src_and_out` due to a failing assertion on native Windows:

```
---- core::config::tests::detect_src_and_out stdout ----
thread 'core::config::tests::detect_src_and_out' panicked at src\core\config\tests.rs:72:13:
assertion `left == right` failed
  left: "E:\\tmp"
 right: "C:\\tmp"
```

Fixes rust-lang#112544 (because now we handle Windows symlinks properly).

try-job: x86_64-msvc
try-job: i686-mingw
try-job: test-various
try-job: armhf-gnu
try-job: aarch64-apple
try-job: aarch64-gnu
bors added a commit to rust-lang-ci/rust that referenced this issue Aug 17, 2024
…nks, r=<try>

bootstrap: fix clean's remove_dir_all implementation

It turns out bootstrap's `clean.rs`'s hand-rolled `rm_rf` (which probably comes before `std::fs::remove_dir_all` was stable) is very broken on native Windows around both read-only files/directories and especially symbolic links. So instead of rolling our own, just use `std::fs::remove_dir_all`.

This is a blocker for compiletest's own `rm_rf` implementation rust-lang#129155 which happens to be also buggy, which in turn is a blocker for the rmake.rs test port rust-lang#128562 that heavily exercises symlinks (I was reviewing rust-lang#128562 and testing it on native Windows which is how I found out).

I also left a FIXME for `detect_src_and_out` due to a failing assertion on native Windows (opened rust-lang#129188):

```
---- core::config::tests::detect_src_and_out stdout ----
thread 'core::config::tests::detect_src_and_out' panicked at src\core\config\tests.rs:72:13:
assertion `left == right` failed
  left: "E:\\tmp"
 right: "C:\\tmp"
```

Fixes rust-lang#112544 (because now we handle Windows symlinks properly).

try-job: x86_64-msvc
try-job: i686-mingw
try-job: test-various
try-job: armhf-gnu
try-job: aarch64-apple
try-job: aarch64-gnu
jieyouxu added a commit to jieyouxu/rust that referenced this issue Aug 18, 2024
…links, r=Kobzol

bootstrap: fix clean's remove_dir_all implementation

It turns out bootstrap's `clean.rs`'s hand-rolled `rm_rf` (which probably comes before `std::fs::remove_dir_all` was stable) is very broken on native Windows around both read-only files/directories and especially symbolic links. So instead of rolling our own, just use `std::fs::remove_dir_all`.

This is a blocker for compiletest's own `rm_rf` implementation rust-lang#129155 which happens to be also buggy, which in turn is a blocker for the rmake.rs test port rust-lang#128562 that heavily exercises symlinks (I was reviewing rust-lang#128562 and testing it on native Windows which is how I found out).

I also left a FIXME for `detect_src_and_out` due to a failing assertion on native Windows (opened rust-lang#129188):

```
---- core::config::tests::detect_src_and_out stdout ----
thread 'core::config::tests::detect_src_and_out' panicked at src\core\config\tests.rs:72:13:
assertion `left == right` failed
  left: "E:\\tmp"
 right: "C:\\tmp"
```

Fixes rust-lang#112544 (because now we handle Windows symlinks properly).

try-job: x86_64-msvc
try-job: i686-mingw
try-job: test-various
try-job: armhf-gnu
try-job: aarch64-apple
try-job: aarch64-gnu
jieyouxu added a commit to jieyouxu/rust that referenced this issue Aug 18, 2024
…links, r=Kobzol

bootstrap: fix clean's remove_dir_all implementation

It turns out bootstrap's `clean.rs`'s hand-rolled `rm_rf` (which probably comes before `std::fs::remove_dir_all` was stable) is very broken on native Windows around both read-only files/directories and especially symbolic links. So instead of rolling our own, just use `std::fs::remove_dir_all`.

This is a blocker for compiletest's own `rm_rf` implementation rust-lang#129155 which happens to be also buggy, which in turn is a blocker for the rmake.rs test port rust-lang#128562 that heavily exercises symlinks (I was reviewing rust-lang#128562 and testing it on native Windows which is how I found out).

I also left a FIXME for `detect_src_and_out` due to a failing assertion on native Windows (opened rust-lang#129188):

```
---- core::config::tests::detect_src_and_out stdout ----
thread 'core::config::tests::detect_src_and_out' panicked at src\core\config\tests.rs:72:13:
assertion `left == right` failed
  left: "E:\\tmp"
 right: "C:\\tmp"
```

Fixes rust-lang#112544 (because now we handle Windows symlinks properly).

try-job: x86_64-msvc
try-job: i686-mingw
try-job: test-various
try-job: armhf-gnu
try-job: aarch64-apple
try-job: aarch64-gnu
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this issue Aug 18, 2024
…links, r=Kobzol

bootstrap: fix clean's remove_dir_all implementation

It turns out bootstrap's `clean.rs`'s hand-rolled `rm_rf` (which probably comes before `std::fs::remove_dir_all` was stable) is very broken on native Windows around both read-only files/directories and especially symbolic links. So instead of rolling our own, just use `std::fs::remove_dir_all`.

This is a blocker for compiletest's own `rm_rf` implementation rust-lang#129155 which happens to be also buggy, which in turn is a blocker for the rmake.rs test port rust-lang#128562 that heavily exercises symlinks (I was reviewing rust-lang#128562 and testing it on native Windows which is how I found out).

I also left a FIXME for `detect_src_and_out` due to a failing assertion on native Windows (opened rust-lang#129188):

```
---- core::config::tests::detect_src_and_out stdout ----
thread 'core::config::tests::detect_src_and_out' panicked at src\core\config\tests.rs:72:13:
assertion `left == right` failed
  left: "E:\\tmp"
 right: "C:\\tmp"
```

Fixes rust-lang#112544 (because now we handle Windows symlinks properly).

try-job: x86_64-msvc
try-job: i686-mingw
try-job: test-various
try-job: armhf-gnu
try-job: aarch64-apple
try-job: aarch64-gnu
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this issue Aug 18, 2024
…links, r=Kobzol

bootstrap: fix clean's remove_dir_all implementation

It turns out bootstrap's `clean.rs`'s hand-rolled `rm_rf` (which probably comes before `std::fs::remove_dir_all` was stable) is very broken on native Windows around both read-only files/directories and especially symbolic links. So instead of rolling our own, just use `std::fs::remove_dir_all`.

This is a blocker for compiletest's own `rm_rf` implementation rust-lang#129155 which happens to be also buggy, which in turn is a blocker for the rmake.rs test port rust-lang#128562 that heavily exercises symlinks (I was reviewing rust-lang#128562 and testing it on native Windows which is how I found out).

I also left a FIXME for `detect_src_and_out` due to a failing assertion on native Windows (opened rust-lang#129188):

```
---- core::config::tests::detect_src_and_out stdout ----
thread 'core::config::tests::detect_src_and_out' panicked at src\core\config\tests.rs:72:13:
assertion `left == right` failed
  left: "E:\\tmp"
 right: "C:\\tmp"
```

Fixes rust-lang#112544 (because now we handle Windows symlinks properly).

try-job: x86_64-msvc
try-job: i686-mingw
try-job: test-various
try-job: armhf-gnu
try-job: aarch64-apple
try-job: aarch64-gnu
jieyouxu added a commit to jieyouxu/rust that referenced this issue Aug 20, 2024
…links, r=Kobzol

bootstrap: fix clean's remove_dir_all implementation

It turns out bootstrap's `clean.rs`'s hand-rolled `rm_rf` (which probably comes before `std::fs::remove_dir_all` was stable) is very broken on native Windows around both read-only files/directories and especially symbolic links. So instead of rolling our own, just use `std::fs::remove_dir_all`.

This is a blocker for compiletest's own `rm_rf` implementation rust-lang#129155 which happens to be also buggy, which in turn is a blocker for the rmake.rs test port rust-lang#128562 that heavily exercises symlinks (I was reviewing rust-lang#128562 and testing it on native Windows which is how I found out).

I also left a FIXME for `detect_src_and_out` due to a failing assertion on native Windows (opened rust-lang#129188):

```
---- core::config::tests::detect_src_and_out stdout ----
thread 'core::config::tests::detect_src_and_out' panicked at src\core\config\tests.rs:72:13:
assertion `left == right` failed
  left: "E:\\tmp"
 right: "C:\\tmp"
```

Fixes rust-lang#112544 (because now we handle Windows symlinks properly).

try-job: x86_64-msvc
try-job: i686-mingw
try-job: test-various
try-job: armhf-gnu
try-job: aarch64-apple
try-job: aarch64-gnu
jieyouxu added a commit to jieyouxu/rust that referenced this issue Aug 20, 2024
…links, r=Kobzol

bootstrap: fix clean's remove_dir_all implementation

It turns out bootstrap's `clean.rs`'s hand-rolled `rm_rf` (which probably comes before `std::fs::remove_dir_all` was stable) is very broken on native Windows around both read-only files/directories and especially symbolic links. So instead of rolling our own, just use `std::fs::remove_dir_all`.

This is a blocker for compiletest's own `rm_rf` implementation rust-lang#129155 which happens to be also buggy, which in turn is a blocker for the rmake.rs test port rust-lang#128562 that heavily exercises symlinks (I was reviewing rust-lang#128562 and testing it on native Windows which is how I found out).

I also left a FIXME for `detect_src_and_out` due to a failing assertion on native Windows (opened rust-lang#129188):

```
---- core::config::tests::detect_src_and_out stdout ----
thread 'core::config::tests::detect_src_and_out' panicked at src\core\config\tests.rs:72:13:
assertion `left == right` failed
  left: "E:\\tmp"
 right: "C:\\tmp"
```

Fixes rust-lang#112544 (because now we handle Windows symlinks properly).

try-job: x86_64-msvc
try-job: i686-mingw
try-job: test-various
try-job: armhf-gnu
try-job: aarch64-apple
try-job: aarch64-gnu
tgross35 added a commit to tgross35/rust that referenced this issue Aug 21, 2024
…links, r=Kobzol

bootstrap: fix clean's remove_dir_all implementation

It turns out bootstrap's `clean.rs`'s hand-rolled `rm_rf` (which probably comes before `std::fs::remove_dir_all` was stable) is very broken on native Windows around both read-only files/directories and especially symbolic links. So instead of rolling our own, just use `std::fs::remove_dir_all`.

This is a blocker for compiletest's own `rm_rf` implementation rust-lang#129155 which happens to be also buggy, which in turn is a blocker for the rmake.rs test port rust-lang#128562 that heavily exercises symlinks (I was reviewing rust-lang#128562 and testing it on native Windows which is how I found out).

I also left a FIXME for `detect_src_and_out` due to a failing assertion on native Windows (opened rust-lang#129188):

```
---- core::config::tests::detect_src_and_out stdout ----
thread 'core::config::tests::detect_src_and_out' panicked at src\core\config\tests.rs:72:13:
assertion `left == right` failed
  left: "E:\\tmp"
 right: "C:\\tmp"
```

Fixes rust-lang#112544 (because now we handle Windows symlinks properly).

try-job: x86_64-msvc
try-job: i686-mingw
try-job: test-various
try-job: armhf-gnu
try-job: aarch64-apple
try-job: aarch64-gnu
tgross35 added a commit to tgross35/rust that referenced this issue Aug 21, 2024
…links, r=Kobzol

bootstrap: fix clean's remove_dir_all implementation

It turns out bootstrap's `clean.rs`'s hand-rolled `rm_rf` (which probably comes before `std::fs::remove_dir_all` was stable) is very broken on native Windows around both read-only files/directories and especially symbolic links. So instead of rolling our own, just use `std::fs::remove_dir_all`.

This is a blocker for compiletest's own `rm_rf` implementation rust-lang#129155 which happens to be also buggy, which in turn is a blocker for the rmake.rs test port rust-lang#128562 that heavily exercises symlinks (I was reviewing rust-lang#128562 and testing it on native Windows which is how I found out).

I also left a FIXME for `detect_src_and_out` due to a failing assertion on native Windows (opened rust-lang#129188):

```
---- core::config::tests::detect_src_and_out stdout ----
thread 'core::config::tests::detect_src_and_out' panicked at src\core\config\tests.rs:72:13:
assertion `left == right` failed
  left: "E:\\tmp"
 right: "C:\\tmp"
```

Fixes rust-lang#112544 (because now we handle Windows symlinks properly).

try-job: x86_64-msvc
try-job: i686-mingw
try-job: test-various
try-job: armhf-gnu
try-job: aarch64-apple
try-job: aarch64-gnu
jieyouxu added a commit to jieyouxu/rust that referenced this issue Aug 22, 2024
…links, r=Kobzol

bootstrap: fix clean's remove_dir_all implementation

It turns out bootstrap's `clean.rs`'s hand-rolled `rm_rf` (which probably comes before `std::fs::remove_dir_all` was stable) is very broken on native Windows around both read-only files/directories and especially symbolic links. So instead of rolling our own, just use `std::fs::remove_dir_all`.

This is a blocker for compiletest's own `rm_rf` implementation rust-lang#129155 which happens to be also buggy, which in turn is a blocker for the rmake.rs test port rust-lang#128562 that heavily exercises symlinks (I was reviewing rust-lang#128562 and testing it on native Windows which is how I found out).

I also left a FIXME for `detect_src_and_out` due to a failing assertion on native Windows (opened rust-lang#129188):

```
---- core::config::tests::detect_src_and_out stdout ----
thread 'core::config::tests::detect_src_and_out' panicked at src\core\config\tests.rs:72:13:
assertion `left == right` failed
  left: "E:\\tmp"
 right: "C:\\tmp"
```

Fixes rust-lang#112544 (because now we handle Windows symlinks properly).

try-job: x86_64-msvc
try-job: i686-mingw
try-job: test-various
try-job: armhf-gnu
try-job: aarch64-apple
try-job: aarch64-gnu
@bors bors closed this as completed in 9fd8a2c Aug 22, 2024
rust-timer added a commit to rust-lang-ci/rust that referenced this issue Aug 22, 2024
Rollup merge of rust-lang#129187 - jieyouxu:squeaky-clean-windows-symlinks, r=Kobzol

bootstrap: fix clean's remove_dir_all implementation

It turns out bootstrap's `clean.rs`'s hand-rolled `rm_rf` (which probably comes before `std::fs::remove_dir_all` was stable) is very broken on native Windows around both read-only files/directories and especially symbolic links. So instead of rolling our own, just use `std::fs::remove_dir_all`.

This is a blocker for compiletest's own `rm_rf` implementation rust-lang#129155 which happens to be also buggy, which in turn is a blocker for the rmake.rs test port rust-lang#128562 that heavily exercises symlinks (I was reviewing rust-lang#128562 and testing it on native Windows which is how I found out).

I also left a FIXME for `detect_src_and_out` due to a failing assertion on native Windows (opened rust-lang#129188):

```
---- core::config::tests::detect_src_and_out stdout ----
thread 'core::config::tests::detect_src_and_out' panicked at src\core\config\tests.rs:72:13:
assertion `left == right` failed
  left: "E:\\tmp"
 right: "C:\\tmp"
```

Fixes rust-lang#112544 (because now we handle Windows symlinks properly).

try-job: x86_64-msvc
try-job: i686-mingw
try-job: test-various
try-job: armhf-gnu
try-job: aarch64-apple
try-job: aarch64-gnu
@jieyouxu
Copy link
Member

Re-opening due to revert #129413.

@jieyouxu jieyouxu reopened this Aug 22, 2024
@jieyouxu jieyouxu self-assigned this Aug 22, 2024
@jieyouxu jieyouxu removed their assignment Oct 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-bug Category: This is a bug. O-windows Operating system: Windows T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants