Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rollup of 5 pull requests #114786

Merged
merged 13 commits into from
Aug 13, 2023
Merged

Rollup of 5 pull requests #114786

merged 13 commits into from
Aug 13, 2023

Conversation

GuillaumeGomez
Copy link
Member

Successful merges:

r? @ghost
@rustbot modify labels: rollup

Create a similar rollup

cuviper and others added 13 commits July 25, 2023 13:11
This changes the bootstrap config `target.*.profiler` from a plain bool
to also allow a string, which will be used as a path to the pre-built
profiling runtime for that target. Then `profiler_builtins/build.rs`
reads that in a `LLVM_PROFILER_RT_LIB` environment variable.
This is inherited from the old PR.

This method returns an iterator over mapped windows of the starting
iterator. Adding the more straight-forward `Iterator::windows` is not
easily possible right now as the items are stored in the iterator type,
meaning the `next` call would return references to `self`. This is not
allowed by the current `Iterator` trait design. This might change once
GATs have landed.

The idea has been brought up by @m-ou-se here:
https://rust-lang.zulipchat.com/#narrow/stream/219381-t-libs/topic/Iterator.3A.3A.7Bpairwise.2C.20windows.7D/near/224587771

Co-authored-by: Lukas Kalbertodt <[email protected]>
This avoids confusion with data structures that actually hold BCB counter
information.
This avoids the need to pass around a separate vector to accumulate into, and
avoids the need to create a fake empty vector when failure occurs.
Storing coverage counter information in `CoverageCounters` has a few advantages
over storing it directly inside BCB graph nodes:

- The graph doesn't need to be mutable when making the counters, making it
easier to see that the graph itself is not modified during this step.

- All of the counter data is clearly visible in one place.

- It becomes possible to use a representation that doesn't correspond 1:1 to
graph nodes, e.g. storing all the edge counters in a single hashmap instead of
several.
…, r=Mark-Simulacrum

Add `Iterator::map_windows`

Tracking issue:  rust-lang#87155.

This is inherited from the old PR  rust-lang#82413.

Unlike rust-lang#82413, this PR implements the `MapWindows` to be lazy: only when pulling from the outer iterator, `.next()` of the inner iterator will be called.

## Implementaion Steps
- [x] Implement `MapWindows` to keep the iterators' [*Laziness*](https://doc.rust-lang.org/std/iter/index.html#laziness) contract.
- [x] Fix the known bug of memory access error.
- [ ] Full specialization of iterator-related traits for `MapWindows`.
    - [x] `Iterator::size_hint`,
    - [x] ~`Iterator::count`~,
    - [x] `ExactSizeIterator` (when `I: ExactSizeIterator`),
    - [x] ~`TrustedLen` (when `I: TrustedLen`)~,
    - [x] `FusedIterator`,
    - [x] ~`Iterator::advance_by`~,
    - [x] ~`Iterator::nth`~,
    - [ ] ...
- [ ] More tests and docs.

## Unresolved Questions:
- [ ] Is there any more iterator-related traits should be specialized?
- [ ] Is the double-space buffer worth?
- [ ] Should there be `rmap_windows` or something else?
- [ ] Taking GAT for consideration, should the mapper function be `FnMut(&[I::Item; N]) -> R` or something like `FnMut(ArrayView<'_, I::Item, N>) -> R`? Where `ArrayView` is mentioned in rust-lang/generic-associated-types-initiative#2.
    - It can save memory, only the same size as the array window is needed,
    - It is more efficient, which requires less data copies,
    - It is possibly compatible with the GATified version of `LendingIterator::windows`.
    - But it prevents the array pattern matching like `iter.map_windows(|_arr: [_; N]| ())`, unless we extend the array pattern to allow matching the `ArrayView`.
…lacrum

Allow using external builds of the compiler-rt profile lib

This changes the bootstrap config `target.*.profiler` from a plain bool
to also allow a string, which will be used as a path to the pre-built
profiling runtime for that target. Then `profiler_builtins/build.rs`
reads that in a `LLVM_PROFILER_RT_LIB` environment variable.
…h726

coverage: Store BCB counter info externally, not directly in the BCB graph

When deciding how to instrument the underlying MIR for coverage, the `InstrumentCoverage` pass builds a simplified “Basic Counter Block” graph, and then allocates coverage counters/expressions to various nodes/edges in the BCB graph as necessary. Those counters/expressions are then injected into the function's MIR.

The awkward thing here is that the code for doing this needs `&mut` access to the graph, in order to associate coverage info with individual nodes, even though it isn't making any structural changes to the graph itself. That makes it harder to understand and modify the instrumentation code.

In addition, the graph alone can't hold all the information that is needed. There ends up being an extra vector of “intermediate expressions” that needs to be passed around separately anyway.

---

This PR simplifies things by instead storing all of that temporary coverage information in a number of side-tables inside `CoverageCounters`.

This makes it easier to see all of the information produced by the make-counters step, and how it is used by the inject-into-mir step.

---

Looking at the combined changes is possible, but I recommend reviewing the commits individually, because the big changes are mostly independent of each other (despite being conceptually related).
…acrum

CI: use smaller machines in PR runs

mingw-check job-linux-16c -> job-linux-4c
~job-linux-4c 20 min in auto job
~job-linux-16c 13 min in pr job
with current pr regressed to almost 21 min, it's ok.

mingw-check-tidy job-linux-16c -> job-linux-4c small enough, so reduce to minimal
~ job-linux-16c 3 min
with current pr regressed to almost 5 min, it's ok.

x86_64-gnu-tools job-linux-16c this is top job by time in PR, so don't touch it
~ job-linux-8c 1.30 hour in auto job
~ job-linux-16c 1 hour in pr job (affected by rust-lang#114613, actual time ~ 30 min)

x86_64-gnu-llvm-15 job-linux-16c don't change too
~ job-linux-8c 1.30 hour in auto job
~ job-linux-16c 30 min in pr job

Noticed while working on rust-lang#114621, so current time affected by always rebuilded docker images (but pr images always rebuilded before too, so nvm)
…lor-32, r=notriddle

Migrate GUI colors test to original CSS color format

Follow-up of rust-lang#111459.

r? `@notriddle`
@rustbot rustbot added A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue. T-libs Relevant to the library team, which will review and decide on the PR/issue. T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue. rollup A PR which is a rollup labels Aug 13, 2023
@GuillaumeGomez
Copy link
Member Author

@bors r+ rollup=never p=5

@bors
Copy link
Contributor

bors commented Aug 13, 2023

📌 Commit be6cda1 has been approved by GuillaumeGomez

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 13, 2023
@bors
Copy link
Contributor

bors commented Aug 13, 2023

⌛ Testing commit be6cda1 with merge 1b198b3...

@bors
Copy link
Contributor

bors commented Aug 13, 2023

☀️ Test successful - checks-actions
Approved by: GuillaumeGomez
Pushing 1b198b3 to master...

@bors bors added the merged-by-bors This PR was explicitly merged by bors. label Aug 13, 2023
@bors bors merged commit 1b198b3 into rust-lang:master Aug 13, 2023
@rustbot rustbot added this to the 1.73.0 milestone Aug 13, 2023
@rust-timer
Copy link
Collaborator

📌 Perf builds for each rolled up PR:

PR# Message Perf Build Sha
#94667 Add Iterator::map_windows cb49ccd4fb09ed1fb3532ea1d58a2f188a1b4086 (link)
#114069 Allow using external builds of the compiler-rt profile lib a7a3dabcfe7ef9c64a8f5c559f9be308322c94b0 (link)
#114354 coverage: Store BCB counter info externally, not directly i… 43847c588fb856a46684040ae95c1a5df048376f (link)
#114625 CI: use smaller machines in PR runs 345a470814af385d831e8ba24685c17de49dfb41 (link)
#114777 Migrate GUI colors test to original CSS color format 5958697b52a0512dc220e330c07351846c3ea6f5 (link)

previous master: ebbd7154a7

In the case of a perf regression, run the following command for each PR you suspect might be the cause: @rust-timer build $SHA

@rust-timer
Copy link
Collaborator

Finished benchmarking commit (1b198b3): comparison URL.

Overall result: ❌✅ regressions and improvements - no action needed

@rustbot label: -perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
0.2% [0.2%, 0.2%] 1
Regressions ❌
(secondary)
- - 0
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
-1.5% [-1.5%, -1.5%] 1
All ❌✅ (primary) 0.2% [0.2%, 0.2%] 1

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
2.7% [2.6%, 2.9%] 3
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
-2.4% [-3.2%, -1.8%] 4
All ❌✅ (primary) - - 0

Cycles

This benchmark run did not return any relevant results for this metric.

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 631.811s -> 633.873s (0.33%)

@GuillaumeGomez GuillaumeGomez deleted the rollup-0cos5gn branch August 14, 2023 08:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-testsuite Area: The testsuite used to check the correctness of rustc merged-by-bors This PR was explicitly merged by bors. rollup A PR which is a rollup S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue. T-libs Relevant to the library team, which will review and decide on the PR/issue. T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants