Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: export metrics about the blocking thread pool #5161

Merged
merged 3 commits into from
Nov 4, 2022

Conversation

duarten
Copy link
Contributor

@duarten duarten commented Nov 3, 2022

Motivation

Add metrics about the blocking thread pool.

Solution

Publish the metrics as thread-safe values, written under the blocking thread
pool's lock and able to be read in a lock-free fashion by any reader.

Fixes #5156

@github-actions github-actions bot added the R-loom Run loom tests on this PR label Nov 3, 2022
@duarten duarten force-pushed the duarte/blocking-metrics branch from d30c5b4 to 659b4c9 Compare November 3, 2022 06:00
@@ -165,7 +202,6 @@ impl BlockingPool {
inner: Arc::new(Inner {
shared: Mutex::new(Shared {
queue: VecDeque::new(),
num_th: 0,
num_idle: 0,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it's also worth it to export this one?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see why not.

@duarten duarten changed the title Export metrics about the blocking thread pool runtime: export metrics about the blocking thread pool Nov 3, 2022
@Darksonn Darksonn added A-tokio Area: The main tokio crate M-metrics Module: tokio/runtime/metrics labels Nov 3, 2022
Copy link
Contributor

@Noah-Kennedy Noah-Kennedy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good. Feel free to add the idle count if you'd like to this PR. I'm going to wait for a second review before we merge this.

@duarten
Copy link
Contributor Author

duarten commented Nov 3, 2022

Added the idle count here: d2ba276

@duarten duarten force-pushed the duarte/blocking-metrics branch from 7291322 to d2ba276 Compare November 3, 2022 18:13
@duarten
Copy link
Contributor Author

duarten commented Nov 3, 2022

Not sure why join_size is failing on CI :/ Probably related to rust 1.65.0?

@Noah-Kennedy
Copy link
Contributor

@duarten it is, see #5164.

@Noah-Kennedy
Copy link
Contributor

@duarten the other PR just merged, so I've taken the liberty of updating this branch with the latest from master.

@duarten duarten force-pushed the duarte/blocking-metrics branch from 4dfe328 to 7de50a6 Compare November 4, 2022 01:21
@Noah-Kennedy
Copy link
Contributor

Waiting for a second approval to merge.

Copy link
Contributor

@Darksonn Darksonn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems good to me.

@carllerche carllerche merged commit 23fdd32 into tokio-rs:master Nov 4, 2022
@carllerche
Copy link
Member

Awesome job everyone.

@duarten duarten deleted the duarte/blocking-metrics branch November 4, 2022 18:12
crapStone pushed a commit to Calciumdibromid/CaBr2 that referenced this pull request Nov 22, 2022
This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [tokio](https://tokio.rs) ([source](https://github.com/tokio-rs/tokio)) | dependencies | minor | `1.21.2` -> `1.22.0` |
| [tokio](https://tokio.rs) ([source](https://github.com/tokio-rs/tokio)) | dev-dependencies | minor | `1.21.2` -> `1.22.0` |

---

### Release Notes

<details>
<summary>tokio-rs/tokio</summary>

### [`v1.22.0`](https://github.com/tokio-rs/tokio/releases/tag/tokio-1.22.0): Tokio v1.22.0

[Compare Source](tokio-rs/tokio@tokio-1.21.2...tokio-1.22.0)

##### Added

-   runtime: add `Handle::runtime_flavor` ([#&#8203;5138])
-   sync: add `Mutex::blocking_lock_owned` ([#&#8203;5130])
-   sync: add `Semaphore::MAX_PERMITS` ([#&#8203;5144])
-   sync: add `merge()` to semaphore permits ([#&#8203;4948])
-   sync: add `mpsc::WeakUnboundedSender` ([#&#8203;5189])

##### Added (unstable)

-   process: add `Command::process_group` ([#&#8203;5114])
-   runtime: export metrics about the blocking thread pool ([#&#8203;5161])
-   task: add `task::id()` and `task::try_id()` ([#&#8203;5171])

##### Fixed

-   macros: don't take ownership of futures in macros ([#&#8203;5087])
-   runtime: fix Stacked Borrows violation in `LocalOwnedTasks` ([#&#8203;5099])
-   runtime: mitigate ABA with 32-bit queue indices when possible ([#&#8203;5042])
-   task: wake local tasks to the local queue when woken by the same thread ([#&#8203;5095])
-   time: panic in release mode when `mark_pending` called illegally ([#&#8203;5093])
-   runtime: fix typo in expect message ([#&#8203;5169])
-   runtime: fix `unsync_load` on atomic types ([#&#8203;5175])
-   task: elaborate safety comments in task deallocation ([#&#8203;5172])
-   runtime: fix `LocalSet` drop in thread local ([#&#8203;5179])
-   net: remove libc type leakage in a public API ([#&#8203;5191])
-   runtime: update the alignment of `CachePadded` ([#&#8203;5106])

##### Changed

-   io: make `tokio::io::copy` continue filling the buffer when writer stalls ([#&#8203;5066])
-   runtime: remove `coop::budget` from `LocalSet::run_until` ([#&#8203;5155])
-   sync: make `Notify` panic safe ([#&#8203;5154])

##### Documented

-   io: fix doc for `write_i8` to use signed integers ([#&#8203;5040])
-   net: fix doc typos for TCP and UDP `set_tos` methods ([#&#8203;5073])
-   net: fix function name in `UdpSocket::recv` documentation ([#&#8203;5150])
-   sync: typo in `TryLockError` for `RwLock::try_write` ([#&#8203;5160])
-   task: document that spawned tasks execute immediately ([#&#8203;5117])
-   time: document return type of `timeout` ([#&#8203;5118])
-   time: document that `timeout` checks only before poll ([#&#8203;5126])
-   sync: specify return type of `oneshot::Receiver` in docs ([#&#8203;5198])

##### Internal changes

-   runtime: use const `Mutex::new` for globals ([#&#8203;5061])
-   runtime: remove `Option` around `mio::Events` in io driver ([#&#8203;5078])
-   runtime: remove a conditional compilation clause ([#&#8203;5104])
-   runtime: remove a reference to internal time handle ([#&#8203;5107])
-   runtime: misc time driver cleanup ([#&#8203;5120])
-   runtime: move signal driver to runtime module ([#&#8203;5121])
-   runtime: signal driver now uses I/O driver directly ([#&#8203;5125])
-   runtime: start decoupling I/O driver and I/O handle ([#&#8203;5127])
-   runtime: switch `io::handle` refs with scheduler:Handle ([#&#8203;5128])
-   runtime: remove Arc from I/O driver ([#&#8203;5134])
-   runtime: use signal driver handle via `scheduler::Handle` ([#&#8203;5135])
-   runtime: move internal clock fns out of context ([#&#8203;5139])
-   runtime: remove `runtime::context` module ([#&#8203;5140])
-   runtime: keep driver cfgs in `driver.rs` ([#&#8203;5141])
-   runtime: add `runtime::context` to unify thread-locals ([#&#8203;5143])
-   runtime: rename some confusing internal variables/fns ([#&#8203;5151])
-   runtime: move `coop` mod into `runtime` ([#&#8203;5152])
-   runtime: move budget state to context thread-local ([#&#8203;5157])
-   runtime: move park logic into runtime module ([#&#8203;5158])
-   runtime: move `Runtime` into its own file ([#&#8203;5159])
-   runtime: unify entering a runtime with `Handle::enter` ([#&#8203;5163])
-   runtime: remove handle reference from each scheduler ([#&#8203;5166])
-   runtime: move `enter` into `context` ([#&#8203;5167])
-   runtime: combine context and entered thread-locals ([#&#8203;5168])
-   runtime: fix accidental unsetting of current handle ([#&#8203;5178])
-   runtime: move `CoreStage` methods to `Core` ([#&#8203;5182])
-   sync: name mpsc semaphore types ([#&#8203;5146])

[#&#8203;4948]: tokio-rs/tokio#4948

[#&#8203;5040]: tokio-rs/tokio#5040

[#&#8203;5042]: tokio-rs/tokio#5042

[#&#8203;5061]: tokio-rs/tokio#5061

[#&#8203;5066]: tokio-rs/tokio#5066

[#&#8203;5073]: tokio-rs/tokio#5073

[#&#8203;5078]: tokio-rs/tokio#5078

[#&#8203;5087]: tokio-rs/tokio#5087

[#&#8203;5093]: tokio-rs/tokio#5093

[#&#8203;5095]: tokio-rs/tokio#5095

[#&#8203;5099]: tokio-rs/tokio#5099

[#&#8203;5104]: tokio-rs/tokio#5104

[#&#8203;5106]: tokio-rs/tokio#5106

[#&#8203;5107]: tokio-rs/tokio#5107

[#&#8203;5114]: tokio-rs/tokio#5114

[#&#8203;5117]: tokio-rs/tokio#5117

[#&#8203;5118]: tokio-rs/tokio#5118

[#&#8203;5120]: tokio-rs/tokio#5120

[#&#8203;5121]: tokio-rs/tokio#5121

[#&#8203;5125]: tokio-rs/tokio#5125

[#&#8203;5126]: tokio-rs/tokio#5126

[#&#8203;5127]: tokio-rs/tokio#5127

[#&#8203;5128]: tokio-rs/tokio#5128

[#&#8203;5130]: tokio-rs/tokio#5130

[#&#8203;5134]: tokio-rs/tokio#5134

[#&#8203;5135]: tokio-rs/tokio#5135

[#&#8203;5138]: tokio-rs/tokio#5138

[#&#8203;5138]: tokio-rs/tokio#5138

[#&#8203;5139]: tokio-rs/tokio#5139

[#&#8203;5140]: tokio-rs/tokio#5140

[#&#8203;5141]: tokio-rs/tokio#5141

[#&#8203;5143]: tokio-rs/tokio#5143

[#&#8203;5144]: tokio-rs/tokio#5144

[#&#8203;5144]: tokio-rs/tokio#5144

[#&#8203;5146]: tokio-rs/tokio#5146

[#&#8203;5150]: tokio-rs/tokio#5150

[#&#8203;5151]: tokio-rs/tokio#5151

[#&#8203;5152]: tokio-rs/tokio#5152

[#&#8203;5154]: tokio-rs/tokio#5154

[#&#8203;5155]: tokio-rs/tokio#5155

[#&#8203;5157]: tokio-rs/tokio#5157

[#&#8203;5158]: tokio-rs/tokio#5158

[#&#8203;5159]: tokio-rs/tokio#5159

[#&#8203;5160]: tokio-rs/tokio#5160

[#&#8203;5161]: tokio-rs/tokio#5161

[#&#8203;5163]: tokio-rs/tokio#5163

[#&#8203;5166]: tokio-rs/tokio#5166

[#&#8203;5167]: tokio-rs/tokio#5167

[#&#8203;5168]: tokio-rs/tokio#5168

[#&#8203;5169]: tokio-rs/tokio#5169

[#&#8203;5171]: tokio-rs/tokio#5171

[#&#8203;5172]: tokio-rs/tokio#5172

[#&#8203;5175]: tokio-rs/tokio#5175

[#&#8203;5178]: tokio-rs/tokio#5178

[#&#8203;5179]: tokio-rs/tokio#5179

[#&#8203;5182]: tokio-rs/tokio#5182

[#&#8203;5189]: tokio-rs/tokio#5189

[#&#8203;5191]: tokio-rs/tokio#5191

[#&#8203;5198]: tokio-rs/tokio#5198

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about these updates again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNC4yNy4xIiwidXBkYXRlZEluVmVyIjoiMzQuMjkuMiJ9-->

Co-authored-by: cabr2-bot <[email protected]>
Reviewed-on: https://codeberg.org/Calciumdibromid/CaBr2/pulls/1651
Reviewed-by: crapStone <[email protected]>
Co-authored-by: Calciumdibromid Bot <[email protected]>
Co-committed-by: Calciumdibromid Bot <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-tokio Area: The main tokio crate M-metrics Module: tokio/runtime/metrics R-loom Run loom tests on this PR
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Export metrics about the blocking thread pool
4 participants