Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

Add group name in task metrics #10196

Merged
merged 18 commits into from
Nov 11, 2021
Merged

Conversation

sandreim
Copy link
Contributor

@sandreim sandreim commented Nov 5, 2021

The subsystem name metric label improves observability of task CPU usage in Grafana - paritytech/polkadot#4198

Added two new spawn methods to SpawnNamed trait. These allow the caller to also specify a subsystem name when starting a task. While the change doesn't break any existing API, it changes the way metrics are logged by adding an extra label. All code using the original spawn methods will default to having subsystem set to substrate-unspecified.

Added optional group parameter to SpawnNamed::spawn() and SpawnNamed::spawn_blocking() methods.

Polkadot companion: paritytech/polkadot#4239

skip check-dependent-cumulus

Signed-off-by: Andrei Sandu <[email protected]>
Signed-off-by: Andrei Sandu <[email protected]>
@sandreim sandreim added A0-please_review Pull request needs code review. B5-clientnoteworthy C1-low PR touches the given topic and has a low impact on builders. labels Nov 5, 2021
@ordian ordian added the D3-trivial 🧸 PR contains trivial changes in a runtime directory that do not require an audit label Nov 5, 2021
Copy link
Member

@bkchr bkchr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think of naming it group instead of subsystem? Because in the end it is something to group different tasks?

primitives/core/src/traits.rs Outdated Show resolved Hide resolved
primitives/core/src/traits.rs Outdated Show resolved Hide resolved
client/service/src/task_manager/mod.rs Outdated Show resolved Hide resolved
@sandreim
Copy link
Contributor Author

sandreim commented Nov 5, 2021

What do you think of naming it group instead of subsystem? Because in the end it is something to group different tasks?

Sounds better, indeed we just want to group by.

Signed-off-by: Andrei Sandu <[email protected]>
Signed-off-by: Andrei Sandu <[email protected]>
Signed-off-by: Andrei Sandu <[email protected]>
Signed-off-by: Andrei Sandu <[email protected]>
Signed-off-by: Andrei Sandu <[email protected]>
Signed-off-by: Andrei Sandu <[email protected]>
Signed-off-by: Andrei Sandu <[email protected]>
@sandreim sandreim changed the title Add subsystem name in task metrics Add group name in task metrics Nov 8, 2021
@sandreim
Copy link
Contributor Author

sandreim commented Nov 8, 2021

Synced offline with @bkchr and decided to drop the new trait methods and just add a the group to the existing spawn() methods.

Signed-off-by: Andrei Sandu <[email protected]>
Copy link
Member

@ordian ordian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as paritytech/polkadot#4239 (comment)

You might also need cargo +nightly fmt.

client/service/src/builder.rs Outdated Show resolved Hide resolved
@sandreim
Copy link
Contributor Author

sandreim commented Nov 8, 2021

Same as paritytech/polkadot#4239 (comment)

You might also need cargo +nightly fmt.

Yep, this seems odd, there is a version mismatch of some sort. I'll rerun.

Signed-off-by: Andrei Sandu <[email protected]>
bin/node-template/node/src/service.rs Outdated Show resolved Hide resolved
bin/node/cli/src/service.rs Outdated Show resolved Hide resolved
bin/node/cli/src/service.rs Outdated Show resolved Hide resolved
client/consensus/common/src/import_queue/basic_queue.rs Outdated Show resolved Hide resolved
client/offchain/src/lib.rs Outdated Show resolved Hide resolved
client/service/src/builder.rs Outdated Show resolved Hide resolved
client/service/src/builder.rs Outdated Show resolved Hide resolved
client/service/src/builder.rs Outdated Show resolved Hide resolved
client/transaction-pool/src/lib.rs Outdated Show resolved Hide resolved
primitives/core/src/traits.rs Outdated Show resolved Hide resolved
- use Option group name in SpawnNamed methods
- switch to kebab case
- implement default group name
- add group name to some tasks

Signed-off-by: Andrei Sandu <[email protected]>
@sandreim sandreim requested a review from bkchr November 11, 2021 07:25
pub fn spawn(
&self,
name: &'static str,
group: Option<&'static str>,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something like:

enum GroupName {
    Default,
    Specific(&'static str),
}

impl From<Option<&'static str>> for GroupName {
    fn from(name: Option<&'static str>) -> Self {
        match name { Some(name) => Self::Specific(name), None => Self::Default }
    }
}

impl From<&'static str> for GroupName {
    fn from(name: &'static str) -> Self {
        Self::Specific(name)
    }
}

would have been super cool :P

Because then you could write:

pub fn spawn(.., group: impl Into<GroupName>, ..)

And stuff like spawn(.., "name", ..) or spawn(.., None, ..) should work :P

This would maybe be the icing on the top, but no requirement.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, you are right, that's more flexible. I'll implement this as a followup PR, at the moment I'm tackling other things.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds like an easy issue for external contributors :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Picking the issue up @ordian

@ordian
Copy link
Member

ordian commented Nov 11, 2021

bot merge

@paritytech-processbot
Copy link

Error: Waiting on code owner review from tomusdrw.

@ordian
Copy link
Member

ordian commented Nov 11, 2021

bot merge

@paritytech-processbot paritytech-processbot bot merged commit f785491 into master Nov 11, 2021
@paritytech-processbot paritytech-processbot bot deleted the per_subsystem_task_metrics branch November 11, 2021 17:15
ordian added a commit that referenced this pull request Nov 12, 2021
* master: (27 commits)
  Bump rustversion from 1.0.4 to 1.0.5 (#10243)
  Kill the light client, CHTs and change tries. (#10080)
  tuple to struct event variants (#10206)
  Bump thiserror from 1.0.26 to 1.0.30 (#10240)
  Warn about usage of pallet collective set members call. (#10156)
  Bump git2 from 0.13.22 to 0.13.23 (#10238)
  Add group name in task metrics  (#10196)
  Bump proc-macro-crate from 1.0.0 to 1.1.0 (#10237)
  Bump parity-util-mem from 0.10.0 to 0.10.2 (#10236)
  Bump substrate-bip39 from 0.4.2 to 0.4.4 (#10213)
  Upgrade jsonrpsee to v0.4.1 (#10022)
  expose substrate-cli service (#10229)
  Intend to reactivate cargo-unleash check (#10167)
  CI: build docs with deps (#9884)
  use CountedMap in pallet-bags-list (#10179)
  Move all example pallets under `examples` folder. (#10215)
  Upgrade wasm builder (#10226)
  upgrade ss58-registry with additional networks. (#10224)
  move wiki -> docs (#10225)
  new remote-ext mode: (#10192)
  ...
grishasobol pushed a commit to gear-tech/substrate that referenced this pull request Mar 28, 2022
* SpawnNamed: add new trait methods

Signed-off-by: Andrei Sandu <[email protected]>

* Implement new methods

Signed-off-by: Andrei Sandu <[email protected]>

* cargo fmt

Signed-off-by: Andrei Sandu <[email protected]>

* SpawnNamed: add new trait methods

Signed-off-by: Andrei Sandu <[email protected]>

* Implement new methods

Signed-off-by: Andrei Sandu <[email protected]>

* cargo fmt

Signed-off-by: Andrei Sandu <[email protected]>

* New approach - spaw() group param

Signed-off-by: Andrei Sandu <[email protected]>

* Update traits: SpawnNamed and SpawnNamed

Signed-off-by: Andrei Sandu <[email protected]>

* Update TaskManager tests

Signed-off-by: Andrei Sandu <[email protected]>

* Update test TaskExecutor

Signed-off-by: Andrei Sandu <[email protected]>

* Fix typo

Signed-off-by: Andrei Sandu <[email protected]>

* grunt work: fix spawn() calls

Signed-off-by: Andrei Sandu <[email protected]>

* cargo fmt

Signed-off-by: Andrei Sandu <[email protected]>

* remove old code

Signed-off-by: Andrei Sandu <[email protected]>

* cargo fmt - the right version

Signed-off-by: Andrei Sandu <[email protected]>

* Implement review feedback

- use Option group name in SpawnNamed methods
- switch to kebab case
- implement default group name
- add group name to some tasks

Signed-off-by: Andrei Sandu <[email protected]>
AurevoirXavier added a commit to darwinia-network/darwinia-common that referenced this pull request Aug 4, 2022
ark0f pushed a commit to gear-tech/substrate that referenced this pull request Feb 27, 2023
* SpawnNamed: add new trait methods

Signed-off-by: Andrei Sandu <[email protected]>

* Implement new methods

Signed-off-by: Andrei Sandu <[email protected]>

* cargo fmt

Signed-off-by: Andrei Sandu <[email protected]>

* SpawnNamed: add new trait methods

Signed-off-by: Andrei Sandu <[email protected]>

* Implement new methods

Signed-off-by: Andrei Sandu <[email protected]>

* cargo fmt

Signed-off-by: Andrei Sandu <[email protected]>

* New approach - spaw() group param

Signed-off-by: Andrei Sandu <[email protected]>

* Update traits: SpawnNamed and SpawnNamed

Signed-off-by: Andrei Sandu <[email protected]>

* Update TaskManager tests

Signed-off-by: Andrei Sandu <[email protected]>

* Update test TaskExecutor

Signed-off-by: Andrei Sandu <[email protected]>

* Fix typo

Signed-off-by: Andrei Sandu <[email protected]>

* grunt work: fix spawn() calls

Signed-off-by: Andrei Sandu <[email protected]>

* cargo fmt

Signed-off-by: Andrei Sandu <[email protected]>

* remove old code

Signed-off-by: Andrei Sandu <[email protected]>

* cargo fmt - the right version

Signed-off-by: Andrei Sandu <[email protected]>

* Implement review feedback

- use Option group name in SpawnNamed methods
- switch to kebab case
- implement default group name
- add group name to some tasks

Signed-off-by: Andrei Sandu <[email protected]>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
A0-please_review Pull request needs code review. C1-low PR touches the given topic and has a low impact on builders. D3-trivial 🧸 PR contains trivial changes in a runtime directory that do not require an audit
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants