Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add metrics to Block Streamer #579

Merged
merged 9 commits into from
Feb 23, 2024
Merged

Conversation

morgsmccauley
Copy link
Collaborator

@morgsmccauley morgsmccauley commented Feb 23, 2024

This PR adds the following metrics to Block Streamer:

  • LAST_PROCESSED_BLOCK - height of last block seen, will be used to calculate the lag from tip of network
  • PROCESSED_BLOCKS_COUNT - count of blocks seen, used to calculate BPS
  • PUBLISHED_BLOCKS_COUNT - count of blocks published to Redis Stream, to determine whether the block stream is publishing messages
  • LOGS_COUNT - count logs by level, so we can alert on number of error logs etc.

I've also slightly refactored RedisClient to support this change.

@morgsmccauley morgsmccauley requested a review from a team as a code owner February 23, 2024 01:36
@morgsmccauley morgsmccauley force-pushed the feat/block-streamer-metrics branch from dd4ba32 to 10561b9 Compare February 23, 2024 01:36
@morgsmccauley morgsmccauley changed the title feat/block streamer metrics feat: Add metrics to Block Streamer Feb 23, 2024
.with(tracing_subscriber::EnvFilter::from_default_env())
.init();

let redis_url = std::env::var("REDIS_URL").expect("REDIS_URL is not set");
let server_port = std::env::var("SERVER_PORT").expect("SERVER_PORT is not set");
let grpc_port = std::env::var("GRPC_PORT").expect("GRPC_PORT is not set");
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will also make this change in Terraform

@morgsmccauley morgsmccauley force-pushed the feat/block-streamer-metrics branch from 10561b9 to 5609feb Compare February 23, 2024 01:42

pub fn last_processed_block_key(&self) -> String {
// TODO: rename to `last_processed_block`
format!("{}:last_published_block", self.get_full_name())
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"published" should mean it actually got added to the Redis Stream

Copy link
Collaborator

@darunrs darunrs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Looking forward to seeing what the metrics show us!

@morgsmccauley morgsmccauley merged commit 22b582e into main Feb 23, 2024
4 checks passed
@morgsmccauley morgsmccauley deleted the feat/block-streamer-metrics branch February 23, 2024 02:18
@morgsmccauley morgsmccauley self-assigned this Feb 23, 2024
@darunrs darunrs mentioned this pull request Feb 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants