Skip to content
This repository has been archived by the owner on Jan 3, 2023. It is now read-only.

Features/add initial subnet metrics #233

Open
wants to merge 39 commits into
base: eudico
Choose a base branch
from

Conversation

cryptoAtwill
Copy link
Contributor

@cryptoAtwill cryptoAtwill commented Jul 28, 2022

Proposed Changes

Add initial metrics to track:

  • Number of created subnets
  • Number of killed subnets
  • Number of active subnets
  • Grafana dashboard
  • Documentation and scripts to setup

Sample dashboard:
image

Checklist

Before you mark the PR ready for review, please make sure that:

  • All commits have a clear commit message.
  • The PR title is in the form of of <PR type>: <area>: <change being made>
    • example: fix: mempool: Introduce a cache for valid signatures
    • PR type: fix, feat, INTERFACE BREAKING CHANGE, CONSENSUS BREAKING, build, chore, ci, docs,perf, refactor, revert, style, test
    • area: api, chain, state, vm, data transfer, market, mempool, message, block production, multisig, networking, paychan, proving, sealing, wallet, deps
  • This PR has tests for new functionality or change in behaviour
  • If new user-facing features are introduced, clear usage guidelines and / or documentation updates should be included in https://lotus.filecoin.io or Discussion Tutorials.
  • CI is green

@cryptoAtwill cryptoAtwill changed the title Features/add initial subnet metrics (WIP) Features/add initial subnet metrics Jul 28, 2022
cmd/eudico/daemon.go Outdated Show resolved Hide resolved
cmd/eudico/daemon.go Outdated Show resolved Hide resolved
metrics/metrics.go Outdated Show resolved Hide resolved
@adlrocha
Copy link
Collaborator

We are making good progress. Thanks! I would try the event listener approach as part of this PR now that you know how to collect metrics. I think it can give all the skeleton that we need to collect "real-time" metrics.

@cryptoAtwill cryptoAtwill changed the title (WIP) Features/add initial subnet metrics Features/add initial subnet metrics Aug 1, 2022
cmd/eudico-stats/main.go Outdated Show resolved Hide resolved
Copy link
Collaborator

@adlrocha adlrocha left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So far this looks good. What about adding an additional metric to test if the 1 second approach works fine for other "less permanent" metrics? Thanks!

Using docker-compose
In the root directory
```shell
docker build -t eudico:latest -f Dockerfile.eudico .
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!!! Have you tried to spawn a network with more than one node with Docker compose?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will give it a try

}

func (s *SubnetNode) Equals(o *SubnetNode) bool {
return s.SubnetID == o.SubnetID &&
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not doing s==o directly?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this does a pointer compare? not the deep compare using pointers?

// The nextEpoch to resync stats with the chain
nextEpoch abi.ChainEpoch
// The list of subnets currently managed by the current subnet
subnets map[address.SubnetID]bool
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't get this variable. Aren´t you handling already the list of subnets being managed in EudicoStatsListener?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one tracks the children in memory.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the list of subnets are for the nodes. This one is to maintain the list of child subnets. Maybe I should give it a differernt name.

ctx context.Context,
id address.SubnetID,
) error {
// TODO: ideally there is no need to add read write lock. Keep in view.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You may need a lock here, what if the listener accesses the map because you start syncing with a new subnet while you are reading from it to collect some metrics? I'd suggest running with race to see if this is the case.

cmd/eudico-stats/main.go Outdated Show resolved Hide resolved
cmd/eudico-stats/stats.go Outdated Show resolved Hide resolved
// FIXME: Consider moving this to an independent optional process
// (so that is not dependent on the main process).
if cctx.Bool("export-metrics") {
go func() {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this still needed? eudico-statsnow runs in its own process, right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There might be metrics that cannot be collected by eudico-stats, such as response time and stuff like that?

@@ -0,0 +1,81 @@
#!/usr/bin/env bash
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about renaming this script eudico-stats or eudico-stats-example?

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants