Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

shed: command to list duplicate messages in tipsets (steb) #5847

Merged
merged 4 commits into from
Apr 29, 2021

Conversation

Stebalien
Copy link
Member

@Stebalien Stebalien commented Mar 19, 2021

Modification of #5841.

This one:

  1. Let's you specify a start/stop height.
  2. Prints out ndjson to stdout, and status/errors to stderr.
  3. Prints one entry per set of duplicates.
  4. Allows filtering messages by method number.
  5. Allows configuring the number of threads.

Usage:

> cd cmd/lotus-shed
> go build
> ./lotus-shed audits duplicate-messages --start=148888 | tee output.ndjson

That will walk back to the mainnet epoch and write the results to stdout and output.ndjson.

TODO

  • Make it possible to start at some height.
  • Make it faster (parallel fetch and/or direct datastore access)?
  • Make the method number filter customizable.
  • Add better docs

@Stebalien
Copy link
Member Author

I sincerely apologize for the git commit messages, I needed to send the code to a different machine for testing, and got tired of writing nice git commit messages. I'll rebase/squash later but don't want to mess with history right now.

@Kubuxu
Copy link
Contributor

Kubuxu commented Mar 22, 2021

@Stebalien for this I use git over ssh with bare repo on the other side and one working repo.
So for example:

devbox1> git init --bare ~/lotus-bare
local> git remote add  devbox ssh://devbox1/~/lotus-bare; git push devbox master
devbox1> git clone ~/lotus-bare ~/lotus

Then if you want to push some branch you push to devbox1 and in ~/lotus dir in devbox you just pull and check it out.
The bare repo in the middle is there because you cannot push to checked out branches.

magik6k and others added 2 commits April 28, 2021 16:23
- Allow a start/end epoch.
- Print one line per duplicate set.
- Allow filtering duplicate messages by method number.
- Make the number of threads configurable.
@Stebalien Stebalien marked this pull request as ready for review April 29, 2021 00:04
@Stebalien Stebalien requested a review from arajasek as a code owner April 29, 2021 00:04
Copy link
Contributor

@arajasek arajasek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, maybe confirm it works with latest changes.

cmd/lotus-shed/balances.go Show resolved Hide resolved
cmd/lotus-shed/balances.go Outdated Show resolved Hide resolved
cmd/lotus-shed/balances.go Outdated Show resolved Hide resolved
@Stebalien
Copy link
Member Author

confirm it works with latest changes

Done. I've also added support for {exclude,include}-{from,to} (which also seems to work).

@Stebalien
Copy link
Member Author

(going to assume this is good to go, it's just a shed command after all).

@Stebalien Stebalien merged commit b23837a into master Apr 29, 2021
@Stebalien Stebalien deleted the steb/audit-dups branch April 29, 2021 01:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants