Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

srml-module: Phragmen election #3364

Merged
merged 55 commits into from
Sep 19, 2019
Merged

srml-module: Phragmen election #3364

merged 55 commits into from
Sep 19, 2019

Conversation

kianenigma
Copy link
Contributor

@kianenigma kianenigma commented Aug 12, 2019

//! # Phragmen Election Module.
//!
//! An election module based on sequential phragmen.
//!
//! ### Term and Round
//!
//! The election happens in _rounds_: every `N` blocks, all previous members are retired and a new
//! set is elected (which may or may not have an intersection with the previous set). Each round
//! lasts for some number of blocks defined by `TermDuration` storage item. The works _term_ and
//! _round_ can be used interchangeably in this context.
//!
//! `TermDuration` might change during a round. This can shorten or extend the length of the round.
//! The next election round's block number is never stored but rather always checked on the fly.
//! Based on the current block height and `TermDuration`, `BlockNumber % TermDuration == 0` will
//! always trigger a new election round.
//!
//! ### Voting
//!
//! Voters can vote for as many of the candidates by providing a list of account ids. Invalid votes
//! (voting for non-candidates) are ignored during election. Yet, a voter _might_ vote for a
//! future candidate. Voters reserve a bond as they vote. Each vote defines a `value`. This amount
//! is locked from the account of the voter and indicated the weight of the vote. Voters can update
//! their votes at any time by calling `vote()` again. This keeps the bond untouched but can
//! optionally change the locked `value`. After a round, votes are kept and might still be valid
//! for further rounds. A voter is responsible to call `remove_voter` upon once they are done to
//! have their bond back and remove the lock.
//!
//! ### Candidacy and Members
//!
//! Candidates also reserve a bond as they submit candidacy. A candidate cannot take their candidacy
//! back. A candidate can end up in one of the below situations:
//! - **Winner**: A winner is kept as a _member_. They must still have a bond in reserve
//! and they are automatically counted as a candidate for the next election.
//! - **Runner-up**: Runner ups are the best candidates immediately after the winners. The number
//! of runner-ups to keep is configurable. Runner-ups serve one major role: if a member is
//! forcefully removed, the next best runner-up is chosen as a replacement. If not, a new
//! election is triggered.
//! - **Loser**: Any of the candidate who are not a winner are left as losers. A loser might be an
//! _outgoing member_, meaning that they are an active member who failed to keep their spot. In
//! this case, the outgoing member will get their bond back. Otherwise, the bond is slashed from
//! the loser candidate.
//!
//! Note that with the members being the default candidates for the next round and votes
//! persisting in storage, the election system is entirely stable given no further input. This means
//! that if the system has a particular set of candidates `C` and voters `V` that lead to a
//! set of members `M` being elected, as long as `V` and `C` don't remove their candidacy and votes,
//! `M` will keep being re-elected at the end of each round.

Before merge, should consider:

  • Add the module to the node/runtime if needed.

Maybe in this PR, maybe later:

  • ability to report other voters.
  • ability to remove candidacy.

@kianenigma kianenigma added the A0-please_review Pull request needs code review. label Aug 12, 2019
srml/staking/src/lib.rs Outdated Show resolved Hide resolved
@gavofyork
Copy link
Member

runtime version should be bumped.

@kianenigma
Copy link
Contributor Author

cc @AlfonsoCev

/// - not be empty.
/// - not be less than the number of candidates.
///
/// `who` cannot be an old voter who has not yet called [`remove_voter`].
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/// `who` cannot be an old voter who has not yet called [`remove_voter`].
/// `who` cannot be an existing voter

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually: it can be an existing voter to update their vote.

It cannot be, exactly as phrased: old voter who has not yet called remove_voter

Copy link
Member

@gavofyork gavofyork Aug 15, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is "old voter" a technical term? it sounds like a voter who is reaching retirement age.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The full story is:

The code does not clean the votes at all and relies on two incentivization that voters themselves will do it. As a voter, once your vote has been counted once, then you are an old voter aka. retired if you want to name it like that. You should manually remove yourself via another tx because:

  1. you want your reserved bond back.
  2. you can't vote and participate anymore in rounds.

If you are fine with the logic I can formulate it better as inactive voter is the module docs. I agree that the term old voter is not clear.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

defunct_voter

/// Set the desired member count; if lower than the current count, then seats will not be up
/// election when they expire. If more, then a new vote will be started if one is not
/// already in progress.
#[weight = SimpleDispatchInfo::FixedOperational(10_000)]
Copy link
Member

@gavofyork gavofyork Aug 14, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does one storage write and it's 10k. the one above does 3 mutations and a binary search and costs 100x as much. wtf?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

about your 3 comments on the steepness of weights:

  1. they are taken by approximately halving the weights that are decided for the other election (maybe a wrong decision, but that's not the main reason of your complaint) via benchmarking, which is not through manual analysis but rather execution time measuring and some manual juice added by me. Some examples:

    1. submitting a candidate (as you recently discussed in some chats) is quite expensive with regards to computing the outcome of the election so I bumped it up a bit.
    2. removing a voter and adding one is more or less the same in terms of cpu/io but I manually reduced the weight of removing a bit for the same reason: it will potentially reduce the block time of the next election.
  2. and more importantly this type of weight that you are analyzing, while I agree with it, is the state-weight which we, could, but at the moment have not yet introduced anywhere. Hence, I didn't apply it here either. The current weight numbers are merely a representation of computation time. Based on this, setting the desired seats could be arguably substantially less computationally intensive than submitting candidacy, hence the 100x difference.

Copy link
Member

@gavofyork gavofyork Aug 14, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure i understand point (2) could you rephrase?

in general, weight should not be used to encode indirect costs because it is used to determine raw computation complexity to manage block fullness and is only incidentally used for fees. this is a good example whereby mixing the two creates a perverse incentive for validators to do less work by choosing transactions like these that are over-priced because they're accounting for a future cost, to be born by Some Other Guy.

if it's one-off indirect cost, then an additional non-weight fee is reasonable, to be charged in a discrete instance as part of the call. if it's an on-going cost, then an accordingly large deposit should be held (ReservableCurrrency can be used for this).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure i understand point (2) could you rephrase?

What I mean is that judging these based on how many mutate/read/writes they have is correct but not something that we yet do; the current weight in all modules is merely based on execution time. regardless of what they do internally. Totally black-boxed. analyzing read/write/mutate etc. will be one of the early next steps after Kusama.

... perverse incentive for validators to do less work by choosing transactions like these that are over-priced because they're accounting for a future cost, to be born by Some Other Guy.

Noted and I can conclude what the little bits of my manual tweaks are wrong. Will reverse them + fix the rest of them in further weight updates.

// split new set into winners and runner ups.
let split_point = desired_seats.min(new_set.len());
let mut new_members = (&new_set[..split_point]).to_vec();
let runner_ups = &new_set[split_point..]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be runners_up. you might want to do a search-replace.

@gavofyork gavofyork added A3-needsresolving and removed A0-please_review Pull request needs code review. labels Sep 13, 2019
@gavofyork gavofyork merged commit e8334c2 into master Sep 19, 2019
@gavofyork gavofyork deleted the kiz-phragmen-election branch September 19, 2019 10:04
en pushed a commit to en/substrate that referenced this pull request Sep 24, 2019
* phragmen election module.

* Add new files.

* Some doc update

* Update weights.

* bump and a few nits.

* Performance improvement.

* Master.into()

* Update srml/elections-phragmen/src/lib.rs

Co-Authored-By: Gavin Wood <[email protected]>

* Fix build

* Some fixes.

* Fix build.

* Proper outgoing and runner-up managment.

* Bit more sensical weight values.

* Update srml/elections-phragmen/src/lib.rs

* Update srml/elections-phragmen/src/lib.rs

Co-Authored-By: Gavin Wood <[email protected]>

* Update srml/elections-phragmen/src/lib.rs

Co-Authored-By: Gavin Wood <[email protected]>

* Update srml/elections-phragmen/src/lib.rs

Co-Authored-By: Gavin Wood <[email protected]>

* fix lock file

* Fix build.

* Remove runner-ups

* Some refactors.

* Add support for reporting voters.

* Fix member check.

* Remove equlize.rs

* Update srml/elections-phragmen/src/lib.rs

* Update srml/elections-phragmen/src/lib.rs

* Update srml/elections-phragmen/src/lib.rs

Co-Authored-By: Gavin Wood <[email protected]>

* Update srml/elections-phragmen/src/lib.rs

Co-Authored-By: Gavin Wood <[email protected]>

* Bring back runner ups.

* use decode_len

* Better weight values.

* Update bogus doc

* Bump.

* Update srml/elections-phragmen/src/lib.rs

Co-Authored-By: Gavin Wood <[email protected]>

* Review comments.

* One more test

* Fix tests

* Fix build

* .. and fix benchmarks.

* Update srml/elections-phragmen/src/lib.rs

* Version bump
kianenigma added a commit to kianenigma/seeding that referenced this pull request Sep 27, 2022
# Membership Request 

Hi, I am Kian Paimani, known as @kianenigma. I have been working on Polkadot/Kusama through Parity since February 2019 and I can categorize my main contributions to Polkadot's ecosystem as follows: 

1. Maintaining and developing the staking sub-system.
2. General FRAME development, especially testing and quality assurance. 
3. Polkadot-native side-projects. 
4. Education 

> My first contribution to Polkadot is also indeed related to staking: paritytech/substrate#1915

### Staking system

I joke as the Polkadot staking to be both my blessing and my curse over the years. I started working on it since the first days that I joined this ecosystem and the work [is ongoing ever since](https://github.com/orgs/paritytech/projects/33/views/9). In the past, I focused on making sure that the staking system is secure and to some extent scalable. More recently, I coordinated the (imminent) launch of Nomination Pools. Nowadays I also put an extra effort on making sure that this sub-system of Polkadot is *sustainable*, through code refactor and educating other core developers. 

Lastly, I have been the main author of the [Polkadot staking newsletter](https://gist.github.com/kianenigma/aa835946455b9a3f167821b9d05ba376), which is my main attempt at making the entire complexity and development of this part of the protocol transparent to the end-users.

I expect myself to contribute *directly* to the staking system for at least another ~12, if not more, and afterwards having the role of an advisor. 

Some notable contributions: 

- paritytech/substrate#4517
- paritytech/substrate#7910
- paritytech/substrate#6242
- paritytech/substrate#9415
- paritytech/polkadot#3141
- paritytech/substrate#11212
- paritytech/substrate#12129

### FRAME 

Historically, I have contributed a variety of domains in FRAME, namely: 

- Early version of the weight system paritytech/substrate#3816 paritytech/substrate#3157
- Early version of the transaction fee system
- Primitive arithmetic types paritytech/substrate#3456
- Council election pallet paritytech/substrate#3364

Many of which were, admittedly, a PoC at most, if not considered "poor". I am happy that nowadays many of the above have been refactored and are being maintained by new domain experts. 

These days, I put most of my FRAME focus on testing and quality assurance. Through my work in the staking system, I have had to deal with the high sensitivity and liveness requirement of protocol development first hand (I believe I had to do among the [very first storage migrations](paritytech/substrate#3948) in Kusama) and consequently I felt the need to make better testing facilities, all of which have been formulated in https://forum.polkadot.network/t/testing-complex-frame-pallets-discussion-tools/356. Some relevant PRs:

- paritytech/substrate#8038
- paritytech/substrate#9788
- paritytech/substrate#10174

Regardless of wearing the staking hat, I plan to remain a direct contributor to FRAME, namely because I consider it to be an important requirements of successfully delivering more features to Polkadot's ecosystem. 

### Polkadot-Native Side Projects

I have started multiple small, mostly non-RUST projects in the polkadot ecosystem that I am very happy about, and I plan to continue doing so. I have not yet found the time to make a "polished product" out of any of these, but I hope that I can help foster our community such that someday a team will do so. I consider my role, for the time being, to *put ideas out there* through these side projects. 

- https://github.com/substrate-portfolio/polkadot-portfolio/
- https://github.com/kianenigma/polkadot-basic-notification/
- https://github.com/paritytech/polkadot-scripts/
- https://github.com/paritytech/substrate-debug-kit/

### Education 

Lastly, aside from having had a number of educational talks over the years (all of which [are listed](https://hello.kianenigma.nl/talks/) in my personal website), I am a big enthusiast of the newly formed Polkadot Blockchain Academy. I have [been an instructor](https://singular.app/collectibles/statemine/16/2) in the first cohort, and continue to contribute for as long and as much as I can, whilst still attending to the former 3 duties. 

---

With all of that being said and done, I consider myself at the beginning of the path to Dan 4, but happy to start at a lower one as well.
bkchr added a commit to polkadot-fellows/seeding that referenced this pull request Sep 27, 2022
# Membership Request 

Hi, I am Kian Paimani, known as @kianenigma. I have been working on Polkadot/Kusama through Parity since February 2019 and I can categorize my main contributions to Polkadot's ecosystem as follows: 

1. Maintaining and developing the staking sub-system.
2. General FRAME development, especially testing and quality assurance. 
3. Polkadot-native side-projects. 
4. Education 

> My first contribution to Polkadot is also indeed related to staking: paritytech/substrate#1915

### Staking system

I joke as the Polkadot staking to be both my blessing and my curse over the years. I started working on it since the first days that I joined this ecosystem and the work [is ongoing ever since](https://github.com/orgs/paritytech/projects/33/views/9). In the past, I focused on making sure that the staking system is secure and to some extent scalable. More recently, I coordinated the (imminent) launch of Nomination Pools. Nowadays I also put an extra effort on making sure that this sub-system of Polkadot is *sustainable*, through code refactor and educating other core developers. 

Lastly, I have been the main author of the [Polkadot staking newsletter](https://gist.github.com/kianenigma/aa835946455b9a3f167821b9d05ba376), which is my main attempt at making the entire complexity and development of this part of the protocol transparent to the end-users.

I expect myself to contribute *directly* to the staking system for at least another ~12, if not more, and afterwards having the role of an advisor. 

Some notable contributions: 

- paritytech/substrate#4517
- paritytech/substrate#7910
- paritytech/substrate#6242
- paritytech/substrate#9415
- paritytech/polkadot#3141
- paritytech/substrate#11212
- paritytech/substrate#12129

### FRAME 

Historically, I have contributed a variety of domains in FRAME, namely: 

- Early version of the weight system paritytech/substrate#3816 paritytech/substrate#3157
- Early version of the transaction fee system
- Primitive arithmetic types paritytech/substrate#3456
- Council election pallet paritytech/substrate#3364

Many of which were, admittedly, a PoC at most, if not considered "poor". I am happy that nowadays many of the above have been refactored and are being maintained by new domain experts. 

These days, I put most of my FRAME focus on testing and quality assurance. Through my work in the staking system, I have had to deal with the high sensitivity and liveness requirement of protocol development first hand (I believe I had to do among the [very first storage migrations](paritytech/substrate#3948) in Kusama) and consequently I felt the need to make better testing facilities, all of which have been formulated in https://forum.polkadot.network/t/testing-complex-frame-pallets-discussion-tools/356. Some relevant PRs:

- paritytech/substrate#8038
- paritytech/substrate#9788
- paritytech/substrate#10174

Regardless of wearing the staking hat, I plan to remain a direct contributor to FRAME, namely because I consider it to be an important requirements of successfully delivering more features to Polkadot's ecosystem. 

### Polkadot-Native Side Projects

I have started multiple small, mostly non-RUST projects in the polkadot ecosystem that I am very happy about, and I plan to continue doing so. I have not yet found the time to make a "polished product" out of any of these, but I hope that I can help foster our community such that someday a team will do so. I consider my role, for the time being, to *put ideas out there* through these side projects. 

- https://github.com/substrate-portfolio/polkadot-portfolio/
- https://github.com/kianenigma/polkadot-basic-notification/
- https://github.com/paritytech/polkadot-scripts/
- https://github.com/paritytech/substrate-debug-kit/

### Education 

Lastly, aside from having had a number of educational talks over the years (all of which [are listed](https://hello.kianenigma.nl/talks/) in my personal website), I am a big enthusiast of the newly formed Polkadot Blockchain Academy. I have [been an instructor](https://singular.app/collectibles/statemine/16/2) in the first cohort, and continue to contribute for as long and as much as I can, whilst still attending to the former 3 duties. 

---

With all of that being said and done, I consider myself at the beginning of the path to Dan 4, but happy to start at a lower one as well.

Co-authored-by: Bastian Köcher <[email protected]>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants