Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

Fuzzing for phragmen #4632

Closed
kianenigma opened this issue Jan 15, 2020 · 3 comments
Closed

Fuzzing for phragmen #4632

kianenigma opened this issue Jan 15, 2020 · 3 comments
Labels
J0-enhancement An additional feature request. U2-some_time_soon Issue is worth doing soon.

Comments

@kianenigma
Copy link
Contributor

There should be a fuzzer that does the following:

  1. (Important -- needed ASAP) Generate a random election edge vector, as such
let mut assignments = vec![
		StakedAssignment {
			who: 1, // voter
			distribution: vec![(10, 10)] // (target, weight)
		},
		StakedAssignment {
			who: 2,
			distribution: vec![
				(10, 15),
				(20, 5),
			],
		},
     ...
]

pass a cloned version of it through the reduce() function, then assert that it is equal to the original value. This can be easily done by using build_support_map and comparing the results. This means that the reduction algorithm, as expected, will just remove some edges but has no effect on the final outcome. I have written a function in my other PR to help with this:

#[allow(dead_code)] // to be used with fuzzing
pub fn assert_assignments_equal(
winners: &Vec<AccountId>,
ass1: &Vec<StakedAssignment<AccountId>>,
ass2: &Vec<StakedAssignment<AccountId>>,
) {
let support_1 = build_support_map::<Balance, AccountId>(winners, ass1);
let support_2 = build_support_map::<Balance, AccountId>(winners, ass2);
for (who, support) in support_1.iter() {
assert_eq!(support.total, support_2.get(who).unwrap().total);
assert_eq!(support.voters, support_2.get(who).unwrap().voters);
}
}
#[allow(dead_code)] // to be used with fuzzing
pub fn reduce_and_compare(
assignment: &Vec<StakedAssignment<AccountId>>,
winners: &Vec<AccountId>,
) {
let mut altered_assignment = assignment.clone();
crate::reduce(&mut altered_assignment);
assert_assignments_equal(
winners,
&assignment,
&altered_assignment,
);
}

Basically just run this over a sensibly large random input.

There should also be an assertion that the encoded length of the result is always less than or equal to that of the original one. We cannot make this strict to just less than since the random generation might actually lead to some graphs that cannot be reduced. A very good bonus is to tell us the size of reduced on average by how many percent.

  1. Likewise, the main elect function can also be compared to the float implementation. This has less priority at the moment. There is also a helper for this: run_and_compare in primitives/phragmen/mock.rs which is doing exactly that. The It just need to be fed some sensible data.

cc @shawntabrizi

@kianenigma kianenigma added J0-enhancement An additional feature request. U2-some_time_soon Issue is worth doing soon. labels Jan 15, 2020
@kianenigma
Copy link
Contributor Author

PR need to made against #4517

@expenses
Copy link
Contributor

@kianenigma I'll definitely try and do this if I have time, but I can't make any guarantees, and I'd prefer not to be assigned to it. Thanks!

@expenses expenses removed their assignment Jan 15, 2020
@kianenigma
Copy link
Contributor Author

Already pushed to #4517

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
J0-enhancement An additional feature request. U2-some_time_soon Issue is worth doing soon.
Projects
None yet
Development

No branches or pull requests

2 participants