Bug: max > T::MaxCandidates; you might need to run benchmarks again

[wilwade]

When running benchmarks I saw this error

max > T::MaxCandidates; you might need to run benchmarks again

Example from Benchmark run: https://github.com/LibertyDSNP/frequency/actions/runs/4726913413/jobs/8386962925

Does this error matter? Why is it happening?

Appears to come from set_desired_candidates in cumulus: pallets/collator-selection/src/lib.rs

[rlaferla]

An observation. The same log file has a script error that occurs earlier:

[rlaferla]

It only occurs for pallet_collator_selection and below is the specific warning logs:

Running benchmarks for pallet_collator_selection
+ ./scripts/../target/release/frequency benchmark pallet --pallet=pallet_collator_selection --extrinsic '*' --chain=frequency-bench --execution=wasm --heap-pages=4096 --wasm-execution=compiled --steps=50 --repeat=20 --output=./scripts/../runtime/common/src/weights --template=./scripts/../.maintain/runtime-weight-template.hbs
2023-06-07 09:01:36 assembling new collators for new session 0 at #0    
2023-06-07 09:01:36 assembling new collators for new session 1 at #0    
2023-06-07 09:01:36 Starting benchmark: pallet_collator_selection::set_invulnerables    
2023-06-07 09:01:37 Starting benchmark: pallet_collator_selection::set_desired_candidates    
2023-06-07 09:01:37 max > T::MaxCandidates; you might need to run benchmarks again    
2023-06-07 09:01:37 max > T::MaxCandidates; you might need to run benchmarks again    
2023-06-07 09:01:37 max > T::MaxCandidates; you might need to run benchmarks again    
2023-06-07 09:01:37 max > T::MaxCandidates; you might need to run benchmarks again    
2023-06-07 09:01:37 max > T::MaxCandidates; you might need to run benchmarks again    
2023-06-07 09:01:37 max > T::MaxCandidates; you might need to run benchmarks again    
2023-06-07 09:01:37 max > T::MaxCandidates; you might need to run benchmarks again    
2023-06-07 09:01:37 max > T::MaxCandidates; you might need to run benchmarks again    
2023-06-07 09:01:37 max > T::MaxCandidates; you might need to run benchmarks again    
2023-06-07 09:01:37 max > T::MaxCandidates; you might need to run benchmarks again

[rlaferla]

	/// Desired number of candidates.
	///
	/// This should ideally always be less than [`Config::MaxCandidates`] for weights to be correct.
	#[pallet::storage]
	#[pallet::getter(fn desired_candidates)]
	pub type DesiredCandidates<T> = StorageValue<_, u32, ValueQuery>;

[rlaferla]

pub type CollatorMaxCandidates = ConstU32<50>;
pub type CollatorMinCandidates = ConstU32<1>;

[rlaferla]

The problem is a bug in Cumulus' collator-selection pallet where the benchmark to set the desired candidates uses a max value of 999 instead of T::MaxCandidates:get() - 1. I will file an issue and/or PR for the Cumulus project.

[rlaferla]

paritytech/cumulus#2715

[rlaferla]

paritytech/cumulus#2716

[rlaferla]