Change classic mod multiplier to 0.98x (for osu!std)

[ssz7-ch2]

Currently, the 0.96x score multiplier does not accurately represent the majority of beatmaps when it comes to slider acc.

Using a random sample of 500 beatmaps and the top 10 (stable leaderboard) scores on them, I calculated the scores with and without slider acc, and the ratio between the two.
spreadsheet

Median and average of the ratios:
Median: 0.982
Average: 0.970

Graphing the ratios in a histogram shows over 70% of scores have a ratio of > 0.96

Graphing the median ratio per beatmap also gives a similar result, with the majority of beatmaps having a ratio between 0.97 and 1

From this data, I suggest changing the classic mod score multiplier to 0.98x to better balance CL scores with non-CL scores.

How the scores were calculated

Circleguard was used to calculate the hitcounts with and without slider acc. The hitcounts were then plugged into the total score equation to get the score.

Notes:

• All sliderticks were assumed to be hit
• All scores were assumed to be FCs
• Circleguard isn't 100% accurate, but it's accurate enough where any errors will have very little effect on the results presented here.

Some extra stats:
Lowest ratio: 0.520 score link
Highest ratio: 1.020 score link
Lowest median ratio: 0.859 beatmap link
Highest median ratio: 1.007 beatmap link

Edit:

• fixed values from using incorrect base score for classic mod sliderends (150 -> 10)
• recalculated values with correct combo progress

[Zyfarok]

Thank you for those graphs !

Maybe 0.98 works better for "the majority of scores" (Edit: actually I didn't realise this was only based on top 10, this is even worse) but:

• Any multiplier that is not x1.00 will always be overly punitive for SS scores (and near SS scores)
• Any multiplier that is not <0.9 will always be overly generous to low acc scores on most maps

And more importantly, if the multiplier is not <0.97, people will have too much incentive to enable classic mod (especially on slider-heavy maps) since they will usually get better scores with it, but if the multiplier is not >0.99, the top of the leaderboard won't make sense, and thus there's no compromise that truly works.

This is why the solution of changing the score conversion to include an estimation of the slider-accuracy was proposed, so that stable scores and lazer scores can be fairly compared, and I understand that having a simple multiplier for mods is desirable because it's easier to understand, but here we're dealing with a situation that requires careful handling since people care about these historical scores, and this complexity is the only way to provide the fairness that players want here. So, I hope the devs change their mind on that topic.
(Note: I don't care how lazer's CL scores are treated and whether they are treated like classic scores, I'm only concerned about stable score here)

[Zyfarok]

Actually, from the 1st graph you can see that:

• for more than 25% of scores, the ideal multiplier should be <0.96 (1st quartile < 0.96)
• for more than 25% of scores, the multiplier should be >0.99 (3rd quartile > 0.99)

That's a lot of under- or over-evaluated scores, no matter which multiplier you pick: >25% will be off by 1.5% or more compared to a fair conversion.

[Zyfarok]

On another note, in case the suboptimal solution of using a mod multiplier to compare classic/lazer is forced to remain, I believe the median is not a good metric to pick the multiplier:
The average (~0.97 here) minimizes the squared-error and seems a better choice for that reason IMO. Also, as I stated above you don't want to incentivize picking the CL mod to get more score easily, so you might want to go lower than the average (which is why the current 0.96 is not that bad).
The only argument for a multiplier above 0.97 is that high-acc/SS scores are more visible in the leaderboard thus they might have more weight in the decision (people care more about them ?), but I'm not sure if it is a good argument...

Edit: also, why only use top 10 scores in leaderboards ? What happens to lower acc scores ?

[Zyfarok]

The api only shows up to the top 100, so don't think that would change much for ranked maps.

The reason I'm asking for that is that the order also matters for local, country and friends leaderboards and there you definitely expect lower accuracy scores.

CL being advantageous to beginners is annoying but it's not a big issue IMO. What I'm more concerned about is CL being advantageous on 5*+ map for most players.

[ssz7-ch2]

What I'm more concerned about is CL being advantageous on 5*+ map for most players.

Looking through some of the scores on the qualified 5*+ maps, the ratio is closer to 0.9 for scores that are a B or C rank, and closer to 0.95-1 if the acc is 95%+. So CL will be advantageous if the player can't play the map well.
Not sure how much of a concern this is, since this is also a problem for the current 0.96x multiplier, just to a lesser extent.

[Zyfarok]

btw @ssz7-ch2, first thx again for the insightful data.

Just to be sure:
You say you took into account that slider-ends base-value is now 150 instead of 10 in stable.
But do you also take into account:

• that (AFAIK) stable computes acc differently (using count of 300/100/50/miss only, ignoring ticks/ends base-values) than lazer (which uses base-score, and thus accounts for slider-ticks/ends base-values) ?
• that in stable, judgements on the slider head is impacted by missed slider-ends/ticks ?
• that in lazer scoring, the ComboScore portion of the score is also impacted by the acc ?

Overall, it would be nice if you could provide us your code to allow us to double-check how you compute things. (be careful to NOT share your API key)

[Zyfarok]

Also, the above makes me actually question how lazer's classic scores and stable scores can actually be represented with absolutely no distinction and the same mod multiplier. Stable scores should probably be treated with a different multiplier than classic because of that.

[ssz7-ch2]

• that (AFAIK) stable computes acc differently (using count of 300/100/50/miss only, ignoring ticks/ends base-values) than lazer (which uses base-score, and thus accounts for slider-ticks/ends base-values) ?

Stable acc wasn't used for the calculations. I should've clarified that I am recalculating the stable scores as if they were set on lazer for both with and without slider acc, not the stable -> lazer converted score. The converted score can be a bit off sometimes, so probably not a good idea to use it in the comparisons.

• that in stable, judgements on the slider head is impacted by missed slider-ends/ticks ?

Missed sliderends are accounted for, but all sliderticks are assumed to be hit (not really an issue except for really crazy slider maps)

• that in lazer scoring, the ComboScore portion of the score is also impacted by the acc ?

Yes. This part of the total score calculation was copied 1 to 1 from lazer.

I will look into uploading the code in a bit. Need to clean it up a bit and add some comments to explain the code.