Dynamic fight_style with target_error results in inconsistent behavior?

[navv1234]

This is just a set of notes for myself, and I guess generic discussion on how we are currently doing the dynamic portion of fight_styles, and how it interacts with target_error. And perhaps more importantly, what we should do about this (if anything)?

Our fight_style system allows a certain amount of dynamic behaviour on the timing of the individual raid when using styles like HecticAddCleave. The dynamic values are calculated during simulator init, and are based on mean times.

Since the simulator iteration lengths are by default varied (+- 20%), and the raid events of the fight_styles are not accounting for fight length variance, we run into situations, where from iteration to iteration, the fight styles behave inconsistently. For example, with HecticAddCleave, no matter what length your fight is, the last add spawn will always be at t=225.

The above is further amplified by the fact that the simulator can be run in two different modes. When the simulator is run with no target error (i.e., fixed iterations), the simulator will run a close to uniform distribution of each fight length in the total range of fight length (max_time +- vary_combat_length). The uniformity is accomplished by adjusting the iteration combat length based on the proportion of work done. What follows is that the largest swings (from iteration to iteration) in combat time are also at the end of the simulation run.

With target_error option, the situation changes from above. As target_error has no fixed number of iterations (although it does converge towards some value), the system described above will result in the iteration fight length distribution to be such that the (vast) majority of iterations are concentrated at the extremes, and as such, the metric values will also be skewed towards the extremes. This leads to more iterations to converge to the wanted target_error value.

So, in total there are in my opinion two issues and/or discussion points:

1. Should the dynamic nature of the raid events issued by fight styles follow the mean values, or should they be computed in relation to an expected iteration time?
2. Should the fight length distribution of target_error converge to an uniform distribution with a range max_time +- vary_combat_length?

[scamille]

For 2, I would definitely say yes. My expectation would be that fight lengths are (roughly) uniformly distributed over +- combat length variation.
I haven't really realized that this is another problem with target_error and not knowing how many iterations you run. I plan to have a look at the code to know how exactly this works right now, and I guess then we can continue this discussion.

[navv1234]

I have a local change that simply makes it draw from the given range, when using target_error. The current behavior is still used for fixed iterations since there it works ok.

[navv1234]

I just went and changed how the target_error and fight length interact in c2dfa28. There is a rather painful pathological case for certain use cases (high target error), where the sim actually does not vary the iteration fight length at all. This happens when you have something like a target_error of 1.0. In this scenario, the first 100 iterations run based on the default "work" settings of the work queue, meaning there's hardly any variance at all (reason being, iteration count for target_error by default is 1,000,000). Since target_error is so high, the error analysis kicks in at cumulative iteration 100, at which point it either stops (target error satisfied), or computes to some low total work iteration count (suddenly we jump to the other end of the spectrum in terms of combat length variance).

[scamille]

Could we fix this by having some function for the fight length variance which results in a better uniform fight length distribution no matter how many iterations we do?
Maybe some mesh / buckets which get refined every 100 iteration, but will still result a roughly uniform distribution in each of these 100 iteration sections.

Unless we go with a completely random selection for the actual fight length variance used in a given iteration. I am not 100% sure if that is a good choice, but I am not statistician.