Perc99 lower than Perc95 and Perc90

[nowakmikolaj]

I encountered a problem, it occasionally happens that Perc99 is smaller than Perc95 and Perc90. How is this possible and why does it happen?

Here is serialized StatsSummary object with threads and iterations info:

{
  "Threads": 100,
  "Iterations": 100,
  "FirstTime": "2023-12-07T17:35:01.284Z",
  "EndTime": "2023-12-07T17:36:21.07Z",
  "Samples": { "Total": 10000, "PerSecond": 125.33527185220464 },
  "SampleTime": {
    "Min": "00:00:00.0430000",
    "Max": "00:00:05.8470000",
    "Mean": "00:00:00.7750000",
    "Median": "00:00:00.8580000",
    "Perc90": "00:00:02.4550000",
    "Perc95": "00:00:02.7660000",
    "Perc99": "00:00:01.6390000"
  },
  "ErrorsCount": 0,
  "SampleTimePercentile99": "00:00:01.6390000",
  "ReceivedBytes": { "Total": 3770000, "PerSecond": 47251.39748828115 },
  "SentBytes": { "Total": 4890000, "PerSecond": 61288.94793572807 }
}

[rabelenda]

That is interesting, thank you for asking!.

One thing to take into consideration is that percentiles are calculated using approximation algorithms. In particular JMeter DSL uses psquare algorithm. This is done as to make an efficient calculation, because if you want the exact value then it need to review all metrics (either keeping them in memory or storing/loading them each time).

That being said, it may happen, moreover when using low amount of samples/measurements, that approximations may deviate from the actual result.

My guess is that the reason for such difference and "incongruent" results is the approximation algorithm used. If anyone has ideas on better algorithms or alternates, it would be awesome to improve it, but as far as we have tested P2 is very efficient and quite good approximating the percentiles.

Can you provide a jtl as evidence and help us further evaluate alternatives and consider potential improvements?

[nowakmikolaj]

@rabelenda
Thanks, here is jtl (I had to change extension to .txt to upload it)

2023-12-07-06-34-59-f7a80746-5c83-4db0-aa2f-62218fe381cc.txt