Can't run Locust distributed with --csv-full-history

[heyman]

Describe the bug

Running Locust distributed and use the --csv-full-history option will cause an exception

Stacktrace

[2020-04-06 13:01:45,700] Jonatans-Air.localdomain/ERROR/stderr: Traceback (most recent call last):
[2020-04-06 13:01:45,700] Jonatans-Air.localdomain/ERROR/stderr: File "/Users/heyman/virtualenvs/locust/bin/locust", line 11, in <module>
[2020-04-06 13:01:45,702] Jonatans-Air.localdomain/ERROR/stderr:
[2020-04-06 13:01:45,702] Jonatans-Air.localdomain/ERROR/stderr: load_entry_point('locustio', 'console_scripts', 'locust')()
[2020-04-06 13:01:45,702] Jonatans-Air.localdomain/ERROR/stderr:
[2020-04-06 13:01:45,702] Jonatans-Air.localdomain/ERROR/stderr: File "/Users/heyman/projects/locust/locust/main.py", line 297, in main
[2020-04-06 13:01:45,703] Jonatans-Air.localdomain/ERROR/stderr:
[2020-04-06 13:01:45,703] Jonatans-Air.localdomain/ERROR/stderr: shutdown(code=code)
[2020-04-06 13:01:45,703] Jonatans-Air.localdomain/ERROR/stderr:
[2020-04-06 13:01:45,704] Jonatans-Air.localdomain/ERROR/stderr: File "/Users/heyman/projects/locust/locust/main.py", line 281, in shutdown
[2020-04-06 13:01:45,705] Jonatans-Air.localdomain/ERROR/stderr:
[2020-04-06 13:01:45,705] Jonatans-Air.localdomain/ERROR/stderr: write_stat_csvs(runner.stats, options.csvfilebase, options.stats_history_enabled)
[2020-04-06 13:01:45,705] Jonatans-Air.localdomain/ERROR/stderr:
[2020-04-06 13:01:45,706] Jonatans-Air.localdomain/ERROR/stderr: File "/Users/heyman/projects/locust/locust/stats.py", line 757, in write_stat_csvs
[2020-04-06 13:01:45,706] Jonatans-Air.localdomain/ERROR/stderr:
[2020-04-06 13:01:45,706] Jonatans-Air.localdomain/ERROR/stderr: f.write(stats_history_csv(stats, stats_history_enabled) + "\n")
[2020-04-06 13:01:45,707] Jonatans-Air.localdomain/ERROR/stderr:
[2020-04-06 13:01:45,707] Jonatans-Air.localdomain/ERROR/stderr: File "/Users/heyman/projects/locust/locust/stats.py", line 871, in stats_history_csv
[2020-04-06 13:01:45,708] Jonatans-Air.localdomain/ERROR/stderr:
[2020-04-06 13:01:45,708] Jonatans-Air.localdomain/ERROR/stderr: percentile_str = ','.join([
[2020-04-06 13:01:45,708] Jonatans-Air.localdomain/ERROR/stderr:
[2020-04-06 13:01:45,708] Jonatans-Air.localdomain/ERROR/stderr: File "/Users/heyman/projects/locust/locust/stats.py", line 872, in <listcomp>
[2020-04-06 13:01:45,709] Jonatans-Air.localdomain/ERROR/stderr:
[2020-04-06 13:01:45,710] Jonatans-Air.localdomain/ERROR/stderr: str(int(s.get_current_response_time_percentile(x) or 0)) for x in PERCENTILES_TO_REPORT])
[2020-04-06 13:01:45,710] Jonatans-Air.localdomain/ERROR/stderr:
[2020-04-06 13:01:45,710] Jonatans-Air.localdomain/ERROR/stderr: File "/Users/heyman/projects/locust/locust/stats.py", line 508, in get_current_response_time_percentile
[2020-04-06 13:01:45,711] Jonatans-Air.localdomain/ERROR/stderr:
[2020-04-06 13:01:45,711] Jonatans-Air.localdomain/ERROR/stderr: raise ValueError("StatsEntry.use_response_times_cache must be set to True if we should be able to calculate the _current_ response time percentile")
[2020-04-06 13:01:45,711] Jonatans-Air.localdomain/ERROR/stderr:
[2020-04-06 13:01:45,711] Jonatans-Air.localdomain/ERROR/stderr: ValueError
[2020-04-06 13:01:45,711] Jonatans-Air.localdomain/ERROR/stderr: :
[2020-04-06 13:01:45,711] Jonatans-Air.localdomain/ERROR/stderr: StatsEntry.use_response_times_cache must be set to True if we should be able to calculate the _current_ response time percentile
[2020-04-06 13:01:45,711] Jonatans-Air.localdomain/ERROR/stderr:

Steps to reproduce

Start a locust master node with the --csv and --csv-full-history options, and at least one Worker node.

Problem & possible solutions

In order to be able to calculate current response time percentiles (for the last 10 seconds) we copy the response times dict every second and cache it for the past 20 seconds. For performance and memory reasons (IIRC) we only do this for the Aggregated StatsEntry, however this seems to have been turned on (probably without knowing the full implications) by d7c7656, but that would not turn on response time caching for the Master node stats when running Locust distributed.

As I see it we have two options.

• It could be that the optimization isn't needed (we've gotten no reports on bad performance since d7c7656 was merged AFAIK), and in that case we should fix so that the response time caching works on the Master node as well. However we would need to do benchmarks to make sure that this doesn't significantly decrease performance or increase memory footprint.
• We could change so that we only report current response times for the Aggregated row in the history CSV file.

[heyman]

I did some benchmarking and found that it should be OK to cache the response times dicts for all stats entries. So I've now pushed a fix that solves the issue with the --csv-full-history option when running distributed.