Memory leak seen when enabling reload on filebeat.config.prospectors

[vjsamuel]

Version: HEAD on master
OS: golang:1.8 docker container

On using the following config:

filebeat.config.prospectors:
  enabled: true
  path: prospectors.d/*.yml
  reload: 
    enabled: true
    period: 5s

It is seen that filebeat memory usage on docker keeps growing until a point that it is OOM killed.

On doing more analysis with pprof, it is seen that there are many unused prospector objects that are still lingering around.

Short term work around can be to just create runner objects only when a new cfg file comes in or a config is removed. This avoids creating Runners which causes the leak in filebeat.

[tsg]

Closing this one as the short term solution was implemented.