Add ability to read config file in test_modules.py for filebeat

[kaiyan-sheng]

Not sure if this is caused by my lack of knowledge on filebeat, I can't find a way to give a config file for a specific module when running GENERATE=1 INTEGRATION_TESTS=1 TESTING_FILEBEAT_MODULES=nginx nosetests -v --nocapture tests/system/test_modules.py.
Right now Im adding hardcoded part into test_modules.py https://github.com/elastic/beats/blob/master/filebeat/tests/system/test_modules.py#L110 to specify variables such as convert.timezone=true to run tests.

Maybe similar to -expected.json, we can have -config.yml and if -config.yml exists, use it as config file when running the tests.

[ruflin]

@kaiyan-sheng +1 on this idea. There is at the moment not good way to do this.

[webmat]

Flexibility to specify a test matrix would be awesome.

A simple yaml file similar to what Kitchen.ci supports:

- suites:
  - default: ~                       # Runs the tests as they run now.
                                     # If test matrix file is absent, run things like right now too
  - timezone_tests:
    - files:                         # limit files used for this test
      - "*-timezone-*.log"
    - config:                        # config overrides
      - convert_timezone: true
    - event:                         # event overrides
      - event.timezone: EST
  - my_plain_log_tests:
    - config:
      - format: plain

Second test suite example would be enough to handle two types of logs like the Logstash module, where there's two pipeline and logs sets, for: "plain" and "json" formatted logs. Right now @ruflin added some very specific code to address the plain/json pattern.

• By default all logs are considered by a test, but you can limit via an array of globs
• By default, configs are at their default values, but you can override
• By default, events are processed as is, but may be useful to be able to specify an override, for very specific tests (to be seen, if actually needed)

[botelastic]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.