Console logging for statuschecker

[HaaiHenkie]

Hello,

I would like robotstatuschecker to show some console logging with regard to passed and failed test cases. When running tests with Robot Framework I usually only look at the console logging and will only open log.html if I see that tests have not passed.

This pull request is a proof of concept and not a fully implemented feature. Also, I am not a professional developer and apart from writing a keyword library I am not very familiar with the programmatic structure of Robot Framework.

My aim was to reproduce the console output of Robot Framework when running the test. This PoC does that quite well already.

I ran into the following issues:

• I wanted the console output to start with 'Postprocessing of results by StatusChecker...' so that it is clear that test execution has finished when robotstatuschecker is automatically started when robot test results are available. To me it seemed logical to do that under the start_result method of the StatusChecker class (implementation of ResultVisitor) because I assumed result is the highest level in the test results hierarchy. That did not work, so I have put it under __init__. That works, but I have no idea if that is the best way to do it.
• After some searching in the code, I found that the console logging is done by LOGGER in robot.output. I realize that this is not part of the robot.api. It works very well for start_suite end end_suite. Unfortunately it does not for end_test, because of AttributeError: 'TestCase' object has no attribute 'template'. I guess this is because of a difference between the result model and the suite model. Could making a slightly different implementation of LOGGER be a solution? This is because I ran it with RIDE Test Runner Plugin that uses a listener version 2 interface which is not compatible with robot.result.model.
• To write some initial separators of the right length around the message in te first bullet, it would be nice to use the same console width as LOGGER is using. I have found that the settings are passed to LOGGER as '**settings.console_output_config' in for example run.py. Is it possible to access those settings from robotstatuschecker.py if you are using it as --prerebotmodifier option? I am using RIDE and I have noticed that RIDE varies the consolewidth it is passing to robot. LOGGER is still using that settings when called from robotstatuschecker.py.
  - I have tried to mimic LOGGER.end_test myself. I do not think this is the way. An alternative is to leave it out and just see the numbers of errors per suite. Disadvantage of that is that you cannot see if statuschecker has added an error message to a testcase that already failed.

[HaaiHenkie]

Hello,

I have submitted a second commit. I found that LOGGER.start_test(test) and LOGGER.end_test(test) are actually working, unless you are using a listener library with a listener version 2 interface.

Unfortunately I just read this in the robot.output.LOGGER documentation

NOTE: This API is likely to change in future versions.

Anyway, here are various tests with the results.

1. From the command line:

    python -m robotstatuschecker output.xml
   

   Console output is as expected.

2. In RIDE with the test runner plugin custom script execution profile with the following cmd file

   :: Custom script for RIDE for testing logging of Robot Framework libraries
   @echo off
   robot %*
   python -m robotstatuschecker %~d2%~p2output.xml
   rebot -d %~d2%~p2 %~d2%~p2output.xml

   Console output is as expected as well. You have to choose between generating log.html and report.html twice or disabling the Report and Log buttons of the test runner plugin by using robot -l NONE -r NONE %*. Also test runner will adapt the console width of robot to the size of its window and the console width of robotstatuschecker will be the default 78 characters.

3. From the command line:

    robot --prerebotmodifier robotstatuschecker.StatusChecker test.robot
   

   Console output is as expected.

4. In RIDE with the test runner plugin robot execution profile and --prerebotmodifier robotstatuschecker.StatusChecker I receive an error because listener TestRunnerAgent.py uses the listener method version 2 that is not compatible with the robot.result.model.

5. Using robotstatuschecker programmatically

   from os.path import abspath, dirname, exists, join
   from shutil import rmtree
   import sys
   from robot import run, rebot
   from robotstatuschecker import process_output
   
   CURDIR = dirname(abspath(__file__))
   sys.path.insert(0, dirname(CURDIR))
   
   
   def check_tests(robot_file):
       results = join(CURDIR, 'reports')
       output = join(results, 'output.xml')
       if exists(results):
           rmtree(results)
       run(join(CURDIR, robot_file), output=output, log=None, report=None, loglevel='DEBUG')
       process_output(output)
       rebot(output, outputdir=results)
   
   
   if __name__ == '__main__':
       check_tests('test.robot')

   Expected output that would be generated by robot.output.LOGGER is not shown. Output generated by robot.api.logger is shown.

[HaaiHenkie]

In commit e8addb9 I added options to suppress console logging.

From the command line:

python -m robotstatuschecker infile --quiet

--quiet can be positioned before, after or between other arguments

In combination with robot and rebot:

robot --prerebotmodifier robotstatuschecker.StatusChecker:False data_sources
rebot --prerebotmodifier robotstatuschecker.StatusChecker:False robot_outputs

Programmatic use:

process_output('infile.xml', 'outfile.xml', verbose=False)

[aaltat]

Are you still interested to work with this PR? If you are could you explain what problem you are trying to solve, because for me it is not quite clear.