XML from --junit-xml filled in with log messages

[AndreaCrotti]

Description

Using Pytest 3.2 and --junit-xml the output XML file gets polluted by logging messages.

Pip list

pytest==3.2.0
coverage==4.4.1
pytest-cov==2.5.1
pytest-reqs==0.0.6
mock==2.0.0

Versions

pytest==3.2.0 and nixos.

Example

This following example is enough to reproduce the issue.
If I run pytest tests.py the logging messages don't show (correctly).
Running instead pytest tests.py --junit-xml output.xml the log messages get into the XML file making it unusable.

import logging

logging.basicConfig()
logging.getLogger().setLevel(logging.INFO)

logger = logging.getLogger(__name__)


def function():
    logger.info("Hello from inside a test")
    return 2


def test_sample():
    assert function() == 2


if __name__ == '__main__':
    function()

[AndreaCrotti]

Noone else had this issue?
How do you guys actually use the --junit-xml plugin if it gets polluted by all the log information?

[nicoddemus]

Hi @AndreaCrotti,

In my experience one should configure logging at the application entry point, not at the module level; this way your tests won't run with logging configured and you can use the pytest-catchlog plugin to automatically capture the log messages for you and display them when tests fail, similar to what's done with stdout and stderr. Not configuring logging at the module level is specially important for libraries (as opposed to applications).

[AndreaCrotti]

Ok makes sense I'll try to see if I can disable the logging set up entirely in this project at work.
I know I generally do that way, but I was still surprised to see that if I do set it up at the module level it behaves like that.

[RonnyPfannschmidt]

the fact that logging will ensure unusable junitxml files is a pytest bug

[nicoddemus]

I disagree, from what I understand the problem is that the XML files contain a lot of logging output; they are working as intended if the tests are generating a ton of logging output.

Or perhaps I'm misunderstanding the issue. @AndreaCrotti can you post a snippet of a xml file you think is unreadable?

[AndreaCrotti]

Well running the code snippet above with pytest I get something like this

<?xml version="1.0" encoding="utf-8"?><testsuite errors="0" failures="0" name="pytest" skips="0" tests="1" time="0.127"><testcase classname="hello" file="hello.py" line="13" name="test_sample" time="0.12134623527526855"><system-err>INFO:hello:Hello from inside a test
</system-err></testcase></testsuite>

Now I just realised that in this case that in this case this is still valid XML, but I think/thought that in the project I tried that it wasn't the case, and some logging stuff was just printed out randomly into the output.
I checked again now and that's not actually the case, the massive log output gets added in the correct system-err elements.

So maybe it wasn't a problem after all, I could not make it work with CircleCI but that was probably something else.

Isn't there a way to actually not fill in these system-err elements at all if I don't need them?
Thanks!

[RonnyPfannschmidt]

@AndreaCrotti you can deactivate capture, currently there is no way to just discard ittho, so it will be in the normal output

[nicoddemus]

Thanks for following up @AndreaCrotti. Should we close this?