inconsistent result in multiple runs

[ihji]

I'm looking into the strange result from pylint on Apache Beam Python SDK. Multiple runs return slightly different results. It looks something related to multi-processing thing because I couldn't find any pattern from this behavior.

Steps to reproduce

1. Clone https://github.com/apache/beam and checkout 0c8ccae9aa608f4d64b22c08d57b9aaa8724bfee
2. Go into sdks/python directory.
3. Run scripts/run_pylint.sh multiple times and compare the result.

Current behavior

beam/sdks/python$ for i in `seq 100`; do scripts/run_pylint.sh; done > lint_100.txt
beam/sdks/python$ grep slots-on-old-class lint_100.txt | wc -l
100
beam/sdks/python$ grep no-self-argument lint_100.txt | wc -l
42

Expected behavior

Consistently return the same result

pylint --version output

Using config file beam/sdks/python/.pylintrc
pylint 1.9.3, 
astroid 1.6.5
Python 2.7.13 (default, Nov 24 2017, 17:33:09) 
[GCC 6.3.0 20170516]

Related issue: https://issues.apache.org/jira/browse/BEAM-5846

[PCManticore]

Thanks for the report! multiprocessing might definitely affect this.

[jazelenk]

I observe this behavior as well. Most commonly, running with multiple jobs enabled gives false negatives- constraining to -j 1 shows more errors.

Here's an easy example:

(venv) $ cat a.py
# pylint: disable=invalid-name,missing-module-docstring,unused-import
import b
(venv) $ cat b.py
# pylint: disable=invalid-name,missing-module-docstring,unused-import
import a
(venv) $ pylint --score=n --reports=n a.py
(venv) $ pylint --score=n --reports=n b.py
(venv) $ pylint -j 2 --score=n --reports=n a.py b.py
(venv) $ pylint -j 1 --score=n --reports=n a.py b.py
************* Module b
b.py:1:0: R0401: Cyclic import (a -> b) (cyclic-import)

[Pierre-Sassoulas]

I can reproduce this with the latest version. Should be fixable by using the MapReduceMixin done by @doublethefish in #3458 that was only added to the duplicate checker.

[Pierre-Sassoulas]

Closing as duplicate of #374