Use daemon mode for python 3.7

[bjoernricks]

Use daemon mode to circrumvent a memory leak (reported at https://bugs.python.org/issue37193).

Daemonic threads are killed immediately on shutdown by the python interpreter without waiting for until they are finished.

Maybe block_on_close = True could work too. In that case the interpreter waits for the threads to finish but doesn't track them in the _threads list.

It seems that the interpreter always waits for non-daemonic threads before finishing. Therefore I currently wonder why the Python devs added thread joining to the ThreadingMixIn (https://github.com/python/cpython/blob/3.7/Lib/socketserver.py#L674) at all when it is done on shutdown automatically already.

Short example script (works with Python 2.7 too) for playing with the different threading behavior.

from __future__ import print_function

import time

from threading import Thread


def run():

    for i in range(1, 10):
        print("I am the daemon thread. I keep on running bg...%i" % (i,))

        time.sleep(2)


if __name__ == "__main__":
    # Main thread

    thread = Thread(target=run)
   
    # play with daemonic and non-daemonic threads
    #  thread.daemon = True
    thread.daemon = False

    thread.start()

    print("My Daemon will take care")

    time.sleep(10)
    print('Exiting')
    
    # play with join
    # thread.join()

[bjoernricks]

Some more background about daemonic/non-daemonic threads can be found at

https://stackoverflow.com/questions/190010/daemon-threads-explanation

The actual behavior change in Python 3.7 is the ThreadingMixIn keeps a list of all created threads to join them on shutdown if daemon_threads is not true and block_on_close is true. By default daemon_threads is false and block_on_close is true therefore this is the default behavior and therefore leaks memory of all run threads.

Compare:

• ThreadingMixin Python 3.5
• ThreadingMixin Python 3.7

[jjnicola]

Tested it!! It works like a charm! The leak seems to be fixed. Thanks a lot @bjoernricks

[bjoernricks]

Just for the record some additional info about the issue

https://salsa.debian.org/debian/python-prometheus-client/commit/5aa256d8aab3b81604b855dc03f260342fc391fb

python/cpython#13893