Race condition on 'select' UNIX call

[ichorid]

Tribler version: 7.1 Next
OS: Ubuntu 18.04

Got this error while running a test of tunnel-community-based download.

Exception in thread StandaloneEndpoint:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/mnt/leecher/tribler/Tribler/dispersy/endpoint.py", line 187, in _loop
    read_list, write_list, _ = select(socket_list, [], [], 0.1)
error: (4, 'Interrupted system call')

The problem appears only when exit code from

os.system('sudo tc qdisc add dev ' + networkDevice + ' root netem delay ' + str(latency) + 'ms')

is non-zero. It doesn't matter what the command is, if it returns with an error, it triggers the problem. If the exit code of the command is insulated by e.g. nohup, like:

os.system('sudo nohup tc qdisc add dev ' + networkDevice + ' root netem delay ' + str(latency) + 'ms')

the problem goes away.
If subprocess.Popen() is used instead of os.system(), the problem goes away too:

Popen("mycmd" + " myarg", shell=True).wait()

If sleep(5) is added after os.system() call, the problem goes away.
So, it is a definite race condition.

As a result, the tunnel community does not start properly, and anonymous downloading does not work.

[ichorid]

Ok, the problem is a race condition triggered by a signal (I guess it is SIGCHLD) generated by sub-shell started by os.system("run something"). It happens in:

File "/mnt/leecher/tribler/Tribler/dispersy/endpoint.py", line 187, in _loop
    read_list, write_list, _ = select(socket_list, [], [], 0.1)

Apparently, when Python interpreter starts a sub-process, and the process does not ends correctly, it could send a SIGCHLD to it's parent process.
If, at this moment, Python interpreter is waiting for data in the Unix "select" call, it would be interrupted by the signal. The signal should be processed by a signal handler, and if the handler is not there, we would get a error: (4, 'Interrupted system call').
Introduction to the problem on a Google mailing list.

[ichorid]

It is interesting to note that Tribler code contains almost no os.system() calls, except in Electrum wallet section.

[qstokkink]

Oh wow. Do you have any clue how to fix this?
I.e. can we catch the error and ignore this, or will it keep failing?

[qstokkink]

As there is a workaround for @ichorid and this is probably not a major issue for normal users, I moved this to the backlog.

[ichorid]

As we had moved to use Twisted endpoint, we don't use select anymore. Closed.