Be robust to disappearing processes when scanning process table

[ericaltendorf]

We scan the process table then inspect each process. During that time, the process can disappear. We should ensure we're robust to that possibility. The following stacktrace suggests we're not:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/psutil/_pslinux.py", line 1517, in wrapper
    return fun(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/psutil/_pslinux.py", line 1637, in cmdline
    with open_text("%s/%s/cmdline" % (self._procfs_path, self.pid)) as f:
  File "/usr/local/lib/python3.8/dist-packages/psutil/_common.py", line 724, in open_text
    return open(fname, "rt", **kwargs)
FileNotFoundError: [Errno 2] No such file or directory: '/proc/2755564/cmdline'
During handling of the above exception, another exception occurred:    
Traceback (most recent call last):
  File "./plotman.py", line 123, in <module>
    interactive.run_interactive()
  File ".../chia/plotman/interactive.py", line 293, in run_interactive
    curses.wrapper(curses_main)
  File "/usr/lib/python3.8/curses/__init__.py", line 105, in wrapper
    return func(stdscr, *args, **kwds)
  File ".../chia/plotman/interactive.py", line 134, in curses_main
    (started, msg) = manager.maybe_start_new_plot(dir_cfg, sched_cfg, plotting_cfg)
  File ".../chia/plotman/manager.py", line 66, in maybe_start_new_plot
    jobs = job.Job.get_running_jobs(dir_cfg['log'])
  File ".../chia/plotman/job.py", line 53, in get_running_jobs
    if proc.name() == 'chia':
  File "/usr/local/lib/python3.8/dist-packages/psutil/__init__.py", line 622, in name
    cmdline = self.cmdline()
  File "/usr/local/lib/python3.8/dist-packages/psutil/__init__.py", line 675, in cmdline
    return self._proc.cmdline()
  File "/usr/local/lib/python3.8/dist-packages/psutil/_pslinux.py", line 1524, in wrapper
    raise NoSuchProcess(self.pid, self._name)
psutil.NoSuchProcess: psutil.NoSuchProcess process no longer exists (pid=2755564, name='/usr/sbin/munin')

[ericaltendorf]

I think this was fixed. Need to find the commit that fixed it, though...

[BasilHorowt]

@ericaltendorf @altendky anyone reported anything like this recently? I had something similar that was (I think) specific to OSX but Kyle fixed it for me.