Auto-dispose/cleanup PyImageJ when Python is shutting down

[elevans]

When PyImageJ is running and is suddenly terminated (e.g. interrupting an active GUI session with Ctrl + C) can sometimes result in a terminal session waiting for JPype Java threads to end. This isn't great behavior -- so we should auto cleanup PyImageJ upon termination and exit cleanly.

It looks like Python has a couple ways to exit:

• sys.exit()
• exit()
• SIGINT

We can use atexit() to run the appropriate cleanup routines. This will likely need some new hooks in scyjava where the JVM related methods live.

Here is a great example using atexit() in addition to catching SIGINT generated from the Ctrl + C keyboard interrupt.

And here's the relevant JPype docs section regarding handling JVM shutdown.

[elevans]

Shutting down the JVM is a little more complicated than what I thought at first glance. Calling jpype.shutdownJVM() after disposing the ImageJ session with ij.dispose() results an strange delay/hang where it seems the JVM is waiting for threads to finish. This issue is summed up nicely here: jpype-project/jpype#936. To summarize there is an old method (< 0.7.5) and new method (> 0.7.5) to JVM shutdown in JPype. We can achieve our intended effect of a quick and clean shutdown with these following changes:

scyjava

def shutdown_jvm():
    """Shutdown the JVM.

    Shutdown the JVM. Set the jpype.config.destroy_jvm flag to true
    to ask JPype to destory the JVM itself. Note that enabling 
    jpype.config.destroy_jvm can lead to delayed shutdown times while
    the JVM is waiting for threads to finish.
    """
    jpype.config.destroy_jvm = False
    jpype.shutdownJVM()

pyimagej

import scyjava as sj
import sys

def _signal_handler(signal=None, frame=None):
    """Handle clean exit at shutdown.

    Handle clean exit at shutdown and also catch Ctlr + C
    keyboard interrupts.
    """
    ij.dispose()
    sj.shutdown_jvm()
    sys.exit(0)

# handle clean exit and catch Ctrl + C KeyboardInterrupt
atexit.register(_signal_handler)
signal.signal(signal.SIGTERM, _signal_handler)
signal.signal(signal.SIGINT, _signal_handler)

This however is not actually a clean shutdown. My understanding is by setting the jpype.config.destroy_jvm flag to False we are telling JPype to skip the cleanup routines and unload the JVM which is abrupt. Is that what we want to do on exit from PyImageJ?

I was going to push this code but found a strange bug I'm not sure how to deal with yet. This code works as intended unless you initialize pyimagej in headless = True and then hit Ctrl + C. Doing so will return AWT blocker activation interrupted: followed by [ERROR] java.lang.InterruptedExcept when you try to hit Ctrl + C again. This absolutely locks python and the terminal. Ctrl + D does not work nor any other key command. I have to either end the python thread thats stuck or close my terminal...which of course is not acceptable. Its pretty clear to me that the JVM is catching the KeyboardInterupts but I'm not sure why its catching the Ctrl + C now and locking the terminal.

[ctrueden]

With scyjava 1.4.0, we now have an extensible when_jvm_stops(callback) function for registering callbacks that occur just prior to JVM shutdown. And with 0079c5d, the ImageJ2 gateway is automatically disposed as part of JVM shutdown in this manner. So this issue is resolved, pending some testing by @elevans on Windows.

[elevans]

Unfortunately I'm still getting a delay/hang on exit on Windows. Note that I'm using updated Windows 10 in an virtual machine (VirtualBox).

After starting pyimagej and calling the GUI with ij.ui().showUI() and then trying to exit with exit() is delayed.

Edit: I'm going to try another box that isn't a virtual machine. I will report back!

[elevans]

We resolved the delay on exit on Windows by ensuring that we dispose of all windows prior to JVM shutdown. For whatever reason there seems to be a hidden window that ij.dispose() is unable to dispose of, which leads the JVM waiting for the window to close. Here's the commit that fixes this issue: scijava/scyjava@c5d8ac1

[ctrueden]

The latest scyjava 1.4.1 has problems shutting down too eagerly in some scenarios; see this gitter thread for details.

[ctrueden]

We ran out of time to investigate this problem any further for the moment. But I added an entry to the Troubleshooting document about it with workarounds (a621108).