Bug in master version

[DavidBAEpstein]

• TIA Toolbox version: not relevant
• Python version: 3.7.13
• Operating System: Linux on Colab

What I Did

I was trying to understand why various versions of jupyter notebooks that I had been editing kept on crashing under various circumstances.

1. Navigated to https://raw.githubusercontent.com/TissueImageAnalytics/tiatoolbox/master/examples/inference-pipelines/idars.ipynb
2. Downloaded to own directory outside tiatoolbox hierarchy.
3. Uploaded to Google Drive/Colab
4. Clicked to run it
5. Removed all outputs.
6. Edited to find version of Python used by Colab 3.7.13 and OS Linux
7. Edited to GPU=True as directed in the comments.
8. Edited workers=8 to workers=1 (did this at two file locations), since Dang had warned me that asking for too many workers could cause memory overflow. (But the issue here is surely unrelated to the number of workers.)
9. Checked "runtime type" on Colab to make sure GPU was being used.
10. Deleted all outputs. Ran all, with resulting crash (saved as no_restart_master.idars.ipynb
11. Repeated above steps, but this time followed instructions from Colab to restart runtime as soon as Colab told me. Another crash (but different, saved as restarted_master.idars.ipynb).
12. Zipped together the two crash report files and uploaded to this issue.
13. Tracebacks are included in the saved files, which are attached. The two tracebacks are different.
    bugs.zip

Since I was previously able to run this file in Colab without difficulty and without intervening, and since it was probably checked on Colab before being merged, my conclusion is that the problem is due to recent changes in tiatoolbox.

IMPORTANT NOTES: The master version of idars.ipynb is dated 31 January 2022, so the change in the repo causing these problems must be subsequent to that. I think that I observed no problem two weeks ago. Problems with this file have nothing to do with defects in my virtual environment (discussed in Issue #352) since everything relevant was done on Colab.

[mostafajahanifar]

Thanks David for pointing out this issues.

As you have already discussed in #355 and you realized before, the first bug related to NOT restarting the Colab after installation of new packages (and hence not having the proper packages loaded) which will easily go away by restarting the runtime as you did here. And for the second bug, it was related to the spelling error in the pretrained IDaRS model weights dictionary which was not updated. This has been resolved in #355 as I have updated the weights dictionary.

As for the errors complaining about the "Pytorch", we just need to delete the cell that tries to install it. We don't need to install the Cuda version of Pytorch separately anymore as it's already included in Colab. In future, we will be eliminating that cell to avoid any errors.

It's really annoying that we need to restart the runtime after the installation of the Colab but there is no other way, AFAIK. We are thinking of adding a comment in the first section of all example notebooks so it warns the user to make sure to restart the runtime before continue.

I'm closing this issue in favor of #355 but feel free to open it you see appropriate.

[DavidBAEpstein]

@mostafajahanifar Thanks for good work, after which it is clear that asking the user to restart runtime is the least bad alternative. We need a text cell explanation of runtime, common to all notebooks.

It's already tough maintaining some kind of compatibility with Colab. We can't afford to spend additional time worrying about Kaggle, where the issues and fixes may be subtly different (though I suspect that Kaggle follows the lead of Colab). The next meeting of TIAToolbox should decide to totally eliminate Kaggle from tiatoolbox as soon as this is feasible. Our only users of Kaggle must be those users banned from Colab for misuse.