Classification + Analysis step Inefficiencies

[JuiceSackey]

Hello,

It seems like classification and analysis step of the program take entirely too long for my dataset (~1 day for each step).
We are analyzing whole brain cfos expression, so there are over 700k cells, which is probably contributing to this issue. I raised an issue on the forums and I was told it may be due to inefficiencies in the coding for these steps and to raise the issue here.
The data is all stored/written to an internal HDD.

This is a log file for just analysis ran:
cellfinder_2021-09-05_09-30-11_log.txt

This is a log file with the classification+analysis step run:
cellfinder_2021-09-07_13-31-22_log.txt

As you can imagine, this is a pretty big bottleneck in processing our data.
Please let me know if you need any more information. Thanks!

My specs are as below:

• OS: Ubuntu
• Version 18.04.5 LTS
• Hardware specs: 64GB RAM, AMD® Ryzen 9 3900x 12-core processor × 24 CPU, GeForce RTX 2080 Ti/PCIe/SSE2 GPU

[adamltyson]

I've removed the bug label, as technically everything is working correctly, just very slow on big datasets, but this does need to be improved.

I don't know when I'll get time to look at this, but I would hope it shouldn't be too complex.

[adamltyson]

There have been many changes to the the way cellfinder works since this issue was raised, many of them with the aim of speeding up this type of analysis. In particular, the next version of cellfinder with have GPU support.

I'm going to close this issue, but if problems persist, please let us know.