'waiting for server to become ready'.... --usemem never starts?

[nextgenusfs]

I'm trying to test the run time of the --usemem versus disk usage. The --usemem option appears to start, but it seems that the sever never actually gets setup? How long does this process usually take?

Here's my system info:
Mac OS
24 cpus, 64 GB RAM
fuNOG database

The disk method runs okay:

 emapper.py -i test.proteins.fa -d fuNOG -o disk --cpu 12
#  emapper-1.0.0-9-g6bc9664
# ./emapper.py  -i test.proteins.fa -d fuNOG -o disk --cpu 12
/usr/local/eggnog-mapper/data/hmmdb_levels/fuNOG_hmm/fuNOG_hmm.all_hmm
Sequence mapping starts now!
26 1415.93303132 0.02 q/s
51 1415.96953607 0.04 q/s
76 1415.98900223 0.05 q/s
101 1416.04119062 0.07 q/s
126 1416.09335566 0.09 q/s
151 1416.15860415 0.11 q/s
176 1416.17889738 0.12 q/s
201 1416.19831896 0.14 q/s
226 1416.22555113 0.16 q/s
251 1416.2860117 0.18 q/s
276 1416.30812669 0.19 q/s
301 1416.33336639 0.21 q/s
326 1416.34853315 0.23 q/s
351 1416.35899854 0.25 q/s
376 1416.37289238 0.27 q/s
401 1416.39509058 0.28 q/s
426 1416.4632082 0.30 q/s
451 1416.50826216 0.32 q/s
 Processed queries:474 total_time:1416.54748416 rate:0.33 q/s
Hit refinement starts now
26 37.9274868965 0.69 q/s (refinement)
51 37.9279999733 1.34 q/s (refinement)
76 59.0786979198 1.29 q/s (refinement)
101 86.2056558132 1.17 q/s (refinement)
126 86.206127882 1.46 q/s (refinement)
151 86.2064938545 1.75 q/s (refinement)
176 86.2070047855 2.04 q/s (refinement)
201 86.2073988914 2.33 q/s (refinement)
226 89.5916199684 2.52 q/s (refinement)
251 89.5920569897 2.80 q/s (refinement)
276 116.3443048 2.37 q/s (refinement)
301 116.344689846 2.59 q/s (refinement)
326 123.786115885 2.63 q/s (refinement)
351 154.735613823 2.27 q/s (refinement)
376 154.735872984 2.43 q/s (refinement)
401 154.736131907 2.59 q/s (refinement)
 Processed queries:420 total_time:154.779358864 rate:2.71 q/s
Reading HMM matches
Functional annotation of refined hits starts now
 Processed queries:420 total_time:3.05103421211 rate:137.66 q/s
Done
   disk.emapper.hmm_hits
   disk.emapper.seed_orthologs
   disk.emapper.annotations
Total time: 1574.71 secs

emapper.py -i test.proteins.fa -d fuNOG -o mem --cpu 12 --usemem
#  emapper-1.0.0-9-g6bc9664
# ./emapper.py  -i test.proteins.fa -d fuNOG -o mem --cpu 12 --usemem
/usr/local/eggnog-mapper/data/hmmdb_levels/fuNOG_hmm/fuNOG_hmm.all_hmm
Loading server at localhost, port 51814-51815
Waiting for server to become ready... localhost 51814
Waiting for server to become ready... localhost 51814
Waiting for server to become ready... localhost 51814
Waiting for server to become ready... localhost 51814
Waiting for server to become ready... localhost 51814
Waiting for server to become ready... localhost 51814
Waiting for server to become ready... localhost 51814
Waiting for server to become ready... localhost 51814
...
...
...

I can see in the system monitor that hmmpgmd is running and occupying ~5-6 GB of RAM (and it doesn't seem to be expanding). But after 10 minutes, it is still giving me the "waiting for server to become ready" message.

Thanks,
Jon

[jhcepas]

ten minutes sounds like too much for that database. Do you see the memory usage to increase? or it got stuck?

[nextgenusfs]

No memory usage increases, seems like it is stuck. hmpgmd memory footprint steady at 5.55GB.

[jhcepas]

and, if you try to start an emapper search connecting to that port, does it work?
something like emapper.py -d fuNOG:localhost:port -i ....

[nextgenusfs]

It does not, says:

emapper.py -d fuNOG:localhost:51814 -i test.proteins.fa -o mem
#  emapper-1.0.0-9-g6bc9664
# ./emapper.py  -d fuNOG:localhost:51814 -i test.proteins.fa -o mem
eggnog-mapper server not found at localhost:51814

[jhcepas]

looks like an issue accessing the sockets port. Never used this mode in OSX, I will try to have a look...

[nextgenusfs]

Thanks. Is the other method that you highlight, i.e. here would that also work? For example, to split the input, run say disk searches in parallel and then combine them after? Or does this only work with diamond search? I'm just trying to make it run faster, i.e. 10,000 proteins (typical fungal genome) is taking a long time with disk search - nearly all the time is spent on the first hmmscan search.

[jhcepas]

for 10k proteins, diamond mode is the best option. No need to split the input file, just call emapper in diamond mode. You can also submit to http://eggnog-mapper.embl.de which would probably run it faster than on a laptop.

[nextgenusfs]

Ok. I was hesitant to do that because the results from diamond mode are not the same as hmmer mode. I was assuming since hmmer was default that those were more accurate, but maybe that is an incorrect assumption?

[jhcepas]

diamond mode works even better than hmm in our benchmarks (distant homologs that diamond might miss are still used for annotation as they are part of the precomputed orhology tables used in the second phase of eggnog mapper).
Diamond mode would only miss annotations for sequences without any (relatively close) representative in the 2000 proteomes covered in eggnog4.5