Velvetg runs out of memory during graph building

[glebkuznetsov]

Running single sample de novo sv calling on c4.xlarge (4 vCPU, 7.5 gb RAM)

Reproducibly happens at same step as indicated by velvet stdout:

ubuntu@ip-172-30-3-201:~/millstone/genome_designer/temp_data/projects/96f5aa69/alignment_groups/297b4097/sample_alignments/3e1f9677$ /home/ubuntu/millstone[48/48$
_designer/conf/../tools/velvet/velvetg /home/ubuntu/millstone/genome_designer/conf/../temp_data/projects/96f5aa69/alignment_groups/297b4097/sample_alignments/3e1f
9677/assembly -ins_length 191 -exp_cov 147.784320298 -scaffolding no -ins_length_sd 87 -cov_cutoff 36.9460800745 -min_contig_lgth 200 -read_trkg yes
[0.000000] Reading read set file /home/ubuntu/millstone/genome_designer/conf/../temp_data/projects/96f5aa69/alignment_groups/297b4097/sample_alignments/3e1f9677/a
ssembly/Sequences;
[4.717501] 2491786 sequences found
[6.728925] Done
[9.928918] Reading pre-graph file /home/ubuntu/millstone/genome_designer/conf/../temp_data/projects/96f5aa69/alignment_groups/297b4097/sample_alignments/3e1f9677/
assembly/PreGraph
[9.929513] Graph has 5502746 nodes and 2491786 sequences
[14.266996] Scanning pre-graph file /home/ubuntu/millstone/genome_designer/conf/../temp_data/projects/96f5aa69/alignment_groups/297b4097/sample_alignments/3e1f967
7/assembly/PreGraph for k-mers
[15.540342] 33128836 kmers found
[18.424569] Sorting kmer occurence table ... 
[28.721637] Sorting done.
[28.721660] Computing acceleration table... 
[28.969931] Computing offsets... 
[29.254735] Ghost Threading through reads 0 / 2491786
[58.288563] Ghost Threading through reads 1000000 / 2491786
[87.722515] Ghost Threading through reads 2000000 / 2491786
[102.786356]  === Ghost-Threaded in 73.531620 s
[102.786389] Threading through reads 0 / 2491786
[143.224738] Threading through reads 1000000 / 2491786
[186.653840] Threading through reads 2000000 / 2491786
[208.575290]  === Threaded in 105.788900 s
[213.815008] Correcting graph with cutoff 0.200000
[214.091538] Determining eligible starting points
[221.738758] Done listing starting nodes
[221.738792] Initializing todo lists
[223.402352] Done with initilization
[223.402384] Activating arc lookup table
[226.714828] Done activating arc lookup table
Killed

From /var/log/syslog:

Sep 15 04:18:35 ip-172-30-3-201 kernel: [24905.320284] velvetg invoked oom-killer: gfp_mask=0x280da, order=0, oom_score_adj=0
Sep 15 04:18:35 ip-172-30-3-201 kernel: [24905.320288] velvetg cpuset=/ mems_allowed=0
...
Sep 15 04:18:35 ip-172-30-3-201 kernel: [24905.320577] Out of memory: Kill process 13461 (velvetg) score 629 or sacrifice child
Sep 15 04:18:35 ip-172-30-3-201 kernel: [24905.324019] Killed process 13461 (velvetg) total-vm:4861440kB, anon-rss:4808560kB, file-rss:4kB