Process bam_ingress:minimap2_alignment (1) terminated with an error exit status (137)

[Yoon90]

Ask away!

Hello!
nextflow.log

Human-variation-workflow has stopped with an error exit status(137)
I guess it has occurred because of the shortage of RAM.

I'm attaching the head and the middle part of the nextflow.config

I found that the minimal required memory is 32GB but this computer cannot provide that much.
Is there any option to change/lower required Memory(RAM)?

Logs:

[0;92m|||||||||| �[0m�[2m_____ ____ ___ ____ __ __ _____ _ _
�[0;92m|||||||||| �[0m�[2m| | _ _ | | / | | | | __ | | ___
�[0;33m||||| �[0m�[2m| | | |) | | __) | |/| | | _| |/ ` | ' / |
�[0;33m||||| �[0m�[2m| || /| | / /| | | | ||| | (| | |) _
�[0;94m|||||||||| �[0m�[2m|_____|| |||| ||| ||_,|./|__/
�[0;94m|||||||||| �[0m�[1mwf-human-variation v1.9.2�[0m
�[2m--------------------------------------------------------------------------------�[0m
�[1mCore Nextflow options�[0m
�[0;34mrunName : �[0;32mtender_pike�[0m
�[0;34mcontainerEngine : �[0;32mdocker�[0m
�[0;34mcontainer : �[0;32montresearch/wf-human-variation:sha0800eade05e4cbb75d45421633c78c4f6320b2f6�[0m
�[0;34mlaunchDir : �[0;32m/home/heeji/epi2melabs/instances/wf-human-variation_01HMQ5WY7W4NNY8QABXDCK57S2�[0m
�[0;34mworkDir : �[0;32m/home/heeji/epi2melabs/instances/wf-human-variation_01HMQ5WY7W4NNY8QABXDCK57S2/work�[0m
�[0;34mprojectDir : �[0;32m/home/heeji/epi2melabs/workflows/epi2me-labs/wf-human-variation�[0m
�[0;34muserName : �[0;32mheeji�[0m
�[0;34mprofile : �[0;32mstandard�[0m
�[0;34mconfigFiles : �[0;32m/home/heeji/epi2melabs/workflows/epi2me-labs/wf-human-variation/nextflow.config�[0m

�[1mWorkflow Options�[0m
�[0;34msv : �[0;32mtrue�[0m
�[0;34msnp : �[0;32mtrue�[0m
�[0;34mcnv : �[0;32mtrue�[0m
�[0;34mstr : �[0;32mtrue�[0m

�[1mMain options�[0m
�[0;34msample_name : �[0;32mDBS_barcode05�[0m
�[0;34mbam : �[0;32m/media/heeji/2023_WES/nanopore/231227_DBS_4_pilot_test/no_sample/20231227_1908_X5_FAU99700_eaea91de/bam_pass/barcode05/merge_barcode05.bam�[0m
�[0;34mref : �[0;32m/media/heeji/2023_WES/hg38.fa�[0m
�[0;34mbam_min_coverage: �[0;32m5�[0m
�[0;34mout_dir : �[0;32m/home/heeji/epi2melabs/instances/wf-human-variation_01HMQ5WY7W4NNY8QABXDCK57S2/output�[0m

!! Only displaying parameters that differ from the pipeline defaults !!
�[2m--------------------------------------------------------------------------------�[0m
If you use epi2me-labs/wf-human-variation for your analysis please cite:

• The nf-core framework
  https://doi.org/10.1038/s41587-020-0439-x

�[2m--------------------------------------------------------------------------------�[0m
This is epi2me-labs/wf-human-variation v1.9.2.

Caused by:
Process bam_ingress:minimap2_alignment (1) terminated with an error exit status (137)

Command executed:

samtools bam2fq -@ 1 -T 1 merge_barcode05.bam | minimap2 -y -t 8 -ax map-ont hg38.fa - | samtools sort -@ 3 --write-index -o DBS_barcode05.bam##idx##DBS_barcode05.bam.bai -O bam --reference hg38.fa -

Command exit status:
137

Command output:
(empty)

Command error:
[M::mm_idx_gen::116.4521.80] collected minimizers
[M::mm_idx_gen::217.7851.78] sorted minimizers
[M::main::217.7921.78] loaded/built the index for 455 target sequence(s)
[M::mm_mapopt_update::219.4621.78] mid_occ = 728
[M::mm_idx_stat] kmer size: 15; skip: 10; is_hpc: 0; #seq: 455
[M::mm_idx_stat::220.401*1.77] distinct minimizers: 100202295 (37.96% are singletons); average occurrences: 5.732; average spacing: 5.587; total length: 3209286105
[M::bam2fq_mainloop] discarded 0 singletons
[M::bam2fq_mainloop] processed 236000 reads
[bam_sort_core] merging from 0 files and 3 in-memory blocks...
.command.sh: line 2: 25 Done samtools bam2fq -@ 1 -T 1 merge_barcode05.bam
26 Killed | minimap2 -y -t 8 -ax map-ont hg38.fa -
28 | samtools sort -@ 3 --write-index -o DBS_barcode05.bam##idx##DBS_barcode05.bam.bai -O bam --reference hg38.fa -

[SamStudio8]

It looks like the alignment process ran out of memory, we are imminently releasing a new version of the workflow with more generous memory directives that should alleviate this.

…

On Mon, 22 Jan 2024, 09:03 Yoon, ***@***.***> wrote: Ask away! Hello! nextflow.log <https://github.com/epi2me-labs/wf-human-variation/files/14007577/nextflow.log> Human-variation-workflow has stopped with an error exit status(137) I guess it has occurred because of the shortage of RAM. I'm attaching the head and the middle part of the nextflow.config I found that the minimal required memory is 32GB but this computer cannot provide that much. Is there any option to change/lower required Memory(RAM)? Logs: [0;92m|||||||||| �[0m�[2m_____ ____ ___ ____ __ __ _____ _ _ �[0;92m|||||||||| �[0m�[2m| *| _ _ | | / | | | | __ | | ___ �[0;33m||||| �[0m�[2m| | | |) | | __) | |/| | | _| |/ ` | ' / | �[0;33m||||| �[0m�[2m| || /| | / /| | | | ||| | (| | |) _ �[0;94m|||||||||| �[0m�[2m|_____|| |**|**|| ||**| ||_,|.**/|*__/ �[0;94m|||||||||| �[0m�[1mwf-human-variation v1.9.2�[0m �[2m--------------------------------------------------------------------------------�[0m �[1mCore Nextflow options�[0m �[0;34mrunName : �[0;32mtender_pike�[0m �[0;34mcontainerEngine : �[0;32mdocker�[0m �[0;34mcontainer : �[0;32montresearch/wf-human-variation:sha0800eade05e4cbb75d45421633c78c4f6320b2f6�[0m �[0;34mlaunchDir : �[0;32m/home/heeji/epi2melabs/instances/wf-human-variation_01HMQ5WY7W4NNY8QABXDCK57S2�[0m �[0;34mworkDir : �[0;32m/home/heeji/epi2melabs/instances/wf-human-variation_01HMQ5WY7W4NNY8QABXDCK57S2/work�[0m �[0;34mprojectDir : �[0;32m/home/heeji/epi2melabs/workflows/epi2me-labs/wf-human-variation�[0m �[0;34muserName : �[0;32mheeji�[0m �[0;34mprofile : �[0;32mstandard�[0m �[0;34mconfigFiles : �[0;32m/home/heeji/epi2melabs/workflows/epi2me-labs/wf-human-variation/nextflow.config�[0m �[1mWorkflow Options�[0m �[0;34msv : �[0;32mtrue�[0m �[0;34msnp : �[0;32mtrue�[0m �[0;34mcnv : �[0;32mtrue�[0m �[0;34mstr : �[0;32mtrue�[0m �[1mMain options�[0m �[0;34msample_name : �[0;32mDBS_barcode05�[0m �[0;34mbam : �[0;32m/media/heeji/2023_WES/nanopore/231227_DBS_4_pilot_test/no_sample/20231227_1908_X5_FAU99700_eaea91de/bam_pass/barcode05/merge_barcode05.bam�[0m �[0;34mref : �[0;32m/media/heeji/2023_WES/hg38.fa�[0m �[0;34mbam_min_coverage: �[0;32m5�[0m �[0;34mout_dir : �[0;32m/home/heeji/epi2melabs/instances/wf-human-variation_01HMQ5WY7W4NNY8QABXDCK57S2/output�[0m !! Only displaying parameters that differ from the pipeline defaults !! �[2m--------------------------------------------------------------------------------�[0m If you use epi2me-labs/wf-human-variation for your analysis please cite: - The nf-core framework https://doi.org/10.1038/s41587-020-0439-x �[2m--------------------------------------------------------------------------------�[0m This is epi2me-labs/wf-human-variation v1.9.2. Caused by: Process bam_ingress:minimap2_alignment (1) terminated with an error exit status (137) Command executed: samtools bam2fq -@ 1 -T 1 merge_barcode05.bam | minimap2 -y -t 8 -ax map-ont hg38.fa - | samtools sort -@ 3 --write-index -o DBS_barcode05.bam##idx##DBS_barcode05.bam.bai -O bam --reference hg38.fa - Command exit status: 137 Command output: (empty) Command error: [M::mm_idx_gen::116.452 *1.80] collected minimizers [M::mm_idx_gen::217.785*1.78] sorted minimizers [M::main::217.792 *1.78] loaded/built the index for 455 target sequence(s) [M::mm_mapopt_update::219.462*1.78] mid_occ = 728 [M::mm_idx_stat] kmer size: 15; skip: 10; is_hpc: 0; #seq: 455 [M::mm_idx_stat::220.401*1.77] distinct minimizers: 100202295 (37.96% are singletons); average occurrences: 5.732; average spacing: 5.587; total length: 3209286105 [M::bam2fq_mainloop] discarded 0 singletons [M::bam2fq_mainloop] processed 236000 reads [bam_sort_core] merging from 0 files and 3 in-memory blocks... .command.sh: line 2: 25 Done samtools bam2fq -@ 1 -T 1 merge_barcode05.bam 26 Killed | minimap2 -y -t 8 -ax map-ont hg38.fa - 28 | samtools sort -@ 3 --write-index -o DBS_barcode05.bam##idx##DBS_barcode05.bam.bai -O bam --reference hg38.fa - — Reply to this email directly, view it on GitHub <#133>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAIN6OXDJUG6AWXWHTN3JZLYPYTNZAVCNFSM6AAAAABCE2PTPKVHI2DSMVQWIX3LMV43ASLTON2WKOZSGA4TGNBRGM3TMNI> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[RenzoTale88]

@Yoon90 as @SamStudio8 mentioned, this is an out of memory issue, therefore decrease the memory usage won't fix the issue.

[Yoon90]

@RenzoTale88
As far as I know the memory usage could be increased.
https://labs.epi2me.io/trouble-shooting/#error-command-exit-status-137

But I guess this only works if the device has enough memory(more than 32GB).
Should i wait for a new version?

[cjw85]

You are correct, the workflow needs a minimum of 32Gb RAM to run, and very often the peak memory usage will be more than this. If your computer has only 32Gb RAM this workflow is unlikely to run reliably.