resume skips samples/chromosomes at isoquant step #76
Description
Description of the bug
Currently in v1.2 the pipeline will silently skip chromosomes of samples during the isoquant step.
If running the command from start to finish (no -resume) it will complete succesfully. However, when pipeline was interrupted or failed and you rerun with -resume it will silently drop samples.
I think it is highly likely due to this: https://midnighter.github.io/nextflow-gotchas/gotchas/join-on-map-fails-resume/
If I add failOnMismatch: true at the join statement in quantify_scrna_isoquant.nf it will indeed start throwing errors after resuming the pipeline.
// MODULE: Isoquant
//
ISOQUANT (
ch_split_bam
.join(ch_split_bai, by: [0], failOnMismatch: true)
.map{
meta, bam, bai ->
bam_basename = bam.toString().split('/')[-1]
split_bam_basename = bam_basename.split(/\./)
chr = [
'chr': split_bam_basename[1].replace("REF_","")
]
[ chr, meta, bam, bai]
}
.combine(ch_split_fasta, by: [0])
.combine(ch_split_fai, by: [0])
.combine(ch_split_gtf, by: [0])
.map{
chr, meta, bam, bai, fasta, fai, gtf ->
meta.sample_name = meta.id.split(/\./)[0]
meta.chr = meta.id.split(/\./)[1]
[ meta, bam, bai, fasta, fai, gtf ]
},
'tag:CB'
)
ch_versions = ch_versions.mix(ISOQUANT.out.versions)
Error:
Join mismatch for the following entries:
- key=[id:ERR9958133.REF_chr21, sample_name:ERR9958133, chr:REF_chr21] values=[/scratch/leuven/364/vsc36452/work/ab/a3b3f7dbc03461f15e9a972060af82/ERR9958133.REF_chr21.bam.bai]
- key=[id:ERR9958133.REF_chr21] values=[/scratch/leuven/364/vsc36452/work/b0/708372fb3bfd2270c0f159384826df/ERR9958133.REF_chr21.bam]
- key=[id:ERR9958134.REF_chr21, sample_name:ERR9958134, chr:REF_chr21] values=[/scratch/leuven/364/vsc36452/work/c5/0598c828f15f6f5636498050987a08/ERR9958134.REF_chr21.bam.bai]
- key=[id:ERR9958134.REF_chr21] values=[/scratch/leuven/364/vsc36452/work/92/e4f020b735be511426df6afacf6849/ERR9958134.REF_chr21.bam]
ERROR ~ Pipeline failed. Please refer to troubleshooting docs: https://nf-co.re/docs/usage/troubleshooting
-- Check '.nextflow.log' file for details
Command used
nextflow run main.nf -profile test,singularity --outdir ./results -resume