Update modules that depend on contains using the (deprecated) V1 manifest style to V2 manifest containers

[robsyme]

Description of feature

The 3.12.3 nf-core/rnaseq workflow pulls a few very old containers that use the V1 manifest version. These older style of containers can cause errors when using the wave.freeze feature of nextflow inspect.

These containers can cause the Wave error

ERROR ~ Unexpected response for 'https://wave.seqera.io/v2/wt/97eadd52dcc4/robsyme/wavetesting/manifests/85f9bc21a4c60b81': [400] {"errors":[{"code":"UNKNOWN","message":"INFO[0000] Retrieving image manifest quay.io/biocontainers/rseqc:3.0.1--py37h516909a_1 \nINFO[0000] Retrieving image quay.io/biocontainers/rseqc:3.0.1--py37h516909a_1 from registry quay.io \nERRO[0001] Error while retrieving image from cache: quay.io/biocontainers/rseqc:3.0.1--py37h516909a_1 unsupported MediaType: \"application/vnd.docker.distribution.manifest.v1+prettyjws\", see https://github.com/google/go-containerregistry/issues/377 \nINFO[0001] Retrieving image manifest quay.io/biocontainers/rseqc:3.0.1--py37h516909a_1 \nINFO[0001] Retrieving image quay.io/biocontainers/rseqc:3.0.1--py37h516909a_1 from registry quay.io \nerror building image: unsupported MediaType: \"application/vnd.docker.distribution.manifest.v1+prettyjws\", see https://github.com/google/go-containerregistry/issues/377\n"}]}

The affected processes and containers are:

• QUALIMAP_RNASEQ:
  • Uses biocontainers/qualimap:2.2.2d--1 (4 years old)
  • Recommendation biocontainers/qualimap:2.3--hdfd78af_0
• RSEQC_.*:
  • Uses biocontainers/rseqc:3.0.1--py37h516909a_1 (4 years old)
  • Recommendation is biocontainers/rseqc:5.0.3--py39hf95cd2a_0

[pinin4fjords]

Bumping RSeQC: nf-core/modules#4601

[robsyme]

Older V1 containers can be discovered with

nextflow inspect nf-core/rnaseq -r master -profile test -format json > config.json

for URL in $(jq -r '.processes | .[] | .container' config.json); do
    docker manifest inspect $URL > /dev/null 2> >(grep -i 'unsupported manifest media') || echo $URL 
done | sort | uniq

Current list (for -profile test) is

quay.io/biocontainers/rseqc:3.0.1--py37h516909a_1
quay.io/biocontainers/qualimap:2.2.2d--1

[drpatelh]

nextflow inspect will only get you the containers for a specific -profile, however, if you want to get ALL of the containers defined in a pipeline using the same container syntax as an nf-core pipeline, you can run this command in the root of a local clone:

$ find ./modules/ -type f -name '*.nf' -exec grep -A 2 'container ' {} \; | grep " }" | awk -v OFS='/' '{print "quay.io", substr($1,2,length($1)-2); }' | sort | uniq | xargs -i sh -c "echo {}; docker manifest inspect {}" | grep -E "quay.io|unsupported manifest" 

quay.io/biocontainers/bbmap:39.01--h5c4e2a8_0
quay.io/biocontainers/bedtools:2.30.0--hc088bd4_0
quay.io/biocontainers/bioconductor-dupradar:1.28.0--r42hdfd78af_0
quay.io/biocontainers/bioconductor-summarizedexperiment:1.24.0--r41hdfd78af_0
quay.io/biocontainers/bioconductor-tximeta:1.12.0--r41hdfd78af_0
quay.io/biocontainers/fastp:0.23.4--h5f740d0_0
quay.io/biocontainers/fastqc:0.12.1--hdfd78af_0
quay.io/biocontainers/fq:0.9.1--h9ee0642_0
quay.io/biocontainers/gffread:0.12.1--h8b12597_0
quay.io/biocontainers/hisat2:2.2.1--h1b792b2_3
quay.io/biocontainers/kallisto:0.48.0--h15996b6_2
quay.io/biocontainers/mulled-v2-1fa26d1ce03c295fe2fdcf85831a92fbcbd7e8c2:1df389393721fc66f3fd8778ad938ac711951107-0
quay.io/biocontainers/mulled-v2-1fa26d1ce03c295fe2fdcf85831a92fbcbd7e8c2:59cdd445419f14abac76b31dd0d71217994cbcc9-0
quay.io/biocontainers/mulled-v2-8849acf39a43cdd6c839a369a74c0adc823e2f91:ab110436faf952a33575c64dd74615a84011450b-0
quay.io/biocontainers/mulled-v2-a97e90b3b802d1da3d6958e0867610c718cb5eb1:2cdf6bf1e92acbeb9b2834b1c58754167173a410-0
quay.io/biocontainers/mulled-v2-cf0123ef83b3c38c13e3b0696a3f285d3f20f15b:64aad4a4e144878400649e71f42105311be7ed87-0
quay.io/biocontainers/multiqc:1.17--pyhdfd78af_0
quay.io/biocontainers/multiqc:1.17--pyhdfd78af_1
quay.io/biocontainers/perl:5.26.2
quay.io/biocontainers/picard:3.0.0--hdfd78af_1
quay.io/biocontainers/preseq:3.1.2--h445547b_2
quay.io/biocontainers/python:3.9--1
quay.io/biocontainers/qualimap:2.2.2d--1
unsupported manifest media type and no default available: application/vnd.docker.distribution.manifest.v1+prettyjws
quay.io/biocontainers/rseqc:3.0.1--py37h516909a_1
unsupported manifest media type and no default available: application/vnd.docker.distribution.manifest.v1+prettyjws
quay.io/biocontainers/salmon:1.10.1--h7e5ed60_0
quay.io/biocontainers/samtools:1.16.1--h6899075_1
quay.io/biocontainers/samtools:1.17--h00cdaf9_0
quay.io/biocontainers/sortmerna:4.3.4--h9ee0642_0
quay.io/biocontainers/stringtie:2.2.1--hecb563c_2
quay.io/biocontainers/subread:2.0.1--hed695b0_0
quay.io/biocontainers/trim-galore:0.6.7--hdfd78af_0
quay.io/biocontainers/ucsc-bedclip:377--h0b8a92a_2
quay.io/biocontainers/ucsc-bedgraphtobigwig:445--h954228d_0
quay.io/biocontainers/umi_tools:1.1.4--py38hbff2b2d_1
quay.io/nf-core/ubuntu:20.04

As you rightly pointed out @robsyme we need to update the following containers;

quay.io/biocontainers/qualimap:2.2.2d--1
quay.io/biocontainers/rseqc:3.0.1--py37h516909a_1

Given that these nf-core modules are now using nf-test and don't want to include this in the upcoming 3.14.0 release, we can patch these modules with nf-core modules patch to bump the container versions.

I have noticed some weird issues when bumping RSeqC in the past but I can't remember now so we will have to be a little cautious when doing this and check all of the releveant tests.

[drpatelh]

Fixed in #1152

[hiuchi]

I have recently started using nf-core and I think I may have a problem regarding this issue.

I have run the following command to build an RRBS analysis pipeline and got an error.

$nextflow run nf-core/methylseq -profile test,docker --outdir res
N E X T F L O W ~ version 23.10.1
Launching `https://github.com/nf-core/methylseq` [high_mestorf] DSL2 - revision: 54f823e102 [master].

As a result, I get the following error

-[nf-core/methylseq] Pipeline completed with errors-
[3c/13954f] NOTE: Process `NFCORE_METHYLSEQ:METHYLSEQ:PRESEQ_LCEXTRAP (Ecoli)` terminated with an error exit status (1) -- Error is ignored
ERROR ~ Error executing process > 'NFCORE_METHYLSEQ:METHYLSEQ:QUALIMAP_BAMQC (Ecoli)'

Caused by:
  Process `NFCORE_METHYLSEQ:METHYLSEQ:QUALIMAP_BAMQC (Ecoli)` terminated with an error exit status (125)

Command executed:

  unset DISPLAY
  mkdir -p tmp
  export _JAVA_OPTIONS=-Djava.io.tmpdir=./tmp
  qualimap \
      --java-mem-size=4915M \
      bamqc \
       \
      -bam Ecoli.sorted.bam \
       \
      -p non-strand-specific \
      --collect-overlap-pairs \
      -outdir Ecoli \
      -nt 2
  
  cat <<-END_VERSIONS > versions.yml
  "NFCORE_METHYLSEQ:METHYLSEQ:QUALIMAP_BAMQC":
      qualimap: $(echo $(qualimap 2>&1) | sed 's/^.*QualiMap v.//; s/Built.*$//')
  END_VERSIONS

Command exit status:
  125

Command output:
  (empty)

Command error:
  Unable to find image 'quay.io/biocontainers/qualimap:2.2.2d--1' locally
  2.2.2d--1: Pulling from biocontainers/qualimap
  docker: [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of quay.io/biocontainers/qualimap:2.2.2d--1 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/.
  See 'docker run --help'.

Finally, thank you very much for your wonderful framework.
I would be grateful for your cooperation.

[pinin4fjords]

@hiuchi Please raise this on nf-core/methylseq- they will likely need to update their copy of this module