ModuleNotFoundError for nf-core v3.0.2 installed from bioconda

[pmoris]

Description of the bug

The most recent version of nf-core tools gives me an error when running most commands like nf-core pipelines create, but only for the version installed from bioconda (mamba create -n nf-core && mamba install -c bioconda nf-core), not for pip/PyPi.

From what I can tell, the problem lies with the different way in which pip and conda resolve optional dependencies (https://stackoverflow.com/questions/42587385/install-extras-with-conda).

Looking at the documentation for markdown-it-py, you can see that one of its optional components is linkify, which pip can pull in by specifying markdown-it-py[linkify]. I suspect this is not happening in the bioconda recipe.

Proposed fix: add linkify to the bioconda recipe: https://github.com/bioconda/bioconda-recipes/blob/master/recipes/nf-core/meta.yaml (and/or ping the trogon devs to do it too?).

### Command used and terminal output

└─▶ nf-core pipelines create


                                          ,--./,-.
          ___     __   __   __   ___     /,-._.--~\ 
    |\ | |__  __ /  ` /  \ |__) |__         }  {
    | \| |       \__, \__/ |  \ |___     \`-._,-`-,
                                          `._,._,'

    nf-core/tools version 3.0.2 - https://nf-co.re


INFO     Launching interactive nf-core pipeline creation tool.                                                                                
╭──────────────────────────────────────────────────── Traceback (most recent call last) ─────────────────────────────────────────────────────╮
│ /home/pmoris/mambaforge/envs/nf-core-3/lib/python3.13/site-packages/textual/widgets/_markdown.py:798 in _on_mount                          │
│                                                                                                                                            │
│    795 │                                                                                        ╭───── locals ──────╮                      │
│    796 │   async def _on_mount(self, _: Mount) -> None:                                         │    _ = Mount()    │                      │
│    797 │   │   if self._markdown is not None:                                                   │ self = Markdown() │                      │
│ ❱  798 │   │   │   await self.update(self._markdown)                                            ╰───────────────────╯                      │
│    799 │                                                                                                                                   │
│    800 │   def _watch_code_dark_theme(self) -> None:                                                                                       │
│    801 │   │   """React to the dark theme being changed."""                                                                                │
│                                                                                                                                            │
│ /home/pmoris/mambaforge/envs/nf-core-3/lib/python3.13/site-packages/textual/widgets/_markdown.py:989 in await_update                       │
│                                                                                                                                            │
│    986 │   │   │   """Update in batches."""                                                                                                │
│    987 │   │   │   BATCH_SIZE = 200                                                                                                        │
│    988 │   │   │   batch: list[MarkdownBlock] = []                                                                                         │
│ ❱  989 │   │   │   tokens = await asyncio.get_running_loop().run_in_executor(                                                              │
│    990 │   │   │   │   None, parser.parse, markdown                                                                                        │
│    991 │   │   │   )                                                                                                                       │
│    992                                                                                                                                     │
│                                                                                                                                            │
│ ╭─────────────────────────────────────────────────── locals ────────────────────────────────────────────────────╮                          │
│ │             batch = []                                                                                        │                          │
│ │        BATCH_SIZE = 200                                                                                       │                          │
│ │          markdown = '\n# Welcome to the nf-core pipeline creation wizard\n\nThis app will help you creat'+614 │                          │
│ │    markdown_block = <DOMQuery query='MarkdownBlock'>                                                          │                          │
│ │            parser = markdown_it.main.MarkdownIt()                                                             │                          │
│ │              self = Markdown()                                                                                │                          │
│ │ table_of_contents = []                                                                                        │                          │
│ ╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────╯                          │
│                                                                                                                                            │
│ /home/pmoris/mambaforge/envs/nf-core-3/lib/python3.13/concurrent/futures/thread.py:58 in run                                               │
│                                                                                                                                            │
│    55 │   │   │   return                                                                       ╭── locals ───╮                             │
│    56 │   │                                                                                    │ self = None │                             │
│    57 │   │   try:                                                                             ╰─────────────╯                             │
│ ❱  58 │   │   │   result = self.fn(*self.args, **self.kwargs)                                                                              │
│    59 │   │   except BaseException as exc:                                                                                                 │
│    60 │   │   │   self.future.set_exception(exc)                                                                                           │
│    61 │   │   │   # Break a reference cycle with the exception 'exc'                                                                       │
│                                                                                                                                            │
│ /home/pmoris/mambaforge/envs/nf-core-3/lib/python3.13/site-packages/markdown_it/main.py:276 in parse                                       │
│                                                                                                                                            │
│   273 │   │   if not isinstance(src, str):                                                                                                 │
│   274 │   │   │   raise TypeError(f"Input data should be a string, not {type(src)}")                                                       │
│   275 │   │   state = StateCore(src, self, env)                                                                                            │
│ ❱ 276 │   │   self.core.process(state)                                                                                                     │
│   277 │   │   return state.tokens                                                                                                          │
│   278 │                                                                                                                                    │
│   279 │   def render(self, src: str, env: EnvType | None = None) -> Any:                                                                   │
│                                                                                                                                            │
│ ╭───────────────────────────────────────────── locals ──────────────────────────────────────────────╮                                      │
│ │   env = {}                                                                                        │                                      │
│ │  self = markdown_it.main.MarkdownIt()                                                             │                                      │
│ │   src = '\n# Welcome to the nf-core pipeline creation wizard\n\nThis app will help you creat'+614 │                                      │
│ │ state = <markdown_it.rules_core.state_core.StateCore object at 0x7f461bf44440>                    │                                      │
│ ╰───────────────────────────────────────────────────────────────────────────────────────────────────╯                                      │
│                                                                                                                                            │
│ /home/pmoris/mambaforge/envs/nf-core-3/lib/python3.13/site-packages/markdown_it/parser_core.py:45 in process                               │
│                                                                                                                                            │
│   42 │   def process(self, state: StateCore) -> None:                                                                                      │
│   43 │   │   """Executes core chain rules."""                                                                                              │
│   44 │   │   for rule in self.ruler.getRules(""):                                                                                          │
│ ❱ 45 │   │   │   rule(state)                                                                                                               │
│   46                                                                                                                                       │
│                                                                                                                                            │
│ ╭──────────────────────────────────── locals ────────────────────────────────────╮                                                         │
│ │  self = <markdown_it.parser_core.ParserCore object at 0x7f461bf44050>          │                                                         │
│ │ state = <markdown_it.rules_core.state_core.StateCore object at 0x7f461bf44440> │                                                         │
│ ╰────────────────────────────────────────────────────────────────────────────────╯                                                         │
│                                                                                                                                            │
│ /home/pmoris/mambaforge/envs/nf-core-3/lib/python3.13/site-packages/markdown_it/rules_core/inline.py:10 in inline                          │
│                                                                                                                                            │
│    7 │   │   if token.type == "inline":                                                                                                    │
│    8 │   │   │   if token.children is None:                                                                                                │
│    9 │   │   │   │   token.children = []                                                                                                   │
│ ❱ 10 │   │   │   state.md.inline.parse(token.content, state.md, state.env, token.children)                                                 │
│   11                                                                                                                                       │
│                                                                                                                                            │
│ ╭──────────────────────────────────── locals ────────────────────────────────────╮                                                         │
│ │ state = <markdown_it.rules_core.state_core.StateCore object at 0x7f461bf44440> │                                                         │
│ │ token = Token(                                                                 │                                                         │
│ │         │   type='inline',                                                     │                                                         │
│ │         │   tag='',                                                            │                                                         │
│ │         │   nesting=0,                                                         │                                                         │
│ │         │   attrs={},                                                          │                                                         │
│ │         │   map=[1, 2],                                                        │                                                         │
│ │         │   level=1,                                                           │                                                         │
│ │         │   children=[],                                                       │                                                         │
│ │         │   content='Welcome to the nf-core pipeline creation wizard',         │                                                         │
│ │         │   markup='',                                                         │                                                         │
│ │         │   info='',                                                           │                                                         │
│ │         │   meta={},                                                           │                                                         │
│ │         │   block=True,                                                        │                                                         │
│ │         │   hidden=False                                                       │                                                         │
│ │         )                                                                      │                                                         │
│ ╰────────────────────────────────────────────────────────────────────────────────╯                                                         │
│                                                                                                                                            │
│ /home/pmoris/mambaforge/envs/nf-core-3/lib/python3.13/site-packages/markdown_it/parser_inline.py:143 in parse                              │
│                                                                                                                                            │
│   140 │   ) -> list[Token]:                                                                                                                │
│   141 │   │   """Process input string and push inline tokens into `tokens`"""                                                              │
│   142 │   │   state = StateInline(src, md, env, tokens)                                                                                    │
│ ❱ 143 │   │   self.tokenize(state)                                                                                                         │
│   144 │   │   rules2 = self.ruler2.getRules("")                                                                                            │
│   145 │   │   for rule in rules2:                                                                                                          │
│   146 │   │   │   rule(state)                                                                                                              │
│                                                                                                                                            │
│ ╭────────────────────────────────── locals ──────────────────────────────────╮                                                             │
│ │    env = {}                                                                │                                                             │
│ │     md = markdown_it.main.MarkdownIt()                                     │                                                             │
│ │   self = <markdown_it.parser_inline.ParserInline object at 0x7f461c99bb60> │                                                             │
│ │    src = 'Welcome to the nf-core pipeline creation wizard'                 │                                                             │
│ │  state = StateInline(pos=[17 of 47], token=0)                              │                                                             │
│ │ tokens = []                                                                │                                                             │
│ ╰────────────────────────────────────────────────────────────────────────────╯                                                             │
│                                                                                                                                            │
│ /home/pmoris/mambaforge/envs/nf-core-3/lib/python3.13/site-packages/markdown_it/parser_inline.py:123 in tokenize                           │
│                                                                                                                                            │
│   120 │   │   │                                                                                                                            │
│   121 │   │   │   if state.level < maxNesting:                                                                                             │
│   122 │   │   │   │   for rule in rules:                                                                                                   │
│ ❱ 123 │   │   │   │   │   ok = rule(state, False)                                                                                          │
│   124 │   │   │   │   │   if ok:                                                                                                           │
│   125 │   │   │   │   │   │   break                                                                                                        │
│   126                                                                                                                                      │
│                                                                                                                                            │
│ ╭──────────────────────────────────── locals ────────────────────────────────────╮                                                         │
│ │        end = 47                                                                │                                                         │
│ │ maxNesting = 20                                                                │                                                         │
│ │         ok = False                                                             │                                                         │
│ │      rules = [                                                                 │                                                         │
│ │              │   <function text at 0x7f461da3dda0>,                            │                                                         │
│ │              │   <function linkify at 0x7f461da3db20>,                         │                                                         │
│ │              │   <function newline at 0x7f461da3dc60>,                         │                                                         │
│ │              │   <function escape at 0x7f461dde85e0>,                          │                                                         │
│ │              │   <function backtick at 0x7f461da3d120>,                        │                                                         │
│ │              │   <function tokenize at 0x7f461da3cc20>,                        │                                                         │
│ │              │   <function tokenize at 0x7f461ddb8860>,                        │                                                         │
│ │              │   <function link at 0x7f461da3d9e0>,                            │                                                         │
│ │              │   <function image at 0x7f461da3d8a0>,                           │                                                         │
│ │              │   <function autolink at 0x7f461da3cf40>,                        │                                                         │
│ │              │   ... +2                                                        │                                                         │
│ │              ]                                                                 │                                                         │
│ │       self = <markdown_it.parser_inline.ParserInline object at 0x7f461c99bb60> │                                                         │
│ │      state = StateInline(pos=[17 of 47], token=0)                              │                                                         │
│ ╰────────────────────────────────────────────────────────────────────────────────╯                                                         │
│                                                                                                                                            │
│ /home/pmoris/mambaforge/envs/nf-core-3/lib/python3.13/site-packages/markdown_it/rules_inline/linkify.py:17 in linkify                      │
│                                                                                                                                            │
│   14 │   if state.linkLevel > 0:                                                                                                           │
│   15 │   │   return False                                                                                                                  │
│   16 │   if not state.md.linkify:                                                                                                          │
│ ❱ 17 │   │   raise ModuleNotFoundError("Linkify enabled but not installed.")                                                               │
│   18 │                                                                                                                                     │
│   19 │   pos = state.pos                                                                                                                   │
│   20 │   maximum = state.posMax                                                                                                            │
│                                                                                                                                            │
│ ╭─────────────────── locals ────────────────────╮                                                                                          │
│ │ silent = False                                │                                                                                          │
│ │  state = StateInline(pos=[17 of 47], token=0) │                                                                                          │
│ ╰───────────────────────────────────────────────╯                                                                                          │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
ModuleNotFoundError: Linkify enabled but not installed.

System information

nf-core/tools version 3.0.2
Nextflow version 24.04.4 build 5917
Python 3.13.0
Operating System: Fedora Linux 40 (KDE Plasma)
Kernel: Linux 6.11.4-201.fc40.x86_64
Architecture: x86-64

[pmoris]

Found the culprit:

We don't directly install markdown-it-py, but it is a dependency of trogon (through its own dependency on textual).

Looking at the conda-forge recipe for textual (https://github.com/conda-forge/textual-feedstock/blob/main/recipe/meta.yaml), we see markdown-it-py listed, without specifying linkify. Meanwhile, the pyproject.toml file for textual does list it as an extra to install (https://github.com/Textualize/textual/blob/main/pyproject.toml). So it's actually this conda-forge recipe that is broken.

[pmoris]

I've added the missing dependency to the bioconda recipe here: bioconda/bioconda-recipes#51762.

I'll also open a PR for adding it explicitly to the pip requirements file (even though it is technically not necessary as long as it remains specified as a mandatory extra component by textual (see https://github.com/Textualize/textual/blob/22770300252deb28d266fe4ed4766d6e2a2f5dd2/pyproject.toml#L44).

[pmoris]

Also created a PR for the conda-forge recipe: conda-forge/textual-feedstock#150.

If/when it gets added there, it would not strictly be necessary to add linkify-it-py to the nf-core requirements anymore. But for the time being this would fix problems with people installing it through bioconda at least.

[mashehu]

should be fixed now, thanks to the updated conda-forge recipe