From 6fc6da2514c0aa73a6c971f639e52bd4d44590cf Mon Sep 17 00:00:00 2001 From: mashehu Date: Tue, 27 Aug 2024 07:30:15 +0200 Subject: [PATCH 01/45] change pipeline template and tooling to allow `main` as default branch --- .github/PULL_REQUEST_TEMPLATE.md | 2 +- .github/RELEASE_CHECKLIST.md | 6 ++-- .github/workflows/branch.yml | 16 ++++----- .github/workflows/pytest.yml | 2 +- README.md | 8 ++--- nf_core/__main__.py | 5 +-- nf_core/components/create.py | 4 +-- .../pipeline-template/.github/CONTRIBUTING.md | 6 ++-- .../.github/PULL_REQUEST_TEMPLATE.md | 4 +-- .../.github/workflows/awsfulltest.yml | 3 +- .../.github/workflows/branch.yml | 18 +++++----- .../.github/workflows/download_pipeline.yml | 4 ++- nf_core/pipeline-template/README.md | 2 +- .../assets/schema_input.json | 2 +- nf_core/pipelines/create/create.py | 34 ++++++++++++------- nf_core/pipelines/download.py | 2 +- nf_core/pipelines/lint/actions_awsfulltest.py | 2 +- nf_core/pipelines/lint/version_consistency.py | 2 +- nf_core/pipelines/schema.py | 2 +- nf_core/synced_repo.py | 3 +- tests/pipelines/lint/test_actions_awstest.py | 2 +- tests/pipelines/test_lint.py | 2 +- tests/pipelines/test_sync.py | 8 ++--- 23 files changed, 78 insertions(+), 61 deletions(-) diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index 71411be1b9..9dbd7a1f6b 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -6,7 +6,7 @@ These are the most common things requested on pull requests (PRs). Remember that PRs should be made against the dev branch, unless you're preparing a release. -Learn more about contributing: https://github.com/nf-core/tools/tree/master/.github/CONTRIBUTING.md +Learn more about contributing: https://github.com/nf-core/tools/tree/main/.github/CONTRIBUTING.md --> ## PR checklist diff --git a/.github/RELEASE_CHECKLIST.md b/.github/RELEASE_CHECKLIST.md index 9a1905c7a0..f96df0f768 100644 --- a/.github/RELEASE_CHECKLIST.md +++ b/.github/RELEASE_CHECKLIST.md @@ -6,11 +6,11 @@ 4. Check that modules/subworkflows in template are up to date with the latest releases 5. Create a PR to `dev` to bump the version in `CHANGELOG.md` and `setup.py` and change the gitpod container to `nfcore/gitpod:latest`. 6. Make sure all CI tests are passing! -7. Create a PR from `dev` to `master` -8. Make sure all CI tests are passing again (additional tests are run on PRs to `master`) +7. Create a PR from `dev` to `main` +8. Make sure all CI tests are passing again (additional tests are run on PRs to `main`) 9. Request review (2 approvals required) 10. Run `rich-codex` to regenerate docs screengrabs (actions `workflow_dispatch` button) -11. Merge the PR into `master` +11. Merge the PR into `main` 12. Wait for CI tests on the commit to passed 13. (Optional but a good idea) Run a manual sync on `nf-core/testpipeline` and check that CI is passing on the resulting PR. 14. Create a new release copying the `CHANGELOG` for that release into the description section. diff --git a/.github/workflows/branch.yml b/.github/workflows/branch.yml index bbac1cc6ff..a116a36225 100644 --- a/.github/workflows/branch.yml +++ b/.github/workflows/branch.yml @@ -1,15 +1,15 @@ name: nf-core branch protection -# This workflow is triggered on PRs to master branch on the repository -# It fails when someone tries to make a PR against the nf-core `master` branch instead of `dev` +# This workflow is triggered on PRs to main/master branch on the repository +# It fails when someone tries to make a PR against the nf-core `main/master` branch instead of `dev` on: pull_request_target: - branches: [master] + branches: [main, master] jobs: test: runs-on: ubuntu-latest steps: - # PRs to the nf-core repo master branch are only ok if coming from the nf-core repo `dev` or any `patch` branches + # PRs to the nf-core repo main/master branch are only ok if coming from the nf-core repo `dev` or any `patch` branches - name: Check PRs if: github.repository == 'nf-core/tools' run: | @@ -21,7 +21,7 @@ jobs: uses: mshick/add-pr-comment@b8f338c590a895d50bcbfa6c5859251edc8952fc # v2 with: message: | - ## This PR is against the `master` branch :x: + ## This PR is against the `main/master` branch :x: * Do not close this PR * Click _Edit_ and change the `base` to `dev` @@ -31,9 +31,9 @@ jobs: Hi @${{ github.event.pull_request.user.login }}, - It looks like this pull-request is has been made against the [${{github.event.pull_request.head.repo.full_name }}](https://github.com/${{github.event.pull_request.head.repo.full_name }}) `master` branch. - The `master` branch on nf-core repositories should always contain code from the latest release. - Because of this, PRs to `master` are only allowed if they come from the [${{github.event.pull_request.head.repo.full_name }}](https://github.com/${{github.event.pull_request.head.repo.full_name }}) `dev` branch. + It looks like this pull-request is has been made against the [${{github.event.pull_request.head.repo.full_name }}](https://github.com/${{github.event.pull_request.head.repo.full_name }}) `main` branch. + The `main/master` branch on nf-core repositories should always contain code from the latest release. + Because of this, PRs to `main/master` are only allowed if they come from the [${{github.event.pull_request.head.repo.full_name }}](https://github.com/${{github.event.pull_request.head.repo.full_name }}) `dev` branch. You do not need to close this PR, you can change the target branch to `dev` by clicking the _"Edit"_ button at the top of this page. Note that even after this, the test will continue to show as failing until you push a new commit. diff --git a/.github/workflows/pytest.yml b/.github/workflows/pytest.yml index dc88031886..3a3d049494 100644 --- a/.github/workflows/pytest.yml +++ b/.github/workflows/pytest.yml @@ -49,7 +49,7 @@ jobs: steps: - name: Check conditions id: conditions - run: echo "run-tests=${{ github.ref == 'refs/heads/master' || (matrix.runner == 'ubuntu-20.04' && matrix.python-version == '3.8') }}" >> "$GITHUB_OUTPUT" + run: echo "run-tests=${{ github.ref == 'refs/heads/master' || github.ref == 'refs/heads/main' || (matrix.runner == 'ubuntu-20.04' && matrix.python-version == '3.8') }}" >> "$GITHUB_OUTPUT" outputs: python-version: ${{ matrix.python-version }} diff --git a/README.md b/README.md index 58fb708a0d..710efed057 100644 --- a/README.md +++ b/README.md @@ -1,12 +1,12 @@

- - nf-core/tools + + nf-core/tools

-[![Python tests](https://github.com/nf-core/tools/workflows/Python%20tests/badge.svg?branch=master&event=push)](https://github.com/nf-core/tools/actions?query=workflow%3A%22Python+tests%22+branch%3Amaster) -[![codecov](https://codecov.io/gh/nf-core/tools/branch/master/graph/badge.svg)](https://codecov.io/gh/nf-core/tools) +[![Python tests](https://github.com/nf-core/tools/workflows/Python%20tests/badge.svg?branch=main&event=push)](https://github.com/nf-core/tools/actions?query=workflow%3A%22Python+tests%22+branch%3Amain) +[![codecov](https://codecov.io/gh/nf-core/tools/branch/main/graph/badge.svg)](https://codecov.io/gh/nf-core/tools) [![code style: prettier](https://img.shields.io/badge/code%20style-prettier-ff69b4.svg)](https://github.com/prettier/prettier) [![code style: Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/charliermarsh/ruff/main/assets/badge/v1.json)](https://github.com/charliermarsh/ruff) diff --git a/nf_core/__main__.py b/nf_core/__main__.py index 0efea13ec9..212278aa2a 100644 --- a/nf_core/__main__.py +++ b/nf_core/__main__.py @@ -4,6 +4,7 @@ import logging import os import sys +from pathlib import Path import rich import rich.console @@ -286,7 +287,7 @@ def command_pipelines_create(ctx, name, description, author, version, force, out @click.option( "--release", is_flag=True, - default=os.path.basename(os.path.dirname(os.environ.get("GITHUB_REF", "").strip(" '\""))) == "master" + default=Path(os.environ.get("GITHUB_REF", "").strip(" '\"")).parent.name in ["master", "main"] and os.environ.get("GITHUB_REPOSITORY", "").startswith("nf-core/") and not os.environ.get("GITHUB_REPOSITORY", "") == "nf-core/tools", help="Execute additional checks for release-ready workflows.", @@ -2139,7 +2140,7 @@ def command_download( @click.option( "--release", is_flag=True, - default=os.path.basename(os.path.dirname(os.environ.get("GITHUB_REF", "").strip(" '\""))) == "master" + default=Path(os.environ.get("GITHUB_REF", "").strip(" '\"")).parent.name in ["master", "main"] and os.environ.get("GITHUB_REPOSITORY", "").startswith("nf-core/") and not os.environ.get("GITHUB_REPOSITORY", "") == "nf-core/tools", help="Execute additional checks for release-ready workflows.", diff --git a/nf_core/components/create.py b/nf_core/components/create.py index c71b128415..6b3b9dad2a 100644 --- a/nf_core/components/create.py +++ b/nf_core/components/create.py @@ -244,7 +244,7 @@ def _get_module_structure_components(self): if self.process_label is None: log.info( "Provide an appropriate resource label for the process, taken from the " - "[link=https://github.com/nf-core/tools/blob/master/nf_core/pipeline-template/conf/base.config#L29]nf-core pipeline template[/link].\n" + "[link=https://github.com/nf-core/tools/blob/main/nf_core/pipeline-template/conf/base.config#L29]nf-core pipeline template[/link].\n" "For example: {}".format(", ".join(process_label_defaults)) ) while self.process_label is None: @@ -260,7 +260,7 @@ def _get_module_structure_components(self): "Where applicable all sample-specific information e.g. 'id', 'single_end', 'read_group' " "MUST be provided as an input via a Groovy Map called 'meta'. " "This information may [italic]not[/] be required in some instances, for example " - "[link=https://github.com/nf-core/modules/blob/master/modules/nf-core/bwa/index/main.nf]indexing reference genome files[/link]." + "[link=https://github.com/nf-core/modules/blob/main/modules/nf-core/bwa/index/main.nf]indexing reference genome files[/link]." ) while self.has_meta is None: self.has_meta = rich.prompt.Confirm.ask( diff --git a/nf_core/pipeline-template/.github/CONTRIBUTING.md b/nf_core/pipeline-template/.github/CONTRIBUTING.md index 5a58501bb2..d0efeb92c8 100644 --- a/nf_core/pipeline-template/.github/CONTRIBUTING.md +++ b/nf_core/pipeline-template/.github/CONTRIBUTING.md @@ -59,9 +59,9 @@ These tests are run both with the latest available version of `Nextflow` and als :warning: Only in the unlikely and regretful event of a release happening with a bug. -- On your own fork, make a new branch `patch` based on `upstream/master`. +- On your own fork, make a new branch `patch` based on `upstream/main` or `upstream/master`. - Fix the bug, and bump version (X.Y.Z+1). -- A PR should be made on `master` from patch to directly this particular bug. +- A PR should be made on `main`/`master` from patch to directly this particular bug. {% if is_nfcore -%} @@ -100,7 +100,7 @@ Once there, use `nf-core pipelines schema build` to add to `nextflow_schema.json ### Default processes resource requirements -Sensible defaults for process resource requirements (CPUs / memory / time) for a process should be defined in `conf/base.config`. These should generally be specified generic with `withLabel:` selectors so they can be shared across multiple processes/steps of the pipeline. A nf-core standard set of labels that should be followed where possible can be seen in the [nf-core pipeline template](https://github.com/nf-core/tools/blob/master/nf_core/pipeline-template/conf/base.config), which has the default process as a single core-process, and then different levels of multi-core configurations for increasingly large memory requirements defined with standardised labels. +Sensible defaults for process resource requirements (CPUs / memory / time) for a process should be defined in `conf/base.config`. These should generally be specified generic with `withLabel:` selectors so they can be shared across multiple processes/steps of the pipeline. A nf-core standard set of labels that should be followed where possible can be seen in the [nf-core pipeline template](https://github.com/nf-core/tools/blob/main/nf_core/pipeline-template/conf/base.config), which has the default process as a single core-process, and then different levels of multi-core configurations for increasingly large memory requirements defined with standardised labels. The process resources can be passed on to the tool dynamically within the process with the `${task.cpus}` and `${task.memory}` variables in the `script:` block. diff --git a/nf_core/pipeline-template/.github/PULL_REQUEST_TEMPLATE.md b/nf_core/pipeline-template/.github/PULL_REQUEST_TEMPLATE.md index dee23ccab1..88d5c7efa6 100644 --- a/nf_core/pipeline-template/.github/PULL_REQUEST_TEMPLATE.md +++ b/nf_core/pipeline-template/.github/PULL_REQUEST_TEMPLATE.md @@ -8,14 +8,14 @@ These are the most common things requested on pull requests (PRs). Remember that PRs should be made against the dev branch, unless you're preparing a pipeline release. -Learn more about contributing: [CONTRIBUTING.md](https://github.com/{{ name }}/tree/master/.github/CONTRIBUTING.md) +Learn more about contributing: [CONTRIBUTING.md](https://github.com/{{ name }}/tree/{{ default_branch }}/.github/CONTRIBUTING.md) --> ## PR checklist - [ ] This comment contains a description of changes (with reason). - [ ] If you've fixed a bug or added code that should be tested, add tests! -- [ ] If you've added a new tool - have you followed the pipeline conventions in the [contribution docs](https://github.com/{{ name }}/tree/master/.github/CONTRIBUTING.md) +- [ ] If you've added a new tool - have you followed the pipeline conventions in the [contribution docs](https://github.com/{{ name }}/tree/{{ default_branch }}/.github/CONTRIBUTING.md) {%- if is_nfcore %} - [ ] If necessary, also make a PR on the {{ name }} _branch_ on the [nf-core/test-datasets](https://github.com/nf-core/test-datasets) repository. {%- endif %} diff --git a/nf_core/pipeline-template/.github/workflows/awsfulltest.yml b/nf_core/pipeline-template/.github/workflows/awsfulltest.yml index dc0450be43..922e535e27 100644 --- a/nf_core/pipeline-template/.github/workflows/awsfulltest.yml +++ b/nf_core/pipeline-template/.github/workflows/awsfulltest.yml @@ -1,11 +1,12 @@ name: nf-core AWS full size tests -# This workflow is triggered on PRs opened against the master branch. +# This workflow is triggered on PRs opened against the main/master branch. # It can be additionally triggered manually with GitHub actions workflow dispatch button. # It runs the -profile 'test_full' on AWS batch on: pull_request: branches: + - main - master workflow_dispatch: pull_request_review: diff --git a/nf_core/pipeline-template/.github/workflows/branch.yml b/nf_core/pipeline-template/.github/workflows/branch.yml index df1a627b15..e0ae1aa8ab 100644 --- a/nf_core/pipeline-template/.github/workflows/branch.yml +++ b/nf_core/pipeline-template/.github/workflows/branch.yml @@ -1,15 +1,17 @@ name: nf-core branch protection -# This workflow is triggered on PRs to master branch on the repository -# It fails when someone tries to make a PR against the nf-core `master` branch instead of `dev` +# This workflow is triggered on PRs to main/master branch on the repository +# It fails when someone tries to make a PR against the nf-core `main/master` branch instead of `dev` on: pull_request_target: - branches: [master] + branches: + - main + - master jobs: test: runs-on: ubuntu-latest steps: - # PRs to the nf-core repo master branch are only ok if coming from the nf-core repo `dev` or any `patch` branches + # PRs to the nf-core repo main/master branch are only ok if coming from the nf-core repo `dev` or any `patch` branches - name: Check PRs if: github.repository == '{{ name }}' run: | @@ -22,7 +24,7 @@ jobs: uses: mshick/add-pr-comment@b8f338c590a895d50bcbfa6c5859251edc8952fc # v2 with: message: | - ## This PR is against the `master` branch :x: + ## This PR is against the `main/master` branch :x: * Do not close this PR * Click _Edit_ and change the `base` to `dev` @@ -32,9 +34,9 @@ jobs: Hi @${{ github.event.pull_request.user.login }}, - It looks like this pull-request is has been made against the [${{github.event.pull_request.head.repo.full_name }}](https://github.com/${{github.event.pull_request.head.repo.full_name }}) `master` branch. - The `master` branch on nf-core repositories should always contain code from the latest release. - Because of this, PRs to `master` are only allowed if they come from the [${{github.event.pull_request.head.repo.full_name }}](https://github.com/${{github.event.pull_request.head.repo.full_name }}) `dev` branch. + It looks like this pull-request is has been made against the [${{github.event.pull_request.head.repo.full_name }}](https://github.com/${{github.event.pull_request.head.repo.full_name }}) `main/master` branch. + The `main/master` branch on nf-core repositories should always contain code from the latest release. + Because of this, PRs to `main/master` are only allowed if they come from the [${{github.event.pull_request.head.repo.full_name }}](https://github.com/${{github.event.pull_request.head.repo.full_name }}) `dev` branch. You do not need to close this PR, you can change the target branch to `dev` by clicking the _"Edit"_ button at the top of this page. Note that even after this, the test will continue to show as failing until you push a new commit. diff --git a/nf_core/pipeline-template/.github/workflows/download_pipeline.yml b/nf_core/pipeline-template/.github/workflows/download_pipeline.yml index 99a42d86d2..93765d0306 100644 --- a/nf_core/pipeline-template/.github/workflows/download_pipeline.yml +++ b/nf_core/pipeline-template/.github/workflows/download_pipeline.yml @@ -2,7 +2,7 @@ name: Test successful pipeline download with 'nf-core pipelines download' # Run the workflow when: # - dispatched manually -# - when a PR is opened or reopened to master branch +# - when a PR is opened or reopened to main/master branch # - the head branch of the pull request is updated, i.e. if fixes for a release are pushed last minute to dev. on: workflow_dispatch: @@ -17,9 +17,11 @@ on: - edited - synchronize branches: + - main - master pull_request_target: branches: + - main - master env: diff --git a/nf_core/pipeline-template/README.md b/nf_core/pipeline-template/README.md index 7718d2e5f8..8074da2f39 100644 --- a/nf_core/pipeline-template/README.md +++ b/nf_core/pipeline-template/README.md @@ -124,7 +124,7 @@ An extensive list of references for the tools used by the pipeline can be found You can cite the `nf-core` publication as follows: {% else -%} -This pipeline uses code and infrastructure developed and maintained by the [nf-core](https://nf-co.re) community, reused here under the [MIT license](https://github.com/nf-core/tools/blob/master/LICENSE). +This pipeline uses code and infrastructure developed and maintained by the [nf-core](https://nf-co.re) community, reused here under the [MIT license](https://github.com/nf-core/tools/blob/{{ default_branch }}/LICENSE). {% endif -%} diff --git a/nf_core/pipeline-template/assets/schema_input.json b/nf_core/pipeline-template/assets/schema_input.json index e76b95fa99..6271f572f2 100644 --- a/nf_core/pipeline-template/assets/schema_input.json +++ b/nf_core/pipeline-template/assets/schema_input.json @@ -1,6 +1,6 @@ { "$schema": "http://json-schema.org/draft-07/schema", - "$id": "https://raw.githubusercontent.com/{{ name }}/master/assets/schema_input.json", + "$id": "https://raw.githubusercontent.com/{{ name }}/{{ default_branch }}/assets/schema_input.json", "title": "{{ name }} pipeline - params.input schema", "description": "Schema for the file provided with params.input", "type": "array", diff --git a/nf_core/pipelines/create/create.py b/nf_core/pipelines/create/create.py index 05b04a5422..71265a7e25 100644 --- a/nf_core/pipelines/create/create.py +++ b/nf_core/pipelines/create/create.py @@ -86,6 +86,10 @@ def __init__( if self.config.outdir is None: self.config.outdir = str(Path.cwd()) + + # Get the default branch name from the Git configuration + self.get_default_branch() + self.jinja_params, self.skip_areas = self.obtain_jinja_params_dict( self.config.skip_features or [], str(self.config.outdir) ) @@ -230,6 +234,7 @@ def obtain_jinja_params_dict( jinja_params["name_docker"] = jinja_params["name"].replace(jinja_params["org"], jinja_params["prefix_nodash"]) jinja_params["logo_light"] = f"{jinja_params['name_noslash']}_logo_light.png" jinja_params["logo_dark"] = f"{jinja_params['name_noslash']}_logo_dark.png" + jinja_params["default_branch"] = self.default_branch if config_yml is not None: if ( hasattr(config_yml, "lint") @@ -251,6 +256,7 @@ def obtain_jinja_params_dict( def init_pipeline(self): """Creates the nf-core pipeline.""" + # Make the new pipeline self.render_template() @@ -417,20 +423,17 @@ def make_pipeline_logo(self): force=bool(self.force), ) - def git_init_pipeline(self) -> None: - """Initialises the new pipeline as a Git repository and submits first commit. - - Raises: - UserWarning: if Git default branch is set to 'dev' or 'TEMPLATE'. - """ - default_branch: Optional[str] = self.default_branch + def get_default_branch(self) -> None: + """Gets the default branch name from the Git configuration.""" try: - default_branch = default_branch or str(git.config.GitConfigParser().get_value("init", "defaultBranch")) + self.default_branch = ( + str(git.config.GitConfigParser().get_value("init", "defaultBranch")) or "main" + ) # default to main except configparser.Error: log.debug("Could not read init.defaultBranch") - if default_branch in ["dev", "TEMPLATE"]: + if self.default_branch in ["dev", "TEMPLATE"]: raise UserWarning( - f"Your Git defaultBranch '{default_branch}' is incompatible with nf-core.\n" + f"Your Git defaultBranch '{self.default_branch}' is incompatible with nf-core.\n" "'dev' and 'TEMPLATE' can not be used as default branch name.\n" "Set the default branch name with " "[white on grey23] git config --global init.defaultBranch [/]\n" @@ -438,12 +441,19 @@ def git_init_pipeline(self) -> None: "Pipeline git repository will not be initialised." ) + def git_init_pipeline(self) -> None: + """Initialises the new pipeline as a Git repository and submits first commit. + + Raises: + UserWarning: if Git default branch is set to 'dev' or 'TEMPLATE'. + """ + log.info("Initialising local pipeline git repository") repo = git.Repo.init(self.outdir) repo.git.add(A=True) repo.index.commit(f"initial template build from nf-core/tools, version {nf_core.__version__}") - if default_branch: - repo.active_branch.rename(default_branch) + if self.default_branch: + repo.active_branch.rename(self.default_branch) try: repo.git.branch("TEMPLATE") repo.git.branch("dev") diff --git a/nf_core/pipelines/download.py b/nf_core/pipelines/download.py index 97453b127e..d153188ffa 100644 --- a/nf_core/pipelines/download.py +++ b/nf_core/pipelines/download.py @@ -1672,7 +1672,7 @@ def tidy_tags_and_branches(self): for tag in tags_to_remove: self.repo.delete_tag(tag) - # switch to a revision that should be kept, because deleting heads fails, if they are checked out (e.g. "master") + # switch to a revision that should be kept, because deleting heads fails, if they are checked out (e.g. "main") self.checkout(self.revision[0]) # delete unwanted heads/branches from repository diff --git a/nf_core/pipelines/lint/actions_awsfulltest.py b/nf_core/pipelines/lint/actions_awsfulltest.py index 7ea167f6c9..2fa7e9082c 100644 --- a/nf_core/pipelines/lint/actions_awsfulltest.py +++ b/nf_core/pipelines/lint/actions_awsfulltest.py @@ -42,7 +42,7 @@ def actions_awsfulltest(self) -> Dict[str, List[str]]: # Check that the action is only turned on for published releases try: - if wf[True]["pull_request"]["branches"] != ["master"]: + if wf[True]["pull_request"]["branches"] != ["master", "main"]: raise AssertionError() if wf[True]["pull_request_review"]["types"] != ["submitted"]: raise AssertionError() diff --git a/nf_core/pipelines/lint/version_consistency.py b/nf_core/pipelines/lint/version_consistency.py index 5fe24ed723..2f9cead83c 100644 --- a/nf_core/pipelines/lint/version_consistency.py +++ b/nf_core/pipelines/lint/version_consistency.py @@ -5,7 +5,7 @@ def version_consistency(self): """Pipeline and container version number consistency. .. note:: This test only runs when the ``--release`` flag is set for ``nf-core pipelines lint``, - or ``$GITHUB_REF`` is equal to ``master``. + or ``$GITHUB_REF`` is equal to ``main``. This lint fetches the pipeline version number from three possible locations: diff --git a/nf_core/pipelines/schema.py b/nf_core/pipelines/schema.py index 7f562bff38..8c9437c7f5 100644 --- a/nf_core/pipelines/schema.py +++ b/nf_core/pipelines/schema.py @@ -427,7 +427,7 @@ def validate_schema_title_description(self, schema=None): if "title" not in self.schema: raise AssertionError("Schema missing top-level `title` attribute") # Validate that id, title and description match the pipeline manifest - id_attr = "https://raw.githubusercontent.com/{}/master/nextflow_schema.json".format( + id_attr = "https://raw.githubusercontent.com/{}/main/nextflow_schema.json".format( self.pipeline_manifest["name"].strip("\"'") ) if self.schema["$id"] != id_attr: diff --git a/nf_core/synced_repo.py b/nf_core/synced_repo.py index e2a76ccaeb..bccd26571a 100644 --- a/nf_core/synced_repo.py +++ b/nf_core/synced_repo.py @@ -10,6 +10,7 @@ from git.exc import GitCommandError from nf_core.components.components_utils import ( + NF_CORE_MODULES_DEFAULT_BRANCH, NF_CORE_MODULES_NAME, NF_CORE_MODULES_REMOTE, ) @@ -186,7 +187,7 @@ def setup_branch(self, branch): if branch is None: # Don't bother fetching default branch if we're using nf-core if self.remote_url == NF_CORE_MODULES_REMOTE: - self.branch = "master" + self.branch = NF_CORE_MODULES_DEFAULT_BRANCH else: self.branch = self.get_default_branch() else: diff --git a/tests/pipelines/lint/test_actions_awstest.py b/tests/pipelines/lint/test_actions_awstest.py index 51b55cb867..01dc9f6168 100644 --- a/tests/pipelines/lint/test_actions_awstest.py +++ b/tests/pipelines/lint/test_actions_awstest.py @@ -24,7 +24,7 @@ def test_actions_awstest_fail(self): new_pipeline = self._make_pipeline_copy() with open(Path(new_pipeline, ".github", "workflows", "awstest.yml")) as fh: awstest_yml = yaml.safe_load(fh) - awstest_yml[True]["push"] = ["master"] + awstest_yml[True]["push"] = ["main"] with open(Path(new_pipeline, ".github", "workflows", "awstest.yml"), "w") as fh: yaml.dump(awstest_yml, fh) diff --git a/tests/pipelines/test_lint.py b/tests/pipelines/test_lint.py index 9ca29d249f..543cc5ae04 100644 --- a/tests/pipelines/test_lint.py +++ b/tests/pipelines/test_lint.py @@ -25,7 +25,7 @@ def setUp(self) -> None: ########################## class TestPipelinesLint(TestLint): def test_run_linting_function(self): - """Run the master run_linting() function in lint.py + """Run the run_linting() function in lint.py We don't really check any of this code as it's just a series of function calls and we're testing each of those individually. This is mostly to check for syntax errors.""" diff --git a/tests/pipelines/test_sync.py b/tests/pipelines/test_sync.py index ffbe75510b..66b6b9623c 100644 --- a/tests/pipelines/test_sync.py +++ b/tests/pipelines/test_sync.py @@ -43,14 +43,14 @@ def mocked_requests_get(url) -> MockResponse: { "state": "closed", "head": {"ref": "nf-core-template-merge-2"}, - "base": {"ref": "master"}, + "base": {"ref": "main"}, "html_url": "pr_url", } ] + [ { "state": "open", "head": {"ref": f"nf-core-template-merge-{branch_no}"}, - "base": {"ref": "master"}, + "base": {"ref": "main"}, "html_url": "pr_url", } for branch_no in range(3, 7) @@ -343,7 +343,7 @@ def test_close_open_pr(self, mock_patch, mock_post) -> None: pr: Dict[str, Union[str, Dict[str, str]]] = { "state": "open", "head": {"ref": "nf-core-template-merge-3"}, - "base": {"ref": "master"}, + "base": {"ref": "main"}, "html_url": "pr_html_url", "url": "url_to_update_pr", "comments_url": "pr_comments_url", @@ -366,7 +366,7 @@ def test_close_open_pr_fail(self, mock_patch, mock_post): pr = { "state": "open", "head": {"ref": "nf-core-template-merge-3"}, - "base": {"ref": "master"}, + "base": {"ref": "main"}, "html_url": "pr_html_url", "url": "bad_url_to_update_pr", "comments_url": "pr_comments_url", From 889e59e15f2d77377ebfd7912ee5205bd853a403 Mon Sep 17 00:00:00 2001 From: mashehu Date: Tue, 27 Aug 2024 08:41:53 +0200 Subject: [PATCH 02/45] handle missing self.default_branch --- nf_core/pipelines/create/create.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/nf_core/pipelines/create/create.py b/nf_core/pipelines/create/create.py index 71265a7e25..2a227329fb 100644 --- a/nf_core/pipelines/create/create.py +++ b/nf_core/pipelines/create/create.py @@ -431,7 +431,7 @@ def get_default_branch(self) -> None: ) # default to main except configparser.Error: log.debug("Could not read init.defaultBranch") - if self.default_branch in ["dev", "TEMPLATE"]: + if self.default_branch is not None and self.default_branch in ["dev", "TEMPLATE"]: raise UserWarning( f"Your Git defaultBranch '{self.default_branch}' is incompatible with nf-core.\n" "'dev' and 'TEMPLATE' can not be used as default branch name.\n" From 5b3dabd588f84f2e2dc111aa639e07a3ebfd3c87 Mon Sep 17 00:00:00 2001 From: mashehu Date: Tue, 27 Aug 2024 08:47:44 +0200 Subject: [PATCH 03/45] fix order of initialization --- nf_core/pipelines/create/create.py | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/nf_core/pipelines/create/create.py b/nf_core/pipelines/create/create.py index 2a227329fb..8fbb049088 100644 --- a/nf_core/pipelines/create/create.py +++ b/nf_core/pipelines/create/create.py @@ -84,6 +84,11 @@ def __init__( # Read features yaml file self.template_features_yml = load_features_yaml() + # Set fields used by the class methods + self.no_git = no_git + self.default_branch = default_branch + self.is_interactive = is_interactive + if self.config.outdir is None: self.config.outdir = str(Path.cwd()) @@ -108,11 +113,6 @@ def __init__( # Set convenience variables self.name = self.config.name - - # Set fields used by the class methods - self.no_git = no_git - self.default_branch = default_branch - self.is_interactive = is_interactive self.force = self.config.force if self.config.outdir == ".": From da915195f6450f6870eec2b2dbccf1dbd7f2206d Mon Sep 17 00:00:00 2001 From: mashehu Date: Tue, 27 Aug 2024 09:00:05 +0200 Subject: [PATCH 04/45] fix schema check --- nf_core/pipeline-template/nextflow_schema.json | 2 +- nf_core/pipelines/schema.py | 12 ++++++------ 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/nf_core/pipeline-template/nextflow_schema.json b/nf_core/pipeline-template/nextflow_schema.json index 4a376330bd..819dd7149e 100644 --- a/nf_core/pipeline-template/nextflow_schema.json +++ b/nf_core/pipeline-template/nextflow_schema.json @@ -1,6 +1,6 @@ { "$schema": "http://json-schema.org/draft-07/schema", - "$id": "https://raw.githubusercontent.com/{{ name }}/master/nextflow_schema.json", + "$id": "https://raw.githubusercontent.com/{{ name }}/{{ default_branch }}/nextflow_schema.json", "title": "{{ name }} pipeline parameters", "description": "{{ description }}", "type": "object", diff --git a/nf_core/pipelines/schema.py b/nf_core/pipelines/schema.py index 8c9437c7f5..1a75029f3d 100644 --- a/nf_core/pipelines/schema.py +++ b/nf_core/pipelines/schema.py @@ -402,7 +402,7 @@ def validate_schema(self, schema=None): def validate_schema_title_description(self, schema=None): """ Extra validation command for linting. - Checks that the schema "$id", "title" and "description" attributes match the piipeline config. + Checks that the schema "$id", "title" and "description" attributes match the pipeline config. """ if schema is None: schema = self.schema @@ -427,11 +427,11 @@ def validate_schema_title_description(self, schema=None): if "title" not in self.schema: raise AssertionError("Schema missing top-level `title` attribute") # Validate that id, title and description match the pipeline manifest - id_attr = "https://raw.githubusercontent.com/{}/main/nextflow_schema.json".format( - self.pipeline_manifest["name"].strip("\"'") - ) - if self.schema["$id"] != id_attr: - raise AssertionError(f"Schema `$id` should be `{id_attr}`\n Found `{self.schema['$id']}`") + id_attr = f"https://raw.githubusercontent.com/{self.pipeline_manifest["name"].strip("\"'")}/main/nextflow_schema.json" + if self.schema["$id"] not in [id_attr, id_attr.replace("/main/", "/master/")]: + raise AssertionError( + f"Schema `$id` should be `{id_attr}` or {id_attr.replace("/main/", "/master/")}. \n Found `{self.schema['$id']}`" + ) title_attr = "{} pipeline parameters".format(self.pipeline_manifest["name"].strip("\"'")) if self.schema["title"] != title_attr: From ed0fb1b4bb0b7c21ea61d6ed84197b692b02905a Mon Sep 17 00:00:00 2001 From: mashehu Date: Tue, 27 Aug 2024 09:08:51 +0200 Subject: [PATCH 05/45] set default value for default branch --- nf_core/pipelines/create/create.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/nf_core/pipelines/create/create.py b/nf_core/pipelines/create/create.py index 8fbb049088..c102441c3c 100644 --- a/nf_core/pipelines/create/create.py +++ b/nf_core/pipelines/create/create.py @@ -56,7 +56,7 @@ def __init__( template_config: Optional[Union[CreateConfig, str, Path]] = None, organisation: str = "nf-core", from_config_file: bool = False, - default_branch: Optional[str] = None, + default_branch: str = "main", is_interactive: bool = False, ) -> None: if isinstance(template_config, CreateConfig): @@ -431,7 +431,7 @@ def get_default_branch(self) -> None: ) # default to main except configparser.Error: log.debug("Could not read init.defaultBranch") - if self.default_branch is not None and self.default_branch in ["dev", "TEMPLATE"]: + if self.default_branch in ["dev", "TEMPLATE"]: raise UserWarning( f"Your Git defaultBranch '{self.default_branch}' is incompatible with nf-core.\n" "'dev' and 'TEMPLATE' can not be used as default branch name.\n" From 9d2a3abf8af09b172b5065e40d7618f0dbfbecca Mon Sep 17 00:00:00 2001 From: mashehu Date: Tue, 27 Aug 2024 09:12:38 +0200 Subject: [PATCH 06/45] fix order in lint --- nf_core/pipelines/lint/actions_awsfulltest.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/nf_core/pipelines/lint/actions_awsfulltest.py b/nf_core/pipelines/lint/actions_awsfulltest.py index 2fa7e9082c..080ae3583e 100644 --- a/nf_core/pipelines/lint/actions_awsfulltest.py +++ b/nf_core/pipelines/lint/actions_awsfulltest.py @@ -42,7 +42,7 @@ def actions_awsfulltest(self) -> Dict[str, List[str]]: # Check that the action is only turned on for published releases try: - if wf[True]["pull_request"]["branches"] != ["master", "main"]: + if wf[True]["pull_request"]["branches"] != ["main", "master"]: raise AssertionError() if wf[True]["pull_request_review"]["types"] != ["submitted"]: raise AssertionError() From 20545509ff899d82683baca5a7395a84febb9c73 Mon Sep 17 00:00:00 2001 From: mashehu Date: Tue, 27 Aug 2024 10:07:47 +0200 Subject: [PATCH 07/45] fix quotes --- nf_core/pipelines/schema.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/nf_core/pipelines/schema.py b/nf_core/pipelines/schema.py index 1a75029f3d..520fbab949 100644 --- a/nf_core/pipelines/schema.py +++ b/nf_core/pipelines/schema.py @@ -427,7 +427,7 @@ def validate_schema_title_description(self, schema=None): if "title" not in self.schema: raise AssertionError("Schema missing top-level `title` attribute") # Validate that id, title and description match the pipeline manifest - id_attr = f"https://raw.githubusercontent.com/{self.pipeline_manifest["name"].strip("\"'")}/main/nextflow_schema.json" + id_attr = f"https://raw.githubusercontent.com/{self.pipeline_manifest['name'].strip('\"\'')}/main/nextflow_schema.json" if self.schema["$id"] not in [id_attr, id_attr.replace("/main/", "/master/")]: raise AssertionError( f"Schema `$id` should be `{id_attr}` or {id_attr.replace("/main/", "/master/")}. \n Found `{self.schema['$id']}`" From 19ec18810aafff259a85bb848520e9671c919672 Mon Sep 17 00:00:00 2001 From: mashehu Date: Tue, 27 Aug 2024 10:30:07 +0200 Subject: [PATCH 08/45] revert to format --- nf_core/pipelines/schema.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/nf_core/pipelines/schema.py b/nf_core/pipelines/schema.py index 520fbab949..b0feab0911 100644 --- a/nf_core/pipelines/schema.py +++ b/nf_core/pipelines/schema.py @@ -427,7 +427,8 @@ def validate_schema_title_description(self, schema=None): if "title" not in self.schema: raise AssertionError("Schema missing top-level `title` attribute") # Validate that id, title and description match the pipeline manifest - id_attr = f"https://raw.githubusercontent.com/{self.pipeline_manifest['name'].strip('\"\'')}/main/nextflow_schema.json" + pipeline_name = self.pipeline_manifest["name"].strip("\"'") + id_attr = f"https://raw.githubusercontent.com/{pipeline_name}/main/nextflow_schema.json" if self.schema["$id"] not in [id_attr, id_attr.replace("/main/", "/master/")]: raise AssertionError( f"Schema `$id` should be `{id_attr}` or {id_attr.replace("/main/", "/master/")}. \n Found `{self.schema['$id']}`" From fa2750a6c3fbc2ce3ccaf1f97e862fbbb301940e Mon Sep 17 00:00:00 2001 From: mashehu Date: Tue, 27 Aug 2024 15:18:27 +0200 Subject: [PATCH 09/45] use format instead of f-string --- nf_core/pipelines/schema.py | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/nf_core/pipelines/schema.py b/nf_core/pipelines/schema.py index b0feab0911..b27a2d9189 100644 --- a/nf_core/pipelines/schema.py +++ b/nf_core/pipelines/schema.py @@ -427,8 +427,9 @@ def validate_schema_title_description(self, schema=None): if "title" not in self.schema: raise AssertionError("Schema missing top-level `title` attribute") # Validate that id, title and description match the pipeline manifest - pipeline_name = self.pipeline_manifest["name"].strip("\"'") - id_attr = f"https://raw.githubusercontent.com/{pipeline_name}/main/nextflow_schema.json" + id_attr = "https://raw.githubusercontent.com/{}/main/nextflow_schema.json".format( + self.pipeline_manifest["name"].strip("\"'") + ) if self.schema["$id"] not in [id_attr, id_attr.replace("/main/", "/master/")]: raise AssertionError( f"Schema `$id` should be `{id_attr}` or {id_attr.replace("/main/", "/master/")}. \n Found `{self.schema['$id']}`" From 805620b5dee7185dd9e5e4f3a409208fdf66c03a Mon Sep 17 00:00:00 2001 From: Edmund Miller Date: Mon, 25 Nov 2024 11:09:22 -0600 Subject: [PATCH 10/45] fix(#3297): Update warning message for pytest-workflow --- nf_core/subworkflows/lint/subworkflow_tests.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/nf_core/subworkflows/lint/subworkflow_tests.py b/nf_core/subworkflows/lint/subworkflow_tests.py index 7ca825f04f..8e9e62430a 100644 --- a/nf_core/subworkflows/lint/subworkflow_tests.py +++ b/nf_core/subworkflows/lint/subworkflow_tests.py @@ -50,7 +50,7 @@ def subworkflow_tests(_, subworkflow: NFCoreComponent): subworkflow.warned.append( ( "test_dir_exists", - "nf-test directory is missing", + "Migrate pytest-workflow to nf-test", subworkflow.nftest_testdir, ) ) From abb0fa2f4bee2123559583f27c2855c52588d3be Mon Sep 17 00:00:00 2001 From: Robert Syme Date: Fri, 29 Nov 2024 14:34:37 -0500 Subject: [PATCH 11/45] Remove toList() channel operation from inside onComplete block This PR resolves an important bug in the nf-core template, whereby all workflows will hang if the --email parameter is supplied. The onComplete block will hang if there are any (queue) channel operations inside the block. All values in the onComplete block must be resolved to single values or value channels _before_ the onComplete block starts. The async channels are not available inside onComplete, so calling the toList() operation will hang forever as the async queue channel will never be completed. --- CHANGELOG.md | 1 + .../local/utils_nfcore_pipeline_pipeline/main.nf | 6 +++++- 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index f418dc0af0..81cb62ede1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -49,6 +49,7 @@ - Update codecov/codecov-action action to v5 ([#3283](https://github.com/nf-core/tools/pull/3283)) - Update python:3.12-slim Docker digest to 2a6386a ([#3284](https://github.com/nf-core/tools/pull/3284)) - Update pre-commit hook astral-sh/ruff-pre-commit to v0.8.0 ([#3299](https://github.com/nf-core/tools/pull/3299)) +- Remove toList() channel operation from inside onComplete block ([#3304](https://github.com/nf-core/tools/pull/3304)) ## [v3.0.2 - Titanium Tapir Patch](https://github.com/nf-core/tools/releases/tag/3.0.2) - [2024-10-11] diff --git a/nf_core/pipeline-template/subworkflows/local/utils_nfcore_pipeline_pipeline/main.nf b/nf_core/pipeline-template/subworkflows/local/utils_nfcore_pipeline_pipeline/main.nf index be5776b836..06692f1dc9 100644 --- a/nf_core/pipeline-template/subworkflows/local/utils_nfcore_pipeline_pipeline/main.nf +++ b/nf_core/pipeline-template/subworkflows/local/utils_nfcore_pipeline_pipeline/main.nf @@ -140,6 +140,10 @@ workflow PIPELINE_COMPLETION { summary_params = [:] {%- endif %} + {%- if multiqc %} + def multiqc_reports = multiqc_report.toList() + {%- endif %} + // // Completion email and summary // @@ -153,7 +157,7 @@ workflow PIPELINE_COMPLETION { plaintext_email, outdir, monochrome_logs, - {% if multiqc %}multiqc_report.toList(){% else %}[]{% endif %} + {% if multiqc %}multiqc_reports.getVal(),{% else %}[]{% endif %} ) } {%- endif %} From bd9608d688f722d17398d8a7fe1d5dfeaeed02be Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Matthias=20H=C3=B6rtenhuber?= Date: Tue, 3 Dec 2024 13:38:33 +0100 Subject: [PATCH 12/45] Update nf_core/components/create.py --- nf_core/components/create.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/nf_core/components/create.py b/nf_core/components/create.py index 6b3b9dad2a..4be165d3db 100644 --- a/nf_core/components/create.py +++ b/nf_core/components/create.py @@ -260,7 +260,7 @@ def _get_module_structure_components(self): "Where applicable all sample-specific information e.g. 'id', 'single_end', 'read_group' " "MUST be provided as an input via a Groovy Map called 'meta'. " "This information may [italic]not[/] be required in some instances, for example " - "[link=https://github.com/nf-core/modules/blob/main/modules/nf-core/bwa/index/main.nf]indexing reference genome files[/link]." + "[link=https://github.com/nf-core/modules/blob/master/modules/nf-core/bwa/index/main.nf]indexing reference genome files[/link]." ) while self.has_meta is None: self.has_meta = rich.prompt.Confirm.ask( From e9ed94ec0f8ece41b406e2f15cd1d7089bc54621 Mon Sep 17 00:00:00 2001 From: mashehu Date: Tue, 3 Dec 2024 14:08:04 +0100 Subject: [PATCH 13/45] set defaultBranch in nextflow config to allow `main` --- nf_core/pipeline-template/nextflow.config | 1 + 1 file changed, 1 insertion(+) diff --git a/nf_core/pipeline-template/nextflow.config b/nf_core/pipeline-template/nextflow.config index 21174bbdc5..000d7cd664 100644 --- a/nf_core/pipeline-template/nextflow.config +++ b/nf_core/pipeline-template/nextflow.config @@ -291,6 +291,7 @@ manifest { homePage = 'https://github.com/{{ name }}' description = """{{ description }}""" mainScript = 'main.nf' + defaultBranch = '{{ default_branch }}' nextflowVersion = '!>=24.04.2' version = '{{ version }}' doi = '' From 00b8ae0da46a7562be763c9efa6b82c696a362ba Mon Sep 17 00:00:00 2001 From: mashehu Date: Tue, 3 Dec 2024 14:18:06 +0100 Subject: [PATCH 14/45] pin java version 21 in CI --- .github/actions/create-lint-wf/action.yml | 1 - .github/workflows/create-lint-wf.yml | 4 +--- .github/workflows/create-test-lint-wf-template.yml | 2 ++ .github/workflows/create-test-wf.yml | 4 +--- nf_core/pipeline-template/.github/workflows/ci.yml | 1 + .../pipeline-template/.github/workflows/download_pipeline.yml | 1 + 6 files changed, 6 insertions(+), 7 deletions(-) diff --git a/.github/actions/create-lint-wf/action.yml b/.github/actions/create-lint-wf/action.yml index 3ef0760513..3ffd960d23 100644 --- a/.github/actions/create-lint-wf/action.yml +++ b/.github/actions/create-lint-wf/action.yml @@ -15,7 +15,6 @@ runs: cd create-lint-wf export NXF_WORK=$(pwd) - # Set up Nextflow - name: Install Nextflow uses: nf-core/setup-nextflow@v2 with: diff --git a/.github/workflows/create-lint-wf.yml b/.github/workflows/create-lint-wf.yml index 37ab71bc3b..78932871e3 100644 --- a/.github/workflows/create-lint-wf.yml +++ b/.github/workflows/create-lint-wf.yml @@ -27,14 +27,12 @@ concurrency: group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }} cancel-in-progress: true -env: - NXF_ANSI_LOG: false - jobs: MakeTestWorkflow: runs-on: ${{ github.event.inputs.runners || github.run_number > 1 && 'ubuntu-latest' || 'self-hosted' }} env: NXF_ANSI_LOG: false + JAVA_HOME: $JAVA_HOME_21_X64 strategy: matrix: NXF_VER: diff --git a/.github/workflows/create-test-lint-wf-template.yml b/.github/workflows/create-test-lint-wf-template.yml index fffa9ffe7a..b0d4c13a4e 100644 --- a/.github/workflows/create-test-lint-wf-template.yml +++ b/.github/workflows/create-test-lint-wf-template.yml @@ -28,6 +28,7 @@ concurrency: env: NXF_ANSI_LOG: false + JAVA_HOME: $JAVA_HOME_21_X64 GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} jobs: @@ -51,6 +52,7 @@ jobs: needs: prepare-matrix env: NXF_ANSI_LOG: false + JAVA_HOME: $JAVA_HOME_21_X64 strategy: matrix: TEMPLATE: ${{ fromJson(needs.prepare-matrix.outputs.all_features) }} diff --git a/.github/workflows/create-test-wf.yml b/.github/workflows/create-test-wf.yml index 93581b9153..f69449dde3 100644 --- a/.github/workflows/create-test-wf.yml +++ b/.github/workflows/create-test-wf.yml @@ -27,15 +27,13 @@ concurrency: group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }} cancel-in-progress: true -env: - NXF_ANSI_LOG: false - jobs: RunTestWorkflow: # use the runner given by the input if it is dispatched manually, run on github if it is a rerun or on self-hosted by default runs-on: ${{ github.event.inputs.runners || github.run_number > 1 && 'ubuntu-latest' || 'self-hosted' }} env: NXF_ANSI_LOG: false + JAVA_HOME: $JAVA_HOME_21_X64 strategy: matrix: NXF_VER: diff --git a/nf_core/pipeline-template/.github/workflows/ci.yml b/nf_core/pipeline-template/.github/workflows/ci.yml index 9db393d9f0..4f005a1bb4 100644 --- a/nf_core/pipeline-template/.github/workflows/ci.yml +++ b/nf_core/pipeline-template/.github/workflows/ci.yml @@ -11,6 +11,7 @@ on: env: NXF_ANSI_LOG: false + JAVA_HOME: $JAVA_HOME_21_X64 NXF_SINGULARITY_CACHEDIR: ${{ github.workspace }}/.singularity NXF_SINGULARITY_LIBRARYDIR: ${{ github.workspace }}/.singularity diff --git a/nf_core/pipeline-template/.github/workflows/download_pipeline.yml b/nf_core/pipeline-template/.github/workflows/download_pipeline.yml index 05397358cf..95ebad9fe4 100644 --- a/nf_core/pipeline-template/.github/workflows/download_pipeline.yml +++ b/nf_core/pipeline-template/.github/workflows/download_pipeline.yml @@ -26,6 +26,7 @@ on: env: NXF_ANSI_LOG: false + JAVA_HOME: $JAVA_HOME_21_X64 jobs: download: From 4b2338ff161a1ddea799c8ab5270e1cefe0f7da0 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?J=C3=BAlia=20Mir=20Pedrol?= Date: Tue, 3 Dec 2024 13:32:17 +0000 Subject: [PATCH 15/45] update modules and subworkflows --- nf_core/pipeline-template/modules.json | 8 +- .../modules/nf-core/fastqc/main.nf | 2 +- .../modules/nf-core/fastqc/meta.yml | 1 + .../nf-core/utils_nextflow_pipeline/main.nf | 2 + .../tests/main.workflow.nf.test | 10 ++- .../nf-core/utils_nfcore_pipeline/main.nf | 89 +++++-------------- .../tests/main.function.nf.test | 52 ----------- .../tests/main.function.nf.test.snap | 30 ------- .../utils_nfschema_plugin/tests/main.nf.test | 4 +- 9 files changed, 40 insertions(+), 158 deletions(-) diff --git a/nf_core/pipeline-template/modules.json b/nf_core/pipeline-template/modules.json index f714eb1d93..397c0cdb03 100644 --- a/nf_core/pipeline-template/modules.json +++ b/nf_core/pipeline-template/modules.json @@ -8,7 +8,7 @@ {%- if fastqc %} "fastqc": { "branch": "master", - "git_sha": "666652151335353eef2fcd58880bcef5bc2928e1", + "git_sha": "dc94b6ee04a05ddb9f7ae050712ff30a13149164", "installed_by": ["modules"] }{% endif %}{%- if multiqc %}{% if fastqc %},{% endif %} "multiqc": { @@ -23,17 +23,17 @@ "nf-core": { "utils_nextflow_pipeline": { "branch": "master", - "git_sha": "3aa0aec1d52d492fe241919f0c6100ebf0074082", + "git_sha": "c2b22d85f30a706a3073387f30380704fcae013b", "installed_by": ["subworkflows"] }, "utils_nfcore_pipeline": { "branch": "master", - "git_sha": "1b6b9a3338d011367137808b49b923515080e3ba", + "git_sha": "85400682a2abac63b09c863c138e91e5df7236b5", "installed_by": ["subworkflows"] }{% if nf_schema %}, "utils_nfschema_plugin": { "branch": "master", - "git_sha": "bbd5a41f4535a8defafe6080e00ea74c45f4f96c", + "git_sha": "2fd2cd6d0e7b273747f32e465fdc6bcc3ae0814e", "installed_by": ["subworkflows"] }{% endif %} } diff --git a/nf_core/pipeline-template/modules/nf-core/fastqc/main.nf b/nf_core/pipeline-template/modules/nf-core/fastqc/main.nf index d8989f4812..752c3a10c6 100644 --- a/nf_core/pipeline-template/modules/nf-core/fastqc/main.nf +++ b/nf_core/pipeline-template/modules/nf-core/fastqc/main.nf @@ -24,7 +24,7 @@ process FASTQC { // Make list of old name and new name pairs to use for renaming in the bash while loop def old_new_pairs = reads instanceof Path || reads.size() == 1 ? [[ reads, "${prefix}.${reads.extension}" ]] : reads.withIndex().collect { entry, index -> [ entry, "${prefix}_${index + 1}.${entry.extension}" ] } def rename_to = old_new_pairs*.join(' ').join(' ') - def renamed_files = old_new_pairs.collect{ old_name, new_name -> new_name }.join(' ') + def renamed_files = old_new_pairs.collect{ _old_name, new_name -> new_name }.join(' ') // The total amount of allocated RAM by FastQC is equal to the number of threads defined (--threads) time the amount of RAM defined (--memory) // https://github.com/s-andrews/FastQC/blob/1faeea0412093224d7f6a07f777fad60a5650795/fastqc#L211-L222 diff --git a/nf_core/pipeline-template/modules/nf-core/fastqc/meta.yml b/nf_core/pipeline-template/modules/nf-core/fastqc/meta.yml index 4827da7af2..2b2e62b8ae 100644 --- a/nf_core/pipeline-template/modules/nf-core/fastqc/meta.yml +++ b/nf_core/pipeline-template/modules/nf-core/fastqc/meta.yml @@ -11,6 +11,7 @@ tools: FastQC gives general quality metrics about your reads. It provides information about the quality score distribution across your reads, the per base sequence content (%A/C/G/T). + You get information about adapter contamination and other overrepresented sequences. homepage: https://www.bioinformatics.babraham.ac.uk/projects/fastqc/ diff --git a/nf_core/pipeline-template/subworkflows/nf-core/utils_nextflow_pipeline/main.nf b/nf_core/pipeline-template/subworkflows/nf-core/utils_nextflow_pipeline/main.nf index 0fcbf7b3f2..d6e593e852 100644 --- a/nf_core/pipeline-template/subworkflows/nf-core/utils_nextflow_pipeline/main.nf +++ b/nf_core/pipeline-template/subworkflows/nf-core/utils_nextflow_pipeline/main.nf @@ -92,10 +92,12 @@ def checkCondaChannels() { channels = config.channels } catch (NullPointerException e) { + log.debug(e) log.warn("Could not verify conda channel configuration.") return null } catch (IOException e) { + log.debug(e) log.warn("Could not verify conda channel configuration.") return null } diff --git a/nf_core/pipeline-template/subworkflows/nf-core/utils_nextflow_pipeline/tests/main.workflow.nf.test b/nf_core/pipeline-template/subworkflows/nf-core/utils_nextflow_pipeline/tests/main.workflow.nf.test index ca964ce8e1..02dbf094cd 100644 --- a/nf_core/pipeline-template/subworkflows/nf-core/utils_nextflow_pipeline/tests/main.workflow.nf.test +++ b/nf_core/pipeline-template/subworkflows/nf-core/utils_nextflow_pipeline/tests/main.workflow.nf.test @@ -52,10 +52,12 @@ nextflow_workflow { } then { - assertAll( - { assert workflow.success }, - { assert workflow.stdout.contains("nextflow_workflow v9.9.9") } - ) + expect { + with(workflow) { + assert success + assert "nextflow_workflow v9.9.9" in stdout + } + } } } diff --git a/nf_core/pipeline-template/subworkflows/nf-core/utils_nfcore_pipeline/main.nf b/nf_core/pipeline-template/subworkflows/nf-core/utils_nfcore_pipeline/main.nf index 5cb7bafef3..9e874fbf0d 100644 --- a/nf_core/pipeline-template/subworkflows/nf-core/utils_nfcore_pipeline/main.nf +++ b/nf_core/pipeline-template/subworkflows/nf-core/utils_nfcore_pipeline/main.nf @@ -56,21 +56,6 @@ def checkProfileProvided(nextflow_cli_args) { } } -// -// Citation string for pipeline -// -def workflowCitation() { - def temp_doi_ref = "" - def manifest_doi = workflow.manifest.doi.tokenize(",") - // Handling multiple DOIs - // Removing `https://doi.org/` to handle pipelines using DOIs vs DOI resolvers - // Removing ` ` since the manifest.doi is a string and not a proper list - manifest_doi.each { doi_ref -> - temp_doi_ref += " https://doi.org/${doi_ref.replace('https://doi.org/', '').replace(' ', '')}\n" - } - return "If you use ${workflow.manifest.name} for your analysis please cite:\n\n" + "* The pipeline\n" + temp_doi_ref + "\n" + "* The nf-core framework\n" + " https://doi.org/10.1038/s41587-020-0439-x\n\n" + "* Software dependencies\n" + " https://github.com/${workflow.manifest.name}/blob/master/CITATIONS.md" -} - // // Generate workflow version string // @@ -150,33 +135,6 @@ def paramsSummaryMultiqc(summary_params) { return yaml_file_text } -// -// nf-core logo -// -def nfCoreLogo(monochrome_logs=true) { - def colors = logColours(monochrome_logs) as Map - String.format( - """\n - ${dashedLine(monochrome_logs)} - ${colors.green},--.${colors.black}/${colors.green},-.${colors.reset} - ${colors.blue} ___ __ __ __ ___ ${colors.green}/,-._.--~\'${colors.reset} - ${colors.blue} |\\ | |__ __ / ` / \\ |__) |__ ${colors.yellow}} {${colors.reset} - ${colors.blue} | \\| | \\__, \\__/ | \\ |___ ${colors.green}\\`-._,-`-,${colors.reset} - ${colors.green}`._,._,\'${colors.reset} - ${colors.purple} ${workflow.manifest.name} ${getWorkflowVersion()}${colors.reset} - ${dashedLine(monochrome_logs)} - """.stripIndent() - ) -} - -// -// Return dashed line -// -def dashedLine(monochrome_logs=true) { - def colors = logColours(monochrome_logs) as Map - return "-${colors.dim}----------------------------------------------------${colors.reset}-" -} - // // ANSII colours used for terminal logging // @@ -245,28 +203,26 @@ def logColours(monochrome_logs=true) { return colorcodes } -// -// Attach the multiqc report to email -// -def attachMultiqcReport(multiqc_report) { - def mqc_report = null - try { - if (workflow.success) { - mqc_report = multiqc_report.getVal() - if (mqc_report.getClass() == ArrayList && mqc_report.size() >= 1) { - if (mqc_report.size() > 1) { +// Return a single report from an object that may be a Path or List +// +def getSingleReport(multiqc_reports) { + switch (multiqc_reports) { + case Path: + return multiqc_reports + case List: + switch (multiqc_reports.size()) { + case 0: + log.warn("[${workflow.manifest.name}] No reports found from process 'MULTIQC'") + return null + case 1: + return multiqc_reports.first() + default: log.warn("[${workflow.manifest.name}] Found multiple reports from process 'MULTIQC', will use only one") - } - mqc_report = mqc_report[0] + return multiqc_reports.first() } - } + default: + return null } - catch (Exception all) { - if (multiqc_report) { - log.warn("[${workflow.manifest.name}] Could not attach MultiQC report to summary email") - } - } - return mqc_report } // @@ -320,7 +276,7 @@ def completionEmail(summary_params, email, email_on_fail, plaintext_email, outdi email_fields['summary'] = summary << misc_fields // On success try attach the multiqc report - def mqc_report = attachMultiqcReport(multiqc_report) + def mqc_report = getSingleReport(multiqc_report) // Check if we are only sending emails on failure def email_address = email @@ -340,7 +296,7 @@ def completionEmail(summary_params, email, email_on_fail, plaintext_email, outdi def email_html = html_template.toString() // Render the sendmail template - def max_multiqc_email_size = (params.containsKey('max_multiqc_email_size') ? params.max_multiqc_email_size : 0) as nextflow.util.MemoryUnit + def max_multiqc_email_size = (params.containsKey('max_multiqc_email_size') ? params.max_multiqc_email_size : 0) as MemoryUnit def smail_fields = [email: email_address, subject: subject, email_txt: email_txt, email_html: email_html, projectDir: "${workflow.projectDir}", mqcFile: mqc_report, mqcMaxSize: max_multiqc_email_size.toBytes()] def sf = new File("${workflow.projectDir}/assets/sendmail_template.txt") def sendmail_template = engine.createTemplate(sf).make(smail_fields) @@ -351,14 +307,17 @@ def completionEmail(summary_params, email, email_on_fail, plaintext_email, outdi if (email_address) { try { if (plaintext_email) { -new org.codehaus.groovy.GroovyException('Send plaintext e-mail, not HTML') } + new org.codehaus.groovy.GroovyException('Send plaintext e-mail, not HTML') + } // Try to send HTML e-mail using sendmail def sendmail_tf = new File(workflow.launchDir.toString(), ".sendmail_tmp.html") sendmail_tf.withWriter { w -> w << sendmail_html } ['sendmail', '-t'].execute() << sendmail_html log.info("-${colors.purple}[${workflow.manifest.name}]${colors.green} Sent summary e-mail to ${email_address} (sendmail)-") } - catch (Exception all) { + catch (Exception msg) { + log.debug(msg) + log.debug("Trying with mail instead of sendmail") // Catch failures and try with plaintext def mail_cmd = ['mail', '-s', subject, '--content-type=text/html', email_address] mail_cmd.execute() << email_html diff --git a/nf_core/pipeline-template/subworkflows/nf-core/utils_nfcore_pipeline/tests/main.function.nf.test b/nf_core/pipeline-template/subworkflows/nf-core/utils_nfcore_pipeline/tests/main.function.nf.test index 1dc317f8f7..e43d208b1b 100644 --- a/nf_core/pipeline-template/subworkflows/nf-core/utils_nfcore_pipeline/tests/main.function.nf.test +++ b/nf_core/pipeline-template/subworkflows/nf-core/utils_nfcore_pipeline/tests/main.function.nf.test @@ -41,58 +41,6 @@ nextflow_function { } } - test("Test Function workflowCitation") { - - function "workflowCitation" - - then { - assertAll( - { assert function.success }, - { assert snapshot(function.result).match() } - ) - } - } - - test("Test Function nfCoreLogo") { - - function "nfCoreLogo" - - when { - function { - """ - input[0] = false - """ - } - } - - then { - assertAll( - { assert function.success }, - { assert snapshot(function.result).match() } - ) - } - } - - test("Test Function dashedLine") { - - function "dashedLine" - - when { - function { - """ - input[0] = false - """ - } - } - - then { - assertAll( - { assert function.success }, - { assert snapshot(function.result).match() } - ) - } - } - test("Test Function without logColours") { function "logColours" diff --git a/nf_core/pipeline-template/subworkflows/nf-core/utils_nfcore_pipeline/tests/main.function.nf.test.snap b/nf_core/pipeline-template/subworkflows/nf-core/utils_nfcore_pipeline/tests/main.function.nf.test.snap index 1037232c9e..02c6701413 100644 --- a/nf_core/pipeline-template/subworkflows/nf-core/utils_nfcore_pipeline/tests/main.function.nf.test.snap +++ b/nf_core/pipeline-template/subworkflows/nf-core/utils_nfcore_pipeline/tests/main.function.nf.test.snap @@ -17,26 +17,6 @@ }, "timestamp": "2024-02-28T12:02:59.729647" }, - "Test Function nfCoreLogo": { - "content": [ - "\n\n-\u001b[2m----------------------------------------------------\u001b[0m-\n \u001b[0;32m,--.\u001b[0;30m/\u001b[0;32m,-.\u001b[0m\n\u001b[0;34m ___ __ __ __ ___ \u001b[0;32m/,-._.--~'\u001b[0m\n\u001b[0;34m |\\ | |__ __ / ` / \\ |__) |__ \u001b[0;33m} {\u001b[0m\n\u001b[0;34m | \\| | \\__, \\__/ | \\ |___ \u001b[0;32m\\`-._,-`-,\u001b[0m\n \u001b[0;32m`._,._,'\u001b[0m\n\u001b[0;35m nextflow_workflow v9.9.9\u001b[0m\n-\u001b[2m----------------------------------------------------\u001b[0m-\n" - ], - "meta": { - "nf-test": "0.8.4", - "nextflow": "23.10.1" - }, - "timestamp": "2024-02-28T12:03:10.562934" - }, - "Test Function workflowCitation": { - "content": [ - "If you use nextflow_workflow for your analysis please cite:\n\n* The pipeline\n https://doi.org/10.5281/zenodo.5070524\n\n* The nf-core framework\n https://doi.org/10.1038/s41587-020-0439-x\n\n* Software dependencies\n https://github.com/nextflow_workflow/blob/master/CITATIONS.md" - ], - "meta": { - "nf-test": "0.8.4", - "nextflow": "23.10.1" - }, - "timestamp": "2024-02-28T12:03:07.019761" - }, "Test Function without logColours": { "content": [ { @@ -95,16 +75,6 @@ }, "timestamp": "2024-02-28T12:03:17.969323" }, - "Test Function dashedLine": { - "content": [ - "-\u001b[2m----------------------------------------------------\u001b[0m-" - ], - "meta": { - "nf-test": "0.8.4", - "nextflow": "23.10.1" - }, - "timestamp": "2024-02-28T12:03:14.366181" - }, "Test Function with logColours": { "content": [ { diff --git a/nf_core/pipeline-template/subworkflows/nf-core/utils_nfschema_plugin/tests/main.nf.test b/nf_core/pipeline-template/subworkflows/nf-core/utils_nfschema_plugin/tests/main.nf.test index 842dc432af..8fb3016487 100644 --- a/nf_core/pipeline-template/subworkflows/nf-core/utils_nfschema_plugin/tests/main.nf.test +++ b/nf_core/pipeline-template/subworkflows/nf-core/utils_nfschema_plugin/tests/main.nf.test @@ -42,7 +42,7 @@ nextflow_workflow { params { test_data = '' - outdir = 1 + outdir = null } workflow { @@ -94,7 +94,7 @@ nextflow_workflow { params { test_data = '' - outdir = 1 + outdir = null } workflow { From a604cb5595e9f93059d58ba1e6a9d613430803b6 Mon Sep 17 00:00:00 2001 From: mashehu Date: Tue, 3 Dec 2024 14:37:17 +0100 Subject: [PATCH 16/45] set the variable manually --- .github/workflows/create-lint-wf.yml | 2 +- .github/workflows/create-test-lint-wf-template.yml | 9 +++++++-- .github/workflows/create-test-wf.yml | 8 +++++++- .github/workflows/pytest.yml | 6 ++++++ nf_core/pipeline-template/.github/workflows/ci.yml | 7 ++++++- .../.github/workflows/download_pipeline.yml | 7 ++++++- nf_core/pipeline-template/.github/workflows/linting.yml | 6 ++++++ 7 files changed, 39 insertions(+), 6 deletions(-) diff --git a/.github/workflows/create-lint-wf.yml b/.github/workflows/create-lint-wf.yml index 78932871e3..fa6c38ef07 100644 --- a/.github/workflows/create-lint-wf.yml +++ b/.github/workflows/create-lint-wf.yml @@ -32,7 +32,7 @@ jobs: runs-on: ${{ github.event.inputs.runners || github.run_number > 1 && 'ubuntu-latest' || 'self-hosted' }} env: NXF_ANSI_LOG: false - JAVA_HOME: $JAVA_HOME_21_X64 + strategy: matrix: NXF_VER: diff --git a/.github/workflows/create-test-lint-wf-template.yml b/.github/workflows/create-test-lint-wf-template.yml index b0d4c13a4e..b4579178bd 100644 --- a/.github/workflows/create-test-lint-wf-template.yml +++ b/.github/workflows/create-test-lint-wf-template.yml @@ -28,7 +28,6 @@ concurrency: env: NXF_ANSI_LOG: false - JAVA_HOME: $JAVA_HOME_21_X64 GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} jobs: @@ -52,7 +51,7 @@ jobs: needs: prepare-matrix env: NXF_ANSI_LOG: false - JAVA_HOME: $JAVA_HOME_21_X64 + strategy: matrix: TEMPLATE: ${{ fromJson(needs.prepare-matrix.outputs.all_features) }} @@ -92,6 +91,12 @@ jobs: python -m pip install --upgrade pip pip install . + # Set up JAVA_HOME for Java 21 + - name: Set Java 21 + run: | + echo "JAVA_HOME=$JAVA_HOME_21_X64" >> $GITHUB_ENV + echo "PATH=$JAVA_HOME_21_X64/bin:$PATH" >> $GITHUB_ENV + - name: Install Nextflow uses: nf-core/setup-nextflow@v2 with: diff --git a/.github/workflows/create-test-wf.yml b/.github/workflows/create-test-wf.yml index f69449dde3..4ad4d98e0d 100644 --- a/.github/workflows/create-test-wf.yml +++ b/.github/workflows/create-test-wf.yml @@ -33,7 +33,7 @@ jobs: runs-on: ${{ github.event.inputs.runners || github.run_number > 1 && 'ubuntu-latest' || 'self-hosted' }} env: NXF_ANSI_LOG: false - JAVA_HOME: $JAVA_HOME_21_X64 + strategy: matrix: NXF_VER: @@ -59,6 +59,12 @@ jobs: python -m pip install --upgrade pip pip install . + # Set up JAVA_HOME for Java 21 + - name: Set Java 21 + run: | + echo "JAVA_HOME=$JAVA_HOME_21_X64" >> $GITHUB_ENV + echo "PATH=$JAVA_HOME_21_X64/bin:$PATH" >> $GITHUB_ENV + - name: Install Nextflow uses: nf-core/setup-nextflow@v2 with: diff --git a/.github/workflows/pytest.yml b/.github/workflows/pytest.yml index 76d5d710c0..45d08d30ba 100644 --- a/.github/workflows/pytest.yml +++ b/.github/workflows/pytest.yml @@ -120,6 +120,12 @@ jobs: id: date run: echo "date=$(date +'%Y-%m')" >> $GITHUB_ENV + # Set up JAVA_HOME for Java 21 + - name: Set Java 21 + run: | + echo "JAVA_HOME=$JAVA_HOME_21_X64" >> $GITHUB_ENV + echo "PATH=$JAVA_HOME_21_X64/bin:$PATH" >> $GITHUB_ENV + - name: Install Nextflow uses: nf-core/setup-nextflow@v2 diff --git a/nf_core/pipeline-template/.github/workflows/ci.yml b/nf_core/pipeline-template/.github/workflows/ci.yml index 4f005a1bb4..5279d8b6d6 100644 --- a/nf_core/pipeline-template/.github/workflows/ci.yml +++ b/nf_core/pipeline-template/.github/workflows/ci.yml @@ -11,7 +11,6 @@ on: env: NXF_ANSI_LOG: false - JAVA_HOME: $JAVA_HOME_21_X64 NXF_SINGULARITY_CACHEDIR: ${{ github.workspace }}/.singularity NXF_SINGULARITY_LIBRARYDIR: ${{ github.workspace }}/.singularity @@ -48,6 +47,12 @@ jobs: - name: Check out pipeline code uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4 + # Set up JAVA_HOME for Java 21 + - name: Set Java 21 + run: | + echo "JAVA_HOME=$JAVA_HOME_21_X64" >> $GITHUB_ENV + echo "PATH=$JAVA_HOME_21_X64/bin:$PATH" >> $GITHUB_ENV + - name: Set up Nextflow uses: nf-core/setup-nextflow@v2 with: diff --git a/nf_core/pipeline-template/.github/workflows/download_pipeline.yml b/nf_core/pipeline-template/.github/workflows/download_pipeline.yml index 95ebad9fe4..4a483f6ed6 100644 --- a/nf_core/pipeline-template/.github/workflows/download_pipeline.yml +++ b/nf_core/pipeline-template/.github/workflows/download_pipeline.yml @@ -26,12 +26,17 @@ on: env: NXF_ANSI_LOG: false - JAVA_HOME: $JAVA_HOME_21_X64 jobs: download: runs-on: ubuntu-latest steps: + # Set up JAVA_HOME for Java 21 + - name: Set Java 21 + run: | + echo "JAVA_HOME=$JAVA_HOME_21_X64" >> $GITHUB_ENV + echo "PATH=$JAVA_HOME_21_X64/bin:$PATH" >> $GITHUB_ENV + - name: Install Nextflow uses: nf-core/setup-nextflow@v2 diff --git a/nf_core/pipeline-template/.github/workflows/linting.yml b/nf_core/pipeline-template/.github/workflows/linting.yml index cfdbcc12a9..ede8a34355 100644 --- a/nf_core/pipeline-template/.github/workflows/linting.yml +++ b/nf_core/pipeline-template/.github/workflows/linting.yml @@ -33,6 +33,12 @@ jobs: - name: Check out pipeline code uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4 + # Set up JAVA_HOME for Java 21 + - name: Set Java 21 + run: | + echo "JAVA_HOME=$JAVA_HOME_21_X64" >> $GITHUB_ENV + echo "PATH=$JAVA_HOME_21_X64/bin:$PATH" >> $GITHUB_ENV + - name: Install Nextflow uses: nf-core/setup-nextflow@v2 From 8bc1a307d1ef53a9311f0ce5e44064217f2e1006 Mon Sep 17 00:00:00 2001 From: mashehu Date: Tue, 3 Dec 2024 14:43:11 +0100 Subject: [PATCH 17/45] nope, setup-java it is --- .github/workflows/create-test-lint-wf-template.yml | 9 ++++----- .github/workflows/create-test-wf.yml | 9 ++++----- .github/workflows/pytest.yml | 9 ++++----- nf_core/pipeline-template/.github/workflows/ci.yml | 9 ++++----- .../.github/workflows/download_pipeline.yml | 9 ++++----- nf_core/pipeline-template/.github/workflows/linting.yml | 9 ++++----- 6 files changed, 24 insertions(+), 30 deletions(-) diff --git a/.github/workflows/create-test-lint-wf-template.yml b/.github/workflows/create-test-lint-wf-template.yml index b4579178bd..78d174434b 100644 --- a/.github/workflows/create-test-lint-wf-template.yml +++ b/.github/workflows/create-test-lint-wf-template.yml @@ -91,11 +91,10 @@ jobs: python -m pip install --upgrade pip pip install . - # Set up JAVA_HOME for Java 21 - - name: Set Java 21 - run: | - echo "JAVA_HOME=$JAVA_HOME_21_X64" >> $GITHUB_ENV - echo "PATH=$JAVA_HOME_21_X64/bin:$PATH" >> $GITHUB_ENV + - uses: actions/setup-java@v4 + with: + java-version: "21" + distribution: "temurin" - name: Install Nextflow uses: nf-core/setup-nextflow@v2 diff --git a/.github/workflows/create-test-wf.yml b/.github/workflows/create-test-wf.yml index 4ad4d98e0d..55b64f9cfb 100644 --- a/.github/workflows/create-test-wf.yml +++ b/.github/workflows/create-test-wf.yml @@ -59,11 +59,10 @@ jobs: python -m pip install --upgrade pip pip install . - # Set up JAVA_HOME for Java 21 - - name: Set Java 21 - run: | - echo "JAVA_HOME=$JAVA_HOME_21_X64" >> $GITHUB_ENV - echo "PATH=$JAVA_HOME_21_X64/bin:$PATH" >> $GITHUB_ENV + - uses: actions/setup-java@v4 + with: + java-version: "21" + distribution: "temurin" - name: Install Nextflow uses: nf-core/setup-nextflow@v2 diff --git a/.github/workflows/pytest.yml b/.github/workflows/pytest.yml index 45d08d30ba..dcf5f0b0cc 100644 --- a/.github/workflows/pytest.yml +++ b/.github/workflows/pytest.yml @@ -120,11 +120,10 @@ jobs: id: date run: echo "date=$(date +'%Y-%m')" >> $GITHUB_ENV - # Set up JAVA_HOME for Java 21 - - name: Set Java 21 - run: | - echo "JAVA_HOME=$JAVA_HOME_21_X64" >> $GITHUB_ENV - echo "PATH=$JAVA_HOME_21_X64/bin:$PATH" >> $GITHUB_ENV + - uses: actions/setup-java@v4 + with: + java-version: "21" + distribution: "temurin" - name: Install Nextflow uses: nf-core/setup-nextflow@v2 diff --git a/nf_core/pipeline-template/.github/workflows/ci.yml b/nf_core/pipeline-template/.github/workflows/ci.yml index 5279d8b6d6..d022850ec5 100644 --- a/nf_core/pipeline-template/.github/workflows/ci.yml +++ b/nf_core/pipeline-template/.github/workflows/ci.yml @@ -47,11 +47,10 @@ jobs: - name: Check out pipeline code uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4 - # Set up JAVA_HOME for Java 21 - - name: Set Java 21 - run: | - echo "JAVA_HOME=$JAVA_HOME_21_X64" >> $GITHUB_ENV - echo "PATH=$JAVA_HOME_21_X64/bin:$PATH" >> $GITHUB_ENV + - uses: actions/setup-java@v4 + with: + java-version: "21" + distribution: "temurin" - name: Set up Nextflow uses: nf-core/setup-nextflow@v2 diff --git a/nf_core/pipeline-template/.github/workflows/download_pipeline.yml b/nf_core/pipeline-template/.github/workflows/download_pipeline.yml index 4a483f6ed6..5d483bd0a4 100644 --- a/nf_core/pipeline-template/.github/workflows/download_pipeline.yml +++ b/nf_core/pipeline-template/.github/workflows/download_pipeline.yml @@ -31,11 +31,10 @@ jobs: download: runs-on: ubuntu-latest steps: - # Set up JAVA_HOME for Java 21 - - name: Set Java 21 - run: | - echo "JAVA_HOME=$JAVA_HOME_21_X64" >> $GITHUB_ENV - echo "PATH=$JAVA_HOME_21_X64/bin:$PATH" >> $GITHUB_ENV + - uses: actions/setup-java@v4 + with: + java-version: "21" + distribution: "temurin" - name: Install Nextflow uses: nf-core/setup-nextflow@v2 diff --git a/nf_core/pipeline-template/.github/workflows/linting.yml b/nf_core/pipeline-template/.github/workflows/linting.yml index ede8a34355..a6150781c3 100644 --- a/nf_core/pipeline-template/.github/workflows/linting.yml +++ b/nf_core/pipeline-template/.github/workflows/linting.yml @@ -33,11 +33,10 @@ jobs: - name: Check out pipeline code uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4 - # Set up JAVA_HOME for Java 21 - - name: Set Java 21 - run: | - echo "JAVA_HOME=$JAVA_HOME_21_X64" >> $GITHUB_ENV - echo "PATH=$JAVA_HOME_21_X64/bin:$PATH" >> $GITHUB_ENV + - uses: actions/setup-java@v4 + with: + java-version: "21" + distribution: "temurin" - name: Install Nextflow uses: nf-core/setup-nextflow@v2 From 1857ebf51658a0100d70fa324303b6e4fa3adb12 Mon Sep 17 00:00:00 2001 From: mashehu Date: Tue, 3 Dec 2024 14:46:55 +0100 Subject: [PATCH 18/45] add missing setup-java action --- .github/actions/create-lint-wf/action.yml | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/.github/actions/create-lint-wf/action.yml b/.github/actions/create-lint-wf/action.yml index 3ffd960d23..01b0fda422 100644 --- a/.github/actions/create-lint-wf/action.yml +++ b/.github/actions/create-lint-wf/action.yml @@ -15,6 +15,11 @@ runs: cd create-lint-wf export NXF_WORK=$(pwd) + - uses: actions/setup-java@v4 + with: + java-version: "21" + distribution: "temurin" + - name: Install Nextflow uses: nf-core/setup-nextflow@v2 with: From 14480d280739292c69c32bc2dd0f1f0127abe136 Mon Sep 17 00:00:00 2001 From: mashehu Date: Tue, 3 Dec 2024 15:26:31 +0100 Subject: [PATCH 19/45] use java v17 --- .github/actions/create-lint-wf/action.yml | 2 +- .github/workflows/create-test-lint-wf-template.yml | 2 +- .github/workflows/create-test-wf.yml | 2 +- .github/workflows/pytest.yml | 2 +- nf_core/pipeline-template/.github/workflows/ci.yml | 2 +- .../pipeline-template/.github/workflows/download_pipeline.yml | 2 +- nf_core/pipeline-template/.github/workflows/linting.yml | 2 +- 7 files changed, 7 insertions(+), 7 deletions(-) diff --git a/.github/actions/create-lint-wf/action.yml b/.github/actions/create-lint-wf/action.yml index 01b0fda422..2573784799 100644 --- a/.github/actions/create-lint-wf/action.yml +++ b/.github/actions/create-lint-wf/action.yml @@ -17,7 +17,7 @@ runs: - uses: actions/setup-java@v4 with: - java-version: "21" + java-version: "17" distribution: "temurin" - name: Install Nextflow diff --git a/.github/workflows/create-test-lint-wf-template.yml b/.github/workflows/create-test-lint-wf-template.yml index 78d174434b..f5e40e8377 100644 --- a/.github/workflows/create-test-lint-wf-template.yml +++ b/.github/workflows/create-test-lint-wf-template.yml @@ -93,7 +93,7 @@ jobs: - uses: actions/setup-java@v4 with: - java-version: "21" + java-version: "17" distribution: "temurin" - name: Install Nextflow diff --git a/.github/workflows/create-test-wf.yml b/.github/workflows/create-test-wf.yml index 55b64f9cfb..b12b503b27 100644 --- a/.github/workflows/create-test-wf.yml +++ b/.github/workflows/create-test-wf.yml @@ -61,7 +61,7 @@ jobs: - uses: actions/setup-java@v4 with: - java-version: "21" + java-version: "17" distribution: "temurin" - name: Install Nextflow diff --git a/.github/workflows/pytest.yml b/.github/workflows/pytest.yml index dcf5f0b0cc..5bb526396d 100644 --- a/.github/workflows/pytest.yml +++ b/.github/workflows/pytest.yml @@ -122,7 +122,7 @@ jobs: - uses: actions/setup-java@v4 with: - java-version: "21" + java-version: "17" distribution: "temurin" - name: Install Nextflow diff --git a/nf_core/pipeline-template/.github/workflows/ci.yml b/nf_core/pipeline-template/.github/workflows/ci.yml index d022850ec5..3bc9db4d76 100644 --- a/nf_core/pipeline-template/.github/workflows/ci.yml +++ b/nf_core/pipeline-template/.github/workflows/ci.yml @@ -49,7 +49,7 @@ jobs: - uses: actions/setup-java@v4 with: - java-version: "21" + java-version: "17" distribution: "temurin" - name: Set up Nextflow diff --git a/nf_core/pipeline-template/.github/workflows/download_pipeline.yml b/nf_core/pipeline-template/.github/workflows/download_pipeline.yml index 5d483bd0a4..326ac0936d 100644 --- a/nf_core/pipeline-template/.github/workflows/download_pipeline.yml +++ b/nf_core/pipeline-template/.github/workflows/download_pipeline.yml @@ -33,7 +33,7 @@ jobs: steps: - uses: actions/setup-java@v4 with: - java-version: "21" + java-version: "17" distribution: "temurin" - name: Install Nextflow diff --git a/nf_core/pipeline-template/.github/workflows/linting.yml b/nf_core/pipeline-template/.github/workflows/linting.yml index a6150781c3..f9ef12335b 100644 --- a/nf_core/pipeline-template/.github/workflows/linting.yml +++ b/nf_core/pipeline-template/.github/workflows/linting.yml @@ -35,7 +35,7 @@ jobs: - uses: actions/setup-java@v4 with: - java-version: "21" + java-version: "17" distribution: "temurin" - name: Install Nextflow From 9c43f15ea589c17374b593dabcf6dea621c07e64 Mon Sep 17 00:00:00 2001 From: mashehu Date: Tue, 3 Dec 2024 15:36:20 +0100 Subject: [PATCH 20/45] avoid confusion by separating the string --- nf_core/pipeline-template/.github/workflows/branch.yml | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/nf_core/pipeline-template/.github/workflows/branch.yml b/nf_core/pipeline-template/.github/workflows/branch.yml index e0ae1aa8ab..482f53f3e2 100644 --- a/nf_core/pipeline-template/.github/workflows/branch.yml +++ b/nf_core/pipeline-template/.github/workflows/branch.yml @@ -1,6 +1,6 @@ name: nf-core branch protection # This workflow is triggered on PRs to main/master branch on the repository -# It fails when someone tries to make a PR against the nf-core `main/master` branch instead of `dev` +# It fails when someone tries to make a PR against the nf-core `main`/`master` branch instead of `dev` on: pull_request_target: branches: @@ -24,7 +24,7 @@ jobs: uses: mshick/add-pr-comment@b8f338c590a895d50bcbfa6c5859251edc8952fc # v2 with: message: | - ## This PR is against the `main/master` branch :x: + ## This PR is against the `main`/`master` branch :x: * Do not close this PR * Click _Edit_ and change the `base` to `dev` @@ -34,9 +34,9 @@ jobs: Hi @${{ github.event.pull_request.user.login }}, - It looks like this pull-request is has been made against the [${{github.event.pull_request.head.repo.full_name }}](https://github.com/${{github.event.pull_request.head.repo.full_name }}) `main/master` branch. - The `main/master` branch on nf-core repositories should always contain code from the latest release. - Because of this, PRs to `main/master` are only allowed if they come from the [${{github.event.pull_request.head.repo.full_name }}](https://github.com/${{github.event.pull_request.head.repo.full_name }}) `dev` branch. + It looks like this pull-request is has been made against the [${{github.event.pull_request.head.repo.full_name }}](https://github.com/${{github.event.pull_request.head.repo.full_name }}) `main`/`master` branch. + The `main`/`master` branch on nf-core repositories should always contain code from the latest release. + Because of this, PRs to `main`/`master` are only allowed if they come from the [${{github.event.pull_request.head.repo.full_name }}](https://github.com/${{github.event.pull_request.head.repo.full_name }}) `dev` branch. You do not need to close this PR, you can change the target branch to `dev` by clicking the _"Edit"_ button at the top of this page. Note that even after this, the test will continue to show as failing until you push a new commit. From 2e4a8ede4b1f7af1efc003cd0ce8524b002e8612 Mon Sep 17 00:00:00 2001 From: mashehu Date: Tue, 3 Dec 2024 16:50:53 +0100 Subject: [PATCH 21/45] use contect to figure out if it master or main Co-authored-by: @mirpredrol --- nf_core/pipeline-template/.github/workflows/branch.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/nf_core/pipeline-template/.github/workflows/branch.yml b/nf_core/pipeline-template/.github/workflows/branch.yml index 482f53f3e2..5c1798d3bd 100644 --- a/nf_core/pipeline-template/.github/workflows/branch.yml +++ b/nf_core/pipeline-template/.github/workflows/branch.yml @@ -24,7 +24,7 @@ jobs: uses: mshick/add-pr-comment@b8f338c590a895d50bcbfa6c5859251edc8952fc # v2 with: message: | - ## This PR is against the `main`/`master` branch :x: + ## This PR is against the `${{github.event.pull_request.base.ref}}` branch :x: * Do not close this PR * Click _Edit_ and change the `base` to `dev` From 229a2329dfefc5981897006c546a00d61da587da Mon Sep 17 00:00:00 2001 From: mashehu Date: Tue, 3 Dec 2024 18:14:33 +0100 Subject: [PATCH 22/45] disambiguate in more places --- nf_core/pipeline-template/.github/workflows/branch.yml | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/nf_core/pipeline-template/.github/workflows/branch.yml b/nf_core/pipeline-template/.github/workflows/branch.yml index 5c1798d3bd..26df52f09c 100644 --- a/nf_core/pipeline-template/.github/workflows/branch.yml +++ b/nf_core/pipeline-template/.github/workflows/branch.yml @@ -1,5 +1,5 @@ name: nf-core branch protection -# This workflow is triggered on PRs to main/master branch on the repository +# This workflow is triggered on PRs to `main`/`master` branch on the repository # It fails when someone tries to make a PR against the nf-core `main`/`master` branch instead of `dev` on: pull_request_target: @@ -24,7 +24,7 @@ jobs: uses: mshick/add-pr-comment@b8f338c590a895d50bcbfa6c5859251edc8952fc # v2 with: message: | - ## This PR is against the `${{github.event.pull_request.base.ref}}` branch :x: + ## This PR is against the `{% raw %}${{github.event.pull_request.base.ref}}{% endraw %}` branch :x: * Do not close this PR * Click _Edit_ and change the `base` to `dev` @@ -34,9 +34,9 @@ jobs: Hi @${{ github.event.pull_request.user.login }}, - It looks like this pull-request is has been made against the [${{github.event.pull_request.head.repo.full_name }}](https://github.com/${{github.event.pull_request.head.repo.full_name }}) `main`/`master` branch. - The `main`/`master` branch on nf-core repositories should always contain code from the latest release. - Because of this, PRs to `main`/`master` are only allowed if they come from the [${{github.event.pull_request.head.repo.full_name }}](https://github.com/${{github.event.pull_request.head.repo.full_name }}) `dev` branch. + It looks like this pull-request is has been made against the [${{github.event.pull_request.head.repo.full_name }}](https://github.com/${{github.event.pull_request.head.repo.full_name }}) {% raw %}${{github.event.pull_request.base.ref}}{% endraw %} branch. + The {% raw %}${{github.event.pull_request.base.ref}}{% endraw %} branch on nf-core repositories should always contain code from the latest release. + Because of this, PRs to {% raw %}${{github.event.pull_request.base.ref}}{% endraw %} are only allowed if they come from the [${{github.event.pull_request.head.repo.full_name }}](https://github.com/${{github.event.pull_request.head.repo.full_name }}) `dev` branch. You do not need to close this PR, you can change the target branch to `dev` by clicking the _"Edit"_ button at the top of this page. Note that even after this, the test will continue to show as failing until you push a new commit. From dc9c3b1763d3f41ae42a4c97a405ba9bfad61878 Mon Sep 17 00:00:00 2001 From: lmReef Date: Wed, 4 Dec 2024 09:43:11 +1300 Subject: [PATCH 23/45] fix: linting error in meta_yml where module.process_name is always "" --- nf_core/components/nfcore_component.py | 12 +++++++++++- nf_core/modules/lint/main_nf.py | 1 - 2 files changed, 11 insertions(+), 2 deletions(-) diff --git a/nf_core/components/nfcore_component.py b/nf_core/components/nfcore_component.py index 37e43a536e..090f55a4b9 100644 --- a/nf_core/components/nfcore_component.py +++ b/nf_core/components/nfcore_component.py @@ -62,7 +62,6 @@ def __init__( # Initialize the important files self.main_nf: Path = Path(self.component_dir, "main.nf") self.meta_yml: Optional[Path] = Path(self.component_dir, "meta.yml") - self.process_name = "" self.environment_yml: Optional[Path] = Path(self.component_dir, "environment.yml") component_list = self.component_name.split("/") @@ -96,6 +95,9 @@ def __init__( self.test_yml = None self.test_main_nf = None + # Set process_name after self.main_nf is defined + self.process_name = self._get_process_name() + def __repr__(self) -> str: return f"" @@ -169,6 +171,13 @@ def _get_included_components_in_chained_tests(self, main_nf_test: Union[Path, st included_components.append(component) return included_components + def _get_process_name(self): + with open(self.main_nf) as fh: + for line in fh: + if re.search(r"^\s*process\s*\w*\s*{", line): + return re.search(r"^\s*process\s*(\w*)\s*{.*", line).group(1) or "" + return "" + def get_inputs_from_main_nf(self) -> None: """Collect all inputs from the main.nf file.""" inputs: Any = [] # Can be 'list[list[dict[str, dict[str, str]]]]' or 'list[str]' @@ -263,3 +272,4 @@ def get_outputs_from_main_nf(self): pass log.debug(f"Found {len(outputs)} outputs in {self.main_nf}") self.outputs = outputs + diff --git a/nf_core/modules/lint/main_nf.py b/nf_core/modules/lint/main_nf.py index 848e17130e..9a1790aeb8 100644 --- a/nf_core/modules/lint/main_nf.py +++ b/nf_core/modules/lint/main_nf.py @@ -256,7 +256,6 @@ def check_process_section(self, lines, registry, fix_version, progress_bar): bioconda_packages = [] # Process name should be all capital letters - self.process_name = lines[0].split()[1] if all(x.upper() for x in self.process_name): self.passed.append(("process_capitals", "Process name is in capital letters", self.main_nf)) else: From b04351db539fc3962b816595ee7ab9eae3a83bd3 Mon Sep 17 00:00:00 2001 From: lmReef Date: Wed, 4 Dec 2024 09:54:53 +1300 Subject: [PATCH 24/45] fix: python linting warn --- nf_core/components/nfcore_component.py | 1 - 1 file changed, 1 deletion(-) diff --git a/nf_core/components/nfcore_component.py b/nf_core/components/nfcore_component.py index 090f55a4b9..eda95be991 100644 --- a/nf_core/components/nfcore_component.py +++ b/nf_core/components/nfcore_component.py @@ -272,4 +272,3 @@ def get_outputs_from_main_nf(self): pass log.debug(f"Found {len(outputs)} outputs in {self.main_nf}") self.outputs = outputs - From d3d9c72d7a20b617e417a8e0ba22ad9b29944d07 Mon Sep 17 00:00:00 2001 From: lmReef Date: Wed, 4 Dec 2024 10:29:10 +1300 Subject: [PATCH 25/45] update changelog --- CHANGELOG.md | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index da5f72c357..29e405903b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -25,6 +25,7 @@ ### Linting - allow mixed `str` and `dict` entries in lint config ([#3228](https://github.com/nf-core/tools/pull/3228)) +- fix meta_yml linting test failing due to module.process_name always being "" (#3317) ### Modules From 4a8e97631b6164b92f039de74e462cd9b9dd94ee Mon Sep 17 00:00:00 2001 From: lmReef Date: Wed, 4 Dec 2024 10:41:19 +1300 Subject: [PATCH 26/45] update changelog entry with link --- CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 29e405903b..7799ef1d4c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -25,7 +25,7 @@ ### Linting - allow mixed `str` and `dict` entries in lint config ([#3228](https://github.com/nf-core/tools/pull/3228)) -- fix meta_yml linting test failing due to module.process_name always being "" (#3317) +- fix meta_yml linting test failing due to module.process_name always being "" ([#3317](https://github.com/nf-core/tools/pull/3317)) ### Modules From b94435b92891b2cee0a78c47890b1d7349d30376 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?J=C3=BAlia=20Mir=20Pedrol?= Date: Wed, 4 Dec 2024 09:48:13 +0000 Subject: [PATCH 27/45] remove too many raw/endraw in branch.yml --- nf_core/pipeline-template/.github/workflows/branch.yml | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/nf_core/pipeline-template/.github/workflows/branch.yml b/nf_core/pipeline-template/.github/workflows/branch.yml index 26df52f09c..110b4a5f5a 100644 --- a/nf_core/pipeline-template/.github/workflows/branch.yml +++ b/nf_core/pipeline-template/.github/workflows/branch.yml @@ -24,7 +24,7 @@ jobs: uses: mshick/add-pr-comment@b8f338c590a895d50bcbfa6c5859251edc8952fc # v2 with: message: | - ## This PR is against the `{% raw %}${{github.event.pull_request.base.ref}}{% endraw %}` branch :x: + ## This PR is against the `${{github.event.pull_request.base.ref}}` branch :x: * Do not close this PR * Click _Edit_ and change the `base` to `dev` @@ -34,9 +34,9 @@ jobs: Hi @${{ github.event.pull_request.user.login }}, - It looks like this pull-request is has been made against the [${{github.event.pull_request.head.repo.full_name }}](https://github.com/${{github.event.pull_request.head.repo.full_name }}) {% raw %}${{github.event.pull_request.base.ref}}{% endraw %} branch. - The {% raw %}${{github.event.pull_request.base.ref}}{% endraw %} branch on nf-core repositories should always contain code from the latest release. - Because of this, PRs to {% raw %}${{github.event.pull_request.base.ref}}{% endraw %} are only allowed if they come from the [${{github.event.pull_request.head.repo.full_name }}](https://github.com/${{github.event.pull_request.head.repo.full_name }}) `dev` branch. + It looks like this pull-request is has been made against the [${{github.event.pull_request.head.repo.full_name }}](https://github.com/${{github.event.pull_request.head.repo.full_name }}) ${{github.event.pull_request.base.ref}} branch. + The ${{github.event.pull_request.base.ref}} branch on nf-core repositories should always contain code from the latest release. + Because of this, PRs to ${{github.event.pull_request.base.ref}} are only allowed if they come from the [${{github.event.pull_request.head.repo.full_name }}](https://github.com/${{github.event.pull_request.head.repo.full_name }}) `dev` branch. You do not need to close this PR, you can change the target branch to `dev` by clicking the _"Edit"_ button at the top of this page. Note that even after this, the test will continue to show as failing until you push a new commit. From f22d7d9e03f33c078967e38bc7174e6a5de6547c Mon Sep 17 00:00:00 2001 From: Rob Syme Date: Wed, 4 Dec 2024 08:09:57 -0500 Subject: [PATCH 28/45] Revert changes to nf_core/pipeline-template/subworkflows/nf-core/utils_nfcore_pipeline/main.nf --- .../nf-core/utils_nfcore_pipeline/main.nf | 40 ++++++++++--------- 1 file changed, 21 insertions(+), 19 deletions(-) diff --git a/nf_core/pipeline-template/subworkflows/nf-core/utils_nfcore_pipeline/main.nf b/nf_core/pipeline-template/subworkflows/nf-core/utils_nfcore_pipeline/main.nf index 9e874fbf0d..228dbff897 100644 --- a/nf_core/pipeline-template/subworkflows/nf-core/utils_nfcore_pipeline/main.nf +++ b/nf_core/pipeline-template/subworkflows/nf-core/utils_nfcore_pipeline/main.nf @@ -203,26 +203,29 @@ def logColours(monochrome_logs=true) { return colorcodes } -// Return a single report from an object that may be a Path or List // -def getSingleReport(multiqc_reports) { - switch (multiqc_reports) { - case Path: - return multiqc_reports - case List: - switch (multiqc_reports.size()) { - case 0: - log.warn("[${workflow.manifest.name}] No reports found from process 'MULTIQC'") - return null - case 1: - return multiqc_reports.first() - default: +// Attach the multiqc report to email +// +def attachMultiqcReport(multiqc_report) { + def mqc_report = null + try { + if (workflow.success) { + mqc_report = multiqc_report.getVal() + if (mqc_report.getClass() == ArrayList && mqc_report.size() >= 1) { + if (mqc_report.size() > 1) { log.warn("[${workflow.manifest.name}] Found multiple reports from process 'MULTIQC', will use only one") - return multiqc_reports.first() + } + mqc_report = mqc_report[0] } - default: - return null + } } + catch (Exception msg) { + log.debug(msg) + if (multiqc_report) { + log.warn("[${workflow.manifest.name}] Could not attach MultiQC report to summary email") + } + } + return mqc_report } // @@ -276,7 +279,7 @@ def completionEmail(summary_params, email, email_on_fail, plaintext_email, outdi email_fields['summary'] = summary << misc_fields // On success try attach the multiqc report - def mqc_report = getSingleReport(multiqc_report) + def mqc_report = attachMultiqcReport(multiqc_report) // Check if we are only sending emails on failure def email_address = email @@ -307,8 +310,7 @@ def completionEmail(summary_params, email, email_on_fail, plaintext_email, outdi if (email_address) { try { if (plaintext_email) { - new org.codehaus.groovy.GroovyException('Send plaintext e-mail, not HTML') - } +new org.codehaus.groovy.GroovyException('Send plaintext e-mail, not HTML') } // Try to send HTML e-mail using sendmail def sendmail_tf = new File(workflow.launchDir.toString(), ".sendmail_tmp.html") sendmail_tf.withWriter { w -> w << sendmail_html } From c3fd9bb084c86e1204e9d4cbeebef413444b1834 Mon Sep 17 00:00:00 2001 From: lmReef Date: Thu, 5 Dec 2024 08:59:07 +1300 Subject: [PATCH 29/45] fix: add explicit str type to self.process_name --- nf_core/components/nfcore_component.py | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/nf_core/components/nfcore_component.py b/nf_core/components/nfcore_component.py index eda95be991..81c0ba98e7 100644 --- a/nf_core/components/nfcore_component.py +++ b/nf_core/components/nfcore_component.py @@ -95,8 +95,7 @@ def __init__( self.test_yml = None self.test_main_nf = None - # Set process_name after self.main_nf is defined - self.process_name = self._get_process_name() + self.process_name: str = self._get_process_name() def __repr__(self) -> str: return f"" From 57a92a0eff27c3570ae2cfcf73a765a66ed3c541 Mon Sep 17 00:00:00 2001 From: lmReef Date: Thu, 5 Dec 2024 09:34:08 +1300 Subject: [PATCH 30/45] fix: module section linter regex --- nf_core/modules/lint/main_nf.py | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/nf_core/modules/lint/main_nf.py b/nf_core/modules/lint/main_nf.py index 848e17130e..599d50cd20 100644 --- a/nf_core/modules/lint/main_nf.py +++ b/nf_core/modules/lint/main_nf.py @@ -96,19 +96,19 @@ def main_nf( for line in iter_lines: if re.search(r"^\s*process\s*\w*\s*{", line) and state == "module": state = "process" - if re.search(r"input\s*:", line) and state in ["process"]: + if re.search(r"^\s*input\s*:", line) and state in ["process"]: state = "input" continue - if re.search(r"output\s*:", line) and state in ["input", "process"]: + if re.search(r"^\s*output\s*:", line) and state in ["input", "process"]: state = "output" continue - if re.search(r"when\s*:", line) and state in ["input", "output", "process"]: + if re.search(r"^\s*when\s*:", line) and state in ["input", "output", "process"]: state = "when" continue - if re.search(r"script\s*:", line) and state in ["input", "output", "when", "process"]: + if re.search(r"^\s*script\s*:", line) and state in ["input", "output", "when", "process"]: state = "script" continue - if re.search(r"shell\s*:", line) and state in ["input", "output", "when", "process"]: + if re.search(r"^\s*shell\s*:", line) and state in ["input", "output", "when", "process"]: state = "shell" continue From 355dff2c8e8fa28b628459bab65efeadf3793dfc Mon Sep 17 00:00:00 2001 From: nf-core-bot Date: Wed, 4 Dec 2024 21:10:36 +0000 Subject: [PATCH 31/45] [automated] Update CHANGELOG.md --- CHANGELOG.md | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index da5f72c357..91c70f4437 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -29,6 +29,7 @@ ### Modules - add a panel around diff previews when updating ([#3246](https://github.com/nf-core/tools/pull/3246)) +- Fix module section linter regex ([#3321](https://github.com/nf-core/tools/pull/3321)) ### Subworkflows From 9996df9e061a7512f6cce1d34fc0e8f7a937df95 Mon Sep 17 00:00:00 2001 From: lmReef Date: Thu, 5 Dec 2024 10:11:58 +1300 Subject: [PATCH 32/45] update changelog --- CHANGELOG.md | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 91c70f4437..b75b865745 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -25,6 +25,7 @@ ### Linting - allow mixed `str` and `dict` entries in lint config ([#3228](https://github.com/nf-core/tools/pull/3228)) +- fix module section regex matching wrong things ([#3321](https://github.com/nf-core/tools/pull/3321)) ### Modules From c22c571cff747b1fb47ae11d3dc1fa76e14a9a2f Mon Sep 17 00:00:00 2001 From: lmReef Date: Thu, 5 Dec 2024 10:14:01 +1300 Subject: [PATCH 33/45] Revert "[automated] Update CHANGELOG.md" This reverts commit 355dff2c8e8fa28b628459bab65efeadf3793dfc. --- CHANGELOG.md | 1 - 1 file changed, 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index b75b865745..67417f940a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -30,7 +30,6 @@ ### Modules - add a panel around diff previews when updating ([#3246](https://github.com/nf-core/tools/pull/3246)) -- Fix module section linter regex ([#3321](https://github.com/nf-core/tools/pull/3321)) ### Subworkflows From ce866c62c141ce1e0126dad9942f879e8bea3f89 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?J=C3=BAlia=20Mir=20Pedrol?= Date: Thu, 5 Dec 2024 11:55:09 +0100 Subject: [PATCH 34/45] Remove not needed setup-java step --- .github/actions/create-lint-wf/action.yml | 5 ----- .github/workflows/create-test-lint-wf-template.yml | 5 ----- .github/workflows/create-test-wf.yml | 5 ----- .github/workflows/pytest.yml | 5 ----- nf_core/pipeline-template/.github/workflows/ci.yml | 5 ----- .../.github/workflows/download_pipeline.yml | 5 ----- nf_core/pipeline-template/.github/workflows/linting.yml | 5 ----- 7 files changed, 35 deletions(-) diff --git a/.github/actions/create-lint-wf/action.yml b/.github/actions/create-lint-wf/action.yml index 2573784799..3ffd960d23 100644 --- a/.github/actions/create-lint-wf/action.yml +++ b/.github/actions/create-lint-wf/action.yml @@ -15,11 +15,6 @@ runs: cd create-lint-wf export NXF_WORK=$(pwd) - - uses: actions/setup-java@v4 - with: - java-version: "17" - distribution: "temurin" - - name: Install Nextflow uses: nf-core/setup-nextflow@v2 with: diff --git a/.github/workflows/create-test-lint-wf-template.yml b/.github/workflows/create-test-lint-wf-template.yml index f5e40e8377..cabd4b9abe 100644 --- a/.github/workflows/create-test-lint-wf-template.yml +++ b/.github/workflows/create-test-lint-wf-template.yml @@ -91,11 +91,6 @@ jobs: python -m pip install --upgrade pip pip install . - - uses: actions/setup-java@v4 - with: - java-version: "17" - distribution: "temurin" - - name: Install Nextflow uses: nf-core/setup-nextflow@v2 with: diff --git a/.github/workflows/create-test-wf.yml b/.github/workflows/create-test-wf.yml index b12b503b27..53f84b72c4 100644 --- a/.github/workflows/create-test-wf.yml +++ b/.github/workflows/create-test-wf.yml @@ -59,11 +59,6 @@ jobs: python -m pip install --upgrade pip pip install . - - uses: actions/setup-java@v4 - with: - java-version: "17" - distribution: "temurin" - - name: Install Nextflow uses: nf-core/setup-nextflow@v2 with: diff --git a/.github/workflows/pytest.yml b/.github/workflows/pytest.yml index 5bb526396d..76d5d710c0 100644 --- a/.github/workflows/pytest.yml +++ b/.github/workflows/pytest.yml @@ -120,11 +120,6 @@ jobs: id: date run: echo "date=$(date +'%Y-%m')" >> $GITHUB_ENV - - uses: actions/setup-java@v4 - with: - java-version: "17" - distribution: "temurin" - - name: Install Nextflow uses: nf-core/setup-nextflow@v2 diff --git a/nf_core/pipeline-template/.github/workflows/ci.yml b/nf_core/pipeline-template/.github/workflows/ci.yml index 3bc9db4d76..9db393d9f0 100644 --- a/nf_core/pipeline-template/.github/workflows/ci.yml +++ b/nf_core/pipeline-template/.github/workflows/ci.yml @@ -47,11 +47,6 @@ jobs: - name: Check out pipeline code uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4 - - uses: actions/setup-java@v4 - with: - java-version: "17" - distribution: "temurin" - - name: Set up Nextflow uses: nf-core/setup-nextflow@v2 with: diff --git a/nf_core/pipeline-template/.github/workflows/download_pipeline.yml b/nf_core/pipeline-template/.github/workflows/download_pipeline.yml index 326ac0936d..05397358cf 100644 --- a/nf_core/pipeline-template/.github/workflows/download_pipeline.yml +++ b/nf_core/pipeline-template/.github/workflows/download_pipeline.yml @@ -31,11 +31,6 @@ jobs: download: runs-on: ubuntu-latest steps: - - uses: actions/setup-java@v4 - with: - java-version: "17" - distribution: "temurin" - - name: Install Nextflow uses: nf-core/setup-nextflow@v2 diff --git a/nf_core/pipeline-template/.github/workflows/linting.yml b/nf_core/pipeline-template/.github/workflows/linting.yml index f9ef12335b..cfdbcc12a9 100644 --- a/nf_core/pipeline-template/.github/workflows/linting.yml +++ b/nf_core/pipeline-template/.github/workflows/linting.yml @@ -33,11 +33,6 @@ jobs: - name: Check out pipeline code uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4 - - uses: actions/setup-java@v4 - with: - java-version: "17" - distribution: "temurin" - - name: Install Nextflow uses: nf-core/setup-nextflow@v2 From a9d97e4f296551fd0c7b9e071a0990007b3b3770 Mon Sep 17 00:00:00 2001 From: "renovate[bot]" <29139614+renovate[bot]@users.noreply.github.com> Date: Thu, 5 Dec 2024 12:16:57 +0000 Subject: [PATCH 35/45] Update python:3.12-slim Docker digest to 2b00791 (#3319) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> --- Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Dockerfile b/Dockerfile index dc9948ea4b..f2141145b8 100644 --- a/Dockerfile +++ b/Dockerfile @@ -1,4 +1,4 @@ -FROM python:3.12-slim@sha256:2a6386ad2db20e7f55073f69a98d6da2cf9f168e05e7487d2670baeb9b7601c5 +FROM python:3.12-slim@sha256:2b0079146a74e23bf4ae8f6a28e1b484c6292f6fb904cbb51825b4a19812fcd8 LABEL authors="phil.ewels@seqera.io,erik.danielsson@scilifelab.se" \ description="Docker image containing requirements for nf-core/tools" From c23af55faa65b9f4bc6735917bdcbb77acdc500e Mon Sep 17 00:00:00 2001 From: nf-core-bot Date: Thu, 5 Dec 2024 15:19:54 +0000 Subject: [PATCH 36/45] [automated] Update CHANGELOG.md --- CHANGELOG.md | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index fed991f850..3272db6c76 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -26,6 +26,7 @@ ### Subworkflows - Add `nf-core subworkflows patch` command ([#2861](https://github.com/nf-core/tools/pull/2861)) +- Improve subworkflow nf-test migration warning ([#3298](https://github.com/nf-core/tools/pull/3298)) ### General From 6ddfe4d35b4a159dd5f18a12c77c6761ec66068f Mon Sep 17 00:00:00 2001 From: mashehu Date: Thu, 5 Dec 2024 16:30:44 +0100 Subject: [PATCH 37/45] fix headers --- docs/api/_src/api/pipelines/bump_version.md | 2 +- docs/api/_src/api/pipelines/create.md | 2 +- docs/api/_src/api/pipelines/download.md | 2 +- docs/api/_src/api/pipelines/launch.md | 2 +- docs/api/_src/api/pipelines/lint.md | 2 +- docs/api/_src/api/pipelines/list.md | 2 +- docs/api/_src/api/pipelines/params-file.md | 2 +- docs/api/_src/api/pipelines/schema.md | 2 +- docs/api/_src/api/pipelines/sync.md | 2 +- docs/api/_src/api/pipelines/utils.md | 2 +- 10 files changed, 10 insertions(+), 10 deletions(-) diff --git a/docs/api/_src/api/pipelines/bump_version.md b/docs/api/_src/api/pipelines/bump_version.md index cd7dc280f6..76db67837a 100644 --- a/docs/api/_src/api/pipelines/bump_version.md +++ b/docs/api/_src/api/pipelines/bump_version.md @@ -1,4 +1,4 @@ -# nf_core.bump_version +# nf_core.pipelines.bump_version ```{eval-rst} .. automodule:: nf_core.pipelines.bump_version diff --git a/docs/api/_src/api/pipelines/create.md b/docs/api/_src/api/pipelines/create.md index 576335e951..5019a5f3c8 100644 --- a/docs/api/_src/api/pipelines/create.md +++ b/docs/api/_src/api/pipelines/create.md @@ -1,4 +1,4 @@ -# nf_core.create +# nf_core.pipelines.create ```{eval-rst} .. automodule:: nf_core.pipelines.create diff --git a/docs/api/_src/api/pipelines/download.md b/docs/api/_src/api/pipelines/download.md index 540fb92c49..afb31ddea6 100644 --- a/docs/api/_src/api/pipelines/download.md +++ b/docs/api/_src/api/pipelines/download.md @@ -1,4 +1,4 @@ -# nf_core.download +# nf_core.pipelines.download ```{eval-rst} .. automodule:: nf_core.pipelines.download diff --git a/docs/api/_src/api/pipelines/launch.md b/docs/api/_src/api/pipelines/launch.md index 0f7fc03f64..0d0260cae6 100644 --- a/docs/api/_src/api/pipelines/launch.md +++ b/docs/api/_src/api/pipelines/launch.md @@ -1,4 +1,4 @@ -# nf_core.launch +# nf_core.pipelines.launch ```{eval-rst} .. automodule:: nf_core.pipelines.launch diff --git a/docs/api/_src/api/pipelines/lint.md b/docs/api/_src/api/pipelines/lint.md index aa62c404b8..91b37c26f6 100644 --- a/docs/api/_src/api/pipelines/lint.md +++ b/docs/api/_src/api/pipelines/lint.md @@ -1,4 +1,4 @@ -# nf_core.lint +# nf_core.pipelines.lint :::{seealso} See the [Lint Tests](/docs/nf-core-tools/api_reference/dev/pipeline_lint_tests) docs for information about specific linting functions. diff --git a/docs/api/_src/api/pipelines/list.md b/docs/api/_src/api/pipelines/list.md index 7df7564544..5f404b91c3 100644 --- a/docs/api/_src/api/pipelines/list.md +++ b/docs/api/_src/api/pipelines/list.md @@ -1,4 +1,4 @@ -# nf_core.list +# nf_core.pipelines.list ```{eval-rst} .. automodule:: nf_core.pipelines.list diff --git a/docs/api/_src/api/pipelines/params-file.md b/docs/api/_src/api/pipelines/params-file.md index 06f27cc592..37e91f458a 100644 --- a/docs/api/_src/api/pipelines/params-file.md +++ b/docs/api/_src/api/pipelines/params-file.md @@ -1,4 +1,4 @@ -# nf_core.params_file +# nf_core.pipelines.params_file ```{eval-rst} .. automodule:: nf_core.pipelines.params_file diff --git a/docs/api/_src/api/pipelines/schema.md b/docs/api/_src/api/pipelines/schema.md index c885d9ed23..4ca1aab480 100644 --- a/docs/api/_src/api/pipelines/schema.md +++ b/docs/api/_src/api/pipelines/schema.md @@ -1,4 +1,4 @@ -# nf_core.schema +# nf_core.pipelines.schema ```{eval-rst} .. automodule:: nf_core.pipelines.schema diff --git a/docs/api/_src/api/pipelines/sync.md b/docs/api/_src/api/pipelines/sync.md index da1f468fe5..f78733bb7d 100644 --- a/docs/api/_src/api/pipelines/sync.md +++ b/docs/api/_src/api/pipelines/sync.md @@ -1,4 +1,4 @@ -# nf_core.sync +# nf_core.pipelines.sync ```{eval-rst} .. automodule:: nf_core.pipelines.sync diff --git a/docs/api/_src/api/pipelines/utils.md b/docs/api/_src/api/pipelines/utils.md index 86b8c3f36f..36c2ecca4d 100644 --- a/docs/api/_src/api/pipelines/utils.md +++ b/docs/api/_src/api/pipelines/utils.md @@ -1,4 +1,4 @@ -# nf_core.utils +# nf_core.pipelines.utils ```{eval-rst} .. automodule:: nf_core.pipelines.utils From f54e1234e9451afbafecb26f124261ba8bc3977b Mon Sep 17 00:00:00 2001 From: nf-core-bot Date: Thu, 5 Dec 2024 16:04:21 +0000 Subject: [PATCH 38/45] [automated] Update CHANGELOG.md --- CHANGELOG.md | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 61d8dbf518..ffc43d2258 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -35,6 +35,7 @@ - Update GitHub Actions ([#3237](https://github.com/nf-core/tools/pull/3237)) - add `--dir/-d` option to schema commands ([#3247](https://github.com/nf-core/tools/pull/3247)) - Update pre-commit hook astral-sh/ruff-pre-commit to v0.7.1 ([#3250](https://github.com/nf-core/tools/pull/3250)) +- fix headers in api docs ([#3323](https://github.com/nf-core/tools/pull/3323)) ## [v3.0.2 - Titanium Tapir Patch](https://github.com/nf-core/tools/releases/tag/3.0.2) - [2024-10-11] From 2c5932e66e2cdbf85728fb0ec03c695e5aebdb8a Mon Sep 17 00:00:00 2001 From: "renovate[bot]" <29139614+renovate[bot]@users.noreply.github.com> Date: Thu, 5 Dec 2024 16:49:54 +0000 Subject: [PATCH 39/45] Update pre-commit hook astral-sh/ruff-pre-commit to v0.8.2 --- .pre-commit-config.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 1494f58182..68a6fa3ed7 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -1,6 +1,6 @@ repos: - repo: https://github.com/astral-sh/ruff-pre-commit - rev: v0.8.1 + rev: v0.8.2 hooks: - id: ruff # linter args: [--fix, --exit-non-zero-on-fix] # sort imports and fix From c03f21288a0ff05dc42f2b5dddf135531564fbed Mon Sep 17 00:00:00 2001 From: nf-core-bot Date: Thu, 5 Dec 2024 16:52:26 +0000 Subject: [PATCH 40/45] [automated] Update CHANGELOG.md --- CHANGELOG.md | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 9c011dfa4e..a4d8b44878 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -57,6 +57,7 @@ - Update pre-commit hook astral-sh/ruff-pre-commit to v0.8.0 ([#3299](https://github.com/nf-core/tools/pull/3299)) - Update gitpod/workspace-base Docker digest to 12853f7 ([#3309](https://github.com/nf-core/tools/pull/3309)) - Run pre-commit when testing linting the template pipeline ([#3280](https://github.com/nf-core/tools/pull/3280)) +- Update pre-commit hook astral-sh/ruff-pre-commit to v0.8.2 ([#3325](https://github.com/nf-core/tools/pull/3325)) ## [v3.0.2 - Titanium Tapir Patch](https://github.com/nf-core/tools/releases/tag/3.0.2) - [2024-10-11] From da16adb69b352d9a2e5120137a5c2ac0be88264a Mon Sep 17 00:00:00 2001 From: Phil Ewels Date: Thu, 5 Dec 2024 23:56:27 +0100 Subject: [PATCH 41/45] Make prompt less nf-core specific --- nf_core/components/components_utils.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/nf_core/components/components_utils.py b/nf_core/components/components_utils.py index 67e05e0ce6..23bf08bbd6 100644 --- a/nf_core/components/components_utils.py +++ b/nf_core/components/components_utils.py @@ -43,10 +43,10 @@ def get_repo_info(directory: Path, use_prompt: Optional[bool] = True) -> Tuple[P if not repo_type and use_prompt: log.warning("'repository_type' not defined in %s", config_fn.name) repo_type = questionary.select( - "Is this repository an nf-core pipeline or a fork of nf-core/modules?", + "Is this repository a pipeline or a modules repository?", choices=[ {"name": "Pipeline", "value": "pipeline"}, - {"name": "nf-core/modules", "value": "modules"}, + {"name": "Modules repository", "value": "modules"}, ], style=nf_core.utils.nfcore_question_style, ).unsafe_ask() From ba447fbf7027312d7562ac533b9ca26ff68574fe Mon Sep 17 00:00:00 2001 From: nf-core-bot Date: Thu, 5 Dec 2024 22:57:35 +0000 Subject: [PATCH 42/45] [automated] Update CHANGELOG.md --- CHANGELOG.md | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index a4d8b44878..4fef8ce401 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -58,6 +58,7 @@ - Update gitpod/workspace-base Docker digest to 12853f7 ([#3309](https://github.com/nf-core/tools/pull/3309)) - Run pre-commit when testing linting the template pipeline ([#3280](https://github.com/nf-core/tools/pull/3280)) - Update pre-commit hook astral-sh/ruff-pre-commit to v0.8.2 ([#3325](https://github.com/nf-core/tools/pull/3325)) +- Make prompt less nf-core specific ([#3326](https://github.com/nf-core/tools/pull/3326)) ## [v3.0.2 - Titanium Tapir Patch](https://github.com/nf-core/tools/releases/tag/3.0.2) - [2024-10-11] From 6a59a9749450bcf796aab68506302145e5f93efd Mon Sep 17 00:00:00 2001 From: Mahesh Binzer-Panchal Date: Fri, 6 Dec 2024 13:50:30 +0000 Subject: [PATCH 43/45] Update gitpod vscode extensions to use nf-core extension pack --- .gitpod.yml | 9 +-------- nf_core/pipeline-template/.gitpod.yml | 11 ++--------- 2 files changed, 3 insertions(+), 17 deletions(-) diff --git a/.gitpod.yml b/.gitpod.yml index efe193f35f..d5948695bf 100644 --- a/.gitpod.yml +++ b/.gitpod.yml @@ -9,11 +9,4 @@ tasks: vscode: extensions: - - esbenp.prettier-vscode # Markdown/CommonMark linting and style checking for Visual Studio Code - - EditorConfig.EditorConfig # override user/workspace settings with settings found in .editorconfig files - - Gruntfuggly.todo-tree # Display TODO and FIXME in a tree view in the activity bar - - mechatroner.rainbow-csv # Highlight columns in csv files in different colors - - nextflow.nextflow # Nextflow syntax highlighting - - oderwat.indent-rainbow # Highlight indentation level - - streetsidesoftware.code-spell-checker # Spelling checker for source code - - charliermarsh.ruff # Code linter Ruff + - nf-core.nf-core-extensionpack # https://github.com/nf-core/vscode-extensionpack diff --git a/nf_core/pipeline-template/.gitpod.yml b/nf_core/pipeline-template/.gitpod.yml index 5907fb59c9..b8165fbc06 100644 --- a/nf_core/pipeline-template/.gitpod.yml +++ b/nf_core/pipeline-template/.gitpod.yml @@ -6,13 +6,6 @@ tasks: nextflow self-update vscode: - extensions: # based on nf-core.nf-core-extensionpack + extensions: #{%- if code_linters -%} - - esbenp.prettier-vscode # Markdown/CommonMark linting and style checking for Visual Studio Code - - EditorConfig.EditorConfig # override user/workspace settings with settings found in .editorconfig files{% endif %} - - Gruntfuggly.todo-tree # Display TODO and FIXME in a tree view in the activity bar - - mechatroner.rainbow-csv # Highlight columns in csv files in different colors - - nextflow.nextflow # Nextflow syntax highlighting - - oderwat.indent-rainbow # Highlight indentation level - - streetsidesoftware.code-spell-checker # Spelling checker for source code - - charliermarsh.ruff # Code linter Ruff + - nf-core.nf-core-extensionpack # https://github.com/nf-core/vscode-extensionpack From 0ba1c7007490f10a0f985a5d70fef474b58f0cac Mon Sep 17 00:00:00 2001 From: nf-core-bot Date: Fri, 6 Dec 2024 13:55:42 +0000 Subject: [PATCH 44/45] [automated] Update CHANGELOG.md --- CHANGELOG.md | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index a4d8b44878..91d9960a81 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -58,6 +58,7 @@ - Update gitpod/workspace-base Docker digest to 12853f7 ([#3309](https://github.com/nf-core/tools/pull/3309)) - Run pre-commit when testing linting the template pipeline ([#3280](https://github.com/nf-core/tools/pull/3280)) - Update pre-commit hook astral-sh/ruff-pre-commit to v0.8.2 ([#3325](https://github.com/nf-core/tools/pull/3325)) +- Update gitpod vscode extensions to use nf-core extension pack ([#3327](https://github.com/nf-core/tools/pull/3327)) ## [v3.0.2 - Titanium Tapir Patch](https://github.com/nf-core/tools/releases/tag/3.0.2) - [2024-10-11] From 7a03d907f6cdf8d75344edbbf2a141854346da11 Mon Sep 17 00:00:00 2001 From: nf-core-bot Date: Mon, 9 Dec 2024 09:47:45 +0000 Subject: [PATCH 45/45] [automated] Fix code linting --- CHANGELOG.md | 1 - 1 file changed, 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 5876c653b7..20f48bb1ce 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -64,7 +64,6 @@ - Make CLI prompt less nf-core specific ([#3326](https://github.com/nf-core/tools/pull/3326)) - Update gitpod vscode extensions to use nf-core extension pack ([#3327](https://github.com/nf-core/tools/pull/3327)) - ## [v3.0.2 - Titanium Tapir Patch](https://github.com/nf-core/tools/releases/tag/3.0.2) - [2024-10-11] ### Template