Merge branch 'lipid_dev_documentation' into 'lipid_dev'

Add lipidomics documentation

See merge request mass-spectrometry/metams!9

.bumpversion.cfg

@@ -1,5 +1,5 @@
 [bumpversion]
-current_version = 2.2.4
+current_version = 2.2.3
 commit = False
 tag = False
 
@@ -16,8 +16,10 @@ values =
 
 [bumpversion:file:README.md]
 
+[bumpversion:file:MAINTAINERS.md]
+
 [bumpversion:file:metaMS/__init__.py]
 
 [bumpversion:file:docs/index.rst]
 
-[bumpversion:file:metaMS/nmdc_lipidomics_metadata_generation/metadata_generator.py]
+#TODO KRH: Add bumpversion to the wdls

.bumpversion_lipid.cfg

@@ -0,0 +1,19 @@
+[bumpversion]
+current_version = 1.0.0
+commit = False
+tag = False
+
+[bumpversion:part:release]
+optional_value = rc1
+values = 
+	alpha
+	beta
+	rc1
+	rc2
+	rc3
+
+[bumpversion:file:docs/index_lipid.rst]
+
+[bumpversion:file:MAINTAINERS.md]
+
+[bumpversion:file:metaMS/nmdc_lipidomics_metadata_generation/metadata_generator.py]

MAINTAINERS.md

@@ -0,0 +1,62 @@
+# Developer Guide
+
+## Table of Contents
+1. [Introduction](#introduction)
+2. [Setting Up the Development Environment](#setting-up-the-development-environment)
+3. [Pushing New Docker Image](#pushing-updated-docker-image)
+4. [Generating Documentation](#generating-documentation)
+5. [Bumping Version Numbers](#bumping-version-numbers)
+
+## Introduction
+This guide provides instructions for developers working on the project. It covers setting up the development environment, pushing Docker images, generating documentation, and bumping version numbers.
+
+## Setting Up the Development Environment
+1. Clone the repository:
+    ```sh
+    git clone https://github.com/your-repo/metams.git
+    cd metams
+    ```
+2. Install dependencies:
+    ```sh
+    pip install -r requirements.txt
+    ```
+3. Install developer-specific dependencies
+    ```sh
+    pip install -r requirements-dev.txt
+    ```
+
+## Steps to releasing a new version
+
+### Pushing Updated Docker Image
+You must be logged into docker hub
+1. Build the push the tagged docker image
+    ```sh
+    make docker-nmdc
+    ```
+
+### Bumping Version Numbers
+The versioning of the repo, docker image and the GC/MS workflow are currently 2.2.3
+The version of the lipid workflow is currently 1.0.0
+
+To bump *both* the repo and the lipid workflow, run one of the following commands.
+
+    ```sh
+    make major
+    ```
+
+    ```sh
+    make minor
+    ```
+
+    ```sh
+    make patch
+    ```
+
+### Generate Documentation
+Documentation of the two workflows can be found the docs folder of this repo.
+
+For the lipidomics workflow, edit only the `docs/index_lipid.rst` file.  Once changes are made there, re-render the `docs/index_lipid.html` and the `docs/README_LCMS_LIPID.md` with the following command.  **This should happen after the version numbers are bumped.**
+
+    ```sh
+    make convert_lipid_rst_to_md
+    ```

Makefile

@@ -17,14 +17,29 @@ mem:
 major:
 
 	@bumpversion major --allow-dirty
+	@bumpversion major --allow-dirty --config-file .bumpversion_lipid.cfg
 
 minor:
 
 	@bumpversion minor --allow-dirty
+	@bumpversion minor --allow-dirty --config-file .bumpversion_lipid.cfg
 
 patch:
 
 	@bumpversion patch --allow-dirty
+	@bumpversion patch --allow-dirty --config-file .bumpversion_lipid.cfg
+
+bump_lipid_major:
+
+	@bumpversion major --allow-dirty --config-file .bumpversion_lipid.cfg
+
+bump_lipid_minor:
+
+	@bumpversion minor --allow-dirty --config-file .bumpversion_lipid.cfg
+
+bump_lipid_patch:
+
+	@bumpversion patch --allow-dirty --config-file .bumpversion_lipid.cfg
 
 install:
 	@source venv/bin/activate
@@ -64,8 +79,10 @@ docker-run:
 	@docker run -v $(data_dir):/metams/data -v $(config_dir):/metams/configuration microbiomedata/metams:latest metaMS run-gcms-workflow /metams/configuration/metams.toml
 
 wdl-run :
-
 	 miniwdl run wdl/metaMS.wdl -i wdl/metams_input.json --verbose --no-cache --copy-input-files
 
-
-
+convert_lipid_rst_to_md:
+    # convert the lipid documentation from rst to md
+	pandoc -f rst -t markdown -o docs/README_LCMS_LIPID.md docs/index_lipid.rst
+	# render the lipid documentation into html from the rst file
+	pandoc -f rst -t html -o docs/index_lipid.html docs/index_lipid.rst

README.md

@@ -1,161 +1,23 @@
 # Table of Contents  
-- Introduction
-  - [MetaMS](#MetaMS)  
+  - [MetaMS](#metams)  
   - [Version](#current-version)  
-  - [Data Input](#data-input-formats)  
-  - [Data Output](#data-output-formats)  
-  - [Data Structure](#data-structure-types)  
-  - [Features](#available-features)  
-  - [Code Documentation](https://emsl-computing.github.io/MetaMS/)  
+  - [Available Workflows](#available-workflows)
+  - [Disclaimer](#disclaimer)
 
-- Installation
-  - [PyPi](#metams-installation)  
-
-- Execution:  
-  - [CLI](#execution)  
-  - [MiniWDL](#MiniWDL)  
-  - [Docker Container](#metams-docker-container)  
 # MetaMS
 
-**MetaMS** is a workflow for metabolomics data processing and annotation
+**MetaMS** is a repository of workflows for metabolomics data processing and annotation in association with the NMDC ([National Microbiome Data Collaborative](https://microbiomedata.org/))
 
 ## Current Version
 
 ### `2.2.3`
 
-### Data input formats
-
-- ANDI NetCDF for GC-MS (.cdf)
-- CoreMS self-containing Hierarchical Data Format (.hdf5)
-- ChemStation Agilent (Ongoing)
-
-### Data output formats
-
-- Pandas data frame (can be saved using pickle, h5, etc)
-- Text Files (.csv, tab separated .txt, etc)
-- Microsoft Excel (xlsx)
-- JSON for workflow metadata
-- Self-containing Hierarchical Data Format (.hdf5) including raw data and ime-series data-point for processed data-sets with all associated metadata stored as json attributes
-
-### Data structure types
-
-- GC-MS
-
-## Available features
-
-### Signal Processing
-
-- Baseline detection, subtraction, smoothing 
-- m/z based Chromatogram Peak Deconvolution,
-- Manual and automatic noise threshold calculation
-- First and second derivatives peak picking methods
-- Peak Area Calculation
-
-
-### Calibration
-
-- Retention Index Linear XXX method 
-
-### Compound Identification
-
-- Automatic local (SQLite) or external (MongoDB or PostgreSQL) database check, generation, and search
-- Automatic molecular match algorithm with all spectral similarity methods 
-
-## MetaMS Installation
-
-Make sure you have python 3.9.13 installed before continue
-
-- PyPi:     
-```bash
-pip3 install metams
-```
-
-- From source:
- ```bash
-pip3 install --editable .
-```
-
-To be able to open chemstation files a installation of pythonnet is needed:
-- Windows: 
-    ```bash
-    pip3 install pythonnet
-    ```
-
-- Mac and Linux:
-    ```bash
-    brew install mono
-    pip3 install pythonnet   
-    ```
-
-## Execution
-
-```bash
-metaMS dump-gcms-toml-template gcms_metams.toml
-```
-```bash
-metaMS dump-gcms-corems-toml-template gcms_corems.toml
-```
-
- Modify the gcms_metams.toml and gcms_corems.toml accordingly to your dataset and workflow parameters
-make sure to include gcms_corems.json path inside the gcms_metams.toml: "corems_toml_path": "path_to_corems.toml" 
-
-```bash
-metaMS run-gcms-workflow path_to_gcms_metams.toml
-```
-
-## MiniWDL 
-
-Make sure you have python 3.9.13 installed before continue
-
-MiniWDL uses the microbiome/metaMS image so there is not need to install metaMS
-
-- Change wdl/gcms_metams_input.json to specify the data location
-
-- Change data/gcms_corems.toml to specify the workflow parameters
-
-Install miniWDL:
-```bash
-pip3 install miniwdl
-```
-
-Call:
-```bash
-miniwdl run wdl/metaMS_gcms.wdl -i wdl/metams_input_gcms.json --verbose --no-cache --copy-input-files
-```
-## MetaMS Docker Container
-
-You will need docker and docker compose: 
-
-If you don't have it installed, the easiest way is to [install docker for desktop](https://www.docker.com/products/docker-desktop/)
-
-- Pull from Docker Registry:
-
-    ```bash
-    docker pull microbiomedata/metams:latest
-    
-    ```
-- or Build the image from source:
-
-    ```bash
-    docker build -t microbiomedata/metams:latest .
-    ```
-- Run Workflow from Container:
+## Available Workflows
 
-    $(data_dir) = full path of directory containing the gcms data
-    $(config_dir) = full path of directory containing configuration and parameters metams.toml and corems.toml
-    ```bash
-    docker run -v $(data_dir):/metaB/data -v $(config_dir):/metaB/configuration microbiomedata/metams:latest metaMS run-gcms-workflow /metaB/configuration/metams.toml
-    ```
+- [GC/MS metabolomics workflow](docs/README_GCMS.md)
+- [LC/MS lipidomics workflow](docs/README_LCMS_LIPID.md)
 
-- Getting the parameters templates:
-    
-    ```bash
-    docker run -v $(config_dir):/metaB/configuration microbiomedata/metams:latest metaMS dump-json-template /metaB/configuration/metams.toml
-    ```
-    
-    ```bash
-    docker run -v $(config_dir):/metaB/configuration microbiomedata/metams:latest metaMS dump-corems-json-template /metaB/configuration/corems.toml
-    ```
+For information about data input, output, and execution for the individual workflows, follow the linked readmes above.
 
 ## Disclaimer