software.md

HuBMAP Product Offerings: Project Management, Communication, Tracking, & Reporting

Last Updaed: 2/19/2021

Description:

The purpose of this document is to categorize HuBMAP efforts through a product management lens in order to structure conversations around, work linkages to, and dependencies among software development level epics and features.

Linked entries below direct you to the associated code repository if one is present.

Infratructure/ Base Product

• API Gateway
• Common Coordinate Framework (CCF)
• Pipeline Container Orchestration
• Documentation (on Portal)
  • CCF Checklist and SOPs
  • MC-IU Onboarding videos
  • IVMOOC
• Hybrid Cloud Infrastructure
• Identity system
• Knowledge Graph
  • Schema
  • Ontology ingest & API service
  • Base biomedical ontologies
• Microservices Architecture
  • Provenance services
• Provenance Graph (+ core database using Docker)
• User registration
• Monitoring services
• Data Ingest
• Data Search

Tissue/ sample prep/ core data

• Dataset generation by data providers (TMCs, RTIs, TTDs)
  • See list of assays on Portal
• DOI generation (DOI registration service)
• Protocols registration tool(protocols.io)

Spatial registration and semantic annotation (via 3D collision detection)

• ASCT+B Reporter
• CCF reference objects (body, organs)
• CCF registration tool (RUI)

Data & Metadata

• Antibody registration tool
• Data & metadata standards & documentation
• Data ingest tool (data & metadata, sample, assay, antibody report, contributor)
• File transfer interface (Globus)
• Manual dataset ingest tools
• Manual donor data ingest process
• Sequencing data dbGaP submission tool
• Tissue & donor registration tool

Pipeline

• Base QA pipeline (IEC dataset one)
• Cells API: cell identification, backend, js client, py client
• Data ingest pipeline
• Pipeline for:
  • CODEX (Cytokit + SPRM)
  • “Example Pipeline”
  • Imaging Mass Spectrometry & MxIF
  • sc/snATAC-seq (SnapTools, SnapATAC, and chromVAR)
  • sc/snRNA-seq (Salmon, Scanpy, scVelo)
  • SPRM (Imaging pipeline)
  • Spatial Transcriptomics (Starfish)
• QA metrics service (assay specific pipeline QA metric sharing)
• Tools
  • Mixed datatype pipeline tools
  • OME.TIFF Pyramid
  • Pipeline deployment
  • Sequencing (FASTQ) file tools
  • Sequencing (snap) file tools
  • Visualization pre-processing
  • Vitessce pre-processing
• Workflow management + Common Workflow Language tool
  • Development workflow management

Reference Mapping

• Azimuth: Automated mapping of query datasets for HuBMAP references

QA

• Approval process (currently manual process)
• Dataset validation tools (data, metadata)
  • Metadata submission conversion
  • Tests supporting validation
• Manual curation tools
• QC metrics

Search & Download

• General Search (Elasticsearch)
• Query tools
• Facets
• Semantic
  • Gene
  • Cell
  • Spatial
  • Multidimensional
• Dataset Download (Portal link + Globus mechanism)

Visualization

• Pipeline visualization (CWL)
• Portal & dataset UI
  • Portal Styling
  • Viewer (Vitessce)
• Spatial visualization (CCF EUI)
  • Cell type
  • Seq-data access (linking with dbGaP)
  • Sample data for CCF demos

Operations

• Data Release reporting tool (internal & external facets)
• Publications (consortium website)