phenopy was developed using Python 3.9 and functions to perform phenotype similarity scoring by semantic similarity. phenopy is a
lightweight but highly optimized command line tool and library to efficiently perform semantic similarity scoring on
generic entities with phenotype annotations from the Human Phenotype Ontology (HPO).
Install using pip:
pip install phenopyInstall from GitHub:
git clone https://github.com/GeneDx/phenopy.git
cd phenopy
pipx install poetry
poetry installphenopy is primarily used as a command line tool. An entity, as described here, is presented as a sample, gene, or
disease, but could be any concept that warrants annotation of phenotype terms.
Use phenopy score to perform semantic similarity scoring in various formats. Write the results of any command to file
using --output-file=/path/to/output_file.txt
-
Score similarity of entities defined by the HPO terms from an input file against all the OMIM diseases in
.phenopy/data/phenotype.hpoa. We provide a test input file in the repo. The default summarization method is to use--summarization-method=BMWAwhich weighs each diseases' phenotypes by the frequency of a phenotype seen in each particular disease.phenopy score tests/data/test.score.txt
Output:
#query entity_id score 118200 210100 0.0 118200 615779 0.0 118200 613266 0.0052 ... -
Score similarity of entities defined by the HPO terms from an input file against all the OMIM diseases in
.phenopy/data/phenotype.hpoa, to use the non-weighted summarization method use--summarization-method=BMAwhich uses a traditional best-match average summarization of semantic similarity scores when comparing terms from record a with terms from record b.phenopy score tests/data/test.score.txt --summarization-method=BMWA
Output:
#query entity_id score 118200 210100 0.0 118200 615779 0.0 118200 613266 0.0052 ... -
Score similarity of an entities defined by the HPO terms from an input file against a custom list of entities with HPO annotations, referred to as the
--records-file. Both files are in the same format.phenopy score tests/data/test.score-short.txt --records-file tests/data/test.score-long.txt
Output:
#query entity_id score 118200 118200 0.0169 118200 300905 0.0156 118200 601098 0.0171 ... -
Score pairwise similarity of entities defined by the HPO terms from an input file using
--self.phenopy score tests/data/test.score-long.txt --threads 4 --self
Output:
#query entity_id score 118200 118200 0.2284 118200 118210 0.1302 118200 118211 0.1302 118210 118210 0.2048 118210 118211 0.2048 118211 118211 0.2048 -
Score age-adjusted pairwise similarity of entities defined in the input file, using phenotype mean age and standard deviation defined in the
--ages_distribution_file, select best-match weighted average as the scoring summarization method--summarization-method BMWA.phenopy score tests/data/test.score-short.txt --ages_distribution_file tests/data/phenotype_age.tsv --summarization-method BMWA --threads 4 --self
Output:
#query entity_id score 118200 210100 0.0 118200 177650 0.0127 118200 241520 0.0 ...The phenotype age file contains hpo-id, mean, sd as tab separated text as follows
HP:0001251 6.0 3.0 HP:0001263 1.0 1.0 HP:0001290 1.0 1.0 HP:0004322 10.0 3.0 HP:0001249 6.0 3.0
If no phenotype ages file is provided --summarization-method=BMWA can be selected to use default, open access literature-derived phenotype ages (~ 1,400 age weighted phenotypes).
phenopy score tests/data/test.score-short.txt --summarization-method BMWA --threads 4For a full list of command arguments use phenopy [subcommand] --help:
phenopy score --helpOutput:
--output_file=OUTPUT_FILE
File path where to store the results. [default: - (stdout)]
--records_file=RECORDS_FILE
An entity-to-phenotype annotation file in the same format as "input_file". This file, if provided, is used to score entries in the "input_file" against entries here. [default: None]
--annotations_file=ANNOTATIONS_FILE
An entity-to-phenotype annotation file in the same format as "input_file". This file, if provided, is used to add information content to the network. [default: None]
--ages_distribution_file=AGES_DISTRIBUTION_FILE
Phenotypes age summary stats file containing phenotype HPO id, mean_age, and std. [default: None]
--self=SELF
Score entries in the "input_file" against itself.
--summarization_method=SUMMARIZATION_METHOD
The method used to summarize the HRSS matrix. Supported Values are best match average (BMA), best match weighted average (BMWA), and maximum (maximum). [default: BMWA]
--threads=THREADS
Number of parallel processes to use. [default: 1]
The phenopy library can be used as a Python module, allowing more control for advanced users.
Generate the hpo network and supporting objects:
import os
from phenopy.build_hpo import generate_annotated_hpo_network
from phenopy.score import Scorer
# data directory
phenopy_data_directory = os.path.join(os.getenv('HOME'), '.phenopy/data')
# files used in building the annotated HPO network
obo_file = os.path.join(phenopy_data_directory, 'hp.obo')
disease_to_phenotype_file = os.path.join(phenopy_data_directory, 'phenotype.hpoa')
# if you have a custom ages_distribution_file, you can set it here.
ages_distribution_file = os.path.join(phenopy_data_directory, 'xa_age_stats_oct052019.tsv')
hpo_network, alt2prim, disease_records = \
generate_annotated_hpo_network(obo_file,
disease_to_phenotype_file,
ages_distribution_file=ages_distribution_file
)Then, instantiate the Scorer class and score hpo term lists.
scorer = Scorer(hpo_network)
terms_a = ['HP:0001263', 'HP:0011839']
terms_b = ['HP:0001263', 'HP:0000252']
print(scorer.score_term_sets_basic(terms_a, terms_b))Output:
0.11213185474495047
The library can be used to prune parent phenotypes from the phenotype.hpoa and store pruned annotations as a file
from phenopy.util import export_phenotype_hpoa_with_no_parents
# saves a new file of phenotype disease annotations with parent HPO terms removed from phenotype lists.
disease_to_phenotype_no_parents_file = os.path.join(phenopy_data_directory, 'phenotype.noparents.hpoa')
export_phenotype_hpoa_with_no_parents(disease_to_phenotype_file, disease_to_phenotype_no_parents_file, hpo_network)phenopy is designed to run with minimal setup from the user, to run phenopy with default parameters (recommended), skip ahead to the Commands overview.
This section provides details about where phenopy stores data resources and config files. The following occurs when you run phenopy for the first time.
- phenopy creates a
.phenopy/directory in your home folder and downloads external resources from HPO into the$HOME/.phenopy/data/directory. - phenopy creates a
$HOME/.phenopy/phenopy.iniconfig file where users can set variables for phenopy to use at runtime.
While we recommend using the default settings for most users, the config file can be modified: $HOME/.phenopy/phenopy.ini.
To run phenopy with a different version of hp.obo, set the path of obo_file in $HOME/.phenopy/phenopy.ini.
We welcome contributions from the community. Please follow these steps to setup a local development environment.
pipenv install --devTo run tests locally:
pipenv shell
coverage run --source=. -m unittest discover --start-directory tests/
coverage report -mThe underlying algorithm which determines the semantic similarity for any two HPO terms is based on an implementation of HRSS, published here.
Please use the following Bibtex to cite this software.
@software{arvai_phenopy_2019,
title = {Phenopy},
rights = {Attribution-NonCommercial-ShareAlike 4.0 International},
url = {https://github.com/GeneDx/phenopy},
abstract = {Phenopy is a Python package to perform phenotype similarity scoring by semantic similarity.
Phenopy is a lightweight but highly optimized command line tool and library to efficiently perform semantic
similarity scoring on generic entities with phenotype annotations from the Human Phenotype Ontology (HPO).},
version = {0.3.0},
author = {Arvai, Kevin and Borroto, Carlos and Gainullin, Vladimir and Retterer, Kyle},
date = {2019-11-05},
year = {2019},
doi = {10.5281/zenodo.3529569}
}