Merge pull request #296 from dice-group/retrieval_via_triplestore

Retrieval via triplestore

.gitignore

@@ -154,8 +154,6 @@ embeddings/
 # Big files
 pre_trained_agents.zip
 embeddings.zip
-KGs/dbpedia_2016-10.owl
-KGs/Nctrer/nctrer.owl
-KGs/Mutagenesis/mutagenesis.owl
-KGs/Carcinogenesis/carcinogenesis.owl
 KGs.zip
+/Fuseki/
+/KGs/

README.md

@@ -10,7 +10,7 @@
 - **CELOE** → [Class Expression Learning for Ontology Engineering](https://www.sciencedirect.com/science/article/abs/pii/S1570826811000023)
 - **OCEL** → A limited version of CELOE
 
-You can find more details about *Ontolearn* and these algorithms and their variations in the [documentation](https://ontolearn-docs-dice-group.netlify.app/index.html).
+You can find more details about these algorithms and what *Ontolearn* has to offer in the [documentation](https://ontolearn-docs-dice-group.netlify.app/index.html).
 
 Quick navigation: 
 - [Installation](#installation)
@@ -65,14 +65,14 @@ the **Family** dataset which is a simple dataset with 202 individuals:
 python deploy_cl.py --model evolearner --path_knowledge_base KGs/Family/family-benchmark_rich_background.owl
 ```
 
-Once you run this command, a local URL where our model is deployed will be provided to you.
+Once you run this command, a local URL where the model is deployed will be provided to you.
 
 
 In the interface you need to enter the positive and the negative examples. For a quick run you can
 click on the **Random Examples** checkbox, but you may as well enter some real examples for
 the learning problem of **Aunt**, **Brother**, **Cousin**, etc. which
-you can find in the folder `examples/synthetic_problems.json`. Just copy and paste the IRIs of
-positive and negative examples for a certain learning problem directly
+you can find in the file `examples/synthetic_problems.json`. Just copy and paste the IRIs of
+positive and negative examples for a specific learning problem directly
 in their respective fields.
 
 Run the help command to see the description on this script usage:
@@ -84,7 +84,7 @@ python deploy_cl.py --help
 ## Usage
 
 In the [examples](https://github.com/dice-group/Ontolearn/tree/develop/examples) folder, you can find examples on how to use
-the learning algorithms. Also in the [tests](https://github.com/dice-group/Ontolearn/tree/develop/tests) folder we have added some test cases.
+the learning algorithms. Also in the [tests](https://github.com/dice-group/Ontolearn/tree/develop/tests) folder you can find the test cases.
 
 For more detailed instructions we suggest to follow the [guides](https://ontolearn-docs-dice-group.netlify.app/usage/06_concept_learners) in the documentation.
 
@@ -108,16 +108,20 @@ negative_examples = {OWLNamedIndividual(IRI.create(NS, 'heinz')),
                      OWLNamedIndividual(IRI.create(NS, 'anna')),
                      OWLNamedIndividual(IRI.create(NS, 'michelle'))}
 
-# Only the class of the learning algorithm is specified
+# Create a model of CELOE using ModelAdapter
+# Only the class of the learning algorithm is specified 
 model = ModelAdapter(learner_type=CELOE,
                      reasoner_type=OWLReasoner_Owlready2_ComplexCEInstances,
                      path="KGs/father.owl")
 
+# Fit the learning problem to the model
 model.fit(pos=positive_examples,
           neg=negative_examples)
 
+# Used to render to description logics syntax
 renderer = DLSyntaxObjectRenderer()
 
+# Print the rendered top best hypothesis
 for desc in model.best_hypotheses(1):
     print('The result:', renderer.render(desc.concept), 'has quality', desc.quality)
 ```
@@ -127,7 +131,8 @@ The output is as follows:
 The result: (¬female) ⊓ (∃ hasChild.⊤) has quality 1.0
 ```
 
-For a quick start on how to use NCES, please refer to the notebook [simple usage NCES](examples/simple-usage-NCES.ipynb)
+NCES is a powerful algorithm implemented recently.
+For a quick start on how to use it, please refer to the notebook [simple usage NCES](examples/simple-usage-NCES.ipynb).
 
 ----------------------------------------------------------------------------
 

docs/usage/01_introduction.md

@@ -2,12 +2,14 @@
 
 **Version:** ontolearn 0.5.4
 
-**GitHub repository:** https://github.com/dice-group/Ontolearn
+**GitHub repository:** [https://github.com/dice-group/Ontolearn](https://github.com/dice-group/Ontolearn)
 
 **Publisher and maintainer:** [DICE](https://dice-research.org/) - data science research group of [Paderborn University](https://www.uni-paderborn.de/en/university).
 
 **Contact**: [[email protected]](mailto:[email protected])
 
+**License:** GNU Affero General Public License v3 or later (AGPLv3+)
+
 --------------------------------------------------------------------------------------------
 
 Ontolearn is an open-source software library for explainable structured machine learning in Python.

docs/usage/03_ontologies.md

@@ -5,7 +5,7 @@ required is an [Ontology](https://www.w3.org/TR/owl2-overview/) with
 In this guide we show the basics of working with ontologies in Ontolearn.
 
 We will frequently **use a sample ontology** to give examples. You can find it in  
-`KGs/father.owl`. Here is a hierarchical diagram that shows the classes and their
+`KGs/father.owl` after you [download the datasets](02_installation.md#download-external-files). Here is a hierarchical diagram that shows the classes and their
 relationships:
 
              Thing

docs/usage/05_reasoner.md

@@ -32,8 +32,12 @@ from. Currently, there are the following reasoners available:
     ```
 
     The structural reasoner requires an ontology ([OWLOntology](ontolearn.owlapy.model.OWLOntology)).
-    There is also the `isolate` argument which isolates the world the reasoner is performing the
-    reasoning. More on that on _[Reasoning Details](07_reasoning_details.md#isolated-world)_.
+    The second argument is `isolate` argument which isolates the world (therefore the ontology) where the reasoner is 
+    performing the reasoning. More on that on _[Reasoning Details](07_reasoning_details.md#isolated-world)_. 
+    The rest of the arguments `use_triplestore` and `triplestore_address` are used in case you want to
+    retrieve instances from a triplestore (go to 
+    [#use-triplestore-to-retrieve-instances](05_reasoner.md#use-triplestore-to-retrieve-instances) for details).
+
 
 
 - [**OWLReasoner_Owlready2_ComplexCEInstances**](ontolearn.owlapy.owlready2.complex_ce_instances.OWLReasoner_Owlready2_ComplexCEInstances) **(CCEI)**
@@ -56,7 +60,8 @@ from. Currently, there are the following reasoners available:
     which is just an enumeration with two possible values: `BaseReasoner_Owlready2.HERMIT` and `BaseReasoner_Owlready2.PELLET`.
     You can set the `infer_property_values` argument to `True` if you want the reasoner to infer
     property values. `infer_data_property_values` is an additional argument when the base reasoner is set to 
-    `BaseReasoner_Owlready2.PELLET`.
+    `BaseReasoner_Owlready2.PELLET`. The rest of the arguments `isolated`, `use_triplestore` and `triplestore_address` 
+    are inherited from the base class.
 
 
 - [**OWLReasoner_FastInstanceChecker**](ontolearn.owlapy.fast_instance_checker.OWLReasoner_FastInstanceChecker) **(FIC)**
@@ -224,6 +229,109 @@ for ind in male_individuals:
     print(ind)
 ```
 
+### Use triplestore to retrieve instances
+
+Instead of going through nodes using expensive computation resources why not just make use of the
+efficient approach of querying a triplestore using SPARQL queries. We have brought this 
+functionality to Ontolearn, and we take care of the conversion part behind the scene. You can use the `instances` 
+method as usual. Let's see what it takes to make use of it.
+
+First of all you need a server which should host the triplestore for your ontology. If you don't
+already have one, see [Loading and launching a triplestore](#loading-and-launching-a-triplestore) below.
+
+As we mentioned earlier, OWLReasoner has two arguments for enabling triplestore 
+retrieval: 
+- `use_triplestore` is a boolean argument. Setting this to True tells `instances` that it
+should query the triplestore hosted on the server address specified by the following argument:
+- `triplestore_address` is a string that contains the URL of the triplestore host.
+
+For example below we are initializing a reasoner that uses the triplestore in the address
+`http://localhost:3030/path/` and retrieving the male instances.
+```python
+reasoner = OWLReasoner_Owlready2(onto,use_triplestore=True, triplestore_address="http://localhost:3030/path/sparql")
+males = reasoner.instances(male, direct=False)
+```
+
+> :warning: **You have to make sure** that the ontology that is loaded in the triplestore is
+> exactly the same as the ontology that is being used by the reasoner, otherwise you may
+> encounter inconsistent results.
+
+> :warning: When using triplestore the `instances` method **will default to the base
+> implementation**. This means that no matter which type of reasoner you are using,
+> the results will be always the same for a given class expression.
+
+> :warning: **You cannot pass these arguments directly to FIC constructor.** 
+> Because of the way it is implemented, if the base reasoner is set to use triplestore,
+> then FIC's `instances` method will default to the base reasoners implementation that uses 
+> triplestore for instance retrieval.
+
+#### Loading and launching a triplestore
+
+We will provide a simple approach to load and launch a triplestore in a local server. For this,
+we will be using _apache-jena_ and _apache-jena-fuseki_. As a prerequisite you need
+JDK 11 or higher and if you are on Windows, you need [Cygwin](https://www.cygwin.com/). In case of
+issues or any further reference please visit the official page of [Apache Jena](https://jena.apache.org/index.html) 
+and check the documentation under "Triple Store".
+
+Having that said, let us now load and launch a triplestore on the "Father" ontology:
+
+Open a terminal window and make sure you are in the root directory. Create a directory to 
+store the files for Fuseki server:
+
+```shell
+mkdir Fuseki && cd Fuseki
+```
+Install _apache-jena_ and _apache-jena-fuseki_. We will use version 4.7.0.
+
+```shell
+# install Jena
+wget https://archive.apache.org/dist/jena/binaries/apache-jena-4.7.0.tar.gz
+#install Jena-Fuseki
+wget https://archive.apache.org/dist/jena/binaries/apache-jena-fuseki-4.7.0.tar.gz
+```
+
+Unzip the files:
+
+```shell
+tar -xzf apache-jena-fuseki-4.7.0.tar.gz
+tar -xzf apache-jena-4.7.0.tar.gz
+```
+
+Make a directory for our 'father' database inside jena-fuseki:
+
+```shell
+mkdir -p apache-jena-fuseki-4.7.0/databases/father/
+```
+
+Now just load the 'father' ontology using the following commands:
+
+```shell
+cd ..
+
+Fuseki/apache-jena-4.7.0/bin/tdb2.tdbloader --loader=parallel --loc Fuseki/apache-jena-fuseki-4.7.0/databases/father/ KGs/father.owl
+```
+
+Launch the server, and it will be waiting eagerly for your queries.
+
+```shell
+cd Fuseki/apache-jena-fuseki-4.7.0 
+
+java -Xmx4G -jar fuseki-server.jar --tdb2 --loc=databases/father /father
+```
+
+Notice that we launched the database found in `Fuseki/apache-jena-fuseki-4.7.0/databases/father` to the path `/father`.
+By default, jena-fuseki runs on port 3030 so the full URL would be: `http://localhost:3030/father`. When 
+you pass this url to `triplestore_address` argument of the reasoner, you have to add the
+`/sparql` sub-path indicating to the server that we are querying via SPARQL queries. Full path now should look like:
+`http://localhost:3030/father/sparql`.
+
+Create a reasoner that uses this URL and retrieve the desired instances:
+
+```python
+father_reasoner = OWLReasoner_Owlready2(onto, use_triplestore=True, triplestore_address="http://localhost:3030/father/sparql")
+# ** use `instances` as you would normally do ** .
+```
+
 -----------------------------------------------------------------------
 
 In this guide we covered the main functionalities of the reasoners in Ontolearn. More

docs/usage/06_concept_learners.md

@@ -8,10 +8,11 @@ of Ontolearn library:
 - [CELOE](ontolearn.concept_learner.CELOE)
 - [OCEL](ontolearn.concept_learner.OCEL)
 
-> **IMPORTANT**!
-> - _DRILL_ is not fully implemented in Ontolear. In the meantime you can refer to 
+
+> :warning: **_DRILL_ is not fully implemented in Ontolearn**. In the meantime you can refer to 
 > [_DRILL's_ GitHub repo](https://github.com/dice-group/drill). 
-> - Documentation for _NCES_ coming soon. In the meantime visit _NCES_ jupyter notebooks
+>
+> :warning: **Documentation for _NCES_ coming soon**. In the meantime visit _NCES_ jupyter notebooks
 > inside [examples folder](https://github.com/dice-group/Ontolearn/tree/develop/examples).
 
 These algorithms are similar in execution, for that reason, we are 
@@ -280,7 +281,7 @@ how to configure it's input properties and how to run it to successfully
 learn class expressions for learning problems in an ontology. There is also a jupyter 
 notebook for each of these concept learners:
 
-- [NCES notebook](../../examples/simple-usage-NCES.ipynb)
-- [EvoLearner notebook](../../examples/evolearner_notebook.ipynb)
-- [CELOE notebook](../../examples/celoe_notebook.ipynb)
-- [OCEL notebook](../../examples/ocel_notebook.ipynb)
+- [NCES notebook](https://github.com/dice-group/Ontolearn/blob/develop/examples/simple-usage-NCES.ipynb)
+- [EvoLearner notebook](https://github.com/dice-group/Ontolearn/blob/develop/examples/evolearner_notebook.ipynb)
+- [CELOE notebook](https://github.com/dice-group/Ontolearn/blob/develop/examples/celoe_notebook.ipynb)
+- [OCEL notebook](https://github.com/dice-group/Ontolearn/blob/develop/examples/ocel_notebook.ipynb)

docs/usage/07_reasoning_details.md

@@ -124,8 +124,8 @@ functionality of our reasoner:
 
 ## Concrete Example
 
-You can find the associated [code](https://github.com/dice-group/Ontolearn/blob/develop/examples/example_reasone∃r.Py)  
-for the following examples inside `examples/example_reasone∃r.Py`
+You can find the associated [code](https://github.com/dice-group/Ontolearn/blob/develop/examples/example_reasone∃r.Py) 
+for the following examples inside `examples/example_reasoner`
 (note that the naming of the classes/relations/individuals may change from the table below). 
 We constructed an ontology for testing purposes. On the table we show for
 each **method** of the reasoner _OWLReasoner_Owlready2_ComplexCEInstances_ the results

docs/usage/08_model_adapter.md

@@ -3,7 +3,7 @@
 To simplify the connection between all the
 components, there is a
 model adaptor available that automatically constructs and connects them.
-Here is how to implement the previous example using the Model Adaptor:
+Here is how to implement the previous example using the [ModelAdapter](ontolearn.mode_adapter.ModelAdapter):
 
 ```python
 from ontolearn.concept_learner import CELOE
@@ -52,4 +52,4 @@ for desc in model.best_hypotheses(1):
 ```
 
 Lines marked with `(*)` are not strictly required as they happen to be
-the default choices. For now, you can use ModelAdaptor only for EvoLearner, CELOE and OCEL.
+the default choices. For now, you can use ModelAdapter only for EvoLearner, CELOE and OCEL.