diff --git a/README.md b/README.md index 3358982..a6f72ae 100644 --- a/README.md +++ b/README.md @@ -1,26 +1,26 @@ -[![Tests Neo4j 5.13](https://github.com/sg-dev/rel2graph/actions/workflows/tests_neo4j5.yaml/badge.svg)](https://github.com/sg-dev/rel2graph/actions/workflows/tests_neo4j5.yaml) -[![Python Versions](https://img.shields.io/badge/python-3.8%20%7C%C2%A03.9%C2%A0%7C%C2%A03.10%C2%A0%7C%203.11%C2%A0%7C%203.12-orange)](https://github.com/sg-dev/rel2graph/actions/workflows) +[![Tests Neo4j 5.13](https://github.com/sg-dev/Data2Neo/actions/workflows/tests_neo4j5.yaml/badge.svg)](https://github.com/sg-dev/Data2Neo/actions/workflows/tests_neo4j5.yaml) +[![Python Versions](https://img.shields.io/badge/python-3.8%20%7C%C2%A03.9%C2%A0%7C%C2%A03.10%C2%A0%7C%203.11%C2%A0%7C%203.12-orange)](https://github.com/sg-dev/Data2Neo/actions/workflows) ---

- rel2graph banner + Data2Neo banner

--- -**Rel2graph** is a library that simplifies the convertion of data in relational format to a graph knowledge database. It reliefs you of the cumbersome manual work of writing the conversion code and let's you focus on the conversion schema and data processing. +**Data2Neo** is a library that simplifies the convertion of data in relational format to a graph knowledge database. It reliefs you of the cumbersome manual work of writing the conversion code and let's you focus on the conversion schema and data processing. The library is built specifically for converting data into a [neo4j](https://neo4j.com/) graph (minimum version 5.2). The library further supports extensive customization capabilities to clean and remodel data. As neo4j python client it uses the native [neo4j python client](https://neo4j.com/docs/getting-started/languages-guides/neo4j-python/). - - [Latest Releases](https://github.com/sg-dev/rel2graph/tags) - - [Documentation](https://rel2graph.jkminder.ch) + - [Latest Releases](https://github.com/sg-dev/Data2Neo/tags) + - [Documentation](https://Data2Neo.jkminder.ch) This library has been developed at the [Chair of Systems Design at ETH Zürich](https://www.sg.ethz.ch). ## Installation If you have setup a private ssh key for your github, copy-paste the command below to install the latest version ([v1.3.3][latest_tag]): ``` -pip install git+ssh://git@github.com/sg-dev/rel2graph@v1.3.3 +pip install git+ssh://git@github.com/sg-dev/Data2Neo@v1.3.3 ``` If you don't have ssh set up, download the latest wheel [here][latest_wheel] and install the wheel with: @@ -32,7 +32,7 @@ If you have cloned the repository you can also build it locally with ``` pip install **path-to-repository** ``` -The rel2graph library supports Python 3.8+. +The Data2Neo library supports Python 3.8+. ## Quick Start A quick example for converting data in a [Pandas](https://pandas.pydata.org) dataframe into a graph. The full example code can be found under [examples](/examples). For more details, please checkout the [full documentation][wiki]. We first define a *convertion schema* in a YAML style config file. In this config file we specify, which entites are converted into which nodes and which relationships. @@ -56,17 +56,17 @@ ENTITY("Person"): RELATIONSHIP(person, "likes", MATCH("Species", Name=Person.FavoriteFlower)): - Since = "4ever" ``` -The library itself has 2 basic elements, that are required for the conversion: the `Converter` that handles the conversion itself and an `Iterator` that iterates over the relational data. The iterator can be implemented for arbitrary data in relational format. Rel2graph currently has preimplemented iterators under: -- `rel2graph.relational_modules.sqlite` for [SQLite](https://www.sqlite.org/index.html) databases -- `rel2graph.relational_modules.pandas` for [Pandas](https://pandas.pydata.org) dataframes +The library itself has 2 basic elements, that are required for the conversion: the `Converter` that handles the conversion itself and an `Iterator` that iterates over the relational data. The iterator can be implemented for arbitrary data in relational format. Data2Neo currently has preimplemented iterators under: +- `Data2Neo.relational_modules.sqlite` for [SQLite](https://www.sqlite.org/index.html) databases +- `Data2Neo.relational_modules.pandas` for [Pandas](https://pandas.pydata.org) dataframes -We will use the `PandasDataFrameIterator` from `rel2graph.relational_modules.pandas`. Further we will use the `IteratorIterator` that can wrap multiple iterators to handle multiple dataframes. Since a pandas dataframe has no type/table name associated, we need to specify the name when creating a `PandasDataFrameIterator`. We also define define a custom function `append` that can be refered to in the schema file and that appends a string to the attribute value. For an entity with `Flower["petal_width"] = 5`, the outputed node will have the attribute `petal_width = "5 milimeters"`. +We will use the `PandasDataFrameIterator` from `Data2Neo.relational_modules.pandas`. Further we will use the `IteratorIterator` that can wrap multiple iterators to handle multiple dataframes. Since a pandas dataframe has no type/table name associated, we need to specify the name when creating a `PandasDataFrameIterator`. We also define define a custom function `append` that can be refered to in the schema file and that appends a string to the attribute value. For an entity with `Flower["petal_width"] = 5`, the outputed node will have the attribute `petal_width = "5 milimeters"`. ```python import neo4j import pandas as pd -from rel2graph.relational_modules.pandas import PandasDataFrameIterator -from rel2graph import IteratorIterator, Converter, Attribute, register_attribute_postprocessor -from rel2graph.utils import load_file +from Data2Neo.relational_modules.pandas import PandasDataFrameIterator +from Data2Neo import IteratorIterator, Converter, Attribute, register_attribute_postprocessor +from Data2Neo.utils import load_file # Setup the neo4j uri and credentials uri = "bolt:localhost:7687" @@ -92,9 +92,9 @@ converter = Converter(load_file("schema.yaml"), iterator, uri, auth) converter() ``` # Known issues -If you encounter a bug or an unexplainable behavior, please check the [known issues](https://github.com/sg-dev/rel2graph/labels/bug) list. If your issue is not found, submit a new one. +If you encounter a bug or an unexplainable behavior, please check the [known issues](https://github.com/sg-dev/Data2Neo/labels/bug) list. If your issue is not found, submit a new one. [latest_version]: v1.3.3 -[latest_tag]: https://github.com/sg-dev/rel2graph/releases/tag/v1.3.3 -[latest_wheel]: https://github.com/sg-dev/rel2graph/releases/download/v1.3.3/rel2graph-1.3.3-py3-none-any.whl -[wiki]: https://rel2graph.jkminder.ch/index.html +[latest_tag]: https://github.com/sg-dev/Data2Neo/releases/tag/v1.3.3 +[latest_wheel]: https://github.com/sg-dev/Data2Neo/releases/download/v1.3.3/Data2Neo-1.3.3-py3-none-any.whl +[wiki]: https://Data2Neo.jkminder.ch/index.html diff --git a/rel2graph/__init__.py b/data2neo/__init__.py similarity index 100% rename from rel2graph/__init__.py rename to data2neo/__init__.py diff --git a/rel2graph/common_modules/__init__.py b/data2neo/common_modules/__init__.py similarity index 100% rename from rel2graph/common_modules/__init__.py rename to data2neo/common_modules/__init__.py diff --git a/rel2graph/common_modules/datetime.py b/data2neo/common_modules/datetime.py similarity index 100% rename from rel2graph/common_modules/datetime.py rename to data2neo/common_modules/datetime.py diff --git a/rel2graph/common_modules/types.py b/data2neo/common_modules/types.py similarity index 100% rename from rel2graph/common_modules/types.py rename to data2neo/common_modules/types.py diff --git a/rel2graph/core/__init__.py b/data2neo/core/__init__.py similarity index 100% rename from rel2graph/core/__init__.py rename to data2neo/core/__init__.py diff --git a/rel2graph/core/converter.py b/data2neo/core/converter.py similarity index 100% rename from rel2graph/core/converter.py rename to data2neo/core/converter.py diff --git a/rel2graph/core/factories/__init__.py b/data2neo/core/factories/__init__.py similarity index 70% rename from rel2graph/core/factories/__init__.py rename to data2neo/core/factories/__init__.py index ada2912..f2ad5bf 100644 --- a/rel2graph/core/factories/__init__.py +++ b/data2neo/core/factories/__init__.py @@ -2,7 +2,7 @@ # -*- coding: utf-8 -*- """ -Factory submodule of rel2graph. This module keeps track off all factories. +Factory submodule of data2neo. This module keeps track off all factories. authors: Julian Minder """ diff --git a/rel2graph/core/factories/factory.py b/data2neo/core/factories/factory.py similarity index 100% rename from rel2graph/core/factories/factory.py rename to data2neo/core/factories/factory.py diff --git a/rel2graph/core/factories/factory_wrappers.py b/data2neo/core/factories/factory_wrappers.py similarity index 100% rename from rel2graph/core/factories/factory_wrappers.py rename to data2neo/core/factories/factory_wrappers.py diff --git a/rel2graph/core/factories/matcher.py b/data2neo/core/factories/matcher.py similarity index 100% rename from rel2graph/core/factories/matcher.py rename to data2neo/core/factories/matcher.py diff --git a/rel2graph/core/factories/registrar.py b/data2neo/core/factories/registrar.py similarity index 100% rename from rel2graph/core/factories/registrar.py rename to data2neo/core/factories/registrar.py diff --git a/rel2graph/core/factories/resource.py b/data2neo/core/factories/resource.py similarity index 100% rename from rel2graph/core/factories/resource.py rename to data2neo/core/factories/resource.py diff --git a/rel2graph/core/global_state.py b/data2neo/core/global_state.py similarity index 100% rename from rel2graph/core/global_state.py rename to data2neo/core/global_state.py diff --git a/rel2graph/core/resource_iterator.py b/data2neo/core/resource_iterator.py similarity index 100% rename from rel2graph/core/resource_iterator.py rename to data2neo/core/resource_iterator.py diff --git a/rel2graph/core/schema_compiler.py b/data2neo/core/schema_compiler.py similarity index 99% rename from rel2graph/core/schema_compiler.py rename to data2neo/core/schema_compiler.py index 1ee3f37..64c0987 100644 --- a/rel2graph/core/schema_compiler.py +++ b/data2neo/core/schema_compiler.py @@ -30,7 +30,7 @@ class SchemaConfigException(ValueError): class SchemaConfigParser: """ - Parses the schema config file into rel2graph instructions that can directly be converted into factories and their arguments. Uses ply to + Parses the schema config file into data2neo instructions that can directly be converted into factories and their arguments. Uses ply to parse and lex the grammar. All t_{name} methods/attributes of the class are used by lex to parser the document into tokens. All p_{name} functions define the grammar for diff --git a/rel2graph/neo4j/__init__.py b/data2neo/neo4j/__init__.py similarity index 100% rename from rel2graph/neo4j/__init__.py rename to data2neo/neo4j/__init__.py diff --git a/rel2graph/neo4j/cypher.py b/data2neo/neo4j/cypher.py similarity index 100% rename from rel2graph/neo4j/cypher.py rename to data2neo/neo4j/cypher.py diff --git a/rel2graph/neo4j/encoder.py b/data2neo/neo4j/encoder.py similarity index 100% rename from rel2graph/neo4j/encoder.py rename to data2neo/neo4j/encoder.py diff --git a/rel2graph/neo4j/graph_elements.py b/data2neo/neo4j/graph_elements.py similarity index 99% rename from rel2graph/neo4j/graph_elements.py rename to data2neo/neo4j/graph_elements.py index 0cd3111..2633dc3 100644 --- a/rel2graph/neo4j/graph_elements.py +++ b/data2neo/neo4j/graph_elements.py @@ -452,7 +452,7 @@ class Node(PropertyDict, Subgraph): All positional arguments passed to the constructor are interpreted as labels and all keyword arguments as properties:: - >>> from rel2graph.neo4j import Node + >>> from data2neo.neo4j import Node >>> a = Node("Person", name="Alice") """ diff --git a/rel2graph/relational_modules/__init__.py b/data2neo/relational_modules/__init__.py similarity index 100% rename from rel2graph/relational_modules/__init__.py rename to data2neo/relational_modules/__init__.py diff --git a/rel2graph/relational_modules/pandas.py b/data2neo/relational_modules/pandas.py similarity index 100% rename from rel2graph/relational_modules/pandas.py rename to data2neo/relational_modules/pandas.py diff --git a/rel2graph/relational_modules/sqlite.py b/data2neo/relational_modules/sqlite.py similarity index 100% rename from rel2graph/relational_modules/sqlite.py rename to data2neo/relational_modules/sqlite.py diff --git a/rel2graph/utils.py b/data2neo/utils.py similarity index 85% rename from rel2graph/utils.py rename to data2neo/utils.py index a6818a3..25f2033 100644 --- a/rel2graph/utils.py +++ b/data2neo/utils.py @@ -2,7 +2,7 @@ # -*- coding: utf-8 -*- """ -Utility functions for working with rel2graph. +Utility functions for working with data2neo. authors: Julian Minder """ diff --git a/docs/requirements.txt b/docs/requirements.txt index ea961b7..360aa10 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -1,4 +1,4 @@ -rel2graph +data2neo sphinx sphinx_autodoc_typehints sphinx_rtd_theme diff --git a/docs/source/api/core.rst b/docs/source/api/core.rst index 8a09837..0c976cd 100644 --- a/docs/source/api/core.rst +++ b/docs/source/api/core.rst @@ -8,7 +8,7 @@ The main api and classes provided by the library. Converter ~~~~~~~~~ -.. autoclass:: rel2graph.Converter +.. autoclass:: data2neo.Converter :members: :show-inheritance: :special-members: __call__ @@ -16,7 +16,7 @@ Converter Resource ~~~~~~~~ -.. autoclass:: rel2graph.Resource +.. autoclass:: data2neo.Resource :members: :show-inheritance: :special-members: __getitem__,__setitem__,__repr__ @@ -24,7 +24,7 @@ Resource ResourceIterator ~~~~~~~~~~~~~~~~ -.. autoclass:: rel2graph.ResourceIterator +.. autoclass:: data2neo.ResourceIterator :members: :show-inheritance: :special-members: __iter__,__len__,__next__ @@ -32,14 +32,14 @@ ResourceIterator IteratorIterator ~~~~~~~~~~~~~~~~ -.. autoclass:: rel2graph.IteratorIterator +.. autoclass:: data2neo.IteratorIterator :members: :show-inheritance: Attribute ~~~~~~~~~ -.. autoclass:: rel2graph.Attribute +.. autoclass:: data2neo.Attribute :members: :noindex: :show-inheritance: @@ -47,7 +47,7 @@ Attribute SubgraphFactoryWrapper ~~~~~~~~~~~~~~~~~~~~~~ -.. autoclass:: rel2graph.SubgraphFactoryWrapper +.. autoclass:: data2neo.SubgraphFactoryWrapper :members: :noindex: :show-inheritance: @@ -55,7 +55,7 @@ SubgraphFactoryWrapper AttributeFactoryWrapper ~~~~~~~~~~~~~~~~~~~~~~~ -.. autoclass:: rel2graph.AttributeFactoryWrapper +.. autoclass:: data2neo.AttributeFactoryWrapper :members: :noindex: :show-inheritance: \ No newline at end of file diff --git a/docs/source/api/neo4j.rst b/docs/source/api/neo4j.rst index c2a8c7a..751f659 100644 --- a/docs/source/api/neo4j.rst +++ b/docs/source/api/neo4j.rst @@ -4,28 +4,28 @@ Neo4j Integration These functions abstract complexity of interacting with Neo4j. Instead of writing Cypher queries, you can use Python objects to create, merge and match nodes and relationships. -.. autofunction:: rel2graph.neo4j.create +.. autofunction:: data2neo.neo4j.create -.. autofunction:: rel2graph.neo4j.merge +.. autofunction:: data2neo.neo4j.merge -.. autofunction:: rel2graph.neo4j.push +.. autofunction:: data2neo.neo4j.push -.. autofunction:: rel2graph.neo4j.pull +.. autofunction:: data2neo.neo4j.pull -.. autofunction:: rel2graph.neo4j.match_nodes +.. autofunction:: data2neo.neo4j.match_nodes -.. autofunction:: rel2graph.neo4j.match_relationships +.. autofunction:: data2neo.neo4j.match_relationships Subgraph ~~~~~~~~ -.. autoclass:: rel2graph.neo4j.Subgraph +.. autoclass:: data2neo.neo4j.Subgraph :members: Node ~~~~ -.. autoclass:: rel2graph.neo4j.Node +.. autoclass:: data2neo.neo4j.Node :members: :show-inheritance: @@ -33,6 +33,6 @@ Node Relationship ~~~~~~~~~~~~ -.. autoclass:: rel2graph.neo4j.Relationship +.. autoclass:: data2neo.neo4j.Relationship :members: :show-inheritance: \ No newline at end of file diff --git a/docs/source/api/relational_modules.rst b/docs/source/api/relational_modules.rst index 2f5d62c..64deaed 100644 --- a/docs/source/api/relational_modules.rst +++ b/docs/source/api/relational_modules.rst @@ -8,13 +8,13 @@ They implement the abstract classes of the main interface. SQLite ====== -Implements the :py:class:`Resource ` and :py:class:`ResourceIterator ` for sqlite based on `sqlite3 `_. +Implements the :py:class:`Resource ` and :py:class:`ResourceIterator ` for sqlite based on `sqlite3 `_. SQLiteResource ~~~~~~~~~~~~~~ -.. autoclass:: rel2graph.relational_modules.sqlite.SQLiteResource +.. autoclass:: data2neo.relational_modules.sqlite.SQLiteResource :members: :show-inheritance: @@ -22,7 +22,7 @@ SQLiteResource SQLiteIterator ~~~~~~~~~~~~~~~~~~ -.. autoclass:: rel2graph.relational_modules.sqlite.SQLiteIterator +.. autoclass:: data2neo.relational_modules.sqlite.SQLiteIterator :members: :show-inheritance: @@ -30,12 +30,12 @@ SQLiteIterator Pandas ====== -Implements the :py:class:`Resource ` and :py:class:`ResourceIterator ` for `pandas `_. +Implements the :py:class:`Resource ` and :py:class:`ResourceIterator ` for `pandas `_. PandasSeriesResource ~~~~~~~~~~~~~~~~~~~~ -.. autoclass:: rel2graph.relational_modules.pandas.PandasSeriesResource +.. autoclass:: data2neo.relational_modules.pandas.PandasSeriesResource :members: :show-inheritance: @@ -43,6 +43,6 @@ PandasSeriesResource PandasDataFrameIterator ~~~~~~~~~~~~~~~~~~~~~~~ -.. autoclass:: rel2graph.relational_modules.pandas.PandasDataFrameIterator +.. autoclass:: data2neo.relational_modules.pandas.PandasDataFrameIterator :members: :show-inheritance: \ No newline at end of file diff --git a/docs/source/api/utils.rst b/docs/source/api/utils.rst index ec4a3c4..2940b96 100644 --- a/docs/source/api/utils.rst +++ b/docs/source/api/utils.rst @@ -2,7 +2,7 @@ Utils ----------------- -General utility functions for working with rel2graph. +General utility functions for working with Data2Neo. -.. autofunction:: rel2graph.utils.load_file \ No newline at end of file +.. autofunction:: data2neo.utils.load_file \ No newline at end of file diff --git a/docs/source/common_modules.rst b/docs/source/common_modules.rst index 51c5424..27195d9 100644 --- a/docs/source/common_modules.rst +++ b/docs/source/common_modules.rst @@ -1,14 +1,14 @@ Common modules ============== -The rel2graph library comes with some predefined wrappers. To use them you must import them from the ``rel2graph.common_modules`` module. The following wrappers are available: +The data2neo library comes with some predefined wrappers. To use them you must import them from the ``data2neo.common_modules`` module. The following wrappers are available: general ------- .. code-block:: python - import rel2graph.common_modules + import data2neo.common_modules **MERGE_RELATIONSHIPS** ~~~~~~~~~~~~~~~~~~~~~~~ @@ -37,7 +37,7 @@ types -------- .. code-block:: python - import rel2graph.common_modules.types + import data2neo.common_modules.types **INT** @@ -65,7 +65,7 @@ datetime -------- .. code-block:: python - import rel2graph.common_modules.datetime + import data2neo.common_modules.datetime **DATETIME** diff --git a/docs/source/conf.py b/docs/source/conf.py index b3a48d1..aa5fa6e 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -17,7 +17,7 @@ # -- Project information ----------------------------------------------------- -project = 'rel2graph' +project = 'data2neo' copyright = '2024, Julian Minder' author = 'Julian Minder' @@ -76,7 +76,7 @@ # so a file named "default.css" will overwrite the builtin "default.css". html_static_path = ['_static'] -html_logo = "./assets/images/rel2graph_banner.png" +html_logo = "./assets/images/data2neo_banner.png" html_theme_options = { 'logo_only': True, 'display_version': False, diff --git a/docs/source/conversion_schema.rst b/docs/source/conversion_schema.rst index 3c3de57..7860d39 100644 --- a/docs/source/conversion_schema.rst +++ b/docs/source/conversion_schema.rst @@ -207,8 +207,8 @@ Assuming you have defined the attributewrappers ``ATTRWRAPPER1`` and ``ATTRWRAPP Note that the library does no semantic checking of your schema. If you apply an attribute wrapper to a node or a relationship, the outcome is undefined and might result in unexpected behaviour/exceptions during runtime. -.. |Resource| replace:: :py:class:`Resource ` -.. |Converter| replace:: :py:class:`Converter ` -.. |ResourceIterator| replace:: :py:class:`ResourceIterator ` +.. |Resource| replace:: :py:class:`Resource ` +.. |Converter| replace:: :py:class:`Converter ` +.. |ResourceIterator| replace:: :py:class:`ResourceIterator ` .. _neo4j: https://neo4j.com/ .. _py2neo: https://py2neo.org/2021.1/index.html diff --git a/docs/source/converter.rst b/docs/source/converter.rst index 4e32064..88b4d53 100644 --- a/docs/source/converter.rst +++ b/docs/source/converter.rst @@ -6,7 +6,7 @@ It is initialised with the *conversion schema* as a string, the iterator, the gr .. code-block:: python - from rel2graph import Converter + from data2neo import Converter converter = Converter(conversion_schema, iterator, uri, auth) @@ -22,8 +22,8 @@ The progress bar class must be an instance of the `tqdm .. code-block:: python - from rel2graph import Converter - from rel2graph.utils import load_file + from data2neo import Converter + from data2neo.utils import load_file from tqdm import tqdm converter = Converter(load_file(conversion_schema_file), iterator, uri, auth) @@ -70,7 +70,7 @@ For converting strings to datetime/date the library provides some predefined wra Global Shared State ~~~~~~~~~~~~~~~~~~~ -If you need to share state between your wrappers you must notify rel2graph explicitly about this. An example of this is a wrapper that needs to keep track of the number of resources it has processed. Because the |Converter| uses multiple workers +If you need to share state between your wrappers you must notify data2neo explicitly about this. An example of this is a wrapper that needs to keep track of the number of resources it has processed. Because the |Converter| uses multiple workers the wrapper may be called in different processes and the state is not shared between the processes (note that parallel processes do not share the same memory). To share state between the processes you need to use the ``GlobalSharedState``. The ``GlobalSharedState`` is a singleton class that can be used to share state between the processes. Before calling the |Converter| you need to register your state with the ``GlobalSharedState`` by simply defining an attribute on it ``GlobalSharedState.my_state = my_state``. By default the ``GlobalSharedState`` will provide the `neo4j driver `_ object to every process under ``GlobalSharedState.graph_driver``. @@ -79,7 +79,7 @@ Other options include ``multiprocessing.Array``, ``multiprocessing.Queue``, ``mu .. code-block:: python - from rel2graph import GlobalSharedState, register_subgraph_preprocessor + from data2neo import GlobalSharedState, register_subgraph_preprocessor @register_subgraph_preprocessor @@ -114,14 +114,14 @@ Other options include ``multiprocessing.Array``, ``multiprocessing.Queue``, ``mu Logging ~~~~~~~ -The whole rel2graph library uses the standard python `logging `_ library. +The whole data2neo library uses the standard python `logging `_ library. See an example of how to use it below. For more information, check out the `official documentation `_. .. code-block:: python import logging - logger = logging.getLogger("rel2graph") # Get Logger + logger = logging.getLogger("data2neo") # Get Logger logger.setLevel(logging.DEBUG) # Set the log level to DEBUG log_formatter = logging.Formatter("%(asctime)s [%(threadName)s]::[%(levelname)s]::%(filename)s: %(message)s") # Specify the format console_handler = logging.StreamHandler() # Create console handler (will output directly to console) @@ -160,9 +160,9 @@ With this: RELATIONSHIP(source, "TO", target): -.. |Resource| replace:: :py:class:`Resource ` -.. |Converter| replace:: :py:class:`Converter ` -.. |ResourceIterator| replace:: :py:class:`ResourceIterator ` +.. |Resource| replace:: :py:class:`Resource ` +.. |Converter| replace:: :py:class:`Converter ` +.. |ResourceIterator| replace:: :py:class:`ResourceIterator ` .. _neo4j: https://neo4j.com/ .. _py2neo: https://py2neo.org/2021.1/index.html .. _tqdm: https://tqdm.github.io diff --git a/docs/source/guides/update1.0.0.rst b/docs/source/guides/update1.0.0.rst index 871edec..6a12a0b 100644 --- a/docs/source/guides/update1.0.0.rst +++ b/docs/source/guides/update1.0.0.rst @@ -1,7 +1,7 @@ Update from pre 1.0.0 ----------------------------- -If you have been using rel2graph prior to version 1.0.0, you will need to update your code. The following changes have been made: +If you have been using Data2Neo prior to version 1.0.0, you will need to update your code. The following changes have been made: RELATION to RELATIONSHIP ~~~~~~~~~~~~~~~~~~~~~~~~ @@ -12,7 +12,7 @@ Simply replace all instances of `RELATION` with `RELATIONSHIP` in your schema. Py2neo to native neo4j driver ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -With the release of rel2graph 1.0.0, the underlying driver for neo4j has been changed from py2neo to the native neo4j driver. Py2neo has been deprecated and is no longer supported. +With the release of Data2Neo 1.0.0, the underlying driver for neo4j has been changed from py2neo to the native neo4j driver. Py2neo has been deprecated and is no longer supported. This means that you will need to update your code to use the new driver. The following changes have been made: 1. The |Converter| now takes a uri and credentials instead of a graph object. @@ -21,13 +21,13 @@ This means that you will need to update your code to use the new driver. The fol # Old from py2neo import Graph - from rel2graph import Converter + from data2neo import Converter graph = Graph("bolt://localhost:7687", auth=("neo4j", "password")) converter = Converter(schema, iterator, graph) # New - from rel2graph import Converter + from data2neo import Converter import neo4j uri = "bolt://localhost:7687" @@ -35,13 +35,13 @@ This means that you will need to update your code to use the new driver. The fol converter = Converter(schema, iterator, uri, auth) -2. If you use wrappers that operate on the py2neo graph object, you will need to update them to use the native neo4j driver. The :ref:`Global Shared State ` provides the neo4j driver with ``GlobalSharedState.graph_driver``. Further the :doc:`neo4j <../neo4j>` module provides the |Node|, |Relationship| and |Subgraph| classes, similar to py2neo. To push a subgraph to the neo4j database, use the :py:func:`create ` and :py:func:`merge ` methods. +2. If you use wrappers that operate on the py2neo graph object, you will need to update them to use the native neo4j driver. The :ref:`Global Shared State ` provides the neo4j driver with ``GlobalSharedState.graph_driver``. Further the :doc:`neo4j <../neo4j>` module provides the |Node|, |Relationship| and |Subgraph| classes, similar to py2neo. To push a subgraph to the neo4j database, use the :py:func:`create ` and :py:func:`merge ` methods. .. code-block:: python # Old from py2neo import Graph - from rel2graph import register_subgraph_postprocessor + from data2neo import register_subgraph_postprocessor from py2neo import Node @register_subgraph_postprocessor @@ -58,8 +58,8 @@ This means that you will need to update your code to use the new driver. The fol return subgraph # New - from rel2graph import register_subgraph_postprocessor, GlobalSharedState - from rel2graph.neo4j import create, Node, Subgraph, Relationship + from data2neo import register_subgraph_postprocessor, GlobalSharedState + from data2neo.neo4j import create, Node, Subgraph, Relationship @register_subgraph_postprocessor def add_node(subgraph): @@ -78,12 +78,12 @@ This means that you will need to update your code to use the new driver. The fol -.. |Resource| replace:: :py:class:`Resource ` -.. |Converter| replace:: :py:class:`Converter ` -.. |ResourceIterator| replace:: :py:class:`ResourceIterator ` -.. |Attribute| replace:: :py:class:`Attribute ` -.. |SubgraphWrapper| replace:: :py:class:`SubgraphFactoryWrapper ` -.. |AttributeWrapper| replace:: :py:class:`AttributeFactoryWrapper ` -.. |Subgraph| replace:: :py:class:`Subgraph ` -.. |Node| replace:: :py:class:`Node ` -.. |Relationship| replace:: :py:class:`Relationship ` \ No newline at end of file +.. |Resource| replace:: :py:class:`Resource ` +.. |Converter| replace:: :py:class:`Converter ` +.. |ResourceIterator| replace:: :py:class:`ResourceIterator ` +.. |Attribute| replace:: :py:class:`Attribute ` +.. |SubgraphWrapper| replace:: :py:class:`SubgraphFactoryWrapper ` +.. |AttributeWrapper| replace:: :py:class:`AttributeFactoryWrapper ` +.. |Subgraph| replace:: :py:class:`Subgraph ` +.. |Node| replace:: :py:class:`Node ` +.. |Relationship| replace:: :py:class:`Relationship ` \ No newline at end of file diff --git a/docs/source/index.rst b/docs/source/index.rst index 106894f..bc2b77c 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -1,12 +1,12 @@ -.. rel2graph documentation master file, created by +.. data2neo documentation master file, created by sphinx-quickstart on Wed Apr 20 19:25:41 2022. You can adapt this file completely to your liking, but it should at least contain the root `toctree` directive. -Welcome to **rel2graph**'s documentation! +Welcome to **data2neo**'s documentation! ========================================= -**rel2graph** is a library that simplifies the conversion of data in relational format +**Data2Neo** is a library that simplifies the conversion of data in relational format to a graph knowledge database. It relieves you of the cumbersome manual work of writing the conversion code and lets you focus on the conversion schema and data processing. @@ -50,8 +50,8 @@ Indices and tables * :ref:`modindex` * :ref:`search` -.. |Resource| replace:: :py:class:`Resource ` -.. |Converter| replace:: :py:class:`Converter ` -.. |ResourceIterator| replace:: :py:class:`ResourceIterator ` +.. |Resource| replace:: :py:class:`Resource ` +.. |Converter| replace:: :py:class:`Converter ` +.. |ResourceIterator| replace:: :py:class:`ResourceIterator ` .. _neo4j: https://neo4j.com/ .. _py2neo: https://py2neo.org/2021.1/index.html diff --git a/docs/source/installation.rst b/docs/source/installation.rst index 222f871..517c1ba 100644 --- a/docs/source/installation.rst +++ b/docs/source/installation.rst @@ -9,13 +9,13 @@ To install the software, follow these steps: .. code-block:: bash - git clone git@github.com:sg-dev/rel2graph.git + git clone git@github.com:sg-dev/data2neo.git 2. Change to the project directory: .. code-block:: bash - cd rel2graph + cd data2neo 3. Install the library: diff --git a/docs/source/introduction.rst b/docs/source/introduction.rst index ed9773f..e3482b1 100644 --- a/docs/source/introduction.rst +++ b/docs/source/introduction.rst @@ -1,9 +1,9 @@ Introduction ============ -This chapter will give you an overview of how *rel2graph* works and a first intuition on +This chapter will give you an overview of how *Data2Neo* works and a first intuition on how to interact with it. Details on how the individual parts of the library work can be -found in later chapters. At the heart of rel2graph is the concept of a factory that +found in later chapters. At the heart of Data2Neo is the concept of a factory that converts an input (relational data) into an output (a neo4j_ graph). The factory input is called a |Resource| which can represent @@ -18,10 +18,10 @@ must connect to their data source. .. image:: assets/images/overview.png :width: 800 - :alt: rel2graph overview + :alt: Data2Neo overview The library supports the use of different resource types, -allowing the user to process various entities. Rel2graph then +allowing the user to process various entities. Data2Neo then constructs one factory per resource type. At the core of the library lies the |Converter|. The |Converter| is the main object the developer interacts with and is responsible for constructing and @@ -39,7 +39,7 @@ it. The |Converter| iteratively asks the |ResourceIterator| for the next resource until the iterator reports no more resources to process. Lastly, it is possible to extend the factories by injecting custom pre- or postprocessing functions, called Wrappers, into -the rel2graph pipeline. :doc:`Wrappers ` allow the user to specify +the Data2Neo pipeline. :doc:`Wrappers ` allow the user to specify arbitrary procedures that are applied to a Resource just before it is passed to the wrapped factory, or to the immediate output of the factory before it is passed back to the Converter. @@ -48,12 +48,12 @@ complex data integration pipelines. :doc:`Read more here. ` .. image:: assets/images/wrapper.png :width: 800 - :alt: rel2graph wrapper + :alt: Data2Neo wrapper -.. |Resource| replace:: :py:class:`Resource ` -.. |Converter| replace:: :py:class:`Converter ` -.. |ResourceIterator| replace:: :py:class:`ResourceIterator ` +.. |Resource| replace:: :py:class:`Resource ` +.. |Converter| replace:: :py:class:`Converter ` +.. |ResourceIterator| replace:: :py:class:`ResourceIterator ` .. |convschema| replace:: :doc:`conversion schema ` .. _neo4j: https://neo4j.com/ .. _py2neo: https://py2neo.org/2021.1/index.html diff --git a/docs/source/neo4j.rst b/docs/source/neo4j.rst index 33c4ec9..39c2326 100644 --- a/docs/source/neo4j.rst +++ b/docs/source/neo4j.rst @@ -1,12 +1,12 @@ Neo4j Integration ================= -The rel2graph library comes with a set of abstract classes that simplify the interaction with neo4j in python. They are derived from the now EOL library py2neo. +The Data2Neo library comes with a set of abstract classes that simplify the interaction with neo4j in python. They are derived from the now EOL library py2neo. This includes python objects to represent |Node| and |Relationship| objects as well as a |Subgraph| object that can be used to represent a set of nodes and relationships. -|Node| and |Relationship| objects are themself a |Subgraph|. The functions :py:func:`create ` and :py:func:`merge ` can be used to create or merge a |Subgraph| into a neo4j database given a neo4j session. To sync local -|Subgraph| objects with the database, use the :py:func:`push ` and :py:func:`pull ` functions.Further use the functions :py:func:`match_nodes ` and :py:func:`match_relationships ` to match elements in the graph and return a list of |Node| or |Relationship|. +|Node| and |Relationship| objects are themself a |Subgraph|. The functions :py:func:`create ` and :py:func:`merge ` can be used to create or merge a |Subgraph| into a neo4j database given a neo4j session. To sync local +|Subgraph| objects with the database, use the :py:func:`push ` and :py:func:`pull ` functions.Further use the functions :py:func:`match_nodes ` and :py:func:`match_relationships ` to match elements in the graph and return a list of |Node| or |Relationship|. We refer to the :doc:`neo4j documentation ` for more information. -.. |Subgraph| replace:: :py:class:`Subgraph ` -.. |Node| replace:: :py:class:`Node ` -.. |Relationship| replace:: :py:class:`Relationship ` +.. |Subgraph| replace:: :py:class:`Subgraph ` +.. |Node| replace:: :py:class:`Node ` +.. |Relationship| replace:: :py:class:`Relationship ` diff --git a/docs/source/quick_start.rst b/docs/source/quick_start.rst index 46940ac..88125a8 100644 --- a/docs/source/quick_start.rst +++ b/docs/source/quick_start.rst @@ -5,13 +5,13 @@ Quick Start This is a quick start guide for the library. For more details, please checkout the full documentation. -As a first example, we will convert the **Northwind Traders** dataset into a Neo4j graph. The complete example code can be found `here `_. +As a first example, we will convert the **Northwind Traders** dataset into a Neo4j graph. The complete example code can be found `here `_. The official Neo4j documentation has a `tutorial `_ on how to import the Northwind Traders dataset using **direct import** with CYPHER. -We will use the same dataset, but instead of using CYPHER, we will use the rel2graph library to convert the data into a graph. +We will use the same dataset, but instead of using CYPHER, we will use the Data2Neo library to convert the data into a graph. -Rel2graph decomposes the data integration process into three distinct parts: +Data2Neo decomposes the data integration process into three distinct parts: -- **Resource Abstraction:** We encapsulate each relational entity in a |Resource|, with each |Resource| possessing a type corresponding to, for example, its table name. A |ResourceIterator| is employed to iterate over all Resources to be converted. This abstraction provides us with the flexibility to accommodate various data sources. Rel2graph currently has pre-implemented iterators for :ref:`SQLite databases ` and :ref:`pandas Dataframes `. +- **Resource Abstraction:** We encapsulate each relational entity in a |Resource|, with each |Resource| possessing a type corresponding to, for example, its table name. A |ResourceIterator| is employed to iterate over all Resources to be converted. This abstraction provides us with the flexibility to accommodate various data sources. Data2Neo currently has pre-implemented iterators for :ref:`SQLite databases ` and :ref:`pandas Dataframes `. - **Conversion Schema:** The :doc:`Convertion Schema ` outlines how each Resource type should be managed, effectively serving as a blueprint for integrating each Resource type. - **Converter:** The |Converter| takes data from the |ResourceIterator| and integrates it into an existing knowledge graph, based on the specifications provided in the :doc:`Convertion Schema `. @@ -55,18 +55,18 @@ The relationship should have the attributes **unitPrice** and **quantity**. The RELATIONSHIP(MATCH("Employee", employeeID = Orders.EmployeeID), "SOLD", order): Refer to the :doc:`conversion schema chapter ` for more details on the syntax. -The full schema for all tables can be found `here `_. If performance is an issue, refer to the :ref:`performance chapter ` for more details on how to optimize the schema. +The full schema for all tables can be found `here `_. If performance is an issue, refer to the :ref:`performance chapter ` for more details on how to optimize the schema. Note that we have used the keywords ``FLOAT`` and ``INT`` in the schema. These are so called :doc:`wrappers `. Wrappers can be used to insert arbitrary Python code into the conversion process, in this case to convert the data to the correct datatype. -Fortunatly, the library comes with a set of :doc:`pre-implemented wrappers `. To use the ``INT`` and ``FLOAT`` wrappers we simply need to import ``import rel2graph.common_modules.types``. +Fortunatly, the library comes with a set of :doc:`pre-implemented wrappers `. To use the ``INT`` and ``FLOAT`` wrappers we simply need to import ``import data2neo.common_modules.types``. -We can also define our Python functions and register them with rel2graph. As an example, let's consider a scenario where the UnitPrice column in the Orders table contains values in different currencies. +We can also define our Python functions and register them with Data2Neo. As an example, let's consider a scenario where the UnitPrice column in the Orders table contains values in different currencies. We can define a function, CONVERTCURRENCY, to standardise these values into a single currency, register it, and then use it in the conversion schema. :doc:`Read more about wrappers here `. .. code-block:: python - from rel2graph import register_attribute_postprocessor, Attribute + from Data2Neo import register_attribute_postprocessor, Attribute @register_attribute_postprocessor def CONVERTCURRENCY(attribute, currency): @@ -87,8 +87,8 @@ We can define a function, CONVERTCURRENCY, to standardise these values into a si Finally, we need to create a |ResourceIterator| that iterates over the data in the Northwind database. -We will use the :py:class:`PandasDataFrameIterator ` from ``rel2graph.relational_modules.pandas``. We will also use the :py:class:`IteratorIterator `, which can wrap multiple iterators to handle multiple dataframes. -Since pandas DataFrames have no types/table names associated with it, we need to specify the name when creating a :py:class:`PandasDataFrameIterator `. +We will use the :py:class:`PandasDataFrameIterator ` from ``data2neo.relational_modules.pandas``. We will also use the :py:class:`IteratorIterator `, which can wrap multiple iterators to handle multiple dataframes. +Since pandas DataFrames have no types/table names associated with it, we need to specify the name when creating a :py:class:`PandasDataFrameIterator `. The Python code looks like this: .. code-block:: python @@ -97,12 +97,12 @@ The Python code looks like this: import pandas as pd from tqdm.notebook import tqdm - from rel2graph.relational_modules.pandas import PandasDataFrameIterator - from rel2graph import IteratorIterator - from rel2graph import Converter - from rel2graph.utils import load_file + from data2neo.relational_modules.pandas import PandasDataFrameIterator + from data2neo import IteratorIterator + from data2neo import Converter + from data2neo.utils import load_file - import rel2graph.common_modules.types # For FLOAT, INT, etc. wrappers + import data2neo.common_modules.types # For FLOAT, INT, etc. wrappers # This is required because the pandas dataframe iterator will convert all values # to int64 which is not supported by neo4j @@ -126,8 +126,8 @@ The Python code looks like this: # Start the conversion converter(progress_bar=tqdm) -`Full Code Example `_ +`Full Code Example `_ -.. |Resource| replace:: :py:class:`Resource ` -.. |Converter| replace:: :py:class:`Converter ` -.. |ResourceIterator| replace:: :py:class:`ResourceIterator ` \ No newline at end of file +.. |Resource| replace:: :py:class:`Resource ` +.. |Converter| replace:: :py:class:`Converter ` +.. |ResourceIterator| replace:: :py:class:`ResourceIterator ` \ No newline at end of file diff --git a/docs/source/resource.rst b/docs/source/resource.rst index 526ca75..b51ab9e 100644 --- a/docs/source/resource.rst +++ b/docs/source/resource.rst @@ -4,7 +4,7 @@ Customising Resource and ResourceIterator The |Resource| and |ResourceIterator| classes are abstract classes and need to be implemented for your specific relational input data type. The library comes with implementations for a few :ref:`existing relational modules `. If you data type is not supported, you can easily create implementations for your data; -see below for an explanation or check out `one of the already implemented modules `_. +see below for an explanation or check out `one of the already implemented modules `_. If you think your implementations could be helpful for others as well, read through the chapter :doc:`Information for Developers ` and create a pull request. @@ -17,7 +17,7 @@ The parent constructor must be called with no arguments. Additionally, you need .. code-block:: python - from rel2graph import Resource + from data2neo import Resource class MyResource(Resource): def __init__(...your constructor parameters...): @@ -74,7 +74,7 @@ Note that your iterator can also traverse resources of different types. .. code-block:: python - from rel2graph import ResourceIterator + from data2neo import ResourceIterator class MyIterator(ResourceIterator): def __init__(self, ...your constructor parameters...) -> None: @@ -112,7 +112,7 @@ There are no restrictions on the "sub"-iterators, as long as they are of type |R .. code-block:: python - from rel2graph import IteratorIterator + from data2neo import IteratorIterator iterator1 = ... # An iterator iterator2 = ... # Another iterator itit = IteratorIterator([iterator1, iterator2]) @@ -124,39 +124,39 @@ Existing relational modules Pandas ------ -With the :py:class:`PandasDataFrameIterator ` you can wrap a `pandas dataframe `_. -If you pass a pandas dataframe to the :py:class:`PandasDataFrameIterator ` it will automatically create :py:class:`PandasSeriesResource ` out of all rows (series) and iterate over them. +With the :py:class:`PandasDataFrameIterator ` you can wrap a `pandas dataframe `_. +If you pass a pandas dataframe to the :py:class:`PandasDataFrameIterator ` it will automatically create :py:class:`PandasSeriesResource ` out of all rows (series) and iterate over them. Since a dataframe has no type associated, you need to also provide a type name. .. code-block:: python - from rel2graph.relational_modules.pandas import PandasDataFrameIterator + from data2neo.relational_modules.pandas import PandasDataFrameIterator iterator = PandasDataFrameIterator(pandas.DataFrame(...), "MyType") SQLite ------ -With the :py:class:`SQLiteIterator ` you can iterate over a sqlite database. You need to provide a connection to the database. -You can also provide a list of tables to iterate over. If you do not provide a list of tables, the iterator will iterate over all tables in the database. Rel2graph requires primary keys, so if your tables do not have primary keys, you need to provide a dictionary with table, primary key pairs. +With the :py:class:`SQLiteIterator ` you can iterate over a sqlite database. You need to provide a connection to the database. +You can also provide a list of tables to iterate over. If you do not provide a list of tables, the iterator will iterate over all tables in the database. data2neo requires primary keys, so if your tables do not have primary keys, you need to provide a dictionary with table, primary key pairs. By default the Iterator will mix all tables together. If you want to iterate over tables one after another, you can set the ``mix_tables`` parameter to ``False``. -The python implementation of sqlite will often throw warnings if a new process is spawned. You can disable these warnings by setting the ``check_same_thread`` parameter to ``False``. Rel2graph does not share the connection between processes, only the master processes requests data from the database. +The python implementation of sqlite will often throw warnings if a new process is spawned. You can disable these warnings by setting the ``check_same_thread`` parameter to ``False``. data2neo does not share the connection between processes, only the master processes requests data from the database. .. code-block:: python - from rel2graph.relational_modules.sqlite import SQLiteIterator + from data2neo.relational_modules.sqlite import SQLiteIterator import sqlite3 connection = sqlite3.connect("mydatabase.db", check_same_thread=False) iterator = SQLiteIterator(connection, filter=["table1", "table2"], primary_keys={"table3": "id"}) -.. |Resource| replace:: :py:class:`Resource ` -.. |Converter| replace:: :py:class:`Converter ` -.. |ResourceIterator| replace:: :py:class:`ResourceIterator ` -.. |IteratorIterator| replace:: :py:class:`IteratorIterator ` +.. |Resource| replace:: :py:class:`Resource ` +.. |Converter| replace:: :py:class:`Converter ` +.. |ResourceIterator| replace:: :py:class:`ResourceIterator ` +.. |IteratorIterator| replace:: :py:class:`IteratorIterator ` .. _neo4j: https://neo4j.com/ .. _py2neo: https://py2neo.org/2021.1/index.html diff --git a/docs/source/wrapper.rst b/docs/source/wrapper.rst index 93a167c..49aaf73 100644 --- a/docs/source/wrapper.rst +++ b/docs/source/wrapper.rst @@ -15,7 +15,7 @@ This chapter will guide you through the creation of your own wrappers. Background ~~~~~~~~~~ -First, we need to be aware of the different factory types *rel2graph* uses. +First, we need to be aware of the different factory types *Data2Neo* uses. The input of every factory is always a resource, but depending on the type the output varies. To write wrappers, we need to distinguish the two main factory types: **SubgraphFactories** and **AttributeFactories**. @@ -30,7 +30,7 @@ AttributeFactories .. code-block:: python - from rel2graph import Attribute + from data2neo import Attribute myattr = Attribute("mykey", "myvalue") key = myattr.key # get the key of the attribute value = myattr.value # get the value of the attribute @@ -75,7 +75,7 @@ The library will not check if your registered functions/classes match the expect If the function behaves other than expected, this will result in undefined behaviour during runtime. Make sure you define your functions/classes correctly. -**Attention**: If you require shared state between wrappers or wrapper calls you need to explicitly tell rel2graph, e.g. a wrapper that counts the number of processed resources. Read more about it in the :ref:`Global Shared State ` chapter. +**Attention**: If you require shared state between wrappers or wrapper calls you need to explicitly tell Data2Neo, e.g. a wrapper that counts the number of processed resources. Read more about it in the :ref:`Global Shared State ` chapter. Preprocessors ~~~~~~~~~~~~~ @@ -93,7 +93,7 @@ Some examples: .. code-block:: python - from rel2graph import register_attribute_preprocessor, register_subgraph_preprocessor + from data2neo import register_attribute_preprocessor, register_subgraph_preprocessor @register_attribute_preprocessor def my_attr_preprocessor(resource: Resource) -> Resource: @@ -129,7 +129,7 @@ Some examples: .. code-block:: python - from rel2graph import register_attribute_postprocessor, register_subgraph_postprocessor, Attribute + from data2neo import register_attribute_postprocessor, register_subgraph_postprocessor, Attribute @register_attribute_postprocessor def attr_append_postprocessor(attribute: Attribute, value=" appendix": str) -> Attribute: @@ -170,7 +170,7 @@ Obviously, everything that can be done with pre/postprocessor functions can also .. code-block:: python - from rel2graph import SubGraphFactoryWrapper, register_wrapper + from data2neo import SubGraphFactoryWrapper, register_wrapper @register_wrapper class REQUIRED(SubgraphFactoryWrapper): @@ -196,15 +196,15 @@ schema.yaml REQUIRED(RELATIONSHIP(from, "relationship type", MATCH("other", key="value")), "No match for label other and key=value"): -.. |Resource| replace:: :py:class:`Resource ` -.. |Converter| replace:: :py:class:`Converter ` -.. |ResourceIterator| replace:: :py:class:`ResourceIterator ` -.. |Attribute| replace:: :py:class:`Attribute ` -.. |SubgraphWrapper| replace:: :py:class:`SubgraphFactoryWrapper ` -.. |AttributeWrapper| replace:: :py:class:`AttributeFactoryWrapper ` -.. |Subgraph| replace:: :py:class:`Subgraph ` -.. |Node| replace:: :py:class:`Node ` -.. |Relationship| replace:: :py:class:`Relationship ` +.. |Resource| replace:: :py:class:`Resource ` +.. |Converter| replace:: :py:class:`Converter ` +.. |ResourceIterator| replace:: :py:class:`ResourceIterator ` +.. |Attribute| replace:: :py:class:`Attribute ` +.. |SubgraphWrapper| replace:: :py:class:`SubgraphFactoryWrapper ` +.. |AttributeWrapper| replace:: :py:class:`AttributeFactoryWrapper ` +.. |Subgraph| replace:: :py:class:`Subgraph ` +.. |Node| replace:: :py:class:`Node ` +.. |Relationship| replace:: :py:class:`Relationship ` .. _neo4j: https://neo4j.com/ .. _py2neo: https://py2neo.org/2021.1/index.html diff --git a/examples/example_iris.ipynb b/examples/example_iris.ipynb index 52953bf..4b5f731 100644 --- a/examples/example_iris.ipynb +++ b/examples/example_iris.ipynb @@ -22,13 +22,13 @@ "import neo4j\n", "import pandas as pd\n", "\n", - "from rel2graph.relational_modules.pandas import PandasDataFrameIterator\n", - "from rel2graph import IteratorIterator\n", - "from rel2graph import Converter\n", - "from rel2graph.utils import load_file\n", - "from rel2graph import register_attribute_postprocessor, Attribute\n", + "from data2neo.relational_modules.pandas import PandasDataFrameIterator\n", + "from data2neo import IteratorIterator\n", + "from data2neo import Converter\n", + "from data2neo.utils import load_file\n", + "from data2neo import register_attribute_postprocessor, Attribute\n", "\n", - "import rel2graph.common_modules.types # For FLOAT, INT, etc. wrappers\n", + "import data2neo.common_modules.types # For FLOAT, INT, etc. wrappers\n", "# This is required because the pandas dataframe iterator will convert all values \n", "# to int64 which is not supported by neo4j" ] @@ -43,7 +43,7 @@ "import logging\n", "\n", "#logging.basicConfig(level=logging.WARNING)\n", - "logger = logging.getLogger(\"rel2graph\")\n", + "logger = logging.getLogger(\"data2neo\")\n", "logger.setLevel(logging.INFO)\n", "log_formatter = logging.Formatter(\"%(asctime)s [%(threadName)s]::[%(levelname)s]::%(filename)s: %(message)s\")\n", "console_handler = logging.StreamHandler()\n", diff --git a/examples/example_northwind.ipynb b/examples/example_northwind.ipynb index 573a878..a912d2d 100644 --- a/examples/example_northwind.ipynb +++ b/examples/example_northwind.ipynb @@ -22,13 +22,13 @@ "import neo4j\n", "import pandas as pd\n", "\n", - "from rel2graph.relational_modules.pandas import PandasDataFrameIterator\n", - "from rel2graph import IteratorIterator\n", - "from rel2graph import Converter\n", - "from rel2graph.utils import load_file\n", - "from rel2graph import register_subgraph_preprocessor\n", + "from data2neo.relational_modules.pandas import PandasDataFrameIterator\n", + "from data2neo import IteratorIterator\n", + "from data2neo import Converter\n", + "from data2neo.utils import load_file\n", + "from data2neo import register_subgraph_preprocessor\n", "\n", - "import rel2graph.common_modules.types # For FLOAT, INT, etc. wrappers\n", + "import data2neo.common_modules.types # For FLOAT, INT, etc. wrappers\n", "# This is required because the pandas dataframe iterator will convert all values \n", "# to int64 which is not supported by neo4j" ] @@ -43,7 +43,7 @@ "import logging\n", "\n", "#logging.basicConfig(level=logging.WARNING)\n", - "logger = logging.getLogger(\"rel2graph\")\n", + "logger = logging.getLogger(\"data2neo\")\n", "logger.setLevel(logging.INFO)\n", "log_formatter = logging.Formatter(\"%(asctime)s [%(threadName)s]::[%(levelname)s]::%(filename)s: %(message)s\")\n", "console_handler = logging.StreamHandler()\n", diff --git a/setup.py b/setup.py index 669d1a6..fc48b88 100644 --- a/setup.py +++ b/setup.py @@ -7,13 +7,13 @@ ] setup( - name = "rel2graph", + name = "data2neo", packages = find_packages(), - version = "1.3.3", + version = "1.4.0", description = "Library for converting relational data into graph data (neo4j)", author = "Julian Minder", author_email = "jminder@ethz.ch", - url = "github.com/sg-dev/rel2graph", + url = "https://github.com/jkminder/data2neo", install_requires = requirements, python_requires = ">=3.8" ) diff --git a/tests/integration/helpers.py b/tests/integration/helpers.py index 870f080..804f46a 100644 --- a/tests/integration/helpers.py +++ b/tests/integration/helpers.py @@ -5,8 +5,8 @@ import os import pytest -from rel2graph import register_subgraph_postprocessor -from rel2graph.neo4j import Node, Relationship, Subgraph +from data2neo import register_subgraph_postprocessor +from data2neo.neo4j import Node, Relationship, Subgraph import warnings diff --git a/tests/integration/test_concurrency.py b/tests/integration/test_concurrency.py index 8e8125b..9fce494 100644 --- a/tests/integration/test_concurrency.py +++ b/tests/integration/test_concurrency.py @@ -13,11 +13,11 @@ import logging import numpy as np -from rel2graph import Converter, IteratorIterator, register_attribute_postprocessor, Attribute, register_subgraph_preprocessor, GlobalSharedState, register_subgraph_postprocessor -from rel2graph.utils import load_file -from rel2graph.relational_modules.pandas import PandasDataFrameIterator -from rel2graph.neo4j import match_relationships, push, pull -from rel2graph.common_modules import MERGE_RELATIONSHIPS +from data2neo import Converter, IteratorIterator, register_attribute_postprocessor, Attribute, register_subgraph_preprocessor, GlobalSharedState, register_subgraph_postprocessor +from data2neo.utils import load_file +from data2neo.relational_modules.pandas import PandasDataFrameIterator +from data2neo.neo4j import match_relationships, push, pull +from data2neo.common_modules import MERGE_RELATIONSHIPS from helpers import * diff --git a/tests/integration/test_dependency.py b/tests/integration/test_dependency.py index 432428c..00008c9 100644 --- a/tests/integration/test_dependency.py +++ b/tests/integration/test_dependency.py @@ -14,8 +14,8 @@ import neo4j import os -from rel2graph import Converter, IteratorIterator, register_attribute_postprocessor, Attribute -from rel2graph.relational_modules.pandas import PandasDataFrameIterator +from data2neo import Converter, IteratorIterator, register_attribute_postprocessor, Attribute +from data2neo.relational_modules.pandas import PandasDataFrameIterator from helpers import * diff --git a/tests/integration/test_end_to_end.py b/tests/integration/test_end_to_end.py index 7267cad..4251254 100644 --- a/tests/integration/test_end_to_end.py +++ b/tests/integration/test_end_to_end.py @@ -9,12 +9,12 @@ import pytest -from rel2graph import Converter -from rel2graph.relational_modules.pandas import PandasDataFrameIterator -from rel2graph.utils import load_file -from rel2graph import IteratorIterator -from rel2graph import register_subgraph_postprocessor -from rel2graph.common_modules import MERGE_RELATIONSHIPS +from data2neo import Converter +from data2neo.relational_modules.pandas import PandasDataFrameIterator +from data2neo.utils import load_file +from data2neo import IteratorIterator +from data2neo import register_subgraph_postprocessor +from data2neo.common_modules import MERGE_RELATIONSHIPS from resources.data_end_to_end import no_duplicates, duplicates, before_update, person_only_nodes_only_result, schema_file_name from resources.data_end_to_end import iris, flower_only_result, full_result, result_parallel @@ -26,7 +26,7 @@ # Set logging level import logging -logging.getLogger("rel2graph").setLevel(logging.DEBUG) +logging.getLogger("data2neo").setLevel(logging.DEBUG) @pytest.mark.parametrize("workers",[1,5]) diff --git a/tests/integration/test_global_state.py b/tests/integration/test_global_state.py index 70f776c..9c4ee86 100644 --- a/tests/integration/test_global_state.py +++ b/tests/integration/test_global_state.py @@ -2,7 +2,7 @@ # -*- coding: utf-8 -*- """ -Integration tests for global state in rel2graph +Integration tests for global state in data2neo authors: Julian Minder """ @@ -12,8 +12,8 @@ import pandas as pd import multiprocessing as mp -from rel2graph import Converter, GlobalSharedState, register_subgraph_postprocessor -from rel2graph.relational_modules.pandas import PandasDataFrameIterator +from data2neo import Converter, GlobalSharedState, register_subgraph_postprocessor +from data2neo.relational_modules.pandas import PandasDataFrameIterator from helpers import * diff --git a/tests/integration/test_merge.py b/tests/integration/test_merge.py index 056004a..fae8cd0 100644 --- a/tests/integration/test_merge.py +++ b/tests/integration/test_merge.py @@ -11,10 +11,10 @@ import pandas as pd -from rel2graph import Converter, IteratorIterator, register_attribute_postprocessor, Attribute -from rel2graph.relational_modules.pandas import PandasDataFrameIterator -import rel2graph.common_modules.types -from rel2graph.common_modules import MERGE_RELATIONS +from data2neo import Converter, IteratorIterator, register_attribute_postprocessor, Attribute +from data2neo.relational_modules.pandas import PandasDataFrameIterator +import data2neo.common_modules.types +from data2neo.common_modules import MERGE_RELATIONS from helpers import * diff --git a/tests/integration/test_wrappers.py b/tests/integration/test_wrappers.py index 70afd10..ca18a51 100644 --- a/tests/integration/test_wrappers.py +++ b/tests/integration/test_wrappers.py @@ -10,13 +10,13 @@ import pytest import pandas as pd -from rel2graph import Converter -from rel2graph import AttributeFactoryWrapper, SubgraphFactoryWrapper -from rel2graph.neo4j import Node, Relationship, Subgraph -from rel2graph.utils import load_file -from rel2graph.relational_modules.pandas import PandasDataFrameIterator -from rel2graph import register_wrapper, register_attribute_postprocessor, register_attribute_preprocessor, register_subgraph_postprocessor, register_subgraph_preprocessor -from rel2graph import Attribute +from data2neo import Converter +from data2neo import AttributeFactoryWrapper, SubgraphFactoryWrapper +from data2neo.neo4j import Node, Relationship, Subgraph +from data2neo.utils import load_file +from data2neo.relational_modules.pandas import PandasDataFrameIterator +from data2neo import register_wrapper, register_attribute_postprocessor, register_attribute_preprocessor, register_subgraph_postprocessor, register_subgraph_preprocessor +from data2neo import Attribute from helpers import * # Turn off reinstantiation warnings diff --git a/tests/unit/common_modules/test_type_conversions.py b/tests/unit/common_modules/test_type_conversions.py index 6f7703d..237b847 100644 --- a/tests/unit/common_modules/test_type_conversions.py +++ b/tests/unit/common_modules/test_type_conversions.py @@ -10,8 +10,8 @@ import pytest import datetime -from rel2graph.common_modules import types -from rel2graph import Attribute +from data2neo.common_modules import types +from data2neo import Attribute @pytest.fixture def s_attr(): diff --git a/tests/unit/common_modules/test_util.py b/tests/unit/common_modules/test_util.py index d2bb47b..493d748 100644 --- a/tests/unit/common_modules/test_util.py +++ b/tests/unit/common_modules/test_util.py @@ -9,8 +9,8 @@ import pytest -from rel2graph.common_modules import datetime -from rel2graph import Attribute +from data2neo.common_modules import datetime +from data2neo import Attribute from datetime import datetime as dt diff --git a/tests/unit/core/test_converter.py b/tests/unit/core/test_converter.py index 4b35a14..6349d69 100644 --- a/tests/unit/core/test_converter.py +++ b/tests/unit/core/test_converter.py @@ -10,7 +10,7 @@ import pytest import neo4j -from rel2graph import Converter +from data2neo import Converter import warnings diff --git a/tests/unit/core/test_matcher.py b/tests/unit/core/test_matcher.py index eb97308..447a799 100644 --- a/tests/unit/core/test_matcher.py +++ b/tests/unit/core/test_matcher.py @@ -13,8 +13,8 @@ import os import warnings -from rel2graph.core.factories import Matcher, AttributeFactory -from rel2graph import Resource +from data2neo.core.factories import Matcher, AttributeFactory +from data2neo import Resource def delete_all(session): session.run("MATCH (n) DETACH DELETE n") diff --git a/tests/unit/core/test_schema_compiler.py b/tests/unit/core/test_schema_compiler.py index 577cad4..f6b0cc1 100644 --- a/tests/unit/core/test_schema_compiler.py +++ b/tests/unit/core/test_schema_compiler.py @@ -9,9 +9,9 @@ import pytest -from rel2graph.core.schema_compiler import compile_schema, SchemaConfigParser, _precompile -from rel2graph import register_attribute_preprocessor, SchemaConfigException -from rel2graph.utils import load_file +from data2neo.core.schema_compiler import compile_schema, SchemaConfigParser, _precompile +from data2neo import register_attribute_preprocessor, SchemaConfigException +from data2neo.utils import load_file ######## TESTS PRECOMPILER ########## diff --git a/tests/unit/neo4j/test_basic_ops.py b/tests/unit/neo4j/test_basic_ops.py index ada2a07..cd71d3b 100644 --- a/tests/unit/neo4j/test_basic_ops.py +++ b/tests/unit/neo4j/test_basic_ops.py @@ -11,8 +11,8 @@ import datetime from neo4j import GraphDatabase, time -from rel2graph.neo4j import Node, Relationship, Subgraph, create, merge, push, pull -from rel2graph.common_modules import MERGE_RELATIONSHIPS +from data2neo.neo4j import Node, Relationship, Subgraph, create, merge, push, pull +from data2neo.common_modules import MERGE_RELATIONSHIPS @pytest.fixture def session(): diff --git a/tests/unit/neo4j/test_graph_elements.py b/tests/unit/neo4j/test_graph_elements.py index 101f6b3..8b91d8a 100644 --- a/tests/unit/neo4j/test_graph_elements.py +++ b/tests/unit/neo4j/test_graph_elements.py @@ -11,8 +11,8 @@ import pytest import pickle -from rel2graph.neo4j import Node, Relationship, Subgraph -from rel2graph import Attribute +from data2neo.neo4j import Node, Relationship, Subgraph +from data2neo import Attribute def test_attributes(): attr = Attribute("key", "value") diff --git a/tests/unit/neo4j/test_match.py b/tests/unit/neo4j/test_match.py index d4f5521..1c79322 100644 --- a/tests/unit/neo4j/test_match.py +++ b/tests/unit/neo4j/test_match.py @@ -11,8 +11,8 @@ import datetime from neo4j import GraphDatabase, time -from rel2graph.neo4j import Node, Relationship, Subgraph, create, merge, match_nodes, match_relationships -from rel2graph.common_modules import MERGE_RELATIONSHIPS +from data2neo.neo4j import Node, Relationship, Subgraph, create, merge, match_nodes, match_relationships +from data2neo.common_modules import MERGE_RELATIONSHIPS @pytest.fixture def session(): diff --git a/tests/unit/relational_modules/test_pandas.py b/tests/unit/relational_modules/test_pandas.py index fb2cb7e..80e7c0a 100644 --- a/tests/unit/relational_modules/test_pandas.py +++ b/tests/unit/relational_modules/test_pandas.py @@ -9,7 +9,7 @@ import pytest -from rel2graph.relational_modules.pandas import PandasSeriesResource, PandasDataFrameIterator +from data2neo.relational_modules.pandas import PandasSeriesResource, PandasDataFrameIterator import pandas as pd import pickle diff --git a/tests/unit/relational_modules/test_sqlite.py b/tests/unit/relational_modules/test_sqlite.py index 065f937..0ca78da 100644 --- a/tests/unit/relational_modules/test_sqlite.py +++ b/tests/unit/relational_modules/test_sqlite.py @@ -9,7 +9,7 @@ import pytest -from rel2graph.relational_modules.sqlite import SQLiteResource, SQLiteIterator +from data2neo.relational_modules.sqlite import SQLiteResource, SQLiteIterator import sqlite3 import pickle diff --git a/tests/unit/utils/test_utils.py b/tests/unit/utils/test_utils.py index fb8a9ba..1804148 100644 --- a/tests/unit/utils/test_utils.py +++ b/tests/unit/utils/test_utils.py @@ -2,14 +2,14 @@ # -*- coding: utf-8 -*- """ -Tests for rel2graph.utils module +Tests for data2neo.utils module authors: Julian Minder """ import pytest -from rel2graph.utils import load_file +from data2neo.utils import load_file def test_load_file(): """