Skip to content
This repository has been archived by the owner on Nov 18, 2023. It is now read-only.

Commit

Permalink
Use typedb_ml as the root package rather than typedb due to otherwi…
Browse files Browse the repository at this point in the history
…se having an import conflict with the client package also called `typedb`.
  • Loading branch information
James Fletcher committed Jul 27, 2022
1 parent c94f439 commit 8ee8997
Show file tree
Hide file tree
Showing 47 changed files with 95 additions and 72 deletions.
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,6 @@ tmp/
__pycache__/

# Data input/output directories
typedb/ml/examples/diagnosis/events/
typedb_ml/examples/diagnosis/events/

*.egg-info/
12 changes: 6 additions & 6 deletions .grabl/automation.yml
Original file line number Diff line number Diff line change
Expand Up @@ -50,35 +50,35 @@ build:
command: |
pyenv install -s 3.7.2
pyenv global 3.7.2 system
bazel test //typedb/ml/pytorch_geometric/... --test_output=streamed --spawn_strategy=standalone --action_env=PATH
bazel test //typedb_ml/pytorch_geometric/... --test_output=streamed --spawn_strategy=standalone --action_env=PATH
test-typedb:
image: vaticle-ubuntu-20.04
timeout: "10m"
command: |
pyenv install -s 3.7.2
pyenv global 3.7.2 system
bazel test //typedb/ml/typedb/... --test_output=streamed --spawn_strategy=standalone --action_env=PATH
bazel test //typedb_ml/typedb/... --test_output=streamed --spawn_strategy=standalone --action_env=PATH
test-networkx:
image: vaticle-ubuntu-20.04
timeout: "10m"
command: |
pyenv install -s 3.7.2
pyenv global 3.7.2 system
bazel test //typedb/ml/networkx/... --test_output=streamed --spawn_strategy=standalone --action_env=PATH
bazel test //typedb_ml/networkx/... --test_output=streamed --spawn_strategy=standalone --action_env=PATH
test-examples:
image: vaticle-ubuntu-20.04
timeout: "10m"
command: |
pyenv install -s 3.7.2
pyenv global 3.7.2 system
bazel test //typedb/ml/examples/... --test_output=streamed --spawn_strategy=standalone --action_env=PATH
bazel test //typedb_ml/examples/... --test_output=streamed --spawn_strategy=standalone --action_env=PATH
test-end-to-end:
image: vaticle-ubuntu-20.04
timeout: "10m"
command: |
pyenv install -s 3.7.2
pyenv global 3.7.2 system
bazel test //typedb/ml/tests/end_to_end:diagnosis --test_output=streamed --spawn_strategy=standalone --action_env=PATH
bazel test //typedb_ml/tests/end_to_end:diagnosis --test_output=streamed --spawn_strategy=standalone --action_env=PATH
deploy-pip-snapshot:
image: vaticle-ubuntu-20.04
# dependencies: [build, test-pytorch-geometric, test-typedb, test-networkx, test-examples, test-end-to-end]
Expand Down Expand Up @@ -106,7 +106,7 @@ build:
./typedb-all-linux/typedb server &
pip install -r requirements.txt
pip install --extra-index-url https://repo.vaticle.com/repository/pypi-snapshot/simple typedb-ml==0.0.0-$GRABL_COMMIT
python -m typedb.ml.examples.diagnosis.diagnosis "./typedb-all-linux" && export TEST_SUCCESS=0 ||
python -m typedb_ml.examples.diagnosis.diagnosis "./typedb-all-linux" && export TEST_SUCCESS=0 ||
export TEST_SUCCESS=1
kill $(jps | awk '/TypeDBServer/ {print $1}')
exit $TEST_SUCCESS
Expand Down
2 changes: 1 addition & 1 deletion BUILD
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ load("@vaticle_dependencies//tool/checkstyle:rules.bzl", "checkstyle_test")

assemble_pip(
name = "assemble-pip",
target = "//typedb:ml",
target = "//typedb_ml:typedb-ml",
package_name = "typedb-ml",
classifiers = [
"Programming Language :: Python :: 3",
Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ There are integrations for [NetworkX](https://networkx.org) and for [PyTorch Geo
- It's most natural to work with `HeteroData` objects since all data in TypeDB has a type. This conversion is available by default in PyG, but TypeDB-ML provides `store_concepts_by_type` to map concepts by type so that they can be re-associated after learning is finished.
- A `FeatureEncoder` to orchestrate encoders to generate features for graphs.
- Encoders for Continuous and Categorical values to apply encodings/embedding spaces to the types and attribute values present in TypeDB data.
- A [full example for link prediction](typedb/ml/examples/diagnosis)
- A [full example for link prediction](typedb_ml/examples/diagnosis)
### Other
- Example usage of Tensorboard for PyG `HeteroData`

Expand All @@ -50,7 +50,7 @@ You may find the following resources useful, particularly to understand why Type

### Run the Example

Take a look at the [PyTorch Geometric heterogeneous link prediction example](typedb/ml/examples/diagnosis) to see how to use TypeDB-ML to build a GNN on TypeDB data.
Take a look at the [PyTorch Geometric heterogeneous link prediction example](typedb_ml/examples/diagnosis) to see how to use TypeDB-ML to build a GNN on TypeDB data.

### Building from source

Expand All @@ -77,7 +77,7 @@ bazel build //...
Run all tests. Requires Python 3.7+ on your `PATH`. Test dependencies are for Linux since that is the CI environment:

```
bazel test //typedb/ml/... --test_output=streamed --spawn_strategy=standalone --action_env=PATH
bazel test //typedb_ml/... --test_output=streamed --spawn_strategy=standalone --action_env=PATH
```

Build the pip distribution. Outputs to `bazel-bin`:
Expand Down
4 changes: 2 additions & 2 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,9 @@
from setuptools import setup

setup(
name='typedb-ml',
name='typedb_ml',
version='',
packages=['typedb', 'ml', 'ml.networkx', 'ml.pytorch_geometric', 'ml.typedb'],
packages=['typedb_ml', 'typedb_ml.networkx', 'typedb_ml.pytorch_geometric', 'typedb_ml.typedb'],
url='',
license='',
author='jmsfltchr',
Expand Down
11 changes: 7 additions & 4 deletions typedb/BUILD → typedb_ml/BUILD
Original file line number Diff line number Diff line change
Expand Up @@ -24,11 +24,14 @@ load("@vaticle_bazel_distribution_pip//:requirements.bzl", deployment_requiremen
load("@vaticle_dependencies//tool/checkstyle:rules.bzl", "checkstyle_test")

py_library(
name = "ml",
name = "typedb-ml",
srcs = [
'__init__.py',
],
deps = [
'//typedb/ml/typedb',
'//typedb/ml/networkx',
'//typedb/ml/pytorch_geometric',
'//typedb_ml/typedb',
'//typedb_ml/networkx',
'//typedb_ml/pytorch_geometric',
],
visibility=['//visibility:public']
)
Expand Down
20 changes: 20 additions & 0 deletions typedb_ml/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
#
# Copyright (C) 2022 Vaticle
#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
#
Original file line number Diff line number Diff line change
Expand Up @@ -29,10 +29,10 @@ py_library(
'diagnosis.py'
],
deps = [
"//typedb/ml/examples/diagnosis/dataset",
"//typedb/ml/pytorch_geometric",
"//typedb/ml/networkx",
"//typedb/ml/typedb",
"//typedb_ml/examples/diagnosis/dataset",
"//typedb_ml/pytorch_geometric",
"//typedb_ml/networkx",
"//typedb_ml/typedb",
"@vaticle_typedb_client_python//:client_python",
vaticle_typedb_ml_requirement("networkx"),
vaticle_typedb_ml_requirement("torch"),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Once you have installed TypeDB-ML (see root README for instructions) you can run

1. Make sure a TypeDB server (version 2.11.1 or later) is running locally

2. Clone TypeDB-ML (shallow clone: `git clone --depth 1`) and from the project root, run the example: `python -m typedb.ml.examples.diagnosis.diagnosis "/path/to/my/typedb/install/directory"`
2. Clone TypeDB-ML (shallow clone: `git clone --depth 1`) and from the project root, run the example: `python -m typedb_ml.examples.diagnosis.diagnosis "/path/to/my/typedb/install/directory"`

3. The database, schema and seed data will be set up and data will be generated synthetically. You should see console output to indicate that the pipeline is running and that the model is learning. Finally, the predicted relations are shown, and they are written back into TypeDB as the type `predicted-diagnosis`.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
import numpy as np
from typedb.client import *

from typedb.ml.examples.diagnosis.dataset.pmf import PMF
from typedb_ml.examples.diagnosis.dataset.pmf import PMF


def get_example_queries(pmf, example_id):
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,15 +31,15 @@
from torch_geometric.nn import HGTConv
from typedb.client import *

from typedb.ml.examples.diagnosis.dataset.generate import generate_example_data
from typedb.ml.networkx.query_graph import QueryGraph
from typedb.ml.pytorch_geometric.dataset.dataset import DataSet
from typedb.ml.pytorch_geometric.transform.binary_link_prediction import LinkPredictionLabeller, \
from typedb_ml.examples.diagnosis.dataset.generate import generate_example_data
from typedb_ml.networkx.query_graph import QueryGraph
from typedb_ml.pytorch_geometric.dataset.dataset import DataSet
from typedb_ml.pytorch_geometric.transform.binary_link_prediction import LinkPredictionLabeller, \
binary_relations_to_edges, binary_link_prediction_edge_triplets
from typedb.ml.pytorch_geometric.transform.common import clear_unneeded_fields, store_concepts_by_type
from typedb.ml.pytorch_geometric.transform.encode import FeatureEncoder, CategoricalEncoder, ContinuousEncoder
from typedb.ml.typedb.load import load_typeql_file, FileType
from typedb.ml.typedb.type import get_thing_types
from typedb_ml.pytorch_geometric.transform.common import clear_unneeded_fields, store_concepts_by_type
from typedb_ml.pytorch_geometric.transform.encode import FeatureEncoder, CategoricalEncoder, ContinuousEncoder
from typedb_ml.typedb.load import load_typeql_file, FileType
from typedb_ml.typedb.type import get_thing_types

DATABASE = "diagnosis"
ADDRESS = "localhost:1729"
Expand Down Expand Up @@ -74,8 +74,8 @@ def diagnosis_example(typedb_binary_directory,
num_graphs,
database=DATABASE,
address=ADDRESS,
schema_file_path="typedb/ml/examples/diagnosis/dataset/schema.tql",
seed_data_file_path="typedb/ml/examples/diagnosis/dataset/seed_data.tql"):
schema_file_path="typedb_ml/examples/diagnosis/dataset/schema.tql",
seed_data_file_path="typedb_ml/examples/diagnosis/dataset/seed_data.tql"):
"""
Args:
typedb_binary_directory: Location of the TypeDB binary for the purpose of loading initial schema and data
Expand Down Expand Up @@ -458,5 +458,5 @@ def write_predictions_to_typedb(predicted_links, tx):
diagnosis_example(args.typedb, args.graphs,
database=DATABASE,
address=ADDRESS,
schema_file_path=cwd + '/' + "typedb/ml/examples/diagnosis/dataset/schema.tql",
seed_data_file_path=cwd + '/' + "typedb/ml/examples/diagnosis/dataset/seed_data.tql")
schema_file_path=cwd + '/' + "typedb_ml/examples/diagnosis/dataset/schema.tql",
seed_data_file_path=cwd + '/' + "typedb_ml/examples/diagnosis/dataset/seed_data.tql")
10 changes: 5 additions & 5 deletions typedb/ml/networkx/BUILD → typedb_ml/networkx/BUILD
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ py_library(
'iterate.py',
],
deps = [
"//typedb/ml/typedb",
"//typedb_ml/typedb",
vaticle_typedb_ml_requirement('networkx'),
vaticle_typedb_ml_requirement('decorator'),
],
Expand All @@ -59,7 +59,7 @@ py_test(
"graph_test_case.py"
],
deps = [
"//typedb/ml/typedb",
"//typedb_ml/typedb",
vaticle_typedb_ml_requirement('networkx'),
]
)
Expand All @@ -73,8 +73,8 @@ py_test(
"graph_test_case.py"
],
deps = [
"//typedb/ml/typedb",
"//typedb/ml/typedb/test",
"//typedb_ml/typedb",
"//typedb_ml/typedb/test",
"@vaticle_typedb_client_python//:client_python",
vaticle_typedb_ml_requirement('networkx'),
],
Expand All @@ -91,7 +91,7 @@ py_test(
"graph_test_case.py"
],
deps = [
"//typedb/ml/typedb",
"//typedb_ml/typedb",
vaticle_typedb_ml_requirement('networkx'),
],
)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,9 @@
import unittest
import networkx as nx

from typedb.ml.typedb.thing import Thing
from typedb.ml.networkx.concept_dict_to_networkx import concept_dict_to_networkx
from typedb.ml.networkx.graph_test_case import GraphTestCase
from typedb_ml.typedb.thing import Thing
from typedb_ml.networkx.concept_dict_to_networkx import concept_dict_to_networkx
from typedb_ml.networkx.graph_test_case import GraphTestCase


class TestConceptDictToTypeDBGraph(GraphTestCase):
Expand Down
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,8 @@
from functools import reduce
import networkx as nx

from typedb.ml.typedb.thing import build_thing
from typedb.ml.networkx.concept_dict_to_networkx import concept_dict_to_networkx
from typedb_ml.typedb.thing import build_thing
from typedb_ml.networkx.concept_dict_to_networkx import concept_dict_to_networkx


def build_graph_from_queries(query_sampler_variable_graph_tuples, transaction,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,10 +26,10 @@
from typedb.api.concept.type.attribute_type import AttributeType
from typedb.client import *

from typedb.ml.networkx.graph_test_case import GraphTestCase
from typedb.ml.networkx.queries_to_networkx import build_graph_from_queries
from typedb.ml.typedb.thing import build_thing
from typedb.ml.typedb.test.base import TypeDBServer
from typedb_ml.networkx.graph_test_case import GraphTestCase
from typedb_ml.networkx.queries_to_networkx import build_graph_from_queries
from typedb_ml.typedb.thing import build_thing
from typedb_ml.typedb.test.base import TypeDBServer


class ITBuildGraphFromQueriesWithRealTypeDB(GraphTestCase):
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,12 +24,12 @@
import networkx as nx
from typedb.api.concept.type.attribute_type import AttributeType

from typedb.ml.networkx.graph_test_case import GraphTestCase
from typedb.ml.networkx.queries_to_networkx import concept_dict_from_concept_map, \
from typedb_ml.networkx.graph_test_case import GraphTestCase
from typedb_ml.networkx.queries_to_networkx import concept_dict_from_concept_map, \
combine_n_graphs, build_graph_from_queries
from typedb.ml.typedb.thing import Thing
from typedb.ml.typedb.test.mock.answer import MockConceptMap
from typedb.ml.typedb.test.mock.concept import MockType, MockAttributeType, MockThing, MockAttribute
from typedb_ml.typedb.thing import Thing
from typedb_ml.typedb.test.mock.answer import MockConceptMap
from typedb_ml.typedb.test.mock.concept import MockType, MockAttributeType, MockThing, MockAttribute


def mock_sampler(input_iter):
Expand Down
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@

import unittest

from typedb.ml.networkx.query_graph import QueryGraph
from typedb_ml.networkx.query_graph import QueryGraph


class TestQueryGraph(unittest.TestCase):
Expand Down
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@
from torch_geometric.utils import from_networkx
from typedb.client import TypeDB, TypeDBSession, SessionType, TypeDBOptions, TransactionType

from typedb.ml.networkx.queries_to_networkx import build_graph_from_queries
from typedb_ml.networkx.queries_to_networkx import build_graph_from_queries


class DataSet:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,8 @@
#
import networkx as nx

from typedb.ml.networkx.iterate import multidigraph_edge_data_iterator
from typedb.ml.typedb.type import get_edge_type_triplets, reverse_edge_type_triplets
from typedb_ml.networkx.iterate import multidigraph_edge_data_iterator
from typedb_ml.typedb.type import get_edge_type_triplets, reverse_edge_type_triplets


class LinkPredictionLabeller:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
# specific language governing permissions and limitations
# under the License.
#
from typedb.ml.networkx.iterate import multidigraph_node_data_iterator, multidigraph_edge_data_iterator
from typedb_ml.networkx.iterate import multidigraph_node_data_iterator, multidigraph_edge_data_iterator


def clear_unneeded_fields(graph):
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
import torch
from torch.nn import Embedding

from typedb.ml.networkx.iterate import (
from typedb_ml.networkx.iterate import (
multidigraph_node_data_iterator,
multidigraph_edge_data_iterator,
)
Expand Down
Loading

0 comments on commit 8ee8997

Please sign in to comment.