Skip to content

Commit

Permalink
Merge pull request #2 from piercefreeman/feature/add-deployment-ci
Browse files Browse the repository at this point in the history
Add CI and deployment pipeline
  • Loading branch information
piercefreeman authored Apr 18, 2023
2 parents 75f0e37 + e3ce248 commit a5fb2d8
Show file tree
Hide file tree
Showing 4 changed files with 96 additions and 12 deletions.
41 changes: 41 additions & 0 deletions .github/workflows/deploy.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
name: Deploy

on:
push:
tags:
- v*

jobs:
deploy:
runs-on: ubuntu-latest

steps:
- name: Checkout repository
uses: actions/checkout@v2

- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: "3.10"

- name: Install Poetry
run: |
curl -sSL https://install.python-poetry.org | python3 -
- name: Configure Poetry
run: |
echo "$(python -c 'import os,sys; print(os.path.realpath(sys.executable))')/$(echo $(python -c 'import sys; print(sys.executable)') | rev | cut -d'/' -f1 | rev)" > "$(python -c 'import site; print(site.USER_BASE)')/bin/python"
echo "PATH=$(python -c 'import site; print(site.USER_BASE)')/bin:$PATH" >> $GITHUB_ENV
- name: Install dependencies with Poetry
run: |
poetry config virtualenvs.create false
poetry install
- name: Build package
run: |
poetry build
- name: Publish package to PyPI
run: |
poetry publish --username ${{ secrets.PYPI_USERNAME }} --password ${{ secrets.PYPI_PASSWORD }}
43 changes: 43 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
name: Pytest

on: push

jobs:
build-and-test:
runs-on: ubuntu-latest

steps:
- name: Checkout repository
uses: actions/checkout@v2

- name: Set up Docker Compose
run: |
sudo apt-get update
sudo apt-get install -y docker-compose
- name: Build and run Docker Compose
run: |
docker-compose up -d
- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: "3.10"

- name: Install Poetry
run: |
curl -sSL https://install.python-poetry.org | python3 -
- name: Configure Poetry
run: |
echo "$(python -c 'import os,sys; print(os.path.realpath(sys.executable))')/$(echo $(python -c 'import sys; print(sys.executable)') | rev | cut -d'/' -f1 | rev)" > "$(python -c 'import site; print(site.USER_BASE)')/bin/python"
echo "PATH=$(python -c 'import site; print(site.USER_BASE)')/bin:$PATH" >> $GITHUB_ENV
- name: Install dependencies with Poetry
run: |
poetry config virtualenvs.create false
poetry install
- name: Run pytest
run: |
poetry run pytest vectordb_orm
22 changes: 11 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,6 @@

`vectordb-orm` is an Object-Relational Mapping (ORM) library designed to work with vector databases, such as Milvus. The project aims to provide a consistent and convenient interface for working with vector data, allowing you to interact with vector databases using familiar ORM concepts and syntax.

## Why use an ORM?

Most vector databases use a JSON-like querying syntax where schemas and objects are specified as dictionary blobs. This makes it difficult to use IDE features like autocomplete or typehinting, and also can lead to error prone code while translating between Python logic and querying syntax.

An ORM provides a high-level, abstracted interface to work with databases. This abstraction makes it easier to write, read, and maintain code, as well as to switch between different database backends with minimal changes. Furthermore, an ORM allows developers to work with databases in a more Pythonic way, using Python objects and classes instead of raw SQL queries or low-level API calls.

## Comparison to SQLAlchemy

While vectordb-orm is inspired by the widely-used SQLAlchemy ORM, it is specifically designed for vector databases, such as Milvus. This means that vectordb-orm offers unique features tailored to the needs of working with vector data, such as similarity search, index management, and efficient data storage. Although the two ORMs share some similarities in terms of syntax and structure, vectordb-orm focuses on providing a seamless experience for working with vector databases.

## Getting Started

Here are some example code snippets demonstrating common behavior with vectordb-orm. vectordb-orm is designed around python typehints. You create a class definition by subclassing `MilvusBase` and providing typehints for the keys of your model, similar to pydantic. These fields also support custom initialization behavior if you want (or need) to modify their configuration options.
Expand Down Expand Up @@ -55,7 +45,7 @@ query_vector = np.array([8.0]*128)
results = session.query(MyObject).filter(MyObject.text == 'bar').order_by_similarity(MyObject.embedding, query_vector).limit(2).all()
```

## Getting started
## Installation

To get started with vectordb-orm, simply install the package and its dependencies, then import the necessary modules:

Expand All @@ -70,6 +60,16 @@ poetry install
poetry run pytest
```

## Why use an ORM?

Most vector databases use a JSON-like querying syntax where schemas and objects are specified as dictionary blobs. This makes it difficult to use IDE features like autocomplete or typehinting, and also can lead to error prone code while translating between Python logic and querying syntax.

An ORM provides a high-level, abstracted interface to work with databases. This abstraction makes it easier to write, read, and maintain code, as well as to switch between different database backends with minimal changes. Furthermore, an ORM allows developers to work with databases in a more Pythonic way, using Python objects and classes instead of raw SQL queries or low-level API calls.

## Comparison to SQLAlchemy

While vectordb-orm is inspired by the widely-used SQLAlchemy ORM, it is specifically designed for vector databases, such as Milvus. This means that vectordb-orm offers unique features tailored to the needs of working with vector data, such as similarity search, index management, and efficient data storage. Although the two ORMs share some similarities in terms of syntax and structure, vectordb-orm focuses on providing a seamless experience for working with vector databases.

## WIP

Please note that vectordb-orm is still a (somewhat large) work in progress. The current implementation focuses on Milvus integration, the goal is to eventually expand support to other vector databases. Contributions and feedback are welcome as we work to improve and expand the capabilities of vectordb-orm.
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[tool.poetry]
name = "milvus-orm"
name = "vectordb-orm"
version = "0.1.0"
description = ""
authors = ["Pierce Freeman <[email protected]>"]
Expand Down

0 comments on commit a5fb2d8

Please sign in to comment.