Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: modernize with PDM #304

Merged
merged 8 commits into from
Jan 8, 2024
Merged

feat: modernize with PDM #304

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
2eeaef4
feat: modernize with PDM
frostming Jan 5, 2024
4c70ba8
doc: update README
frostming Jan 5, 2024
21f18f4
run under pdm env
frostming Jan 5, 2024
768cce6
requires-python `>=3.7`
wey-gu Jan 8, 2024
6c54e0d
fix: update lockfile
frostming Jan 8, 2024
e69fbcd
fix: separate example tests
frostming Jan 8, 2024
71e6d82
fix lint
frostming Jan 8, 2024
048f1b0
fix: try fixing by upgrading coverage
frostming Jan 8, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 13 additions & 18 deletions .github/workflows/deploy_release.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,31 +10,26 @@ jobs:
runs-on: ubuntu-20.04

steps:
- uses: actions/checkout@v2
- uses: actions/setup-python@v4
- uses: actions/checkout@v3
- uses: pdm-project/setup-pdm@v3
with:
python-version: '3.x'
cache: true
- name: Install dependencies
run: |
python3 -m pip install --upgrade pip
pip3 install setuptools wheel twine
pip install .
pip install -r requirements/dev.txt
run: pdm install
- name: Test with pytest
run: |
docker-compose -f docker-compose.yaml up -d
docker-compose -f tests/docker-compose.yaml up -d
sleep 20
pytest -s -v -k "not SSL"
working-directory: tests
pdm test
- name: Test SSL connection with pytest
run: |
enable_ssl=true docker-compose -f docker-compose-ssl.yaml up -d
enable_ssl=true docker-compose -f tests/docker-compose-ssl.yaml up -d
sleep 20
pytest -s -v -k "SSL"
working-directory: tests
pdm test-ssl

- name: Build and publish
env:
TWINE_USERNAME: ${{ secrets.PYPI_NAME }}
TWINE_PASSWORD: ${{ secrets.PYPI_PASSWORD }}
run: |
python3 setup.py sdist bdist_wheel
twine upload dist/*
PDM_PUBLISH_USERNAME: ${{ secrets.PYPI_NAME }}
PDM_PUBLISH_PASSWORD: ${{ secrets.PYPI_PASSWORD }}
run: pdm publish
57 changes: 35 additions & 22 deletions .github/workflows/run_test.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,11 @@ on:

jobs:
ci:
runs-on: ubuntu-20.04
runs-on: ubuntu-22.04
strategy:
max-parallel: 2
matrix:
python-version: [3.9, '3.10', 3.11, 3.12]
python-version: [3.7, 3.8, 3.9, '3.10', 3.11, 3.12]

steps:
- name: Maximize runner space
Expand All @@ -27,35 +27,48 @@ jobs:

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Nicole00 we could add 3.6 3.7 3.8 back in separate PR after this being merged. :D

Copy link
Contributor Author

@frostming frostming Jan 8, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAIK the setup-python action can no longer install 3.6 easily. And some of our dev dependencies don't work on 3.6.

nebula-python/pyproject.toml

Lines 32 to 42 in 2eeaef4

dev = [
"black==22.8.0; python_version >= \"3.7\"",
]
test = [
"pytest; python_full_version >= \"3.7.1\"",
"pytest-cov; python_full_version >= \"3.7.1\"",
]
example = [
"prettytable; python_full_version >= \"3.7.1\"",
"pandas; python_full_version >= \"3.7.1\"",
]

That's the trouble part what I mean.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, got it, let's put CI with 3.9+ as is, and keep the requirement broader to defer the decision of changing minimal py.

We could make CI best effort in a slightly broader in other PR.

@Nicole00 This seems to be a good enough timing for at least dropping 3.6 :D

- uses: actions/checkout@v3
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v4
uses: pdm-project/setup-pdm@v3
with:
python-version: ${{ matrix.python-version }}
- name: lint
run: make fmt-check
cache: true

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install .
pip install -r requirements/dev.txt
run: pdm install
- name: lint
run: pdm fmt-check
- name: Test with pytest
run: |
docker-compose -f docker-compose.yaml up -d
docker-compose -f tests/docker-compose.yaml up -d
sleep 20
pytest -s -v -k "not SSL" --cov=../nebula3 --cov-report=xml --cov-append
working-directory: tests
pdm test
- name: Test SSL connection with pytest
run: |
enable_ssl=true docker-compose -f docker-compose-ssl.yaml up -d
sleep 20
pytest -s -v -k "SSL" --cov=../nebula3 --cov-report=xml --cov-append
working-directory: tests
- name: Test example
run: |
docker-compose -f ../tests/docker-compose.yaml up -d
enable_ssl=true docker-compose -f tests/docker-compose-ssl.yaml up -d
sleep 20
for f in *.py; do python "$f"; done
working-directory: example
pdm test-ssl
- name: Upload Coverage to Codecov
uses: codecov/codecov-action@v3
with:
files: ./tests/coverage.xml
files: coverage.xml

example:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: [3.11, 3.12]
steps:
- uses: actions/checkout@v3
- name: Set up Python ${{ matrix.python-version }}
uses: pdm-project/setup-pdm@v3
with:
python-version: ${{ matrix.python-version }}
cache: true
- name: Install dependencies
run: pdm install -G example
- name: Setup containers
run: |
docker-compose -f tests/docker-compose.yaml up -d
sleep 20
- name: Test example
run: |
for f in example/*.py; do pdm run python "$f"; done
1 change: 1 addition & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -115,3 +115,4 @@ nebula/graph.thrift
# CI data
tests/data
tests/logs
.pdm-python
25 changes: 0 additions & 25 deletions Makefile
View file
Open in desktop

This file was deleted.

61 changes: 35 additions & 26 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

This repository holds the official Python API for NebulaGraph.

[![pdm-managed](https://img.shields.io/badge/pdm-managed-blueviolet)](https://pdm.fming.dev)

## Before you start

Before you start, please read this section to choose the right branch for you. The compatibility between the API and NebulaGraph service can be found in [How to choose nebula-python](#How-to-choose-nebula-python). The current master branch is compatible with NebulaGraph 3.x.
Expand All @@ -13,24 +15,24 @@ Before you start, please read this section to choose the right branch for you. T
|
|-- nebula3 // client code
| |-- fbthrift // the fbthrift lib code
| |-- common
| |-- data
| |-- graph
| |-- meta
| |-- common
| |-- data
| |-- graph
| |-- meta
| |-- net // the net code for graph client
| |-- storage
| |-- storage
| |-- Config.py // the pool config
| |__ Exception.py // the define exception
|
|
|-- examples
| |-- GraphClientMultiThreadExample.py // the multi thread example
| |-- GraphClientSimpleExample.py // the simple example
| |__ ScanVertexEdgeExample.py
| |__ ScanVertexEdgeExample.py
|
|-- tests // the test code
|
|
|-- setup.py // used to install or package
|
|
|__ README.md // the introduction of nebula3-python

```
Expand Down Expand Up @@ -154,7 +156,7 @@ connection_pool.close()

## Quick example to use storage-client to scan vertex and edge

You should make sure the scan client can connect to the address of storage which see from `SHOW HOSTS`
You should make sure the scan client can connect to the address of storage which see from `SHOW HOSTS`

```python
from nebula3.mclient import MetaCache, HostAddr
Expand Down Expand Up @@ -182,7 +184,7 @@ while resp.has_next():
result = resp.next()
for vertex_data in result:
print(vertex_data)

resp = graph_storage_client.scan_edge(
space_name='ScanSpace',
edge_name='friend')
Expand All @@ -195,16 +197,16 @@ while resp.has_next():
## How to choose nebula-python

| Nebula-Python Version | NebulaGraph Version |
|---|---|
| 1.0 | 1.x |
| 2.0.0 | 2.0.0/2.0.1 |
| 2.5.0 | 2.5.0 |
| 2.6.0 | 2.6.0/2.6.1 |
| 3.0.0 | 3.0.0 |
| 3.1.0 | 3.1.0 |
| 3.3.0 | 3.3.0 |
| 3.4.0 | >=3.4.0 |
| master | master |
| --------------------- | ------------------- |
| 1.0 | 1.x |
| 2.0.0 | 2.0.0/2.0.1 |
| 2.5.0 | 2.5.0 |
| 2.6.0 | 2.6.0/2.6.1 |
| 3.0.0 | 3.0.0 |
| 3.1.0 | 3.1.0 |
| 3.3.0 | 3.3.0 |
| 3.4.0 | >=3.4.0 |
| master | master |

## How to contribute to nebula-python

Expand All @@ -215,18 +217,25 @@ git clone https://github.com/{username}/nebula-python.git
cd nebula-python
```

Install the package in the editable mode, then install all the dev dependencies:
We use [PMD](https://github.com/pdm-project/pdm) to manage the package, install it first:

```
pipx install pdm
```

Visit the [PDM documentation](https://pdm-project.org) for alternative installation methods.

Install the package and all dev dependencies:
```
pip install -e .
pip install -r requirements/dev.txt
pdm install
```

Make sure the Nebula server in running, then run the tests with pytest:
```
pytest
pdm test
```
Using the default formatter with [black](https://github.com/psf/black).

Please run `make fmt` to format python code before submitting.
Please run `pdm fmt` to format python code before submitting.

See [How to contribute](https://github.com/vesoft-inc/nebula-community/blob/master/Contributors/how-to-contribute.md) for the general process of contributing to Nebula projects.
1 change: 1 addition & 0 deletions example/FormatResp.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@

import pandas as pd
import prettytable

from nebula3.data.DataObject import Value, ValueWrapper
from nebula3.data.ResultSet import ResultSet

Expand Down
Loading