Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add import entries in cli #210

Merged
merged 16 commits into from
Sep 9, 2022
Merged

Add import entries in cli #210

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
620c435
feat: add importentries
kod-kristoff Sep 2, 2022
626f439
test: add some fsctories
kod-kristoff Sep 2, 2022
c06eb3a
build: clean & update deps
kod-kristoff Sep 8, 2022
c851580
test: make unit tests pass
kod-kristoff Sep 8, 2022
8975a10
fix: merge handling for AddEntries and...
kod-kristoff Sep 8, 2022
0c8cf85
test: remove duplication
kod-kristoff Sep 8, 2022
d2ea639
build: remove lock-file
kod-kristoff Sep 8, 2022
2e9a5e7
refactor: merge ImportingEntries and ...
kod-kristoff Sep 8, 2022
f31de74
test: improve resource_id
kod-kristoff Sep 8, 2022
73a6b48
build: ignore poetry.lock
kod-kristoff Sep 8, 2022
e8001a9
ci: update workflows
kod-kristoff Sep 9, 2022
a3ed500
ci: add env
kod-kristoff Sep 9, 2022
6ffd0a9
ci: disable python 3.11-dev
kod-kristoff Sep 9, 2022
6c09e27
build: clean & update deps
kod-kristoff Sep 9, 2022
af255a9
ci: fix typo in workflow
kod-kristoff Sep 9, 2022
2750fca
ci: add tag for latest-changes workflow
kod-kristoff Sep 9, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 16 additions & 54 deletions .github/workflows/build-publish.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,25 +5,23 @@ on:
tags:
- v*

env:
POETRY_VERSION: "1.2.0"
PYTHON_VERSION: "3.10"
jobs:
lint:
runs-on: ${{ matrix.os }}
name: Lint with pylint (${{ matrix.os }})
strategy:
matrix:
os: [ubuntu-latest]
python-version: ["3.9"]
python-version: ["3.10"]
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
# - name: Load cached $HOME/.local
# uses: actions/[email protected]
# with:
# path: ~/.local
# key: dotlocal-${{ runner.os }}-${{ matrix.python-version }}-${{ env.POETRY_VERSION }}-${{ hashFiles('.github/workflows/build.yml') }}

- name: Install Poetry
uses: snok/[email protected]
Expand All @@ -32,15 +30,7 @@ jobs:
virtualenvs-create: true
virtualenvs-in-project: true

# - name: Load cached venv
# id: cached-poetry-dependencies
# uses: actions/cache@v2
# with:
# path: .venv
# key: venv-${{ runner.os }}-${{ matrix.python-version }}-${{ env.POETRY_VERSION }}-${{ hashFiles('**/poetry.lock') }}-${{ hashFiles('.github/workflows/build-publish.yml') }}

- name: Install dependencies
# if: steps.cached-poetry-dependencies.outputs.cache-hit != 'true'
run: make install-dev

- name: Setup up environment
Expand All @@ -57,15 +47,14 @@ jobs:
fail-fast: false
matrix:
os: [ubuntu-latest]
python-version: ["3.9", "3.10"]
python-version: ["3.10"]
es-version: ["6.8.23"]
task:
- name: Run tests
run: make all-tests-w-coverage
report_coverage: true

services:
mysql:
mariadb:
image: mariadb:10.3
ports:
- 3306
Expand All @@ -81,16 +70,11 @@ jobs:
--health-retries 10

steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
# - name: Load cached $HOME/.local
# uses: actions/[email protected]
# with:
# path: ~/.local
# key: dotlocal-${{ runner.os }}-${{ matrix.python-version }}-${{ env.POETRY_VERSION }}-${{ hashFiles('.github/workflows/build.yml') }}

- name: Install Poetry
uses: snok/[email protected]
Expand All @@ -99,15 +83,7 @@ jobs:
virtualenvs-create: true
virtualenvs-in-project: true

# - name: Load cached venv
# id: cached-poetry-dependencies
# uses: actions/cache@v2
# with:
# path: .venv
# key: venv-${{ runner.os }}-${{ matrix.python-version }}-${{ env.POETRY_VERSION }}-${{ hashFiles('**/poetry.lock') }}-${{ hashFiles('.github/workflows/build-publish.yml') }}

- name: Install dependencies
# if: steps.cached-poetry-dependencies.outputs.cache-hit != 'true'
run: make install-dev

- name: Setup up environment
Expand All @@ -129,33 +105,19 @@ jobs:
DB_USER: karp
DB_PASSWORD: testing
DB_HOST: 127.0.0.1
DB_PORT: ${{ job.services.mysql.ports['3306'] }}
- name: Upload coverage to Codecov
if: matrix.task.report_coverage
uses: codecov/codecov-action@v2
with:
# directory: ./coverage/reports/
# env_vars: OS,PYTHON
fail_ci_if_error: true
# files: ./coverage1.xml,./coverage2.xml
# flags: unittests
# name: codecov-umbrella
verbose: true
# - name: Coveralls
# uses: coverallsapp/github-action@master
# with:
# github-token: ${{ secrets.github_token }}
DB_PORT: ${{ job.services.mariadb.ports['3306'] }}

publish:
needs: [lint, test]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Set up Python 3.9
uses: actions/setup-python@v2
- name: Set up Python ${{ env.PYTHON_VERSION }}
uses: actions/setup-python@v4
with:
python-version: "3.9"
python-version: ${{ env.PYTHON_VERSION }}
- name: install poetry
run: pip install poetry
- name: Build distribution
Expand Down
118 changes: 32 additions & 86 deletions .github/workflows/build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ on:
- "*"

env:
POETRY_VERSION: 1.1.10
POETRY_VERSION: 1.2.0

jobs:
checks:
Expand All @@ -22,85 +22,37 @@ jobs:
max-parallel: 4
fail-fast: false
matrix:
python-version: ["3.9", "3.10"]
python-version: ["3.10"]
# python-version: ["3.10", "3.11-dev"] TODO: enable when greenlet works
os: [ubuntu-latest]
task:
- name: Lint code
run: make lint
allow_failure: true
- name: Typecheck
run: make check-mypy
run: make type-check
allow_failure: true
# - name: Lint w bandit
# run: make lint-security-issues
# allow_failure: false

steps:
- uses: actions/checkout@v2

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}

# - name: Load cached $HOME/.local
# uses: actions/[email protected]
# with:
# path: ~/.local
# key: dotlocal-${{ runner.os }}-${{ matrix.python-version }}-${{ env.POETRY_VERSION }}-${{ hashFiles('.github/workflows/build.yml') }}

- name: Install Poetry
uses: snok/[email protected]
with:
version: ${{ env.POETRY_VERSION }}
virtualenvs-create: true
virtualenvs-in-project: true

- name: Load cached venv
id: cached-poetry-dependencies
uses: actions/cache@v2
with:
path: .venv
key: venv-${{ runner.os }}-${{ matrix.python-version }}-${{ env.POETRY_VERSION }}-${{ hashFiles('**/poetry.lock') }}-${{ hashFiles('.github/workflows/build.yml') }}

- name: Install dependencies
if: steps.cached-poetry-dependencies.outputs.cache-hit != 'true'
run: make install-dev

- name: Setup up environment
run: cp -v .env.ci .env

- name: ${{ matrix.task.name }}
run: ${{ matrix.task.run }}
continue-on-error: ${{ matrix.task.allow_failure }}
env:
DB_DRIVER: sqlite

unit-tests:
name: ${{ matrix.task.name }} py-${{ matrix.python-version }} on ${{ matrix.os }}
runs-on: ${{ matrix.os }}

strategy:
max-parallel: 4
fail-fast: false
matrix:
python-version: ["3.9", "3.10"]
os: [ubuntu-latest]
task:
- name: Unit tests
run: make unit-tests-w-coverage
run: make unit-tests
allow_failure: false
# include:
# - task.allow_failure: false
# - task.allow_failure: true
# python-version: "3.11-dev"
# - name: Lint w bandit
# run: make lint-security-issues
# allow_failure: false

steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}

# - name: Load cached $HOME/.local
# uses: actions/cache@v2.1.6
# uses: actions/cache@v3.1.6
# with:
# path: ~/.local
# key: dotlocal-${{ runner.os }}-${{ matrix.python-version }}-${{ env.POETRY_VERSION }}-${{ hashFiles('.github/workflows/build.yml') }}
Expand All @@ -114,10 +66,10 @@ jobs:

- name: Load cached venv
id: cached-poetry-dependencies
uses: actions/cache@v2
uses: actions/cache@v3
with:
path: .venv
key: venv-${{ runner.os }}-${{ matrix.python-version }}-${{ env.POETRY_VERSION }}-${{ hashFiles('**/poetry.lock') }}-${{ hashFiles('.github/workflows/build.yml') }}
key: venv-${{ runner.os }}-${{ matrix.python-version }}-${{ env.POETRY_VERSION }}-${{ hashFiles('**/pyproject.toml') }}-${{ hashFiles('.github/workflows/build.yml') }}

- name: Install dependencies
if: steps.cached-poetry-dependencies.outputs.cache-hit != 'true'
Expand All @@ -131,17 +83,12 @@ jobs:
continue-on-error: ${{ matrix.task.allow_failure }}
env:
DB_DRIVER: sqlite
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v2
with:
fail_ci_if_error: false
verbose: true

tests:
name: ${{ matrix.task.name }} py-${{ matrix.python-version }} es-${{ matrix.es.version }} on ${{ matrix.os }}
runs-on: ${{ matrix.os }}
services:
mysql:
mariadb:
image: mariadb:10.3
ports:
- 3306
Expand All @@ -160,7 +107,8 @@ jobs:
max-parallel: 4
fail-fast: false
matrix:
python-version: ["3.9", "3.10"]
python-version: ["3.10"]
# python-version: ["3.10", "3.11-dev"] TODO: enable when greenlet works
es:
- use: false
version: NO
Expand All @@ -170,24 +118,29 @@ jobs:
task:
- name: Integration tests
run: make integration-tests-w-coverage
allow_failure: false
- name: End-to-end tests
run: make e2e-tests-w-coverage
allow_failure: false
# include:
# - task.allow_failure: true
# python-version: "3.11-dev"
exclude:
- es:
use: false
task:
name: End-to-end tests

steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}

# - name: Load cached $HOME/.local
# uses: actions/cache@v2.1.6
# uses: actions/cache@v3.1.6
# with:
# path: ~/.local
# key: dotlocal-${{ runner.os }}-${{ matrix.python-version }}-${{ env.POETRY_VERSION }}-${{ hashFiles('.github/workflows/build.yml') }}
Expand All @@ -201,10 +154,10 @@ jobs:

- name: Load cached venv
id: cached-poetry-dependencies
uses: actions/cache@v2
uses: actions/cache@v3
with:
path: .venv
key: venv-${{ runner.os }}-${{ matrix.python-version }}-${{ env.POETRY_VERSION }}-${{ hashFiles('**/poetry.lock') }}-${{ hashFiles('.github/workflows/build.yml') }}
key: venv-${{ runner.os }}-${{ matrix.python-version }}-${{ env.POETRY_VERSION }}-${{ hashFiles('**/pyproject.toml') }}-${{ hashFiles('.github/workflows/build.yml') }}

- name: Install dependencies
if: steps.cached-poetry-dependencies.outputs.cache-hit != 'true'
Expand Down Expand Up @@ -235,21 +188,14 @@ jobs:
echo "TEST_ES_HOME=`realpath elasticsearch-${{ matrix.es.version }}`" >> .env
cat .env
pip install -e .[elasticsearch7]
#- name: Setup sql index
#if: ! ${{ matrix.es.use }}
#run: echo "SEARCH_CONTEXT=sql_index" >> .env
- name: List tree
run: tree .
- name: ${{ matrix.task.name }}
run: ${{ matrix.task.run }}
continue-on-error: ${{ matrix.task.allow_failure }}
env:
DB_TEST_DATABASE: karp
DB_USER: karp
DB_PASSWORD: testing
DB_HOST: 127.0.0.1
DB_PORT: ${{ job.services.mysql.ports['3306'] }}
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v2
with:
fail_ci_if_error: false
verbose: true
DB_PORT: ${{ job.services.mariadb.ports['3306'] }}
Loading