incompatible dependency with json_schema

[lascott]

Description
Incompatible dependency constraints, or undocumented resolution of jsconschema v airflow

airflow jsonschema constraint

• jsonschema >=3.0.0,<3.1.0 # [py<38]

• ensure jsonschema version is < 3.0.0 in setup.py

  https://github.com/conda-forge/airflow-feedstock/blob/master/recipe/meta.yaml
  Very few online results, but in poetry, this was resolve by dropping to a version < 3.0.0
  AttributeError occurred when install dependencies: module 'jsonschema' has no attribute 'Draft7Validator' python-poetry/poetry#2312
  set jsonschema max version spec-first/connexion#708

Related Issues:
#8993 #8880 #8605

Apache Airflow version:
airflow 1.10.10 py37hc8dfbb8_0 conda-forge
airflow-with-celery 1.10.10 0 conda-forge
airflow-with-jira 1.10.10 0 conda-forge
airflow-with-postgres 1.10.10 0 conda-forge
airflow-with-redis 1.10.10 0 conda-forge
airflow-with-samba 1.10.10 0 conda-forge
airflow-with-ssh 1.10.10 0 conda-forge

Kubernetes version (if you are using kubernetes) (use kubectl version): None

Environment:

• Cloud provider or hardware configuration: local

• OS Linux Mint 19.1 Tessa

• Kernel : 4.15.0-101-generic Question about TaskInstance.is_queueable #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

• Install tools:

  conda env create -f avm-${FILE_IN}.yaml

avm-prod-test.yaml minimal airflow set to reproduce

# https://docs.anaconda.com/anaconda/packages/pkg-docs/ # # Production overnight schedulers, # You can add your own there # --------------------------------- # name: avm-prod-test channels: - conda-forge - defaults

dependencies:

• python=3.7
• pip

Core scientific python

• pandas

Jupyter Environment

• autopep8
• jupyter

io

• Path
• sqlite
• sqlalchemy==1.3.15
• pyarrow
• csvkit

Production only airflow used to reproduce prolem

• airflow
• airflow-with-postgres
• airflow-with-samba
• airflow-with-ssh
• airflow-with-redis
• airflow-with-celery
• airflow-with-jira
• jsonschema>=3.0.1

IDE preferences

• rope
• spyder

What happened:

Running airflow initdb cannot avoid jsonschema error

airflow initdb Traceback (most recent call last): File "/home/louis/miniconda/envs/avm-prod-test/bin/airflow", line 26, in from airflow.bin.cli import CLIFactory File "/home/louis/miniconda/envs/avm-prod-test/lib/python3.7/site-packages/airflow/bin/cli.py", line 79, in api_module = import_module(conf.get('cli', 'api_client')) # type: Any File "/home/louis/miniconda/envs/avm-prod-test/lib/python3.7/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "/home/louis/miniconda/envs/avm-prod-test/lib/python3.7/site-packages/airflow/api/client/local_client.py", line 24, in from airflow.api.common.experimental import delete_dag File "/home/louis/miniconda/envs/avm-prod-test/lib/python3.7/site-packages/airflow/api/common/experimental/delete_dag.py", line 25, in from airflow.models.serialized_dag import SerializedDagModel File "/home/louis/miniconda/envs/avm-prod-test/lib/python3.7/site-packages/airflow/models/serialized_dag.py", line 32, in from airflow.serialization.serialized_objects import SerializedDAG File "/home/louis/miniconda/envs/avm-prod-test/lib/python3.7/site-packages/airflow/serialization/serialized_objects.py", line 491, in class SerializedDAG(DAG, BaseSerialization): File "/home/louis/miniconda/envs/avm-prod-test/lib/python3.7/site-packages/airflow/serialization/serialized_objects.py", line 521, in SerializedDAG _json_schema = load_dag_schema() File "/home/louis/miniconda/envs/avm-prod-test/lib/python3.7/site-packages/airflow/serialization/json_schema.py", line 77, in load_dag_schema jsonschema.Draft7Validator.check_schema(schema) AttributeError: module 'jsonschema' has no attribute 'Draft7Validator'

What you expected to happen:
Successful initialization

What do you think went wrong?
Incompatible dependency constraints, or undocumented resolution.

airflow jsonschema constraint

https://github.com/conda-forge/airflow-feedstock/blob/master/recipe/meta.yaml

• jsonschema >=3.0.0,<3.1.0 # [py<38]
• Very few online results, but in poetry, this was resolve by dropping to a version < 3.0.0
  see:
  AttributeError occurred when install dependencies: module 'jsonschema' has no attribute 'Draft7Validator' python-poetry/poetry#2312
  set jsonschema max version spec-first/connexion#708
  Changes proposed in this pull request:
  ensure jsonschema version is < 3.0.0 in setup.py

How to reproduce it:

rebuild miniconda with airflow and call initdb.
# be sure to check path PATH="$HOME/miniconda/bin:$HOME/miniconda/envs/avm-prod-test:$PATH"
bash ~/miniconda.sh -b -p $HOME/miniconda || exit 1
source $HOME/miniconda/bin/activate
conda config --set always_yes yes --set show_channel_urls true --set changeps1 no
conda update -q conda
conda config --add channels conda-forge
conda info -a
conda env create -f avm-prod-test.yaml
airflow initdb

Anything else we need to know: Occurs every time

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template!

[lascott]

I made an effort to submit after reading the issue template and adhering to it.
Something more than a bot would have been the polite thing to do by now.
New participants should feel welcomed if a community is to grow.

[lascott]

'jsonschema' has no attribute 'Draft7Validator' error resolution.

boundary-layer only runs with airflow < 1.10.7 at present.
Also requires werkzeug<1.0

Will inform boundary-layer forum. Not an airflow issue (probably).
Marking resolved.

name: avm-test
channels:

• conda-forge
• defaults

dependencies:

• python=3.7
• pip
• airflow<=1.10.6
• pip:
  • boundary-layer

requires airflow<1.10.7 for now

Core scientific python

• scikit-learn
• matplotlib
• pyqt
• seaborn
• tqdm
• kiwisolver
• geopandas
• jsonschema
• werkzeug<1.0

[liaodalin19903]

requires airflow<1.10.7 for now

Core scientific python

• scikit-learn
• matplotlib
• pyqt
• seaborn
• tqdm
• kiwisolver
• geopandas
• jsonschema
• werkzeug<1.0

Hi, @lascott
what's the meaning of requires airflow<1.10.7 for now?