Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add compatibility for Dask 2021.06.0 #70

Merged
merged 15 commits into from
Jun 9, 2021
Merged

Add compatibility for Dask 2021.06.0 #70

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
9c23dca
Add compatibility for Dask 2021.06.0
jrbourbeau Jun 5, 2021
07a99fb
Pin recent pyarrow
philippjfr Jun 7, 2021
c3b25f2
Switch to conda-forge
philippjfr Jun 7, 2021
250b488
Losen pyarrow pin
philippjfr Jun 7, 2021
00c3bd1
Adjust timeout
philippjfr Jun 7, 2021
a6b1310
Simplify dependencies
philippjfr Jun 7, 2021
dcac8c4
Set channel-priority to strict
philippjfr Jun 7, 2021
1fdd9df
Use same Python version
jrbourbeau Jun 7, 2021
f5d5b67
Have setup-miniconda and 'doit env_create' use the same conda environ…
jrbourbeau Jun 7, 2021
6a4f77d
Install hilbertcurve from conda-forge
jrbourbeau Jun 8, 2021
bdf52f3
Skip Windows 3.6 CI build due to fiona issue on conda-forge
jrbourbeau Jun 8, 2021
595e328
Prioritize conda-forge
jrbourbeau Jun 8, 2021
0e976a0
Attempt using geopandas-base
philippjfr Jun 9, 2021
dc132de
Trigger CI
jrbourbeau Jun 9, 2021
69720f8
Another attempt at fixing test env
philippjfr Jun 9, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 7 additions & 12 deletions .github/workflows/test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,14 +22,17 @@ jobs:
python-version: 3.6
- os: macos-latest
python-version: 3.8
timeout-minutes: 40
# Fiona for Python 3.6 on Windows has build issues
# See https://github.com/conda-forge/fiona-feedstock/issues/171 for more details
- os: windows-latest
python-version: 3.6
timeout-minutes: 60
defaults:
run:
shell: bash -l {0}
env:
PYTHON_VERSION: ${{ matrix.python-version }}
CHANS_DEV: "-c pyviz/label/dev"
CHANS_OSX: "-c pyviz/label/dev -c conda-forge"
CHANS_DEV: "-c conda-forge -c pyviz/label/dev"
CHANS: "-c pyviz"
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
steps:
Expand All @@ -42,6 +45,7 @@ jobs:
- uses: conda-incubator/setup-miniconda@v2
with:
miniconda-version: "latest"
python-version: ${{ matrix.python-version }}
- name: Fetch
run: git fetch --prune --tags
- name: conda setup
Expand All @@ -50,21 +54,12 @@ jobs:
conda install -c pyviz "pyctdev>=0.5"
doit ecosystem_setup
doit env_create ${{ env.CHANS_DEV}} --python=${{ matrix.python-version }}
- name: doit develop_install osx
if: contains(matrix.os, 'macos')
run: |
eval "$(conda shell.bash hook)"
conda activate test-environment
doit develop_install ${{ env.CHANS_OSX }} -o tests
pip install hilbertcurve
- name: doit develop_install
if: (!contains(matrix.os, 'macos'))
run: |
eval "$(conda shell.bash hook)"
conda activate test-environment
conda list
doit develop_install ${{ env.CHANS_DEV }} -o tests
pip install hilbertcurve
- name: doit env_capture
run: |
eval "$(conda shell.bash hook)"
Expand Down
26 changes: 7 additions & 19 deletions setup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
import sys

import param

from setuptools import find_packages, setup

extras_require = {
'tests': [
'codecov',
'flake8',
'geopandas',
'hilbertcurve',
'geopandas-base',
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that geopandas-base is only for conda-forge, so this won't work for pip

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it, that's okay for now. I guess we should disallow pip install spatialpandas[test] somehow but for development, testing and CI we recommend conda anyway.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Our build setup (pyctdev) does have ways to deal with x vs x-base or x-core differing between pip and conda, but I am not sure exactly how it handles that.

'hypothesis',
'pytest-cov',
'pytest',
Expand All @@ -29,27 +29,15 @@
install_requires = [
'fsspec',
'numba',
'pandas>=0.25',
'pandas >=0.25',
'param',
'pyarrow>=0.15',
'pyarrow >=1.0',
'python-snappy',
'retrying',
'numpy',
'dask[complete] >=2.0'
]

# Checking for platform explicitly because
# pyctdev does not handle dependency conditions
# such as 'numpy<1.20;platform_system=="Darwin"'
if sys.platform == 'darwin':
install_requires.extend([
'dask[complete]>=2.0,<2020.12',
'numpy<1.20',
])
else:
install_requires.extend([
'dask[complete]>=2.0',
'numpy',
])

setup_args = dict(
name='spatialpandas',
version=param.version.get_setup_version(
Expand Down
11 changes: 8 additions & 3 deletions spatialpandas/dask.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,12 @@
from dask import delayed
from dask.dataframe.core import get_parallel_type
from dask.dataframe.partitionquantiles import partition_quantiles
from dask.dataframe.utils import make_array_nonempty, make_meta, meta_nonempty
from dask.dataframe.extensions import make_array_nonempty
try:
from dask.dataframe.dispatch import make_meta_dispatch
from dask.dataframe.backends import meta_nonempty
except ImportError:
from dask.dataframe.utils import make_meta as make_meta_dispatch, meta_nonempty

from .geodataframe import GeoDataFrame
from .geometry.base import GeometryDtype, _BaseCoordinateIndexer
Expand Down Expand Up @@ -99,7 +104,7 @@ def persist(self, **kwargs):
)


@make_meta.register(GeoSeries)
@make_meta_dispatch.register(GeoSeries)
def make_meta_series(s, index=None):
result = s.head(0)
if index is not None:
Expand Down Expand Up @@ -546,7 +551,7 @@ def __getitem__(self, key):
return result


@make_meta.register(GeoDataFrame)
@make_meta_dispatch.register(GeoDataFrame)
def make_meta_dataframe(df, index=None):
result = df.head(0)
if index is not None:
Expand Down