Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Drop support for Python 3.6 #4390

Merged
merged 14 commits into from
Mar 9, 2021
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 1 addition & 8 deletions .github/workflows/ci-windows.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ jobs:
strategy:
fail-fast: false
matrix:
python-version: ["3.6", "3.7", "3.8"]
python-version: ["3.7", "3.8"]

steps:
- name: Checkout source
Expand All @@ -23,13 +23,6 @@ jobs:
activate-environment: dask-distributed
auto-activate-base: false

- name: Install contextvars
shell: bash -l {0}
run: |
if [[ "${{ matrix.python-version }}" = "3.6" ]]; then
conda install -c conda-forge contextvars
fi

- name: Install tornado
shell: bash -l {0}
run: |
Expand Down
3 changes: 1 addition & 2 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,7 @@ dist: trusty

env:
matrix:
- PYTHON=3.6 TESTS=true COVERAGE=true PACKAGES="lz4" TORNADO=5 CRICK=true
- PYTHON=3.7 TESTS=true PACKAGES="python-snappy python-blosc" TORNADO=6
- PYTHON=3.7 TESTS=true COVERAGE=true PACKAGES="lz4 python-snappy python-blosc" CRICK=true TORNADO=5
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think here you removed the test case for when snappy and blosc are not installed

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the new Python 3.7 environment file is not including snappy or blosc, but please recheck and make sure this looks ok 🙂

- PYTHON=3.8 TESTS=true PACKAGES="python-snappy python-blosc" TORNADO=6

matrix:
Expand Down
4 changes: 0 additions & 4 deletions continuous_integration/travis/install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -67,10 +67,6 @@ conda create -n dask-distributed -c conda-forge -c defaults \

source activate dask-distributed

if [[ $PYTHON == 3.6 ]]; then
conda install -c conda-forge -c defaults contextvars
fi

if [[ $PYTHON == 3.8 ]]; then
# Install pytorch to run related tests
conda install -c pytorch -c conda-forge -c defaults pytorch torchvision
Expand Down
3 changes: 0 additions & 3 deletions distributed/cli/tests/test_dask_scheduler.py
Original file line number Diff line number Diff line change
Expand Up @@ -185,9 +185,6 @@ def test_interface(loop):
assert all("127.0.0.1" == d["host"] for d in info["workers"].values())


@pytest.mark.skipif(
sys.version_info < (3, 7), reason="Intermittent failure on old Python version"
)
def test_pid_file(loop):
def check_pidfile(proc, pidfile):
start = time()
Expand Down
9 changes: 0 additions & 9 deletions distributed/client.py
Original file line number Diff line number Diff line change
Expand Up @@ -758,15 +758,6 @@ def as_current(self):
method to return self. Any Future objects deserialized inside this context
manager will be automatically attached to this Client.
"""
# In Python 3.6, contextvars are thread-local but not Task-local.
# We can still detect a race condition though.
if sys.version_info < (3, 7) and _current_client.get() not in (self, None):
raise RuntimeError(
"Detected race condition where multiple asynchronous clients tried "
"entering the as_current() context manager at the same time. "
"Please upgrade to Python 3.7+."
)

tok = _current_client.set(self)
try:
yield
Expand Down
3 changes: 1 addition & 2 deletions distributed/comm/tests/test_comms.py
Original file line number Diff line number Diff line change
Expand Up @@ -222,8 +222,7 @@ async def handle_comm(comm):
assert not handle_comm_called

writer.close()
if hasattr(writer, "wait_closed"): # always true for python >= 3.7, but not for 3.6
await writer.wait_closed()
await writer.wait_closed()


@pytest.mark.asyncio
Expand Down
13 changes: 0 additions & 13 deletions distributed/compatibility.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,16 +10,3 @@
PYPY = platform.python_implementation().lower() == "pypy"
WINDOWS = sys.platform.startswith("win")
TORNADO6 = tornado.version_info[0] >= 6
PY37 = sys.version_info[:2] >= (3, 7)

if sys.version_info[:2] >= (3, 7):
from asyncio import get_running_loop
else:

def get_running_loop():
from asyncio import _get_running_loop

loop = _get_running_loop()
if loop is None:
raise RuntimeError("no running event loop")
return loop
3 changes: 0 additions & 3 deletions distributed/deploy/tests/test_local.py
Original file line number Diff line number Diff line change
Expand Up @@ -1031,9 +1031,6 @@ async def test_capture_security(cleanup, temporary):


@pytest.mark.asyncio
@pytest.mark.skipif(
sys.version_info < (3, 7), reason="asyncio.all_tasks not implemented"
)
async def test_no_danglng_asyncio_tasks(cleanup):
start = asyncio.all_tasks()
async with LocalCluster(asynchronous=True, processes=False):
Expand Down
4 changes: 1 addition & 3 deletions distributed/pubsub.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
import asyncio
from collections import defaultdict, deque
from contextlib import suppress
import logging
import threading
import weakref
Expand Down Expand Up @@ -421,8 +420,7 @@ async def _():
try:
await asyncio.wait_for(_(), timeout2)
finally:
with suppress(RuntimeError): # Python 3.6 fails here sometimes
self.condition.release()
self.condition.release()

return self.buffer.popleft()

Expand Down
5 changes: 0 additions & 5 deletions distributed/tests/test_client.py
Original file line number Diff line number Diff line change
Expand Up @@ -3670,7 +3670,6 @@ async def test_reconnect_timeout(c, s):

@pytest.mark.slow
@pytest.mark.skipif(WINDOWS, reason="num_fds not supported on windows")
@pytest.mark.skipif(sys.version_info < (3, 7), reason="TODO: intermittent failures")
@pytest.mark.parametrize("worker,count,repeat", [(Worker, 100, 5), (Nanny, 10, 20)])
def test_open_close_many_workers(loop, worker, count, repeat):
psutil = pytest.importorskip("psutil")
Expand Down Expand Up @@ -4055,10 +4054,6 @@ def run2():
t2.join()


@pytest.mark.xfail(
sys.version_info < (3, 7),
reason="Python 3.6 contextvars are not copied on Task creation",
)
@gen_cluster(client=False)
async def test_as_current_is_task_local(s, a, b):
l1 = asyncio.Lock()
Expand Down
5 changes: 0 additions & 5 deletions distributed/tests/test_core.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
import asyncio
import os
import socket
import sys
import threading
import weakref
import warnings
Expand Down Expand Up @@ -209,10 +208,6 @@ class MyServer(Server):
default_port = 8756


@pytest.mark.skipif(
sys.version_info < (3, 7),
reason="asynccontextmanager not avaiable before Python 3.7",
)
@pytest.mark.asyncio
async def test_server_listen():
"""
Expand Down
2 changes: 0 additions & 2 deletions distributed/tests/test_diskutils.py
Original file line number Diff line number Diff line change
Expand Up @@ -276,8 +276,6 @@ def _test_workspace_concurrency(tmpdir, timeout, max_procs):
def test_workspace_concurrency(tmpdir):
if WINDOWS:
raise pytest.xfail.Exception("TODO: unknown failure on windows")
if sys.version_info < (3, 7):
raise pytest.xfail.Exception("TODO: unknown failure on Python 3.6")
_test_workspace_concurrency(tmpdir, 5.0, 6)


Expand Down
3 changes: 0 additions & 3 deletions distributed/tests/test_scheduler.py
Original file line number Diff line number Diff line change
Expand Up @@ -1811,9 +1811,6 @@ async def test_get_task_duration(c, s, a, b):


@pytest.mark.asyncio
@pytest.mark.skipif(
sys.version_info < (3, 7), reason="asyncio.all_tasks not implemented"
)
async def test_no_danglng_asyncio_tasks(cleanup):
start = asyncio.all_tasks()
async with Scheduler(port=0) as s:
Expand Down
4 changes: 2 additions & 2 deletions distributed/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@
except ImportError:
PollIOLoop = None # dropped in tornado 6.0

from .compatibility import PYPY, WINDOWS, get_running_loop
from .compatibility import PYPY, WINDOWS
from .metrics import time


Expand Down Expand Up @@ -1156,7 +1156,7 @@ def reset_logger_locks():

if is_kernel():
try:
get_running_loop()
asyncio.get_running_loop()
except RuntimeError:
is_kernel_and_no_running_loop = True

Expand Down
3 changes: 1 addition & 2 deletions distributed/variable.py
Original file line number Diff line number Diff line change
Expand Up @@ -89,8 +89,7 @@ async def _(): # Python 3.6 is odd and requires special help here

await asyncio.wait_for(_(), timeout=left)
finally:
with suppress(RuntimeError): # Python 3.6 loses lock on finally clause
self.started.release()
self.started.release()

record = self.variables[name]
if record["type"] == "Future":
Expand Down
4 changes: 2 additions & 2 deletions docs/source/install.rst
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,6 @@ Notes
<https://trac.macports.org/ticket/50058>`_. with Python from macports that
makes executables be placed in a location that is not available by default. A
simple solution is to extend the ``PATH`` environment variable to the location
where Python from macports install the binaries. For example, for Python 3.6::
where Python from macports install the binaries. For example, for Python 3.7::

$ export PATH=/opt/local/Library/Frameworks/Python.framework/Versions/3.6/bin:$PATH
$ export PATH=/opt/local/Library/Frameworks/Python.framework/Versions/3.7/bin:$PATH
1 change: 0 additions & 1 deletion requirements.txt
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
click >= 6.6
cloudpickle >= 1.5.0
contextvars;python_version<'3.7'
dask>=2020.12.0
msgpack >= 0.6.0
psutil >= 5.0
Expand Down
3 changes: 1 addition & 2 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@
url="https://distributed.dask.org",
maintainer="Matthew Rocklin",
maintainer_email="[email protected]",
python_requires=">=3.6",
python_requires=">=3.7",
license="BSD",
package_data={
"": ["templates/index.html", "template.html"],
Expand All @@ -89,7 +89,6 @@
"License :: OSI Approved :: BSD License",
"Operating System :: OS Independent",
"Programming Language :: Python",
"Programming Language :: Python :: 3.6",
"Programming Language :: Python :: 3.7",
"Programming Language :: Python :: 3.8",
"Topic :: Scientific/Engineering",
Expand Down