Don't use capture_output or update minimum Python version

[holdenk]

What happened:

Exception ignored in: <object repr() failed>
Traceback (most recent call last):
  File "/home/holden/repos/scalingpythonml/venv_dask/lib/python3.6/site-packages/distributed/deploy/cluster.py", line 98, in __del__
    if self.status != Status.closed:
AttributeError: 'HelmCluster' object has no attribute 'status'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/holden/repos/scalingpythonml/scratch/dask-kubernetes/dask_kubernetes/helm.py", line 79, in __init__
    encoding="utf-8",
  File "/usr/lib/python3.6/subprocess.py", line 423, in run
    with Popen(*popenargs, **kwargs) as process:
TypeError: __init__() got an unexpected keyword argument 'capture_output'

What you expected to happen:

Connect to cluster

Minimal Complete Verifiable Example:

Create a virtual env with Python 3.6

from dask_kubernetes import HelmCluster
cluster = HelmCluster(release_name="myrelease")

Anything else we need to know?:

Environment:

• Dask version: dask==2.27.0
• Dask Kubernetes Version: -e [email protected]:dask/dask-kubernetes.git@fdb77c0#egg=dask_kubernetes
• Python version: Python 3.6.9 (default, Jul 17 2020, 12:50:27)
• Operating System: DISTRIB_DESCRIPTION="Ubuntu 18.04.5 LTS"
• Install method (conda, pip, source): pip

[mrocklin]

I'd be inclined to bump the minimum to 3.7. We're going that way soonish anyway.

…

On Sat, Sep 26, 2020 at 6:54 PM Holden Karau ***@***.***> wrote: *What happened*: Exception ignored in: <object repr() failed> Traceback (most recent call last): File "/home/holden/repos/scalingpythonml/venv_dask/lib/python3.6/site-packages/distributed/deploy/cluster.py", line 98, in __del__ if self.status != Status.closed: AttributeError: 'HelmCluster' object has no attribute 'status' Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/holden/repos/scalingpythonml/scratch/dask-kubernetes/dask_kubernetes/helm.py", line 79, in __init__ encoding="utf-8", File "/usr/lib/python3.6/subprocess.py", line 423, in run with Popen(*popenargs, **kwargs) as process: TypeError: __init__() got an unexpected keyword argument 'capture_output' *What you expected to happen*: Connect to cluster *Minimal Complete Verifiable Example*: Create a virtual env with Python 3.6 from dask_kubernetes import HelmClustercluster = HelmCluster(release_name="myrelease") *Anything else we need to know?*: *Environment*: - Dask version: dask==2.27.0 - Dask Kubernetes Version: -e ***@***.*** ***@***.*** <fdb77c0> #egg=dask_kubernetes - Python version: Python 3.6.9 (default, Jul 17 2020, 12:50:27) - Operating System: DISTRIB_DESCRIPTION="Ubuntu 18.04.5 LTS" - Install method (conda, pip, source): pip — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#269>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACKZTFVKPA5PS6DEENKY6DSH2LOHANCNFSM4R3IBTEA> .

[quasiben]

@holdenk thanks for checking out dask-kubernetes. To give a bit more context for @mrocklin's comment, the Dask and many projects within the PyData community are moving away from Python 3.6 because NumPy is dropping 3.6 support.
Pandas, for example, dropped 3.6 support last week.

dask/community#66 for even more reading :)

[holdenk]

I think updating this to 3.7 is the right thing to do, we're also pushing minimum versions forward more aggressively in Spark.