Removing asyncio.coroutine syntax from HASS core #12509

Julius2342 · 2018-02-18T22:14:51Z

Description

Changed old asyncio.coroutine/ yield from syntax to async def / await syntax within core libraries.

Current state

Runs here on my home on production without problems.

13 unit tests fail.

    3299 passed
      13 failed
         - tests/test_bootstrap.py:43 test_home_assistant_core_config_validation[pyloop]
         - tests/components/emulated_hue/test_hue_api.py:219 test_put_light_state[pyloop]
         - tests/components/emulated_hue/test_hue_api.py:346 test_put_with_form_urlencoded_content_type[pyloop]
         - tests/components/mqtt/test_discovery.py:24 test_invalid_topic[pyloop]
         - tests/components/mqtt/test_discovery.py:37 test_invalid_json[pyloop]
         - tests/components/mqtt/test_discovery.py:51 test_only_valid_components[pyloop]
         - tests/components/notify/test_apns.py:357 TestApns.test_disable_when_unregistered
         - tests/components/sensor/test_coinmarketcap.py:31 TestCoinMarketCapSensor.test_setup
         - tests/helpers/test_discovery.py:27 TestHelpersDiscovery.test_listen
         - tests/scripts/test_check_config.py:120 TestCheckConfig.test_component_platform_not_found
         - tests/scripts/test_check_config.py:73 TestCheckConfig.test_config_component_platform_fail_validation
         - tests/scripts/test_check_config.py:57 TestCheckConfig.test_config_platform_valid
         - tests/util/test_logging.py:9 test_sensitive_data_filter

balloob · 2018-02-18T22:53:33Z

For the test that failed on config entries, try swapping the two decorators around in config/config_entries.py line 100 (or convert to async/await)

houndci-bot · 2018-02-23T20:13:00Z

homeassistant/components/group/__init__.py

@@ -569,8 +568,7 @@ def async_will_remove_from_hass(self):
            self._async_unsub_state_changed()
            self._async_unsub_state_changed = None

-    @asyncio.coroutine
-    def _async_state_changed_listener(self, entity_id, old_state, new_state):
+    async def _async_state_changed_listener(self, entity_id, old_state, new_state):


line too long (83 > 79 characters)

houndci-bot · 2018-02-23T20:13:00Z

homeassistant/components/group/__init__.py

@@ -569,8 +568,7 @@ def async_will_remove_from_hass(self):
            self._async_unsub_state_changed()
            self._async_unsub_state_changed = None

-    @asyncio.coroutine
-    def _async_state_changed_listener(self, entity_id, old_state, new_state):
+    async def _async_state_changed_listener(self, entity_id, old_state, new_state):


line too long (83 > 79 characters)

balloob · 2018-02-23T23:13:16Z

Please limit changes to code that is covered by tests (but do not update the tests in the same PR).

MartinHjelmare · 2018-02-24T09:37:44Z

This is not only core changes. Please update PR title.

Julius2342 · 2018-02-24T12:57:26Z

@balloob : how should I handle linting problems within components? (pylint complained at various places that a function from core is (no longer) iterable? I made the functions async where the complexity is low and added a pylint ignore where I saw the risk of breaking something.

I'm also not sure how to handle failed unit tests...

balloob · 2018-02-24T16:26:04Z

You will have to fix the unit tests..

balloob · 2018-02-24T18:29:13Z

homeassistant/components/config/config_entries.py

    @RequestDataValidator(vol.Schema({
        vol.Required('domain'): str,
    }))
+    @asyncio.coroutine


Since this is covered by tests, we could also just add async keyword to function.

balloob · 2018-02-24T18:38:11Z

homeassistant/requirements.py

@@ -24,9 +23,9 @@ def async_process_requirements(hass, name, requirements):
    pip_install = partial(pkg_util.install_package,
                          **pip_kwargs(hass.config.config_dir))

-    with (yield from pip_lock):
+    with (await pip_lock):


I think that this can be replaced with async with pip_lock:

balloob · 2018-02-24T20:01:28Z

Okay I addressed my own comment and fixed the merge conflict, however the logging test still fails. I am unable to repro it locally on Python 3.5.3 🤔 This is why smaller PRs will help, makes it easy to see why something is causing an error.

Julius2342 · 2018-02-25T10:18:12Z

regarding util/test_logging.py :

it is reproducable locally when running all tests, but not when running the single test
it already fails when initializing logging_util.HideSensitiveDataFilter

I fixed this test by doing:

diff --git a/tests/util/test_logging.py b/tests/util/test_logging.py
index 94c8568..c67b2ae 100644
--- a/tests/util/test_logging.py
+++ b/tests/util/test_logging.py
@@ -6,7 +6,6 @@ import threading
 import homeassistant.util.logging as logging_util
 
 
-@asyncio.coroutine
 def test_sensitive_data_filter():
     """Test the logging sensitive data filter."""
     log_filter = logging_util.HideSensitiveDataFilter('mock_sensitive')

(and JFTR: the flaky test_pilight.py test is already flaky on current dev )

Julius2342 requested a review from a team as a code owner February 18, 2018 22:14

homeassistant added cla-signed core labels Feb 18, 2018

homeassistant added the cla-signed label Feb 18, 2018

Julius2342 changed the title ~~[WIP] Removing asyncio.coroutine syntax from HASS core~~ Removing asyncio.coroutine syntax from HASS core Feb 23, 2018

homeassistant added the cla-signed label Feb 23, 2018

houndci-bot reviewed Feb 23, 2018

View reviewed changes

Julius2342 requested a review from a team as a code owner February 23, 2018 21:27

homeassistant added the cla-signed label Feb 23, 2018

Julius2342 changed the title ~~Removing asyncio.coroutine syntax from HASS core~~ Removing asyncio.coroutine syntax from HASS core a d Feb 24, 2018

Julius2342 changed the title ~~Removing asyncio.coroutine syntax from HASS core a d~~ Removing asyncio.coroutine syntax from HASS core and some places within components Feb 24, 2018

homeassistant added the cla-signed label Feb 24, 2018

Julius2342 added 3 commits February 24, 2018 18:03

changed asyncio.coroutine syntax to new async def/await

ea85a2c

removed py34 from tox environment

6daa2ec

reverted some changes within entity.py

eee2f44

Julius2342 force-pushed the async-core branch from c7e5d3e to eee2f44 Compare February 24, 2018 17:15

homeassistant added the cla-signed label Feb 24, 2018

Julius2342 added 7 commits February 24, 2018 18:26

-

0495739

reverted changes within bootstrap.py

046f231

reverted changes within discovery.py

50bed7e

switched decorators

5638ce3

Reverted change within aiohttp_client.py

8e8f86d

reverted change within logging.py

4b4f768

switched decorators

8b6671a

Julius2342 changed the title ~~Removing asyncio.coroutine syntax from HASS core and some places within components~~ Removing asyncio.coroutine syntax from HASS core Feb 24, 2018

balloob reviewed Feb 24, 2018

View reviewed changes

balloob approved these changes Feb 24, 2018

View reviewed changes

Await lock properly

2ccf0e3

homeassistant added the cla-signed label Feb 24, 2018

Merge branch 'dev' into async-core

ffd931e

Julius2342 added 2 commits February 25, 2018 10:37

removed asyncio.coroutine from test

a1fb42f

Merge remote-tracking branch 'upstream/dev' into async-core

c38a6da

balloob merged commit 16cb738 into home-assistant:dev Feb 25, 2018

Julius2342 deleted the async-core branch February 25, 2018 19:52

kellerza mentioned this pull request Feb 26, 2018

Check_config await error #12722

Merged

balloob mentioned this pull request Mar 9, 2018

0.65 #12995

Merged

home-assistant locked and limited conversation to collaborators May 29, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Removing asyncio.coroutine syntax from HASS core #12509

Removing asyncio.coroutine syntax from HASS core #12509

Julius2342 commented Feb 18, 2018 •

edited

Loading

balloob commented Feb 18, 2018

houndci-bot Feb 23, 2018

houndci-bot Feb 23, 2018

balloob commented Feb 23, 2018

MartinHjelmare commented Feb 24, 2018

Julius2342 commented Feb 24, 2018

balloob commented Feb 24, 2018

balloob Feb 24, 2018

balloob Feb 24, 2018

balloob commented Feb 24, 2018

Julius2342 commented Feb 25, 2018 •

edited

Loading

Removing asyncio.coroutine syntax from HASS core #12509

Removing asyncio.coroutine syntax from HASS core #12509

Conversation

Julius2342 commented Feb 18, 2018 • edited Loading

Description

Current state

balloob commented Feb 18, 2018

houndci-bot Feb 23, 2018

Choose a reason for hiding this comment

houndci-bot Feb 23, 2018

Choose a reason for hiding this comment

balloob commented Feb 23, 2018

MartinHjelmare commented Feb 24, 2018

Julius2342 commented Feb 24, 2018

balloob commented Feb 24, 2018

balloob Feb 24, 2018

Choose a reason for hiding this comment

balloob Feb 24, 2018

Choose a reason for hiding this comment

balloob commented Feb 24, 2018

Julius2342 commented Feb 25, 2018 • edited Loading

Julius2342 commented Feb 18, 2018 •

edited

Loading

Julius2342 commented Feb 25, 2018 •

edited

Loading