Update ZHA to also poll Philips Hue lights with new firmware

[TheJulianJES]

Proposed change

The older Philips Hue lights (ZLL profile) do not send attribute reports and have to be polled more often.
In firmware updates from early this year, Signify changed some of the manufacturer from "Philips" to "Signify Netherlands B.V.". This change was only made to some of the newer-style Philips Hue lights (ZHA profile with Bluetooth). These lights do not have to be polled anymore (hence this changed wasn't made back then).
New firmware which doesn't seem to be fully rolled out yet (not even available through the Hue Bluetooth app yet) also changes the manufacturer name on the older Hue bulbs (which have to be polled).

These new firmware images can be found here: Koenkk/zigbee-OTA#47

I'll still have to test this minor change once I get all different types of Hue lights updated. I'll update the PR.

Also, would it (easily) be possible to only match HueLight to the older (ZLL profile) Hue bulbs?

Signature of an older Hue bulb (ZLL profile) which needs this "fast poll": https://paste.ubuntu.com/p/RKbpqqD4Rt/
Signature of a newer Hue bulb (ZHA profile) which does not need any special treatment:

• before re-pair: https://paste.ubuntu.com/p/hM2HGFVqkT/
• after re-pair: https://paste.ubuntu.com/p/D87NfZcFQ8/ (note: quirk not applying because of cluster change 0xfc02 -> 0xfc03) (This will need a separate PR in zha-device-handlers to support both cases)

Type of change

• Dependency upgrade
• Bugfix (non-breaking change which fixes an issue)
• New integration (thank you!)
• New feature (which adds functionality to an existing integration)
• Breaking change (fix/feature causing existing functionality to break)
• Code quality improvements to existing code or addition of tests

Additional information

• This PR fixes or closes issue: fixes #
• This PR is related to issues:

1. Update Hue firmware images (breaking?) Koenkk/zigbee-OTA#47
2. (Add "Signify Netherlands B.V." as a manufacturer for ZHA Philips lights zigpy/zha-device-handlers#728)

• Link to documentation pull request:

Checklist

• The code change is tested and works locally.
• Local tests pass. Your PR cannot be merged unless tests pass
• There is no commented out code in this PR.
• I have followed the development checklist
• The code has been formatted using Black (black --fast homeassistant tests)
• Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

• Documentation added/updated for www.home-assistant.io

If the code communicates with devices, web services, or third-party tools:

• The manifest file has all fields filled out correctly.
  Updated and included derived files by running: python3 -m script.hassfest.
• New or updated dependencies have been added to requirements_all.txt.
  Updated by running python3 -m script.gen_requirements_all.
• Untested files have been added to .coveragerc.

The integration reached or maintains the following Integration Quality Scale:

• No score or internal
• 🥈 Silver
• 🥇 Gold
• 🏆 Platinum

To help with the load of incoming pull requests:

• I have reviewed two other open pull requests in this repository.

[probot-home-assistant]

Hey there @dmulcahey, @Adminiuga, mind taking a look at this pull request as it has been labeled with an integration (zha) you are listed as a code owner for? Thanks!
_{^{(message by CodeOwnersMention)}}

[Adminiuga]

Signature of an older Hue bulb (ZLL profile) which needs this "fast poll":

{
  "node_descriptor": "NodeDescriptor(logical_type=<LogicalType.Router: 1>, complex_descriptor_available=0, user_descriptor_available=0, reserved=0, aps_flags=0, frequency_band=<FrequencyBand.Freq2400MHz: 8>, mac_capability_flags=<MACCapabilityFlags.AllocateAddress|RxOnWhenIdle|MainsPowered|FullFunctionDevice: 142>, manufacturer_code=4107, maximum_buffer_size=71, maximum_incoming_transfer_size=45, server_mask=0, maximum_outgoing_transfer_size=45, descriptor_capability_field=<DescriptorCapability.NONE: 0>, *allocate_address=True, *is_alternate_pan_coordinator=False, *is_coordinator=False, *is_end_device=False, *is_full_function_device=True, *is_mains_powered=True, *is_receiver_on_when_idle=True, *is_router=True, *is_security_capable=False)",
  "endpoints": {
    "11": {
      "profile_id": 49246,
      "device_type": "0x0210",
      "in_clusters": [
        "0x0000",
        "0x0003",
        "0x0004",
        "0x0005",
        "0x0006",
        "0x0008",
        "0x0300",
        "0x1000",
        "0xfc01",
        "0xfc03"
      ],
      "out_clusters": [
        "0x0019"
      ]
    },
    "242": {
      "profile_id": 41440,
      "device_type": "0x0061",
      "in_clusters": [
        "0x0021"
      ],
      "out_clusters": [
        "0x0021"
      ]
    }
  },
  "manufacturer": "Signify Netherlands B.V.",
  "model": "LCT010",
  "class": "zigpy.device.Device"
}

[MattWestb]

LWB010 before update:

Zigbee info
IEEE: 00:17:88:01:02:c6:86:59
Nwk: 0xf1f3


{
  "node_descriptor": "NodeDescriptor(logical_type=<LogicalType.Router: 1>, complex_descriptor_available=0, user_descriptor_available=0, reserved=0, aps_flags=0, frequency_band=<FrequencyBand.Freq2400MHz: 8>, mac_capability_flags=<MACCapabilityFlags.AllocateAddress|RxOnWhenIdle|MainsPowered|FullFunctionDevice: 142>, manufacturer_code=4107, maximum_buffer_size=71, maximum_incoming_transfer_size=45, server_mask=0, maximum_outgoing_transfer_size=45, descriptor_capability_field=<DescriptorCapability.NONE: 0>, *allocate_address=True, *is_alternate_pan_coordinator=False, *is_coordinator=False, *is_end_device=False, *is_full_function_device=True, *is_mains_powered=True, *is_receiver_on_when_idle=True, *is_router=True, *is_security_capable=False)",
  "endpoints": {
    "11": {
      "profile_id": 49246,
      "device_type": "0x0100",
      "in_clusters": [
        "0x0000",
        "0x0003",
        "0x0004",
        "0x0005",
        "0x0006",
        "0x0008",
        "0x1000"
      ],
      "out_clusters": [
        "0x0019"
      ]
    },
    "242": {
      "profile_id": 41440,
      "device_type": "0x0061",
      "in_clusters": [
        "0x0021"
      ],
      "out_clusters": [
        "0x0021"
      ]
    }
  },
  "manufacturer": "Philips",
  "model": "LWB010",
  "class": "zhaquirks.philips.zlldimmablelight.ZLLDimmableLight"
}

Adding OTA file and restarting HA:

2021-08-12 12:33:52 DEBUG (SyncWorker_1) [zigpy.ota.provider] ImageKey(manufacturer_id=4107, image_type=268): /config/zigpy_ota/100B-010C-01002100-ConfLight-Lamps_0012.zigbee, version: 16785664, hw_ver: (None, None), OTA string: 

2021-08-12 12:48:13 DEBUG (MainThread) [zigpy.zcl] [0xf1f3:11:0x0019] OTA query_next_image handler for 'Philips LWB010': field_control=0, manufacture_id=4107, image_type=268, current_file_version=16783874, hardware_version=None
2021-08-12 12:48:13 DEBUG (MainThread) [zigpy.zcl] [0xf1f3:11:0x0019] OTA image version: 16785664, size: 267644. Update needed: True
2021-08-12 12:48:13 INFO (MainThread) [zigpy.zcl] [0xf1f3:11:0x0019] Updating: Philips LWB010

2021-08-12 13:00:48 DEBUG (MainThread) [zigpy.zcl] [0xf1f3:11:0x0019] ZCL request 0x0006: [<Status.SUCCESS: 0>, 4107, 268, 16785664]
2021-08-12 13:00:48 DEBUG (MainThread) [zigpy.zcl] [0xf1f3:11:0x0019] OTA upgrade_end handler for 'Philips LWB010': status=Status.SUCCESS, manufacturer_id=4107, image_type=268, file_version=16785664

Looks like the new names is not being loaded by repower the light so i was deleting it and readding and was getting:

Device info
LWB010
by Signify Netherlands B.V.
tuya ZBGW EZSP


{
  "node_descriptor": "NodeDescriptor(logical_type=<LogicalType.Router: 1>, complex_descriptor_available=0, user_descriptor_available=0, reserved=0, aps_flags=0, frequency_band=<FrequencyBand.Freq2400MHz: 8>, mac_capability_flags=<MACCapabilityFlags.AllocateAddress|RxOnWhenIdle|MainsPowered|FullFunctionDevice: 142>, manufacturer_code=4107, maximum_buffer_size=71, maximum_incoming_transfer_size=45, server_mask=0, maximum_outgoing_transfer_size=45, descriptor_capability_field=<DescriptorCapability.NONE: 0>, *allocate_address=True, *is_alternate_pan_coordinator=False, *is_coordinator=False, *is_end_device=False, *is_full_function_device=True, *is_mains_powered=True, *is_receiver_on_when_idle=True, *is_router=True, *is_security_capable=False)",
  "endpoints": {
    "11": {
      "profile_id": 49246,
      "device_type": "0x0100",
      "in_clusters": [
        "0x0000",
        "0x0003",
        "0x0004",
        "0x0005",
        "0x0006",
        "0x0008",
        "0x1000",
        "0xfc03"
      ],
      "out_clusters": [
        "0x0019"
      ]
    },
    "242": {
      "profile_id": 41440,
      "device_type": "0x0061",
      "in_clusters": [
        "0x0021"
      ],
      "out_clusters": [
        "0x0021"
      ]
    }
  },
  "manufacturer": "Signify Netherlands B.V.",
  "model": "LWB010",
  "class": "zigpy.device.Device"
}

Name changed and quirk is not loaded (i dont knowing if its needed after the last commits) but looks working as normal (have not looking if its still not aging out its neighbors and children but its going back in the "Decommissioned things" box).

Edit:
All strings from the signature looks the same so i think its only the name is being changed and perhaps some bug fixes (likely only in the Philips code then the SDK is very old).

[TheJulianJES]

Signature of a newer Hue bulb (ZHA profile) which does not need any special treatment:

• before re-pairing:

{
  "node_descriptor": "NodeDescriptor(logical_type=<LogicalType.Router: 1>, complex_descriptor_available=0, user_descriptor_available=0, reserved=0, aps_flags=0, frequency_band=<FrequencyBand.Freq2400MHz: 8>, mac_capability_flags=<MACCapabilityFlags.AllocateAddress|RxOnWhenIdle|MainsPowered|FullFunctionDevice: 142>, manufacturer_code=4107, maximum_buffer_size=82, maximum_incoming_transfer_size=128, server_mask=11264, maximum_outgoing_transfer_size=128, descriptor_capability_field=<DescriptorCapability.NONE: 0>, *allocate_address=True, *is_alternate_pan_coordinator=False, *is_coordinator=False, *is_end_device=False, *is_full_function_device=True, *is_mains_powered=True, *is_receiver_on_when_idle=True, *is_router=True, *is_security_capable=False)",
  "endpoints": {
    "11": {
      "profile_id": 260,
      "device_type": "0x010d",
      "in_clusters": [
        "0x0000",
        "0x0003",
        "0x0004",
        "0x0005",
        "0x0006",
        "0x0008",
        "0x0300",
        "0x1000",
        "0xfc01",
        "0xfc02"
      ],
      "out_clusters": [
        "0x0019"
      ]
    },
    "242": {
      "profile_id": 41440,
      "device_type": "0x0061",
      "in_clusters": [],
      "out_clusters": [
        "0x0021"
      ]
    }
  },
  "manufacturer": "Signify Netherlands B.V.",
  "model": "LCA001",
  "class": "zhaquirks.philips.zhaextendedcolorlight.ZHAExtendedColorLight"
}

• after re-pairing:

{
  "node_descriptor": "NodeDescriptor(logical_type=<LogicalType.Router: 1>, complex_descriptor_available=0, user_descriptor_available=0, reserved=0, aps_flags=0, frequency_band=<FrequencyBand.Freq2400MHz: 8>, mac_capability_flags=<MACCapabilityFlags.AllocateAddress|RxOnWhenIdle|MainsPowered|FullFunctionDevice: 142>, manufacturer_code=4107, maximum_buffer_size=82, maximum_incoming_transfer_size=128, server_mask=11264, maximum_outgoing_transfer_size=128, descriptor_capability_field=<DescriptorCapability.NONE: 0>, *allocate_address=True, *is_alternate_pan_coordinator=False, *is_coordinator=False, *is_end_device=False, *is_full_function_device=True, *is_mains_powered=True, *is_receiver_on_when_idle=True, *is_router=True, *is_security_capable=False)",
  "endpoints": {
    "11": {
      "profile_id": 260,
      "device_type": "0x010d",
      "in_clusters": [
        "0x0000",
        "0x0003",
        "0x0004",
        "0x0005",
        "0x0006",
        "0x0008",
        "0x0300",
        "0x1000",
        "0xfc01",
        "0xfc03"
      ],
      "out_clusters": [
        "0x0019"
      ]
    },
    "242": {
      "profile_id": 41440,
      "device_type": "0x0061",
      "in_clusters": [],
      "out_clusters": [
        "0x0021"
      ]
    }
  },
  "manufacturer": "Signify Netherlands B.V.",
  "model": "LCA001",
  "class": "zigpy.device.Device"
}

Note: The quirk not applying because of cluster change 0xfc02 -> 0xfc03)
(This will need a separate PR in zha-device-handlers to support both cases)

[MattWestb]

Sorry i was not looking so good my have also getting one 0xfc03 cluster ;-((

Do we needing quirk for fixing on/off and light level after power reset or is it implanted in the ZCL ?

[TheJulianJES]

zigpy/zigpy#716 and zigpy/zha-device-handlers#872 would fully implement ZCL v7 making this redundant. (Still WIP)

However, it looks like zigpy/zigpy#774 was merged as a fix for now (not fully implementing ZCL v7) but at least providing start_up_on_off start_up_current_level.
The quirk also does that (power_on_state and power_on_level) but it also provides power_on_color_temperature/0x4010 on the Color cluster.

@puddly seems to also have implemented that attribute in the pending PR zigpy/zigpy#716.
If start_up_color_temperature/power_on_color_temperature is added to zigpy, all Philips Hue quirks (for the lights at least) could probably be removed.

It might be best if start_up_color_temperature is first implemented in zigpy before removing the quirks, as a couple of people may use power_on_color_temperature.

PR to add start_up_color_temperature to zigpy: zigpy/zigpy#788
PR to remove all Philips Hue lighting related quirks from zha-device-handlers: zigpy/zha-device-handlers#993

[TheJulianJES]

Tested and working.
After this change, all Hue bulbs (regardless of whether they're old or new Hue lights) are polled in a 3 to 5 minute interval.
I think this change should be implemented for now, as older Hue bulbs (sometimes) do not correctly show their current state for 45 to 75 minutes (normal refresh interval). This happens with (long) transitions, bound devices turning them on/off directly, and so on. Basically, it just applies current functionality to newer firmware Hue bulbs where the manufacturer name changed.

In the future, it should be considered to disable the fast polling for newer (ZHA profile) Hue bulbs (only match HueLight to ZLL profile bulbs).

[Adminiuga]

Lgtm