feat(Zigbee): Recall bounded devices after reboot + IEEE address option for commands #10676

P-R-O-C-H-Y · 2024-12-03T19:08:01Z

Description of Change

This PR adds new feature/fix to recall bounded devices from the binding table after device reboot, with that also the isBound() will return correct value, as the devices will be found if there were some before rebooting.

Other updates:

Fixes print of the IEEE address, which had reverted bytes order
printBoundDevices() can now be called with Print parameter, so printing to serial instead of log_i is possible.
Added missing Zigbee locks, fixing the assert failed: vPortExitCritical port.c:632 (port_uxCriticalNesting[0] > 0) error
Added a cmd timeout 10s for reading manufacturer and model
mirror updates on Switch/ColorDimSwitch, so there is now option to send the command also to the endpoint num + ieee (long address) combination.

Tests scenarios

Tested locally with Switch example + 1 or 3 light/s connected

Test Results

61 files 61 suites 5m 33s ⏱️
21 tests 21 ✅ 0 💤 0 ❌
144 runs 144 ✅ 0 💤 0 ❌

Results for commit 3ee00ea.

♻️ This comment has been updated with latest results.

github-actions · 2024-12-03T21:47:27Z

Memory usage test (comparing PR against master branch)

The table below shows the summary of memory usage change (decrease - increase) in bytes and percentage for each target.

Memory	FLASH [bytes]		FLASH [%]		RAM [bytes]		RAM [%]
Target	DEC	INC	DEC	INC	DEC	INC	DEC	INC
ESP32C6	0	‼️ +18K	0.00	‼️ +3.46	0	⚠️ +1696	0.00	‼️ +5.71
ESP32H2	0	‼️ +15K	0.00	‼️ +2.71	0	⚠️ +1596	0.00	‼️ +5.50

Click to expand the detailed deltas report [usage change in BYTES]

Target	ESP32C6		ESP32H2
Example	FLASH	RAM	FLASH	RAM
Zigbee/examples/Zigbee_Color_Dimmable_Light	‼️ +18K	⚠️ +1680	‼️ +15K	⚠️ +1580
Zigbee/examples/Zigbee_Color_Dimmer_Switch	‼️ +10K	⚠️ +936	‼️ +7K	⚠️ +860
Zigbee/examples/Zigbee_On_Off_Light	‼️ +18K	⚠️ +1696	‼️ +14K	⚠️ +1596
Zigbee/examples/Zigbee_On_Off_Switch	‼️ +11K	⚠️ +920	‼️ +7K	⚠️ +860
Zigbee/examples/Zigbee_Scan_Networks	‼️ +14K	⚠️ +1260	‼️ +10K	⚠️ +1128
Zigbee/examples/Zigbee_Temp_Hum_Sensor_Sleepy	‼️ +16K	⚠️ +972	‼️ +12K	⚠️ +832
Zigbee/examples/Zigbee_Temperature_Sensor	‼️ +13K	⚠️ +944	‼️ +9K	⚠️ +836
Zigbee/examples/Zigbee_Thermostat	‼️ +12K	⚠️ +608	‼️ +9K	⚠️ +540

SuGlider

maybe just a log_i() left behind...

SuGlider · 2024-12-06T02:45:52Z

libraries/Zigbee/src/ZigbeeEP.cpp

+    log_i("Bound devices:");
+    for ([[maybe_unused]]
+    const auto &device : _bound_devices) {
+      log_i("Device on endpoint %d, short address: 0x%x, ieee address: %02x:%02x:%02x:%02x:%02x:%02x:%02x:%02x", device->endpoint, device->short_addr, 


Suggested change

log_i("Device on endpoint %d, short address: 0x%x, ieee address: %02x:%02x:%02x:%02x:%02x:%02x:%02x:%02x", device->endpoint, device->short_addr,

log_d("Device on endpoint %d, short address: 0x%x, ieee address: %02x:%02x:%02x:%02x:%02x:%02x:%02x:%02x", device->endpoint, device->short_addr,

Let's keep log_i for this scenario. If user calls printBoundDevices, and do not provide the Serial where to print, it will be printed with Log Info.

me-no-dev · 2024-12-09T09:37:46Z

libraries/Zigbee/src/ZigbeeEP.cpp

-       const auto &device : _bound_devices) {
-    log_i("Device on endpoint %d, short address: 0x%x", device->endpoint, device->short_addr);
-    print_ieee_addr(device->ieee_addr);
+void ZigbeeEP::printBoundDevices(Print *print) {


Such functions in Arduino should use Print & print instead.

@me-no-dev But can I then use a default parameter? So if there is nothing passed, it will print in log_i?
void printBoundDevices(Print *print = nullptr);

me-no-dev · 2024-12-09T09:41:12Z

PR is OK, but please fix the Print function

P-R-O-C-H-Y · 2024-12-09T10:05:50Z

Added a factory reset, when the device is removed from the network.

P-R-O-C-H-Y · 2024-12-09T10:06:40Z

Ready now.

me-no-dev · 2024-12-09T10:33:26Z

libraries/Zigbee/examples/Zigbee_On_Off_Switch/Zigbee_On_Off_Switch.ino

@@ -191,6 +190,6 @@ void loop() {
  static uint32_t lastPrint = 0;
  if (millis() - lastPrint > 10000) {
    lastPrint = millis();
-    zbSwitch.printBoundDevices();
+    zbSwitch.printBoundDevices(&Serial);


should be zbSwitch.printBoundDevices(Serial); with the new API

* fix(zigbee): Increase timeout, commision again on failure + setScanDuration (espressif#10651) * fix(zigbee): Increase timeout, commision again on failure * fix(zigbee): Update library keywords * feat(Matter): add new MatterColorLight endpoint (espressif#10654) * feat(matter): adds Matter Color Light endpoint * feat(matter): New example => Wifi Prov within Matter as alternative for wireless network provisioning (espressif#10658) * feat(matter): Arduino WiFi Prov example for Matter * feat(matter): Adds Matter Enhanced Color Light Endpoint (CW/WW/RGB) (espressif#10657) * feat(matter): created enhanced color light new matter endpoint and example * feat(matter): Adds a new Matter Endpoint: Generic Switch (smart button) (espressif#10662) * feat(matter): adds new matter generic switch endpoint * fix(matter): no need of arduino preferences here * ci(pre-commit): Apply automatic fixes --------- Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com> * BLECharacteristic optimization (espressif#10665) * BLECharacteristic::notify() optimization GeneralUtils::hexDump() doesn't output anything if the log level is not "VERBOSE". Additionally, it is very CPU intensive, even when it doesn't output anything. So it is much better to *not* call it at all if not needed. In a simple program which calls BLECharacteristic::notify() every 50 ms, the performance gain of this little optimization is 37% in release mode (-O3) and 57% in debug mode. Of course, the "#if ARDUHAL_LOG_LEVEL >= ARDUHAL_LOG_LEVEL_VERBOSE" guard could also be put inside the GeneralUtils::hexDump() function itself. But it's better to put it here also, as it is clearer (indicating a verbose log thing) and it allows to remove the "m_value.getValue().c_str()" call, which is in itself quite CPU intensive. * BLECharacteristic optimization Calls to BLEUtils::buildHexData() don't output anything when the log level is not "VERBOSE" or "DEBUG". As this function is quite CPU intensive, it is better to not call it when not needed. * Remove 3rd party references in code and documentation (espressif#10666) * feat(support): documentation adjustment * feat(support): readme files, commentaries and examples * fix(template): Fix Issue Report Template ----------------------------- Co-authored-by: Lucas Saavedra Vaz <[email protected]> * Tasmota changes * optional Ethernet support (JL1101 driver added) * esp-modem only esp32, esp32s2 and esp32s3 * remove `OpenThread` * remove all BT BLE libraries * remove zigbee * remove SPIFFS * remove Client Secure * remove Provisioning * remove TfLite, Insights and Rainmaker * make GPIOViewer working see arendst/Tasmota@9696118 * remove FS log which is just littering * refactor(uart): Refactor UART test to work with any number of UARTs (espressif#10593) * refactor(uart): Refactor UART test to work with any number of UARTs Co-authored-by: Rodrigo Garcia <[email protected]> * fix(uart): Set CPU freq on ESP32 * ci(pre-commit): Apply automatic fixes * fix(spelling): Fix codespell error --------- Co-authored-by: Rodrigo Garcia <[email protected]> Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com> * feat (Variants) Added custom boards variants GLYPH C3, GLYPHC6 & GLYPHH2 (espressif#10671) * Added custom boards GLYPH C3, GLYPHC6 & GLYPHH2 based on ESP32C3, ESP32C6 & ESP32H2 * feat(Variants) : Added custom boards GLYPH C3, GLYPHC6 & GLYPHH2 Added custom boards variants from PCBCUPID - GLYPH C3, GLYPHC6 & GLYPHH2 based on ESP32C3, ESP32C6 & ESP32H2 * added required fix : moved the variants to end and fixed build.board * ci(pre-commit): Apply automatic fixes --------- Co-authored-by: srini <[email protected]> Co-authored-by: Jan Procházka <[email protected]> Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com> * feat(Zigbee): Recall bounded devices after reboot + IEEE address option for commands (espressif#10676) * feat(zigbee): Recall bound devices after reboot * fix(zigbee): Add missing locks + allow printBoundDevices to Serial * fix(Zigbee): Add locks to temp sensor setReporting * fix(Zigbee): remove unnecessary space in formatting * fix(Zigbee): proper parameter in printBoundDevices * feat(Zigbee): factory reset when removed from network * fix(zigbee): Update comment * fix(zigbee): fix serial and add missing factoryReset to example * ci(pre-commit): Apply automatic fixes --------- Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com> * ci(pre-commit): Bump hooks versions and fix leftover files (espressif#10680) * update(hooks): Bump pre-commit hooks versions * fix(formatting): Fix python script formatting * fix(formatting): Fix leftover files on protected folders * feat(Matter): Creates New Matter Fan Controller Endpoint (espressif#10691) * feat(matter): creates new matter fan controller endpoint * change(tools): Push generated binaries to PR * ci(pre-commit): Add bash script formatter and linter (espressif#10681) * ci(pre-commit): Add check for bash scripts * fix(formatting): Fix bash scripts * docs(pre-commit): Add info about the included hooks * fix ESP32-U4WDH chip detection by ESP.getChipModel() (espressif#10696) * fixes chip detection for ESP32-U4WDH * fix(push): Re-comment unused code * feat(matter): adds new Temperature Sensor Matter Endpoint (espressif#10698) * feat(matter): adds new temperature sensor matter endpoint * feat(matter): commentaries review and fixes * feat(matter): commentaries review and fixes * feat(matter): commentaries review and fixes * feat(matter): commentaries review and fixes * feat(matter): commentaries review and fixes * feat(matter): commentaries review and fixes * feat(matter): general commentaries and code review * feat(matter): keeping arduino style for local variables (lower case) * feat(matter): applies a generic temperature unit to the implementation and example * fix(matter): fixed problem with begin(float) implementation * fix(matter): fixed begin(float) initiallization * feat(matter): updated matter temperature keywords with new api * ci(pre-commit): Apply automatic fixes * fix(matter): fixed code spell ci errors in matter temperature sensor --------- Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com> * feat(chip): Add definition for BOOT_PIN for all chips (espressif#10700) * feat(chip): Add definition for BOOT_PIN for all chips For use in sketches as default button * fix(core): Make BOOT_PIN static * fix(hal): BOOT_PIN should always be defined * feat(Matter): Adds New Matter Humidity Sensor Endpoint (espressif#10703) * feat(matter): adds matter humidity sensor endpoint * Update and rename platformio-build.py to pioarduino-build.py * feat(matter_examples): apply boot button change to all examples (espressif#10702) * feat(matter_examples): apply boot button change to all examples * ci(tests): Re-enable UART test for P4 * Delete libraries/Matter directory --------- Co-authored-by: Rodrigo Garcia <[email protected]> Co-authored-by: Jan Procházka <[email protected]> Co-authored-by: Me No Dev <[email protected]> Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com> Co-authored-by: Sylvain Quendez <[email protected]> Co-authored-by: Lucas Saavedra Vaz <[email protected]> Co-authored-by: PCB CUPID <[email protected]> Co-authored-by: srini <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Michael Stegen <[email protected]>

tofurky · 2024-12-28T21:34:57Z

Hi @P-R-O-C-H-Y, I think there may be a subtle issue with this.

In handling ESP_ZB_ZDO_SIGNAL_LEAVE, there are apparently 2 leave types: ZB_NWK_LEAVE_TYPE_REJOIN and ZB_NWK_LEAVE_TYPE_RESET. However it's ignored and it always calls Zigbee.factoryReset().

My ESP32C6 seemed to randomly factory reset itself, so I started looking into the code. Months ago, in packet captures, I had noticed an issue where IKEA bulbs (Silicon Designs stack, I think) were telling my Lutron Zigbee remotes (Atmel?) to "leave", which dropped them off the network. I think something similar may have happened here.

So, perhaps we should only be resetting if ZB_NWK_LEAVE_TYPE_RESET is set? And/or if it's from the coordinator?

[26523678][V][ZigbeeTempSensor.cpp:112] setHumidity(): Updating humidity sensor value...
[26523678][D][ZigbeeTempSensor.cpp:114] setHumidity(): Setting humidity to 2090
[26523679][V][ZigbeeTempSensor.cpp:71] setTemperature(): Updating temperature sensor value...
[26523679][D][ZigbeeTempSensor.cpp:73] setTemperature(): Setting temperature to 2430
[26524179][I][o2.ino:140] zigbee_reading_handler(): Sent average of readings of oxygen=20.9%, temperature=24.3 C (60 total readings)
[26524677][D][o2.ino:57] sensor_read(): oxygen=20.9% temperature=24.3 C (2518 us)
[26525677][D][o2.ino:57] sensor_read(): oxygen=20.9% temperature=24.4 C (2416 us)
[26526677][D][o2.ino:57] sensor_read(): oxygen=20.9% temperature=24.3 C (2485 us)
[26526711][I][ZigbeeCore.cpp:225] esp_zb_app_signal_handler(): Device started up in non factory-reset mode
[26526711][I][ZigbeeCore.cpp:238] esp_zb_app_signal_handler(): Device rebooted
[26526712][D][ZigbeeCore.cpp:466] searchBindings(): Requesting binding table for address 0x81e6
[26526713][D][ZigbeeCore.cpp:406] bindingTableCb(): Binding table callback for address 0x81e6 with status 0
[26526714][D][ZigbeeCore.cpp:409] bindingTableCb(): Binding table info: total 2, index 0, count 2
[26526714][D][ZigbeeCore.cpp:419] bindingTableCb(): Binding table record: src_endp 10, dst_endp 1, cluster_id 0x0402, dst_addr_mode 3
[26526715][D][ZigbeeCore.cpp:436] bindingTableCb(): Device bound to EP 10 -> device endpoint: 1, short addr: 0x0000, ieee addr: <censored>
[26526715][D][ZigbeeCore.cpp:419] bindingTableCb(): Binding table record: src_endp 10, dst_endp 1, cluster_id 0x0405, dst_addr_mode 3
[26526716][D][ZigbeeCore.cpp:436] bindingTableCb(): Device bound to EP 10 -> device endpoint: 1, short addr: 0x0000, ieee addr: <censored>
[26526717][D][ZigbeeCore.cpp:457] bindingTableCb(): Filling bounded devices finished
[26527677][D][o2.ino:57] sensor_read(): oxygen=20.9% temperature=24.4 C (2539 us)
[26528677][D][o2.ino:57] sensor_read(): oxygen=20.9% temperature=24.4 C (2408 us)
[26529677][D][o2.ino:57] sensor_read(): oxygen=20.9% temperature=24.3 C (2336 us)
[26530677][D][o2.ino:57] sensor_read(): oxygen=20.9% temperature=24.2 C (2474 us)
[26531677][D][o2.ino:57] sensor_read(): oxygen=20.9% temperature=24.4 C (2433 us)
[26532677][D][o2.ino:57] sensor_read(): oxygen=20.9% temperature=24.3 C (2415 us)
[26533677][D][o2.ino:57] sensor_read(): oxygen=20.9% temperature=24.3 C (2457 us)
[26534677][D][o2.ino:57] sensor_read(): oxygen=20.9% temperature=24.4 C (2414 us)
[26535677][D][o2.ino:57] sensor_read(): oxygen=20.9% temperature=24.3 C (2399 us)
[26536677][D][o2.ino:57] sensor_read(): oxygen=20.9% temperature=24.3 C (2451 us)
[26537677][D][o2.ino:57] sensor_read(): oxygen=20.9% temperature=24.4 C (2413 us)
[26538677][D][o2.ino:57] sensor_read(): oxygen=20.9% temperature=24.5 C (2408 us)
[26539677][D][o2.ino:57] sensor_read(): oxygen=20.9% temperature=24.3 C (2449 us)
[26540677][D][o2.ino:57] sensor_read(): oxygen=20.9% temperature=24.3 C (2526 us)
[26541677][D][o2.ino:57] sensor_read(): oxygen=20.9% temperature=24.3 C (2402 us)
[26542032][V][ZigbeeCore.cpp:344] factoryReset(): Factory resetting Zigbee stack, device will reboot
ESP-ROM:esp32c6-20220919
Build:Sep 19 2022
rst:0xc (SW_CPU),boot:0xc (SPI_FAST_FLASH_BOOT)
Saved PC:0x4001975a
SPIWP:0xee
mode:DIO, clock div:2
load:0x40875720,len:0x1228
load:0x4086c110,len:0xd9c
load:0x4086e610,len:0x2f74
entry 0x4086c110

P-R-O-C-H-Y self-assigned this Dec 3, 2024

P-R-O-C-H-Y added the Area: Libraries Issue is related to Library support. label Dec 3, 2024

P-R-O-C-H-Y force-pushed the feat/zigbee-recall-binding-table branch from 78c6725 to 97643ec Compare December 3, 2024 21:42

P-R-O-C-H-Y added the Status: Review needed Issue or PR is awaiting review label Dec 4, 2024

P-R-O-C-H-Y requested review from me-no-dev, lucasssvaz and SuGlider December 4, 2024 07:45

P-R-O-C-H-Y added Status: In Progress Issue is in progress and removed Status: Review needed Issue or PR is awaiting review labels Dec 4, 2024

feat(zigbee): Recall bound devices after reboot

d45a267

P-R-O-C-H-Y force-pushed the feat/zigbee-recall-binding-table branch from 97643ec to d45a267 Compare December 4, 2024 11:45

P-R-O-C-H-Y added Status: Review needed Issue or PR is awaiting review and removed Status: In Progress Issue is in progress labels Dec 4, 2024

fix(zigbee): Add missing locks + allow printBoundDevices to Serial

a6c9d5c

P-R-O-C-H-Y mentioned this pull request Dec 4, 2024

Zigbee: problem with battery powered devices after power off/on #10612

Closed

1 task

fix(Zigbee): Add locks to temp sensor setReporting

fa53967

SuGlider approved these changes Dec 6, 2024

View reviewed changes

fix(Zigbee): remove unnecessary space in formatting

40c56b5

lucasssvaz approved these changes Dec 6, 2024

View reviewed changes

me-no-dev reviewed Dec 9, 2024

View reviewed changes

P-R-O-C-H-Y added 2 commits December 9, 2024 11:03

fix(Zigbee): proper parameter in printBoundDevices

c47324a

feat(Zigbee): factory reset when removed from network

21409b4

P-R-O-C-H-Y force-pushed the feat/zigbee-recall-binding-table branch from 9319ce5 to 21409b4 Compare December 9, 2024 10:05

fix(zigbee): Update comment

8e6cbfe

me-no-dev reviewed Dec 9, 2024

View reviewed changes

fix(zigbee): fix serial and add missing factoryReset to example

c8f7c1b

me-no-dev approved these changes Dec 9, 2024

View reviewed changes

me-no-dev added Status: Pending Merge Pull Request is ready to be merged and removed Status: Review needed Issue or PR is awaiting review labels Dec 9, 2024

ci(pre-commit): Apply automatic fixes

3ee00ea

me-no-dev merged commit 92dd841 into espressif:release/v3.1.x Dec 9, 2024
49 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(Zigbee): Recall bounded devices after reboot + IEEE address option for commands #10676

feat(Zigbee): Recall bounded devices after reboot + IEEE address option for commands #10676

P-R-O-C-H-Y commented Dec 3, 2024 •

edited

Loading

github-actions bot commented Dec 3, 2024 •

edited

Loading

github-actions bot commented Dec 3, 2024 •

edited

Loading

github-actions bot commented Dec 3, 2024 •

edited

Loading

SuGlider left a comment

SuGlider Dec 6, 2024

P-R-O-C-H-Y Dec 6, 2024

me-no-dev Dec 9, 2024

P-R-O-C-H-Y Dec 9, 2024

P-R-O-C-H-Y Dec 9, 2024

me-no-dev commented Dec 9, 2024

P-R-O-C-H-Y commented Dec 9, 2024

P-R-O-C-H-Y commented Dec 9, 2024

me-no-dev Dec 9, 2024

tofurky commented Dec 28, 2024

	log_i("Device on endpoint %d, short address: 0x%x, ieee address: %02x:%02x:%02x:%02x:%02x:%02x:%02x:%02x", device->endpoint, device->short_addr,
	log_d("Device on endpoint %d, short address: 0x%x, ieee address: %02x:%02x:%02x:%02x:%02x:%02x:%02x:%02x", device->endpoint, device->short_addr,

feat(Zigbee): Recall bounded devices after reboot + IEEE address option for commands #10676

feat(Zigbee): Recall bounded devices after reboot + IEEE address option for commands #10676

Conversation

P-R-O-C-H-Y commented Dec 3, 2024 • edited Loading

Description of Change

Tests scenarios

Related links

github-actions bot commented Dec 3, 2024 • edited Loading

github-actions bot commented Dec 3, 2024 • edited Loading

Test Results

github-actions bot commented Dec 3, 2024 • edited Loading

Memory usage test (comparing PR against master branch)

SuGlider left a comment

Choose a reason for hiding this comment

SuGlider Dec 6, 2024

Choose a reason for hiding this comment

P-R-O-C-H-Y Dec 6, 2024

Choose a reason for hiding this comment

me-no-dev Dec 9, 2024

Choose a reason for hiding this comment

P-R-O-C-H-Y Dec 9, 2024

Choose a reason for hiding this comment

P-R-O-C-H-Y Dec 9, 2024

Choose a reason for hiding this comment

me-no-dev commented Dec 9, 2024

P-R-O-C-H-Y commented Dec 9, 2024

P-R-O-C-H-Y commented Dec 9, 2024

me-no-dev Dec 9, 2024

Choose a reason for hiding this comment

tofurky commented Dec 28, 2024

P-R-O-C-H-Y commented Dec 3, 2024 •

edited

Loading

github-actions bot commented Dec 3, 2024 •

edited

Loading

github-actions bot commented Dec 3, 2024 •

edited

Loading

github-actions bot commented Dec 3, 2024 •

edited

Loading