XMTZC04HM Extension

[DigiH]

Latest XMTZC04HM Update for testing with multiple model and property conditions.
Should work fine with kg for person and object weighing recognition, possibly also already for lbs, but needs testing and verification for person and object lbs weighings.

Checklist:

• The pull request is done against the latest development branch
• Only one feature/fix was added per PR and the code change compiles without warnings
• I accept the DCO.

[DigiH]

@h2zero with my last test trial commit I changed the model condition down from 4 OR to only 2 before the AND for the tests not to fail

"condition":["servicedata", "index", 0, "2", "|", "servicedata", "index", 0, "a", "|", "servicedata", "index", 0, "6", "|", "servicedata", "index", 0, "e", "&", "uuid", "contain", "181d"],

to

"condition":["servicedata", "index", 0, "2", "|", "servicedata", "index", 0, "6", "&", "uuid", "contain", "181d"],

where we both assumed that it should also work with more than 2 OR.

Could you please have a look at what might be the issue there, or can you see any other mixup previously?

Thanks

Should be still valid for testing with the "2" and "6" (display still on stabilised) for person and object weighing.

[DigiH]

Yes, just did a quick check with just adding a 3rd OR model condition back in. Causes tests to fail.

Reverting back to only 2 OR.

[h2zero]

@DigiH yes, you're correct. That condition set would fail in this case if one of the "or" cases matched and the "and" case was not the next case tested.

To correct this please try adding this block:

        } else if (match && *cond_str == '|') {
          i += 3; 

          while (*cond_str != '&' && i < condition.size()) {
            cond_str = condition[++i].as<const char*>();
          }

          if (i < condition.size() && cond_str != nullptr) {
            i++;
            continue;
          }
        }

at this line:

decoder/src/decoder.cpp

Line 266 in 04c103c

}

[DigiH]

@h2zero unfortunately that didn't quite work as expected. Reverted back to just the two working OR before the AND without the decoder.cpp change.

[1technophile]

@DigiH with this branch my XMTZC04HM is detected as a XMTZC05HM for info

[DigiH]

@1technophile That is very odd, as this branch clearly has "&", "uuid", "contain", "181d" for XMTZC04HM and "&", "uuid", "contain", "181b" for XMTZC05HM in the model conditions. So your "0x181d" really shouldn't successfully trigger the XMTZC05HM decoder ;)

Since your servicedata index 0 shows 0 which most likely indicates unstabilised it is understood that it doesn't get caught by the XMTZC04HM decoder, especially with the currently only implemented 2 and 6 index 0 model conditions, But the fact that the XMTZC05HM decoder then kicked in is not good.

Did this also happen for the followed (green) changed update in MQTTExplorer. where index 0 is a?

@h2zero while the XMTZC05HM works flawlessly for my XMTZC05HM in all cases and all the included test cases, could it be that, while the 2 OR plus 1 AND condition checking routine works there, it also somehow catches @1technophile's XMTZC04HM after being rejected by the preceding XMTZC04HM decoder, still having a true match lingering for the last UUID AND check there, and then triggering with the first true OR condition in the XMTZC05HM decoder?

[1technophile]

@DigiH let me retest it again in a couple of hours

[DigiH]

@1technophile the person weighing with the display still on and an object weighing with the scale display still on should both work with the current XMTZC04HM decoder, that's what I left the 2 and 6 in for testing. Hope these work fine for you.

[h2zero]

@h2zero unfortunately that didn't quite work as expected. Reverted back to just the two working OR before the AND without the decoder.cpp change.

That's unfortunate, I'll do some testing when the rest of this change is stable and see then what can be done.

@h2zero while the XMTZC05HM works flawlessly for my XMTZC05HM in all cases and all the included test cases, could it be that, while the 2 OR plus 1 AND condition checking routine works there, it also somehow catches @1technophile's XMTZC04HM after being rejected by the preceding XMTZC04HM decoder, still having a true match lingering for the last UUID AND check there, and then triggering with the first true OR condition in the XMTZC05HM decoder?

I don't see how it would be possible. I'll see what I can do for test data to be sure.

[1technophile]

So not sure what happens with the previous test, here is my last data.

For an object it works:

{"id":"C8","mac_type":0,"name":"MI SCALE2","manufacturerdata":"5701c8","rssi":-81,"servicedata":"625c03b207010100122b","servicedatauuid":"0x181d","brand":"Xiaomi","model":"Mi_Smart_Scale","model_id":"XMTZC01HM/XMTZC04HM","weighing_mode":"object","unit":"kg","weight":4.3}

but not for a person:

{"id":"C8","mac_type":0,"name":"MI SCALE2","manufacturerdata":"5701c8","rssi":-86,"servicedata":"a29231b2070101001403","servicedatauuid":"0x181d"}

Not sure now why I have an a instead of a 2 for the first char.

[DigiH]

Yes, the first

{"id":"C8","mac_type":0,"name":"MI SCALE2","manufacturerdata":"5701c8","rssi":-81,"servicedata":"625c03b207010100122b","servicedatauuid":"0x181d","brand":"Xiaomi","model":"Mi_Smart_Scale","model_id":"XMTZC01HM/XMTZC04HM","weighing_mode":"object","unit":"kg","weight":4.3}

is an object with the scale display still on, which I already implemented with the currently limited 2 OR conditions available.

{"id":"C8","mac_type":0,"name":"MI SCALE2","manufacturerdata":"5701c8","rssi":-80,"servicedata":"e25c03b2070101001236","servicedatauuid":"0x181d"}

I think this is still an object, but the scale light off now, with the leading e, which you previously stated as weighing an object, but not yet implemented due to the currently still limited OR options. The same with "a" in the next one, which should be a person and the scale light off.

The only second option to work should be person with light stil on, which should have the servicedata starting with a 2.

Not sure now why I have an a instead of a 2 for the first char.

I know, you previously said you've never seen the a, it should be a person weighing but the scale light off.

So with the current limit to two OR model conditions, only these work already in the XMTZC04HM decoder.

index 0 = 2 - person, scale display still on
index 0 = 6 - object, scale display still on

[1technophile]

I will do some repeated testing to see if I get back the 2 and at what frequency.

[DigiH]

2 usually requires a short TimeBtwRead as it's only the 1st broadcast of the weighing, after that it's always a, for 20 minutes on the XMTZC05HM.

[h2zero]

Just to update my previous comment on the incorrect decoder selection; I have tested with the data provided in @1technophile's post and the result is "device not found (-1)".

[DigiH]

I've just started a discussion, but we can also continue here. Since I suspected above that, now that we have two consecutive decoders with the pseudo nested 2 OR and one AND, it seems that the last AND UUID condition of the XMTZC04HM tiggers a match and then if the first OR condition of the following XMTZC05HM decoder is also a match it triggers that following decoder. Hence it only happened to @1technophile when his servciedata had

"servicedata", "index", 2, "2"

but not in his more recent test

{"id":"C8","mac_type":0,"name":"MI SCALE2","manufacturerdata":"5701c8","rssi":-86,"servicedata":"a29231b2070101001403","servicedatauuid":"0x181d"}

To test this I have created a dummy XXX decoder and placed it after the nicely working XMTZC05HM decoder

https://github.com/DigiH/decoder/blob/multipleconditionstest/src/devices/XXX_json.h

So if I now weigh an object but keep jiggling it, so that there is not stablised XMTZC05HM decoder recognition I get
{"id":"","mac_type":0,"name":"MIBFS","manufacturerdata":"57010c9541d1e284","rssi":-83,"servicedata":"0204e607021803162600004e02","servicedatauuid":"0x181b","brand":"XXXTEST","model":"XXX","model_id":"XXYY","unit":"BOOM"}

So the XMTZC05HM decoder matches the UUID, this match is somehow kept, and I created the dummy XXX decoder to always match

"servicedata", "index", 18, "0", "|", "servicedata", "index", 19, "0"

which are always 0 when weighing objects (it's the impedance)

I have tested with the data provided in @1technophile's post and the result is "device not found (-1)".

So not sure if this is a real world vs. test discrepancy

[h2zero]

I can confirm this, the AND is not being processed correctly. I've rigged up a test and will update shortly.

[h2zero]

@DigiH please rebase and test with this branch https://github.com/theengs/decoder/tree/multi-or-and-detect

In my testing, multiple OR cases are working properly and the final AND case should now always be checked.

[DigiH]

Thanks a lot @h2zero all kg tests working fine now with 4 OR and the one AND condition.

@1technophile should be all ready for you to test with your kg setting.

Best case scenario is that lbs will only need the same 4 OR condtions, worst casse an addtional 4.

Ready for lbs testing.

[h2zero]

You're welcome, thanks for the hard work on these decodes!

[DigiH]

Thanks for putting up with me. No more such complex decoders from me in the near future - promise :)

[h2zero]

Lol, for some reason I doubt that promise 😄

In all seriousness, this is great collaboration and a good opportunity to tweak the decoder for the future. Still more work to do in the nested conditions category but it's working well so far.

[DigiH]

In all seriousness, with such quick implementations and fixes as you've done it really is a joy to collaborate! :)

[DigiH]

@github768484 or anyone else using the Mi Smart Scale with lbs setting, the branch is ready for testing with changing the decoder to

decoder = https://github.com/DigiH/decoder#XMTZC04HM-Extension

in platform.ini

Additionally it would be great to have

'-DpubKnownBLEServiceData=true'

in the environment build_flags, to also see the related servicedata.

Thanks

[1technophile]

I confirm that it works with kg, thanks @DigiH

[github768484]

Help me out gents.. How might I load this branch onto the ESP32dev via Arduino IDE? I can also flash the .bin files if provided.

[DigiH]

Hi @github768484 , can you try with the binary here

https://github.com/DigiH/TestBin/raw/main/Bins/XMTZC04HMTest.bin

or zipped

https://github.com/DigiH/TestBin/raw/main/Bins/XMTZC04HMTest.zip

Would be great if you could try all four possible scenarios and show us the mqtt broker JSON output for them.

person weighing - scale lights still on

person weighing - scale lights off

object weighing - scale lights still on

object weighing - scale lights off

Press on the scale to turn it on. Wait until “0.00” is displayed and place the item you want to weigh onto the scale.

Thanks a lot

[github768484]

@DigiH I was expecting to see "primary" bin along with a partitions.bin. I have a bootloader bins prepared. Please forgive my ignorance with this. We are a the boundary of my knowledge with programming. I am more than happy to help out where I can.. just need a bit of handholding in these waters.

[DigiH]

@github768484 Now I'm showing my ignorance, not knowing too much of what you're talking about ;) as my knowledge of the Flash Tool is virtually non-existent.

Am I right in assuming that you usually download these files

esp32dev-ble-firmware.bin
esp32dev-ble-libraries.zip
esp32dev-ble-partitions.bin 

from here https://github.com/1technophile/OpenMQTTGateway/releases, and then upload them with the Flash Tool?

EDITED:

Here are the three relevant test files

https://github.com/DigiH/TestBin/raw/main/esp32dev-ble-firmware.bin
https://github.com/DigiH/TestBin/raw/main/esp32dev-ble-libraries.zip
https://github.com/DigiH/TestBin/raw/main/esp32dev-ble-partitions.bin

Please let me know if I assumed right.

Thanks

[github768484]

@DigiH We are in business. I am getting a reading for both an object and a person. It appears that the reading continues to post the last measured value until the scale is updated with another value. The log below should help you see what I am referring to. This was only a short duration test of about 4 minutes in total. Please let me know if I can assist you further. I currently get BT data from the Mi Scale as well as an Inkbird Temperature module.

N: Scan begin
N: Send on /BTtoMQTT/C8478CD10EBE msg {"id":"C8:47:8C:D1:0E:BE","mac_type":0,"name":"MI SCALE2","rssi":-49,"servicedata":"637404e607030113271c","brand":"Xiaomi","model":"Mi_Smart_Scale","model_id":"XMTZC01HM/XMTZC04HM","weighing_mode":"object","unit":"lb","weight":11.4}

N: Send on /BTtoMQTT/C8478CD10EBE msg {"id":"C8:47:8C:D1:0E:BE","mac_type":0,"name":"MI SCALE2","rssi":-48,"servicedata":"e37404e6070301132739","brand":"Xiaomi","model":"Mi_Smart_Scale","model_id":"XMTZC01HM/XMTZC04HM","weighing_mode":"object","unit":"lb","weight":11.4}

N: Send on /BTtoMQTT/C8478CD10EBE msg {"id":"C8:47:8C:D1:0E:BE","mac_type":0,"name":"MI SCALE2","rssi":-49,"servicedata":"e37404e6070301132739","brand":"Xiaomi","model":"Mi_Smart_Scale","model_id":"XMTZC01HM/XMTZC04HM","weighing_mode":"object","unit":"lb","weight":11.4}

N: Send on /BTtoMQTT/C8478CD10EBE msg {"id":"C8:47:8C:D1:0E:BE","mac_type":0,"name":"MI SCALE2","rssi":-53,"servicedata":"a3984ee6070301132a00","brand":"Xiaomi","model":"Mi_Smart_Scale","model_id":"XMTZC01HM/XMTZC04HM","weighing_mode":"person","unit":"lb","weight":201.2}

N: Send on /BTtoMQTT/C8478CD10EBE msg {"id":"C8:47:8C:D1:0E:BE","mac_type":0,"name":"MI SCALE2","rssi":-46,"servicedata":"a3984ee6070301132a00","brand":"Xiaomi","model":"Mi_Smart_Scale","model_id":"XMTZC01HM/XMTZC04HM","weighing_mode":"person","unit":"lb","weight":201.2}

N: Send on /BTtoMQTT/C8478CD10EBE msg {"id":"C8:47:8C:D1:0E:BE","mac_type":0,"name":"MI SCALE2","rssi":-46,"servicedata":"a3984ee6070301132a00","brand":"Xiaomi","model":"Mi_Smart_Scale","model_id":"XMTZC01HM/XMTZC04HM","weighing_mode":"person","unit":"lb","weight":201.2}

N: Send on /BTtoMQTT/C8478CD10EBE msg {"id":"C8:47:8C:D1:0E:BE","mac_type":0,"name":"MI SCALE2","rssi":-95,"servicedata":"a3984ee6070301132a00","brand":"Xiaomi","model":"Mi_Smart_Scale","model_id":"XMTZC01HM/XMTZC04HM","weighing_mode":"person","unit":"lb","weight":201.2}

Notes on the firmware install: I generated my on bootloader bins and installed the following bins provided by you:
https://github.com/DigiH/TestBin/raw/main/esp32dev-ble-firmware.bin
https://github.com/DigiH/TestBin/raw/main/esp32dev-ble-partitions.bin

The ble-libraries provided by you should have been compiled into the bin files provided by you. I believe that this "compiled" statement is correct.

Cheers,

JA

[DigiH]

We are in business. I am getting a reading for both an object and a person. It appears that the reading continues to post the last measured value until the scale is updated with another value.

Great, thanks. On my XMTZC05HM Mi Body Composition Scale the values are being repeated for 20 minutes, then they stop and the scale goes into stand by not broadcasting anything, so it might be similar on your XMTZC04HM. This now allows to not have to set the TimeBtwRead to a low setting to get stabilised readings.

The ble-libraries provided by you should have been compiled into the bin files provided by you. I believe that this "compiled" statement is correct.

That's correct, just me not knowing that much about the Flash Tool installation i wanted to make sure that I provided all the usually available files :) for direct upload to an ESP32 they would have be far too large anyway.

This was only a short duration test of about 4 minutes in total.

There is one instance missing, the servicedata starting with "2…", which is the case of person weighing and the lights still on, but I think we can safely assume that it'll be the same, as all the other 3 possibilities are exactly the same as with kg weighing.

I can now add the additional lb test cases to this PR.

Thanks a lot again for your testing, @github768484!

[DigiH]

Checklist:

• The pull request is done against the latest development branch
• Only one feature/fix was added per PR and the code change compiles without warnings
• I accept the DCO.

[1technophile]

Thanks for the dev and the test!