possible improvement of perfomance

[Anto79-ops]

hello!

I'm using Bermuda with 3 Olimex's, Shelly, and NUC BT for both active and passive, on my NUC 12 with HAOS bare-metal. When searching for perfomance bottlenecks on my system, turns out Bermuda BLE comes in 4th, Performance was assesed with callgrinds using the Profiler integration in HAOS. Turns out Bermuda is performing about 30000/updates/min plus 90000 calculations. Some visuals here:

If you would like more details, feel free to ping

thanks

[agittins]

Howdy! Yes, definitely interested in more info.

Is Bermuda actually causing performance issues on this system? I ask because profiling info is great, but unless it's pointing to actual issues then it can easily turn into a wild goose chase. :-)

Bermuda will always use more resources than most integrations, because every second we're doing calculations to build a picture of the entire RF space around the home. But I am very interested in being able to tell if there are big dumb areas that I've missed or where optimisations will make a sizeable difference. It's a bit alarming that 3 other integrations are taking more cpu time than Bermuda! 😅

Note that we should be seeing something in the order of (device_count * scanner_count) hits to update_advertisement per second, where device_count is the number of seen BLE devices, not just configured ones.

Have you ever had the hass_bluetooth_proxy integration installed? The one that lets you use Android devices as scanners? I ask because I suspect there's an interaction between it and Bermuda that I need to track down and might be a significant performance issue.

[bdraco]

1,2,3 are asyncio, selector, and python internal overhead. Bermuda is 4

As far as integrations go in the profile, Bermuda is solidly #1

[agittins]

Hah, winning? 😅

That makes more sense, other integrations might do more processing from time to time, but for operations done every single second, Bermuda is probably hard to match. On my test systems the overhead is usually "fairly low" with updates taking around 10ms, but with varied hardware it's hard to know - plus I've just worked out that it seems that at least one integration saves every single advert it ever sees, forever in the cache, which is causing some systems to crumble when Bermuda starts up and starts crunching 6 months worth of old mac addresses :-(

I'm definitely interested in what's happening on Anto79-ops' system.

[Anto79-ops]

Thanks for the comments. I'm not using the hass_bluetooth_proxy integration. I mean I have bluetooth enabled on my NUC12, hci0.

I originally had 7 wifi-based ESP32 BT proxies and was using those (plus a few other proxies) for sensors and room presence. It was OK, not great for room presence. But then I decided to clean up the 2.4 ghz spectrum in my home, so I got a few of the olimex ones and took the 7 wifi based devices offline. I monitor 6 beacons.

With the olimex's range, room presence becomes.... well, bad. But your integration provided an option to see distance for all proxies at once. Hence, I can get a rough idea for room presence using the olimexs.

I did notice a tick up in cpu usage with your integration, hence we're here discussing it.

Thanks for your integration!

[agittins]

Great, thanks for the context!

You mentioned getting distances to each proxy - does that mean you've enabled a number of the "distance to..." and "unfiltered distance to..." sensors?

They definitely will impact performance, as they don't have the same rate-limiting the area and distance sensors use, so they'll make the ui a lot slower due to the amount of data and rate of change (even if they're not in any dashboards i think the front end still tracks their state).

It will also beef up the amount of data you are logging to the database, with keeping an eye on if you're short on space or i/o (ssd should be fine, but if you are using non-stock recorder settings it'd be worth taking a look at them).

Happy to chat further to see how the default area matching fell short for you, and to see if we can tweak some things to get you more out of the system! First step would probably be a "download diagnostics" (it goes quicker if you've recently restarted, it might chug away for a while) and if you are able, the result of clicking "enable debug logging" in Bermuda for about 30 seconds or so.

The diagnostics will have personal info like Mac address (but not device names) redacted, but the debug log won't, so you could email it to me at [email protected] if you're comfortable with that, or we can just start with the diagnostics.

[Anto79-ops]

hello,

Yes, I enabled the distance to entities but not the unfiltered, for 5 (and soon 6) beacons.

When I attempted to download the diaganostics....yikes...my system crashed! sorry, I was not able to obtain the logs. Having said that, I have temporarily removed the integration. I'd be happy to try it again later, once some of the performance bottlenecks have improved, at least for me system.

Starred and watching your repo.

[Lash-L]

Edit: I take back for now, improvements were not as significant as I hoped once I worked with real data instead of dummy data

Hi! I have some optimizations to calculate data that seem to cut runtime for that function about in half. I'm gonna build up a pretty robust data set from my local machine and test and then generate new data to test with as well, but I'll let you know