[WIP] Bluetooth Binding

[kaikreuzer]

This PR is a follow-up replacement of #3633. It includes heavy refactoring especially wrt discovery and lifecylcle. The namespace has also been changed from BLE to Bluetooth.

Overview

This provides an initial PR for a Bluetooth binding, with a BlueGiga USB adapter and a YeelightBlue device support implemented. It is still WIP.

Work to be completed

There is further testing, tidying and improving required.

Better connection management is needed to cope with disconnects. There is also some arbitration required in the BlueGiga handler to handle concurrency clashes between different thing handlers.

Bluetooth Binding overview

The Bluetooth binding is implemented to allow bundles to extend the main Bluetooth bundle in order to add new Bluetooth adapter as well as device support. This architecture means that such extension bundles must utilise the binding name bluetooth.

A base class structure is defined in the org.eclipse.smarthome.binding.bluetooth bundle. This includes the main classes required to implement Bluetooth -:

• BluetoothAdapter. This interface defines the main functionality required to be implemented by a Bluetooth adapter, including device discovery. Typically, this interface is implemented by a BridgeHandler and then registered as an OSGi service
• BluetoothDiscoveryParticipant. An interface to be implemented by services that can identify specific Bluetooth devices.
• BluetoothDevice. This implements a Bluetooth device. It manages the notifications of device notifications, Bluetooth service and characteristic management, and provides the main interface to communicate to a Bluetooth device.
• BluetoothService. Implements the Bluetooth service. A service holds a number of characteristics.
• BluetoothCharacteristic. Implements the Bluetooth characteristic. This is the basic component for communicating data to and from a Bluetooth device
• BluetoothDescriptor. Implements the Bluetooth descriptors for each characteristic.

Implementing a new Bluetooth Adapter bundle

Bluetooth adapters are modelled as a Bridge in ESH. The bridge handler provides the link with the Bluetooth hardware (eg a dongle, or system Bluetooth API). An adapter bundle needs to implement two main classes: the BridgeHandler which should implement BluetoothAdapter (any be registered as a service), and a ThingFactory, which is required to instantiate the handler.

The bridge handler must implement any functionality required to interface to the Bluetooth layer. It is responsible for managing the Bluetooth scanning, device discovery (ie the device interrogation to get the list of services and characteristics) and reading and writing of characteristics. The bridge needs to manage any interaction between the interface with any things it provides – this needs to account for any constraints that a interface may impose such that Things do not need to worry about any peculiarities imposed by a specific interface.

Classes such as BluetoothCharacteristic or BluetoothService may be extended to provide additional functionality to interface to a specific library if needed.

Implementing specific Bluetooth device support

A specific Bluetooth thing handler provides the functionality required to interact with a specific Bluetooth device. The new thing bundle needs to implement three main classes – a BluetoothDiscoveryParticipant, a ThingHandler and a ThingFactory, which is required to instantiate the handler.

Two fundamental communications methods can be employed in BLE – beacons, and connected mode. A Bluetooth thing handler can implement one or both of these communications. In practice, a connected mode Thing implementation would normally handle the beacons in order to provide as a minimum the RSSI data.

Thing Naming

To avoid naming conflicts with different Bluetooth bundles a strict naming policy for things and thing xml files is proposed. This should use the bundle name and the thing name, separated with an underscore - eg for the Yeelight binding Blue2 thing, the thing type is yeelight_blue2.

Connected Mode Implementation

The connected mode BluetoothThingHandler needs to handle the following functionality

• Extend the connected bluetooth thing handler. This holds the adapter through which all communication is done.
• Call the adapter.getDevice() method to get the BluetoothDevice class for the requested device. The getDevice() method will return a BluetoothDevice class even if the device is not currently known.
• Implement the BluetoothDeviceListener methods. These provide callbacks for various notifications regarding device updates – eg when the connection state of a device changes, when the device discovery is complete, when a read and write completes, and when beacon messages are received.
• The parent class calls the device.connect() method to connect to the device. Once the device is connected, the BluetoothDeviceListener.onConnectionStateChange() callback will be called.
• The parent class calls the device.discoverServices() method to discover all the BluetoothServices and BluetoothCharacteristics implemented by the device. Once this is complete, the BluetoothDeviceListener.onServicesDiscovered() callback will be called.
• Call the readCharacteristic or writeCharacteristic methods to interact with the device. The BluetoothDeviceListener.onCharacteristicReadComplete() and BluetoothDeviceListener.onCharacteristicWriteComplete() methods will be called on completion.
• Implement the BluetoothDeviceListener.onCharacteristicUpdate() method to process any read responses or unsolicited updates of a characteristic value.

Beacon Mode Implementation

The beacon mode thing handler needs to handle the following functionality:

• Extend the beacon bluetooth thing handler. This holds the adapter through which all communication is done.
• Call the adapter.getDevice() method to get the BluetoothDevice class for the requested device. The getDevice() method will return a BluetoothDevice class even if the device is not currently known.
• Implement the BluetoothDeviceListener.onScanRecordReceived() method to process the beacons. The notification will provide the current receive signal strength (RSSI), the raw beacon data, and various elements of generally useful beacon data is provided separately.

Generic Bluetooth Device Support

The core Bluetooth binding already includes generic "beacon" and "connected" Bluetooth thing types. All devices for which no discovery participant defines a specific thing type are added to the inbox as a beacon device. The corresponding handler implementation (BeaconBluetoothHandler) uses Beacon mode and merely defines a channel for RSSI for such devices.
The "connected" thing type can be used by manually defining a thing. The corresponding handler implementation (ConnectedBluetoothHandler) uses Connected mode and thus immediately connects to the device and reads its services. Common services are added as channels (t.b.d.).

[martinvw]

@cdjackson / @kaikreuzer I'm planning the look at https://github.com/openhab/openhab2-addons/pull/2491 to build in support of BLE there and see what I bump in to.

Is it the idea that the bridge lives in the Bluetooth binding and the thing in the Miflora binding?

[martinvw]

Note, travis reports the following error, I suppose missing xml's because they are auto-generated, I'll include it while working on it.

[ERROR] Failed to execute goal org.eclipse.tycho:tycho-packaging-plugin:1.0.0:package-plugin (default-package-plugin) on project org.eclipse.smarthome.binding.bluetooth.yeelightblue: /home/travis/build/eclipse/smarthome/extensions/binding/org.eclipse.smarthome.binding.bluetooth.yeelightblue/build.properties: bin.includes value(s) [OSGI-INF/] do not match any files. -> [Help 1]

[kaikreuzer]

@martinvw openhab/openhab2-addons#2491 would be cool :-)

travis reports the following error, I suppose missing xml's because they are auto-generated

I'll check why they are not generated during the Maven build. Your IDE should correctly generated those files.

[kaikreuzer]

Is it the idea that the bridge lives in the Bluetooth binding and the thing in the Miflora binding?

I would have hoped that the PR description already answers such questions :-)
TL;DR: Specific device support only defines things and implement their handlers. They currently have to reference ALL available Bluetooth adapters as potential bridges - this is something that still needs to be addressed in the framework as we obviously would not want such a dependency.

Just as some warning: This PR here is far from being merged, so if you look at openhab/openhab2-addons#2491 , please only do so in order to come up with improvements and suggestions for THIS PR here. Our efforts should be to collaborate on this and not already start too many specific device implementations on it. We should rather find some Bluetooth device that is a good candidate for initial support&testing.

[martinvw]

I would have hoped that the PR description already answers such questions :-)
TL;DR: Specific device support only defines things and implement their handlers. They currently have to reference ALL available Bluetooth adapters as potential bridges - this is something that still needs to be addressed in the framework as we obviously would not want such a dependency.

I did read and saw the same limitation so I wanted to validate my summary :-)

Just as some warning: This PR here is far from being merged, so if you look at openhab/openhab2-addons#2491 , please only do so in order to come up with improvements and suggestions for THIS PR here. Our efforts should be to collaborate on this and not already start too many specific device implementations on it.

Yes will do

We should rather find some Bluetooth device that is a good candidate for initial support&testing.

Do you mean device as in thing or device as in the bridge because I'm focussing now on a:

• BlueGiga BLED112-V1
• Miflora

or optionally a connected toothbrush ;-)

[kaikreuzer]

Do you mean device as in thing or device as in the bridge because I'm focussing now on a:

Devices as a thing like the Miflora.
Note that the PR already includes a bundle for the BlueGiga dongle as an "BluetoothAdapter" (currently it actually is the only one we have).

[maggu2810]

Should be moved to a separate PR 😉

[kaikreuzer]

Indeed, removed it.

[kaikreuzer]

@martinvw Build issues are fixed.

[kaikreuzer]

FTR, this is the architecture diagram that I had sketched out here and which has been followed in this PR:

The missing part is the BlueZ-over-DBUS bundle - I'll try to add such to the PR as well (while I am not yet certain whether porting @cdjackson's implementation or betting on TinyB is the better choice for the start...)

[kaikreuzer]

@cdjackson I have tried to port your BlueZ-over-DBUS code. The result is a skeleton that can be built on, but I learned that the code is really still in an early stage (especially wrt error handling) and that it also need heavy refactoring as the communication logic has to be moved out from the BlueZBluetoothGATT/Service/Characteristic/Descriptors to the Device and handler classes instead. I left the code in a state where everything compiles (and thus removed the communication logic from the old locations), but I haven't added an implementation to the BlueZBluetoothDevice and the BlueZBridgeHandler.

Do you think you will find any time to look into finishing this code? Or shall we rather go for the TinyB library as an alternate BlueZ integration?

[martinvw]

This hurts on of the static code check:

[WARNING] .binding.miflora\ESH-INF\thing\miflora.xml:[0]
Missing the supported bridge with id bluegiga

[kaikreuzer]

@martinvw Note that the yeelight binding is not o.esh.binding.yeelight, but o.esh.binding.bluetooth.yeelight - the same should be done for miflora, i.e. it would just be an extension bundle to the bluetooth binding - and as such have the "bluetooth" binding id.
But in any case you are right: "bluegiga" is not known in general, we will have to define an abstract bridge type that can be referenced by any extension, while having multiple implementations for it. This is not yet supported by the framework.

[martinvw]

Then we have one more challenge, the milfora binding strived to wrap both an mqtt connector and a bluetooth connector in a single binding (which feels good imho) in the end it would be great of that would be possible somehow...

[martinvw]

o.esh.binding.bluetooth.yeelight

But how can it be like that in openHAB...

[kaikreuzer]

Ah, that's an interesting question indeed...
Probably best to directly contribute it to ESH :-)

[martinvw]

@hakan42 wdyt I intended to start from your openHAB miflora binding

@kaikreuzer what about the other question:

Then we have one more challenge, the milfora binding strived to wrap both an mqtt connector and a bluetooth connector in a single binding (which feels good imho) in the end it would be great of that would be possible somehow...

[kaikreuzer]

We would need to be able to reference bridges from another binding in that case - another feature that isn't yet supported by the framework.
But we would anyhow first need the MQTT bridge infrastructure in place as well. The according PR looks a bit stalled since September :-(.

[hakan42]

@martinvw , feel free to take over the binding, I'd love if it started getting some proper love. Unfortunately, I am extremely swamped with work at the moment, so I won't be able to do much myself.

[martinvw]

@cdjackson this call pass null to the listener very often, can you agree with me that maybe passing characteristic to the listeners might make more sense, or do I overlook some detail here?

[cdjackson]

Yes - you are right - this should be characteristic.

[kaikreuzer]

Hm, but why does the if check for procedureCharacteristic? Should that be changed to characteristic as well? Or should it be an additional check?

[kaikreuzer]

I've made some improvements to this PR:

• adapted to 0.10.0-SNAPSHOT version
• updated license headers to EPLv2
• included BlueGiga sources to facilitate quick bug fixing
• replaced BlueZ-over-DBUS bundle by an initial version using TinyB (ARM-only for now) to give quicker and more robust results

I will next go and extend the BlueZ bundle to also support proper pairing and service access on devices, so that custom device support bundles can be built on top.

[kaikreuzer]

FTR: Currently, there still is an issue with the native TinyB libraries on a RaspPi when using Zulu JVM - with OracleJDK, it works as expected.

[kaikreuzer]

Another update: I have split the "generic" thing type into "beacon" and "connected", so that both modes are supported out of the box by the core bluetooth binding.
The idea is to implement support for common Bluetooth characteristics (such as battery status etc.) directly for the connected thing type. I've updated the description above accordingly.

[kaikreuzer]

Battery level channel support has been added for "connected" devices and I have pushed quite some further improvements for the stability.

[kaikreuzer]

Pushed another update with improvements.
With 93d2da4, I also added another specific device support bundle (Blukii) which demonstrates how to deal with custom characteristics. I chose this one as I have a device at hand (which isn't the case for the Yeelight Blue bulb).

[vkolotov]

The battery service unfortunately is no longer accessible through the TinyB library with the most recent Bluez (v5.48): intel-iot-devkit/tinyb#131

[kaikreuzer]

@vkolotov Many thanks for the heads-up, that sounds like a very unfortunate bug/feature 😟

[kaikreuzer]

I've now updated this PR to have just a diff commit on top of #4997.

[mronus]

@kaikreuzer As I see you have closed this PR. Actually we've been looking forward to use official bluetooth binding for a long time :) I would like to know about your roadmap about having a complete official bluetooth binding support in OH.

[kaikreuzer]

@mronus Please note that there is already an official Bluetooth binding, which has been merged with #4997.
This PR here only contained BlueGiga stick support and a few draft specific device support implementations - those will rather be done as individual PRs, but I left the branch available as a reference.