Initial Commit of the Hue Emulator Binding

[digitaldan]

Signed-off-by: Dan Cunningham [email protected]

[digitaldan]

This Binding emulates a Hue Bridge, allowing the echo to directly control it. This has two big benefits over using a alexa hosted app. One is that it is super fast, control is done over the local network and not from the cloud. Two is that there is no app word, you just say "Alexa Turn lights ON" or "Alexa Set Thermostat to 70".

This should also allow other Hue compatible devices (like the Logitech Harmony) to see openhab, I plan to make those work as well, but one step at a time.

To use the binding drop the jar in your addons folder. Items are exposed to the echo using tags, the format is "he:Voice Name" so for example
Switch TestSwitch [ "he:Kicthen Lights" ]
Number TestNumber [ "he:Thermostat" ]

After adding tags, say "Alexa search for connected devices" and see what happens.

Right now this supports ON/OFF and Decimal/Percent commands. Couple known issues, first is that sometimes I have to search more than once to find devices. Also I have no idea what happens if you are using the Hue binding (your feedback is appreciated), I would imagine I need to filter local requests or something. Also there is no security here, I plan to add a include button/channel to mimic how the actual Hue bridge does this. Finally, I need to clean up the code, but I wanted to get feedback asap, so I'm exposing the dirty laundry so to speak.

[digitaldan]

@kaikreuzer couple things....

As you pointed out it's not really a binding, I was planning to have a thing for this however, a switch channel that one could turn on to allow an Echo (or something else) to pair while on, this simulates an actual hue bridge and would help with security.

Hue emulator was the best name I could come up with, I'm open to suggestions ;-)

Originally I had this all done with JAXRS which was very nice and clean. Unfortunately this only worked under the path "/rest" which will not work with the Hue API. Since "/rest" is set as the "root" property I could not seem to get around this (and I tried everything!). One option, which might be the best, is to start another jetty listener on another port, this would also mean I would not pollute the main OH paths (it needs to be on /api/*). Also this might help when we add username and password support, as the API needs to be open from that. I'm not sure how to do this in our OSGI environmen so I would need some help. Right now I'm using a HttpServlet and Jackson.

If I think of anything else I will post back, thanks!

[kaikreuzer]

Thanks for the PR, @digitaldan!

I didn't yet test it or looked at the code in detail, but let me quickly respond to some of your topics:

I think this functionality is pretty similar to what @beowulfe did for HomeKit. There, we have decided to name it org.openhab.io.homekit, so my suggestion here would be org.openhab.io.hueemulation.

I was planning to have a thing for this however, a switch channel that one could turn on

This is rather a "setup&configuration" issue than a "daily operation" - so I think we can make this available by specifying a config description. This is possible for any kind of service, it does not have to be a binding.

Since "/rest" is set as the "root" property I could not seem to get around this (and I tried everything!).

I also would not know any way to use JAX-RS, but add the resources on a different URI :-/ So I guess your approach with servlets is the best we can do.

Right now I'm using a HttpServlet and Jackson.

Note that the preferred library for JSON processing is GSON, not Jackson. GSON is always available, Jackson is only bundled, if needed (e.g. for 1.x compatibility). If it is easy for you to switch to GSON, please do so. If not, please do not bother to much and we can add Jackson as a requirement for this feature, so that it is installed together with this bundle.

Items are exposed to the echo using tags

Tags are actually not meant to be used for "specific" extensions - your he:xxx looks pretty much like a binding configuration in openHAB 1.x. The idea is that tags provide a (generally useful) semantic information. They are supposed to reference well-defined ontologies. These ontologies are not yet defined, but homekit seemed to be one to me that will probably have some weight in the future.
So my suggestion would be to actually use the very same tags in the hueemulation as well - they seem to apply pretty much the same way, so all a user would have to do is to tag his items and he could instantly use Amazon Echo and Siri in exactly the same way; wouldn't that be cool? The name to expose for the item would be the default label of the item, that's how the homekit integration does it as well.

[digitaldan]

I think this functionality is pretty similar to what @beowulfe did for HomeKit. There, we have decided to name it org.openhab.io.homekit, so my suggestion here would be org.openhab.io.hueemulation.

Yep, Sounds good.

This is rather a "setup&configuration" issue than a "daily operation" - so I think we can make this available by specifying a config description. This is possible for any kind of service, it does not have to be a binding.

Agreed

I also would not know any way to use JAX-RS, but add the resources on a different URI :-/ So I guess your approach with servlets is the best we can do.

I was hoping you wouldn't say that ;-) I'll try and clean it up as much as I can.

Note that the preferred library for JSON processing is GSON, not Jackson. GSON is always available, Jackson is only bundled, if needed (e.g. for 1.x compatibility). If it is easy for you to switch to GSON, please do so. If not, please do not bother to much and we can add Jackson as a requirement for this feature, so that it is installed together with this bundle.

This shouldn't be a problem, I used very simple pojos, I would imagine it would be easy to swap.

Tags are actually not meant to be used for "specific" extensions - your he:xxx looks pretty much like a binding configuration in openHAB 1.x. The idea is that tags provide a (generally useful) semantic information.

I actually based the tag format off the homekit one, there they were using tag formats like "homekit:DimmableLightbulb", it also occurred to me that this felt more like a configuration string then a label.

So my suggestion would be to actually use the very same tags in the hueemulation as well - they seem to apply pretty much the same way, so all a user would have to do is to tag his items and he could instantly use Amazon Echo and Siri in exactly the same way; wouldn't that be cool? The name to expose for the item would be the default label of the item, that's how the homekit integration does it as well.

My thinking originally was that the label name might be different than what you say. Either phonetically (to help get around so so recognition), or you want something different then the label for other reasons. One example is that I have my echo in the kitchen, so while I have a Group called Kitchen_lights with a label of "Kitchen Lights", I want the voice label to be "lights" so I could say "Turn lights off" , making my label "Lights" would not make sense in other scenarios. I was also thinking about allowing multiple tags, so in this case you could support both "lights" and "Kitchen Lights" on the same device.

I guess this could be accomplished by having redundant items/groups to have different labels? I'll go ahead and make the change and see how this works out in my system.

In terms of the tag, would you suggest using something like "hedevice", or "voicedevice" ? I don't think I need to specify the hue device type (like in the homekit tag).

[kgoderis]

@digitaldan and @kaikreuzer : the work around setting a different root folder for the http part consists in setting the same httpContext on all servlets in openhab and use a filter chain to dispatch the http request to the correct servlet. Now each servlet uses a unique http context so you can not pass on http requests.

In the Caldav stuff I have written for ESH I have inserted a solution to this problem.....

[kaikreuzer]

I guess this could be accomplished by having redundant items/groups to have different labels?

This could be a valid workaround for the time being, yes. In general, the label on the item is anyhow only considered to be a fallback/default. As sitemaps can override the default label (done through the ItemUIProvider), we could possibly introduce something similar for voice. This might actually be something to introduce in the context of eclipse-archived/smarthome#1021 as we also need to feed in some vocabulary to the STT engines. The list of "aliases" for your item is probably a subset of such a vocabulary.

In terms of the tag, would you suggest using something like "hedevice", or "voicedevice" ?

No, my suggestion is to use the homekit tags as well, not inventing any new ones. These are

homekit:Lightbulb
homekit:DimmableLightbulb
homekit:Switch

and possibly also

homekit:heatingThreshold
homekit:coolingThreshold

If we define an ESH ontology one day, we can simply use different tags then.

[digitaldan]

No, my suggestion is to use the homekit tags as well, not inventing any new ones.

I see the convenience of this, but I don't like using "homekit" which is a very Apple specific (and proprietary/copyrighted?) name. I think this is confusing for users.

Does the HomeKit stuff need to specify device type information in the tag? Since the Hue API is so simple, I didn't think I need to do this, ON/OFF and Decimal/Percent (and I guess color when I get around to it) can all be automatically inferred by the items accepted types.

Since we are early with both bindings, can we agree on something more generic? I don't mean to be a pain about it ; -)

[andylintner]

I agree, using the HomeKit name here would be confusing. The HomeKit protocol is strongly typed, and exposes different interfaces depending on the device type. Siri, and other HomeKit apps, use that device information, so it's important to have a way to tag them.

[digitaldan]

@kgoderis when you say you inserted a solution, what do you mean? I ended up registering a servlet to the HttpService which seems to work, unfortunately registering a jaxrs service on this same object forced it to live under the root path of "/rest". I think you are doing something similar?

[kgoderis]

@digitaldan i made a post on the ESH forum but the more general solution is to use servlet filtering if you have path/name clashes

[kaikreuzer]

I see the convenience of this, but I don't like using "homekit" which is a very Apple specific (and proprietary/copyrighted?) name. I think this is confusing for users.

Let me repeat what I said above: "The idea is that tags provide a (generally useful) semantic information. They are supposed to reference well-defined ontologies. These ontologies are not yet defined, but homekit seemed to be one to me that will probably have some weight in the future. [...] If we define an ESH ontology one day, we can simply use different tags then."

The "semantic tagging" is a concept that is discussed for ESH and it will be defined in the future. As there are many people in the industry trying to define suitable ontologies for smart homes (HomeKit, Weave, AllJoyn, OIC, oneM2M, IPSO, SAREF, etc.), I am reluctant to invent one for ESH as well and always hoped that something will become a de-facto standard in the industry. Possibly we end up copying stuff from other ontologies together into an "esh:" ontology, just to have a neutral naming and not to confuse users or have trademark issues. But in any case this means that we should use the same tags for all "io-integrations". And since we have the homekit ones in place and they make sense for the use cases we have right now, I think we should use this for a start. As these are merely constants in the code, it should be pretty easy to change them any time in the future to something else.
@beowulfe: This then also has to apply for the homekit integration: If we define an esh-ontology instead of the current tags, the homekit-bundle should make use of it as well - so it will be on us to come up with the right definitions that make sense in general, but do not break anything important on any specific integration.

[andylintner]

That makes sense. There certainly are a lot of ontologies out there right now - I'd include UPNP HA as another interesting example. HomeKit is as good as any of the others. It would be interesting to think about an ESH ontology - but you're probably right to sit back and see how things shake out.

[kaikreuzer]

but you're probably right to sit back and see how things shake out.

I am sitting back since 9 months already and not much is shaking out yet... That's why I feel that we will end up with our own ontology after all - as soon as work on that starts at ESH, I will invite you in the discussion to have the HomeKit view covered nicely :-)

[digitaldan]

@kaikreuzer , ok I can support the homekit tags, how do you propose I coordinate? Should i look for any items that begin with "homekit:" or do I need to be in sync with the homekit binding and whatever tags they have now as well as ones they add/modify or delete? Certain tags are easier to support like

homekit:Lightbulb
homekit:DimmableLightbulb
homekit:Switch

The temperature tags will be a bit harder, also in my setup I expose other numeric items such as volume via tags, I guess users will need to tag them as a "homekit:DimmableLightbulb" ? Or should we consider adding something more generic like "homekit:numeric" and "homekit:switch" ?

[kaikreuzer]

Should i look for any items that begin with "homekit:"

You could directly query for items having the tags I listed in https://github.com/openhab/openhab2-addons/pull/654#issuecomment-183856348. No need to sync with any changes in the HomeKit integration - simply define some constants for these tags within your code.

The temperature tags will be a bit harder

Why? Can't you simply expose these two tags as a dimmable hue light that expects numeric values?

I expose other numeric items such as volume via tags, I guess users will need to tag them as a "homekit:DimmableLightbulb" ?

Yes, this would be the workaround that people have to use. Actually, this is what the Hue emulation is doing anyhow - it claims that your volume is a dimmable hue light to the outside world :-)

[smhgit]

The approach here is to use tags to expose items to the echo. Have you consider exposing items to echo by group assignment?
Why not to define a group which any items belongs to this group will be exposed to the echo device? The group can be pre-defined group or via configuration file.

[kaikreuzer]

How would you identify the group item? By a tag? Or by its name? I think this isn't really flexible and would feel more like a workaround.
The nice things about tags is that bindings can define tags themselves - so if you add a new light bulb to your openHAB instance in future, this will be automatically exposed to the Echo without needing any further configuration.

[smhgit]

In the echo case, the echo is not binding specific, which means that every item from any binding can be attached to echo device. Do you think that in this case adding the tag by the binding is still something feasible? Why any binding should know about echo device?
Also, what will be the case if you have two (or more :-)) echo devices in the same home. How do you select which item goes to which device (assuming you want to handle this case)?
One option can be that the echo service / binding will add the group when it is created. The group can be tagged, but this time only the group is tagged and not every item (and maybe also it doesn't needs to be tagged as the service knows which group he created). Using groups also simplifies handling more than one echo.

[kaikreuzer]

Why any binding should know about echo device?

They don't have to - that's why I said in https://github.com/openhab/openhab2-addons/pull/654#issuecomment-184356643 that "tags provide a (generally useful) semantic information", i.e. they are neither tight to certain bindings nor to a specific integration (HomeKit/Echo...).

handling more than one echo.

This is anyhow not feasible, because what the emulator does is simply exposing the items to the outside world. It is not supporting "Echo devices", that's why it isn't a binding.

[digitaldan]

I have changed this to an "io" plugin and renamed it to hueemulation. It will also use "homekit:" style tags as well. I have the basic security framework in place and would like to be able to have users turn the setup/include functionality on from something like the paper ui. I used the basicui bundle as an example of a service configuration ( I added props to my service comp xml file, add config.xml to ESH folder) but I could not get this bundle to show up under configuration-> services in the paper ui. @kaikreuzer do you have any advice here?

[smhgit]

@digitaldan

I think you have an issue here (probably you need ||):

if (user.username == null | user.username.length() == 0) {
user.username = UUID.randomUUID().toString();
}

[kaikreuzer]

@kaikreuzer do you have any advice here?

Is this code already in this PR? I cannot see any ESH-INF folder in it and I only see that you even removed ESH-INF from the build.properties in the latest commit...

[digitaldan]

I think you have an issue here (probably you need ||):

Is this code already in this PR? I

No ;-) I have the security scaffolding in place, but have not tried it out yet. I also need to run this with the Hue binding to make sure nothing bad happens. I'll check in my attempt to make this a configuration service, I'm guessing I missing some OSGI wiring thing. Once I get these out of the way I should be ready for a review and to get some more users to try it out.

[kaikreuzer]

I you commit your code to the branch, I can help analyzing what might be missing. Without it, I do not stand a chance :-)

[digitaldan]

I you commit your code to the branch, I can help analyzing what might be missing. Without it, I do not stand a chance :-)

I just don't think you're trying hard enough ;-)

So I have added properties to my component xml property and added a config.xml to the ESH folder but I'm not sure how a) how these get wired up and b) how I get this config variable (i'm assuming it comes through my activate(Map<String,Object> config) method, or the modify method)

Thanks for the help.

[kaikreuzer]

Ok, I am trying harder then! I have created digitaldan#1, which makes it appear in the Paper UI - the problem was that the service needs to register as some interface, otherwise it is simply not listed in the service registry of OSGi and the Paper UI cannot find it.

Btw, the emulation seems very successful, openHAB finds itself now 😆

Not sure if and how we should prevent this - we would possibly have to tweak the Hue binding to ignore such UPnP devices (although it is trained to specifically find hue bridges ;-).

Another observation: jUPnP doesn't seem to fully like the responses from the emulation, I see this in my log a lot:

2016-02-17 14:23:01.059 [DEBUG] [h.internal.HueEmulationServlet:136  ] - GET /api/discovery.xml
Feb 17, 2016 2:23:01 PM org.jupnp.binding.xml.UDA10DeviceDescriptorBinderImpl hydrateServiceList
WARNING: UPnP specification violation, skipping invalid service declaration. Can't parse service type string (namespace/type/version): (null)

Probably something not fully UPNP compliant in the discovery.xml?

[digitaldan]

Ok, I am trying harder then! I have created digitaldan#1, which makes it appear in the Paper UI - the problem was that the service needs to register as some interface, otherwise it is simply not listed in the service registry of OSGi and the Paper UI cannot find it.

Awesome! Also thanks for fixing my spelling issues, you would think English is not my primary language 😆

Btw, the emulation seems very successful, openHAB finds itself now

I was pleased when I saw that as well, I even tried to add it, but unfortunately that didn't work. I think the Hue binding is ignoring the port in the UPNP BaseURL element and just defaulting to 80. I was thinking about submitting a PR now that I have a (fake) Hue bridge to test against 😆

Probably something not fully UPNP compliant in the discovery.xml?

Yes, I was actually looking at something similar last night and noticed I had left some elements empty, I suspect this is something else, but I'll definitely look at it. Thanks for the pointer.

[digitaldan]

Hey @kaikreuzer it's now showing up in the paper UI but when I click on it I get a 404 toaster message, I can see we are making a rest call to "http://localhost:8080/rest/config-descriptions/io:hueemulation" which is returning a 404. I also don't see anything listed under "http://localhost:8080/rest/config-descriptions", so I must be missing something else, I'm just not sure what.

[digitaldan]

@kaikreuzer Forget my last comment! As soon as I was ready to give up it hit me that my ESH folder was not being included in build.properties, problem solved ;-)

[kaikreuzer]

Thanks, @digitaldan, I am through with my review! Should be only small things to fix.
Could you please also add a feature for it in https://github.com/openhab/openhab2-addons/blob/master/features/openhab-addons/src/main/feature/feature.xml?

[digitaldan]

@kaikreuzer thanks for the review and feedback. I have gone ahead and made your suggested changes, added a feature for it and squashed everything.

[kaikreuzer]

As per my previous comment, this has to be in a finally block in case an exception is thrown before.

[kaikreuzer]

I have gone ahead and made your suggested changes,

Not all as it seems - so issue back to you :-)

[digitaldan]

Not all as it seems - so issue back to you :-)

Yep i missed a few, that's what happens when you rush it ;-) I'll get those fixed soon.

[kaikreuzer]

@digitaldan Any update?

[digitaldan]

@kaikreuzer yes, I was testing this last night on a real OH2 system to validate, I'll have it finalized in the next few hours!

[digitaldan]

@kaikreuzer just committed my changes.

[kaikreuzer]

@digitaldan One more update, please! (https://github.com/openhab/openhab2-addons/pull/654/files#r53811793)

[digitaldan]

@kaikreuzer , so sorry about that. I'm juggling many open projects and not doing a great job on focusing. Just pushed again.

[kaikreuzer]

Thanks! I think now it all looks good to me :-)