Feature: support FastOn / FastOff

[jtmoderate876]

The FastOn / FastOff feature (when you "double click" a light on or off) is handy too, for example, turn of a whole room rather than just the one light you are at.

Burnt and Daniel Pfrommer (the guys that wrote InsteonTerminal) also reverse engineered these things for Openhab.

Someone "code-savvy" might look at how they are doing it at:
https://github.com/openhab/openhab1-addons/tree/master/bundles/binding/org.openhab.binding.insteonplm

There they also support battery level and light level (for things like the motion sensors).

[TD22057]

It's trivial to do. I didn't bother because I couldn't figure out what the difference between "instant" and "fast" on was. So I put in support for instant on but not fast. It's easy to add that as a command to send to the devices (just a different cmd1 code). I think the only real work here is add separate MQTT messages and templates for a fast on/off message vs the regular on/off commands. It would be nice to just have that be a flag in the regular on/off template but I expect most home automation systems will need a separate topic and message to easily differentiate between them.

[TD22057]

Some notes: The switches currently respond to fast on (0x12) and fast off (0x14) as regular on and off commands. That's fine since the load does turn on and off so updating the switch state is required. The questions is what should happen on the MQTT and HA side?

One option: a separate MQTT topic/payload for reporting fast on and off since a user might want to handle those as separate commands? The problem with this is that now there are two switches in HA which basically have the same state. That might not be a huge deal since Insteon-MQTT will keep both updated. But I'm not sure if the HA automation system will allow sending commands if the switch is already in that state which might be an issue.

[Juggler00]

My preference would be a separate topic. This gives the most flexibility. If they are to refer to the same device, this can always be configured with triggers in HA.

[crxporter]

It would be great to see HA with the ability to react to "on", "off", "fast on", and "fast off" at any time, regardless of the previous switch state.

This would allow all of our switches and dimmers to essentially have 4 functions to start scenes and automations. obviously I don't know if this is a possibility, I just saw this conversation and decided to share my opinion.

[TD22057]

The issue is what else happens? I can certainly send a fast on and off message. But the device (switch/dimmer) also turns the load on and off (or turns the light on/off on the switch for no load switches) and triggers the scene. So a fast on is also an "regular on" in the sense that the load is on. So by default, the regular switch state message also needs to be sent. Maybe that could be turned off as an option but I'll have to think about that. It would also mean adding per device mqtt inputs perhaps or at least per device options to control that behavior.

[jtmoderate876]

Yes, as you point out the FastOn / FastOff also have the effect of turning off the local light so that topic needs to update too and this is a "two-topic effect" (like the motion sensor battery and light levels).

Back when Houselinc was a viable software, I used FastOn/FastOff in a couple of scenarios.

One was to turn on and off "whole rooms".

The other was to bypass "auto-off" timers.
So if I turn on the Range Hood Fan it turns off automatically after an hour.
If I double-click it on, which people don't generally do, then the fan stays on and bypasses the auto-off timer.
Same for bathroom fans and low-use room lights - timer off unless deliberately instructed otherwise.

You have breathed life into my Insteon investment - which has been painful and was languishing with software that was either one or more of outdated, complicated, unreliable, etc.
Thank you.

[TD22057]

Thanks - I'm just glad others found it useful.

The question I have is how to really report the fast on/off stuff - especially with home assistant (HA). Let's say there are two messages STATE and FAST_STATE. A normal click sends STATE=ON and STATE=OFF and a double clicks sends FAST_STATE=ON and OFF. The basic HA set up is to define a switch SW1 that sends ON/OFF commands to the switch and updates it's state when STATE=ON/OFF messages arrive.

So what happens if you double click? If I just send FAST_STATE=ON, then in HA you can link that to a different (virtual) switch (SW2) and do some kinds of automation on it. But the regular switch (SW1) is now showing the wrong state. So if we change it to send a STATE and FAST_STATE message, now what happens?

Let's take your example w/ the fan timer. In HA, this could be configured as something like this: If STATE=ON arrives, start timer and shut off after 30 minutes. If FAST_STATE=ON arrives, turn SW1 on w/ no timer. But when you turn SW1=ON as part of the fast state automation, that sends a message to Insteon to turn it on which then reports back that it happened. So now a STATE=ON message arrives back and starts the timer which isn't what you want.

Another approach is to send two messages. Start by sending the regular STATE=ON (which updates SW1 and starts the timer) and then send a FAST_STATE=ON. So in that case, the fast state automation would be to cancel the timer that state started. This would rely on the message order being preserved in MQTT.

[jtmoderate876]

If you double-click most devices On it turns on immediately locally but it doesn't send a regular On command on to the "network" it only sends out a Command FastOn.

So right now, as things are you can double-tap FastOn / FastOff devices and they turn on and off but insteon-mqtt doesn't see that (until they are refreshed).

EDIT
My understanding of the way @pfrommerd and @berndpfrommer handled "on" versus "faston" in on openhab isn't totally accurate so take the following with a grain of salt.
EDIT
My flawed understanding of the way OpenHAB handles (and probably HouseLinc) is that device state REQUESTS from PLM are received as device state updates - in MQTT this might be thought of as the topic: #/state. Queries for state would update that same #/state topic.

However,
Device-driven actions such as On/Off/FastOn/FastOff messages are received as COMMANDS (from the device, received by PLM) and reflect topics of, say #/command = on/off/faston/fastoff and in software the #/state would be updated but not interpreted as a change in action.

I suppose in our case this state-matching could be performed in EITHER in the driver (your insteon-mqtt in this case) or the HA software (home assistant in your case, Node-RED in mine) - I think Bernd Pfrommer did it in the driver for OpenHAB.

I'll walk through the approach as if insteon-mqtt were to handle everything...

So in our previous fan timer scenario where we want double-tap to override timer to leave the fan on:
1 the switch is physically double-tapped-on and sends insteon FastOn
2 that gets interpreted by the insteon-mqtt as both a #/command = "faston" and a #/state = "on"
3 the HA software takes action off the #/command="faston" and does not turn on enable the timer that would have been enabled had the command been a single-tap #/command="on".

In the case where "we" from the HA side send an on/off/faston/fastoff command, insteon-mqtt would just update the status either right away or after getting an ack from the device (if acks exist).

I would think you are dealing with a similar thing with scenes where scenes are commands - that set one to many device/state (s).

In all of this it becomes an exercise keeping the device/state in sync with the device.

[jtmoderate876]

Here is that code from @pfrommerd and @berndpfrommer
but it is not sinking in for me yet:
https://github.com/openhab/openhab1-addons/blob/master/bundles/binding/org.openhab.binding.insteonplm/src/main/java/org/openhab/binding/insteonplm/internal/device/CommandHandler.java

Search "faston" and compare to "onoff" just above that.

[TD22057]

Thanks - the command codes are already in insteon-mqtt. It's more a question of what to do when those messages arrive than how to implement it.

[krkeegan]

@jtmoderate876 agree, I have done many of these tricks too.

These types of design issues are irritating, sadly I think there are many more to come.

I would argue that the fast commands are a subset of On and Off. So in your example of having two virtual switches a

Fast_On should result in SW1=SW2=ON
Fast_Off should result in SW1=SW2=OFF

However, this breaks down for the regular commands, as most systems only see changes in state and sending the following sequence FAST_OFF->ON->FAST_OFF would result in SW2 remaining OFF the entire time, meaning your second FAST_OFF wouldn't trigger.

This is ugly, but maybe we need yet a third virtual switch? SW1 is regular ON/OFF. SW2 is FAST_ON and SW3 is FAST_OFF? The resulting states would look like:

Fast_On should result in SW1=SW2=ON SW3=OFF
Fast_Off should result in SW1=SW2=OFF SW3=ON
On should result in SW1=ON SW2=SW3=OFF
Off should result in SW1=SW2=SW3=OFF

That is really ugly. A flag would be much less ugly on our end, but at least in the case of HA, I think this would require a bit of work for the user. I think they could use the sensor type to extract the relevant Fast_ states? https://www.home-assistant.io/components/sensor.mqtt/

[jtmoderate876]

@krkeegan - you found the best solution to balance trade offs between Insteon and HA differences. 3 switches: on/off, faston, and fastoff, would provide all the switch functionality I can think of.

It probably require Insteon-MQTT driver programming gymnastics but if completed, would be great. I suppose we need to remember dimmers in the on/off part.

Thanks for keeping the thinking moving here.

[TD22057]

Does this sound like what you want?

Human physically clicks a switch either with regular on/off or fast on/off. In this case, we have a separate "fast sensor" topic which would also be put into HA. The switch will always send a fast on/off sensor update first, then a regular on/off update whenever the state changes. As long as the fast message is sent first, automations can be written to respond to the regular on/off change and use the fast on/off sensor to change their behavior.

It would be nice if we could add custom attributes to the MQTT light or JSON light in HA but I don't think that's possible. Then the sensor wouldn't be needed - the fast boolean would just be an additional flag in the on/off message. For non-HA users, the fast flag can be an option in the insteon/mqtt templates and only one message would be required.

I think the other way around scenario: sending a fast on/off command to the switch is already handled via the input MQTT commands.

[jtmoderate876]

If master bedroom main light is on, you should still be able to double-tap off that switch to turn off the other master bedroom lights.

If your scenario still propagates Insteon to MQTT the CMD off for fast , even if sender is already off the next that works for my MQTT node-red perfectly...

But I think @krkeegan is saying HA won’t process more offs if it perceived the device is already off so HASS may need his approach - but you guys know that part way more than I,

[TD22057]

I think I see what you mean. I can make insteon/mqtt send updates even if the state isn't different w/o a big issue. In HA, the "fast" sensor can be created with force_updates=true which will also work to signal changes even if the state doesn't change. I'll try and prototype something up this weekend.

[NickWaterton]

Any progress on fast_on/off?
I currently use OpenHab, but the Insteon support is lagging (I think Bernd has moved on), and it’s all written in impenetrable java anyway (and I program in Python)
This project looks like it will breath new life into my Insteon setup. My plan is to integrate it into OpenHab (I have a 2012 hub, and PLM modem run in parallel). I have done similar things with other mqtt-bridges, and they work well. I have a working Harmony-mqtt bridge, a sonoff-mqtt bridge (and I actually wrote the Rooma-mqtt bridge used in HA).
The sticking point right now is fast_on/off support. I use this extensively for various things. For example the mini remotes support fast_on/off, so you can have single click does one thing, and double click does another. In OpenHab these are two separate items, so it’s easy to detect the difference.
I implemented fast_on/off and dimming for mini remotes in the insteonplm binding. I also have one of the alarm modules, and I implemented the code to support that also.
So if I can help out to get fast_on/off working, or implement siren support, dimming on mini remotes etc. I’m more than willing to lend a hand.
Can you let me know what the current state of implementation is?

[NickWaterton]

So I looked through how this works (the jinja templates is genius - I was struggling with this in my sonoff-bridge code - so I'll borrow it for my sonoff project and credit you guys).

I've implemented faston and fastoff for Switches, Dimmers and Remotes. This is just a state publish (new variable 'faston" = 0/1, and new topic 'fastonstate' (defined in the template) which is only published if the physical device is toggled 'faston/off'. That way you can decide whether to receive the faston state in the regular state topic, and/or in a separate topic. The fastonstate topic receives the state ON or OFF (no level) as by definition it's an ON/OFF binary option.

There didn't seem to be much point in sending faston/off to the device as it's the same as instant on/off, and you can't send to remotes anyway, so there is no send for faston/off implemented. I could implement a scene send for faston/off, but again, not sure if it's needed.

I've put together a basic template for OpenHab as well (OpenHab doesn't do well sending json via mqtt).

Once I've tested it out thoroughly I can submit a pull request if there is any interest.

[krkeegan]

I think it is certainly worth making a PR so we can see the code you have written. Your solution sounds reasonable. We can also add a fast_on as a variable in the MQTT template to enable the possibility of a JSON message in the /state topic containing the fast_on details.

[NickWaterton]

I already added the fast_on variable in the templates, and added the KeypadLincs as well. I’m adding the Siren module right now as a new device, which is turning out to be quite complex.

I have it working, but I’m still figuring out how to set the various options on the module (it’s not just on/off (which I have working), there is armed home, armed away, disarmed, delay, alarm type, alarm countdown and more. I have all the methods written, just working out how to link it to the command syntax. The Siren module actually reports a fast_on/off event while its sounding every second (not a regular on/off at all). Should be able to post it towards the end of this week with any luck.

[TD22057]

Thanks. Please make 2 separate PR's. For the fast on, what are you doing about the normal state topic? I never had time to test HA or other services but I'm pretty sure we need to send the state update topic after the fast on topic goes out (assuming both are defined). Otherwise HA won't know that the light has change state.

[NickWaterton]

The normal state topic gets published as usual, just after the fast_on message. You can also just include the fast_on value in the normal state message as a json string eg { “state”: “on”, “fast_on”: 1 }. I’m using 1 and 0 to indicate if it was a fast_on event.
I fixed manual dimming up and down on remotes while I was adding fast_on as well. I’ll see what I can do about two PR’s but new devices are integrated in in many places, so it’s hard to separate it out once it’s in.

[TD22057]

I appreciate what you're doing, but I don't want to try and merge new devices and fast on/off changes at the same time. I'd really appreciate it if you can implement the fast on/off for the existing devices, get that working, and file a PR for that change. Then you can branch from that to add a new device and file a separate PR for that when it's done. Thanks.

[jtmoderate876]

Nick & Ted, thanks for breathing life back into fast on/off. I have loved Insteon-mqtt reliability and moved entirely to it but miss “double-click” functions.

[TD22057]

FYI in reviewing the PR I came to a some conclusions. There are two parts to this - the first and easiest is reporting fast on/off commands. That just adds a new output template parameter that you can use. The second is an input to send a fast on/off command. The issue is that there are really 3-4 modes: normal, fast, instant (on only), manual and obviously they are mutually exclusive. So having separate flags for them doesn't make a lot of sense on the input. I'm going to take the code in the PR (just the fast on/off portion) and add in a new SwitchMode class which will store an manage all the possible on/off codes. This means that rather than having a bunch of flags in the classes like set_state( on, is_fast, is_instant, is_manual ), it can be something more like set_state( on, mode ). This eliminates a lot of error checking for mismatching flags and gives all the on/off classes a common place to handle these mode flags. I'm hoping to have something in dev for you to test by early next week.

[TD22057]

OK - I took the code from PR #103, made a lot of modifications (mostly related to creating a Mode enumeration that has values NORMAL, FAST, INSTANT, and have worked that in. My basic testing shows this to working now. But - I have a question for people:

Do we really need a separate mode state topic to be published?

I've added new template variables to the state payloads of "mode" = "normal"/"fast"/"instant" as well as "fast" = 0/1, "instant" = 0/1. I think it's better to report the mode in the message with the state change - that way there are no ordering issues or anything else. This works fine in Home Assistant - here's an example automation that triggers and turns off a different light when a dimmer is double-clicked off (even if it's already off):

- id: test_fast
  trigger:
    platform: mqtt
    topic: "insteon/4e.d1.d4/state"
  condition:
    condition: template
    value_template: '{{ trigger.payload_json.state == "OFF" and trigger.payload_json.fast == 1 }}'
  action:
    - service: light.turn_off
      entity_id: light.den

So - is there any reason to have separate mode flags when the same state topic for the device can be subscribed to get the mode state out?

[crxporter]

This is one of the big ones keeping me from jumping over to Insteon-mqtt (fast on/off).

Your setup looks good for me with one condition- is a message sent regardless of the previous state of the light? I have several switches that we click or double click off when the light was already off. I would hope these messages will be sent in that case?

[TD22057]

Yes - it still sends the fast message. It works that way now - basically pushing any button even if it's already in that state will send the message.

[TD22057]

FYI - if you want to test this out, I just pushed the update to the faston branch on github. I still need to write docs and some test cases before it's really finished and get be merged into dev.

[jtmoderate876]

Thank you guys for implementing this. Trying now... Sincerely, Jeff Tapia

[jtmoderate876]

I mostly set it and forgot it on insteon-mqtt - it has "just worked" so I am a bit rusty on configuration, but...

I think I need something in my config.yaml file (or somewhere else) but I'm not getting any faston/off instant on/off in my mqtt subscriptions.

I know you mentioned documentation is still needed but I got stuck.

I will try some more tomorrow.

[TD22057]

Check the default config in the branch. You'll need to add something like this to your state templates for the devices you want to report: "fast" : {{fast}} or "mode" : "{{mode}}" will yield "fast" : 0/1 and "mode" : "normal/fast"

[TD22057]

This functionality has been merged into the dev branch in commit 80a8b6b. See the sample config.yaml file or the detailed MQTT docs for details.

[jtmoderate876]

I can't get my head around faston implementation.

I'm not great with git yet but I'm pretty sure...
I used the faston branch a few days ago.
Upgrade to the dev brach yesterday.

Modified Config.yaml with entries like the following:
dimmer:
# Output state change topic and payload. This message is sent
# whenever the device state changes for any reason. Available
# variables for templating are:
# address = 'aa.bb.cc'
# name = 'device name'
# on = 0/1
# on_str = 'off'/'on'
# level_255 = 0-255
# level_100 = 0-100
# mode = 'normal'/'fast'/'instant'
# fast = 0/1
# instant = 0/1
state_topic: 'insteon/{{name}}/state'
state_payload: >
{ "state" : "{{on_str.upper()}}", "brightness" : {{level_255}}, "mode": {{mode}} }

But my mode part always returns nothing:
{ "state" : "ON", "brightness" : 160, "mode": }

Does anyone have mode working with fast (faston/fastoff) and can share your successful config.yaml or anything else I missed?

[TD22057]

Looks like it's not running the dev branch would be my guest. I just tried it again and it works fine for me.

[jtmoderate876]

Thx ted, I’ll try some more Sincerely, Jeff Tapia

[jtmoderate876]

Thx again, things are working now. My lack of understanding python, pip, bdis_wheel error, insteon user password (I don't think my insteon user has a password) etc. made it difficult (for me) to follow your setup instructions - but I seem to have it now.

[jtmoderate876]

Geez Ted, you've done a really nice job.

I need to get my head around a clean install because I don't think it works on raspberry pi without banging around a bit with the scripts, but...

Once set up it is fantastic with Node-RED MQTT.

I changed the /state concept on buttons so /state is always the last:
original:
state_topic: 'insteon/{{name}}/state/{{button}}'
subtopic friendly:
state_topic: 'insteon/{{name}}/button/{{button}}/state'

This way, for complex, json payloads I can search for an ending "/state" and append each json key/value pair as separate "subtopics" (/state, /brightness, /mode) for destinations that require simple data types (int, float/double, boolean, string) versus json objects.

Anyway, in short, this is EXCELLENT, incredibly thoughtful, configurable, professional.

Wow - thx.