Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create a new doc page for Main UI‘s special features #2188

Closed
wants to merge 14 commits into from
43 changes: 39 additions & 4 deletions configuration/multimedia.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,16 @@ The framework supports some base [functions](https://openhab.org/javadoc/latest/

### Actions

You can set and get the volume in DSL rules by using these functions:

- `setMasterVolume(float volume)` : Sets the volume of the host machine (volume in range 0-1)
- `setMasterVolume(PercentType percent)` : Sets the volume of the host machine
- `increaseMasterVolume(float percent)` : Increases the volume by the given percent
- `decreaseMasterVolume(float percent)` : Decreases the volume by the given percent
- `float getMasterVolume()` : Returns the current volume as a float between 0 and 1

Please refer to the documentation of the [Automation add-ons](/addons/#automation) on how to use these actions from the respective language, e.g. JavaScript or JRuby.

## Audio Capture

openHAB is able to capture audio.
Expand Down Expand Up @@ -55,8 +59,16 @@ The distribution comes with these options built-in:
| `enhancedjavasound` | System Speaker (with mp3 support) | This uses the JRE sound drivers plus an additional 3rd party library, which adds support for mp3 files. |
| `webaudio` | Web Audio | Convenient, if sounds should not be played on the server, but on the client: This sink sends the audio stream through HTTP to web clients, which then cause it to be played back by the browser. Obviously, the browser needs to be opened and have a compatible openHAB UI running. Currently, this feature is supported by Main UI, Basic UI and HABPanel. |

Please refer to the [Main UI docs]({{base}}/ui/mainui.html#web-audio-sink) for setting up web audio in Main UI.

Additionally, certain bindings register their supported devices as audio sinks, e.g. Sonos speakers.

### Default Audio Sink

You can configure a default audio sink, which will be used if no audio sink is provided in audio and voice actions.

You can define the default audio sink either by textual configuration in `$OPENHAB_CONF/services/runtime.cfg` or in the UI by visitting the **Settings** page and opening **System Settings** -> **Audio**.

### Console commands

To check which audio sinks are available, you can use the console:
Expand All @@ -67,8 +79,6 @@ openhab> openhab:audio sinks
Web Audio (webaudio)
```

You can define the default audio sink either by textual configuration in `$OPENHAB_CONF/services/runtime.cfg` or in the UI in `Settings->Audio`.

In order to play a sound, you can use the following commands on the console:

```text
Expand Down Expand Up @@ -97,6 +107,17 @@ Alternatively the [`playSound()`](https://www.openhab.org/javadoc/latest/org/ope
- `playStream(String url)` : plays an audio stream from an url to the default sink (set url to `null` if streaming should be stopped)
- `playStream(String sink, String url)` : plays an audio stream from an url to the given sink(s) (set url to `null` if streaming should be stopped)

If no audio sink is provided, the default audio sink will be used.

Please refer to the documentation of the [Automation add-ons](/addons/#automation) on how to use these actions from the respective language, e.g. JavaScript or JRuby.

UI-based rules support audio actions as well.
Just create or edit a rule, add a new action, select "Audio & Voice" and the UI will then guide you trough the setup:

![Audio action setup in the UI](images/rule-play-audio.png)

Visit the [Blockly docs]({{base}}/configuration/blockly/rules-blockly-voice-and-multimedia.html) to learn how to use audio actions from Blockly.

#### Examples

```java
Expand All @@ -109,12 +130,18 @@ playStream("example.com")
playStream("sonos:PLAY5:kitchen", "example.com")
```

You will find more examples in the documentation of the [Automation add-ons](/addons/#automation) and the [Blockly docs]({{base}}/configuration/blockly/rules-blockly-voice-and-multimedia.html).

## Voice

### Text-to-Speech

In order to use text-to-speech, you need to install at least one [TTS service](/addons/#voice).

#### Default TTS Service & Voice

You can define a default TTS service and a default voice to use either by textual configuration in `$OPENHAB_CONF/services/runtime.cfg` or in the UI by visitting the **Settings** page and opening **System Settings** -> **Voice**.

#### Console Commands

To check which Text-to-Speech services are available, you can use the console:
Expand Down Expand Up @@ -147,8 +174,6 @@ openhab> openhab:voice voices
VoiceRSS - vietnamien (Vietnam) - default (voicerss:viVN)
```

You can define a default TTS service and a default voice to use either by textual configuration in `$OPENHAB_CONF/services/runtime.cfg` or in the UI in `Settings->Voice`.

In order to say a text, you can enter such a command on the console (The default voice and default audio sink will be used):

```text
Expand All @@ -169,6 +194,14 @@ Alternatively you can execute such commands within DSL rules by using the [`say(
You can select a particular voice (second parameter) and a particular audio sink (third parameter).
If no voice or no audio sink is provided, the default voice and default audio sink will be used.

Please refer to the documentation of the [Automation add-ons](/addons/#automation) on how to use these actions from the respective language, e.g. JavaScript or JRuby.

UI-based rules support voice actions as well.
Just create or edit a rule, add a new action, select "Audio & Voice" and the UI will then guide you trough the setup.
The presented dialog will look similar to the one shown [above](#actions-2).

Visit the [Blockly docs]({{base}}/configuration/blockly/rules-blockly-voice-and-multimedia.html) to learn how to use voice actions from Blockly.

##### Examples

```java
Expand All @@ -180,6 +213,8 @@ say("Hello world!", "voicerss:enUS", "sonos:PLAY5:kitchen")
say("Hello world!", "voicerss:enUS", "sonos:PLAY5:kitchen", new PercentType(25))
```

You will find more examples in the documentation of the [Automation add-ons](/addons/#automation) and the [Blockly docs]({{base}}/configuration/blockly/rules-blockly-voice-and-multimedia.html).

### Speech-to-Text

In order to use Speech-to-Text, you need to install at least one [STT service](/addons/#voice).
Expand Down
56 changes: 2 additions & 54 deletions settings/aboutpage.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,60 +75,8 @@ The first image depicts the normal white setting (off), the second image shows i

- **Disable card expansion animations**: Disables the animations when expanding a widget card.
- **Disable page transition animations**: Disables the transitions when switching from one page to the other.

### Enable Web Audio sink support

- This setting is also part of the miscellaneous settings.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From here on:

I have the feeling you deleted most of what I have written here. Please don't as I have put quite an amount of work by providing context, background and examples into it.

You move around redundant information but don't drop it, thanks.

- The intent of that option is to allow sending audio to that device, e.g. if that device is a tablet that is hanging on the wall in a room and a specific audio should be sent to that device for notification purposes.
- To activate audio output of that particular device, switch on this option.
In case you want to activate audio on multiple devices, each device needs to be activated by opening the about page on the device.
Technically, this information (and all the other miscellaneous settings) are stored in the local storage of the browser.
- For me information [see here](https://github.com/openhab/openhab-webui/pull/1422)

To test if it works play some static content that comes with openHAB in the /sounds dir using an openHAB console command.
Use [openHABs console](docs/administration/console.html#using-the-console) or just run the following from the linux shell:

> openhab-cli console

and then enter the following command:

```text
openhab> audio play webaudio barking.mp3
```

If you have a dog that gets triggered by barking you may opt for playing doorbell.mp3 instead.
If you play doorbell.mp3 your dog will then probably play its own barking audio sound. 😉
Note that most browsers block web pages from playing audio/video automatically nowadays (though some browsers have a setting to activate automatic play)

Here are some useful actions available to rules in which you can specify webaudio as sink, see:

- [playSound](/docs/configuration/multimedia.html#actions-2)

```text
playSound("webaudio", "barking.mp3")
```

- [say](/docs/configuration/multimedia.html#actions-3)

```text
say("Hello world!", "voicerss:enUS", "webaudio")
```

You can also configure webaudio as sink when creating rules in the UI:

![play-rule.png](images/play-rule.png)

or using it in a blockly rule:

![webaudio-blockly.png](images/webaudio-blockly.png)

It is important to understand that you cannot address one particular device.
When you play a sound on webaudio, it will be played on each device where webaudio is activated.

If you really like webaudio, you can also make it the default sink in the "Audio" settings - see [Settings->Audio](/docs/settings/services_system.html#Audio).
Then you can omit the webaudio sink ID from commands/actions.

![settings-audio.png](images/settings-audio.png)
- **Enable Web Audio sink support**: Please refer to the [Web Audio Sink docs]({{base}}/ui/mainui.html#web-audio-sink).
- **Listen for UI commands to**: Please refer to the [UI Command Item docs]({{base}}/ui/mainui.html#ui-command-item).

## Reload

Expand Down
Binary file removed settings/images/webaudio-blockly.png
Binary file not shown.
28 changes: 1 addition & 27 deletions tutorials/getting_started/tips-and-tricks.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,33 +44,7 @@ Imagine you need to find all usages of an Item in rules or pages.
You could open all your rules and pages to check them, or just use the developer sidebar search.
Enter your Item name, and you will see in which rules, scenes, scripts and pages that Item is used.

The developer sidebar search ignores case and searches inside the following entities:

- Things
- UID
- label
- Items
- name
- label
- tags (requires exact match)
- metadata
- Pages
- uid
- label
- content (widgets, tabs, charts, etc. with their configuration)
- Transformations
- uid
- label
- type
- Rules, Scenes & Scripts
- Item name & Thing UID of triggers, actions & conditions
- script code (e.g. Rules DSL, JavaScript, and even Blockly)
- script MIME types (requires exact match)
- Blockly scripts (using 'block', 'blockly' or 'blocksource' as search string, where case is ignored)
- tags (requires exact match)
- Persistence Configurations:
- label & service id of persistence service
- Items persisted by persistence service
For more information about the search and which entities are searchable, please refer to the [Developer Sidebar docs]({{base}}/ui/mainui.html#developer-sidebar-search).

### Pinned Objects

Expand Down
Binary file added ui/images/notification.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
113 changes: 113 additions & 0 deletions ui/mainui.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
---
layout: documentation
title: Main UI
---

# Main UI

Since openHAB 3.0, the so-called "Main UI" is the main web user interface of openHAB.
Main UI allows the administrator (you) to set up and configure openHAB, and provides many types of pages to visualize data and control Items.

This page aims at providing an overview over Main UI's "special" features, which means it does not try to document Main UI in general or provide a comprehensive overview over all its many capabilities and configuration pages.
If you are new to openHAB, please refer to the [Getting Started guide]({{base}}/tutorial/), which is introducing you to Main UI by using it to set up openHAB.

[[toc]]

## Developer Sidebar

Main UI provides a developer sidebar for administrator user(s), that is accessible from anywhere in the UI.
The developer sidebar is split into a "tools" section with several useful tools including a universal search across nearly all UI-configurable entities and a widget expression tester, and a "help" section aiming at providing help and useful information regarding configuration and setup.

If your device's screen is wide enough (it has to be at least 1280 pixels wide), you can open the developer side-panel by:

- Opening the **Developer Tools** page from the menu and enabling the developer sidebar.
- Clicking on the help icon displayed in the upper right corner of most pages.
- Pressing the key combination <kbd>Alt</kbd><kbd>Shift</kbd><kbd>D</kbd> on your keyboard.

### Developer Sidebar Search

As already mentioned, the developer sidebar provides a universal search across nearly all UI-configurable entities.
This means, you can search through most configuration available in Main UI, including Things, Items, pages.

Imagine you need to find all usages of an Item.
You could start to open all rules, scenes and scripts, check persistence configuration and have a look at pages and sitemaps to find usages of that Item - Or you use the developer sidebar search.

The developer sidebar search ignores case (so it doesn't matter whether you search `LivingRoom_Light` or `livingroom_light`) and searches inside the following entities:

- Things
- UID
- label
- Items
- name
- label
- tags (requires exact match, i.e. if the Item has a tag `Lightbulb`, only the search query `Lightbulb` (case doesn't matter) will match)
- metadata
- Pages (including Sitemaps)
- uid
- label
- content (widgets, tabs, charts, etc. with their configuration)/slots
- Transformations
- uid
- label
- type
- Rules, Scenes & Scripts
- Item name & Thing UID of triggers, actions & conditions
- script code (e.g. Rules DSL, JavaScript, and even Blockly)
- script MIME types (requires exact match)
- Blockly scripts (using `block`, `blockly` or `blocksource` as search string, where case is ignored)
- tags (requires exact match)
- Persistence Configurations:
- label & service id of persistence service
- Items persisted by persistence service

## Web Audio Sink
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This Info is also available here: https://www.openhab.org/docs/settings/aboutpage.html#enable-web-audio-sink-support
Please merge the content of the two pages and link accordingly.


Main UI implements a web audio sink which allows the openHAB server to play audio through the web browser.

To enable web audio sink support for a client, open the **Help & About** page from the menu and turn on the **Enable Web Audio sink support** option.
As it is setting specific to your browser environment, web audio support has to be enabled for every client where it should be used.

Please note that due to limitations in Safari (and possibly Chrome as well), a user interaction is required after the first audio stream has been sent to actually play it.
This means, that after opening Main UI, the first audio that should be played on the web audio sink is only player after the user interacts with Main UI in any way (i.e. touching the screen is enough).
For subsequent audio playback, that is not required anymore and the audio is played immediately.

Please also note that you cannot address a particular web audio-enabled device.
When you play a sound on web audio, it will be played on all devices where web audio is enabled.

Web audio sinks can be used for example on wall-mounted tablets to make announcements using a [text-to-speech (TTS) engine]({{base}}/configuration/multimedia.html#text-to-speech) and play notification or alarm sounds.

For more information about audio sinks as well as audio and voice actions, please refer to the [multimedia configuration docs]({{base}}/configuration/multimedia.html), where you will also find examples including UI-based rules and Blockly.

## UI Command Item

Main UI can be controlled from the openHAB server by setting up a so-called UI command Item.

Similarly to web audio support, this is a setting specific to your local browser environment, so you have to define the UI command Item for each client, which also allows you to different UI command Items for different clients.
To set up an Item as the UI command Item on your device, first create a particular Item of type `String`, then open the **Help & About** page from the menu and select an Item for the **Listen for UI command to** option.

The command Item has to be a `String` Item.
The UI is listening for commands to that Item, and if an Item command is a valid command string, the UI executes the given command.

Currently, the supported commands are:

- `navigate:$path`:
Navigate to a given `$path`, e.g. `navigate:/locations` to navigate to the **Locations** tab of the home page or `navigate:/page/my_custom_page` to navigate to a page with the ID `my_custom_page`.
For example, you can use this to open a video camera page of the door automatically when the doorbell is rung.
- `popup:$target`, `popover:$target` and `sheet:$target`:
Open a `$target`, which can be a page (`$target` is `page:` + a page ID), a custom widget (`$target` is `widget:` + widget ID) or any [`oh-` component]({{base}}/ui/components/) (`$target` is the component name), in a popup, popover or sheet.

For example, to open `oh-clock-card` inside a popup, send `popup:oh-clock-card` to the UI command Item.
- `notification:$text:$title:$subtitle:$titleRight:$closeTimeout`:
stefan-hoehn marked this conversation as resolved.
Show resolved Hide resolved
Display a notification inside the UI:
The `$text` parameter is mandatory, all other parameters are optional.
`$closeTimeout` is in milliseconds and defaults 5000, which means by default a notification will be closed after 5 seconds.
To leave out a parameter, leave a colon space empty, e.g. `notification:$text:$title:::$closeTimeout`.

For example, send `notification:This is the text.:This is the title.:This is the subtitle.:This is the right title.` to the UI command Item to display the following notification:
<p align="center"><img style="max-width: 40%;" src="images/notification.png"/></p>
- `close`:
Close all popups, popovers and sheets.
- `back`:
Navigate back.
- `reload`:
Reload the current page.
22 changes: 11 additions & 11 deletions ui/widget-expressions-variables.md
Original file line number Diff line number Diff line change
Expand Up @@ -126,31 +126,31 @@ Sometimes, however, it is helpful to add more information to a variable and for
The widget expression system can create objects in two different ways:

- JavaScript object syntax:

Objects can be defined within the expression system using the standard JavaScript syntax: `{'key1':'value1','key2':'value2'}`.
If a key doesn't contain special characters such as spaces, the quotes around keys can usually be ommited: `{key1:'value1',key2:'value2'}`.

::: tip

Due to the special meaning of `:[space]` in yaml, it is best to have no spaces between the `:` and the value.
If you have `:[space]` anywhere in your expression it will raise a YAML error unless you enclose the entire expression (= included) in another layer of quotes.

:::

Example:

```yaml
actionVariable: myObject
actionVariableValue: ={'name':props.item,'selected':true}
```

In this example, a variable is set to an object with keys `name` and `selected` using JavaScript object expression syntax.
- YAML object syntax
- YAML object syntax:

The other way to create objects is to take advantage of the relationship between YAML and JSON and place the key:value pairs as YAML keys under the initial key.

Here is a variable definition with the same results as the one above using the YAML syntax.

```yaml
actionVariable: myObject
actionVariableValue:
Expand Down Expand Up @@ -214,5 +214,5 @@ Substract one week from the state of `DateTime` and return a relative time repre

## Debugging Expressions

Expressions can be tested in the Widgets Expression Tester found in the Developer Sidebar
Expressions can be tested in the Widgets Expression Tester found in the [Developer Sidebar](mainui.html#developer-sidebar)
(<kbd>Shift</kbd><kbd>Alt</kbd><kbd>D</kbd>).