Config Subentries

[balloob]

Context

There are integrations that have authentication stored in a config entry. Some of these integrations really are building blocks that have to be built on top of:

• OpenAI and Google AI contain authentication tokens.
• MQTT contains the broker connection
• KNX contains the Bus connection

MQTT and KNX have in common that we use the config entry to store the authentication, and the entities are created on top of it. They have to be configured in YAML.

For OpenAI and Google AI we decided to limit the user configuration to a single thing, and merge that into the config entry (in this case the prompt). It has the annoying side effect that a user has to re-enter API key for each OpenAI prompt it wants to use.

There are also notifications that are in dire need of a solution. If you add an SMS gateway, how would you define targets? Right now you need to keep your targets in your scripts. Wouldn’t it be great if you could pre-create targets and those are represented as entities ? The scrape integration actually "solved" this and allows defining the sensors based on the scrape information via a very elaborate options flow.

Solution

We create a new data type called “config subentry”. It is owned by a config entry. It is set up as part of async_setup_entry. Config subentries are created by config subentry flows and updated by config subentry options flows. This works similarly to how we create/edit config flows.

Config subentries will have a type such that we can decide what config subentry flows to show where in the UI.

For example, config subentries of type ai_agent could be shown on a dedicated page like we do for helpers. It would have a button “Add AI Agent” that opens an “Add integration” like flow, but for picking integration/config entry that supports adding “ai_agent” config subentries via a flow.

Example use cases

Create MQTT entities from UI

New flow with live preview with what the messages from MQTT will result in.

Create conversation entities

Add config subentry flows to Google and OpenAI to allow creating customized conversation entities. This will users to create different entities with different prompts, instead of having both OpenAI API key and prompt in 1 config entry.

Notification targets

Create targets for notification integrations, like an SMS gateway or Telegram.

Weather: add different locations

Add different weather stations to a single API key.

AI Agents

Example of how different AI providing integrations and their config subentries can provide different prompts and capabilities

And a UI to manage all config subentries across different integrations could look like this:

Lifecycle

Pre-config subentries, the data hierarchy is like this (source):

With subentries, we are introducing a new optional block between "Config Entry" and "Device Registry Entry". A device can either be linked directly to a config entry or a config subentry, not both. Deleting, disabling or re-enabling a config subentry will follow the existing logic:

Delete, disable or re-enable any object and all objects below will be adjusted accordingly.

Alternatives

An alternative is to get all integrations to put all config subentries inside their config entry.

The downside of this is that it will not be possible to group config subentries from different integrations together on a single page, like showing a page with all AI agents, weather stations or notification targets.

Having the config subentries as a new concept will also allow us to offer edit buttons for entities if we know the relationship between config subentry and entity is 1:1.

Note

It is not the goal to put automations and scripts into config subentries. Those are inside YAML as per our ADRs and will be kept there.

[joostlek]

If I read this some more examples come to mind, can you validate that these are a logical next step?

1. Integrations with an API key per "application". For example Trafikverket_train or Nederlandse spoorwegen. They have 1 API key, but theoretically could use multiple routes. So the API key is the main config entry, and every train/route you want to add is a config subentry. This would also be applicable for integrations like WAQI or weather providers.
2. Integrations with devices that take more information to set up. I think myuplink could make use of this. Recently I have been looking into meross, and that has a part cloud/part local approach, that would also benefit from this as the cloud part could be the config entry (to get device info and controls) and the sub entry to define where this device is located in the local network.

[joostlek]

I am also thinking about added value for bringing an integration like bayesian to the config flow (since that's one binary sensor that's using a lot of factors with a lot of attributes)

[balloob]

I think 1. very much yes. It's about splitting up the authentication from the configuration, if multiple configurations can exist.

Number 2 I might not fully understand. If it's just asking for more information, it could be an options flow.

[elupus]

If this also allow discovery, rfxtrx could use this and get rid of its annoying/complex config flow.

[balloob]

Didn't think about discovery but that would be an option for sure.

[janiversen]

This is a very nice idea, and actually very much like what I have been "toying" with for modbus, at least if I understand it correctly.

Thinking in terms of modbus, there would be a subentry for each transport type, with the specific transport parameters. So in the config entry the user select transport type, and can then set the parameters in the subentry.

The idea that the subentry have a type, is important, because it helps have different subentries.

[balloob]

Not sure what "transport type" means here, but I could see a future where one could add a modbus entity under a modbus communication type using a config subentry.

[janiversen]

tranport type is "serial", "tcp", "tls" and "udp" (the different ways modbus equipment can be attached to HA) each with a unique set of comm. params.

Using subentries for entries, would demand that one config_entry can hold a list of subentries ! But that would solve the biggest problem I see at the moment.

Being able to "offer" the modbus config_entry as subentry to other integrations, would help overcome some of the configuration issues we see today.

[balloob]

A config entry can hold many config subentries. That's the idea :)

Integration: modbus
Config entry: transport (serial, tcp, tls, udp)
Config subentries: 1 subentry per entity

[joostlek]

I think we also need to think about a way to define the flow for the integration. For example, MQTT would maybe be sufficient to only have set up the main config entry, and quit the flow, while another integration might want to register an x amount of flows directly at creation which the user should finish before it can load (To take MyUplink for example, they have devices that can expose up to 600 entities which are not all relevant (but they can't create a generic way to differentiate on that). So in that case it would be nice that the user first has to finish the device flow before the device is added (so the user had to select their relevant data points before adding the device).

Maybe some sort of way to allow the integration or the config flow to "discover" new subentries the user has to configure, that'd be great

[balloob]

I don't think we should use this as a way to ask the user more information that is necessary for set up. 600 entities is insane and very much an edge case.

I do believe that setting up a config entry should create config subentries. For example, configuring OpenAI should create a config subentry for providing an agent for the conversation integration.

[janiversen]

What way would you suggest we follow, to ask the user for more information.

Think of the following scenario (modbus, added more text to make it easier understand) ?

Currently I can only do the following:

"Select the way your modbus device is attached (serial, tcp, tls, udp)" ?

and simultaneously show all parameters for each type of attachment (something like 20 in total).

I would like to use something like subentry, to ask the user for more information, like ?

"Select the way your modbus device is attached (serial, tcp, tls, udp)" ?

then show the relevant parameters for the selected way of attachment.

It might be this is not 100% subentry, but having config_flow allowing a hierarchy would make a number of configurations a lot easier and in case of modbus allow a switch to config_flow.

[Kane610]

Why can't you just have the next step in the condig flow be based on the previous selection?

[farmio]

I would imagine the modbus example to have connection parameters in the config flow:

• menu step for serial, tcp, udp etc.
• next step detailed infos for the specific type of attachment

and the subentry flow to configure individual entities for that connection

• menu step for platform
• next step(s) for addresses, registers etc.

[janiversen]

@farmio you expressed my thinking a lot better !

[Jc2k]

Could matter use this?

Right now, the integration has a config entry for the connection to the matter addon. With this each pairing would be a sub-config-entry.

Pairings could then be deleted by the normal config entry delete hook.

This could potentially unlock using normal HA discovery flow for adding new matter devices too? AIUI For devices already visible by zeroconf we wouldn't need the app involvement, but previously that didn't really work because the discovery flows end in the creation of config entries.

[balloob]

I don't know. In the case of Matter the "pairing" is stored in the Matter SDK, running inside the Matter add-on. The integration is only having representation of that pairing.

We actually looked at using a normal config flow for Matter pairing and decided against it. It's so complicated that we want to control the UX to bring the optimal experience.

[elupus]

Thinking about sub entries... sub entries from another domain could make sense. Say modbus main domain, some heat pump needing to piggy back on the modbus bus config.

[jbouwh]

When it is about sub entries from an other domain, it is important we can specialize the schema.
E.g. When we would use person sub entries to supply the recipient on a notify entity implementation, we should be able to customize the data schema.
So in that case, an integration expose a sub entry feature to be used / linked with additional data to another integration.

For MQTT an alternative method to add manual MQTT items (now it is YAML only) would be a case. Not sure the subentries should be platform subentries from the entity domain, but that could be an idea as well.

[MartinHjelmare]

A device can either be linked directly to a config entry or a config subentry, not both.

It's not clear to me why a device can only be linked to either a config entry or a sub entry. Today it's up to integrations to decide how to link entities to devices/services. Eg one config entry can have single device/service with multiple entities or multiple devices/services with one or many entities. From an integration perspective, adding configuration via config subentries doesn't mean that an integration has a specific relationship between devices/services and its entities. It means that the integration needs to be able to add entities with their own configuration.

I think we should be very careful before we start deciding the relationship between devices/services and entities on behalf of integrations if that's not really needed.

[balloob]

I meant with "linked" is about a single relationship. A device can have multiple relationships with a combination of config entries and config subentries. This is all driven by the entities:

Today a device is created by adding an entity to Home Assistant. That entity is tied to either a config subentry or a config entry. That is the one that owns that entity. So if the config subentry is removed, it will remove the config subentry entities, and if the device has no other relationships, the device. The same for if a config entry is removed today.

[MartinHjelmare]

Does this mean that entities coming from different config subentries of the same config entry can be linked to the same device as long as that device isn't linked to those different config subentries but to the config entry?

[emontnemery]

It means that a device can have multiple subentry links for each config entry it's linked to.
So for example, the links could be:

(
    (config entry 1, no subentry),
    (config entry 1, subentry 1),
    (config entry 1, subentry 2),
    (config entry 2, subentry 1),
    (config entry 3, no subentry)
)

If one of those links are removed, the entities corresponding to that link are removed.
Links are automatically removed if a config entry or subentry is removed, and can also be manually removed by integrations.

[MartinHjelmare]

Thanks!

[MartinHjelmare]

This proposal has been approved by the core meeting.