[Question] Will policies stay 1:1 or will they be mergeable?

[MarcusCaepio]

Hi all,
I know, all this is in beta, but I just want to share my thoughts with you :)
The status quo of the policy to agent mechanism is, that I can choose only one policy for an agent.
While trying all this in my cluster, I quickly mentioned, that it could end up in many policies and many admin work, to define all the several cases, when you have many server with multiple applications running on them.
E.g. I have "default" server, server with apache, with mysql, php-fpm, with LAMP and everything mixed in every imaginable combination. Only this case would be a policy set of 8 or 9, when you define every combination. I don't know what happens, when I e.g. assign a policy with mysql integration to a server, which has no mysql installed. Any errors? Overhead?

What I am wishing here, would be a multi select of policies, which I can assign to an agent and which are merged to a complete agent config. E.g. I have my default policy, which applies to all my agents. Then I could simply add a policy with "LAMP" integrations or a policy for only "MySQL" + "Apache" and so on. There should be also a check, if the different policies don't have two same integrations applied, to avoid overwriting or errors.

Cheers,
Marcus

[elasticmachine]

Pinging @elastic/agent (Team:Agent)

[andresrc]

cc @ph @mostlyjason

[mostlyjason]

@MarcusCaepio thanks for giving this feedback! You're right that our agent policies are 1:1 to hosts. The reason is that a monolithic policy is easier apply and you know exactly what is running on your host. With merged policies there may be conflicts to deal with.

I don't know what happens, when I e.g. assign a policy with mysql integration to a server, which has no mysql installed. Any errors? Overhead?

It would check the path provided to see if there are any log files. If not, no data will be sent. This will be similar for metrics. I assume there will be some small overhead but it will it will work.

We also have a capability that is still in the early stage but we are planning to build on it. Its called input conditions. Basically, you can apply a condition that says if this is X type of host, then activate these integrations https://www.elastic.co/guide/en/fleet/current/dynamic-input-configuration.html. The conditions and variables can be a powerful combination. It is available in the standalone mode and we are planning to add it to the UI later. Would this provide the flexibility you need?

[ruflin]

An other thing that could be interesting here is the concept of reusable integration policies which we discussed in the past: elastic/kibana#75867

[MarcusCaepio]

I think, working with conditions will be again a big need of time, when you want to manage +1000 servers. But I will test it, when everything is out of Beta Status. Thanks for the explanations.