Add Elastic Agent container

[jalvz]

The default policy should start apm-server.

Critically, this requires a service that installs the APM package prior to any data ingestion.

[graphaelli]

I think this should depend on the version of the stack. One idea:

For any version:

• --with-elastic-agent - plain old agent

For >= 8.0:

• --with-apm-server - agent with automatically configured apm-server.

For < 8.0:

• --with-apm-server - no change, apm-server container started
• --with-apm-server-agent - agent with automatically configured apm-server. Looking ahead to implementation, this should probably automatically set --apm-server-url but container dependencies are going to be tricky.

[jalvz]

Sounds good to me, except first bullet point - we only have agent/apm-server combo starting 7.11, so what is it for?

[graphaelli]

That would be for non-apm integrations

[graphaelli]

#1005 added --with-elastic-agent.

To automatically install the APM Integration this needs automation:

curl 'http://kibana:5601/api/fleet/package_policies' -H 'Content-Type: application/json' --data-raw '{"name":"apm-1","description":"","namespace":"default","policy_id":"27f16170-3b38-11eb-af1b-f5e1393cd644","enabled":true,"output_id":"","inputs":[{"type":"apm","enabled":true,"streams":[],"vars":{"enable_rum":{"type":"bool","value":true}}}],"package":{"name":"apm","title":"Elastic APM","version":"0.1.0-dev.1"}}'

where policy_id and package.version are variable.

@jalvz :

1. Is there a reason not to attach the integration to the the default policy?
2. How often does the package version change?

It seems we can look up the default policy in a number of ways. /api/fleet/agent_policies appears to be the most proper but I haven't verified whether that has filtering params or if they must all be paginated through looking for is_default: true.

Querying the global fleet settings also seems to do it and require no pagination but seems like a hack -

GET http://kibana:5601/api/fleet/settings
{"item":{"id":"27ee0610-3b38-11eb-af1b-f5e1393cd644","agent_auto_upgrade":true,"package_auto_upgrade":true,"kibana_urls":["http://kibana:5601"]}}

[jalvz]

1. AFAIK the default policy will tell agent to start metricbeat and filebeat. I'm not sure whether that reason is strong enough to create a separate policy only for APM, I guess the worst that happens is that elastic-agent runs metricbeat and filebeat always too?
   If it is not too hard I'd vote separate policy, but I don't have an strong opinion.

2. The-plan-right-now ™️ is to align with stack versioning after GA, so in production only with each release.
   For snapshot releases you'll notice the version qualifer in https://epr-snapshot.elastic.co/package/apm/0.1.0-dev.1/, which I intended to bump with each new push to the registry. But we don't have to do it that way.

[simitt]

I can take a stab at adding the --with-apm-server-agent automation, since I need that anyways.

[simitt]

The-plan-right-now ™️ is to align with stack versioning after GA, so in production only with each release.

+1; this will also help with automated upgrades of installed integrations that are part of a solution whenever Kibana is upgraded.

[simitt]

IMO it's fine to update the default policy; different deployment environments probably have the need for different default policies (e.g. kubernetes) for which cases it makes sense to update the shipped default policy before starting up the cluster. I suggest we do the same for the APM Server integration here.

It seems we can look up the default policy in a number of ways. /api/fleet/agent_policies appears to be the most proper but I haven't verified whether that has filtering params or if they must all be paginated through looking for is_default: true.

fyi - we can use curl "http://localhost:5601/api/fleet/agent_policies?kuery=ingest-agent-policies.is_default:true"

[graphaelli]

Great, sounds like we don't need to get fancy with either version lookups (use static) or policy setups (use default).

fyi - we can use curl "http://localhost:5601/api/fleet/agent_policies?kuery=ingest-agent-policies.is_default:true"

perfect

I can take a stab at adding the --with-apm-server-agent automation, since I need that anyways.

All yours then @simitt, thank you.

[jalvz]

I think this is done, we will have to update things when fleet server comes, but for now we are good. Feel free to reopen if you disagree.