[ APM & Fleet]: 'apm.log' logs NOT available after enabling Logs and metrics and restarting Kibana from Cloud deployment page.

[dikshachauhan-qasource]

Kibana version:
7.13 BC7 Kibana Cloud Cloud-QA environment

Host OS and Browser version:
APM & Fleet, All

Preconditions

1. 7.13 BC7 Kibana Cloud Cloud-QA environment should be available.

Kibana version: 7.13 BC7
BUILD 40864
COMMIT 6ce6847436ff9bef0ad91268b6585e0f9339c9fd
Artifact link: https://staging.elastic.co/7.13.0-8eb98cbf/summary-7.13.0.html    BC7

2. Managed host should be avilable under Agents tab.

Steps to reproduce:

1. Enable logs and metrics at cloud deployment.
2. Wait for 1-2 mins for action to complete.
3. Got to stack monitoring Tab in kibana.
4. Validate data sync at apm & fleet overview resource usage.
5. Restart Kibana.
6. Go to logs tab.
7. Apply event.dataset filter for below: "apm.log"

Reference Link:
#99470

Actual Result:

• "elastic_agent.apm" data events not available on Logs tab after enabling logs collection settings and restarting Kibana at cloud deployment.

Screenshots:

NOTE:

• Also, attempted to trigger an error for APM manually, by running curl -i /config/v1/agents. However, no authorization error logs was returned at logs Tab.
  

[elasticmachine]

Pinging @elastic/fleet (Feature:Fleet)

[dikshachauhan-qasource]

@karanbirsingh-qasource Please review

[amolnater-qasource]

Reviewed and assigned to @EricDavisX

[EricDavisX]

I'd like to know if this is reproducible only in cloud or also in the self managed stack deploy. either way, cloud is critical - and I know too little about it to help triage. I'll pull in @ruflin @ph @simitt to help, please.

[simitt]

According to the screenshot above the request did not trigger an error. The APM Server log level is set to error, so it is expected that in this case no error is logged.
If you send the above request without providing an Auth header, or request some non-existent endpoint, for example curl -i https://test-deployment.qa.cld.elstc.co:9243/invalid then an error is returned and logged. I just tested and can confirm that everything works as expected, also after a Kibana restart.

[EricDavisX]

Silvia For The Win! Thank you @simitt .

@amolnater-qasource let's follow through and make sure we update any relating test for this to note the intentional 'wrong' usage to throw an error. I'll close this out, I see you have sent an email to the group - thanks again!

[dikshachauhan-qasource]

Hi @EricDavisX

We have updated our testcase as per above feedback from Silvia.

Testcase link: C79018

Thanks
QAS