Logs are missing from diagnostics bundle

[ycombinator]

For confirmed bugs, please report:

• Version: 8.8.0-SNAPSHOT built from ff5964f.

  $ sudo elastic-agent version
  

• Operating System: Linux

• Steps to Reproduce:

1. Install Agent in Standalone mode.

   sudo ./elastic-agent install
   

2. Get diagnostics bundle.

   sudo ./elastic-agent diagnostics
   

3. Look at the logs folder in the diagnostics bundle. It should contain at least one .ndjson log file but it is empty.

   unzip elastic-agent-diagnostics-*.zip -d diag
   ls -al diag/logs
   

[ycombinator]

I'm git bisect'ing to try and narrow down the specific commit that introduced this bug. I'll also try to figure out if this bug exists in 8.7.

[ycombinator]

Can confirm that this bug exists in 8.7 too, specifically in this build:

$ sudo elastic-agent version
Binary: 8.7.1-SNAPSHOT (build: 988ac05f8916bf66b8a6e395a62bac7aa68e79bd at 2023-04-06 20:33:07 +0000 UTC)
Daemon: 8.7.1-SNAPSHOT (build: 988ac05f8916bf66b8a6e395a62bac7aa68e79bd at 2023-04-06 20:33:07 +0000 UTC)

[ycombinator]

A few rounds of git bisect on main and it looks like the bug was introduced in c4c2ac0.

Likewise, on 8.7, it looks like the bug was introduced in 1ae907d.

[ycombinator]

Did some more digging on this issue, comparing a build from the "bad" commit (c4c2ac0) with one from the previous "good" commit (df3d52d) and here's what I found:

• The issue is NOT with diagnostics log collection (as this issue originally stated; that was merely the symptom). In the case of both commits, the diagnostics code path is attempting to collect logs from the same logs folder path (for my darwin builds, this was /Library/Elastic/Agent/data/elastic-agent-{short-sha}/logs/).
• The issue is with log creation. In the case of the bad commit, no logs (i.e. elastic-agent-*.ndjson files) are being written to the logs folder path, whereas they are being written to that that path in the case of the good commit.

I've updated the title and description of this issue accordingly.

[ycombinator]

Following up on my previous comment, it turns out logs (*.ndjson files) are being created in both cases ("good" and "bad" commits), but in different locations. Assuming darwin builds, after running sudo ./elastic-agent install:

• with the "good" commit (df3d52d), logs are being created under /Library/Elastic/Agent/data/elastic-agent-{short-sha}/logs.
• with the "bad" commit (c4c2ac0), logs are being created under /Library/Elastic/Agent/.

Of course, in either case, elastic-agent diagnostics is looking for logs under /Library/Elastic/Agent/data/elastic-agent-{short-sha}/logs. Which is why it finds them in the case of the "good" commit but not in the case of the "bad" commit.

Reverting issue tite and description to the original one, as I'm not sure if the fix needs to be in the diagnostics tool (log collection) or in the log creation code paths. I suspect it's the latter, but I'm not a 100% sure.