Datasets system.auth and system.syslog are not available for AL2023 under Data streams tab.

[amolnater-qasource]

Kibana Build details:

VERSION: 8.13.0 SNAPSHOT
BUILD: 71393
COMMIT: 4f3bc35472dfeb88c02466790bd3c96dcc98f4de
Artifact Link: https://snapshots.elastic.co/8.13.0-e2cda7bd/downloads/beats/elastic-agent/elastic-agent-8.13.0-SNAPSHOT-linux-x86_64.tar.gz

Host OS: Amazon Linux 2023(AL2023)

Preconditions:

1. 8.13.0 SNAPSHOT Kibana cloud environment should be available.

Steps to reproduce:

1. Install agent on AL2023 with agent policy having System integration.
2. Navigate to Data streams tab and observe no data available for system.auth and system.syslog datasets.

Screenshots:

Expected Result:
Datasets system.auth and system.syslog should be available for AL2023 under Data streams tab.

Testing Under:
https://github.com/elastic/ingest-dev/issues/2942

[elasticmachine]

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

[amolnater-qasource]

@manishgupta-qasource Please review.

[pierrehilbert]

The issue seems to be the same than #3650, AL2023 is using journald instead of rsyslog

[manishgupta-qasource]

Secondary review for this ticket is Done

[pierrehilbert]

Will be covered by elastic/beats#37086

[elasticmachine]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

[belimawr]

While elastic/beats#37086 paves the path to close this issue, it is also needed to update the system integration to use the journald input on hosts that have moved away from traditional log files.

[nimarezainia]

@belimawr is this also resolved by elastic/integrations#11618 ? I would like to get the AL2023 qualified.

[amolnater-qasource]

Hi Team,
We have revalidated this issue on latest 8.17.0 BC6 kibana cloud environment and found it fixed now.

Observations:

• Datasets system.auth and system.syslog is available for AL2023 under Data streams tab.

Screenshots:

Build details:
VERSION: 8.17.0 BC6
BUILD: 80521
COMMIT: e8a820624a03a412433584d3e3df951838e4c63c
Artifact Link: https://staging.elastic.co/8.17.0-6b31e673/downloads/beats/elastic-agent/elastic-agent-8.17.0-linux-x86_64.tar.gz

Hence we are closing and marking this issue as QA:Validated.

Thanks!