[receiver/awscloudwatch] Correct config to fetch logs from multiple log groups in a single config

[ceevaaa]

Component(s)

receiver/awscloudwatch

What happened?

Description

Can someone please explain me a config that can fetch logs from a set of log groups (all streams of each log group).

The groups are -

1. /aws/kafka/*
2. /aws/elasticache/*
3. /aws/redis/*
4. /aws/rds/*
5. /aws/lambda/*

For just one log group you can write -

awscloudwatch:
    region: ap-south-1
    logs:
      poll_interval: 1m
      groups:
        autodiscover:
          limit: 100
          prefix: /aws/kafka/

But how will you do it for 5 different groups ?
Do I need to create five different AWSCloudWatch receivers ?

Any pointers would help here.
Thanks and Regards.

Collector version

v0.83.0

Environment information

OS
NAME="Amazon Linux"
VERSION="2"
ID="amzn"
ID_LIKE="centos rhel fedora"
VERSION_ID="2"
PRETTY_NAME="Amazon Linux 2"
ANSI_COLOR="0;33"
CPE_NAME="cpe:2.3⭕amazon:amazon_linux:2"
HOME_URL="https://amazonlinux.com/"

OpenTelemetry Collector configuration

No response

Log output

No response

Additional context

No response

[github-actions]

Pinging code owners:

• receiver/awscloudwatch: @djaglowski @schmikei

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[djaglowski]

@ceevaaa, you can use the named field to list the five log groups:

awscloudwatch:
  region: ap-south-1
  logs:
    poll_interval: 1m
    groups:
      named:
        /aws/kafka/:
        /aws/elasticache/:
        /aws/redis/:
        /aws/rds/:
        /aws/lambda/:

If you need to autodiscover log groups based on five different prefixes, then I believe you'll need to create five instances of the receiver.

[ceevaaa]

I see @djaglowski.
Just to be sure I have understood correctly, I am going to explain it.

I want to fetch all the logs from all the streams of these below log groups :

/aws/rds
/aws/rds/cluster
/aws/rds/instance
/aws/rds/foo/bar

/aws/kafka/
/aws/kafka/A
/aws/kafka/B

/aws/lambda/abc
/aws/lambda/xyz
/aws/lambda/

I think you get the idea.

Now as you have explained I will need to use five different awscloudwatchreceiver for this, which might look something like this -

awscloudwatch/rds:
    region: ap-south-1
    logs:
      poll_interval: 1m
      groups:
        autodiscover:
          limit: 50
          prefix: /aws/rds

awscloudwatch/kafka:
    region: ap-south-1
    logs:
      poll_interval: 1m
      groups:
        autodiscover:
          limit: 50
          prefix: /aws/kafka

awscloudwatch/lambda:
    region: ap-south-1
    logs:
      poll_interval: 1m
      groups:
        autodiscover:
          limit: 50
          prefix: /aws/lambda

.
.
.

Correct ?