[Metricbeat] AWS Module panic

[sorantis]

Metricbeat version: 7.6.1

While running Metricbeat with AWS module enabled and no permissions to run iam:ListAccountAliases I got the following error:

2020-04-01T12:57:03.827+0200	ERROR	runtime/panic.go:199	recovered from panic while
fetching 'aws/ec2' for host ''. Recovering, but please report this.	{"panic": "runtime error: 
invalid memory address or nil pointer dereference", "stack": 
"github.com/elastic/beats/libbeat/logp.Recover\n\t/go/src/github.com/elastic/beats/libbeat/logp/
global.go:105\nruntime.gopanic\n\t/usr/local/go/src/runtime/panic.go:679\nruntime.panicmem\n
\t/usr/local/go/src/runtime/panic.go:199\nruntime.sigpanic\n\t/usr/local/go/src/runtime/signal_un
ix.go:394\ngithub.com/elastic/beats/x-pack/metricbeat/module/aws/ec2.
(*MetricSet).createCloudWatchEvents\n\t/go/src/github.com/elastic/beats/x-
pack/metricbeat/module/aws/ec2/ec2.go:238\ngithub.com/elastic/beats/x-
pack/metricbeat/module/aws/ec2.(*MetricSet).Fetch\n\t/go/src/github.com/elastic/beats/x-
pack/metricbeat/module/aws/ec2/ec2.go:133\ngithub.com/elastic/beats/metricbeat/mb/module.
(*metricSetWrapper).fetch\n\t/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.
go:249\ngithub.com/elastic/beats/metricbeat/mb/module.
(*metricSetWrapper).startPeriodicFetching\n\t/go/src/github.com/elastic/beats/metricbeat/mb/
module/wrapper.go:217\ngithub.com/elastic/beats/metricbeat/mb/module.
(*metricSetWrapper).run\n\t/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go
:201\ngithub.com/elastic/beats/metricbeat/mb/module.
(*Wrapper).Start.func1\n\t/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:1
40"}

[elasticmachine]

Pinging @elastic/integrations-platforms (Team:Platforms)

[kaiyan-sheng]

@sorantis Thanks for creating this issue!! I can't reproduce this error with no iam:ListAccountAliases permission. I assume you are running the ec2 metricset with 7.6.1 metricbeat? I suspect it's caused by something wrong with DescribeInstances API call.

[kaiyan-sheng]

If only iam:ListAccountAliases is missing, Metricbeat should only return a warning:

2020-04-01T15:15:43.008-0600    WARN    [aws.ec2]       aws/aws.go:100  failed to list account aliases, please check permission setting: AccessDenied: User: arn:aws:iam::428152502467:user/test-mb is not authorized to perform: iam:ListAccountAliases on resource: *
        status code: 403, request id: 5baff3b1-f539-4cf0-b871-7230c0fade5c

[sorantis]

Here's what I was doing:

• Enabled aws module
• Left the config as is, only added credential_profile_name
• the profile I used didn't have all the required permissions and also was configured for a short session (1hr). As soon as a session expired I started seeing session errors.
• At this point I tried to stop metricbeat, which didn't really work, so I had to kill -9 it.
• Then I renewed a session and restarted metricbeat.
• Got the error.

[kaiyan-sheng]

@sorantis Awesome! Thanks for the info! I will try to reproduce this again!