AWS usage with HTTP_PROXY on non-EC2

[dlvenable]

Bug Report

Describe the bug

Setting both HTTP_PROXY and HTTPS_PROXY on non-AWS network results in EC2 instance metadata error. In our case we don't even want to use EC2 for the AWS credentials. In fact, there is a default configuration provided in ${HOME}/.aws/config.

To Reproduce

Create a FluentBit configuration which outputs using http output with AWS SigV4.

When running FluentBit, set both HTTP_PROXY and HTTPS_PROXY.

[ info] [fluent bit] version=3.1.8, commit=, pid=410709...
[debug] [upstream] config->http_proxy: http://<user>:<pass> @<host>:<port> 
[debug] [imds] ec2_imds_client tcp host must be set to 169.254.169.254 
[error] [aws_credentials] EC2 IMDS configuration error 
[error] [output:http:http.0] Failed to create AWS Credential Provider 
[error] [output:http:http.0] failed to create aws credential provider for sigv4 auth 

Expected behavior

The ./.aws/config had a default configuration. Because of this, I'd expect that the failure in EC2 should not matter. I'm not sure why we even need to check it in the credentials chain.

Screenshots

N/A

Your Environment

• Version used: 3.1.8
• Configuration:
• Environment name and version (e.g. Kubernetes? What version?): Machine not on EC2
• Server type and version: N/A
• Operating System and version: Linux
• Filters and plugins: http output

Additional context

We were able to work-around this by adding the following export:

export NO_PROXY=169.254.169.254

However, failures with the EC2 credentials provider shouldn't prevent usage of other AWS credentials.