Init doesn't work in GovCloud

[dalgibbard]

Describe the question/issue

When using aws-for-fluentbit in the AWS GovCloud environment, loading from S3 using the init tagged images is not possible because of this line: https://github.com/aws/aws-for-fluent-bit/blob/mainline/init/fluent_bit_init_process.go#L205

Which assumes that the ARN prefix is always arn:aws:* (13 chars to the bucket name start), whereas, in GovCloud environments, this is actually arn:aws-us-gov:* (20 chars to the bucket name start)

This results in truncation on the wrong part, resulting in a target bucket/path of:

Configuration

Using envvars in ECS Fargate to load from S3; eg:

                {
                    "name": "aws_fluent_bit_init_s3_1",
                    "value": "arn:aws-us-gov:s3:::my_bucket_name/my_config_file.conf"
                },

Fluent Bit Log Output

Note the Bucket name starting with v:s3::: due to misaligned truncation

time="2023-03-30T16:16:17Z" level=fatal msg="[FluentBit Init Process] Cannot get bucket region of v:s3:::my_bucket_name + my_config_file.conf, you must be the bucket owner to implement this operation\n"

Fluent Bit Version Info

2.31.5

Cluster Details

ECS Fargate

Application Details

N/A

Steps to reproduce issue

See above

Related Issues

N/A

[tabacco]

The SDK even has a library specifically for parsing ARNs, which is used in that very same file: https://github.com/aws/aws-for-fluent-bit/blob/mainline/init/fluent_bit_init_process.go#L91-L94

[PettitWesley]

Thanks/Sorry! We will work on fixing this. Definitely should use the ARN parsing library.

[PettitWesley]

Testing the fix with this image, which can be pulled by any AWS account if you want to test/use as well:

144718711470.dkr.ecr.us-west-2.amazonaws.com/aws-for-fluent-bit:init-s3-arn-parse-fix

[dalgibbard]

Confirmed, this fixes the issue for me running in GovCloud environment :)

[dalgibbard]

Thank you @PettitWesley on the super fast turnaround on this, I really appreciate it!

[dalgibbard]

@PettitWesley just wondering - what's the release cadence like for this repo? Just waiting on a new image to be available for us to consume :)

[PettitWesley]

@dalgibbard probably next week I hope (no guarantee)... we're working out some issues in our build pipeline right now...

Also we needs to do a CVE release first:
https://github.com/aws/aws-for-fluent-bit/pull/627/files#r1160057787