Cherry-pick #18370 to 7.8: [Filebeat] Unescape characters in s3 file names #18412
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Cherry-pick of PR #18370 to 7.8 branch. Original message:
What does this PR do?
This PR unescape 3-byte encoded substring to regular string. For example: "%3D" to "=".
Why is it important?
When user uses folders in S3 bucket to organize log files, SQS notification actually encode the log file path like below:
File in S3 is
year=2020/month=05/test1.txt
but in SQS is converted toyear%3D2020/month%3D05/test1.txt
.This conversion needs to be undo when Filebeat tries to read the S3 file pointed by SQS message. Otherwise, this file will not be found.
Checklist
CHANGELOG.next.asciidoc
orCHANGELOG-developer.next.asciidoc
.Related issues
https://discuss.elastic.co/t/filebeat-s3-issue/230441