Skip to content

Commit

Permalink
Remove url.QueryUnescape()
Browse files Browse the repository at this point in the history
We introduced [^1] the `url.QueryUnescape()` function to unescape
object keys from S3 notification in SQS messages.

However, the object keys in the S3 list object responses do not
require [^2] unescape.

We must remove the unescape to avoid unintended changes to the S3
object key.

[^1]: elastic#18370
[^2]: elastic#38012 (comment)
  • Loading branch information
zmoog committed Feb 23, 2024
1 parent a27e439 commit a4323ab
Showing 1 changed file with 1 addition and 9 deletions.
10 changes: 1 addition & 9 deletions x-pack/filebeat/input/awss3/s3.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@ import (
"context"
"errors"
"fmt"
"net/url"
"sync"
"time"

Expand Down Expand Up @@ -208,14 +207,7 @@ func (p *s3Poller) GetS3Objects(ctx context.Context, s3ObjectPayloadChan chan<-
// Metrics
p.metrics.s3ObjectsListedTotal.Add(uint64(totListedObjects))
for _, object := range page.Contents {
// Unescape s3 key name. For example, convert "%3D" back to "=".
filename, err := url.QueryUnescape(*object.Key)
if err != nil {
p.log.Errorw("Error when unescaping object key, skipping.", "error", err, "s3_object", *object.Key)
continue
}

state := newState(bucketName, filename, *object.ETag, p.listPrefix, *object.LastModified)
state := newState(bucketName, *object.Key, *object.ETag, p.listPrefix, *object.LastModified)
if p.states.MustSkip(state, p.store) {
p.log.Debugw("skipping state.", "state", state)
continue
Expand Down

0 comments on commit a4323ab

Please sign in to comment.