Feature request: prefix in bucket list. Limit list to specfic subdirectory

[jalonsoa]

Hi,
we have a bucket with many files from different applications and we want ingest only files from one directory. At this moment we can filter with include / exclude patterns, but we get the complete list of the bucket each interval.

We need a bucket_prefix option to limit lists to only this prefix.

I create a pull request with the changes #16

[logicminds]

Suffering from the same problem. We should be able to search the bucket instead of pull down everything and filter out what we don't want. Wonder if there is a query parameter.

[kdHub]

The pull request worked well and supported both full and partial match to bucket prefix for my use case. Thanks for sharing and also support merging this as a feature