You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
Right now, to load files from S3 in pipelines.yaml in any plugin pipeline author has to provide AWS credentials in each plugin. This is making the pipeline look noisy. Currently we support loading ISM policy(ism_policy_file ) and index template(template_file) from S3.
Describe the solution you'd like
Leverage the extensions work and create an s3_configuration extension in data-prepper-config.yaml.
Is your feature request related to a problem? Please describe.
Right now, to load files from S3 in
pipelines.yaml
in any plugin pipeline author has to provide AWS credentials in each plugin. This is making the pipeline look noisy. Currently we support loading ISM policy(ism_policy_file
) and index template(template_file
) from S3.Describe the solution you'd like
Leverage the extensions work and create an
s3_configuration
extension indata-prepper-config.yaml
.Now, pipeline author don't have to configure the credentials in pipeline and just use S3 URI path to load files from S3.
The text was updated successfully, but these errors were encountered: