Bagnowka should be added as a DBS source

[Libisch]

Overview

• Bagnowka is an old website which will not be updated.
• As such, there is no need to process it's data more than ones.
• All the data is scraped, currently available in a single json file.
• All images were uploaded to an AWS s3 bucket.
  • Image URLs are already in the json file, only for the main_image.
  • Since this is an external source, the main_image is simply the first image (or the only image, in case there is just one).
  • Images other than main_image have IDs, which can be used for the full image_url path (by using the same base url as the main_image).

Next Step
Determine how to convert and sync to ES: build a new pipeline or handle it differently, since it's a one time job.

[Libisch]

@OriHoch?

[OriHoch]

you need to add a bagnowka pipeline spec (under bagnowka/pipeline-spec.yaml)

this pipeline should load the bagnowka json file and return common docs which can be synced to ES

this is very similar to what we are doing with clearmash - just instead of loading from making API calls, you load from a json file

you can see examples of pipelines specs here - https://github.com/OpenBudget/budgetkey-data-pipelines/tree/master/budgetkey_data_pipelines/pipelines
and this is the documentation of the datapackage_pipelines framework - https://github.com/frictionlessdata/datapackage-pipelines

[Libisch]

Done (PR #71), moving on to #87