Use Bagnowka images & data

[Libisch]

Overview:

• Bagnowka stores thousands pre-war views of Poland and Eastern Europe from before 1945. BH has permission to scrape and add the data to our database.
• All photos (and connected data) from A-Z Galleries should be merged with BH DBS to be displayed as one of our collection.

Status:

• We have a working scraping script, and the data is stored in a json dataset.
• Tested uploading to AWS s3.

Next steps:

• Upload images to usable storage and get new URLs.
• Update script and create a new dataset with valid image URLs.
• Convert the compete dataset to mach/use with API.

[Libisch]

@Beit-Hatfutsot/active-contributors

[OriHoch]

• for upload images I opened this issue (should review the code for uploading and check how hard it is to convert to google storage dbs-bagnowka-scrape#4) - for checking uploading to google storage (if it's simple it's worth to do rather then adding another provider)

[OriHoch]

• issue for researching how to match the bagnowka images json format to our mongo / elasticsearch document format - investigate how to match bagnowka image to dbs mongo photoUnit and/or photo documents #131
• should have a script that scrapes the bagnowka images and writes them as photos/photoUnits to BH dbs - should have a script that scrapes the bagnowka images and writes them as photos/photoUnits to BH dbs #132

[Libisch]

@TheGrandVizier @OriHoch
Bagnowka is now added to ES via pipelines, so this is not relevant, right?

[OriHoch]

issue is still relevant, but if it's done, feel free to close / move to QA