Add backoff to Stocksnap DAG for 5XX errors

[AetherUnbound]

Airflow log link

Note: Airflow is currently only accessible to maintainers & those given
access. If you would like access to Airflow, please reach out to a member of
@WordPress/openverse-maintainers.

https://airflow.openverse.org/dags/stocksnap_workflow/grid?dag_run_id=manual__2024-09-05T18%3A39%3A42%2B00%3A00&task_id=ingest_data.pull_image_data&base_date=2024-09-05T18%3A39%3A42%2B0000&tab=logs

Description

It looks like we're starting to encounter ephemeral 5XX errors with certain Stocksnap errors (separate from #4101). We should add a backoff.on_exception wrapper to the _get_filesize function of the ingestion class so these errors can be retried:

openverse/catalog/dags/providers/provider_api_scripts/stocksnap.py

Line 160 in 7f4fb7c

def _get_filesize(self, image_url):

This can be done using a decorator like Freesound:

openverse/catalog/dags/providers/provider_api_scripts/freesound.py

Line 182 in 05f8c55

@backoff.on_exception(backoff.expo, flaky_exceptions, max_tries=3)

Although we'd want the check to look like the global one for Science Museum (see #4715)

openverse/catalog/dags/providers/provider_api_scripts/science_museum.py

Lines 50 to 59 in 81d8da8

	# Science Museum's API tends to be flaky, so we add a backoff on every request
	# for 5XX error codes.
	# See: https://github.com/WordPress/openverse/issues/4710
	get_response_json = backoff.on_exception(
	backoff.expo,
	HTTPError,
	max_time=60 * 30, # 30 minutes
	# Only retry on 5XX errors
	giveup=lambda e: e.response.status_code not in {502, 503, 504},
	)(ProviderDataIngester.get_response_json)

Reproduction

Since these are temporal issues, I wasn't able to reproduce the 502 in the logs

DAG status

I've left this enabled, though I will add a silenced alert clause for this linking to this issue.

[MarleaM]

@AetherUnbound I'd love to work on this issue!

[AetherUnbound]

Hi @MarleaM, thank you for your interest in contributing to Openverse! I've assigned this issue to you. If you have any questions, you may leave them here.

Please check out our welcome and general setup documentation pages for getting started with setting up your local environment.

[MarleaM]

hi @AetherUnbound ! I believe I have a solution for this ready, however I am having trouble getting ov to run (i'm on windows & having issues with WSL).
I think that what I have should address the issue, though. Should I go ahead and push?

Thanks for your help : )