Skip to content
This repository has been archived by the owner on Aug 4, 2023. It is now read-only.

Add audio to the Database #69

Closed
wants to merge 10 commits into from
Closed

Add audio to the Database #69

wants to merge 10 commits into from

Conversation

obulat
Copy link
Contributor

@obulat obulat commented May 21, 2021

Related to WordPress / openverse # 19
This is a cleaned-up version of #64 related to saving the Audio data to the database.

This PR creates a new Audio database. The ingestion workflow now differs for audio and image tables, and the selection is based on the media_type, which is recorded in the tsv filename.

To finalize this PR, we need to agree on the audio metadata that we are going to save.

@obulat obulat mentioned this pull request May 21, 2021
3 tasks
Base automatically changed from airflow_update to master May 24, 2021 21:19
@obulat obulat requested review from krysal and zackkrida May 25, 2021 05:31
@obulat obulat changed the base branch from master to audio_storage May 27, 2021 10:28
@obulat obulat changed the title Add audio ingestion Add audio to the Database May 27, 2021
@zackkrida zackkrida requested a review from dhruvkb June 2, 2021 21:41
@obulat obulat changed the base branch from audio_storage to add_audio_storage June 3, 2021 17:00
obulat added 5 commits June 3, 2021 20:04
Refactor `image_store` into `media_store`, `image_store` and `audio_store`
Audio table definition is added to the `local_postgres/03_openledger_media_schema.sql` (renamed from `openledger_image_schema.sql`)
Docker executes sql files in alphabetical order, so the numbers were added to .sql file names to ensure correct loading order
@obulat obulat marked this pull request as draft June 9, 2021 08:16
@obulat obulat force-pushed the add_audio_storage branch 3 times, most recently from f08299a to 6b7ba33 Compare June 21, 2021 08:38
obulat added 2 commits June 24, 2021 09:06
# Conflicts:
#	src/cc_catalog_airflow/dags/common/storage/audio.py
#	src/cc_catalog_airflow/dags/common/storage/media.py
#	src/cc_catalog_airflow/dags/common/storage/test_audio.py
@obulat obulat mentioned this pull request Jun 25, 2021
@obulat obulat closed this Jun 25, 2021
@obulat obulat deleted the audio_db branch August 21, 2021 03:35
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant