Use factory_boy for catalog test data generation #4751
Labels
🤖 aspect: dx
Concerns developers' experience with the codebase
✨ goal: improvement
Improvement to an existing user-facing feature
🟩 priority: low
Low priority and doesn't need to be rushed
🧱 stack: catalog
Related to the catalog and Airflow DAGs
Problem
The API uses factory_boy for generating test data. The catalog tests do not, and as such, require far more manual configuration of test data, for example:
https://github.com/WordPress/openverse/blob/change/tag-upsert-strategy/catalog/tests/dags/database/test_batched_update.py#L77-L106
This code is in a private function in one test suite, and defaults must manually be configured instead of automatically handled.
This increases the overhead of testing with sample data, which makes writing tests less pleasant over time.
Description
factory_boy
has bulk creation methods, and other useful approaches for making the generation of test data a non-issue when writing tests, and centralises the implementation of test data generation so improvements for one test can be shared by all existing and future tests.We can use
factory_boy
to generate test data, but inserting it into the database will still require translating the class instance data, because while it supports several ORMs, our Airflow DAGs do not use any of them, not even for table description.Once we implement #416 we can use the SQLAlchemy (or similar) integrations available for
factory_boy
, so that test data is easy to generate and maintain.In the mean time,
factory_boy
can be used to generate instances of DTOs that get translated to the dicts used to insert data into one table or another as needed, using our own methods to translate the DTOs into those dicts.Alternatives
Implement our own version of this with bespoke functions similar to the one in the code I linked above. That could be fine too, it just doesn't come with the benefits of years of maintenance and experience in a library like factory_boy meant to make this specific thing easy.
The text was updated successfully, but these errors were encountered: