Make the index names configurable using env vars #712

dhruvkb · 2022-05-20T12:38:34Z

Description

This PR provides the mechanism to set the index mappings for the media types using environment variables. If not set, they will default to the name of the media type (which is the current configuration).

Checklist

My pull request has a descriptive title (not a vague title like Update index.md).
My pull request targets the default branch of the repository (main) or a parent feature branch.
My commit messages follow best practices.
My code follows the established code style of the repository.
I added or updated tests for the changes I made (if applicable).
I added or updated documentation (if applicable).
I tried running the project locally and verified that there are no visible errors.

Developer Certificate of Origin

Developer Certificate of Origin
Version 1.1

Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129

Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.


Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
    have the right to submit it under the open source license
    indicated in the file; or

(b) The contribution is based upon previous work that, to the best
    of my knowledge, is covered under an appropriate open source
    license and I have the right under that license to submit that
    work with modifications, whether created in whole or in part
    by me, under the same open source license (unless I am
    permitted to submit under a different license), as indicated
    in the file; or

(c) The contribution was provided directly to me by some other
    person who certified (a), (b) or (c) and I have not modified
    it.

(d) I understand and agree that this project and the contribution
    are public and that a record of the contribution (including all
    personal information I submit with it, including my sign-off) is
    maintained indefinitely and may be redistributed consistent with
    this project or the open source license(s) involved.

github-actions · 2022-05-20T12:40:18Z

API Developer Docs Preview: Ready

https://wordpress.github.io/openverse-api/_preview/712

Please note that GitHub pages takes a little time to deploy newly pushed code, if the links above don't work or you see old versions, wait 5 minutes and try again.

You can check the GitHub pages deployment action list to see the current status of the deployments.

sarayourfriend

What's the goal behind this change?

api/catalog/api/views/audio_views.py

obulat

LGTM!

dhruvkb · 2022-05-20T20:18:55Z

@sarayourfriend since we will be using versioned indices we might have more than one index of each media type in the Elasticsearch store (during transition periods). This PR allows us to keep those changes in the environment instead of modifying the code everytime there is a new index version.

openverse-bot · 2022-06-27T21:22:54Z

Based on the medium urgency of this PR, the following reviewers are being
gently reminded to review this PR:

@krysal
@sarayourfriend

This reminder is being automatically generated due to the urgency configuration.
Ignoring weekend days, this PR was updated 23 day(s) ago. PRs
labelled with medium urgency are expected to be reviewed within 4.

@dhruvkb, if this PR is not ready for a review, please draft it to prevent reviewers
from getting further unnecessary pings.

sarayourfriend

LGTM!

…o index_indep

dhruvkb · 2022-07-18T22:15:33Z

I'm going to merge this. This is backwards-compatible as it defaults to the media type when the env vars are not defined.

In #712 Elasticsearch configuration was moved out of the search controller area into the Django settings module. This unifies access to ES from across the application and is part of the refactoring being done to support the ES versioned index, multi-stage deployment. This commit keeps those changes and applies them to the old search_controller.py module that did not exist at the time the configuration change was applied.

* Revert "Fix related media endpoint (#808)" This reverts commit d7962f1. * Selectively revert "Add logging and fix all logging contexts (#790)" This reverts commit ce33572. * Revert "Refactor search controller for consistency and clarity (#778)" Preserves #730 This reverts commit d26df92. * Fix divergent Elasticsearch configurations In #712 Elasticsearch configuration was moved out of the search controller area into the Django settings module. This unifies access to ES from across the application and is part of the refactoring being done to support the ES versioned index, multi-stage deployment. This commit keeps those changes and applies them to the old search_controller.py module that did not exist at the time the configuration change was applied. * Lint * Prevent circular import We can replace the Any back with the serializer type in the future. * Add back related media endpoint integration tests * Lint

dhruvkb added 🟨 priority: medium Not blocking but should be addressed soon 🌟 goal: addition Addition of new feature 💻 aspect: code Concerns the software code in the repository labels May 20, 2022

dhruvkb requested a review from a team as a code owner May 20, 2022 12:38

dhruvkb requested review from krysal and obulat May 20, 2022 12:38

dhruvkb changed the title ~~Move AWS constants to configuration module~~ Make the index names configurable using env vars May 20, 2022

sarayourfriend reviewed May 20, 2022

View reviewed changes

obulat reviewed May 20, 2022

View reviewed changes

api/catalog/api/views/audio_views.py Outdated Show resolved Hide resolved

obulat approved these changes May 20, 2022

View reviewed changes

Base automatically changed from es_rand to main May 25, 2022 15:07

dhruvkb force-pushed the index_indep branch from de1360d to ddc915a Compare May 25, 2022 15:24

Make the index names configurable using env vars

e9d97a6

dhruvkb force-pushed the index_indep branch from ddc915a to e9d97a6 Compare May 25, 2022 15:27

dhruvkb requested a review from sarayourfriend May 25, 2022 15:33

sarayourfriend approved these changes Jul 11, 2022

View reviewed changes

dhruvkb added 2 commits July 15, 2022 15:41

Merge branch 'main' of https://github.com/WordPress/openverse-api int…

2f192d7

…o index_indep

Merge branch 'main' of https://github.com/WordPress/openverse-api int…

0a8d2a7

…o index_indep

dhruvkb merged commit e9cfd3c into main Jul 18, 2022

dhruvkb deleted the index_indep branch July 18, 2022 22:15

sarayourfriend mentioned this pull request Aug 3, 2022

MEDIA_INDEX_MAPPING not retrievable from settings #845

Closed

sarayourfriend mentioned this pull request Aug 5, 2022

Selectively revert suspect commits causing API instability #852

Merged

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make the index names configurable using env vars #712

Make the index names configurable using env vars #712

dhruvkb commented May 20, 2022

github-actions bot commented May 20, 2022 •

edited

Loading

sarayourfriend left a comment

obulat left a comment

dhruvkb commented May 20, 2022

openverse-bot commented Jun 27, 2022

sarayourfriend left a comment

dhruvkb commented Jul 18, 2022

Make the index names configurable using env vars #712

Make the index names configurable using env vars #712

Conversation

dhruvkb commented May 20, 2022

Description

Checklist

Developer Certificate of Origin

github-actions bot commented May 20, 2022 • edited Loading

sarayourfriend left a comment

Choose a reason for hiding this comment

obulat left a comment

Choose a reason for hiding this comment

dhruvkb commented May 20, 2022

openverse-bot commented Jun 27, 2022

sarayourfriend left a comment

Choose a reason for hiding this comment

dhruvkb commented Jul 18, 2022

github-actions bot commented May 20, 2022 •

edited

Loading