-
Notifications
You must be signed in to change notification settings - Fork 212
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add Elasticsearch cluster maintenance documentation #3032
Conversation
Full-stack documentation: https://docs.openverse.org/_preview/3032 Please note that GitHub pages takes a little time to deploy newly pushed code, if the links above don't work or you see old versions, wait 5 minutes and try again. You can check the GitHub pages deployment action list to see the current status of the deployments. Changed files 🔄: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for documenting all the learnings from the ES update. This doc is excellent 💯
and production until the final steps! | ||
``` | ||
|
||
#### 5. Run a reindex with the new staging cluster |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wonder how restoring a backup between versions, if possible, compares in cost and time to a total reindexing? 💭
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It might be possible, but upgrading between major versions doesn't allow you to do that, at least not 7 to 8. The data nodes use a separate block device, the SSD, that would be easy to image and restore from, but I don't know what benefit that gives us. We still need to test indexing against the new cluster anyway, so it's not like we could skip reindexing on staging and a data refresh.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me! A small nit but no blockers.
Co-authored-by: Dhruv Bhanushali <[email protected]>
Fixes
Related to #2744 which identified a missed step in the ES cluster upgrade process.
Description
I realised there was not a single place where ES cluster maintenance was comprehensively documented. This document includes everything I know about our ES cluster, including how to make configuration changes.
Docs preview: https://docs.openverse.org/_preview/3032/meta/maintenance/elasticsearch_cluster.html
Testing Instructions
Read the new document and identify any areas that need clarification. Remember the audience is the Openverse maintainers with infrastructure access, so any of the @WordPress/openverse-maintainers with the responsibility for infrastructure maintenance should be able to understand and employ this document.
Checklist
Update index.md
).main
) or a parent feature branch.Developer Certificate of Origin
Developer Certificate of Origin