Sub-epic: Elasticsearch

[JonellaCulmer]

What we’re after: We need to add the ability to load archive MURs, test Elasticsearch backup and restore snapshots, and try the new beta service (coming in July 2020) before the current service is deprecated in October 2020.

Why we’re after it: To improve the reliability of the FEC’s Elasticsearch system and prepare for the deprecation of the current Elasticsearch service.

Tickets

• Research searching penalties in enforcement cases  fec-cms#4122
• Test and upload all legal data ES management commands on stage and prod space openFEC#5377
• Deploy the new ES service in production openFEC#4582
• In Dev space, update current elasticsearch service instance to es-medium plan openFEC#4729
• Upgrade Elasticsearch7.4 service in Prod spac openFEC#4663
• Upgrade Elasticsearch7.4 service in Stage space openFEC#4662
• Upgrade Elasticsearch7.4 service in Dev space openFEC#4659
• upgrade to Elasticsearch v7.4 beta service openFEC#4505
• Create elasticsearch snapshots of each index openFEC#4583
• Elasticsearch backups failing openFEC#4464
• upload arch-mur data to ES56 openFEC#4589
• Add python code to load the Arch-MUR to the ES openFEC#4490
• Copy 11 tables under schema mur_arch from Oracle to PG/Aurora openFEC#4465
• Load archived MUR data into postgres openFEC#4151
• Tweak the existing python code to load the AMUR to the ES service openFEC#4527
• Identify the new ES service upgrade version  openFEC#4474
• Preserve archived MUR data in Postgres or archived MUR XML openFEC#4129
• Work with the db team to copy the AMUR schemas/tables from Oracle to PG/Aurora openFEC#4473