Make plan for scheduling regular updating of global search

[dorothyyeager]

Summary

What we are after:
Users need to be able to access our content pages, including new ones, from the fec.gov search box. The awesome @patphongs has just finished scraping the site to update the global search, and it's working great. To keep it that way, we need to figure out when to schedule regular updates.

Related issues

• Global search not indexing all of Wagtail CMS pages #3281
• Research better ways of not forgetting major sections to index #3287

Completion criteria

• Decide how often the task of updating global search should be scheduled.
• Determine whether we want this to be an automated process at this point, or a schedule for manual updates based on the previously determined schedule
• Make follow-up ticket for the next time it should be scheduled

[lbeaufort]

Ideas for automation - can we have the api celery-beat run the management command for the cms and then open a pull request?

[patphongs]

Here's the XML ticket, which may help to automate our search indexing with search.gov: #3280

[dorothyyeager]

We've made a plan! cc: @AmyKort @PaulClark2

The plan
@patphongs @lbeaufort and I discussed the global search. We definitely want to automate it. The first step to doing that is to make the sitemap.xml file. Pat has a ticket in at #3280 which we've moved into PI11 planning. We'll also need to identify the transition pages that need to be included. See #3279 . And we need to make sure retired pages aren't captured. See #3376

Our thought is that moving to automation for this and resolving the two issues above can be a content epic for PI11.

Until we automate though, we will need to continue manual updates. We agreed that once a sprint makes sense. That way it will capture new pages created during a sprint. Pat says that the process, now that we've done it once, is easy. So I've created the issue for the next sprint, and part of the completion criteria will be to make the ticket for the sprint after that. See #3391 .

The plan is place, so we can close this ticket.

[JonellaCulmer]

@patphongs Do we have the process for manual updates documented somewhere?

cc: @dorothyyeager @lbeaufort

[dorothyyeager]

@JonellaCulmer I think it's in here: #3278 (I've referenced this issue in the ticket for the innovation sprint).