API memory leak

[sarayourfriend]

Description

The API's memory consumption pattern changed significantly in the last month.

This corresponds with the switch over to ES 8 in production and with this changelog: https://docs.openverse.org/changelogs/api/2023.08.28.20.43.01.html

The changelog doesn't have any changes that seem like they could possibly be related, so I suspect this has something to do with the ES 8 switch.

First thing that comes to mind is that we forgot to update the ES-py client version to match the cluster.

I'll get a PR for that up ASAP and then run a memory analysis locally to see if I can identify a leak.

Marked as critical because we need to resolve this before the ASGI switch.

[sarayourfriend]

I've assigned this to myself as I'm working on the ES client upgrade, but will need to hand this off if that doesn't solve the issue.

[sarayourfriend]

We deployed the ES client upgrade 12 hours ago, but it has not solved the memory leak. @obulat identified that it is still happening in production.

I unassigned myself the issue and am pinging @WordPress/openverse-api to hand off to someone else.

[dhruvkb]

I'm lowering the priority of this issue to high, because we're not actively working on this till the ASGI migration has been completed.

[AetherUnbound]

We believe this issue has been mitigated for now, as completed by the efforts of #3047 and #3048