OOM crash while sending out a batch of requests

[laurentS]

We had the OOM Killer show up today while a batch was being sent to ~1050 public bodies. Once I restarted nginx everything resumed without problems, but that memory issue should not have happened.

There was no particularly high traffic on the site when this happened. The ruby process was using over 2GB of RAM when it was killed, which looks like a memory leak to me (or at least a linearly growing memory footprint with the size of the batch, if it's not a "leak" 😉). This was with alaveteli 0.43.1.0.

I will investigate further and report back here (hopefully with a fix).

This is similar to #7829 which was merged in 0.44.0.0 but in a slightly different context (sending the batch, as opposed to expiring embargo).

[garethrees]

FWIW we set the maximum batch size at 500.

a linearly growing memory footprint with the size of the batch, if it's not a "leak"

Yeah this sounds quite plausible.

[garethrees]

This is desirable, but unlikely to be worked on in the next 12 months so closing for now.