Retrieve emails as a generator

[samuelhwilliams]

Summary

We have been getting intermittent 504 timeouts on our DOS email upload
job and they trace back to DMMailChimpClient's
get_email_addresses_from_list job, which tries to get emails back in
batches of 1000. This seems to take too long sometimes, so have reduced
it to a page size of 100 and converted the method to a generator to
reduce overhead.

Ticket

https://trello.com/c/4cK1wTvq/236

[galund]

Seems like the right change to make anyway, but it feels equally likely to me that a smaller page size / more calls would result in more timeouts, as result in fewer.

[samuelhwilliams]

Hopefully not...

[risicle]

We've typically given some kind of hint in the method name when returning an iterator, e.g. the api client's *_iter() methods.

[samuelhwilliams]

This is fair enough, although I think reading the method name you'd still expect to be given an iterable back - just from the fact it's plural on email_addresses. Happy enough to rename it if we think this is for the best, but I don't think it'll make the library more consistent with anything. Might need to be done as a "one fell swoop" sort of thing. 🤔

[risicle]

🤷‍♂️

[risicle]

FWIW if you wanted to show off you could do this as a yield from (member['email_address'] for member in member_data["members"])

[samuelhwilliams]

Aha. I had a feeling something like this existed but didn't follow through on that feeling to actually look it up.

Thanks