Recommended Redis/AWS Elasticache Configuration for BullMQ?

[dannyjlaurence-exp]

Sorry if this is a fairly basic question, but I'm struggling to wrap my arms around what I think to be a fairly straightforward question - what are the recommended settings for Redis (and in particular, AWS Elasticache) for BullMQ?

I imagine that the answer is that it depends a bit on how many jobs and queues we'll expect to have, but I have a feeling that some of the big picture questions could be answered in the documentation, such as:

• How much data is stored per queue or job? (To help with estimating the amount of memory we need to have)
• Are there any gotchas with using a clustered environment? Is a single Redis instance fine for most applications? (Thinking about things like failing over, or scaling up or down)
• An example Redis configuration that goes with an example Bullmq implementation

I'm sorry if this stuff is written down somewhere, the only page I could find that even mentions this is: https://docs.bullmq.io/guide/connections

These questions very well may fall outside the bounds of this project's concern, but I'm not quite sure where else to find this information. Anything to point me in the right direction would be helpful. Thank you very much.

[manast]

• Are there any gotchas with using a clustered environment? Is a single Redis instance fine for most applications? (Thinking about things like failing over, or scaling up or down)

Clustered is of no use for BullMQ, however elasticache is highly available by default since it uses a redundant replica that will be promoted to master if the current master fails.

[manast]

• How much data is stored per queue or job? (To help with estimating the amount of memory we need to have)

This is something we could put in the documentation, like average job size minus the data payload of course.

[manast]

• An example Redis configuration that goes with an example Bullmq implementation

Not sure what you mean exactly, but we could add some examples on how to connect to different providers.

[dannyjlaurence-exp]

Yea didn't have a specific request, more just the I go to elastic cache to set up a new Redis instance for Bullmq and the feeling of I have no idea what to pick among these options. Your answers here honestly are enough for most to get started. though, so I don't think there's a ton more to do. Thank you for your response!