Help to implement a separate queue for each user in NestJS

[antizhyk]

Hey, everybody. In short, I need to implement the ability to create a queue for each user in which a series of asynchronous tasks will be executed at a given interval (which in the future will be changed at the user's request).

Application description (description is general, just for understanding):
I create a bot that will be engaged in mailing specified letters to certain contacts and with a certain interval. All this can be customized by the user.

What I tried to do:
Since I am using NestJS I tried using Bull to customize the queues. But I encountered the following problems:

1. I don't understand how to create a queue for a single user, there is a possibility to create a fixed number of threads that is specified in the module, but users can be 5 and 10 and more, how to realize dynamic creation of threads is not clear.
2. The intervals that are specified in the instructions do not take into account the time of task completion. That is, if the interval is 1 second and the task is executed 5 seconds. Then either the task will be created every second, if the number of threads allows, or one after the other so that the waiting time of 1 second has already passed at the moment of task execution (5 seconds).

And these processes should be convenient to manage. For example, to pause or change intervals.

I also tried to use BullMQ with its fictitious multithreading, but there you can not set intervals, because in fact the thread is the main task in which the smaller ones are executed.

[manast]

I am not sure I understand all the requirements you have, but it sounds like groups could be very close to what you want to do: https://docs.bullmq.io/bullmq-pro/groups (not that this is only available on the Pro version).

[antizhyk]

All this time I tried to get into the description of the work of the groups and could not understand whether it is a solution (probably problems with the translator, well, with me because I still have not learned English properly)). I have drawn a diagram of the application, maybe it will help someone to understand whether Bull will work for me.

[manast]

There are several ways to do this, one that I have used successfully consists in having 1 repeatable job with a sensible repetition interval maybe something like 5 minutes (of course this time will vary depending on your requirements). This job will check in a database which users needs to run the "task", and for the users that needs it, we create a new job in a different queue.
So basically, instead of having a repeatable job per user you can have one global repeatable job, that checks if new jobs needs to be added for some users or not.