Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: provide connection details in getting started (fixes #2838) #2897

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion docs/gitbook/README (1).md
Original file line number Diff line number Diff line change
Expand Up @@ -45,12 +45,15 @@ Jobs are added to the queue and can be processed at any time, with at least one

```typescript
import { Worker } from 'bullmq';
import IORedis from 'ioredis';

const connection = new IORedis({ maxRetriesPerRequest: null });

const worker = new Worker('foo', async job => {
// Will print { foo: 'bar'} for the first job
// and { qux: 'baz' } for the second.
console.log(job.data);
});
}, { connection });
```

{% hint style="info" %}
Expand Down
57 changes: 42 additions & 15 deletions docs/gitbook/guide/connections.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,32 +7,59 @@ Every class will consume at least one Redis connection, but it is also possible
Some examples:

```typescript
import { Queue, Worker } from 'bullmq'
import { Queue, Worker } from 'bullmq';

// Create a new connection in every instance
const myQueue = new Queue('myqueue', { connection: {
host: "myredis.taskforce.run",
port: 32856
}});

const myWorker = new Worker('myqueue', async (job)=>{}, { connection: {
host: "myredis.taskforce.run",
port: 32856
}});
const myQueue = new Queue('myqueue', {
connection: {
host: 'myredis.taskforce.run',
port: 32856,
},
});

const myWorker = new Worker('myqueue', async job => {}, {
connection: {
host: 'myredis.taskforce.run',
port: 32856,
},
});
```

```typescript
import { Queue, Worker } from 'bullmq';
import { Queue } from 'bullmq';
import IORedis from 'ioredis';

const connection = new IORedis();

// Reuse the ioredis instance
const myQueue = new Queue('myqueue', { connection });
const myWorker = new Worker('myqueue', async (job)=>{}, { connection });
// Reuse the ioredis instance in 2 different producers
const myFirstQueue = new Queue('myFirstQueue', { connection });
const mySecondQueue = new Queue('mySecondQueue', { connection });
```

Note that in the second example, even though the ioredis instance is being reused, the worker will create a duplicated connection that it needs internally to make blocking connections. Consult the [ioredis](https://github.com/luin/ioredis/blob/master/API.md) documentation to learn how to properly create an instance of `IORedis.`
```typescript
import { Worker } from 'bullmq';
import IORedis from 'ioredis';

const connection = new IORedis({ maxRetriesPerRequest: null });
Copy link
Collaborator

@roggervalf roggervalf Nov 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

something to take in count is that we are recommending to not add this option for producers (Queue), only for consumers (Workers). Pls change it to reflect it base on https://docs.bullmq.io/bull/patterns/persistent-connections#queue


// Reuse the ioredis instance in 2 different consumers
const myFirstWorker = new Worker('myFirstWorker', async job => {}, {
connection,
});
const mySecondWorker = new Worker('mySecondWorker', async job => {}, {
connection,
});
```

Note that in the third example, even though the ioredis instance is being reused, the worker will create a duplicated connection that it needs internally to make blocking connections. Consult the [ioredis](https://github.com/luin/ioredis/blob/master/API.md) documentation to learn how to properly create an instance of `IORedis`.

Also note that simple Queue instance used for managing the queue such as adding jobs, pausing, using getters, etc. usually has different requirements from the worker.

For example, say that you are adding jobs to a queue as the result of a call to an HTTP endpoint - producer service. The caller of this endpoint cannot wait forever if the connection to Redis happens to be down when this call is made. Therefore the `maxRetriesPerRequest` setting should either be left at its default (which currently is 20) or set it to another value, maybe 1 so that the user gets an error quickly and can retry later.

On the other hand, if you are adding jobs inside a Worker processor, this process is expected to happen in the background - consumer service. In this case you can share the same connection.

For more details, refer to the [persistent connections](https://docs.bullmq.io/bull/patterns/persistent-connections) page.

{% hint style="danger" %}
When using ioredis connections, be careful not to use the "keyPrefix" option in [ioredis](https://redis.github.io/ioredis/interfaces/CommonRedisOptions.html#keyPrefix) as this option is not compatible with BullMQ, which provides its own key prefixing mechanism.
Expand Down
Loading