PostgreSQL

RateLimiterPostgres

PostgreSQL >= 9.5

Usage

It works with pg, knex and sequelize.

It is required to provide ready callback as the second option of new RateLimiterPostgres(opts, ready) to react on errors during creating table(s) for rate limiters. See example below.

If tableCreated option is true, ready callback can be omitted.

See detailed options description here

// createRateLimiter.js file
const {RateLimiterPostgres} = require('rate-limiter-flexible');

module.exports = async (opts) => {
  return new Promise((resolve, reject) => {
    let rateLimiter
    const ready = (err) => {
      if (err) {
        reject(err)
      } else {
        resolve(rateLimiter)
      }
    }

    rateLimiter = new RateLimiterPostgres(opts, ready)
  })
}

const {RateLimiterRes} = require('rate-limiter-flexible')
const { Pool } = require('pg');
const createRateLimiter = require('./createRateLimiter')

const client = new Pool({
  host: '127.0.0.1',
  port: 5432,
  database: 'root',
  user: 'root',
  password: 'secret',
});

const opts = {
  storeClient: client,
  points: 5, // Number of points
  duration: 1, // Per second(s)

  // Custom options
  tableName: 'mytable', // if not provided, keyPrefix used as table name
  keyPrefix: 'myprefix', // must be unique for limiters with different purpose
};

createRateLimiter(opts)
  .then((rateLimiter) => {
    rateLimiter.consume(userId)
      .then((rateLimiterRes) => {
        // There were enough points to consume
      })
      .catch((rejRes) => {
        if (rejRes instanceof Error) {
          // Some Postgres error
          // Never happen if `insuranceLimiter` set up
        } else {
          // Can't consume
          console.log(rejRes instanceof RateLimiterRes)
        }
      });
    })
  .catch((err) => {
    console.error(err)
    process.exit()
  })

Data expired more than an hour ago, removed every 5 minutes by setTimeout.

By default, RateLimiterPostgres creates separate table by keyPrefix for every limiter.

All limits are stored in one table if tableName option is set.

Note: Carefully test performance, if your application limits more than 500 requests per second.

Sequelize and Knex support

It gets internal connection from Sequelize or Knex to make raw queries. Connection is released after any query or transaction, so workflow is clean.

const rateLimiter = new RateLimiterPostgres({
      storeClient: sequelizeInstance,
}, ready);

const rateLimiter = new RateLimiterPostgres({
      storeClient: knexInstance,
      storeType: `knex`, // knex requires this option 
}, ready);

See detailed options description here

Benchmark

Endpoint is pure NodeJS endpoint launched in node:10.5.0-jessie and postgres:latest Docker containers with 4 workers

User key is random number from 0 to 300.

Endpoint is limited by RateLimiterPostgres with config:

new RateLimiterPostgres({
  storeClient: pgClient,
  points: 5, // Number of points
  duration: 1, // Per second(s)
});

Statistics        Avg      Stdev        Max
  Reqs/sec       995.09     303.79    2010.29
  Latency        7.48ms     5.30ms    51.60ms
  Latency Distribution
     50%     5.25ms
     75%     8.07ms
     90%    16.36ms
     95%    21.85ms
     99%    29.42ms
  HTTP codes:
    1xx - 0, 2xx - 8985, 3xx - 0, 4xx - 21024, 5xx - 0

Heap snapshot statistic on high traffic