[#161020787] Handle too many request responses from backend API

[chiadav]

[#161020787] too many request

[digitalcitizenship]

Affected stories

• 🌟 #161020787: Il client che contatta le API di notifica / preferenze deve fare retry in caso di status == 429

New dependencies added: mock-http-server and abortcontroller-polyfill.

mock-http-server

Author: Marco Pracucci

Description: Controllable HTTP Server Mock for your functional tests

Homepage: https://github.com/spreaker/node-mock-http-server

Created	almost 4 years ago
Last Updated	11 days ago
License	MIT
Maintainers	7
Releases	11
Direct Dependencies	body-parser, connect, multiparty and underscore

README

Node.js HTTP Server Mock

Mockable HTTP Server your functional tests.

npm install mock-http-server

Full working example

var ServerMock = require("mock-http-server");

describe('Test', function() {

    // Run an HTTP server on localhost:9000
    var server = new ServerMock({ host: "localhost", port: 9000 });

    beforeEach(function(done) {
        server.start(done);
    });

    afterEach(function(done) {
        server.stop(done);
    });

    it('should do something', function(done) {
        server.on({
            method: 'GET',
            path: '/resource',
            reply: {
                status:  200,
                headers: { "content-type": "application/json" },
                body:    JSON.stringify({ hello: "world" })
            }
        });

        // Now the server mock will handle a GET http://localhost:9000/resource
        // and will reply with 200 `{"hello": "world"}`
        done();
    });
});

Methods

Constructor

new ServerMock(httpConfig, httpsConfig) instance a new mockable HTTP/HTTPS Server. If httpConfig is defined, creates an HTTP server, while if httpsConfig is defined, creates an HTTPS server. They can be both defined.

Example:

var server = new ServerMock({
    host: "localhost",
    port: 80
}, {
    host: "localhost",
    port: 443,
    key: fs.readFileSync("private-key.pem"),
    cert: fs.readFileSync("certificate.pem")
});

start(callback)

Starts the server and invokes the callback once ready to accept connections.

Example:

beforeEach(function(done) {
    server.start(done);
});

stop(callback)

Stops the server and invokes the callback all resources have been released.

Example:

afterEach(function(done) {
    server.stop(done);
});

on(options)

Defines a request handler. Multiple calls to on() can be chained together.

Option	Default	Description
method	GET	HTTP method to match. Can be * to match any method.
path		HTTP request path to match. Can be * to match any path (to be used in conjunction with filter to allow custom matching)
filter		The value is a filter function fn(request): if it returns true the handler gets executed.
reply.status	200	HTTP response status code. Can be a number or a synchronous function fn(request) that returns the response status code.
reply.headers	{ "content-type": "application/json" }	HTTP response headers. content-length is managed by the server implementation.
reply.headersOverrides	{ "content-length": 1000 }	HTTP response headers to override to default headers (ie. content-length). If a value is set to undefined, the header gets removed from the response.
reply.body	empty string	HTTP response body. Can be a string, a synchronous function fn(request) that returns the body, or an asynchronous function fn(request, reply) that send the response body invoking reply(body).
reply.end	true	End the response once the body has been sent (default behaviour). If false, it will keep the response connection open indefinitely (useful to test special cases on the client side - ie. read timeout after partial body response sent).
delay	0	Delays the response by X milliseconds.

Example:

server.on({
    method: 'GET',
    path: '/resource',
    reply: {
        status:  200,
        headers: { "content-type": "application/json" },
        body:    JSON.stringify({ hello: "world" })
    }
});

or:

server.on({
    method: '*',
    path: '/resource',
    reply: {
        status:  200,
        headers: { "content-type": "application/json" },
        body:    function(req) {
            return req.method === "GET" ? JSON.stringify({ action: "read" }) : JSON.stringify({ action: "edit" });
        }
    }
});

or:

server.on({
    method: '*',
    path: '/resource',
    reply: {
        status:  200,
        headers: { "content-type": "application/json" },
        body:    function(req, reply) {
            setTimeout(function() {
                reply(req.method === "GET" ? JSON.stringify({ action: "read" }) : JSON.stringify({ action: "edit" }));
            }, 100);
        }
    }
});

or (JSON is parsed when the Content-Type is 'application/json'):

server.on({
    method: 'POST',
    path: '/resources',
    filter: function (req) {
      return _.isEqual(req.body, {
        name: 'someName',
        someOtherValue: 1234
      })
    },
    reply: {
        status:  201,
        headers: { "content-type": "application/json" },
        body: {
          id: 987654321,
          name: 'someName',
          someOtherValue: 1234
        }
    }
});

requests(filter)

Returns an array containing all requests received. If filter is defined, it allows to filter requests by method and/or path.

Filter	Description
method	Filter requests by method.
path	Filter requests by path.

Example:

// Returns all requests
server.requests();

// Returns all GET requests
server.requests({ method: "GET" });

// Returns all GET requests to /resource
server.requests({ method: "GET", path: "/resource" });

connections()

Returns an array containing all active connections.

getHttpPort()

Returns the port at which the HTTP server is listening to or null otherwise. This may be useful if you configure the HTTP server port to 0 which means the operating system will assign an arbitrary unused port.

getHttpsPort()

Returns the port at which the HTTPS server is listening to or null otherwise. This may be useful if you configure the HTTPS server port to 0 which means the operating system will assign an arbitrary unused port.

resetHandlers()

Clears all request handlers that were previously set using on() method.

Contribute

How to publish a new version

Release npm package:

1. Update version in package.json and package-lock.json
2. Update CHANGES.md
3. Release new version on GitHub
4. Run npm publish

License

MIT

abortcontroller-polyfill

Author: Martin Olsson

Description: Polyfill/ponyfill for the AbortController DOM API + optional patching of fetch (stub that calls catch, doesn't actually abort request).

Homepage: https://github.com/mo/abortcontroller-polyfill#readme

Created	about 2 years ago
Last Updated	6 months ago
License	MIT
Maintainers	1
Releases	31

README

AbortController polyfill for abortable fetch()

Minimal stubs so that the AbortController DOM API for terminating fetch() requests can be used
in browsers that doesn't yet implement it. This "polyfill" doesn't actually close the connection
when the request is aborted, but it will call .catch() with err.name == 'AbortError'
instead of .then().

const controller = new AbortController();
const signal = controller.signal;
fetch('/some/url', {signal})
  .then(res => res.json())
  .then(data => {
    // do something with "data"
  }).catch(err => {
    if (err.name == 'AbortError') {
      return;
    }
  });
// controller.abort(); // can be called at any time

You can read about the AbortController API in the DOM specification.

How to use

$ npm install --save abortcontroller-polyfill

If you're using webpack or similar, you then import it early in your client entrypoint .js file using

import 'abortcontroller-polyfill/dist/polyfill-patch-fetch'
// or:
require('abortcontroller-polyfill/dist/polyfill-patch-fetch')

Using it on browsers without fetch

If you need to support browsers where fetch is not available at all (for example
Internet Explorer 11), you first need to install a fetch polyfill and then
import the abortcontroller-polyfill afterwards.

The unfetch npm package offers a minimal fetch()
implementation (though it does not offer for example a Request class). If you need a polyfill that
implements the full Fetch specification, use the
whatwg-fetch npm package instead. Typically you will
also need to load a polyfill that implements ES6 promises, for example
promise-polyfill, and of course you need to avoid
ES6 arrow functions and template literals.

Example projects showing abortable fetch setup so that it works even in Internet Explorer 11, using
both unfetch and GitHub fetch, is available
here.

Using it along with 'create-react-app'

create-react-app enforces the no-undef eslint rule at compile time so if your
version of eslint does not list AbortController etc as a known global for
the browser environment, then you might run into an compile error like:

  'AbortController' is not defined  no-undef

This can be worked around by (temporarily, details here) adding a declaration like:

  const AbortController = window.AbortController;

Using the AbortController/AbortSignal without patching fetch

If you just want to polyfill AbortController/AbortSignal without patching fetch
you can use:

import 'abortcontroller-polyfill/dist/abortcontroller-polyfill-only'

Using it on Node.js

You can either import it as a ponyfill without modifying globals:

const { AbortController, abortableFetch } = require('abortcontroller-polyfill/dist/cjs-ponyfill');
const { fetch } = abortableFetch(require('node-fetch'));
// or
// import AbortController, { abortableFetch } from 'abortcontroller-polyfill/dist/cjs-ponyfill';
// import _fetch from 'node-fetch';
// const { fetch } = abortableFetch(_fetch);

or if you're lazy

global.fetch = require('node-fetch');
require('abortcontroller-polyfill/dist/polyfill-patch-fetch');

If you also need a Request class with support for aborting you can do:

const { AbortController, abortableFetch } = require('abortcontroller-polyfill/dist/cjs-ponyfill');
const _nodeFetch = require('node-fetch');
const { fetch, Request } = abortableFetch({fetch: _nodeFetch, Request: _nodeFetch.Request});

const controller = new AbortController();
const signal = controller.signal;
controller.abort();
fetch(Request("http://api.github.com", {signal}))
  .then(r => r.json())
  .then(j => console.log(j))
  .catch(err => {
      if (err.name === 'AbortError') {
          console.log('aborted');
      }
  })

See also Node.js examples here

Using it on Internet Explorer 11 (MSIE11)

The abortcontroller-polyfill works on Internet Explorer 11. However, to use it you must first
install separate polyfills for promises and for fetch(). For the promise polyfill, you can
use the promise-polyfill package from npm, and for fetch() you can use either the whatwg-fetch npm package (complete fetch implementation) or the unfetch npm package (not a complete polyfill but it's only 500 bytes large and covers a lot of the basic use cases).

If you choose unfetch, the imports should be done in this order for example:

import 'promise-polyfill/src/polyfill';
import 'unfetch/polyfill';
import 'abortcontroller-polyfill';

See example code here.

Using it on Internet Explorer 8 (MSIE8)

The abortcontroller-polyfill works on Internet Explorer 8. However, since github-fetch
only supports IE 10+ you need to use the fetch-ie8 npm package instead and also note that IE 8 only
implements ES 3 so you need to use the es5-shim package (or similar). Finally, just like with
IE 11 you also need to polyfill promises. One caveat is that CORS requests will not work out of the box on IE 8.

Here is a basic example of abortable fetch running in IE 8.

Contributors

• Martin Olsson
• Jimmy Wärting
• silverwind
• Rasmus Jacobsen
• João Vieira
• Cyril Auburtin
• Leonardo Apiwan
• Jake Champion
• Sai Srinivasan

License

MIT

Generated by 🚫 dangerJS

[codecov]

Codecov Report

Merging #1148 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #1148   +/-   ##
=======================================
  Coverage   42.82%   42.82%           
=======================================
  Files         220      220           
  Lines        5555     5555           
  Branches     1155     1155           
=======================================
  Hits         2379     2379           
  Misses       3168     3168           
  Partials        8        8

[codecov]

Codecov Report

Merging #1148 into master will decrease coverage by 0.04%.
The diff coverage is 58.33%.

@@            Coverage Diff             @@
##           master    #1148      +/-   ##
==========================================
- Coverage   42.22%   42.18%   -0.05%     
==========================================
  Files         258      259       +1     
  Lines        6910     6946      +36     
  Branches     1538     1544       +6     
==========================================
+ Hits         2918     2930      +12     
- Misses       3970     3994      +24     
  Partials       22       22

[Undermaken]

@cloudify Could you review it?

[cloudify]

I'd like to see a test for retryLogicForTransientResponseError

[Undermaken]

@chiadav tests and lint fails. Could you check?

[Undermaken]

@chiadav could you solve conflicts?