Implement config for the backoff middleware #85

rilder-almeida · 2023-08-28T19:20:09Z

Issue:

During an incident in which we experienced a message bottleneck that was being indefinitely reattempted, we realized that the backoff middleware was instantiating the default configurations of the retrier and not allowing these configurations to be passed as parameters.

Solution:

Add to the configuration structure of the pipelines configurations for the backoff middleware's retrier. To maintain backward compatibility, if these configurations are not defined, default configurations will be used. Additionally, if a Dead Letter Queue (DLQ) topic is defined in the config but the number of retries is set to infinite, a warning log indicating the risk of an infinite loop will be emitted during resource instantiation.

Problema: Durante um incidente, onde tivemos um gargalo de mensagens que ficaram sendo retentadas infinitamente, percebemos que o middleware de backoff instanciava as configs padrões do retrier e não permitia que estas configs fossem passadas como parâmetro. Solução: Adiciona na estrutura de configuração do pipelineas configs para o retrier do backoff middleware. Para não haver nenhuma quebra de combatibilidade, caso estas configs não sejam definidas, configs padrões serão usadas. E se na presença de um tópico de DLQ definido nas config mas o numero de retentativas for infinito, um log de warnning sinalizando o risco de um loop infinito será emitido durante a instanciação dos recursos. Links: - https://app.clickup.com/t/86a0j6ea8

rjfonseca

LGTM

But I pointed out some concerns. Mainly that this new configuration could mislead the developers in thinking that there is a DLQ when in practice it will never be used. I would recommend that this particular case should raise an error instead of a warning.