Resilience

Resilience

DotPulsar will under some circumstances try to connect/reconnect or retry an operation (like sending or acknowledging a message) indefinitely. We try our hardest to give some sensible defaults, but if we are uncertain as to what is wrong, we choose to fault the producer/consumer/reader (which will change their state to 'Faulted'). This is necessary to prevent an endless loop that the user is totally unaware of. However we can't foresee all scenarios and the wanted behavior will certainly vary on a per-use-case basis, so naturally, we will provide the option of overwriting the defaults. When creating a PulsarClient we can optionally add an exception handler. This handler will get the chance to evaluate the exception before the default handler. Here's how it's done.

Implementing IHandleException

Let's overwrite the default behavior that will fault (rethrow) SocketExceptions with a SocketErrorCode of HostNotFound, HostUnreachable and NetworkUnreachable, so that all SocketExceptions are always retried with a delay of 5 seconds.
DotPulsar.Abstractions.IHandleException only requires us to implement one method (OnException) that takes an ExceptionContext.

public sealed class CustomExceptionHandler : IHandleException
{
    private readonly TimeSpan _retryInterval;

    public CustomExceptionHandler(TimeSpan retryInterval) => _retryInterval = retryInterval;

    public async ValueTask OnException(ExceptionContext exceptionContext)
    {
        if (exceptionContext.Exception is SocketException socketException)
        {
            await Task.Delay(_retryInterval, exceptionContext.CancellationToken);
            exceptionContext.Result = FaultAction.Retry;
            exceptionContext.ExceptionHandled = true;
        }
    }
}

Setting ExceptionContext.ExceptionHandled to true will ensure that no other exception handlers (including the default) will be called afterward.
ExceptionContext.Result can be set to Retry (will retry), Rethrow (will rethrow the exception and fault the producer/consumer/reader) or ThrowException (will throw ExceptionContext.Exception and fault the producer/consumer/reader. ExceptionContext.Exception has a public setter, just in case we want to convert the exception to something else).

We can also use ExceptionHandlers for logging and/or enriching the exception.

All there is left to do now is to add the exception handler when creating the PulsarClient.

Adding a custom exception handler

Please note that the same instance is used for all producers, consumers and readers.

var retryInterval = TimeSpan.FromSeconds(5);
var customExceptionHandler = new CustomExceptionHandler(retryInterval);
await using var client = PulsarClient.Builder()
                                     .ExceptionHandler(customExceptionHandler)
                                     .Build();

Multiple exception handlers can be added. They will be called in the same order as they were added.

Using the ExceptionHandler shortcuts

Explicitly implementing the IHandleException interface can be a bit verbose, so on the IPulsarClientBuilder we have created a shortcut.

await using var client = PulsarClient.Builder()
                                     .ExceptionHandler(CustomExceptionHandler)
                                     .Build(); //Connecting to pulsar://localhost:6650

Both Action<ExceptionContext> and Func<ExceptionContext, ValueTask> are supported.

private void CustomExceptionHandler(ExceptionContext exceptionContext)
{
    Console.WriteLine("I got this awesome exception: " + exceptionContext.Exception);
}