Question: Parallel integration tests

[MeikelLP]

Hello,

my app is a simple (ASP.NET Core) Web Api. I'm using the AuditApiAttribute to decorate endpoints that should log audit events. This works totally fine and as expected.

Sadly I cannot test these logs in my integration/unit test due to the nature of Audit.NET: A singleton implementation. I tried to look into the implementation details of AuditApiAttribute but I saw it uses simple constructor calls (new X()) instead of dependency injection. I currently don't see how I can replace the default audit scope creation on a per request/app basis.

Most of my integration/unit tests are run in parallel so the global data provider cannot be used. If I use it it is replaced in every test setup which causes the audit logs to go to any collection (it's random basically). I do want my tests to be isolated and do not want all of them to write to a big collection.

I saw talks about this issue in the past (#315) but I'm not really sure how it was resolved. I don't want to write my own api attribute if possible. I also do not want to use the middleware if possible because I want a whitelist approach. Not a blacklist approach :)

Any ideas?

[thepirat000]

If you are testing whether the audit for an action is generated, you could setup an in-memory data provider for your tests (just like you would do to test an Entity Framework context)

I guess you can act on an action method and test whether the audit logs were saved as expected or not.

// Arrange
Audit.Core.Configuration.Setup().UseInMemoryProvider();

// Act
CallControllerMethod();

// Assert
var events = (Audit.Core.Configuration.DataProvider as InMemoryDataProvider).GetAllEvents();
Assert.AreEqual(1, events.Count);

Also, the InMemoryDataProvider is thread-safe so it should work fine with parallel tests

[MeikelLP]

This does not work as this just puts all messages in the same collection and I would have to pick my logs out of it. This is not good if u want to run isolated tests.

I set up a repo to reproduce this issue. Just run dotnet test and you will get The collection was expected to contain a single element, but it contained 4 elements. for all cases.

[MeikelLP]

This was most likely not noticed as NUnit does not run tests in parallel at all by default. However XUnit does this on a per class basis per default.

[thepirat000]

I have an idea to allow injecting the AuditDataProvider from the service collection for Audit.WebApi.
Of course, I need to test it and see if it is compatible with the other use cases, but if it is, you should be able to test it like this afterward:

// In your Program.cs:
builder.Services.AddSingleton<AuditDataProvider>(new InMemoryDataProvider());

// In your unit test, assert like this:
var dp = testApp.Services.GetRequiredService<AuditDataProvider>() as InMemoryDataProvider;
Assert.Single(dp.GetAllEvents());

So the AuditApi attribute and the middleware will honor the AuditDataProvider from the service collection and use the static
configuration only if the data provider was not found in the registered services.

[MeikelLP]

@thepirat000 please look at my PR which is linked above your message (#559). It's basically your idea. But just for WebApi attributes. It works. The first commit adds a test for that which fails. The second one uses the DI container for injecting the provider and thus fixes the tests.

Feel free to give feedback. I'm willing to help your project :)

[thepirat000]

Awesome, thanks!

Yes, the same idea. I'll also add it to the middleware and run some more tests.

[thepirat000]

This fix was included in version 20.1.3, please upgrade your references and re-test

https://github.com/thepirat000/Audit.NET/blob/master/src/Audit.WebApi/README.md#output

[MeikelLP]

Thank you! I think this a big quality improvement overall. Thanks for your work and response time :)

My tests all turned green now. No more randomness :D