Unclear how I can reliably set the Partition Key in DynamoDB in my EF repository layer

[davidbrinton]

I'm using Audit.Net for all of our audits and have it turned on to audit every create or update event using the Entity Framework interceptor. We're pushing our events to DynamoDB. Up until now we've set the Partition Key to a random UUID for every event written, but this is not a great option as we can't look up all changes to a particular object. For example, if we want to see all of the events against the Orders table for order with OrderId 123. For this reason we're wanting to change the Partition Key to always be the primary key value of the table we're auditing.

It's not clear to me how I can do this in a thread safe way that supports all tables. I think I can do something like this in a basic operation

public async Task UpdateNoteAsync(Note note)
{
    using (AuditScope scope = AuditScope.Create(new AuditScopeOptions {  ExtraFields = new Dictionary<string, object> { { "EntityId", noteId } }}))
    using (NoteContext context = _ContextFactory.GetContext<NoteContext>())
    {
        // Update the note here

        await context.SaveChangesAsync();
    }
}

I think that will cause it to write the noteId value into the EntityId field and I think that would be thread safe. Is that right? If so, then, my next wonder would be what would happen if I am updating more than one table in a single block like that? For instance if I'm updating the Note table and the Order table? How would the scope know when to put NoteId in the EntityId field and when to put OrderId in the EntityId field?

I'm thinking there must be some way to achieve what I'm trying to achieve more easily though. For example, if there was a global configuration I could do like this?

Configuration.Setup()
    .UseDynamoDB(config => config
    .UseUrl(auditingDatabaseConnection)
    .Table(AUDITING_DATABASE_TABLE_NAME)
    .SetAttribute(AUDITING_DATABASE_ATTRIBUTE_NAME, ev => Guid.NewGuid()))
    .TableSettings(new List<TableSetting>
    { 
        new TableSetting { TableName = "Order", SourceFieldName = "OrderId", OutputFieldName = "EntityId", OutputFieldPrefix = "order_" },
        new TableSetting { TableName = "Note", SourceFieldName = "NoteId", OutputFieldName = "EntityId", OutputFieldPrefix = "note_" }
    });

Thanks for the help on learning what my options are here.

[thepirat000]

If you added the EF interceptor and surrounded the save changes with an AuditScope.Create, you'll end up having 2 events logged.
Is that what you want?

If so, you'll need something to co-relate the events. For example, if you're in an ASP.NET Core application, you could use the request trace identifier. I think the easiest way to attach the TraceIdentifier to the event is via a Custom Action in your startup logic:

public void ConfigureServices(IServiceCollection services)
{
    services.AddHttpContextAccessor();
    ...
}

public void Configure(IApplicationBuilder app, IHttpContextAccessor contextAccessor)
{
    Audit.Core.Configuration.AddCustomAction(ActionType.OnScopeCreated, scope =>
    {
        var httpContext = contextAccessor.HttpContext;
        scope.Event.CustomFields["CorrelationId"] = httpContext.TraceIdentifier;
    });
    ...
}

Now all your events will have the "CorrelationId" custom field set to a unique identifier by request

[davidbrinton]

@thepirat000 OK, I've started implementing this and I have two questions.

I don't see how to tie this into Entity Framework since adding a row to a table in the database won't trigger an event. I know I can't use the interceptor since that only works on SaveChanges. I've looked at the documentation once again and it looks like there is an Entity Framework Data Provider which looks like it expects to store the audit events using Entity Framework, which I don't think supports DynamoDB and there is also an Audit.EntityFramework extension which contains the interceptor I've been using, but also a low-level interceptor that looks promising but I guess I'm still unsure as to how to connect this custom AuditDataProvider to events that occur in Entity Framework.

The other question I had was, as this is an InsertEventAsync override, does that mean it only works for row creation statements, or can I also use this for UPDATE and DELETE operations?

[thepirat000]

The interceptors are only aware of things that happen within your code using an intercepted EF DbContext, they will not intercept operations that happen externally, like manually adding a row to a table, etc. If you need to audit any operation at the server level, you should consider using a native tool for your database, like SQL Server Audit Components.

Also, this is what connects the data provider to all the events that happen within Audit.NET:

Audit.Core.Configuration.Setup()
    .Use(new MyCustomDynamoDataProvider("THE URL"));

Here you're settling the global data provider that will handle all the audit events.

Note the InsertEventAsync in the data provider refers to the Audit Event insertion, not the operation in the database. Each EF audit event will contain 1 or more operations to the database. So, if you're only creating EF audit events, you can just override InsertEvent / InsertEventAsync like in my previous example.

The Data provider also has ReplaceEvent / ReplaceEventAsync methods, but you don't need to override those for the EF events, since they are never replaced.

[davidbrinton]

@thepirat000 OK, I think I understand, this AuditDataProvider customizes how events are mapped to the DynamoDB. So, can I still use the interceptor to trigger the event and this provider will split the different updates out into multiple rows? I think that's what you're saying, is that right?

[thepirat000]

That's correct!
Sorry if I wasn't clear, my english's not the best

[davidbrinton]

@thepirat000 I think your English is quite clear, I just was missing an understanding about how some of the underlying functionality in Audit.Net worked.

OK, I'm going to give this a shot. Thank you.

[thepirat000]

Another option is to use the DbContext.ContextId as a custom field to correlate the events, you'll need to change the order of the usings:

using (NoteContext context = _ContextFactory.GetContext<NoteContext>())
using (AuditScope scope = AuditScope.Create(new AuditScopeOptions { ExtraFields = new { EntityId = noteId, ContextId = context.ContextId.ToString() } }))
{
   ...

So now the manual event will also include the ContextId property (EF events already includes the context id by default) which is unique per DbContext instance