Audit Field Changes on Django Models

A Django app for auditing field changes on database models.

Installation

pip install django-field-audit

Documentation

Django Settings

To enable the app, add it to your Django INSTALLED_APPS configuration and run migrations. Settings example:

INSTALLED_APPS = [
    # ...
    "field_audit",
]

The "auditor chain" (see FIELD_AUDIT_AUDITORS in the Custom settings table below) is configured out of the box with the default auditors. If change_context auditing is desired for authenticated Django requests, add the app middleware to your Django MIDDLEWARE configuration. For example:

MIDDLEWARE = [
    # ...
    "field_audit.middleware.FieldAuditMiddleware",
]

The audit chain can be updated to use custom auditors (subclasses of field_audit.auditors.BaseAuditor). If change_context auditing is not desired, the audit chain can be cleared to avoid extra processing:

FIELD_AUDIT_AUDITORS = []

Custom settings details

Name	Description	Default value when unset
FIELD_AUDIT_AUDITEVENT_MANAGER	A custom manager to use for the AuditEvent Model.	field_audit.models.DefaultAuditEventManager
FIELD_AUDIT_AUDITORS	A custom list of auditors for acquiring change_context info.	["field_audit.auditors.RequestAuditor", "field_audit.auditors.SystemUserAuditor"]

Model Auditing

To begin auditing Django models, import the field_audit.audit_fields decorator and decorate models specifying which fields should be audited for changes. Example code:

# flight/models.py

from django.db import models
from field_audit import audit_fields


@audit_fields("tail_number", "make_model", "operated_by")
class Aircraft(models.Model):
    id = AutoField(primary_key=True)
    tail_number = models.CharField(max_length=32, unique=True)
    make_model = models.CharField(max_length=64)
    operated_by = models.CharField(max_length=64)

Audited DB write operations

By default, Model and QuerySet methods are audited, with the exception of four "special" QuerySet methods:

DB Write Method	Audited
Model.delete()	Yes
Model.save()	Yes
QuerySet.bulk_create()	No
QuerySet.bulk_update()	No
QuerySet.create()	Yes (via Model.save())
QuerySet.delete()	No
QuerySet.get_or_create()	Yes (via QuerySet.create())
QuerySet.update()	No
QuerySet.update_or_create()	Yes (via QuerySet.get_or_create() and Model.save())

Auditing Special QuerySet Writes

Auditing for the four "special" QuerySet methods that perform DB writes (labeled No in the table above) can be enabled. This requires three extra usage details:

Warning Enabling auditing on these QuerySet methods might have significant performance implications, especially on large datasets, since audit events are constructed in memory and bulk written to the database.

1. Enable the feature by calling the audit decorator specifying @audit_fields(..., audit_special_queryset_writes=True).
2. Configure the model class so its default manager is an instance of field_audit.models.AuditingManager.
3. All calls to the four "special" QuerySet write methods require an extra audit_action keyword argument whose value is one of:
   • field_audit.models.AuditAction.AUDIT
   • field_audit.models.AuditAction.IGNORE

Important Notes

• Specifying audit_special_queryset_writes=True (step 1 above) without setting the default manager to an instance of AuditingManager (step 2 above) will raise an exception when the model class is evaluated.
• At this time, QuerySet.delete(), QuerySet.update(), and QuerySet.bulk_create() "special" write methods can actually perform change auditing when called with audit_action=AuditAction.AUDIT. QuerySet.bulk_update() is not currently implemented and will raise NotImplementedError if called with that action. Implementing this remaining method remains a task for the future, see TODO below. All four methods do support audit_action=AuditAction.IGNORE usage, however.
• All audited methods use transactions to ensure changes to audited models are only committed to the database if audit events are successfully created and saved as well.

Bootstrap events for models with existing records

In the scenario where auditing is enabled for a model with existing data, it can be valuable to generate "bootstrap" audit events for all of the existing model records in order to ensure that there is at least one audit event record for every model instance that currently exists. There is a migration utility for performing this bootstrap operation. Example code:

# flight/migrations/0002_bootstrap_aircarft_auditing.py

from django.db import migrations, models
from field_audit.utils import run_bootstrap

from flight.models import Aircraft


class Migration(migrations.Migration):

    dependencies = [
        ('flight', '0001_initial'),
    ]

    operations = [
        run_bootstrap(Aircraft, ["tail_number", "make_model", "operated_by"])
    ]

Bootstrap events via management command

If bootstrapping is not suitable during migrations, there is a management command for performing the same operation. The management command does not accept arbitrary field names for bootstrap records, and uses the fields configured by the existing audit_fields(...) decorator on the model. Example (analogous to migration action shown above):

manage.py bootstrap_field_audit_events init Aircraft

Additionally, if a post-migration bootstrap "top up" action is needed, the the management command can also perform this action. A "top up" operation creates bootstrap audit events for any existing model records which do not have a "create" or "bootstrap" AuditEvent record. Note that the management command is currently the only way to "top up" bootstrap audit events. Example:

manage.py bootstrap_field_audit_events top-up Aircraft

Using with SQLite

This app uses Django's JSONField which means if you intend to use the app with a SQLite database, the SQLite JSON1 extension is required. If your system's Python sqlite3 library doesn't ship with this extension enabled, see this article for details on how to enable it.

Contributing

All feature and bug contributions are expected to be covered by tests.

Setup for developers

Create/activate a python virtualenv and install the required dependencies.

cd django-field-audit
mkvirtualenv django-field-audit  # or however you choose to setup your environment
pip install django pynose flake8 coverage

Running tests

Note: By default, local tests use an in-memory SQLite database. Ensure that your local Python's sqlite3 library ships with the JSON1 extension enabled (see Using with SQLite).

• Tests

  nosetests

• Style check

  flake8 --config=setup.cfg

• Coverage

  coverage run -m nose
  coverage report -m

Adding migrations

The example manage.py is available for making new migrations.

python example/manage.py makemigrations field_audit

Uploading to PyPI

First bump the package version in the field_audit/__init__.py file. Then create a changelog entry in the CHANGELOG.md file. After these changes are merged, you should tag the main branch with the new version. Then, package and upload the generated files to PyPI.

pip install -r pkg-requires.txt

python setup.py sdist bdist_wheel
twine upload dist/*

TODO

• Implement auditing for the remaining "special" QuerySet write operations:
  • bulk_update()
• Write full library documentation using github.io.
• Switch to pytest to support Python 3.10.

Backlog

• Add to optimization for instance.save(save_fields=[...]) [maybe].
• Support adding new audit fields on the same model at different times (instead of raising AlreadyAudited) [maybe].