Run tests against a read-only "dashboard" database connection

[simonw]

Related to #16. I want to encourage using a separate "dashboard" database alias which is configured something like this:

DATABASES = {
    "default": {
        "ENGINE": "django.db.backends.postgresql_psycopg2",
        "NAME": "mydb",
    },
    "dashboard": {
        "ENGINE": "django.db.backends.postgresql_psycopg2",
        "NAME": "mydb",
        "OPTIONS": {"options": "-c default_transaction_read_only=on -c statement_timeout=100"},
    },
}

I want to write the tests against this - but I'm running into some trouble because the test framework isn't designed to handle read-only database connections like this. I'm seeing errors like this:

Got an error creating the test database: cannot execute CREATE DATABASE in a read-only transaction

[simonw]

I tried doing this in my pytest_use_postgresql.py plugin module:

import os

import pytest
from dj_database_url import parse
from django.conf import settings
from testing.postgresql import Postgresql

_POSTGRESQL = Postgresql()


@pytest.hookimpl(tryfirst=True)
def pytest_load_initial_conftests(early_config, parser, args):
    os.environ["DJANGO_SETTINGS_MODULE"] = early_config.getini("DJANGO_SETTINGS_MODULE")
    settings.DATABASES["default"] = parse(_POSTGRESQL.url())
    settings.DATABASES["dashboard"] = parse(_POSTGRESQL.url())
    settings.DATABASES["dashboard"]["OPTIONS"] = {
        "options": "-c default_transaction_read_only=on -c statement_timeout=100"
    }


def pytest_unconfigure(config):
    _POSTGRESQL.stop()

That's what triggered the above errors.

[simonw]

https://pytest-django.readthedocs.io/en/latest/database.html#django-db-setup describes the pytest-django django_db_setup() fixture and says:

The default implementation creates the test database by applying migrations and removes databases after the test run.

You can override this fixture in your own conftest.py to customize how test databases are constructed.

So maybe I need to over-ride that fixture for my project?

Or I could try django_db_modify_db_settings() which says:

This fixture allows modifying django.conf.settings.DATABASES just before the databases are configured.

[simonw]

The implementation of the pytest-django fixtures is in https://github.com/pytest-dev/pytest-django/blob/edd33115e9ed205d241a6833839dcea5186d7d03/pytest_django/fixtures.py#L84

[simonw]

I moved that logic to conftest.py and it mostly seems to work:

import pytest

@pytest.fixture(scope="session")
def django_db_modify_db_settings():
    from django.conf import settings

    settings.DATABASES["dashboard"]["OPTIONS"] = {
        "options": "-c default_transaction_read_only=on -c statement_timeout=100"
    }

Just one catch: at the end of the tests I get these warnings:

test_project/test_dashboard.py::test_anonymous_users_denied
  /Users/simon/.local/share/virtualenvs/django-sql-dashboard-ZqMBSvf9/lib/python3.9/site-packages/
  django/db/backends/postgresql/base.py:304: RuntimeWarning:
  Normally Django will use a connection to the 'postgres' database to avoid running initialization queries
  against the production database when it's not needed (for example, when running tests). Django was
  unable to create a connection to the 'postgres' database and will use the first PostgreSQL database instead.
    warnings.warn(

test_project/test_dashboard.py::test_anonymous_users_denied
  /Users/simon/Dropbox/Development/django-sql-dashboard:0:
PytestWarning: Error when trying to teardown test databases: RuntimeError("generator didn't stop after throw()")

[simonw]

Oh this is nasty: tests on my laptop are now flaky. Running pytest some of the time gets me a clean test run, and other times it dies with nasty Got an error creating the test database: cannot execute CREATE DATABASE in a read-only transaction errors inside the django/test/utils.py:170: in setup_databases function.

[simonw]

CI is failing too: https://github.com/simonw/django-sql-dashboard/runs/2107262024 - lots of messages like this one:

==================================== ERRORS ====================================
__________________ ERROR at setup of test_superusers_allowed ___________________

self = <django.db.backends.utils.CursorWrapper object at 0x7f93bb35f7b8>
sql = 'CREATE DATABASE "test_test" ', params = None
ignored_wrapper_args = (False, {'connection': <django.db.backends.postgresql.base.DatabaseWrapper object at 0x7f93bb35f518>, 'cursor': <django.db.backends.utils.CursorWrapper object at 0x7f93bb35f7b8>})

    def _execute(self, sql, params, *ignored_wrapper_args):
        self.db.validate_no_broken_transaction()
        with self.db.wrap_database_errors:
            if params is None:
                # params default might be backend specific.
>               return self.cursor.execute(sql)
E               psycopg2.errors.ReadOnlySqlTransaction: cannot execute CREATE DATABASE in a read-only transaction

[simonw]

https://github.com/django/django/blob/3.1.7/django/test/utils.py#L155-L196 is that setup_databases() function.

[simonw]

Alternative idea: have the test framework create the default database as usual, then use the settings fixture to add the extra dashboard database later on:

@pytest.fixture(autouse=True)
def use_dummy_cache_backend(settings):
    settings.CACHES = {
        "default": {
            "BACKEND": "django.core.cache.backends.dummy.DummyCache",
        }
    }

https://pytest-django.readthedocs.io/en/latest/configuring_django.html#overriding-individual-settings

[simonw]

I tried that recipe but I got this error:

ScopeMismatch: You tried to access the 'function' scoped fixture 'django_db_modify_db_settings' with a 'session' scoped request object, involved factories

So I'm going to use an explicit fixture instead.