Alembic related failures on Focal

[kushaldas]

Description

We have various alembic related failures on Focal application tests.

Steps to Reproduce

• Create a fresh virtualenv from develop
• To test on focal, first move to dev_focal branch. git checkout dev_focal.
• make test-focal

Expected Behavior

Tests should pass.

Actual Behavior

FAILED tests/test_alembic.py::test_alembic_head_matches_db_models - AssertionError: Schema for ('table', 'journalist_login_attempt', 'journalist_login_attempt') did not match:
FAILED tests/test_alembic.py::test_alembic_migration_downgrade[60f41bb14d98] - subprocess.CalledProcessError: Command '['alembic', 'downgrade', 'f2833ac34bb6']' returned non-zero exit status 1.
FAILED tests/test_alembic.py::test_alembic_migration_downgrade[523fff3f969c] - subprocess.CalledProcessError: Command '['alembic', 'downgrade', 'f2833ac34bb6']' returned non-zero exit status 1.
FAILED tests/test_alembic.py::test_alembic_migration_downgrade[3da3fcab826a] - subprocess.CalledProcessError: Command '['alembic', 'downgrade', 'f2833ac34bb6']' returned non-zero exit status 1.
FAILED tests/test_alembic.py::test_alembic_migration_downgrade[48a75abc0121] - subprocess.CalledProcessError: Command '['alembic', 'downgrade', 'f2833ac34bb6']' returned non-zero exit status 1.
FAILED tests/test_alembic.py::test_alembic_migration_downgrade[a9fe328b053a] - subprocess.CalledProcessError: Command '['alembic', 'downgrade', 'f2833ac34bb6']' returned non-zero exit status 1.
FAILED tests/test_alembic.py::test_alembic_migration_downgrade[35513370ba0d] - subprocess.CalledProcessError: Command '['alembic', 'downgrade', 'f2833ac34bb6']' returned non-zero exit status 1.
FAILED tests/test_alembic.py::test_alembic_migration_downgrade[b58139cfdc8c] - subprocess.CalledProcessError: Command '['alembic', 'downgrade', 'f2833ac34bb6']' returned non-zero exit status 1.
FAILED tests/test_alembic.py::test_alembic_migration_downgrade[f2833ac34bb6] - subprocess.CalledProcessError: Command '['alembic', 'downgrade', 'e0a525cbab83']' returned non-zero exit status 1.
FAILED tests/test_alembic.py::test_schema_unchanged_after_up_then_downgrade[60f41bb14d98] - AssertionError: Schema for ('table', 'revoked_tokens', 'revoked_tokens') did not match:
FAILED tests/test_alembic.py::test_schema_unchanged_after_up_then_downgrade[fccf57ceef02] - AssertionError: Schema for ('table', 'submissions', 'submissions') did not match:
FAILED tests/test_alembic.py::test_schema_unchanged_after_up_then_downgrade[3d91d6948753] - AssertionError: Schema for ('table', 'replies', 'replies') did not match:
FAILED tests/test_alembic.py::test_schema_unchanged_after_up_then_downgrade[e0a525cbab83] - AssertionError: Schema for ('table', 'replies', 'replies') did not match:
FAILED tests/test_alembic.py::test_schema_unchanged_after_up_then_downgrade[b58139cfdc8c] - subprocess.CalledProcessError: Command '['alembic', 'downgrade', '-1']' returned non-zero exit status 1.
FAILED tests/test_alembic.py::test_schema_unchanged_after_up_then_downgrade[f2833ac34bb6] - AssertionError: Schema for ('table', 'journalist_login_attempt', 'journalist_login_attempt') did not match:
FAILED tests/test_alembic.py::test_schema_unchanged_after_up_then_downgrade[2d0ce3ee5bdc] - AssertionError: Schema for ('table', 'journalist_login_attempt', 'journalist_login_attempt') did not match:
FAILED tests/test_alembic.py::test_downgrade_with_data[b58139cfdc8c] - subprocess.CalledProcessError: Command '['alembic', 'downgrade', '-1']' returned non-zero exit status 1.

Comments

Suggestions to fix, any other relevant information.

[kushaldas]

For BASE_OS=focal securedrop/bin/dev-shell bin/run-test -s -v tests/test_alembic.py -k test_alembic_head_matches_db_models

The error is:

test_alembic_head_matches_db_models failed; it passed 0 out of the required 1 times.
        <class 'AssertionError'>
        Schema for ('table', 'journalist_login_attempt', 'journalist_login_attempt') did not match:
Left:
CREATE TABLE journalist_login_attempt (
        id INTEGER NOT NULL, 
        timestamp DATETIME, 
        journalist_id INTEGER, 
        PRIMARY KEY (id), 
        FOREIGN KEY(journalist_id) REFERENCES "journalists_tmp" (id)
)
Right:
CREATE TABLE journalist_login_attempt (
        id INTEGER NOT NULL, 
        timestamp DATETIME, 
        journalist_id INTEGER, 
        PRIMARY KEY (id), 
        FOREIGN KEY(journalist_id) REFERENCES journalists (id)
)
        [<TracebackEntry /root/code/securedrop/securedrop/tests/test_alembic.py:122>, <TracebackEntry /root/code/securedrop/securedrop/tests/test_alembic.py:61>]

[kushaldas]

Too many other errors are also related to this journalists_tmp table.

[kushaldas]

Here is the dump of the schema from our code:

This a9fe328b053a -> 60f41bb14d98 is where last time we are creating journalists_tmp table and dropping at the end.

main: /tmp/pytest-of-www-data/pytest-0/test_alembic_head_matches_db_m2/data/db.sqlite                                                                                                                               
INFO  [alembic.runtime.migration] Context impl SQLiteImpl.                                                                                                                                                          
INFO  [alembic.runtime.migration] Will assume non-transactional DDL.                                                                                                                                                
INFO  [alembic.runtime.migration] Running upgrade  -> 15ac9509fc68, init                                                                                                                                            
INFO  [alembic.runtime.migration] Running upgrade 15ac9509fc68 -> faac8092c123, enable security pragmas                                                                                                             
INFO  [alembic.runtime.migration] Running upgrade faac8092c123 -> 3d91d6948753, Create source UUID column                                                                                                           
INFO  [alembic.runtime.migration] Running upgrade 3d91d6948753 -> fccf57ceef02, create submission uuid column                                                                                                       
INFO  [alembic.runtime.migration] Running upgrade fccf57ceef02 -> 2d0ce3ee5bdc, added passphrase_hash column to journalists table                                                                                   
INFO  [alembic.runtime.migration] Running upgrade 2d0ce3ee5bdc -> e0a525cbab83, add column to track source deletion of replies                                                                                      
/opt/venvs/securedrop-app-code/lib/python3.8/site-packages/alembic/util/messaging.py:69: UserWarning: Skipping unsupported ALTER for creation of implicit constraint                                                
  warnings.warn(msg)                                                                                                                                                                                                
INFO  [alembic.runtime.migration] Running upgrade e0a525cbab83 -> 6db892e17271, add reply UUID                                                                                                                      
INFO  [alembic.runtime.migration] Running upgrade 6db892e17271 -> f2833ac34bb6, add UUID column for users table                                                                                                     
INFO  [alembic.runtime.migration] Running upgrade f2833ac34bb6 -> b58139cfdc8c, add checksum columns and revoke token table                                                                                         
Revision ID: b58139cfdc8c                                                                                                                                                                                           
Revises: f2833ac34bb6                                                                                                                                                                                               
Create Date: 2019-04-02 10:45:05.178481                                                                                                                                                                             
INFO  [alembic.runtime.migration] Running upgrade b58139cfdc8c -> a9fe328b053a, Migrations for SecureDrop's 0.14.0 release
INFO  [alembic.runtime.migration] Running upgrade a9fe328b053a -> 60f41bb14d98, Add Session Nonce To Journalist                                                                                                     
INFO  [alembic.runtime.migration] Running upgrade 60f41bb14d98 -> 3da3fcab826a, delete orphaned submissions and replies                                        
INFO  [alembic.runtime.migration] Running upgrade 3da3fcab826a -> 523fff3f969c, add versioned instance config                                                                                                       
INFO  [alembic.runtime.migration] Running upgrade 523fff3f969c -> 35513370ba0d, add Source.deleted_at                                                                 
INFO  [alembic.runtime.migration] Running upgrade 35513370ba0d -> 48a75abc0121, add seen tables   
Printing schema                                         
[('index', 'sqlite_autoindex_alembic_version_1', 'alembic_version', None),                                                                                                                                          
 ('index', 'sqlite_autoindex_instance_config_1', 'instance_config', None),
 ('index', 'sqlite_autoindex_journalists_1', 'journalists', None),                                                                                                                                                  
 ('index', 'sqlite_autoindex_journalists_2', 'journalists', None),
 ('index', 'sqlite_autoindex_replies_1', 'replies', None),                                                                                                                                                          
 ('index', 'sqlite_autoindex_revoked_tokens_1', 'revoked_tokens', None),
 ('index', 'sqlite_autoindex_seen_files_1', 'seen_files', None),
 ('index', 'sqlite_autoindex_seen_messages_1', 'seen_messages', None),
 ('index', 'sqlite_autoindex_seen_replies_1', 'seen_replies', None),                                                                                                                                                
 ('index', 'sqlite_autoindex_sources_1', 'sources', None),                                                                                                                                                          
 ('index', 'sqlite_autoindex_sources_2', 'sources', None),
 ('index', 'sqlite_autoindex_submissions_1', 'submissions', None),                                                                                                                                    
 ('table', 'alembic_version', 'alembic_version', 'CREATE TABLE alembic_version (\n\tversion_num VARCHAR(32) NOT NULL, \n\tCONSTRAINT alembic_version_pkc PRIMARY KEY (version_num)\n)'),
 ('table', 'instance_config', 'instance_config', 'CREATE TABLE instance_config (\n\tversion INTEGER NOT NULL, \n\tvalid_until DATETIME, \n\tallow_document_uploads BOOLEAN, \n\tPRIMARY KEY (version), \n\tUNIQUE (v
alid_until), \n\tCHECK (allow_document_uploads IN (0, 1))\n)'),
 ('table', 'journalist_login_attempt', 'journalist_login_attempt', 'CREATE TABLE journalist_login_attempt (\n\tid INTEGER NOT NULL, \n\ttimestamp DATETIME, \n\tjournalist_id INTEGER, \n\tPRIMARY KEY (id), \n\tFOR
EIGN KEY(journalist_id) REFERENCES "journalists_tmp" (id)\n)'),                 
 ('table', 'journalists', 'journalists', 'CREATE TABLE journalists (\n\tid INTEGER NOT NULL, \n\tuuid VARCHAR(36) NOT NULL, \n\tusername VARCHAR(255) NOT NULL, \n\tfirst_name VARCHAR(255), \n ... (286 characters
truncated) ... ast_access DATETIME, \n\tPRIMARY KEY (id), \n\tUNIQUE (username), \n\tUNIQUE (uuid), \n\tCHECK (is_admin IN (0, 1)), \n\tCHECK (is_totp IN (0, 1))\n)'),
 ('table', 'replies', 'replies', 'CREATE TABLE replies (\n\tid INTEGER NOT NULL, \n\tuuid VARCHAR(36) NOT NULL, \n\tjournalist_id INTEGER, \n\tsource_id INTEGER, \n\tfilename VARCHAR( ... (157 characters truncate
d) ...  REFERENCES "journalists_tmp" (id), \n\tFOREIGN KEY(source_id) REFERENCES sources (id), \n\tUNIQUE (uuid), \n\tCHECK (deleted_by_source IN (0, 1))\n)'),

 ('table', 'revoked_tokens', 'revoked_tokens', 'CREATE TABLE revoked_tokens (\n\tid INTEGER NOT NULL, \n\tjournalist_id INTEGER, \n\ttoken TEXT NOT NULL, \n\tPRIMARY KEY (id), \n\tFOREIGN KEY(journalist_id) REFER
ENCES "journalists_tmp" (id), \n\tUNIQUE (token)\n)'),                                                                                                                                                
 ('table', 'seen_files', 'seen_files', 'CREATE TABLE seen_files (\n\tid INTEGER NOT NULL, \n\tfile_id INTEGER NOT NULL, \n\tjournalist_id INTEGER, \n\tPRIMARY KEY (id), \n\tUNIQUE (file_id, journalist_id), \n\tFO
REIGN KEY(file_id) REFERENCES submissions (id), \n\tFOREIGN KEY(journalist_id) REFERENCES journalists (id)\n)'),                                                                                                    
 ('table', 'seen_messages', 'seen_messages', 'CREATE TABLE seen_messages (\n\tid INTEGER NOT NULL, \n\tmessage_id INTEGER NOT NULL, \n\tjournalist_id INTEGER, \n\tPRIMARY KEY (id), \n\tUNIQUE (message_id, journal
ist_id), \n\tFOREIGN KEY(message_id) REFERENCES submissions (id), \n\tFOREIGN KEY(journalist_id) REFERENCES journalists (id)\n)'),                                                                                  
 ('table', 'seen_replies', 'seen_replies', 'CREATE TABLE seen_replies (\n\tid INTEGER NOT NULL, \n\treply_id INTEGER NOT NULL, \n\tjournalist_id INTEGER, \n\tPRIMARY KEY (id), \n\tUNIQUE (reply_id, journalist_id)
, \n\tFOREIGN KEY(reply_id) REFERENCES replies (id), \n\tFOREIGN KEY(journalist_id) REFERENCES journalists (id)\n)'),                                                                                               
 ('table', 'source_stars', 'source_stars', 'CREATE TABLE source_stars (\n\tid INTEGER NOT NULL, \n\tsource_id INTEGER, \n\tstarred BOOLEAN, \n\tPRIMARY KEY (id), \n\tFOREIGN KEY(source_id) REFERENCES "sources_tmp
" (id), \n\tCHECK (starred IN (0, 1))\n)'),                                                                      
 ('table', 'sources', 'sources', 'CREATE TABLE sources (\n\tid INTEGER NOT NULL, \n\tuuid VARCHAR(36) NOT NULL, \n\tfilesystem_id VARCHAR(96), \n\tjournalist_designation VARCHAR(255)  ... (123 characters truncate
d) ... ted_at DATETIME, \n\tPRIMARY KEY (id), \n\tUNIQUE (uuid), \n\tUNIQUE (filesystem_id), \n\tCHECK (flagged IN (0, 1)), \n\tCHECK (pending IN (0, 1))\n)'),
 ('table', 'submissions', 'submissions', 'CREATE TABLE submissions (\n\tid INTEGER NOT NULL, \n\tuuid VARCHAR(36) NOT NULL, \n\tsource_id INTEGER, \n\tfilename VARCHAR(255) NOT NULL, \n\tsize ... (45 characters t
runcated) ... ecksum VARCHAR(255), \n\tPRIMARY KEY (id), \n\tFOREIGN KEY(source_id) REFERENCES sources (id), \n\tUNIQUE (uuid), \n\tCHECK (downloaded IN (0, 1))\n)')]
['CREATE', 'TABLE', 'instance_config', '(', 'version', 'INTEGER', 'NOT', 'NULL,', 'valid_until', 'DATETIME,', 'allow_document_uploads', 'BOOLEAN,', 'PRIMARY', 'KEY', '(version),', 'UNIQUE', '(valid_until),', 'CHE
CK', '(allow_document_uploads', 'IN', '(0,', '1))', ')']
['CREATE', 'TABLE', 'instance_config', '(', 'version', 'INTEGER', 'NOT', 'NULL,', 'valid_until', 'DATETIME,', 'allow_document_uploads', 'BOOLEAN,', 'PRIMARY', 'KEY', '(version),', 'UNIQUE', '(valid_until),', 'CHE
CK', '(allow_document_uploads', 'IN', '(0,', '1))', ')']
['CREATE', 'TABLE', 'journalist_login_attempt', '(', 'id', 'INTEGER', 'NOT', 'NULL,', 'timestamp', 'DATETIME,', 'journalist_id', 'INTEGER,', 'PRIMARY', 'KEY', '(id),', 'FOREIGN', 'KEY(journalist_id)', 'REFERENCES
', '"journalists_tmp"', '(id)', ')']
['CREATE', 'TABLE', 'journalist_login_attempt', '(', 'id', 'INTEGER', 'NOT', 'NULL,', 'timestamp', 'DATETIME,', 'journalist_id', 'INTEGER,', 'PRIMARY', 'KEY', '(id),', 'FOREIGN', 'KEY(journalist_id)', 'REFERENCES
', 'journalists', '(id)', ')']

[kushaldas]

On Xenial:

# sqlite3 --version
3.11.0 2016-02-15 17:29:24 3d862f207e3adc00f78066799ac5a8c282430a5f

On Focal:

# sqlite3 --version
3.31.1 2020-01-27 19:55:54 3bfa9cc97da10598521b342961df8f5f68c7388fa117345eeb516eaa837balt1

[kushaldas]

Digging more into the sqlite db:

root@03731129f84e:~/code/securedrop/securedrop# sqlite3 /tmp/pytest-of-www-data/pytest-0/test_alembic_head_matches_db_m0/data/db.sqlite                                                                             
SQLite version 3.31.1 2020-01-27 19:55:54                                                                                                                                                                           
Enter ".help" for usage hints.                                                                                                                                                                                      
sqlite> .schema                                                                                                                                                                                                     
CREATE TABLE alembic_version (                                                                                                                                                                                      
        version_num VARCHAR(32) NOT NULL,                                                                                                                                                                           
        CONSTRAINT alembic_version_pkc PRIMARY KEY (version_num)                                                                                                                                                    
);                                                                                                                                                                                                                  
CREATE TABLE journalist_login_attempt (                                                                                                                                                                             
        id INTEGER NOT NULL,                                                                                                                                                                                        
        timestamp DATETIME,   
        journalist_id INTEGER,         
        PRIMARY KEY (id),                  
        FOREIGN KEY(journalist_id) REFERENCES "journalists_tmp" (id)
);                                                            
CREATE TABLE source_stars (                            
        id INTEGER NOT NULL,
        source_id INTEGER,          
        starred BOOLEAN,          
        PRIMARY KEY (id),     
        FOREIGN KEY(source_id) REFERENCES "sources_tmp" (id),
        CHECK (starred IN (0, 1))        
);                         
CREATE TABLE sources (                                        
        id INTEGER NOT NULL,           
        uuid VARCHAR(36) NOT NULL,
        filesystem_id VARCHAR(96),
        journalist_designation VARCHAR(255) NOT NULL,
        flagged BOOLEAN,
        last_updated DATETIME,
        pending BOOLEAN,
        interaction_count INTEGER NOT NULL, deleted_at DATETIME,
        PRIMARY KEY (id),
        UNIQUE (uuid),
        UNIQUE (filesystem_id),
        CHECK (flagged IN (0, 1)),
        CHECK (pending IN (0, 1))
);
CREATE TABLE submissions (
        id INTEGER NOT NULL,
        uuid VARCHAR(36) NOT NULL,
        source_id INTEGER,
        filename VARCHAR(255) NOT NULL,
        size INTEGER NOT NULL,
        downloaded BOOLEAN, checksum VARCHAR(255),
        PRIMARY KEY (id),
        FOREIGN KEY(source_id) REFERENCES sources (id),
        UNIQUE (uuid),
        CHECK (downloaded IN (0, 1))
);
CREATE TABLE replies (                                                                                                                                                                                      [42/105]
        id INTEGER NOT NULL,
        uuid VARCHAR(36) NOT NULL,
        journalist_id INTEGER,
        source_id INTEGER,
        filename VARCHAR(255) NOT NULL,
        size INTEGER NOT NULL,
        deleted_by_source BOOLEAN NOT NULL, checksum VARCHAR(255),
        PRIMARY KEY (id),
        FOREIGN KEY(journalist_id) REFERENCES "journalists_tmp" (id),
        FOREIGN KEY(source_id) REFERENCES sources (id),
        UNIQUE (uuid),
        CHECK (deleted_by_source IN (0, 1))
);
CREATE TABLE revoked_tokens (
        id INTEGER NOT NULL,
        journalist_id INTEGER,
        token TEXT NOT NULL,
        PRIMARY KEY (id),
        FOREIGN KEY(journalist_id) REFERENCES "journalists_tmp" (id),
        UNIQUE (token)
);
CREATE TABLE journalists (                                                                                                                                                                                  [20/105]
        id INTEGER NOT NULL,
        uuid VARCHAR(36) NOT NULL,
        username VARCHAR(255) NOT NULL,
        first_name VARCHAR(255),
        last_name VARCHAR(255),
        pw_salt BLOB,
        pw_hash BLOB,
        passphrase_hash VARCHAR(256),
        is_admin BOOLEAN,
        session_nonce INTEGER NOT NULL,
        otp_secret VARCHAR(16),
        is_totp BOOLEAN,
        hotp_counter INTEGER,
        last_token VARCHAR(6),
        created_on DATETIME,
        last_access DATETIME,
        PRIMARY KEY (id),
        UNIQUE (username),
        UNIQUE (uuid),
        CHECK (is_admin IN (0, 1)),
        CHECK (is_totp IN (0, 1))
);
CREATE TABLE instance_config (
        version INTEGER NOT NULL,
        valid_until DATETIME,
        allow_document_uploads BOOLEAN,
        PRIMARY KEY (version),
        UNIQUE (valid_until),
        CHECK (allow_document_uploads IN (0, 1))
);
CREATE TABLE seen_files (
        id INTEGER NOT NULL,
        file_id INTEGER NOT NULL,
        journalist_id INTEGER,
        PRIMARY KEY (id),
        UNIQUE (file_id, journalist_id),
        FOREIGN KEY(file_id) REFERENCES submissions (id),
        FOREIGN KEY(journalist_id) REFERENCES journalists (id)
);
CREATE TABLE seen_messages (
        id INTEGER NOT NULL,
        message_id INTEGER NOT NULL,
        journalist_id INTEGER,
        PRIMARY KEY (id),
        UNIQUE (message_id, journalist_id),
        FOREIGN KEY(message_id) REFERENCES submissions (id),
        FOREIGN KEY(journalist_id) REFERENCES journalists (id)
);
CREATE TABLE seen_replies (
        id INTEGER NOT NULL,
        reply_id INTEGER NOT NULL,
        journalist_id INTEGER,
        PRIMARY KEY (id),
        UNIQUE (reply_id, journalist_id),
        FOREIGN KEY(reply_id) REFERENCES replies (id),
        FOREIGN KEY(journalist_id) REFERENCES journalists (id)
);

[kushaldas]

Found the issue on https://sqlite.org/lang_altertable.html:

Compatibility Note: The behavior of ALTER TABLE when renaming a table was enhanced in versions 3.25.0 (2018-09-15) and 3.26.0 (2018-12-01) in order to carry the rename operation forward into triggers and views that reference the renamed table. This is considered an improvement. Applications that depend on the older (and arguably buggy) behavior can use the PRAGMA legacy_alter_table=ON statement or the SQLITE_DBCONFIG_LEGACY_ALTER_TABLE configuration parameter on sqlite3_db_config() interface to make ALTER TABLE RENAME behave as it did prior to version 3.25.0.

[kushaldas]

We can solve it for now by using the following diff in a PR:

diff --git a/securedrop/alembic/env.py b/securedrop/alembic/env.py
index c16d34a5a..d6bce65b5 100644
--- a/securedrop/alembic/env.py
+++ b/securedrop/alembic/env.py
@@ -5,6 +5,8 @@ import sys
 
 from alembic import context
 from sqlalchemy import engine_from_config, pool
+from sqlalchemy.engine import Engine
+from sqlalchemy import event
 from logging.config import fileConfig
 from os import path
 
@@ -16,6 +18,12 @@ fileConfig(config.config_file_name)
 sys.path.insert(0, path.realpath(path.join(path.dirname(__file__), '..')))
 from db import db  # noqa
 
+@event.listens_for(Engine, "connect")
+def set_sqlite_pragma(dbapi_connection, connection_record):
+    cursor = dbapi_connection.cursor()
+    cursor.execute("PRAGMA legacy_alter_table=ON")
+    cursor.close()
+
 try:
     # These imports are only needed for offline generation of automigrations.
     # Importing them in a prod-like environment breaks things.

But, I personally think in future we should rewrite the migration files. I will wait to talk to the team before I open the PR.

[rmol]

If we don't rewrite the offending migrations, where we've used the procedure labeled incorrect under Making Other Kinds Of Table Schema Changes, but just call that pragma first thing within them instead of globally in env.py, we can avoid the risk that we change their results in some way that causes site-disabling failures with production databases, and still default to the improved ALTER TABLE behavior in future migrations.

I moved the fix to the beginning of the migrations that rename tables, the Alembic tests pass, and a database created with alembic upgrade head looks OK to me (though that's exactly the kind of verification I want to not depend on 😂). The adapted version is just:

    conn = op.get_bind()
    conn.execute("PRAGMA legacy_alter_table=ON")

If I remove the pragma from one of the later migrations, the tests fail, so that suggests we're getting a connection per migration, and so these local fixes shouldn't affect new migrations.