Add foreign key constraint to event_forward_extremities.

[erikjohnston]

The bulk of this PR is adding helper functions to make it easier to add constraints.

This is to help with #13476

Reviewable commit-by-commit.

[erikjohnston]

Note to self: we should probably log when we delete stuff during the validation?</del?

[DMRobertson]

Drive-by: are these types being used for validation or is it "just" a dataclass? (If the former, recommend a short unit test to sanity check that the validation behaves as you expect)

[erikjohnston]

Ermh, it's mostly just used as a data class I think? So long as the pydantic models correctly round trip it should be fine?

[MadLittleMods]

Is there a more automatic way we can copy things instead of relying on our copy-paste skills here to make sure it's right?

Simple, stupid way is also good but I feel like there might be a time when we overlook and miss an index in the future. Or a race with PRs where some new column is added but isn't considered in this background update.

We can kick that can down the road though.

[erikjohnston]

Actually, probably using the special sqlite_schema table. Let me give it a go

[erikjohnston]

I have changed it to automatically port the indexes for SQLite, but I can't see a nice way of automating the table creation without having to parse the SQL.

[MadLittleMods]

Fancy!!

---

From some Googling as well, doesn't seem possible for SQLite out of box 👍

With Postgres, we could use CREATE TABLE new_table LIKE old_table; (reference) which is exactly what we want to copy the schema/structure only.

SQLite does have CREATE TABLE ... AS SELECT but it doesn't copy the primary key, constraints, or defaults. Kinda sucks as well because it populates the table as well even if we LIMIT the SELECT. Even though, it doesn't copy constraints (I assume that means indexes as well) but we could work around it with the code you already have to copy those over. Perhaps we could overcome all of these problems by applying all indexes/constraints (currently the built-in ones are excluded with AND sql IS NOT NULL)

synapse/synapse/storage/background_updates.py

Lines 1103 to 1110 in 2451d8d

	# Fetch the indexes/triggers/etc. Note that `sql` column being null is
	# due to indexes being auto created based on the class definition (e.g.
	# PRIMARY KEY), and so don't need to be recreated.
	txn.execute(
	"""
	SELECT sql FROM sqlite_master
	WHERE tbl_name = ? AND type != 'table' AND sql IS NOT NULL
	""",

[MadLittleMods]

(approved in case you want to move on without #15751 (comment), we can iterate on that later)

[erikjohnston]

(approved in case you want to move on without #15751 (comment), we can iterate on that later)

Thanks! I think its good enough for now, as the biggest footgun was forgetting to port over some indexes, which is harder to do now.