roachtest: systemschema/validate-after-version-upgrade failed

[cockroach-teamcity]

roachtest.systemschema/validate-after-version-upgrade failed with artifacts on release-23.1 @ 3551dc089ac3efcb3772df2109164dcb01e79d8f:

test artifacts and logs in: /artifacts/systemschema/validate-after-version-upgrade/run_1
(validate_system_schema_after_version_upgrade.go:99).1: After upgrading, ``USE system; SHOW CREATE ALL TABLES;`` does not match expected output after version upgrade.
Diff:
@@ -340,11 +340,18 @@
 	INDEX execution_count_idx (aggregated_ts ASC, app_name ASC, execution_count DESC) WHERE app_name NOT LIKE '$ internal%':::STRING,
 	INDEX service_latency_idx (aggregated_ts ASC, app_name ASC, service_latency DESC) WHERE app_name NOT LIKE '$ internal%':::STRING,
 	INDEX cpu_sql_nanos_idx (aggregated_ts ASC, app_name ASC, cpu_sql_nanos DESC) WHERE app_name NOT LIKE '$ internal%':::STRING,
 	INDEX contention_time_idx (aggregated_ts ASC, app_name ASC, contention_time DESC) WHERE app_name NOT LIKE '$ internal%':::STRING,
 	INDEX total_estimated_execution_time_idx (aggregated_ts ASC, app_name ASC, total_estimated_execution_time DESC) WHERE app_name NOT LIKE '$ internal%':::STRING,
-	INDEX p99_latency_idx (aggregated_ts ASC, app_name ASC, p99_latency DESC) WHERE app_name NOT LIKE '$ internal%':::STRING
+	INDEX p99_latency_idx (aggregated_ts ASC, app_name ASC, p99_latency DESC) WHERE app_name NOT LIKE '$ internal%':::STRING,
+	FAMILY "primary" (crdb_internal_aggregated_ts_app_name_fingerprint_id_node_id_plan_hash_transaction_fingerprint_id_shard_8, aggregated_ts, fingerprint_id, transaction_fingerprint_id, plan_hash, app_name, node_id, agg_interval, metadata, statistics, plan, index_recommendations),
+	FAMILY fam_14_execution_count (execution_count),
+	FAMILY fam_15_service_latency (service_latency),
+	FAMILY fam_16_cpu_sql_nanos (cpu_sql_nanos),
+	FAMILY fam_17_contention_time (contention_time),
+	FAMILY fam_18_total_estimated_execution_time (total_estimated_execution_time),
+	FAMILY fam_19_p99_latency (p99_latency)
 );
 CREATE TABLE public.transaction_statistics (
 	aggregated_ts TIMESTAMPTZ NOT NULL,
 	fingerprint_id BYTES NOT NULL,
 	app_name STRING NOT NULL,
@@ -364,11 +371,18 @@
 	INDEX execution_count_idx (aggregated_ts ASC, app_name ASC, execution_count DESC) WHERE app_name NOT LIKE '$ internal%':::STRING,
 	INDEX service_latency_idx (aggregated_ts ASC, app_name ASC, service_latency DESC) WHERE app_name NOT LIKE '$ internal%':::STRING,
 	INDEX cpu_sql_nanos_idx (aggregated_ts ASC, app_name ASC, cpu_sql_nanos DESC) WHERE app_name NOT LIKE '$ internal%':::STRING,
 	INDEX contention_time_idx (aggregated_ts ASC, app_name ASC, contention_time DESC) WHERE app_name NOT LIKE '$ internal%':::STRING,
 	INDEX total_estimated_execution_time_idx (aggregated_ts ASC, app_name ASC, total_estimated_execution_time DESC) WHERE app_name NOT LIKE '$ internal%':::STRING,
-	INDEX p99_latency_idx (aggregated_ts ASC, app_name ASC, p99_latency DESC) WHERE app_name NOT LIKE '$ internal%':::STRING
+	INDEX p99_latency_idx (aggregated_ts ASC, app_name ASC, p99_latency DESC) WHERE app_name NOT LIKE '$ internal%':::STRING,
+	FAMILY "primary" (crdb_internal_aggregated_ts_app_name_fingerprint_id_node_id_shard_8, aggregated_ts, fingerprint_id, app_name, node_id, agg_interval, metadata, statistics),
+	FAMILY fam_9_execution_count (execution_count),
+	FAMILY fam_10_service_latency (service_latency),
+	FAMILY fam_11_cpu_sql_nanos (cpu_sql_nanos),
+	FAMILY fam_12_contention_time (contention_time),
+	FAMILY fam_13_total_estimated_execution_time (total_estimated_execution_time),
+	FAMILY fam_14_p99_latency (p99_latency)
 );
 CREATE TABLE public.database_role_settings (
 	database_id OID NOT NULL,
 	role_name STRING NOT NULL,
 	settings STRING[] NOT NULL,

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=4 , ROACHTEST_encrypted=false , ROACHTEST_fs=ext4 , ROACHTEST_localSSD=true , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

Jira issue: CRDB-26100

[ajwerner]

@ericharmeling didn't you just do something here?

[ericharmeling]

didn't you just do something here?

Yes. This should be fixed in bb11b30.

[ericharmeling]

I'm going to close this for now, as the commit that failed here (3551dc0) does not include the changes in #99800, which I've verified two ways should resolve this failure.