Skip to content

2.23.1.1501-b6

Summary:
The output of ysql_dumpall is already being generated by pg_upgrade. However, the call to prepare_new_globals() in pg_upgrade.c:main() is #ifdef'd out. Remove this call from the #ifdef to apply the global dump during the upgrade, before upgrading databases.

Add a basic upgrade test for globals (roles, role memberships, and tablespaces), which requires fixing the process of turning on per-database catalog version mode in the current upgrade-from version, 2.20.2.2. Now, we first make the call to yb_fix_catalog_version_table(true), and then we restart the cluster in per-db catalog version mode. Before this fix, creating role memberships has an error caused by per-database catalog version mode not being fully set up if it's run quickly after the fix function has executed, as happens in the test script.

Also, since the test expects permission errors for some cases, adjust the ysqlsh function in common.sh so that the caller can see output from stderr, and compare it in line with other expected output.

Adding the call to prepare_new_globals() exposes a couple of bugs:
 1) YugabyteDB doesn't use the install superuser the same way that PostgreSQL does. PostgreSQL expects the operator to initdb the initial database with a particular install superuser that's given hardcoded OID 10, and then to run initdb for the upgrade target, and run pg_upgrade to the upgrade target, all using this same user. YugabyteDB takes the default postgres user as OID 10 and then creates its own superuser called yugabyte, which is used for most operations. PostgreSQL's expectation during pg_dumpall (our ysql_dumpall) is that it can skip dumping the current user, OID 10, because this is created by the new version's initdb. In our case, because postgres user was created as OID 10 but we don't use this mechanism to skip it, explicitly skip creation of user postgres in ysql_dumpall (in pg_dumpall.c).
 2) Similar to how PostgreSQL creates some builtin roles beginning with "pg_" in the bootstrap process in pg_authid.dat, YugabyteDB creates some of its own roles there, and those roles begin with "yb_". Apply similar logic to PostgreSQL to skip creating those users in the invocation of ysql_dumpall, because they're created by the new version's initdb.

Test Plan:
Jenkins

On MacOS 14 arm64:
  ./yb_build.sh release --sj
  pg15_tests/get_shell_test_specs.sh | grep upgrade | pg15_tests/run_tests.sh
  ./yb_build.sh fastdebug --sj
  pg15_tests/get_shell_test_specs.sh | grep upgrade | pg15_tests/run_tests.sh

Reviewers: jason

Reviewed By: jason

Subscribers: smishra, yql

Differential Revision: https://phorge.dev.yugabyte.com/D37873
Assets 2
Loading