Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add MySQL 8 Support to Backup Tests #10691

Merged
merged 27 commits into from
Jul 20, 2022

Conversation

mattlord
Copy link
Contributor

@mattlord mattlord commented Jul 13, 2022

Description

This PR does the following:

  1. Modifies the MySQL 8.0 CI workflow template to use the Percona apt repo instead of the MySQL one
    • This then aligns the mysqld and xtrabackup versions
    • xtrabackup 8.0.28 is latest and does not support mysqld 8.0.29, which is the latest server version upstream
  2. Adds explicit MySQL 8.0 and XtraBackup 8.0 support to the relevant endtoend backup tests
  3. Creates a new (xb_backup) mysql80 workflow to test XtraBackup 8.0 with MySQL 8.0
  4. Renames the test shard/workflow "19" to the more meaningful "vtbackup_transform"
  5. Modifies how the xtrabackup related tests run with 5.7 so that the only tests done (not skipped) are the tar stream specific ones
    • Meaning that the xbstream and vtctlbackup tests are only done with 8.0
    • This is because while XtraBackup 2.4 defaults to using the tar stream method, XtraBackup 8.0 does not have a tar stream method, only xbstream

Related Issue(s)

Checklist

  • "Backport me!" label has been added if this change should be backported
  • Tests were added or are not required
  • Documentation was added or is not required

@vitess-bot
Copy link
Contributor

vitess-bot bot commented Jul 13, 2022

Review Checklist

Hello reviewers! 👋 Please follow this checklist when reviewing this Pull Request.

General

  • Ensure that the Pull Request has a descriptive title.
  • If this is a change that users need to know about, please apply the release notes (needs details) label so that merging is blocked unless the summary release notes document is included.
  • If a new flag is being introduced, review whether it is really needed. The flag names should be clear and intuitive (as far as possible), and the flag's help should be descriptive. Additionally, flag names should use dashes (-) as word separators rather than underscores (_).
  • If a workflow is added or modified, each items in Jobs should be named in order to mark it as required. If the workflow should be required, the GitHub Admin should be notified.

Bug fixes

  • There should be at least one unit or end-to-end test.
  • The Pull Request description should either include a link to an issue that describes the bug OR an actual description of the bug and how to reproduce, along with a description of the fix.

Non-trivial changes

  • There should be some code comments as to why things are implemented the way they are.

New/Existing features

  • Should be documented, either by modifying the existing documentation or creating new documentation.
  • New features should have a link to a feature request issue or an RFC that documents the use cases, corner cases and test cases.

Backward compatibility

  • Protobuf changes should be wire-compatible.
  • Changes to _vt tables and RPCs need to be backward compatible.
  • vtctl command output order should be stable and awk-able.

Signed-off-by: Matt Lord <[email protected]>
@mattlord mattlord added Component: Build/CI release notes (needs details) This PR needs to be listed in the release notes in a dedicated section (deprecation notice, etc...) Type: Internal Cleanup Type: CI/Build and removed Type: Internal Cleanup labels Jul 13, 2022
Signed-off-by: Matt Lord <[email protected]>
@mattlord mattlord removed the release notes (needs details) This PR needs to be listed in the release notes in a dedicated section (deprecation notice, etc...) label Jul 13, 2022
This needed to be done after the shard was setup in order to satisfy
the semantic assumptions related to semi-sync in the tests.

Signed-off-by: Matt Lord <[email protected]>
Signed-off-by: Matt Lord <[email protected]>
@mattlord mattlord changed the base branch from main to release-14.0 July 13, 2022 22:41
@mattlord mattlord changed the base branch from release-14.0 to main July 13, 2022 22:45
@mattlord mattlord changed the title Add xb_backup mysql 8 workflow with new mysql 8 backup test support Add MySQL 8 support to backup tests Jul 14, 2022
Signed-off-by: Matt Lord <[email protected]>
@rsajwani
Copy link
Contributor

rsajwani commented Jul 14, 2022

changes looks good. These are precisely all the issues I encounter on my local box. Thanks for fixing them.

@mattlord mattlord marked this pull request as ready for review July 14, 2022 23:48
@mattlord mattlord changed the title Add MySQL 8 support to backup tests Add MySQL 8 Support to Backup Tests Jul 15, 2022
Copy link
Member

@deepthi deepthi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work @mattlord, thank you for taking this on.
I'm a little iffy about changing ALL 8.0 tests to use percona. I would prefer to do that only for the xtrabackup test clusters.

@@ -58,7 +58,7 @@ func printContraction1(g *codegen.Generator, wa *weightarray, incont []uca.Contr
trie[r] = append(trie[r], cont)
}
if depth == len(cont.Path) {
leaf = &cont
leaf = &cont // nolint:exportloopref
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was fixed in #10765, so we should remove the nolint directive.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! I'll remove and rebase.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work @mattlord, thank you for taking this on.
I'm a little iffy about changing ALL 8.0 tests to use percona. I would prefer to do that only for the xtrabackup test clusters.

I don't see a problem with it. Percona 8.0 has little changes beyond upstream as most of the patches that were in 5.7 and before (especially 5.6 and 5.5) are now no longer relevant. I also like that we're testing what I think many of our users are using. But I understand the concern.

I can make this change in only the XtraBackup workflows though. I'll just need to create a separate template for them and handle them specially in the generator. If you feel strongly about it then I'll do that.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In that case, this LGTM

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's a patch we can leverage if we do decide to change this decision in the future and limit Percona Server 8.0 usage to XtraBackup tests: https://gist.github.com/mattlord/341fc54e4fa3f36d11bd03d8bdf57124

@deepthi deepthi merged commit bf13249 into vitessio:main Jul 20, 2022
@deepthi deepthi deleted the xb_backup_e2e_mysql8 branch July 20, 2022 19:54
timvaillancourt pushed a commit to slackhq/vitess that referenced this pull request Aug 17, 2023
* Add support for MySQL 8.0 in backup tests

Signed-off-by: Matt Lord <[email protected]>

* Add 8.0 workflow

Signed-off-by: Matt Lord <[email protected]>

* whitespace

Signed-off-by: Matt Lord <[email protected]>

* Use vtctldclient SetKeyspaceDurabilityPolicy to manage semi-sync

This needed to be done after the shard was setup in order to satisfy
the semantic assumptions related to semi-sync in the tests.

Signed-off-by: Matt Lord <[email protected]>

* Remove extraneous changes

Signed-off-by: Matt Lord <[email protected]>

* We need lz4 for TestXtrabackupStreamWithlz4Compression

Signed-off-by: Matt Lord <[email protected]>

* Try using Percona Repo for MySQL 8 to align mysqld and xtrabackup versions

Signed-off-by: Matt Lord <[email protected]>

* Specify stream type everywhere

Signed-off-by: Matt Lord <[email protected]>

* Remove repeated server install

Signed-off-by: Matt Lord <[email protected]>

* Moar...

Signed-off-by: Matt Lord <[email protected]>

* Move vtctlbackup test to 8.0

Signed-off-by: Matt Lord <[email protected]>

* Rename vtbackup test and move to MySQL 8

Signed-off-by: Matt Lord <[email protected]>

* Split the xbstream tests so the workflow doesn't time out

Otherwise it was going over the 10min limit and getting
killed.

Signed-off-by: Matt Lord <[email protected]>

* Use MySQL 8 compat method for setting passwords

Signed-off-by: Matt Lord <[email protected]>

* Test increasing timeout at another level

Signed-off-by: Matt Lord <[email protected]>

* Don't use the init passwords file with 8.0

mysqlctl doesn't start...

This is likely due to the change in initialization behavior with
MySQL 8.0 as it goes though two phases and you can't simply
start up mysqld and pass it data, it has to initialize and restart
first.

Signed-off-by: Matt Lord <[email protected]>

* Fix incorrect password update statement for vt_repl user

Signed-off-by: Rohit Nayak <[email protected]>

* Bump timeouts for 8.0 backup tests

Signed-off-by: Matt Lord <[email protected]>

* Bump it more :(

Signed-off-by: Matt Lord <[email protected]>

* Increase backup/restore timeout in backup_utils

Signed-off-by: Matt Lord <[email protected]>

* Apply new 8.0 template everywhere

Signed-off-by: Matt Lord <[email protected]>

* Fix bugs around how the compression flags were getting passed

Signed-off-by: Matt Lord <[email protected]>

* Use 45m timeout for the workflow, 30m for the run.

Signed-off-by: Matt Lord <[email protected]>

* These changes were no longer needed so limiting diff.

Signed-off-by: Matt Lord <[email protected]>

* Explicitly skip new linter check

Signed-off-by: Matt Lord <[email protected]>

Co-authored-by: Rohit Nayak <[email protected]>
timvaillancourt added a commit to slackhq/vitess that referenced this pull request Aug 30, 2023
…test + compressors (#119)

* Delete all legacy sharding related code (vitessio#10278)

* Delete all legacy sharding related code

Signed-off-by: Matt Lord <[email protected]>

* Move used until functions from initialsharding to cluster

Signed-off-by: Matt Lord <[email protected]>

* Remove vtctl commands

Signed-off-by: Matt Lord <[email protected]>

* Kill vtworker and SetKeyspaceServedFrom cmd

Signed-off-by: Matt Lord <[email protected]>

* WaitForDrain related stragglers

Signed-off-by: Matt Lord <[email protected]>

* Legacy local straggler workflow

Signed-off-by: Matt Lord <[email protected]>

* Get rid of SetKeyspaceShardingInfo & wait for drain stragglers

Signed-off-by: Matt Lord <[email protected]>

* Remove vtworker stragglers

Signed-off-by: Matt Lord <[email protected]>

* Update throttlerservice protobuf

Signed-off-by: Matt Lord <[email protected]>

* Rename test 24, add hashicorp vault test to it (now mysql_server_vault)

Signed-off-by: Matt Lord <[email protected]>

* Remove last mentiones of legacy sharding in vtctl

Signed-off-by: Matt Lord <[email protected]>

* remove binlog_use_v3_resharding_mode

Signed-off-by: Matt Lord <[email protected]>

* Address review comments

Signed-off-by: Matt Lord <[email protected]>

* Address review comments

Signed-off-by: Matt Lord <[email protected]>

* Correct vtgate help output

Signed-off-by: Matt Lord <[email protected]>

* go fmt

Signed-off-by: Tim Vaillancourt <[email protected]>

* Remove v2 resharding fields (vitessio#10409)

* cleanup: remove sharding_column_name and sharding_column_type

Signed-off-by: Arvind Murty <[email protected]>

* cleanup: remove sharding_column_name and sharding_column_type

Signed-off-by: Arvind Murty <[email protected]>

* cleanup: remove sharding_column_name and sharding_column_type

Signed-off-by: Arvind Murty <[email protected]>

* cleanup: remove sharding_column_name and sharding_column_type

Signed-off-by: Arvind Murty <[email protected]>

* generate vtadmin files

Signed-off-by: Arvind Murty <[email protected]>

* cleanup: remove sharding_column_name and sharding_column_type from vtadmin

Signed-off-by: Arvind Murty <[email protected]>

* Merge from main

Signed-off-by: Arvind Murty <[email protected]>

* Fix bad merge conflict resolution

Signed-off-by: Tim Vaillancourt <[email protected]>

* Fix missing 'sharding' import

Signed-off-by: Tim Vaillancourt <[email protected]>

* Fix bad conflict resolution in go/test/endtoend/cellalias/cell_alias_test.go

Signed-off-by: Tim Vaillancourt <[email protected]>

* Backup/Restore: add support for external compressors and decompressors (vitessio#10558)

* change to support an external decompressor

Signed-off-by: Renan Rangel <[email protected]>
Signed-off-by: Rameez Sajwani <[email protected]>

* add external compressor support + builtin additional compressors

Signed-off-by: Renan Rangel <[email protected]>
Signed-off-by: Rameez Sajwani <[email protected]>

* wrap external compressor/decompressor

Signed-off-by: Renan Rangel <[email protected]>
Signed-off-by: Rameez Sajwani <[email protected]>

* go mod tidy + comments

Signed-off-by: Renan Rangel <[email protected]>
Signed-off-by: Rameez Sajwani <[email protected]>

* add copyright notices

Signed-off-by: Renan Rangel <[email protected]>
Signed-off-by: Rameez Sajwani <[email protected]>

* add support for builtin engine

Signed-off-by: Renan Rangel <[email protected]>
Signed-off-by: Rameez Sajwani <[email protected]>

* Adding test case for buckup compression

Signed-off-by: Rameez Sajwani <[email protected]>

* Fixing unit test and run mod tidy

Signed-off-by: Rameez Sajwani <[email protected]>

* Removing unwanted unit tests

Signed-off-by: Rameez Sajwani <[email protected]>

* Increase timeout of backup tests

Signed-off-by: Rameez Sajwani <[email protected]>

* fixing linter errors

Signed-off-by: Rameez Sajwani <[email protected]>

* Change test logic to accomodate running selective tests

Signed-off-by: Rameez Sajwani <[email protected]>

* removing lint warning

Signed-off-by: Rameez Sajwani <[email protected]>

* fixing test failure

Signed-off-by: Rameez Sajwani <[email protected]>

* Removing un-necessary test

Signed-off-by: Rameez Sajwani <[email protected]>

* Fixing code review feeback

Signed-off-by: Rameez Sajwani <[email protected]>

* Change builtinEngine to consider 'auto' decompressor

Signed-off-by: Rameez Sajwani <[email protected]>

* fixing Upgrade/Downgrade test

Signed-off-by: Rameez Sajwani <[email protected]>

* Fix type & add summary under release notes

Signed-off-by: Rameez Sajwani <[email protected]>

* Fixing typos in summary

Signed-off-by: Rameez Sajwani <[email protected]>

* Fixing flag name typos

Signed-off-by: Rameez Sajwani <[email protected]>

Co-authored-by: Renan Rangel <[email protected]>
Co-authored-by: Renan Rangel <[email protected]>

* Add MySQL 8 Support to Backup Tests (vitessio#10691)

* Add support for MySQL 8.0 in backup tests

Signed-off-by: Matt Lord <[email protected]>

* Add 8.0 workflow

Signed-off-by: Matt Lord <[email protected]>

* whitespace

Signed-off-by: Matt Lord <[email protected]>

* Use vtctldclient SetKeyspaceDurabilityPolicy to manage semi-sync

This needed to be done after the shard was setup in order to satisfy
the semantic assumptions related to semi-sync in the tests.

Signed-off-by: Matt Lord <[email protected]>

* Remove extraneous changes

Signed-off-by: Matt Lord <[email protected]>

* We need lz4 for TestXtrabackupStreamWithlz4Compression

Signed-off-by: Matt Lord <[email protected]>

* Try using Percona Repo for MySQL 8 to align mysqld and xtrabackup versions

Signed-off-by: Matt Lord <[email protected]>

* Specify stream type everywhere

Signed-off-by: Matt Lord <[email protected]>

* Remove repeated server install

Signed-off-by: Matt Lord <[email protected]>

* Moar...

Signed-off-by: Matt Lord <[email protected]>

* Move vtctlbackup test to 8.0

Signed-off-by: Matt Lord <[email protected]>

* Rename vtbackup test and move to MySQL 8

Signed-off-by: Matt Lord <[email protected]>

* Split the xbstream tests so the workflow doesn't time out

Otherwise it was going over the 10min limit and getting
killed.

Signed-off-by: Matt Lord <[email protected]>

* Use MySQL 8 compat method for setting passwords

Signed-off-by: Matt Lord <[email protected]>

* Test increasing timeout at another level

Signed-off-by: Matt Lord <[email protected]>

* Don't use the init passwords file with 8.0

mysqlctl doesn't start...

This is likely due to the change in initialization behavior with
MySQL 8.0 as it goes though two phases and you can't simply
start up mysqld and pass it data, it has to initialize and restart
first.

Signed-off-by: Matt Lord <[email protected]>

* Fix incorrect password update statement for vt_repl user

Signed-off-by: Rohit Nayak <[email protected]>

* Bump timeouts for 8.0 backup tests

Signed-off-by: Matt Lord <[email protected]>

* Bump it more :(

Signed-off-by: Matt Lord <[email protected]>

* Increase backup/restore timeout in backup_utils

Signed-off-by: Matt Lord <[email protected]>

* Apply new 8.0 template everywhere

Signed-off-by: Matt Lord <[email protected]>

* Fix bugs around how the compression flags were getting passed

Signed-off-by: Matt Lord <[email protected]>

* Use 45m timeout for the workflow, 30m for the run.

Signed-off-by: Matt Lord <[email protected]>

* These changes were no longer needed so limiting diff.

Signed-off-by: Matt Lord <[email protected]>

* Explicitly skip new linter check

Signed-off-by: Matt Lord <[email protected]>

Co-authored-by: Rohit Nayak <[email protected]>

* Fix test file merge issues

Signed-off-by: Tim Vaillancourt <[email protected]>

* Go 1.18.7 to fix error

Signed-off-by: Tim Vaillancourt <[email protected]>

* Fix lint

Signed-off-by: Tim Vaillancourt <[email protected]>

* Fix vttablet.txt

Signed-off-by: Tim Vaillancourt <[email protected]>

* Remove sharding_column_name and sharding_column_type from vtctld2 (vitessio#10459)

* Remove sharding_column_name and sharding_column_type

Signed-off-by: notfelineit <[email protected]>

* Run make web_build to compile production files

Signed-off-by: notfelineit <[email protected]>

* Remove vars from test

Signed-off-by: notfelineit <[email protected]>

* Remove sharding column type select box test from web test

Signed-off-by: notfelineit <[email protected]>

---------

Signed-off-by: Matt Lord <[email protected]>
Signed-off-by: Tim Vaillancourt <[email protected]>
Signed-off-by: Arvind Murty <[email protected]>
Signed-off-by: notfelineit <[email protected]>
Co-authored-by: Matt Lord <[email protected]>
Co-authored-by: Arvind Murty <[email protected]>
Co-authored-by: Rameez Sajwani <[email protected]>
Co-authored-by: Renan Rangel <[email protected]>
Co-authored-by: Renan Rangel <[email protected]>
Co-authored-by: Rohit Nayak <[email protected]>
Co-authored-by: Frances Thai <[email protected]>
timvaillancourt added a commit to slackhq/vitess that referenced this pull request Sep 1, 2023
…test + compressors (#119)

* Delete all legacy sharding related code (vitessio#10278)

* Delete all legacy sharding related code

Signed-off-by: Matt Lord <[email protected]>

* Move used until functions from initialsharding to cluster

Signed-off-by: Matt Lord <[email protected]>

* Remove vtctl commands

Signed-off-by: Matt Lord <[email protected]>

* Kill vtworker and SetKeyspaceServedFrom cmd

Signed-off-by: Matt Lord <[email protected]>

* WaitForDrain related stragglers

Signed-off-by: Matt Lord <[email protected]>

* Legacy local straggler workflow

Signed-off-by: Matt Lord <[email protected]>

* Get rid of SetKeyspaceShardingInfo & wait for drain stragglers

Signed-off-by: Matt Lord <[email protected]>

* Remove vtworker stragglers

Signed-off-by: Matt Lord <[email protected]>

* Update throttlerservice protobuf

Signed-off-by: Matt Lord <[email protected]>

* Rename test 24, add hashicorp vault test to it (now mysql_server_vault)

Signed-off-by: Matt Lord <[email protected]>

* Remove last mentiones of legacy sharding in vtctl

Signed-off-by: Matt Lord <[email protected]>

* remove binlog_use_v3_resharding_mode

Signed-off-by: Matt Lord <[email protected]>

* Address review comments

Signed-off-by: Matt Lord <[email protected]>

* Address review comments

Signed-off-by: Matt Lord <[email protected]>

* Correct vtgate help output

Signed-off-by: Matt Lord <[email protected]>

* go fmt

Signed-off-by: Tim Vaillancourt <[email protected]>

* Remove v2 resharding fields (vitessio#10409)

* cleanup: remove sharding_column_name and sharding_column_type

Signed-off-by: Arvind Murty <[email protected]>

* cleanup: remove sharding_column_name and sharding_column_type

Signed-off-by: Arvind Murty <[email protected]>

* cleanup: remove sharding_column_name and sharding_column_type

Signed-off-by: Arvind Murty <[email protected]>

* cleanup: remove sharding_column_name and sharding_column_type

Signed-off-by: Arvind Murty <[email protected]>

* generate vtadmin files

Signed-off-by: Arvind Murty <[email protected]>

* cleanup: remove sharding_column_name and sharding_column_type from vtadmin

Signed-off-by: Arvind Murty <[email protected]>

* Merge from main

Signed-off-by: Arvind Murty <[email protected]>

* Fix bad merge conflict resolution

Signed-off-by: Tim Vaillancourt <[email protected]>

* Fix missing 'sharding' import

Signed-off-by: Tim Vaillancourt <[email protected]>

* Fix bad conflict resolution in go/test/endtoend/cellalias/cell_alias_test.go

Signed-off-by: Tim Vaillancourt <[email protected]>

* Backup/Restore: add support for external compressors and decompressors (vitessio#10558)

* change to support an external decompressor

Signed-off-by: Renan Rangel <[email protected]>
Signed-off-by: Rameez Sajwani <[email protected]>

* add external compressor support + builtin additional compressors

Signed-off-by: Renan Rangel <[email protected]>
Signed-off-by: Rameez Sajwani <[email protected]>

* wrap external compressor/decompressor

Signed-off-by: Renan Rangel <[email protected]>
Signed-off-by: Rameez Sajwani <[email protected]>

* go mod tidy + comments

Signed-off-by: Renan Rangel <[email protected]>
Signed-off-by: Rameez Sajwani <[email protected]>

* add copyright notices

Signed-off-by: Renan Rangel <[email protected]>
Signed-off-by: Rameez Sajwani <[email protected]>

* add support for builtin engine

Signed-off-by: Renan Rangel <[email protected]>
Signed-off-by: Rameez Sajwani <[email protected]>

* Adding test case for buckup compression

Signed-off-by: Rameez Sajwani <[email protected]>

* Fixing unit test and run mod tidy

Signed-off-by: Rameez Sajwani <[email protected]>

* Removing unwanted unit tests

Signed-off-by: Rameez Sajwani <[email protected]>

* Increase timeout of backup tests

Signed-off-by: Rameez Sajwani <[email protected]>

* fixing linter errors

Signed-off-by: Rameez Sajwani <[email protected]>

* Change test logic to accomodate running selective tests

Signed-off-by: Rameez Sajwani <[email protected]>

* removing lint warning

Signed-off-by: Rameez Sajwani <[email protected]>

* fixing test failure

Signed-off-by: Rameez Sajwani <[email protected]>

* Removing un-necessary test

Signed-off-by: Rameez Sajwani <[email protected]>

* Fixing code review feeback

Signed-off-by: Rameez Sajwani <[email protected]>

* Change builtinEngine to consider 'auto' decompressor

Signed-off-by: Rameez Sajwani <[email protected]>

* fixing Upgrade/Downgrade test

Signed-off-by: Rameez Sajwani <[email protected]>

* Fix type & add summary under release notes

Signed-off-by: Rameez Sajwani <[email protected]>

* Fixing typos in summary

Signed-off-by: Rameez Sajwani <[email protected]>

* Fixing flag name typos

Signed-off-by: Rameez Sajwani <[email protected]>

Co-authored-by: Renan Rangel <[email protected]>
Co-authored-by: Renan Rangel <[email protected]>

* Add MySQL 8 Support to Backup Tests (vitessio#10691)

* Add support for MySQL 8.0 in backup tests

Signed-off-by: Matt Lord <[email protected]>

* Add 8.0 workflow

Signed-off-by: Matt Lord <[email protected]>

* whitespace

Signed-off-by: Matt Lord <[email protected]>

* Use vtctldclient SetKeyspaceDurabilityPolicy to manage semi-sync

This needed to be done after the shard was setup in order to satisfy
the semantic assumptions related to semi-sync in the tests.

Signed-off-by: Matt Lord <[email protected]>

* Remove extraneous changes

Signed-off-by: Matt Lord <[email protected]>

* We need lz4 for TestXtrabackupStreamWithlz4Compression

Signed-off-by: Matt Lord <[email protected]>

* Try using Percona Repo for MySQL 8 to align mysqld and xtrabackup versions

Signed-off-by: Matt Lord <[email protected]>

* Specify stream type everywhere

Signed-off-by: Matt Lord <[email protected]>

* Remove repeated server install

Signed-off-by: Matt Lord <[email protected]>

* Moar...

Signed-off-by: Matt Lord <[email protected]>

* Move vtctlbackup test to 8.0

Signed-off-by: Matt Lord <[email protected]>

* Rename vtbackup test and move to MySQL 8

Signed-off-by: Matt Lord <[email protected]>

* Split the xbstream tests so the workflow doesn't time out

Otherwise it was going over the 10min limit and getting
killed.

Signed-off-by: Matt Lord <[email protected]>

* Use MySQL 8 compat method for setting passwords

Signed-off-by: Matt Lord <[email protected]>

* Test increasing timeout at another level

Signed-off-by: Matt Lord <[email protected]>

* Don't use the init passwords file with 8.0

mysqlctl doesn't start...

This is likely due to the change in initialization behavior with
MySQL 8.0 as it goes though two phases and you can't simply
start up mysqld and pass it data, it has to initialize and restart
first.

Signed-off-by: Matt Lord <[email protected]>

* Fix incorrect password update statement for vt_repl user

Signed-off-by: Rohit Nayak <[email protected]>

* Bump timeouts for 8.0 backup tests

Signed-off-by: Matt Lord <[email protected]>

* Bump it more :(

Signed-off-by: Matt Lord <[email protected]>

* Increase backup/restore timeout in backup_utils

Signed-off-by: Matt Lord <[email protected]>

* Apply new 8.0 template everywhere

Signed-off-by: Matt Lord <[email protected]>

* Fix bugs around how the compression flags were getting passed

Signed-off-by: Matt Lord <[email protected]>

* Use 45m timeout for the workflow, 30m for the run.

Signed-off-by: Matt Lord <[email protected]>

* These changes were no longer needed so limiting diff.

Signed-off-by: Matt Lord <[email protected]>

* Explicitly skip new linter check

Signed-off-by: Matt Lord <[email protected]>

Co-authored-by: Rohit Nayak <[email protected]>

* Fix test file merge issues

Signed-off-by: Tim Vaillancourt <[email protected]>

* Go 1.18.7 to fix error

Signed-off-by: Tim Vaillancourt <[email protected]>

* Fix lint

Signed-off-by: Tim Vaillancourt <[email protected]>

* Fix vttablet.txt

Signed-off-by: Tim Vaillancourt <[email protected]>

* Remove sharding_column_name and sharding_column_type from vtctld2 (vitessio#10459)

* Remove sharding_column_name and sharding_column_type

Signed-off-by: notfelineit <[email protected]>

* Run make web_build to compile production files

Signed-off-by: notfelineit <[email protected]>

* Remove vars from test

Signed-off-by: notfelineit <[email protected]>

* Remove sharding column type select box test from web test

Signed-off-by: notfelineit <[email protected]>

---------

Signed-off-by: Matt Lord <[email protected]>
Signed-off-by: Tim Vaillancourt <[email protected]>
Signed-off-by: Arvind Murty <[email protected]>
Signed-off-by: notfelineit <[email protected]>
Co-authored-by: Matt Lord <[email protected]>
Co-authored-by: Arvind Murty <[email protected]>
Co-authored-by: Rameez Sajwani <[email protected]>
Co-authored-by: Renan Rangel <[email protected]>
Co-authored-by: Renan Rangel <[email protected]>
Co-authored-by: Rohit Nayak <[email protected]>
Co-authored-by: Frances Thai <[email protected]>
timvaillancourt added a commit to slackhq/vitess that referenced this pull request Sep 4, 2023
…test + compressors (#119) (#124)

* Delete all legacy sharding related code (vitessio#10278)

* Delete all legacy sharding related code



* Move used until functions from initialsharding to cluster



* Remove vtctl commands



* Kill vtworker and SetKeyspaceServedFrom cmd



* WaitForDrain related stragglers



* Legacy local straggler workflow



* Get rid of SetKeyspaceShardingInfo & wait for drain stragglers



* Remove vtworker stragglers



* Update throttlerservice protobuf



* Rename test 24, add hashicorp vault test to it (now mysql_server_vault)



* Remove last mentiones of legacy sharding in vtctl



* remove binlog_use_v3_resharding_mode



* Address review comments



* Address review comments



* Correct vtgate help output



* go fmt



* Remove v2 resharding fields (vitessio#10409)

* cleanup: remove sharding_column_name and sharding_column_type



* cleanup: remove sharding_column_name and sharding_column_type



* cleanup: remove sharding_column_name and sharding_column_type



* cleanup: remove sharding_column_name and sharding_column_type



* generate vtadmin files



* cleanup: remove sharding_column_name and sharding_column_type from vtadmin



* Merge from main



* Fix bad merge conflict resolution



* Fix missing 'sharding' import



* Fix bad conflict resolution in go/test/endtoend/cellalias/cell_alias_test.go



* Backup/Restore: add support for external compressors and decompressors (vitessio#10558)

* change to support an external decompressor




* add external compressor support + builtin additional compressors




* wrap external compressor/decompressor




* go mod tidy + comments




* add copyright notices




* add support for builtin engine




* Adding test case for buckup compression



* Fixing unit test and run mod tidy



* Removing unwanted unit tests



* Increase timeout of backup tests



* fixing linter errors



* Change test logic to accomodate running selective tests



* removing lint warning



* fixing test failure



* Removing un-necessary test



* Fixing code review feeback



* Change builtinEngine to consider 'auto' decompressor



* fixing Upgrade/Downgrade test



* Fix type & add summary under release notes



* Fixing typos in summary



* Fixing flag name typos






* Add MySQL 8 Support to Backup Tests (vitessio#10691)

* Add support for MySQL 8.0 in backup tests



* Add 8.0 workflow



* whitespace



* Use vtctldclient SetKeyspaceDurabilityPolicy to manage semi-sync

This needed to be done after the shard was setup in order to satisfy
the semantic assumptions related to semi-sync in the tests.



* Remove extraneous changes



* We need lz4 for TestXtrabackupStreamWithlz4Compression



* Try using Percona Repo for MySQL 8 to align mysqld and xtrabackup versions



* Specify stream type everywhere



* Remove repeated server install



* Moar...



* Move vtctlbackup test to 8.0



* Rename vtbackup test and move to MySQL 8



* Split the xbstream tests so the workflow doesn't time out

Otherwise it was going over the 10min limit and getting
killed.



* Use MySQL 8 compat method for setting passwords



* Test increasing timeout at another level



* Don't use the init passwords file with 8.0

mysqlctl doesn't start...

This is likely due to the change in initialization behavior with
MySQL 8.0 as it goes though two phases and you can't simply
start up mysqld and pass it data, it has to initialize and restart
first.



* Fix incorrect password update statement for vt_repl user



* Bump timeouts for 8.0 backup tests



* Bump it more :(



* Increase backup/restore timeout in backup_utils



* Apply new 8.0 template everywhere



* Fix bugs around how the compression flags were getting passed



* Use 45m timeout for the workflow, 30m for the run.



* These changes were no longer needed so limiting diff.



* Explicitly skip new linter check





* Fix test file merge issues



* Go 1.18.7 to fix error



* Fix lint



* Fix vttablet.txt



* Remove sharding_column_name and sharding_column_type from vtctld2 (vitessio#10459)

* Remove sharding_column_name and sharding_column_type



* Run make web_build to compile production files



* Remove vars from test



* Remove sharding column type select box test from web test



---------

Signed-off-by: Matt Lord <[email protected]>
Signed-off-by: Tim Vaillancourt <[email protected]>
Signed-off-by: Arvind Murty <[email protected]>
Signed-off-by: notfelineit <[email protected]>
Co-authored-by: Matt Lord <[email protected]>
Co-authored-by: Arvind Murty <[email protected]>
Co-authored-by: Rameez Sajwani <[email protected]>
Co-authored-by: Renan Rangel <[email protected]>
Co-authored-by: Renan Rangel <[email protected]>
Co-authored-by: Rohit Nayak <[email protected]>
Co-authored-by: Frances Thai <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants