Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor vitess/lite Dockerfiles. #5592

Merged
merged 1 commit into from
Dec 19, 2019

Conversation

enisoc
Copy link
Member

@enisoc enisoc commented Dec 19, 2019

We no longer build off of vitess/base, which fixes the problem of automated vitess/lite builds sometimes having the incorrect binary versions. Note that we still have that problem for the docker/k8s family of images.

I also tried to pull common logic out of the various Dockerfiles into scripts and make rules to make it easier to keep them in sync with each other going forward.

Lastly, this adds XtraBackup to all flavors that support it.

cc @aquarapid

Copy link
Contributor

@morgo morgo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Member

@deepthi deepthi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Nicely done!

@enisoc enisoc merged commit 7a49ea3 into vitessio:master Dec 19, 2019
@enisoc enisoc deleted the docker-lite-refactor branch December 19, 2019 17:41
ajeetj added a commit to planetscale/vitess that referenced this pull request Dec 20, 2019
* Rebase test_master with latest Cluster code (#22)

* pick latest fix from cluster_for_test

Signed-off-by: Arindam Nayak <[email protected]>

* Tabletmanager2 test cases in GO using cluster (#23)

* tabletmanager lock unlock table test case

Signed-off-by: Ajeet jain <[email protected]>

* Vtctld Test cases in Go using cluster (#25)

* converted vtctld_test.py to go

Signed-off-by: Arindam Nayak <[email protected]>

* Converted schema.py testcase (#26)

* migrated one of testcase from schema.py to schema_test.go

Signed-off-by: Arindam Nayak <[email protected]>

* Tablet Manager test cases in Go using cluster (#27)

* converted tabletmanager test cases to go

Signed-off-by: Ajeet jain <[email protected]>

* Converted keyspace test to go (#28)

* ported testcase of keyspace_test.py

Signed-off-by: Arindam Nayak <[email protected]>

* Converted sharded test from py to go (#29)

* Converted sharded test from py to go

Signed-off-by: Arindam Nayak <[email protected]>

* initial commit for sharding test

Signed-off-by: Arindam Nayak <[email protected]>

* added reconciliation assert

Signed-off-by: Arindam Nayak <[email protected]>

* added insert and stream replication

Signed-off-by: Arindam Nayak <[email protected]>

* added rest assertions including migrateservedtypes

Signed-off-by: Arindam Nayak <[email protected]>

* completed remaining assertions and fixed non-workings

Signed-off-by: Arindam Nayak <[email protected]>

* converted to v3 way of resharding

Signed-off-by: Arindam Nayak <[email protected]>

* restructured the testcases

Signed-off-by: Arindam Nayak <[email protected]>

* added testcase with bytes as sharding key type

Signed-off-by: Arindam Nayak <[email protected]>

* Rebase test_master with latest Cluster code (#22)

* pick latest fix from cluster_for_test

Signed-off-by: Arindam Nayak <[email protected]>

* Tabletmanager2 test cases in GO using cluster (#23)

* tabletmanager lock unlock table test case

Signed-off-by: Ajeet jain <[email protected]>

* Vtctld Test cases in Go using cluster (#25)

* converted vtctld_test.py to go

Signed-off-by: Arindam Nayak <[email protected]>

* Converted schema.py testcase (#26)

* migrated one of testcase from schema.py to schema_test.go

Signed-off-by: Arindam Nayak <[email protected]>

* Tablet Manager test cases in Go using cluster (#27)

* converted tabletmanager test cases to go

Signed-off-by: Ajeet jain <[email protected]>

* Converted keyspace test to go (#28)

* ported testcase of keyspace_test.py

Signed-off-by: Arindam Nayak <[email protected]>

* Converted sharded test from py to go (#29)

* Converted sharded test from py to go

Signed-off-by: Arindam Nayak <[email protected]>

* Added testcase for mysqlctl process

Signed-off-by: saurabh <[email protected]>

* removed unused variables

Signed-off-by: saurabh <[email protected]>

* removing comment

Signed-off-by: saurabh <[email protected]>

* addressed review comments

Signed-off-by: saurabh <[email protected]>

* removed cleandir flag from vttablet teardown

Signed-off-by: Arindam Nayak <[email protected]>

* added v3 sharding test

Signed-off-by: Arindam Nayak <[email protected]>

* vtgate buffer testcase migrated in go

Signed-off-by: saurabh <[email protected]>

* inital code for resharding

Signed-off-by: Ajeet jain <[email protected]>

* completed the cluster code

Signed-off-by: Ajeet jain <[email protected]>

* more progress

Signed-off-by: Ajeet jain <[email protected]>

* pull graceful shutdown code

Signed-off-by: Ajeet jain <[email protected]>

* added fe more cases in resharding_test.go

Signed-off-by: Ajeet jain <[email protected]>

* fixed merge issues

Signed-off-by: Ajeet jain <[email protected]>

* running mysql in parallel mode and fe more assertions

Signed-off-by: Ajeet jain <[email protected]>

* used rbr mysql mode

Signed-off-by: Arindam Nayak <[email protected]>

* almost finished with reshading v2 version

Signed-off-by: Ajeet jain <[email protected]>

* Rebase test_master with latest Cluster code (#22)

* pick latest fix from cluster_for_test

Signed-off-by: Arindam Nayak <[email protected]>

* Tabletmanager2 test cases in GO using cluster (#23)

* tabletmanager lock unlock table test case

Signed-off-by: Ajeet jain <[email protected]>

* Vtctld Test cases in Go using cluster (#25)

* converted vtctld_test.py to go

Signed-off-by: Arindam Nayak <[email protected]>

* Converted schema.py testcase (#26)

* migrated one of testcase from schema.py to schema_test.go

Signed-off-by: Arindam Nayak <[email protected]>

* Tablet Manager test cases in Go using cluster (#27)

* converted tabletmanager test cases to go

Signed-off-by: Ajeet jain <[email protected]>

* Converted keyspace test to go (#28)

* ported testcase of keyspace_test.py

Signed-off-by: Arindam Nayak <[email protected]>

* Converted sharded test from py to go (#29)

* Converted sharded test from py to go

Signed-off-by: Arindam Nayak <[email protected]>

* Added testcase for mysqlctl process

Signed-off-by: saurabh <[email protected]>

* removed unused variables

Signed-off-by: saurabh <[email protected]>

* removing comment

Signed-off-by: saurabh <[email protected]>

* addressed review comments

Signed-off-by: saurabh <[email protected]>

* removed cleandir flag from vttablet teardown

Signed-off-by: Arindam Nayak <[email protected]>

* vtgate buffer testcase migrated in go

Signed-off-by: saurabh <[email protected]>

* changed code to use v3 way of sharding

Signed-off-by: Arindam Nayak <[email protected]>

* converted to v3 way

Signed-off-by: Ajeet jain <[email protected]>

* added multi split diff and multi sharding

Signed-off-by: Arindam Nayak <[email protected]>

* converted to v3 way

Signed-off-by: Ajeet jain <[email protected]>

* TabletManager remaining test cases

TabletManager remaining test cases in GO
Signed-off-by: Ajeet jain <[email protected]>

* moved sql start to non-blocking mode (#35)

* moved sql start to non-blocking mode

Signed-off-by: Arindam Nayak <[email protected]>

* added testcase for vtgate vars

Signed-off-by: Arindam Nayak <[email protected]>

* implement pr suggestion

Signed-off-by: Arindam Nayak <[email protected]>

* trying to work with throttler

Signed-off-by: Ajeet jain <[email protected]>

* use ks1 mysql ports for other tablets in ks2

Signed-off-by: Arindam Nayak <[email protected]>

* converted on int keys

Signed-off-by: Ajeet jain <[email protected]>

* Rebase test_master with latest Cluster code (#22)

* pick latest fix from cluster_for_test

Signed-off-by: Arindam Nayak <[email protected]>
Signed-off-by: Arindam Nayak <[email protected]>

* Tabletmanager2 test cases in GO using cluster (#23)

* tabletmanager lock unlock table test case

Signed-off-by: Ajeet jain <[email protected]>

Signed-off-by: Arindam Nayak <[email protected]>

* Vtctld Test cases in Go using cluster (#25)

* converted vtctld_test.py to go

Signed-off-by: Arindam Nayak <[email protected]>
Signed-off-by: Arindam Nayak <[email protected]>

* Converted schema.py testcase (#26)

* migrated one of testcase from schema.py to schema_test.go

Signed-off-by: Arindam Nayak <[email protected]>
Signed-off-by: Arindam Nayak <[email protected]>

* Tablet Manager test cases in Go using cluster (#27)

* converted tabletmanager test cases to go

Signed-off-by: Ajeet jain <[email protected]>

Signed-off-by: Arindam Nayak <[email protected]>

* Converted keyspace test to go (#28)

* ported testcase of keyspace_test.py

Signed-off-by: Arindam Nayak <[email protected]>
Signed-off-by: Arindam Nayak <[email protected]>

* Converted sharded test from py to go (#29)

* Converted sharded test from py to go

Signed-off-by: Arindam Nayak <[email protected]>
Signed-off-by: Arindam Nayak <[email protected]>

* Added testcase for mysqlctl process

Signed-off-by: saurabh <[email protected]>
Signed-off-by: Arindam Nayak <[email protected]>

* removed unused variables

Signed-off-by: saurabh <[email protected]>
Signed-off-by: Arindam Nayak <[email protected]>

* removing comment

Signed-off-by: saurabh <[email protected]>
Signed-off-by: Arindam Nayak <[email protected]>

* addressed review comments

Signed-off-by: saurabh <[email protected]>
Signed-off-by: Arindam Nayak <[email protected]>

* removed cleandir flag from vttablet teardown

Signed-off-by: Arindam Nayak <[email protected]>

* vtgate buffer testcase migrated in go

Signed-off-by: saurabh <[email protected]>
Signed-off-by: Arindam Nayak <[email protected]>

* TabletManager remaining test cases

TabletManager remaining test cases in GO
Signed-off-by: Ajeet jain <[email protected]>
Signed-off-by: Arindam Nayak <[email protected]>

* moved sql start to non-blocking mode (#35)

* moved sql start to non-blocking mode

Signed-off-by: Arindam Nayak <[email protected]>
Signed-off-by: Arindam Nayak <[email protected]>

* added testcase for vtgate vars

Signed-off-by: Arindam Nayak <[email protected]>

* redirect process errors to log files (#38)

Signed-off-by: Arindam Nayak <[email protected]>

* debug the test error in ci

Signed-off-by: Arindam Nayak <[email protected]>

* fix initial sharding multi

Signed-off-by: Arindam Nayak <[email protected]>

* fixed protected mysqlctl teardown

Signed-off-by: Arindam Nayak <[email protected]>

* removed sleep time

Signed-off-by: Ajeet jain <[email protected]>

* refactor mysqlctl start and stop

Signed-off-by: Arindam Nayak <[email protected]>

* Fixed multisplitdiff command

Signed-off-by: Arindam Nayak <[email protected]>

* added log statement for error condition

Signed-off-by: Arindam Nayak <[email protected]>

* Ignoring Python tests which are moved to GO (#40)

Signed-off-by: Ajeet jain <[email protected]>

* readme for go endtoend test cases

Signed-off-by: Ajeet jain <[email protected]>

* Update README.md

* Fixed byte column test cases

Signed-off-by: Ajeet jain <[email protected]>

* lint fixes

Signed-off-by: Ajeet jain <[email protected]>

* updated comments

Signed-off-by: Arindam Nayak <[email protected]>

* restructure of launch cluster

Signed-off-by: Ajeet jain <[email protected]>

* fix tear down code

Signed-off-by: Ajeet jain <[email protected]>

* Refactor code

Signed-off-by: Arindam Nayak <[email protected]>

* fix the throttler test cases

Signed-off-by: Ajeet jain <[email protected]>

* fix the static checks

Signed-off-by: Ajeet jain <[email protected]>

* disable running python test cases in travis

Signed-off-by: Ajeet jain <[email protected]>

* fixed static checks

Signed-off-by: Ajeet jain <[email protected]>

* removed unrequired line to set RBR mode

Signed-off-by: Ajeet jain <[email protected]>

* Add matrix build for mysql

Signed-off-by: Morgan Tocker <[email protected]>

* Add additional flavors, move flaky test

Signed-off-by: Morgan Tocker <[email protected]>

* vindex: add required NeedVCursor function

There's currently no way to know if a vindex requires the vcursor.
We need this knowledge to prevent vreplication from choosing
a vindex that requires a VCursor. This change requires all vindexes
to report if they need the vcursor or not.

Signed-off-by: Sugu Sougoumarane <[email protected]>

* vrepl: choose best vindex instead of primary

Signed-off-by: Sugu Sougoumarane <[email protected]>

* few of the reparent test cases

Signed-off-by: Ajeet jain <[email protected]>

* vttablet-up.sh : Adds detection of mysqld launch failure.  Given failure, tests for further for          presence of apparmor.  Exits with error and recommendation to disable.

Signed-off-by: ChrisR <[email protected]>

* completed all the reparent tests

Signed-off-by: Ajeet jain <[email protected]>

* Remove MariaDB 10.1

Re-enable unit race
Change CI git checkout to be v2.0
Disable test caching

Signed-off-by: Morgan Tocker <[email protected]>

* Revert some of the MariaDB changes, fix unit_race

Signed-off-by: Morgan Tocker <[email protected]>

* Add debugging for macOS issue

Signed-off-by: Morgan Tocker <[email protected]>

* Add additional flaky test

Signed-off-by: Morgan Tocker <[email protected]>

* Add sleep for macOS

Signed-off-by: Morgan Tocker <[email protected]>

* Disable macOS local example for now (flaky)

Signed-off-by: Morgan Tocker <[email protected]>

* Disable unit_race again

Signed-off-by: Morgan Tocker <[email protected]>

* updated test cases for reparent

Signed-off-by: Ajeet jain <[email protected]>

* Resharding endtoend testcases in Go migrated from Python (#44)


Signed-off-by: Ajeet jain <[email protected]>

* fix vtgate schema testcases

Signed-off-by: Ajeet jain <[email protected]>

* Convert binlog.py to go version (#41)


Signed-off-by: Arindam Nayak <[email protected]>

* fixed range based reparenting

Signed-off-by: Ajeet jain <[email protected]>

* removed unrequired comments

Signed-off-by: Ajeet jain <[email protected]>

* Modified apparmor detection to passive, added additional investigation info.

Signed-off-by: ChrisR <[email protected]>

* make-release-packages.sh: Disable shellcheck 1091 and Add README pointing to local install instructions

Signed-off-by: Gary Edgar <[email protected]>

* review comments addressed

Signed-off-by: Ajeet jain <[email protected]>

* Splitting github endtoend action into 2 files

Signed-off-by: Ajeet jain <[email protected]>

* vindex: NeedVCursor->NeedsVCursor

Signed-off-by: Sugu Sougoumarane <[email protected]>

* Set correct tablet type to fix failing test

Signed-off-by: Arindam Nayak <[email protected]>

* Fixing initial sharding multi

Signed-off-by: Arindam Nayak <[email protected]>

* fix the assertions

Signed-off-by: Arindam Nayak <[email protected]>

* fixed master-spare test case

Signed-off-by: Ajeet jain <[email protected]>

* Testing dockerized test in github workflow

Signed-off-by: Arindam Nayak <[email protected]>

* Vertical split testcase migrated in go (#46)

Vertical split endtoend testcases in Go migrated from Python

Signed-off-by: saurabh <[email protected]>

* Adjusted shard number for testcase

Signed-off-by: Arindam Nayak <[email protected]>

* Reverting changes for config.json

Signed-off-by: Arindam Nayak <[email protected]>

* Refactor vitess/lite Dockerfiles. (vitessio#5592)

Signed-off-by: Anthony Yeh <[email protected]>

Co-authored-by: Arindam Nayak <[email protected]>
Co-authored-by: saurabh408 <[email protected]>
Co-authored-by: Morgan Tocker <[email protected]>
Co-authored-by: Sugu Sougoumarane <[email protected]>
Co-authored-by: ChrisR <[email protected]>
Co-authored-by: Gary Edgar <[email protected]>
Co-authored-by: Anthony Yeh <[email protected]>
Co-authored-by: Deepthi Sigireddi <[email protected]>
systay pushed a commit that referenced this pull request Jul 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants