Vertical split testcase migrated in go #42

saurabh408 · 2019-12-12T14:02:28Z

Vertical split testcases migrated in go.

Signed-off-by: saurabh [email protected]

* pick latest fix from cluster_for_test Signed-off-by: Arindam Nayak <[email protected]>

* tabletmanager lock unlock table test case Signed-off-by: Ajeet jain <[email protected]>

* converted vtctld_test.py to go Signed-off-by: Arindam Nayak <[email protected]>

* migrated one of testcase from schema.py to schema_test.go Signed-off-by: Arindam Nayak <[email protected]>

* converted tabletmanager test cases to go Signed-off-by: Ajeet jain <[email protected]>

* ported testcase of keyspace_test.py Signed-off-by: Arindam Nayak <[email protected]>

* Converted sharded test from py to go Signed-off-by: Arindam Nayak <[email protected]>

Signed-off-by: Arindam Nayak <[email protected]>

* pick latest fix from cluster_for_test Signed-off-by: Arindam Nayak <[email protected]>

* tabletmanager lock unlock table test case Signed-off-by: Ajeet jain <[email protected]>

* converted vtctld_test.py to go Signed-off-by: Arindam Nayak <[email protected]>

* migrated one of testcase from schema.py to schema_test.go Signed-off-by: Arindam Nayak <[email protected]>

* converted tabletmanager test cases to go Signed-off-by: Ajeet jain <[email protected]>

* ported testcase of keyspace_test.py Signed-off-by: Arindam Nayak <[email protected]>

* Converted sharded test from py to go Signed-off-by: Arindam Nayak <[email protected]>

Signed-off-by: saurabh <[email protected]>

Signed-off-by: Arindam Nayak <[email protected]>

Signed-off-by: saurabh <[email protected]>

* moved sql start to non-blocking mode Signed-off-by: Arindam Nayak <[email protected]> Signed-off-by: Arindam Nayak <[email protected]>

Signed-off-by: Arindam Nayak <[email protected]>

Signed-off-by: Ajeet jain <[email protected]> * readme for go endtoend test cases Signed-off-by: Ajeet jain <[email protected]> * Update README.md

Signed-off-by: Arindam Nayak <[email protected]>

Signed-off-by: saurabh <[email protected]>

saurabh408 · 2019-12-12T15:47:34Z

@deepthi @ajeetj one of the assertion i.e https://github.com/planetscale/vitess/blob/tal_vsplit_tests/go/test/endtoend/sharding/verticalsplit/vertical_split_test.go#L554 is failing.I am looking into it.

Signed-off-by: saurabh <[email protected]>

saurabh408 · 2019-12-13T05:40:36Z

@deepthi @ajeetj one of the assertion i.e https://github.com/planetscale/vitess/blob/tal_vsplit_tests/go/test/endtoend/sharding/verticalsplit/vertical_split_test.go#L554 is failing.I am looking into it.

All assertions are now working.

ajeetj

MInor comments, RLGTM

ajeetj · 2019-12-16T10:06:05Z

go/test/endtoend/cluster/vtgate_process.go

+	resultMap := make(map[string]interface{})
+	resp, err := http.Get(vtgate.VerifyURL)
+	if err != nil {
+		return nil


Should atleast log the error as we are not throwing it.

Done. Refer PR46

ajeetj · 2019-12-16T10:06:28Z

go/test/endtoend/cluster/vtgate_process.go

+		respByte, _ := ioutil.ReadAll(resp.Body)
+		err := json.Unmarshal(respByte, &resultMap)
+		if err != nil {
+			panic(err)


Let's not do panic, log error and return nil

Done. Refer PR46

ajeetj · 2019-12-16T10:08:18Z

go/test/endtoend/sharding/verticalsplit/vertical_split_test.go

+
+func TestVerticalSplit(t *testing.T) {
+	// setting RBR mode
+	os.Setenv("EXTRA_MY_CNF", path.Join(os.Getenv("VTROOT"), "config", "mycnf", "rbr.cnf"))


Please verify, after using the latest code we should not need this one.

Done. Refer PR46

ajeetj · 2019-12-16T10:10:23Z

go/test/endtoend/sharding/verticalsplit/vertical_split_test.go

+	err = clusterInstance.TopoProcess.ManageTopoDir("mkdir", "/vitess/"+"test_ca")
+	err = clusterInstance.VtctlProcess.AddCellInfo("test_ca")
+	err = clusterInstance.TopoProcess.ManageTopoDir("mkdir", "/vitess/"+"test_ny")
+	err = clusterInstance.VtctlProcess.AddCellInfo("test_ny")


should do assert.Nil(t.err) for all the above statements

Done. Refer PR46

ajeetj · 2019-12-16T10:11:35Z

go/test/endtoend/sharding/verticalsplit/vertical_split_test.go

+
+	// rdOnly_2 tablet init
+	_ = clusterInstance.VtctlclientProcess.InitTablet(&sourceRdOnlyTablet2, cellj, sourceKeyspace, hostname, sourceShard.Name)
+	_ = sourceRdOnlyTablet2.VttabletProcess.CreateDB(sourceKeyspace)


catch and assert error for all the above

you can also loop over a list of tablets as that the only parameter that is changing.

Done. Refer PR46

ajeetj · 2019-12-16T10:23:32Z

go/test/endtoend/sharding/verticalsplit/vertical_split_test.go

+	if err != nil {
+		t.Fatal(err)
+	}


Done. Refer PR46

ajeetj · 2019-12-16T10:24:44Z

go/test/endtoend/sharding/verticalsplit/vertical_split_test.go

+		_ = clusterInstance.VtctlclientProcess.ExecuteCommand("ApplySchema", "--sql=drop table "+t, "source_keyspace")
+	}
+	for _, t := range []cluster.Vttablet{sourceMasterTablet, sourceReplicaTablet, sourceRdOnlyTablet1, sourceRdOnlyTablet2} {
+		_ = clusterInstance.VtctlclientProcess.ExecuteCommand("ReloadSchema", t.Alias)


catch the error, and assert Nil for above 2 places.

Done. Refer PR46

ajeetj · 2019-12-16T10:25:19Z

go/test/endtoend/sharding/verticalsplit/vertical_split_test.go

+	resultCountStr := fmt.Sprintf("%v", reflect.ValueOf(resultTableMap["Count"]))
+	assert.Equal(t, "2", resultCountStr, fmt.Sprintf("unexpected value for VttabletCall(Execute.source_keyspace.0.replica) inside %s", resultCountStr))
+
+	// getting ['VtgateApi']['Histograms']['ExecuteKeyRanges.destination_keyspace.master']['Count']


remove unrequired code

Done. Refer PR46

ajeetj · 2019-12-16T10:25:24Z

go/test/endtoend/sharding/verticalsplit/vertical_split_test.go

+
+func checkStats(t *testing.T) {
+
+	//getting ['VttabletCall']['Histograms']['Execute.source_keyspace.0.replica']['Count']


remove unrequired code

Done. Refer PR46

ajeetj · 2019-12-16T10:27:49Z

go/test/endtoend/sharding/verticalsplit/vertical_split_test.go

+			tabletUID := clusterInstance.GetAndReserveTabletUID()
+			tablet := &cluster.Vttablet{
+				TabletUID: tabletUID,
+				HTTPPort:  clusterInstance.GetAndReservePort(),
+				GrpcPort:  clusterInstance.GetAndReservePort(),
+				MySQLPort: clusterInstance.GetAndReservePort(),
+				Alias:     fmt.Sprintf("%s-%010d", clusterInstance.Cell, tabletUID),
+			}
+			if i == 0 { // Make the first one as master
+				tablet.Type = "replica"
+			} else if i == 1 {
+				tablet.Type = "replica"
+			} else {
+				tablet.Type = "rdonly"
+			}


We can use vttablet.GetVttabletInstance

Done. Refer PR46

deepthi

@ajeetj seems to have already done a comprehensive review. I have just a couple of nits to add to that.

deepthi · 2019-12-16T21:06:37Z

go/test/endtoend/cluster/vtgate_process.go

@@ -131,7 +132,7 @@ func (vtgate *VtgateProcess) WaitForStatus() bool {
 }

 // GetStatusForTabletOfShard function gets status for a specific tablet of a shard in keyspace
-func (vtgate *VtgateProcess) GetStatusForTabletOfShard(name string) bool {
+func (vtgate *VtgateProcess) GetStatusForTabletOfShard(name string, count int) bool {


what is this count for? it is not obvious.

count is "how many endpoints to wait for"
Since there are 2 rdOnly tablets we need to wait for 2(count) endpoints.

then this should be WaitFor... instead of GetStatusFor..

GetStatusForTabletOfShard(vtgate_process.go) is called mostly in conjuction with WaitForStatusOfTabletInShard (vtgate_process.go)) where wait for endpoint is mentioned. I will make a update comment in GetStatusForTabletOfShard to label it as : number of endpoints.
Done. Refer PR46

deepthi · 2019-12-16T21:08:38Z

go/test/endtoend/sharding/verticalsplit/vertical_split_test.go

+	_ = clusterInstance.VtctlclientProcess.InitTablet(&sourceReplicaTablet, cellj, sourceKeyspace, hostname, sourceShard.Name)
+	_ = sourceReplicaTablet.VttabletProcess.CreateDB(sourceKeyspace)
+
+	// rdOnly_1 tablet init


in vitess lingo rdonly is one word. Let's also stop using _, that is python style

Done. Refer PR46

Signed-off-by: saurabh <[email protected]>

saurabh408 · 2019-12-19T05:09:42Z

closing it as new branch created with review comments addressed.

arindamnayak and others added 30 commits October 31, 2019 12:17

Rebase test_master with latest Cluster code (#22)

d538d86

* pick latest fix from cluster_for_test Signed-off-by: Arindam Nayak <[email protected]>

Tabletmanager2 test cases in GO using cluster (#23)

f638869

* tabletmanager lock unlock table test case Signed-off-by: Ajeet jain <[email protected]>

Vtctld Test cases in Go using cluster (#25)

ae542e3

* converted vtctld_test.py to go Signed-off-by: Arindam Nayak <[email protected]>

Converted schema.py testcase (#26)

c8dac4a

* migrated one of testcase from schema.py to schema_test.go Signed-off-by: Arindam Nayak <[email protected]>

Tablet Manager test cases in Go using cluster (#27)

3dc75a9

* converted tabletmanager test cases to go Signed-off-by: Ajeet jain <[email protected]>

Converted keyspace test to go (#28)

c99d4e1

* ported testcase of keyspace_test.py Signed-off-by: Arindam Nayak <[email protected]>

Converted sharded test from py to go (#29)

98a5e40

* Converted sharded test from py to go Signed-off-by: Arindam Nayak <[email protected]>

initial commit for sharding test

aa3b68b

Signed-off-by: Arindam Nayak <[email protected]>

added reconciliation assert

0f90bf9

Signed-off-by: Arindam Nayak <[email protected]>

added insert and stream replication

ed5a789

Signed-off-by: Arindam Nayak <[email protected]>

added rest assertions including migrateservedtypes

8cec35b

Signed-off-by: Arindam Nayak <[email protected]>

completed remaining assertions and fixed non-workings

c73595f

Signed-off-by: Arindam Nayak <[email protected]>

converted to v3 way of resharding

a48bb48

Signed-off-by: Arindam Nayak <[email protected]>

restructured the testcases

586ba3b

Signed-off-by: Arindam Nayak <[email protected]>

added testcase with bytes as sharding key type

52e88d9

Signed-off-by: Arindam Nayak <[email protected]>

added v3 sharding test

4fb8a33

Signed-off-by: Arindam Nayak <[email protected]>

used rbr mysql mode

8e83cd5

Signed-off-by: Arindam Nayak <[email protected]>

Rebase test_master with latest Cluster code (#22)

fc1a866

* pick latest fix from cluster_for_test Signed-off-by: Arindam Nayak <[email protected]>

Tabletmanager2 test cases in GO using cluster (#23)

62be70d

* tabletmanager lock unlock table test case Signed-off-by: Ajeet jain <[email protected]>

Vtctld Test cases in Go using cluster (#25)

11e9809

* converted vtctld_test.py to go Signed-off-by: Arindam Nayak <[email protected]>

Converted schema.py testcase (#26)

20d7016

* migrated one of testcase from schema.py to schema_test.go Signed-off-by: Arindam Nayak <[email protected]>

Tablet Manager test cases in Go using cluster (#27)

291566d

* converted tabletmanager test cases to go Signed-off-by: Ajeet jain <[email protected]>

Converted keyspace test to go (#28)

23a0872

* ported testcase of keyspace_test.py Signed-off-by: Arindam Nayak <[email protected]>

Converted sharded test from py to go (#29)

5f08311

* Converted sharded test from py to go Signed-off-by: Arindam Nayak <[email protected]>

Added testcase for mysqlctl process

55d1cd5

Signed-off-by: saurabh <[email protected]>

removed unused variables

0681679

Signed-off-by: saurabh <[email protected]>

removing comment

333f424

Signed-off-by: saurabh <[email protected]>

addressed review comments

dfd992e

Signed-off-by: saurabh <[email protected]>

removed cleandir flag from vttablet teardown

848c29c

Signed-off-by: Arindam Nayak <[email protected]>

vtgate buffer testcase migrated in go

4277b59

Signed-off-by: saurabh <[email protected]>

arindamnayak and others added 16 commits December 6, 2019 16:04

moved sql start to non-blocking mode (#35)

a13e72f

* moved sql start to non-blocking mode Signed-off-by: Arindam Nayak <[email protected]> Signed-off-by: Arindam Nayak <[email protected]>

added testcase for vtgate vars

ee21e42

Signed-off-by: Arindam Nayak <[email protected]>

redirect process errors to log files (#38)

69e9b3d

Signed-off-by: Arindam Nayak <[email protected]>

debug the test error in ci

42c4b84

Signed-off-by: Arindam Nayak <[email protected]>

fix initial sharding multi

91212cb

Signed-off-by: Arindam Nayak <[email protected]>

Merge branch 'tal_test_master' into tal_test_initial_sharding_v3

5a88206

fixed protected mysqlctl teardown

99f1b28

Signed-off-by: Arindam Nayak <[email protected]>

refactor mysqlctl start and stop

19a5225

Signed-off-by: Arindam Nayak <[email protected]>

Fixed multisplitdiff command

fa5edf5

Signed-off-by: Arindam Nayak <[email protected]>

added log statement for error condition

db7c8d2

Signed-off-by: Arindam Nayak <[email protected]>

Ignoring Python tests which are moved to GO (#40)

76293e5

Signed-off-by: Ajeet jain <[email protected]> * readme for go endtoend test cases Signed-off-by: Ajeet jain <[email protected]> * Update README.md

Merge branch 'tal_test_master' into tal_test_initial_sharding_v3

b14ecc4

updated comments

ae8e4c0

Signed-off-by: Arindam Nayak <[email protected]>

Refactor code

c2633e4

Signed-off-by: Arindam Nayak <[email protected]>

Merge branch 'tal_test_master' into tal_test_initial_sharding_v3

c7b2f3c

Signed-off-by: Arindam Nayak <[email protected]>

vertical split tests migrated in go

840de68

Signed-off-by: saurabh <[email protected]>

saurabh408 requested review from deepthi and ajeetj December 12, 2019 14:02

saurabh408 requested a review from sougou as a code owner December 12, 2019 14:02

fixed missed assertion

16f7eb4

Signed-off-by: saurabh <[email protected]>

ajeetj reviewed Dec 16, 2019

View reviewed changes

deepthi reviewed Dec 16, 2019

View reviewed changes

addressed review comments

6ce07fa

Signed-off-by: saurabh <[email protected]>

saurabh408 mentioned this pull request Dec 18, 2019

Vertical split testcase migrated in go #46

Merged

ajeetj changed the base branch from tal_test_initial_sharding_v3 to tal_test_master December 19, 2019 03:13

saurabh408 closed this Dec 19, 2019

saurabh408 deleted the tal_vsplit_tests branch December 20, 2019 05:05

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Vertical split testcase migrated in go #42

Vertical split testcase migrated in go #42

saurabh408 commented Dec 12, 2019

saurabh408 commented Dec 12, 2019

saurabh408 commented Dec 13, 2019

ajeetj left a comment

ajeetj Dec 16, 2019

saurabh408 Dec 18, 2019 •

edited

Loading

ajeetj Dec 16, 2019

saurabh408 Dec 18, 2019 •

edited

Loading

ajeetj Dec 16, 2019

saurabh408 Dec 18, 2019 •

edited

Loading

ajeetj Dec 16, 2019

saurabh408 Dec 18, 2019 •

edited

Loading

ajeetj Dec 16, 2019

saurabh408 Dec 18, 2019 •

edited

Loading

ajeetj Dec 16, 2019

saurabh408 Dec 18, 2019 •

edited

Loading

ajeetj Dec 16, 2019

saurabh408 Dec 18, 2019

ajeetj Dec 16, 2019

saurabh408 Dec 18, 2019

ajeetj Dec 16, 2019

saurabh408 Dec 18, 2019

ajeetj Dec 16, 2019

saurabh408 Dec 18, 2019

deepthi left a comment

deepthi Dec 16, 2019

saurabh408 Dec 17, 2019

deepthi Dec 18, 2019

saurabh408 Dec 19, 2019

deepthi Dec 16, 2019

saurabh408 Dec 18, 2019

saurabh408 commented Dec 19, 2019


		func checkStats(t *testing.T) {

		//getting ['VttabletCall']['Histograms']['Execute.source_keyspace.0.replica']['Count']

Vertical split testcase migrated in go #42

Vertical split testcase migrated in go #42

Conversation

saurabh408 commented Dec 12, 2019

saurabh408 commented Dec 12, 2019

saurabh408 commented Dec 13, 2019

ajeetj left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

saurabh408 Dec 18, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

saurabh408 Dec 18, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

saurabh408 Dec 18, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

saurabh408 Dec 18, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

saurabh408 Dec 18, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

saurabh408 Dec 18, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

deepthi left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

saurabh408 commented Dec 19, 2019

saurabh408 Dec 18, 2019 •

edited

Loading

saurabh408 Dec 18, 2019 •

edited

Loading

saurabh408 Dec 18, 2019 •

edited

Loading

saurabh408 Dec 18, 2019 •

edited

Loading

saurabh408 Dec 18, 2019 •

edited

Loading

saurabh408 Dec 18, 2019 •

edited

Loading