*: write binlog to unix socket. #1660

coocood · 2016-08-30T08:47:29Z

No description provided.

coocood · 2016-08-30T08:48:44Z

@shenli @disksing @iamxy @qiuyesuifeng PTAL

disksing · 2016-08-30T08:54:08Z

Do we need a switch to turn on/off this feature? Not all user need it.

disksing · 2016-08-30T09:04:04Z

infoschema/infoschema.go

 	"github.com/pingcap/tidb/table"
 	"github.com/pingcap/tidb/terror"
 	// import table implementation to init table.TableFromMeta
+	"github.com/pingcap/tidb/perfschema"


Any special reason to move the import?

coocood · 2016-08-31T03:59:43Z

@disksing @hanfei1991 PTAL

zimulala · 2016-08-31T09:44:53Z

table/tables/tables.go

@@ -515,7 +525,38 @@ func (t *Table) RemoveRecord(ctx context.Context, h int64, r []types.Datum) erro
 	if err != nil {
 		return errors.Trace(err)
 	}
-
+	if binloginfo.Enable {


Here can use a function.

shenli · 2016-09-10T02:12:56Z

sessionctx/binloginfo/binloginfo.go

+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package binloginfo


How about rename it to binlog?

Then the package name will conflict with tipb/go-binlog.

shenli · 2016-09-10T02:18:13Z

For same statements, we use another sql to do some sub jobs. For example, set global variable, it will run an update statement. Do we need to avoid add an update binlog when set global variable?

coocood · 2016-09-13T09:11:15Z

@iamxy @shenli @zimulala PTAL

coocood · 2016-09-18T07:05:25Z

@iamxy @shenli @zimulala PTAL

zimulala · 2016-09-18T08:53:46Z

session.go

@@ -184,18 +185,29 @@ func (s *session) finishTxn(rollback bool) error {
 	if s.txn == nil {
 		return nil
 	}
+	sessVar := variable.GetSessionVars(s)


sessVar seems useless.

zimulala · 2016-09-18T09:13:40Z

store/tikv/txn_committer.go

+		req := &binlog.WriteBinlogReq{ClusterID: binloginfo.ClusterID, Payload: rollbackData}
+		resp, err := binloginfo.PumpClient.WriteBinlog(context.Background(), req)
+		if err != nil {
+			log.Error(err)


Add more log information. The same as the line 431.

zimulala · 2016-09-18T09:17:10Z

store/tikv/txn_committer.go

+				err = errors.New(resp.Errmsg)
+			}
+		}
+		ch <- err


err -> errors.Trace(err) ?

coocood · 2016-09-20T05:02:32Z

@shenli @zimulala PTAL

shenli

Rest LGTM

shenli · 2016-09-20T07:27:28Z

store/tikv/txn_committer.go

+}
+
+func (c *txnCommitter) writeCommitBinlog() {
+	if binloginfo.PumpClient == nil {


Use shouldWriteBinlog?

shenli · 2016-09-20T07:27:37Z

store/tikv/txn_committer.go

+}
+
+func (c *txnCommitter) rollbackBinlog() {
+	if binloginfo.PumpClient == nil {


shouldWriteBinlog?

shenli · 2016-09-20T07:27:46Z

store/tikv/txn_committer.go

 	return nil
 }

+func (c *txnCommitter) prewriteBinlog() chan error {
+	if binloginfo.PumpClient == nil {


shouldWriteBinlog?

support write binlog to a unix socket through grpc.

zimulala · 2016-09-20T10:20:24Z

tidb-server/main.go

+			log.Fatal(errors.ErrorStack(err1))
+		}
+		binloginfo.ClusterID = id
+	}


Can line 101-112 be extract into a function？I think simpler and shorter code in the function of main is better. The same as line 122-130.

zimulala · 2016-09-20T10:22:13Z

tidb-server/main.go

+	if err != nil {
+		log.Fatal(errors.ErrorStack(err))
+	}
+	if cluster := u.Query().Get("cluster"); cluster != "" {


If cluster == "" it has some problems?

zimulala · 2016-09-20T10:55:51Z

store/tikv/txn_committer.go

 			return errors.Trace(err)
 		}
 		log.Warnf("txn commit succeed with error: %v, tid: %d", err, c.startTS)
 	}
+	c.writeCommitBinlog()


Why not put c.writeCommitBinlog() and c.rollbackBinlog() in tikvTxn.Commit()?

zimulala · 2016-09-21T03:07:10Z

store/tikv/txn_committer.go

+				PrewriteKey:   c.keys[0],
+				PrewriteValue: prewriteValue.([]byte),
+			}
+			prewriteData, _ := binPrewrite.Marshal()


Check the error. The same as line 418 and line 442

zimulala · 2016-09-21T03:12:21Z

store/tikv/txn_committer.go

+	go func() {
+		prewriteValue := c.txn.us.GetOption(kv.BinlogData)
+		var err error
+		if prewriteValue != nil {


If the return value of c.shouldWriteBinlog() is ture, the value of prewriteValue != nil also is true.

zimulala · 2016-09-21T03:14:43Z

store/tikv/txn_committer.go

+			}
+			prewriteData, _ := binPrewrite.Marshal()
+			req := &binlog.WriteBinlogReq{ClusterID: binloginfo.ClusterID, Payload: prewriteData}
+			var resp *binlog.WriteBinlogResp


This line can be removed.

zimulala · 2016-09-21T04:55:29Z

LGTM

disksing reviewed Aug 30, 2016
View reviewed changes

zimulala reviewed Aug 31, 2016
View reviewed changes

coocood changed the title *: save binlog when commit [DNM]*: save binlog when commit Sep 1, 2016

shenli reviewed Sep 10, 2016
View reviewed changes

coocood force-pushed the coocood/add-binlog-event branch from f4c066f to 1fdbfa5 Compare September 13, 2016 09:09

coocood changed the title ~~[DNM]*: save binlog when commit~~ *: write binlog to unix socket. Sep 13, 2016

coocood force-pushed the coocood/add-binlog-event branch from a62b19f to 17a03f2 Compare September 18, 2016 03:25

coocood force-pushed the coocood/add-binlog-event branch from 78c7fa8 to 6c68530 Compare September 18, 2016 07:56

zimulala reviewed Sep 18, 2016

View reviewed changes

shenli reviewed Sep 20, 2016

View reviewed changes

*: write binlog

af7b814

support write binlog to a unix socket through grpc.

coocood force-pushed the coocood/add-binlog-event branch from 482b454 to af7b814 Compare September 20, 2016 10:22

zimulala reviewed Sep 20, 2016

View reviewed changes

*: address comment

5cd4845

coocood force-pushed the coocood/add-binlog-event branch from def9152 to 5cd4845 Compare September 20, 2016 11:07

zimulala reviewed Sep 21, 2016

View reviewed changes

*: address comment

6bd0747

Merge branch 'master' into coocood/add-binlog-event

7d97d73

coocood merged commit 1dd9e89 into master Sep 21, 2016

coocood deleted the coocood/add-binlog-event branch September 21, 2016 05:09

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

*: write binlog to unix socket. #1660

*: write binlog to unix socket. #1660

coocood commented Aug 30, 2016

coocood commented Aug 30, 2016

disksing commented Aug 30, 2016

disksing Aug 30, 2016

coocood commented Aug 31, 2016

zimulala Aug 31, 2016

shenli Sep 10, 2016

coocood Sep 18, 2016

shenli commented Sep 10, 2016

coocood commented Sep 13, 2016

coocood commented Sep 18, 2016

zimulala Sep 18, 2016

zimulala Sep 18, 2016 •

edited

Loading

zimulala Sep 18, 2016

coocood commented Sep 20, 2016

shenli left a comment

shenli Sep 20, 2016

shenli Sep 20, 2016

shenli Sep 20, 2016

zimulala Sep 20, 2016 •

edited

Loading

zimulala Sep 20, 2016

zimulala Sep 20, 2016 •

edited

Loading

zimulala Sep 21, 2016 •

edited

Loading

zimulala Sep 21, 2016

zimulala Sep 21, 2016

zimulala commented Sep 21, 2016

*: write binlog to unix socket. #1660

*: write binlog to unix socket. #1660

Conversation

coocood commented Aug 30, 2016

coocood commented Aug 30, 2016

disksing commented Aug 30, 2016

Choose a reason for hiding this comment

coocood commented Aug 31, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shenli commented Sep 10, 2016

coocood commented Sep 13, 2016

coocood commented Sep 18, 2016

Choose a reason for hiding this comment

zimulala Sep 18, 2016 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

coocood commented Sep 20, 2016

shenli left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zimulala Sep 20, 2016 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zimulala Sep 20, 2016 • edited Loading

Choose a reason for hiding this comment

zimulala Sep 21, 2016 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zimulala commented Sep 21, 2016

zimulala Sep 18, 2016 •

edited

Loading

zimulala Sep 20, 2016 •

edited

Loading

zimulala Sep 20, 2016 •

edited

Loading

zimulala Sep 21, 2016 •

edited

Loading