Skip to content
This repository has been archived by the owner on Nov 24, 2023. It is now read-only.

config: support setting session variable #687

Merged
merged 8 commits into from
May 29, 2020
Merged

config: support setting session variable #687

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
3f3ee25
add session config
GMHDBJD May 27, 2020
cd3bc52
change session config to map
GMHDBJD May 27, 2020
2ad6fc6
change to DSN
GMHDBJD May 27, 2020
36e8103
format dsn
GMHDBJD May 28, 2020
9ec51cb
add integration_test for session config
GMHDBJD May 28, 2020
5d57a5b
add comment
GMHDBJD May 28, 2020
839be69
Merge branch 'master' into support-session
GMHDBJD May 28, 2020
1a08eee
Merge branch 'master' into support-session
GMHDBJD May 28, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 6 additions & 5 deletions dm/config/subtask.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,11 +81,12 @@ func (c *RawDBConfig) SetMaxIdleConns(value int) *RawDBConfig {

// DBConfig is the DB configuration.
type DBConfig struct {
Host string `toml:"host" json:"host" yaml:"host"`
Port int `toml:"port" json:"port" yaml:"port"`
User string `toml:"user" json:"user" yaml:"user"`
Password string `toml:"password" json:"-" yaml:"password"` // omit it for privacy
MaxAllowedPacket *int `toml:"max-allowed-packet" json:"max-allowed-packet" yaml:"max-allowed-packet"`
Host string `toml:"host" json:"host" yaml:"host"`
Port int `toml:"port" json:"port" yaml:"port"`
User string `toml:"user" json:"user" yaml:"user"`
Password string `toml:"password" json:"-" yaml:"password"` // omit it for privacy
MaxAllowedPacket *int `toml:"max-allowed-packet" json:"max-allowed-packet" yaml:"max-allowed-packet"`
Session map[string]string `toml:"session" json:"session" yaml:"session"`

RawDBCfg *RawDBConfig `toml:"-" json:"-" yaml:"-"`
}
Expand Down
6 changes: 6 additions & 0 deletions pkg/conn/basedb.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,10 +56,16 @@ func (d *DefaultDBProviderImpl) Apply(config config.DBConfig) (*BaseDB, error) {
}
maxIdleConns = rawCfg.MaxIdleConns
}

for key, val := range config.Session {
dsn += fmt.Sprintf("&%s=\"%s\"", key, val)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to use different formats for types of val? see https://github.com/go-sql-driver/mysql#system-variables

Copy link
Collaborator Author

@GMHDBJD GMHDBJD May 28, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because we don't restrict user config, it's hard to judge the type of val.
I use ' for all types and url.QueryEscape to escape string in 36e8103. MySQL extracts the relevant type from a string.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can it work if add "" for number value?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes，see

dm/tests/all_mode/conf/dm-task.yaml

Line 20 in 9ec51cb

tidb_retry_limit: "10"

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MySQL extracts the relevant type from a string.

Are there any docs/links explain about this?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems not official. go-sql-driver/mysql#405 (comment)

Copy link
Member

@csuzhangxc csuzhangxc May 28, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 you can add this as a comment above the code (the link in the about comment has out of date because it's not a permanent link).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add commet in 5d57a5b

}

db, err := sql.Open("mysql", dsn)
if err != nil {
return nil, terror.DBErrorAdapt(err, terror.ErrDBDriverError)
}

db.SetMaxIdleConns(maxIdleConns)

return NewBaseDB(db), nil
Expand Down
5 changes: 5 additions & 0 deletions tests/all_mode/conf/dm-task.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,11 @@ target-database:
port: 4000
user: "root"
password: ""
session:
sql_mode: "ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION"
tidb_skip_utf8_check: 1
tidb_disable_txn_auto_retry: off
tidb_retry_limit: "10"
csuzhangxc marked this conversation as resolved.
Show resolved Hide resolved

mysql-instances:
- source-id: "mysql-replica-01"
Expand Down