Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make v3 discovery and etcdctl reuse the same ClientConfig #13745

Closed
wants to merge 1 commit into from

Conversation

ahrtr
Copy link
Member

@ahrtr ahrtr commented Feb 28, 2022

Part of 13624, and it's one of the follow-ups to PR 13635.

Move the original clientConfig from etcdctl to clientv3, so that it can be reused by both etcdctl and v3 discovery.

The original clientConfig has some dependencies on pks/cobrautil, such as (https://github.com/etcd-io/etcd/blob/main/etcdctl/ctlv3/command/global.go#L180). However, clientv3 isn't allowed to depend on pkg, see go.mod#L50. Actually the original pks/cobrautil is only used by etcdctl or etcdutl, so move it into the clientv3 package as well.

Note that etcd uses golang's standard package flag to process command line flags, while etcdctl uses github.com/spf13/pflag, and etcdctl and v3 discovery has different flag names, such as --endpoints vs --discovery-endpoints. So it's difficult to share the same code to process flags between them. So the command line flags keep unchanged in this PR.

cc @ptabor @serathius @spzala

@ahrtr ahrtr added this to the etcd-v3.6 milestone Feb 28, 2022
@codecov-commenter
Copy link

codecov-commenter commented Feb 28, 2022

Codecov Report

Merging #13745 (1f91c58) into main (fb55910) will decrease coverage by 0.12%.
The diff coverage is 44.60%.

❗ Current head 1f91c58 differs from pull request most recent head 35e2852. Consider uploading reports for the commit 35e2852 to get more accurate results

Impacted file tree graph

@@            Coverage Diff             @@
##             main   #13745      +/-   ##
==========================================
- Coverage   72.72%   72.59%   -0.13%     
==========================================
  Files         467      468       +1     
  Lines       38286    38309      +23     
==========================================
- Hits        27842    27811      -31     
- Misses       8635     8694      +59     
+ Partials     1809     1804       -5     
Flag Coverage Δ
all 72.59% <44.60%> (-0.13%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
client/v3/config.go 0.00% <0.00%> (ø)
etcdctl/ctlv2/command/cluster_health.go 0.00% <ø> (ø)
etcdctl/ctlv2/command/exec_watch_command.go 0.00% <ø> (ø)
etcdctl/ctlv2/command/get_command.go 0.00% <ø> (ø)
etcdctl/ctlv2/command/ls_command.go 0.00% <ø> (ø)
etcdctl/ctlv2/command/mk_command.go 0.00% <ø> (ø)
etcdctl/ctlv2/command/mkdir_command.go 0.00% <ø> (ø)
etcdctl/ctlv2/command/rm_command.go 0.00% <ø> (ø)
etcdctl/ctlv2/command/rmdir_command.go 0.00% <ø> (ø)
etcdctl/ctlv2/command/set_command.go 0.00% <ø> (ø)
... and 71 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fb55910...35e2852. Read the comment docs.

@ahrtr ahrtr force-pushed the reuse_client_config branch 2 times, most recently from 1a34acb to cb4fd9c Compare February 28, 2022 09:06
client/v3/config.go Outdated Show resolved Hide resolved
client/v3/config.go Outdated Show resolved Hide resolved
client/v3/config.go Outdated Show resolved Hide resolved
client/v3/config.go Outdated Show resolved Hide resolved
return client
}

func NewClientCfg(endpoints []string, dialTimeout, keepAliveTime, keepAliveTimeout time.Duration, scfg *SecureCfg, acfg *AuthCfg) (*Config, error) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would recommend constructor only be responsible for providing default values and not setting or validating. For configuration structs I expect values to be provided via json.Unmarshall or flags.VarString. Both of those functions take reference to struct/field making the constructor you wrote not very useful.

Example on how to define configuration (based on how K8s code does configuration):

type ClientConfig struct {
  DialTimeout time.Duration `json:"dial-timeout"`
}

func NewClientConfig() ClientConfig {
  return ClientConfig{
    DialTimeout: 5*time.Second,
  }
}

func (c ClientConfig) Validate() err {
  if c.DialTimeout < 0 {
    return fmt.Error("Timeout cannot be negative")
  }
  return nil
}

func (c ClientConfig) AddFlags(fs flags.FlagSet) {
  fs.VarString("dial-timeout", &c.DialTimeout, c.DialTimeout, "Dial timeout")
}

func main() {
  
  config := NewClientConfig()
  config.AddFlags(flags.CommandLine) // or something like
  err := config.Validate()
  if err != nil {
    fmt.Print(err)
    os.Exit(1)
  }
}

This code is very draftly, however shows overall direction. As for registering single ClientConfig as different flags we could use aliasing. For example:

func addClientConfigFlags(c ClientConfig, prefix string, fs flags.FlagSet) {
  fs.VarString(prefix + "-dial-timeout", &c.DialTimeout, c.DialTimeout, "Dial timeout")
}

type DiscoveryClientConfig ClientConfig
type EtcdctlClientConfig ClientConfig

func (c DialClientConfig) AddFlags(fs flags.FlagSet) {
  addClientConfigFlags(c, "discovery", fs)
}

func (c EtcdctlClientConfig) AddFlags(fs flags.FlagSet) {
  addClientConfigFlags(c, "client", fs)
}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just moved the function from etcdctl into clientv3, and exported the function. Apart from that, I did not change anything else. If we want to change anything else, I'd like to do it in a separate PR. WDYT?

@serathius
Copy link
Member

This PR is little hard to read due to lot of dependency changes. Could you please create a separate PR/commiut that just fixes the dependencies? It should be fast and easy to merge, allowing us to focus the discussion on the config changes.

@ahrtr
Copy link
Member Author

ahrtr commented Feb 28, 2022

This PR is little hard to read due to lot of dependency changes. Could you please create a separate PR/commiut that just fixes the dependencies? It should be fast and easy to merge, allowing us to focus the discussion on the config changes.

All the changes in go.mod and go.sum are made by go mod tidy automatically.

Just as I mentioned in the description of this PR, I moved the package pkg/corbautil into clientv3 due to some reasons, and it causes some dependency changes. So I have to fix the dependency changes in this PR.

@ahrtr ahrtr force-pushed the reuse_client_config branch 3 times, most recently from 4b38211 to 91e2c44 Compare February 28, 2022 15:30
client/v3/config.go Outdated Show resolved Hide resolved
move the original clientConfig from etcdctl to clientv3. The original
pks/cobrautil is only used by etcdctl or etcdutl, so move it into the
clientv3 package.
@serathius
Copy link
Member

Just as I mentioned in the description of this PR, I moved the package pkg/corbautil into clientv3 due to some reasons, and it causes some dependency changes. So I have to fix the dependency changes in this PR.

Can you create a separate PR to move pkg/corbautil to clientv3 then? It will make the review much much easier.

@ahrtr
Copy link
Member Author

ahrtr commented Feb 28, 2022

Just as I mentioned in the description of this PR, I moved the package pkg/corbautil into clientv3 due to some reasons, and it causes some dependency changes. So I have to fix the dependency changes in this PR.

Can you create a separate PR to move pkg/corbautil to clientv3 then? It will make the review much much easier.

Actually I want to do it in a separate PR as well, but I can't. I mentioned the reason in the description of this PR. Pasted it again. The original clientConfig has some dependencies on pks/cobrautil, such as (https://github.com/etcd-io/etcd/blob/main/etcdctl/ctlv3/command/global.go#L180). I moved the clientConfig into clientv3 in this PR. However, clientv3 isn't allowed to depend on pkg, see go.mod#L50. Actually the original pks/cobrautil is only used by etcdctl or etcdutl, so it makes sense to move it into the clientv3 package as well.

@ahrtr ahrtr force-pushed the reuse_client_config branch from 91e2c44 to 35e2852 Compare February 28, 2022 15:42
@serathius
Copy link
Member

serathius commented Feb 28, 2022

I don't think that tools for cobra should be put into client package. We should not put dependency on cobra for users that just want to call etcd API. I think we should update the code to stop using cobrautil as dependency.

As etcdctl and etcdutl are command line programs it's resonable they depend on cobra, however this is not true for client which is a library.

@ahrtr
Copy link
Member Author

ahrtr commented Feb 28, 2022

Actually it just depends on cobrautl.ExitBadArgs and cobrautl.ExitBadConnection.

If you are OK, I can remove the dependency in this PR by change them with the related number directly in this PR, see below,

cobrautl.ExitWithError(128, err)

cobrautl.ExitWithError(2, err)

Then I can update all the dependency related changes in a separate PR. WDYT? @serathius

@serathius
Copy link
Member

serathius commented Feb 28, 2022

I think that while trying to reuse ClientConfig we are stumbling upon difference in behavior between library vs binary. Here the difference is seen in error handling. As binary know what command line framework it uses cobra and knows what error codes it want to return it can use them directly. This is however untrue for libraries that should return error to user and let them decide if program should exit or not.

By reusing ClientConfig we are trying it to become a library, however the code itself needs to be changed. I think that instead of calling the exit codes directly we should return error that would be used by binary to decide which error code to return.

@ahrtr
Copy link
Member Author

ahrtr commented Feb 28, 2022

Overall I agree with you. I just submitted another PR 13751. Thanks.

@ahrtr ahrtr closed this Feb 28, 2022
@ahrtr ahrtr removed this from the etcd-v3.6 milestone Jul 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

3 participants