Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

auth jwt: grpc context timed out for ErrAuthOldRevision #11033

Closed
xqzhang2015 opened this issue Aug 14, 2019 · 7 comments
Closed

auth jwt: grpc context timed out for ErrAuthOldRevision #11033

xqzhang2015 opened this issue Aug 14, 2019 · 7 comments
Assignees
Labels

Comments

@xqzhang2015
Copy link

xqzhang2015 commented Aug 14, 2019

We're trying to build a C++ client for V3, with supporting TLS and auth. But when testing auth error ErrAuthOldRevision, client-side doesn't receive any expected response and just grpc context timed out. Not sure it's an etcd cluster issue or grpc c++ issue.

You could reproduce it by sync or async get call. Steps:

  1. authenticate and call get
  2. add a new user(with it, auth revision will be increased with 1)
  3. call get again(Expectation: ErrAuthOldRevision response, Physical: grpc context timed out)

[etcd --version]
etcd Version: 3.3.13
Git SHA: 98d3084
Go Version: go1.10.8
Go OS/Arch: linux/amd64

[grpc version]
grpc C++, 1.14.0

[System]
Linux 3.10.0-229.el7.x86_64 #1 SMP Fri Mar 6 11:36:42 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

@mitake
Copy link
Contributor

mitake commented Aug 15, 2019

@xqzhang2015 can you reproduce this on the latest master branch? Probably it would be caused by the invalid version number mechanism implemented in the server side (see also #10408).

@xqzhang2015
Copy link
Author

@mitake thanks for the info. And it's really caused by the infinite loop issue of JWT token revision staleness.

Once this client timed out issue happened, the CPU utilization of etcd cluster will become super high, just as same as the #10408 or #10218 .

With that patch for v3.3.13, the client timed out caused by JWT token staleness disappeared.

I noticed that v3.4.0-rc.1 has included this patch, but #10468 is not included, will it be applied? BTW, will the official v3.4.0, instead of experimental version, be released on 2019-08-30?

Furthermore, will it be included by v3.3.14? If yes, when will the official version be released?

Thanks again. Really appreciate your confirmation.

@xqzhang2015
Copy link
Author

@mitake or @xiang90 would you like to give any comment for my question above?

@mitake
Copy link
Contributor

mitake commented Aug 21, 2019

@xqzhang2015 sorry for my late reply. @gyuho how do you think about backporting the change? If it's ok for you, I'll open a PR for backporting.

@xqzhang2015
Copy link
Author

I noticed v3.3.15 was released recently and think it's ok for me to release it to next v3.3.x version.

@mitake mitake self-assigned this Aug 26, 2019
@mitake
Copy link
Contributor

mitake commented Aug 26, 2019

@xqzhang2015 got it, let me handle this.

@stale
Copy link

stale bot commented Apr 6, 2020

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 21 days if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Apr 6, 2020
@stale stale bot closed this as completed Apr 28, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Development

No branches or pull requests

2 participants