Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cli: node status output misaligns decommission status #27807

Closed
neeral opened this issue Jul 20, 2018 · 1 comment
Closed

cli: node status output misaligns decommission status #27807

neeral opened this issue Jul 20, 2018 · 1 comment
Labels
C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) O-community Originated from the community

Comments

@neeral
Copy link
Contributor

neeral commented Jul 20, 2018 •
edited
Loading

BUG REPORT

Please describe the issue you observed:
cockroach node status will hide decommissioning or dead nodes, unless the user requests all nodes to be shown. The data shown is collected from two different RPC calls and these do not line up. For example, a node's liveness will be put together with a different node's decommission status.

This only affects 2.0 and earlier. In 2.1, the logic for cockroach node status was changed.

  • What did you do?
  1. Decommission NodeID 1 in a cluster
  2. Run cockroach node status --decommission and cockroach node status
  3. Observe the columns is_live compared to the others, e.g. updated_at. You will see it is wrong.
  • What did you expect to see?
neeral@neeral-M51AC:~/go/src/github.com/cockroachdb/cockroach$ cockroach node status --insecure --port 10002
+----+--------------------+-------------------------+---------------------+---------------------+---------+
| id |      address       |          build          |     updated_at      |     started_at      | is_live |
+----+--------------------+-------------------------+---------------------+---------------------+---------+
|  2 | neeral-M51AC:10002 | v2.0.4-5-g7fa050c-dirty | 2018-07-20 16:38:07 | 2018-07-20 16:30:32 | true    |
|  3 | neeral-M51AC:10003 | v2.0.4-5-g7fa050c-dirty | 2018-07-20 16:38:07 | 2018-07-20 16:30:34 | true    |
|  4 | neeral-M51AC:10004 | v2.0.4-5-g7fa050c-dirty | 2018-07-20 16:38:06 | 2018-07-20 16:36:56 | true    |
+----+--------------------+-------------------------+---------------------+---------------------+---------+
(3 rows)
neeral@neeral-M51AC:~/go/src/github.com/cockroachdb/cockroach$ cockroach node status --insecure --port 10002 --all
+----+--------------------+-------------------------+---------------------+---------------------+---------+------------------+-----------------------+--------+--------------------+------------------------+------------+-----------+-------------+--------------+--------------+-------------------+--------------------+-------------+
| id |      address       |          build          |     updated_at      |     started_at      | is_live | replicas_leaders | replicas_leaseholders | ranges | ranges_unavailable | ranges_underreplicated | live_bytes | key_bytes | value_bytes | intent_bytes | system_bytes | gossiped_replicas | is_decommissioning | is_draining |
+----+--------------------+-------------------------+---------------------+---------------------+---------+------------------+-----------------------+--------+--------------------+------------------------+------------+-----------+-------------+--------------+--------------+-------------------+--------------------+-------------+
|  1 | neeral-M51AC:10001 | v2.0.4-5-g7fa050c-dirty | 2018-07-20 16:37:56 | 2018-07-20 16:34:52 |   false |                0 |                     0 |      0 |                  0 |                      0 |          0 |         0 |           0 |            0 |            0 |                 0 |               true |    true     |
|  2 | neeral-M51AC:10002 | v2.0.4-5-g7fa050c-dirty | 2018-07-20 16:38:17 | 2018-07-20 16:30:32 |    true |               11 |                    11 |     20 |                  0 |                      0 |    4670684 |     54588 |     4664514 |            0 |        17045 |                21 |              false |    false    |
|  3 | neeral-M51AC:10003 | v2.0.4-5-g7fa050c-dirty | 2018-07-20 16:38:17 | 2018-07-20 16:30:34 |    true |                6 |                     6 |     20 |                  0 |                      0 |    4670684 |     54588 |     4664514 |            0 |        17045 |                21 |              false |    false    |
|  4 | neeral-M51AC:10004 | v2.0.4-5-g7fa050c-dirty | 2018-07-20 16:38:16 | 2018-07-20 16:36:56 |    true |                3 |                     2 |     20 |                  0 |                      0 |    4607768 |     54576 |     4601575 |            0 |        17045 |                21 |              false |    false    |
+----+--------------------+-------------------------+---------------------+---------------------+---------+------------------+-----------------------+--------+--------------------+------------------------+------------+-----------+-------------+--------------+--------------+-------------------+--------------------+-------------+
(4 rows)
  • What did you see instead?
neeral@neeral-M51AC:~/go/src/github.com/cockroachdb/cockroach$ cockroach node status --insecure --port 10002 --all
+----+--------------------+-------------------------+---------------------+---------------------+---------+------------------+-----------------------+--------+--------------------+------------------------+------------+-----------+-------------+--------------+--------------+-------------------+--------------------+-------------+
| id |      address       |          build          |     updated_at      |     started_at      | is_live | replicas_leaders | replicas_leaseholders | ranges | ranges_unavailable | ranges_underreplicated | live_bytes | key_bytes | value_bytes | intent_bytes | system_bytes | gossiped_replicas | is_decommissioning | is_draining |
+----+--------------------+-------------------------+---------------------+---------------------+---------+------------------+-----------------------+--------+--------------------+------------------------+------------+-----------+-------------+--------------+--------------+-------------------+--------------------+-------------+
|  1 | neeral-M51AC:10001 | v2.0.4-5-g7fa050c-dirty | 2018-07-20 16:37:56 | 2018-07-20 16:34:52 |   false |                0 |                     0 |      0 |                  0 |                      0 |          0 |         0 |           0 |            0 |            0 |                 0 |               true |    true     |
|  2 | neeral-M51AC:10002 | v2.0.4-4-gfb69397       | 2018-07-20 16:39:54 | 2018-07-20 16:39:54 |    true |                0 |                     0 |     20 |                  0 |                      0 |    4922350 |     54984 |     4916941 |            0 |        17033 |                21 |              false |    false    |
|  3 | neeral-M51AC:10003 | v2.0.4-5-g7fa050c-dirty | 2018-07-20 16:39:57 | 2018-07-20 16:30:34 |    true |               14 |                    13 |     20 |                  0 |                      0 |    5195505 |     55518 |     5190439 |            0 |        17059 |                21 |              false |    false    |
|  4 | neeral-M51AC:10004 | v2.0.4-5-g7fa050c-dirty | 2018-07-20 16:39:56 | 2018-07-20 16:36:56 |    true |                5 |                     4 |     20 |                  0 |                     20 |    5132062 |     55260 |     5127182 |            0 |        17059 |                21 |              false |    false    |
+----+--------------------+-------------------------+---------------------+---------------------+---------+------------------+-----------------------+--------+--------------------+------------------------+------------+-----------+-------------+--------------+--------------+-------------------+--------------------+-------------+
(4 rows)
neeral@neeral-M51AC:~/go/src/github.com/cockroachdb/cockroach$ cockroach node status --insecure --port 10002
+----+--------------------+-------------------------+---------------------+---------------------+---------+
| id |      address       |          build          |     updated_at      |     started_at      | is_live |
+----+--------------------+-------------------------+---------------------+---------------------+---------+
|  2 | neeral-M51AC:10002 | v2.0.4-4-gfb69397       | 2018-07-20 16:39:54 | 2018-07-20 16:39:54 | false   |
|  3 | neeral-M51AC:10003 | v2.0.4-5-g7fa050c-dirty | 2018-07-20 16:39:57 | 2018-07-20 16:30:34 | true    |
|  4 | neeral-M51AC:10004 | v2.0.4-5-g7fa050c-dirty | 2018-07-20 16:39:56 | 2018-07-20 16:36:56 | true    |
+----+--------------------+-------------------------+---------------------+---------------------+---------+
@neeral neeral mentioned this issue Jul 20, 2018
Merged
@knz knz changed the title node status output misaligns decommission status cli: node status output misaligns decommission status Jul 21, 2018
@knz knz added C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) O-community Originated from the community A-cli labels Jul 21, 2018
craig bot pushed a commit that referenced this issue Aug 3, 2018
@neeral
Merge #27808
41b1a63 
27808: cli: fix alignment in `node status` output r=benesch a=neeral

See #27807

`cockroach node status` will hide decommissioning or dead nodes, unless
the user requests all nodes to be shown. The data shown is collected
from two different RPC calls and these do not line up. For example, a
node's liveness will be put together with a different node's
decommission status.

To view this, decommission the node with NodeID 1 in a cluster. Run
`cockroach node status --decommission` and `cockroach node status`. Look
at the columns IsLive and UpdatedAt. You'll see they go out of sync.

Release note: None

Co-authored-by: neeral <[email protected]>
@knz
Copy link
Contributor

knz commented Aug 7, 2018

Fixed by #27808

@knz knz closed this as completed Aug 7, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) O-community Originated from the community
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@knz @neeral