ui: The "status" column in the UI seems discrepant from the status in SHOW JOBS when a high-water timestamp is present

[jocrl]

Broken out from #71563 (comment) to separate the issues.

Here, these three jobs are either canceled or paused, but the canceled and paused statuses are not shown in the UI. This can be confusing.

Previous discussion:

@jocrl: Poking around, I still don't have a good understanding of what a highwater timestamp is. But I'm wondering if we maybe want to show a running/cancelled/paused etc. badge in addition to the highwater timestamp?

@Annebirzin: I'm not sure I'm fully understanding the discrepancy but seems like adding the status would make the true status more clear (and assuming this would be consistent with the jobs table in the SQL shell)

@miretskiy, via slack: Highwater makes sense for Running or Paused jobs (perhaps retrying too)...
pause-requested is a transient state... I suppose it does make sense to show it there as well...

Jira issue: CRDB-15410

[kevin-v-ngo]

Do we know why cancelled and paused statuses are not shown in the UI and is it because they have a high-water mark? Would all jobs that have a high-water mark have an empty status in the UI? Curious what makes it confusing and if someone reported this. Perhaps we need some form of an 'uknown' status prompting someone to drill into the jobs details page.

Note high-water mark seems like a changefeed specific concept where one would set up alerts to understand if the feed is keeping up: https://www.cockroachlabs.com/docs/v20.1/change-data-capture.html#monitor-a-changefeed. It seems more natural to be in the job details page for changefeed jobs?

[jocrl]

Do we know why cancelled and paused statuses are not shown in the UI and is it because they have a high-water mark?
...
Note high-water mark seems like a changefeed specific concept where one would set up alerts to understand if the feed is keeping up: https://www.cockroachlabs.com/docs/v20.1/change-data-capture.html#monitor-a-changefeed. It seems more natural to be in the job details page for changefeed jobs?

I don't know. @amruss, would you know?

Would all jobs that have a high-water mark have an empty status in the UI?

As currently implemented, yes.

[dhartunian]

hi folks, sorry for the confusion but I think I fixed this here: #81213

I filed the issue #80496 via this support ticket: https://github.com/cockroachlabs/support/issues/1552

@kevin-v-ngo @jocrl it never crossed my mind that this would be a SQL issue, sorry.

[dhartunian]

Also, I don't have a good reason why for jobs with a highwater timestamp, we omitted the status. I think this was a historical quirk I kept around when modifying the jobs page since I didn't have a lot of context at the time either.

Anyway, I strongly believe that matching SQL output should always automatically "win" the debate.

[jocrl]

Ah, thanks David! I'm closing this issue; solved by #81213.

@kevin-v-ngo, @amruss, and @dhartunian, I think there's a question of how to coordinate jobs page work 😛

@kevin-v-ngo, I think there's also a question of who we can go to for decisions on requirements for the Jobs page, and who can make a decision that is a green flag to do it. I was going to bring up in our 1:1 that I went ahead with showing both at once based on feedback from @miretskiy, after asking on the #jobs channel. I know @amruss has been OOO for a while; and perhaps this (understanding requirements) is an area we can more explicitly ask for her and the Jobs team's help with.

Thanks everyone!