Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Execution time tracking tweaks #1994

Merged
merged 5 commits into from
Aug 6, 2024
Merged

Execution time tracking tweaks #1994

merged 5 commits into from
Aug 6, 2024

Conversation

ikreymer
Copy link
Member

@ikreymer ikreymer commented Aug 6, 2024

Tweaks to how execution time is tracked for more accuracy + excluding waiting states:

  • don't update if crawl state is in a 'waiting state' (waiting for capacity or waiting for org limit)
  • rename start states -> waiting states for clarity
  • reset lastUpdatedTime if two consecutive updates of non-running state, to ensure non-running states don't count, but also account for occasional hiccups -- if only one update detects non-running state, don't reset
  • webhooks: move start webhook to when crawl actually starts for first time (db lastUpdatedTime is not yet + crawl is running)
  • don't set lastUpdatedTime until pods actually running
  • set crawljob update interval to every 10 seconds for more accurate execution time tracking
  • frontend: show seconds in 'Execution Time' display

- don't update if crawl state is not in running state
- reset lastUpdatedTime if two consecutive updates of non-running state, to ensure non-running states don't count
- webhooks: move start webhook to when crawl actually starts for first time (db lastUpdatedTime is not yet + crawl is running)
- don't set lastUpdatedTime until pods actually running
- frontend: show seconds in 'Execution Time' display
@ikreymer ikreymer requested a review from tw4l August 6, 2024 06:42
don't count execution time when in WAITING_STATES
…ng with crawl last update time, properly

detecting when it hasn't been set yet
Copy link
Member

@tw4l tw4l left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice changes! Tested and seems to be working well

@ikreymer ikreymer merged commit 7fa2b61 into main Aug 6, 2024
6 checks passed
@ikreymer ikreymer deleted the exec-time-tweaks branch August 6, 2024 16:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants