-
Notifications
You must be signed in to change notification settings - Fork 14.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add average duration markline in task and dagrun duration charts. #38214
Conversation
That is very nice! |
The value is calculated by echarts using type option. It only supports average, min, max and median values. But since runduration and queued duration is calculated in javascript the p50, p70 values can also be calculated to be supplied to markline and they can have different colors. As per my understanding p50 and median are equivalent which could be more useful than average. https://echarts.apache.org/en/option.html#series-bar.markLine.data.0.type Here median is red and average is blue for the same task as above. |
That is cool and useful!
yes |
Nice. Percentile calculations are going to be very useful. |
Nice work! If we do median and average, let's change the tick style so we rely on more than just color. Also, we should make sure the colors are separate from any state colors like red and green. I also agree that we need a legend. Later on, we can move more of this logic to the backend and we can do more complex metrics. I would still like to do a historic gantt chart with a box-and-whisker plot of task durations. |
But if we do just average, this looks good. Does this also work for Run Duration when "Show Landing Times" is checked? |
After using this in several charts of different variations I agree with @eladkal that median looks more useful since one of the outlier task that takes more time to run skews the average to be higher when other tasks executed in few minutes and median looks more useful in the chart compared to average value. I am updating the PR to show median value only to skip average. Please feel free to suggest color that is accessible since I just used the default one provided by echarts. Thanks |
fd38476
to
67d62e6
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice work!
I think I'll move the dag runs summary from the Details panel to below the graph on the Run Duration page. Then we can show appropriate stats.
…ache#38214) * Add average duration markline in task and dagrun duration charts. * Use median instead of average.
Add average duration for run and queued time in task duration and dag run duration. This can help in visualizing potential delays in runs like below dag for one month where there is more sleep time in the last 5 days compared to previous days.
Related: #22132
Ref comment in PR review : #35863 (comment)