Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ML] Improve progress reportings for DF analytics #45856

Commits on Aug 22, 2019

  1. [ML] Improve progress reportings for DF analytics

    Previously, the stats API reports a progress percentage
    for DF analytics tasks that are running and are in the
    `reindexing` or `analyzing` state.
    
    This means that when the task is `stopped` there is no progress
    reported. Thus, one cannot distinguish between a task that never
    run to one that completed.
    
    In addition, there are blind spots in the progress reporting.
    In particular, we do not account for when data is loaded into the
    process. We also do not account for when results are written.
    
    This commit addresses the above issues. It changes progress
    to being a list of objects, each one describing the phase
    and its progress as a percentage. We currently have 4 phases:
    reindexing, loading_data, analyzing, writing_results.
    
    When the task stops, progress is persisted as a document in the
    state index. The stats API now reports progress from in-memory
    if the task is running, or returns the persisted document
    (if there is one).
    dimitris-athanasiou committed Aug 22, 2019
    Configuration menu
    Copy the full SHA
    e33d175 View commit details
    Browse the repository at this point in the history

Commits on Aug 23, 2019

  1. Configuration menu
    Copy the full SHA
    a12aea2 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    0c61cb3 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    d0d6227 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    eb81450 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    11c446e View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    24e506a View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    d9d97e4 View commit details
    Browse the repository at this point in the history
  8. Update x-pack/plugin/ml/src/main/java/org/elasticsearch/xpack/ml/acti…

    …on/TransportGetDataFrameAnalyticsStatsAction.java
    
    Co-Authored-By: Benjamin Trent <[email protected]>
    dimitris-athanasiou and benwtrent authored Aug 23, 2019
    Configuration menu
    Copy the full SHA
    d567c54 View commit details
    Browse the repository at this point in the history