Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Recommender status info #1066

Closed
8 tasks done
reckart opened this issue Mar 25, 2019 · 3 comments
Closed
8 tasks done

Recommender status info #1066

reckart opened this issue Mar 25, 2019 · 3 comments
Assignees
Labels
Milestone

Comments

@reckart
Copy link
Member

reckart commented Mar 25, 2019

Is your feature request related to a problem? Please describe.
There is currently no way for the user to see the status of recommenders.

Describe the solution you'd like
Add recommender status information to the recommender sidebar showing some or all of the following information:

  • recommender name (in sidebar)
  • training set size (in sidebar)
  • training time (in recommender log available from sidebar)
  • evaluation set size (in sidebar)
  • evaluation time (in recommender log available from sidebar)
  • selection time (in recommender log available from sidebar)
  • accuracy (in sidebar)
  • active/not active (in sidebar)

It would also be nice if there would be an indication telling if the recommender is currently ready to generate predictions / being (re-)trained, e.g.: initial training (not ready), re-training (ready but being updated), ready (ready, not being updated atm)

Describe alternatives you've considered
No alternative ideas.

@reckart reckart added this to the 0.10.0 milestone Mar 25, 2019
@reckart reckart modified the milestones: 0.10.0, 0.11.0 Jun 6, 2019
@reckart reckart assigned reckart and unassigned reckart Jul 12, 2019
@reckart
Copy link
Member Author

reckart commented Jul 15, 2019

@UWinch @Sibgha360 I have added a list of recommenders into the sidebar now, but it �is not yet very pretty and doesn't show all the info listed above.

I think it would be great to look how to make it such that it updates via websocket whenever training/selection tasks start/complete.

Also it would probably be a good idea to only show the recommender title and state by default and allow the user to click on some button or on the title to expand it and only then show all the details.

UWinch added a commit that referenced this issue Jul 18, 2019
- started on websocket messaging
UWinch added a commit that referenced this issue Jul 19, 2019
- conform to existing events by extending applicationevent
UWinch added a commit that referenced this issue Jul 19, 2019
- fixed injecting wicket application
@UWinch
Copy link
Contributor

UWinch commented Jul 22, 2019

@reckart I started working on this and would like to clarify:

  1. What exactly do you feel should be prettier, only the collapsible list or also other things?
  2. You added the evaluationresult to the activeRecommenders to access them in this list. Is this a workaround that should be changed and now be updated via websocket messages?
    Thx.

@reckart
Copy link
Member Author

reckart commented Jul 22, 2019

Wrt. 1: Well, I think it doesn't look particularly slick. I used the bootstrap list group and badges because it was quick, not because I think they are very pretty. I think they work ok on the project overview page, but in a sidebar, I feel they take up quite a bit of space. Anyway... here are some more things I thought about:

  • color the active/off indicator
  • when websocked is enabled, add some (animated) indicator saying whether training is in progress, training is complete, or training had an error
  • provide access to the messages which we now (can) have in the RecommenderContext, and actually produce some messages. Maybe even replace the static display of the four metrics that I added by a generic access to the messages and simply write the metrics out to the messages.
  • add buttons to enable/disable a particular recommender per user
  • add buttons to show/hide the suggestions produced by a particular recommender - this could also control whether the respective recommender is shown in the graph
  • remove the labels from the graph and instead add an indicator next to each recommender in the list below saying which color it is has in the graph
  • add font-awesome glyphs to the buttons
  • think about how to do scrolling in the sidebar, i.e. which parts should always be visible and which parts should scroll?
  • ... these just off the top of my mind

Wrt 2. I think they should stay. As for the WS messages: we could send the metrics/evaluation result along with the message or we could just use the message as a trigger to re-render the metrics which are picked up from the evaluation result that is attached to the respective active recommender. Naively, I'd probably go with the latter - sounds like you had the former in mind.

UWinch added a commit that referenced this issue Jul 23, 2019
- deleted unecessary event
- started debugging
UWinch added a commit that referenced this issue Jul 25, 2019
- getting user from annotatorstate to use outside requestcycle
UWinch added a commit that referenced this issue Jul 25, 2019
- added error message to events
reckart added a commit that referenced this issue Jul 26, 2019
- Register WS endpoing when deploying in Eclipse with "serve modules without publishing" enabled - otherwise WS doesn't work
reckart added a commit that referenced this issue Jul 26, 2019
- Added missing dependency
UWinch added a commit that referenced this issue Aug 9, 2019
- only publish events instead of callback use
- fixed constructor bug
UWinch added a commit that referenced this issue Aug 9, 2019
- started re-structuring models for info-panel
@reckart reckart modified the milestones: 0.11.0, 0.12.0 Aug 11, 2019
UWinch added a commit that referenced this issue Aug 12, 2019
- finished adding second model to info-panel for immediate update
UWinch added a commit that referenced this issue Aug 13, 2019
-fix for unit test
@reckart reckart modified the milestones: 0.11.0, 0.12.0 Aug 14, 2019
UWinch added a commit that referenced this issue Aug 21, 2019
-requested changes
reckart added a commit that referenced this issue Oct 1, 2019
- Do not log EOF exceptioin when user leaves page
- Re-render the learning curve chart via websocket instead of by the RenderAnnotationsEvent
@reckart reckart modified the milestones: 0.13.0, 0.14.0 Nov 11, 2019
@reckart reckart modified the milestones: 0.14.0, 0.15.0 Nov 26, 2019
@jcklie jcklie modified the milestones: 0.15.0, 0.16.0 Jan 14, 2020
UWinch added a commit that referenced this issue Mar 17, 2020
@reckart reckart modified the milestones: 0.16.0, 0.17.0 May 4, 2020
@jcklie jcklie modified the milestones: 0.17.0, 0.18.0 Jul 21, 2020
@reckart reckart added the Stale Has not need updates for a while. Do not merge. label Jul 28, 2020
@reckart reckart modified the milestones: 0.18.0, Feature backlog Jul 28, 2020
@reckart reckart modified the milestones: ⭐️ Feature backlog, 29.0 May 30, 2023
@reckart reckart removed the Stale Has not need updates for a while. Do not merge. label May 30, 2023
@reckart reckart assigned reckart and unassigned UWinch May 30, 2023
reckart added a commit that referenced this issue May 30, 2023
- Fix logging of start and end events for selection, training and prediction and always show the time the steps too in the recommender log
reckart added a commit that referenced this issue May 30, 2023
@reckart reckart closed this as completed May 30, 2023
reckart added a commit that referenced this issue Jun 13, 2023
* main: (189 commits)
  No issue. Minor additions to BioC format description
  #4062 - ViewportTracker should focus on block-like elements
  #4032 - Allow using externalized strings from backend code
  #4060 - Clean up redundant code in annotation handlers
  #4026: Support for error tracking with Sentry
  #3673 - Update dependencies
  update dead link to the new file
  #4055 - Editor scrolls up when left sidebar is opened/closed
  [maven-release-plugin] prepare for next development iteration
  [maven-release-plugin] prepare release inception-28.2
  #4052 - Admins no longer see all projects in the project overview
  #3673 - Update dependencies
  #4048 - Document navigation options not visible to manager when viewing other users document
  #3673 - Update dependencies
  #3673 - Update dependencies
  #1496 - Some spans are missing begin offset field
  #1511 - External recommender fails when CAS contains control characters
  #1496 - Some spans are missing begin offset field
  #4040 - Ability to store preferences from client-side code
  #1066 - Recommender status info
  ...

% Conflicts:
%	inception/inception-api-annotation/src/main/java/de/tudarmstadt/ukp/clarin/webanno/api/annotation/page/AnnotationPageBase.java
%	inception/inception-brat-editor/src/main/java/de/tudarmstadt/ukp/clarin/webanno/brat/annotation/BratAnnotationEditor.java
%	inception/inception-diam/src/main/java/de/tudarmstadt/ukp/inception/diam/service/DiamWebsocketController.java
%	inception/inception-documents/src/test/java/de/tudarmstadt/ukp/inception/documents/DocumentServiceImplConcurrencyTest.java
%	inception/inception-external-search-solr/pom.xml
%	inception/inception-html-editor/src/main/java/de/tudarmstadt/ukp/inception/htmleditor/docview/HtmlDocumentViewControllerImpl.java
%	inception/inception-html-editor/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports
%	inception/inception-preferences/src/main/java/de/tudarmstadt/ukp/inception/preferences/config/PreferencesServiceAutoConfig.java
%	inception/inception-recommendation/src/main/java/de/tudarmstadt/ukp/inception/recommendation/service/LearningRecordServiceImpl.java
%	inception/inception-recommendation/src/main/java/de/tudarmstadt/ukp/inception/recommendation/service/RecommendationServiceImpl.java
%	inception/inception-recommendation/src/main/java/de/tudarmstadt/ukp/inception/recommendation/tasks/TrainingTask.java
%	inception/inception-recommendation/src/test/java/de/tudarmstadt/ukp/inception/recommendation/footer/RecommendationEventWebsocketControllerImplTest.java
%	inception/inception-support/pom.xml
%	inception/inception-ui-core/pom.xml
%	inception/inception-websocket/pom.xml
%	inception/pom.xml
@reckart reckart added this to Kanban Aug 7, 2024
@reckart reckart moved this to 🍹 Done in Kanban Aug 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Archived in project
Development

No branches or pull requests

3 participants