Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature : Feedback Loop for Ramp #108

Merged
merged 39 commits into from
Jun 14, 2023
Merged

Feature : Feedback Loop for Ramp #108

merged 39 commits into from
Jun 14, 2023

Conversation

kshitijrajsharma
Copy link
Member

@kshitijrajsharma kshitijrajsharma commented May 2, 2023

Issue:

What does this PR DO ?

PR Implements the Awaiting feedback loop method for RAMP Basemodel !
PR Focuses Mainly on Backend changes to implement feedback method

  • Introduces Feedback Model
  • CURD API for Feedback
  • Connects feedback data to training Methodology
  • Basic Frontend to showcase backend flow ( Mapper view and Validator View )

Methodology
When new feedback is submitted backend picks up the loaded model , zoom level of prediction and base image when mapper is providing feedback , That Published model acts as base model where new feedback will be fit into . Initially Feedback will go back to manager/moderators of model's page . Moderators can see the feedbacks for their published model . They have option to validate good feedbacks while discarding the bad feedbacks ! Feedback loops runs with 1 batch size and 1 epoch by default to avoid overfitting . Maintains different version of model, it will be a new training instance which means manager can always go back to previous version of model if things doesn't work out after applying feedback !

Limitation :

  • Feedbacks needs to be completed for the grid single feedback or couple of feedback can't be supplied to model , for eg : if model makes two bad prediction in area while submitting feedback full input for whole grid should be supplied Like this , if one feature is accepted /modified or deleted for feedback mapper needs to make sure all the yellow box is completed for feedback

image

How to test ?

  • Test Feedback CURD API

image

Run predictions from published model , Submit feedbacks from Mapper view , Validate them from Manager View and Apply those to published model and publish new version of model with feedback ! **Mapper View :**

image

**Manager View :**

image

Currently Accepted Feedback :

  • Missing Feature : Mapper can create new feature if missing
  • Modify Feature : Mapper can correct if predictions are not correct / misaligned
  • Delete Feature : Mapper can delete invalid outliers

TODO :

  • Stabilize This Method
  • Add good methods and direction to frontend

@kshitijrajsharma kshitijrajsharma marked this pull request as ready for review June 14, 2023 09:00
@kshitijrajsharma kshitijrajsharma merged commit 094905a into master Jun 14, 2023
@kshitijrajsharma kshitijrajsharma deleted the feature/feedback branch June 14, 2023 09:01
jeafreezy pushed a commit to jeafreezy/fAIr that referenced this pull request Sep 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add Backend Model and CURD API for feedback
1 participant