DISCUSSION : Apply Feedback Loop on the Predictions

[kshitijrajsharma]

• Predictions are now on fAIr dev , Plan is to start discussion on how we can include feedback loop on the predictions done by model and improve it accordingly

[kshitijrajsharma]

IMO , I think let's start with simple :
I propose to implement this step by step , First lets do wrong prediction feedback
We can display popup on frontend , To let user provide feedback if prediction done by fAIr is right or wrong , if right user will keep as it is if wrong lets change color of that particular user and collect until user user is finished , Once user says submit my feedback lets get those wrong predictions and keep it in backend which will be associated with that particular published training model , It should have a validation option , we can't live train / fit model as user gives feedback someone needs to validate the feedback so that intentionally one won't be able to mislead the model , It should be visible to project manager who published the model or in usergroup of model management , he/should be able to see the feedback for the model he published and its up to him which feedback he wants to validate once validate project manager can fit the model accordingly apply feedback and push model !
This idea will require a proper research how we can fit model and publish new version of it , since feedback data may be small without misleading /overfitting the model
Frontend needs new feature
CURD API should be there , table should be created for feedback which will have , id , geom , training_id , validation_status , feedback_type .Lets also include feedback type so that on future we will be able to incorporate / accept multiple feedback_type from user and also user information for sure !
Frontend for project managers should be there , they should be able to see submitted feedbacks and validate them