Rubric Generator Expedition - Silicone Sages

[asleem]

Summary

This Pull Request is for the Rubric Generator expedition, a new feature added to the Marvel-AI that allows educators to generate a customized rubric for an assignment based on the learning standard, point_scale and grade level specified. Moreover, educators can download the generated rubric as a pdf document.
In this PR we are using the extended file support (document loaders) that was created in epic 7.1 so educators can upload an assignment in a wide range of file type/format in order to generate the corresponding rubric.

Changes

• Created new folder within app/features named rubric_generator that contains the following;
  • Core.py created the executor function that takes the input parameter and calls the create_rubric method
  • Metadata.json created the validation criteria for router.py to guaranty the parameter handling of rubric_generator_args
  • Tools.py created the RubricGenerator class with all necessary functions and classes for our rubric creation feature
  • Tests/test_core.py created 2 unit test with pytest
  • Prompt/rubric-generator-prompt.txt createdthe prompt sent to the LLM
  • document_loaders.py reused the document loader from epic 7.1 PR#79 Silicon Sages - Epic 7.1 - Multiple Choice Assessments Generator #79
• Created RubricGeneratorArgs class defining the pydantic schema in app/services/schema.py
• Created convert_rubric_generator_args_to_pydantic method in app/api/tool_utilities.py
• Added “15” as the tool nb for the rubric generator feature in app/api/tools_config.json
• Changed the output of submit_tool method to be able to return a downloadable pdf file instead of json response
• Updated the requirements.txt file and added packages versions to fix dependencies conflict

Testing

Results

The result is a downloadable pdf document that contain the rubric generated in a table template.

rubric.pdf

Notes

First, the LLM generates the rubric in JSON format. Next, we create a LaTeX document and insert the rubric into a table template. Finally, the LaTeX document is compiled into a PDF file.
The max number for the point scale is set to 8, the suggested value is 4 for optimal granularity in grading and to better fit in the generated rubric table template in the pdf document.
A feedback is added to the generated pdf document; this is an evaluation of the rubric created by the LLM for more assistance for the educators.

Request interface

Schema:

Example:

{
  "user": {
    "id": "string",
    "fullName": "string",
    "email": "string"
  },
  "type": "chat",
  "tool_data": {
    "tool_id": 15,
    "inputs": [
      {
        "name": "rubric_generator_args",
        "value": {
             "standard": "To make an ensemble Machine Learning model",
             "point_scale":8,
             "grade_level": "college",
             "file_type": "pdf",
             "lang": "en",
             "file_url": "https://raw.githubusercontent.com/asleem/uploaded_files/main/assignment_build_LM.pdf"
        }
      }
    ]
  }
}

Response interface

Schema:

Example:
rubric-47.pdf

[kyl88]

Code implementation of the API and Schema looks solid :)

The hyperparameters were implemented well

It does match the criteria on PDF documents

Pytest test cases were implemented well

tool utilities and routing of app is good

[kyl88]

@wilfredo

Any feedback from you with regard to Abir implementation and test cases

[AaronSosaRamos]

@yunusj I've refactored the Rubric Generator moving the document loaders to a global utils folder for better reusability.

[yunusj]

Response needs to be JSON instead of file download as our frontend displays the JSON in editable document viewer

[AaronSosaRamos]

Hey @yunusj, I've refactored the PR in order to return a JSON instead of a file download