Master thesis with code investigating methods for incorporating long-context reasoning in low-resource languages, without the need to pre-train from scratch. We investigated if multilingual models could inherit these properties by making it an Efficient Transformer (s.a. the Longformer architecture).

Convert pretrained RoBerta models to various long-document transformer models

LongPegasus package is used for inducing longformer self attention over base pegasus abstractive summarization model to increase the token limit and performance.

Discourse_classification is a LongFormer model designed to identify different discourse elements in a student's writing.

An algorithm that works which can take the following PDF and extract key information from the same for an investor looking to evaluate the company. The investor looks at key elements such as future growth prospects, key changes in the business, key triggers, important information that might have a material effect on next year's earnings and growth.

Exploratory notebooks and utils related to the "Feedback Prize - Evaluating Student Writing" Kaggle competition.

Large Language Model for Semantic Table Interpretation