Introduction_to_Information_Extraction/Introduction_to_Information_Extraction.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "gpuType": "T4",
      "authorship_tag": "ABX9TyMtEHsz3VFP6lJaciCraDgW",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/ua-datalab/NLP-Speech/blob/main/Introduction_to_Information_Extraction/Introduction_to_Information_Extraction.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<center><h1> Introduction to Information Extraction </h1> </center>\n",
        "\n",
        "\n",
        "\n",
        "![](https://www.tex-ai.com/wp-content/uploads/2020/10/NLP-techniques-for-information-extraction.jpg)"
      ],
      "metadata": {
        "id": "Y1Qe19Kv212z"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Housekeeping\n",
        "1. Check that the recording is on\n",
        "2. Check audio and screenshare\n",
        "3. Share link to notebook in chat\n",
        "4. Light mode and readable font size"
      ],
      "metadata": {
        "id": "h00WlRlEOwuN"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## What is information extraction?\n",
        "\n",
        "Information extraction (IE) involves converting *unstructured or semi-structured information* from *machine readable documents* into *structured knowledge*, which can be queried to *automatically access* specific information *at scale*.\n",
        "\n",
        "Broadly, it is the culmintion of NLP, dataset creation, and  search and retrieval.\n",
        "\n",
        "It is a move from data processing, to incorporating knowledge into tasks.\n",
        "\n",
        "### Terminology\n",
        "- *Unstructured or semi-structured information*\n",
        "- *Machine readable documents*\n",
        "- *Structured knowledge*\n",
        "- *Automatically access*\n",
        "- *At scale*"
      ],
      "metadata": {
        "id": "_ntT6SjC3BNF"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Processing the World Wide Web- an infinite knowledge source\n",
        "- Millions of contributors, and agregate agreement on correctness of information\n",
        "- Easy to train algorithms to access human-created information\n",
        "- Large group of people who can agree on how entities are related to each other\n",
        "- Semantic Web- formally representing web metadata, entities and locations to utilize information in scalable ways\n",
        "\n",
        "## Shallow vs Advanced Information extraction\n",
        "\n",
        "- Reguar Expressions\n",
        "  - Matching patterns (such as capitalized letters) or digits (money)\n",
        "- Matching keywords against registries\n",
        "  - Names\n",
        "  - Geographical entities\n",
        "  - Currency\n",
        "\n",
        "With the advent of NLP, we are able to add a lot of information to text, and with LLMs, we can add embeddings that better model relationships between words in a document. So, we can move from extracting expected information to processing documents better to search for information better."
      ],
      "metadata": {
        "id": "-zazPxFMPLEa"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## What can we extract?\n",
        "\n",
        "### Named entity recognition (NER) or identification\n",
        "- Aims at finding real-world objects in texts.\n",
        "- Classifies them into predefined categories such as names of persons, organizations, locations, temporal expressions, products, etc.\n",
        "\n",
        "\n",
        "Question: is this an example best solved by shallow or deep IE?\n",
        "\n",
        "### Quantities and monetary values\n",
        "- Currency\n",
        "- Stocks\n",
        "- Number\n",
        "\n",
        "### Entity Disambiguation and Term Evolution\n",
        "- Same name may point to two different entities\n",
        "- An entity may have a new name\n",
        "\n",
        "Question: How does one differenciate information about the Bush administrations of two different presidents?\n",
        "\n",
        "Question: How do we connect news about Twitter from 2010-2024?\n",
        "\n",
        "### Entity classes\n",
        "- categories assigned to an entity\n",
        "- Determines its relationship with other entities\n",
        "\n",
        "A given entity can belong to more than one class.\n",
        "\n",
        "Question: what are some classes Mark Zuckerburg belongs to?\n",
        "\n",
        "### Entity Relations\n",
        "- Meaning building involves connecting concepts\n",
        "- IE includes extracting those connections"
      ],
      "metadata": {
        "id": "8s_CT6R-PcfC"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        "# Creating the components of a knowledge base\n",
        "\n",
        "Information extraction is thus motivated to enrich a knowledge base that can be imporved and addded to. This happens by:\n",
        "\n",
        " - Selecting data sources (newspaper articles, reddit posts, question-answer datasets)\n",
        " - Extracting entities, classes, and relationships in the dataset\n",
        " - Consistntly integrating and linking new information in the right place in an existing knowledge base"
      ],
      "metadata": {
        "id": "uDTLDlm9a2L5"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# How do we carry out information extraction?\n",
        "\n",
        "- Data processing:\n",
        "  - Tokenization\n",
        "  - Tagging\n",
        "  - Stop word removal\n",
        "  - Dependency parsing\n",
        "\n",
        "- Using a language model to:\n",
        "  - Carry out the data processing\n",
        "  - Utilize tags to assess entities and relations\n",
        "  - Make predictions and using them to extract information\n",
        "\n",
        "- Processing the output\n",
        "  - Human-readability and organization\n",
        "  - Natural Language Understanding\n",
        "  - Utilizing the extracted information for downstream tasks (search, text classification, summarization, returning related content)"
      ],
      "metadata": {
        "id": "b2LuVSTk3XJl"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## How do we measure the success of IE ?\n",
        "### TF/IDF\n",
        "- Term Frequency-Inverse Document Frequency\n",
        "- Evaluates the importance of a word in one given document, relative to a collection of documents.\n",
        "- Helps find exact documents relevant to an entity,when it is frequent in one specific document, but rare across the entire dataset\n",
        "- Also provides a measure for \"common\" entities\n",
        "\n",
        "### Precision\n",
        "- The proportion of true positives among all positive (true and false) predictions made by a model.\n",
        "- Measure of how well a model identifies relevant instances.\n",
        "\n",
        "### Recall\n",
        "- The proportion of true positives among all actual positive instances (true positives and false negatives).\n",
        "- It indicates a model's ability to capture relevant cases.\n",
        "- Focus on minimizing false negatives.\n"
      ],
      "metadata": {
        "id": "FlXyRVPTUvCv"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Why Information Extraction?\n",
        "\n",
        "Essential step for downstream tasks, such as text summarization, text classification, sentiment analysis, and finding causality."
      ],
      "metadata": {
        "id": "O1RPvP_1XnFl"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Working examples of information extraction\n",
        "\n",
        "1. Information Collection\n",
        "2. Process Data\n",
        "3. Choosing the Right Model\n",
        "4. Evaluation of the Model\n",
        "5. Deploying Model in Production"
      ],
      "metadata": {
        "id": "p5infNPp3oMQ"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Entity extraction with SpaCy\n",
        "\n",
        "Source: https://www.dataquest.io/blog/tutorial-text-classification-in-python-using-spacy/"
      ],
      "metadata": {
        "id": "4pIHBiAG7bF7"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# ! pip install spacy\n",
        "# ! python -m spacy download en_core_web_sm\n",
        "import spacy\n",
        "\n",
        "# import and load the English language model for vocabluary, syntax & entities\n",
        "import en_core_web_sm\n",
        "nlp = en_core_web_sm.load()\n",
        "\n",
        "#for visualization of Entity detection importing displacy from spacy:\n",
        "from spacy import displacy\n",
        "\n",
        "# For querying tags\n",
        "from spacy import explain"
      ],
      "metadata": {
        "id": "_Wt6rXSuYgy2"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "\n",
        "\n",
        "nytimes= nlp(u\"\"\"New York City on Tuesday declared a public health emergency and ordered mandatory measles vaccinations amid an outbreak, becoming the latest national flash point over refusals to inoculate against dangerous diseases.\n",
        "\n",
        "At least 285 people have contracted measles in the city since September, mostly in Brooklyn’s Williamsburg neighborhood. The order covers four Zip codes there, Mayor Bill de Blasio (D) said Tuesday.\n",
        "\n",
        "The mandate orders all unvaccinated people in the area, including a concentration of Orthodox Jews, to receive inoculations, including for children as young as 6 months old. Anyone who resists could be fined up to $1,000.\"\"\")\n",
        "\n",
        "entities=[(i, i.label_, i.label) for i in nytimes.ents]\n",
        "entities"
      ],
      "metadata": {
        "id": "-2BW1M9ZYfmT"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "displacy.render(nytimes, style = \"ent\",jupyter = True)\n"
      ],
      "metadata": {
        "id": "9wCiWOtPZD4o"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Query tags:\n",
        "import spacy\n",
        "spacy.explain(\"GPE\")"
      ],
      "metadata": {
        "id": "GTMdObXKZUtU"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Discussion\n",
        "\n",
        "This small and light language model is able to parse, tag and extract entities with relative speed. I used the small English model for this task. Changing the size of the language model may give different results."
      ],
      "metadata": {
        "id": "5o-xnpI60UHS"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Simple LLM-based relation extractor\n",
        "\n",
        "For this example, we will focus on extracting relations as well as entities. The model used is Rebel, a simple but powerful relation extractor in the form of a triplet (Entity1, Relation, Entity2). This can run on CPU, as we are accessing a pretrained model.\n",
        "\n",
        "Note: the output of the model is not human readable in this example.\n",
        "\n",
        "Model source: https://huggingface.co/Babelscape/rebel-large"
      ],
      "metadata": {
        "id": "Je8K2NWf72n-"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "cEfeRAeW21L1",
        "collapsed": true
      },
      "outputs": [],
      "source": [
        "#Load model and format for creating the triplets:\n",
        "\n",
        "from transformers import AutoModelForSeq2SeqLM, AutoTokenizer\n",
        "\n",
        "# Load model and tokenizer\n",
        "tokenizer = AutoTokenizer.from_pretrained(\"Babelscape/rebel-large\")\n",
        "model = AutoModelForSeq2SeqLM.from_pretrained(\"Babelscape/rebel-large\")\n",
        "gen_kwargs = {\n",
        "    \"max_length\": 256,\n",
        "    \"length_penalty\": 0,\n",
        "    \"num_beams\": 3,\n",
        "    \"num_return_sequences\": 3,\n",
        "}\n",
        "\n",
        "# Function to extract triplets from text:\n",
        "def extract_triplets(text):\n",
        "    triplets = []\n",
        "    relation, subject, relation, object_ = '', '', '', ''\n",
        "    text = text.strip()\n",
        "    current = 'x'\n",
        "    for token in text.replace(\"<s>\", \"\").replace(\"<pad>\", \"\").replace(\"</s>\", \"\").split():\n",
        "        if token == \"<triplet>\":\n",
        "            current = 't'\n",
        "            if relation != '':\n",
        "                triplets.append({'head': subject.strip(), 'type': relation.strip(),'tail': object_.strip()})\n",
        "                relation = ''\n",
        "            subject = ''\n",
        "        elif token == \"<subj>\":\n",
        "            current = 's'\n",
        "            if relation != '':\n",
        "                triplets.append({'head': subject.strip(), 'type': relation.strip(),'tail': object_.strip()})\n",
        "            object_ = ''\n",
        "        elif token == \"<obj>\":\n",
        "            current = 'o'\n",
        "            relation = ''\n",
        "        else:\n",
        "            if current == 't':\n",
        "                subject += ' ' + token\n",
        "            elif current == 's':\n",
        "                object_ += ' ' + token\n",
        "            elif current == 'o':\n",
        "                relation += ' ' + token\n",
        "    if subject != '' and relation != '' and object_ != '':\n",
        "        triplets.append({'head': subject.strip(), 'type': relation.strip(),'tail': object_.strip()})\n",
        "    return triplets\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# function for processing a text corpus:\n",
        "def print_triplets(text):\n",
        "  # Tokenizer text\n",
        "  model_inputs = tokenizer(text, max_length=1000, padding=True, truncation=True, return_tensors = 'pt')\n",
        "\n",
        "  # Generate\n",
        "  generated_tokens = model.generate(\n",
        "      model_inputs[\"input_ids\"].to(model.device),\n",
        "      attention_mask=model_inputs[\"attention_mask\"].to(model.device),\n",
        "      **gen_kwargs,\n",
        "  )\n",
        "\n",
        "  # Extract text\n",
        "  decoded_preds = tokenizer.batch_decode(generated_tokens, skip_special_tokens=False)\n",
        "\n",
        "  # Print triplets:\n",
        "  for idx, sentence in enumerate(decoded_preds):\n",
        "      print(f'Prediction triplets {idx}')\n",
        "      [print(f\"Entity1: {item['head']}\\n Relation:{item['type']}\\n Entity2:{item['tail']}\") for item in extract_triplets(sentence)]\n",
        "      break #we only print 1 prediction from the model, comment this for more predictions\n",
        "\n"
      ],
      "metadata": {
        "id": "Rhm7aSJQZqnd"
      },
      "execution_count": 34,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Example 1: relation extraction for a given sentence:\n",
        "text1 = \"Batman was created by the artist Bob Kane and writer Bill Finger,\\\n",
        "        and debuted in the 27th issue of the comic book Detective Comics on March 30, 1939.\"\n",
        "print_triplets(text1)"
      ],
      "metadata": {
        "id": "-ugoqjklRIPt",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "441e3dcb-4d43-492c-e761-365bc1973539"
      },
      "execution_count": 35,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Prediction triplets 0\n",
            "Entity1: Batman\n",
            " Relation:creator\n",
            " Entity2:Bob Kane\n",
            "Entity1: Batman\n",
            " Relation:creator\n",
            " Entity2:Bill Finger\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# function for processing a text corpus:\n",
        "def extract_triplets_form_corpus(corpus):\n",
        "  sents = corpus.split('.')\n",
        "  while(\"\" in sents):\n",
        "    sents.remove(\"\")\n",
        "  print(f\"No. of sentences: {len(sents)}\")\n",
        "  for n, sent in enumerate(sents):\n",
        "      print(f\"Sentence no. {n} to be processed: {sent}\")\n",
        "      # Tokenizer text\n",
        "      model_inputs = tokenizer(sent, max_length=1000, padding=True, truncation=True, return_tensors = 'pt')\n",
        "      print(f\"sent: {n} tokenized.\")\n",
        "\n",
        "      # Generate\n",
        "      generated_tokens = model.generate(\n",
        "          model_inputs[\"input_ids\"].to(model.device),\n",
        "          attention_mask=model_inputs[\"attention_mask\"].to(model.device),\n",
        "          **gen_kwargs,\n",
        "      )\n",
        "      print(f\"model output for sent: {n} generated.\")\n",
        "\n",
        "      # Extract text\n",
        "      decoded_preds = tokenizer.batch_decode(generated_tokens, skip_special_tokens=False)\n",
        "      print(f\"text from sent: {n} extracted.\")\n",
        "\n",
        "      # Print triplets:\n",
        "      for idx, sentence in enumerate(decoded_preds):\n",
        "          print(f'Prediction triplets {idx}')\n",
        "          [print(f\"Entity1: {item['head']}\\n Relation:{item['type']}\\n Entity2:{item['tail']}\")\n",
        "          for item in extract_triplets(sentence)]\n",
        "          break #we only print 1 prediction"
      ],
      "metadata": {
        "id": "SvXX8QfAp2Hs"
      },
      "execution_count": 36,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Text to extract triplets from\n",
        "# Source: https://en.wikipedia.org/wiki/Batman\n",
        "text = 'Batman, created by the artist Bob Kane and writer Bill Finger, is a superhero who appears in American comic books published by DC Comics \\\n",
        "        and debuted in the 27th issue of the comic book Detective Comics on March 30, 1939.\\\n",
        "        In the DC Universe, Batman is the alias of Bruce Wayne, a wealthy American playboy, \\\n",
        "        philanthropist, and industrialist who resides in Gotham City.\\\n",
        "        His origin story features him swearing vengeance against criminals after witnessing the murder of his parents,\\\n",
        "        Thomas and Martha.'\n"
      ],
      "metadata": {
        "id": "-KysOyHJm1VQ"
      },
      "execution_count": 37,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "extract_triplets_form_corpus(text)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "KVnScQkGu58k",
        "outputId": "0e364e3a-04fc-4071-c902-85223bfc4dcb"
      },
      "execution_count": 38,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "No. of sentences: 3\n",
            "Sentence no. 0 to be processed: Batman, created by the artist Bob Kane and writer Bill Finger, is a superhero who appears in American comic books published by DC Comics         and debuted in the 27th issue of the comic book Detective Comics on March 30, 1939\n",
            "sent: 0 tokenized.\n",
            "model output for sent: 0 generated.\n",
            "text from sent: 0 extracted.\n",
            "Prediction triplets 0\n",
            "Entity1: Batman\n",
            " Relation:creator\n",
            " Entity2:Bob Kane\n",
            "Entity1: Batman\n",
            " Relation:instance of\n",
            " Entity2:superhero\n",
            "Entity1: Batman\n",
            " Relation:inception\n",
            " Entity2:March 30, 1939\n",
            "Entity1: Detective Comics\n",
            " Relation:publisher\n",
            " Entity2:DC Comics\n",
            "Sentence no. 1 to be processed:         In the DC Universe, Batman is the alias of Bruce Wayne, a wealthy American playboy,         philanthropist, and industrialist who resides in Gotham City\n",
            "sent: 1 tokenized.\n",
            "model output for sent: 1 generated.\n",
            "text from sent: 1 extracted.\n",
            "Prediction triplets 0\n",
            "Entity1: Batman\n",
            " Relation:residence\n",
            " Entity2:Gotham City\n",
            "Sentence no. 2 to be processed:         His origin story features him swearing vengeance against criminals after witnessing the murder of his parents,        Thomas and Martha\n",
            "sent: 2 tokenized.\n",
            "model output for sent: 2 generated.\n",
            "text from sent: 2 extracted.\n",
            "Prediction triplets 0\n",
            "Entity1: Thomas\n",
            " Relation:spouse\n",
            " Entity2:Martha\n",
            "Entity1: Martha\n",
            " Relation:spouse\n",
            " Entity2:Thomas\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Discussion\n",
        "\n",
        "When we provide a text based on both real and fictitious events, our output connects entities with a variety of relations, including \"subclass of\", \"spouse\", and \"residence\".\n",
        "\n",
        "Since \"Batman\" refers to both a fictitious person as well as a work of art, our relations extractor may offer all kinds of relations. We can use this pipeline to extract information based on our needs."
      ],
      "metadata": {
        "id": "BWBJyrEuwCbw"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Try this example on your own:\n",
        "# Source: https://en.wikipedia.org/wiki/Leonardo_da_Vinci\n",
        "text = 'Leonardo da Vinci was born on 15 April 1452 in, or close to, \\\n",
        "        the Tuscan hill town of Vinci, 20 miles from Florence.\\\n",
        "        He was born to Piero da Vinci a Florentine legal notary, and Caterina di Meo Lippi (c. 1434–1494),\\\n",
        "        from the lower class.\\\n",
        "        He was an Italian polymath of the High Renaissance who was active as a painter, draughtsman, \\\n",
        "        engineer, scientist, theorist, sculptor, and architect.'\n",
        "extract_triplets_form_corpus(text)"
      ],
      "metadata": {
        "id": "ggP_BDfoHwL3"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Automatic resume scraping with LLMs\n",
        "\n",
        "How are automatic tracking systems used for collecting information from resumes?\n",
        "\n",
        "Source: https://huggingface.co/foduucom/resume-extractor\n",
        "\n",
        "This simple example uses an LLM to extract information from a resume PDF. It uses Ollama to access Llama3, a large language model.\n",
        "\n",
        "Note: This will take a long while, and requires a GPU to run in a reasonable timeframe.\n",
        "\n"
      ],
      "metadata": {
        "id": "wZX9RZh27P6h"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# install and download ollama with dependencies\n",
        "! sudo apt-get install -y pciutils\n",
        "!curl https://ollama.ai/install.sh | sh\n",
        "!pip install ollama langchain_community pdfminer.six\n",
        "\n",
        "# import necessary python libraries\n",
        "import os\n",
        "import threading\n",
        "import subprocess\n",
        "import requests\n",
        "import json\n",
        "\n",
        "def ollama():\n",
        "    os.environ['OLLAMA_HOST'] = '0.0.0.0:11434'\n",
        "    os.environ['OLLAMA_ORIGINS'] = '*'\n",
        "    subprocess.Popen([\"ollama\", \"serve\"])\n"
      ],
      "metadata": {
        "id": "NKNXtLGhW9Xs",
        "collapsed": true
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# start Ollama\n",
        "ollama_thread = threading.Thread(target=ollama)\n",
        "ollama_thread.start()"
      ],
      "metadata": {
        "id": "IfAukBYRAL4y"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# run embedding model\n",
        "!ollama run llama3"
      ],
      "metadata": {
        "id": "wCfNgPzWAMq6",
        "collapsed": true
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "from langchain_community.llms import Ollama\n",
        "\n",
        "# Format for the extracted output:\n",
        "json_content = \"\"\"{{\n",
        "    \"name\": \"\",\n",
        "    \"email\" : \"\",\n",
        "    \"phone_1\": \"\",\n",
        "    \"phone_2\": \"\",\n",
        "    \"address\": \"\",\n",
        "    \"city\": \"\",\n",
        "    \"linkedin\": \"\",\n",
        "    \"professional_experience_in_years\": \"\",\n",
        "    \"highest_education\": \"\",\n",
        "    \"is_fresher\": \"yes/no\",\n",
        "    \"is_student\": \"yes/no\",\n",
        "    \"skills\": [\"\",\"\"],\n",
        "    \"applied_for_profile\": \"\",\n",
        "    \"education\": [\n",
        "        {{\n",
        "            \"institute_name\": \"\",\n",
        "            \"year_of_passing\": \"\",\n",
        "            \"score\": \"\"\n",
        "        }},\n",
        "        {{\n",
        "            \"institute_name\": \"\",\n",
        "            \"year_of_passing\": \"\",\n",
        "            \"score\": \"\"\n",
        "        }}\n",
        "    ],\n",
        "    \"professional_experience\": [\n",
        "        {{\n",
        "            \"organisation_name\": \"\",\n",
        "            \"duration\": \"\",\n",
        "            \"profile\": \"\"\n",
        "        }},\n",
        "        {{\n",
        "            \"organisation_name\": \"\",\n",
        "            \"duration\": \"\",\n",
        "            \"profile\": \"\"\n",
        "        }}\n",
        "    ]\n",
        "}}\"\"\"\n",
        "\n",
        "class InputData:\n",
        "    # LLM Prompt\n",
        "    def input_data(text):\n",
        "\n",
        "        input = f\"\"\"Extract relevant information from the following resume text and fill the provided JSON template.\n",
        "                    Ensure all keys in the template are present in the output,\n",
        "                    even if the value is empty or unknown.\n",
        "                    If a specific piece of information is not found in the text, use 'Not provided' as the value.\n",
        "\n",
        "        Resume text:\n",
        "        {text}\n",
        "\n",
        "        JSON template:\n",
        "        {json_content}\n",
        "\n",
        "        Instructions:\n",
        "        1. Carefully analyse the resume text.\n",
        "        2. Extract relevant information for each field in the JSON template.\n",
        "        3. If a piece of information is not explicitly stated, make a reasonable inference based on the context.\n",
        "        4. Ensure all keys from the template are present in the output JSON.\n",
        "        5. Format the output as a valid JSON string.\n",
        "\n",
        "        Output the filled JSON template only, without any additional text or explanations.\"\"\"\n",
        "\n",
        "        return input\n",
        "    # run LLM:\n",
        "    def llm():\n",
        "        llm = Ollama(model=\"llama3\")\n",
        "        return llm\n",
        "\n",
        "# Process resume and print results:\n",
        "from pdfminer.high_level import extract_text\n",
        "import sys\n",
        "sys.path.append(\"/content/resume-extractor/\")\n",
        "\n",
        "def extract_text_from_pdf(pdf_path):\n",
        "    return extract_text(pdf_path)"
      ],
      "metadata": {
        "id": "S6Sz0e4vLN1k"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Extraction:\n",
        "text = extract_text_from_pdf(r'/content/anti-cv.pdf')\n",
        "\n",
        "llm = input.llm()\n",
        "data = llm.invoke(input.input_data(text))\n",
        "\n",
        "print(data)"
      ],
      "metadata": {
        "collapsed": true,
        "id": "2RFadcllHc0D"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Discussion\n",
        "\n",
        "With more computing power and a larger LLMs, we can automate the entire pipeline for our information extraction, and work using simple prompts.\n",
        "\n",
        "Our pipeline is able to take an opaque text, extract a document out of it, process it, and conduct an information extraction task, all in one shot, with very good results. However, running a model like this will require a GPU and more overhead."
      ],
      "metadata": {
        "id": "FzWzySwYzjYB"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# References\n",
        "\n",
        "- https://nanonets.com/blog/information-extraction/\n",
        "- https://www.geeksforgeeks.org/information-extraction-in-nlp/\n",
        "- \"Introduction to Information Extraction: Basic Notions and Current Trends\"\n"
      ],
      "metadata": {
        "id": "zcvBYzTW3uH-"
      }
    }
  ]
}