Examples _ End-to-End Neural Entity Linking.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Try on your own input\n",
    "The following notebook shows how to practice with the code repository with you example text.\n",
    "**Pre-requisites**\n",
    "- Make sure you have installed and downloaded everything as the [README](https://github.com/dalab/end2end_neural_el#trying-the-system-on-random-user-input-text) mentions on the github page.\n",
    "- Once you follow the instructions you will have the server running at https://localhost:5555.\n",
    "- To quickly. run the following cell.- "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Response [200]>"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import requests, json\n",
    "requests.post(\"http://localhost:5555\") ## if Response is [200] then it means the server is running"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[17, 7, \"Germany\"], [49, 6, \"Angela_Merkel\"], [0, 5, \"Barack_Obama\"]]"
     ]
    }
   ],
   "source": [
    "## Check using curl. (it is not nessasary to run the code)\n",
    "!curl -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' -d \"{ \\\"text\\\": \\\"Obama will visit Germany and have a meeting with Merkel tomorrow.\\\", \\\"spans\\\": []  }\" 'http://localhost:5555'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Utility Functions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pprint\n",
    "import pandas as pd\n",
    "from IPython.display import Markdown\n",
    "\n",
    "\n",
    "def query(text):\n",
    "    ## Takes the input string and passes it to the service and gets the reponse back.\n",
    "    myjson = { \"text\": text, \"spans\": []  }\n",
    "    r = requests.post(\"http://localhost:5555\", json=myjson)\n",
    "    return json.loads(r.content)\n",
    "\n",
    "def printmd(string):\n",
    "    ## displays the annotated/tagged input text in jupyter's Markdown format\n",
    "    display(Markdown(string))\n",
    "    \n",
    "    \n",
    "def format_index_output(text):\n",
    "    ## main function which sends the input text to the service, gets the response back and formats the output\n",
    "    ## in a presentable form to evaluate.\n",
    "    \n",
    "    ents = query(text)\n",
    "    ents.sort(key=lambda tup: tup[0], reverse=True)  \n",
    "    for i, ent in enumerate(ents):\n",
    "        text = text[:ent[0]] + '['+text[ent[0]:ent[0]+ent[1]]+'](https://en.wikipedia.org/wiki/'+ ent[2] +')' + text[ent[0]+ent[1]:]\n",
    "    \n",
    "    # Replace $ sign : Quick fix since $ sign is a keyword in jupyter markdowns\n",
    "    text = text.replace(\"$\",\"\\\\$\")\n",
    "    \n",
    "    printmd(text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "text = [\"Eminem is the best rapper of all time. MMLP was his best album, Eminem and Dre have produced this album\",\n",
    "        \"Pakistan has one of the best teams in cricket. The pakistani squad for ICC Cricket World Cup has Shahid Afridi and Shoaib Akhtar\",\n",
    "        \"KIEV: Separatist rebels have not fulfilled conditions like handing back border posts or laying down their weapons, Ukraine’s president said on Monday in a phone call with the leaders of Russia, Germany and France as he pondered whether to extend a ceasefire.The call between President Petro Poroshenko, Russia’s Vladimir Putin, Germany’s Chancellor Angela Merkel and France’s Francois Hollande took place as an expiration deadline neared for Ukraine’s shaky, unilateral ceasefire\"\n",
    "       ]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "[Eminem](https://en.wikipedia.org/wiki/Eminem) is the best rapper of all time. [MMLP](https://en.wikipedia.org/wiki/The_Marshall_Mathers_LP) was his best album, [Eminem](https://en.wikipedia.org/wiki/Eminem) and [Dre](https://en.wikipedia.org/wiki/Dr._Dre) have produced this album"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "Pakistan has one of the best teams in cricket. The pakistani squad for [ICC Cricket World Cup](https://en.wikipedia.org/wiki/Cricket_World_Cup) has [Shahid Afridi](https://en.wikipedia.org/wiki/Shahid_Afridi) and Shoaib Akhtar"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "[KIEV](https://en.wikipedia.org/wiki/Kiev): Separatist rebels have not fulfilled conditions like handing back border posts or laying down their weapons, [Ukraine](https://en.wikipedia.org/wiki/Ukraine)’s president said on Monday in a phone call with the leaders of [Russia](https://en.wikipedia.org/wiki/Russia), [Germany](https://en.wikipedia.org/wiki/Germany) and [France](https://en.wikipedia.org/wiki/France) as he pondered whether to extend a ceasefire.The call between President [Petro Poroshenko](https://en.wikipedia.org/wiki/Petro_Poroshenko), [Russia](https://en.wikipedia.org/wiki/Russia)’s [Vladimir Putin](https://en.wikipedia.org/wiki/Vladimir_Putin), [Germany](https://en.wikipedia.org/wiki/Germany)’s Chancellor [Angela Merkel](https://en.wikipedia.org/wiki/Angela_Merkel) and [France](https://en.wikipedia.org/wiki/France)’s [Francois Hollande](https://en.wikipedia.org/wiki/François_Hollande) took place as an expiration deadline neared for [Ukraine](https://en.wikipedia.org/wiki/Ukraine)’s shaky, unilateral ceasefire"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "for t in text:\n",
    "    format_index_output(t)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [conda env:end2end_nel]",
   "language": "python",
   "name": "conda-env-end2end_nel-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}