03_Recommendations_Part1.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {}
   },
   "source": [
    "# Recommendations: Part 1\n",
    "\n",
    "In this notebook you will learn how to make recommendations using Neo4j. \n",
    "\n",
    "Execute the code to import the libraries (remember to unset Reset all runtimes before running):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "pycharm": {}
   },
   "outputs": [],
   "source": [
    "from py2neo import Graph\n",
    "import pandas as pd\n",
    "\n",
    "import matplotlib \n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "plt.style.use('fivethirtyeight')\n",
    "pd.set_option('display.float_format', lambda x: '%.3f' % x)\n",
    "pd.set_option('display.max_colwidth', 100)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {}
   },
   "source": [
    "Update the cell below to use the IP Address, Bolt Port, and Password, as you did previously."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "pycharm": {}
   },
   "outputs": [],
   "source": [
    "# Change the line of code below to use the IP Address, Bolt Port, and Password of your Neo4j database instance.\n",
    "# graph = Graph(\"bolt://<IP Address>:<Bolt Port>\", auth=(\"neo4j\", \"<Password>\")) \n",
    " \n",
    "graph = Graph(\"bolt://localhost:7687\", auth=(\"neo4j\", \"letmein\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {}
   },
   "source": [
    "##  Finding popular authors\n",
    "\n",
    "Since you will be making collaborator suggestions find authors who have written the most articles so that you have some data to work with."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "pycharm": {}
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>author.name</th>\n",
       "      <th>articlesPublished</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Peter G. Neumann</td>\n",
       "      <td>89</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Peter J. Denning</td>\n",
       "      <td>80</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Moshe Y. Vardi</td>\n",
       "      <td>72</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Pamela Samuelson</td>\n",
       "      <td>71</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Bart Preneel</td>\n",
       "      <td>65</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Vinton G. Cerf</td>\n",
       "      <td>56</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Barry W. Boehm</td>\n",
       "      <td>53</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Mark Guzdial</td>\n",
       "      <td>49</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Edwin R. Hancock</td>\n",
       "      <td>47</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Josef Kittler</td>\n",
       "      <td>46</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        author.name  articlesPublished\n",
       "0  Peter G. Neumann                 89\n",
       "1  Peter J. Denning                 80\n",
       "2    Moshe Y. Vardi                 72\n",
       "3  Pamela Samuelson                 71\n",
       "4      Bart Preneel                 65\n",
       "5    Vinton G. Cerf                 56\n",
       "6    Barry W. Boehm                 53\n",
       "7      Mark Guzdial                 49\n",
       "8  Edwin R. Hancock                 47\n",
       "9     Josef Kittler                 46"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "popular_authors_query = \"\"\"\n",
    "MATCH (author:Author)\n",
    "RETURN author.name, size((author)<-[:AUTHOR]-()) AS articlesPublished\n",
    "ORDER BY articlesPublished DESC\n",
    "LIMIT 10\n",
    "\"\"\"\n",
    "\n",
    "graph.run(popular_authors_query).to_data_frame()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {}
   },
   "source": [
    "Pick one of these authors..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "pycharm": {}
   },
   "outputs": [],
   "source": [
    "author_name = \"Peter G. Neumann\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {}
   },
   "source": [
    "Retrieve the articles they've published and how many citations they've received:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "pycharm": {}
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>article</th>\n",
       "      <th>year</th>\n",
       "      <th>citations</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Security by obscurity</td>\n",
       "      <td>2003</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>The foresight saga, redux</td>\n",
       "      <td>2012</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Risks of automation: a cautionary total-system perspective of our cyberfuture</td>\n",
       "      <td>2016</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Robust open-source software</td>\n",
       "      <td>1999</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Are dependable systems feasible</td>\n",
       "      <td>1993</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Crypto policy perspectives</td>\n",
       "      <td>1994</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Risks of National Identity Cards</td>\n",
       "      <td>2001</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Information system security redux</td>\n",
       "      <td>2003</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>The foresight saga</td>\n",
       "      <td>2006</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Computers, ethics, and values</td>\n",
       "      <td>1991</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Identity-Ralated Misuse.</td>\n",
       "      <td>1997</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Alternative passwords</td>\n",
       "      <td>1994</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>Animal crackers</td>\n",
       "      <td>1993</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>Accidental financial losses</td>\n",
       "      <td>1992</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>Anticipating disasters</td>\n",
       "      <td>2005</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>Survivable systems</td>\n",
       "      <td>1992</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>Friendly fire</td>\n",
       "      <td>1994</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>Computers: boon or bane?</td>\n",
       "      <td>2001</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>Y2K update</td>\n",
       "      <td>1998</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>Responsibilities of technologists</td>\n",
       "      <td>2005</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                                          article  \\\n",
       "0                                                           Security by obscurity   \n",
       "1                                                       The foresight saga, redux   \n",
       "2   Risks of automation: a cautionary total-system perspective of our cyberfuture   \n",
       "3                                                     Robust open-source software   \n",
       "4                                                 Are dependable systems feasible   \n",
       "5                                                      Crypto policy perspectives   \n",
       "6                                                Risks of National Identity Cards   \n",
       "7                                               Information system security redux   \n",
       "8                                                              The foresight saga   \n",
       "9                                                   Computers, ethics, and values   \n",
       "10                                                       Identity-Ralated Misuse.   \n",
       "11                                                          Alternative passwords   \n",
       "12                                                                Animal crackers   \n",
       "13                                                    Accidental financial losses   \n",
       "14                                                         Anticipating disasters   \n",
       "15                                                             Survivable systems   \n",
       "16                                                                  Friendly fire   \n",
       "17                                                       Computers: boon or bane?   \n",
       "18                                                                     Y2K update   \n",
       "19                                              Responsibilities of technologists   \n",
       "\n",
       "    year  citations  \n",
       "0   2003          2  \n",
       "1   2012          2  \n",
       "2   2016          1  \n",
       "3   1999          1  \n",
       "4   1993          1  \n",
       "5   1994          1  \n",
       "6   2001          1  \n",
       "7   2003          1  \n",
       "8   2006          1  \n",
       "9   1991          1  \n",
       "10  1997          0  \n",
       "11  1994          0  \n",
       "12  1993          0  \n",
       "13  1992          0  \n",
       "14  2005          0  \n",
       "15  1992          0  \n",
       "16  1994          0  \n",
       "17  2001          0  \n",
       "18  1998          0  \n",
       "19  2005          0  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "author_articles_query = \"\"\"\n",
    "MATCH (:Author {name: $authorName})<-[:AUTHOR]-(article)\n",
    "RETURN article.title AS article, article.year AS year, size((article)<-[:CITED]-()) AS citations\n",
    "ORDER BY citations DESC\n",
    "LIMIT 20\n",
    "\"\"\"\n",
    "\n",
    "graph.run(author_articles_query,  {\"authorName\": author_name}).to_data_frame()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {}
   },
   "source": [
    "Find the author's collaborators:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "pycharm": {}
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>coauthor</th>\n",
       "      <th>collaborations</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Susan Landau</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Lauren Weinstein</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Whitfield Diffie</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Steven Michael Bellovin</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Rebecca T. Mercuri</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Matt Blaze</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Herbert S. Lin</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Alfred Z. Spector</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>David Lorge Parnas</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Lawrence Robinson</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                  coauthor  collaborations\n",
       "0             Susan Landau               3\n",
       "1         Lauren Weinstein               3\n",
       "2         Whitfield Diffie               3\n",
       "3  Steven Michael Bellovin               2\n",
       "4       Rebecca T. Mercuri               2\n",
       "5               Matt Blaze               2\n",
       "6           Herbert S. Lin               1\n",
       "7        Alfred Z. Spector               1\n",
       "8       David Lorge Parnas               1\n",
       "9        Lawrence Robinson               1"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "collaborations_query = \"\"\"\n",
    "MATCH (:Author {name: $authorName})<-[:AUTHOR]-(article)-[:AUTHOR]->(coauthor)\n",
    "RETURN coauthor.name AS coauthor, count(*) AS collaborations\n",
    "ORDER BY collaborations DESC\n",
    "LIMIT 10\n",
    "\"\"\"\n",
    "\n",
    "graph.run(collaborations_query,  {\"authorName\": author_name}).to_data_frame()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {}
   },
   "source": [
    "How would you suggest some future collaborators for this author? One way is by looking at the collaborators of their collaborators!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "pycharm": {}
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>coauthor</th>\n",
       "      <th>collaborations</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>John Ioannidis</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Scott Bradner</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Angelos D. Keromytis</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>John Kelsey</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Virgil D. Gligor</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Ran Canetti</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Peter Wolcott</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>William Aiello</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>David K. Gifford</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Gerald Jay Sussman</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "               coauthor  collaborations\n",
       "0        John Ioannidis              10\n",
       "1         Scott Bradner               9\n",
       "2  Angelos D. Keromytis               8\n",
       "3           John Kelsey               7\n",
       "4      Virgil D. Gligor               5\n",
       "5           Ran Canetti               4\n",
       "6         Peter Wolcott               4\n",
       "7        William Aiello               4\n",
       "8      David K. Gifford               4\n",
       "9    Gerald Jay Sussman               4"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "collaborations_query = \"\"\"\n",
    "MATCH (author:Author {name: $authorName})<-[:AUTHOR]-(article)-[:AUTHOR]->(coauthor),\n",
    "      (coauthor)<-[:AUTHOR]-()-[:AUTHOR]->(coc)\n",
    "WHERE not((coc)<-[:AUTHOR]-()-[:AUTHOR]->(author)) AND coc <> author      \n",
    "RETURN coc.name AS coauthor, count(*) AS collaborations\n",
    "ORDER BY collaborations DESC\n",
    "LIMIT 10\n",
    "\"\"\"\n",
    "\n",
    "graph.run(collaborations_query,  {\"authorName\": author_name}).to_data_frame()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {}
   },
   "source": [
    "Each of these people have collaborated with someone that Peter has worked with before, so they might be able to do an introduction."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {}
   },
   "source": [
    "## Exercise\n",
    "\n",
    "1. Can you find the top 20 suggested collaborators for 'Brian Fitzgerald' instead of 'Peter G. Neumann'?\n",
    "2. How many of these potential collaborators have collaborated with Brian's collaborators more than 3 times?\n",
    "\n",
    "Keep the results of this exercise handy as they may be useful for the quiz at the end of this module."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}