notebooks/affinity_tutorial.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Getting Started with Affinity"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook gives useful examples on how to make use of the Affinity endpoints in the Signal AI API.\n",
    "\n",
    "Full technical documentation of the Affinity endpoints are available here\n",
    "https://api.signal-ai.com/docs#tag/Concept-Affinity"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## What is Affinity?\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The Affinity API endpoints allow API users to leverage the power of the **Signal AI Knowledge Graph**, derived from billions of documents and updated regularly.\n",
    "\n",
    "The Signal AI Knowledge Graph consists of:\n",
    "\n",
    "- nodes which represent concepts such as entities and topics\n",
    "- edges represent connections describing relationships between these concepts\n",
    "\n",
    "Using the Affinity API, users can retrieve data for hundreds of thousands of entities and topics. The sole relationship type currently accessible via the Affinity API is `proximity`. The proximity between an entity (e.g. Tesla) and a topic (e.g. Product recall) is a measure of how they are related over a period of time (e.g. a certain month).  It reflects the likelihood of a salient mention of the entity with the topic.\n",
    "\n",
    "The proximity relationships can be used for discovery and comparison use cases as we will show in this notebook"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 0. Setting things up!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 0.1 Prerequisites"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Please make sure that you have familiarised yourself with the Signal AI API using the [Getting Started with the Search API notebook](getting_started.ipynb)\n",
    "\n",
    "In particular you would need to be able to use the following endpoints:\n",
    "\n",
    "1- Authentication: to be able to access the API\n",
    "\n",
    "2- Discovery: to be able to search for entities and topics of interest"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note: run `pip install -r requirements.txt` to install the dependencies for this notebook"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "import backoff\n",
    "import requests\n",
    "import os\n",
    "import pandas as pd\n",
    "import datetime"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 0.2 Check if authentication works! "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You will need a client_id and client_secret to gain access to the API. The code below will assume they have been set and the environment variables SIGNAL_API_CLIENT_ID and SIGNAL_API_CLIENT_SECRET respectively.\n",
    "\n",
    "Using your credentials you can request a temporary access token from the API using the url:\n",
    "https://api.signal-ai.com/auth/token"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "def authenticate(client_id, client_secret, url = \"https://api.signal-ai.com\"):\n",
    "    \"\"\" obtain a temporary access token using user credentials \"\"\"\n",
    "    token_url = f'{url}/auth/token'\n",
    "    payload = {\n",
    "        \"grant_type\": \"client_credentials\",\n",
    "        \"client_id\": client_id,\n",
    "        \"client_secret\": client_secret\n",
    "    }\n",
    "    response = requests.post(token_url, data=payload)\n",
    "    return response.json().get(\"access_token\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Congratulations! You have an access token, it will last for 24 hours before you will need to reauthenticate by repeating this step\n"
     ]
    }
   ],
   "source": [
    "TEMP_ACCESS_TOKEN = authenticate(os.environ['SIGNAL_API_ID'], os.environ['SIGNAL_API_SECRET'])\n",
    "if TEMP_ACCESS_TOKEN:\n",
    "    print('Congratulations! You have an access token, it will last for 24 hours before you will need to reauthenticate by repeating this step')\n",
    "else:\n",
    "    print('Error: Perhaps the credentials are incorrect?')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "## 1. Proximity Examples"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.1. Discovery: entities most associated with a topic (at a certain time)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Example use case:\n",
    "- Who are currently the main active organisations associated with a certain industry? \n",
    "\n",
    "\n",
    "How: \n",
    "Using the Affinity end points, rank organisations by their proximity score to a topic (representing the industry) in the last month.\n",
    "\n",
    "Note that you would need to have the ID of the topic as an input. For that please consult [Getting Started notebook](getting_started.ipynb) to search for topics and obtain the ID."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "## Use the topics end point as per the getting started notebook to search for the ids \n",
    "##  but we provide them here for convineince\n",
    "topics = {\n",
    "    'Wearables': 'f861d9df-5a65-41fe-8077-1714b838f1e1' \n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Example**: top 20 organisations in the Wearables industry in the last month"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "last_month = (pd.to_datetime(\"today\").to_period('M') - 1).strftime('%Y-%m')\n",
    "topic_id = topics['Wearables']\n",
    "response = requests.post(\n",
    "    # call the entities endpoint\n",
    "    'https://api.signal-ai.com/affinity',\n",
    "    json={\n",
    "        'source-concept': {\n",
    "            'id': topic_id\n",
    "        },\n",
    "        'relationship': {\n",
    "            'type': 'proximity',\n",
    "            'date': {\n",
    "                'start': last_month,\n",
    "                'end': last_month\n",
    "            },\n",
    "            'interval': 'month',\n",
    "            'limit-per-interval': 20\n",
    "        },\n",
    "        'target-concepts': {\n",
    "            'types': ['entity/organisation']\n",
    "        }\n",
    "    },\n",
    "    # include the access token in the header\n",
    "    headers={\n",
    "        \"Authorization\": f'Bearer {TEMP_ACCESS_TOKEN}',\n",
    "        \"Content-Type\": \"application/json\",\n",
    "    }\n",
    ")\n",
    "\n",
    "result = response.json()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>target-concept.name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Fitbit</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Garmin</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Omega SA</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>TAG Heuer</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Xiaomi</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Xiaomi Tech</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Mobvoi</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Apple Inc.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Huawei</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>The Swatch Group</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Ray-Ban</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Google</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>WHOOP</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>Nippon Electric Glass</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>Fuyao</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>Fossil Group</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>Polar Electro</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>Nest Labs</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>Nippon Sheet Glass</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>Withings</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       target-concept.name\n",
       "0                   Fitbit\n",
       "1                   Garmin\n",
       "2                 Omega SA\n",
       "3                TAG Heuer\n",
       "4                   Xiaomi\n",
       "5              Xiaomi Tech\n",
       "6                   Mobvoi\n",
       "7               Apple Inc.\n",
       "8                   Huawei\n",
       "9         The Swatch Group\n",
       "10                 Ray-Ban\n",
       "11                  Google\n",
       "12                   WHOOP\n",
       "13  Nippon Electric Glass \n",
       "14                   Fuyao\n",
       "15            Fossil Group\n",
       "16           Polar Electro\n",
       "17               Nest Labs\n",
       "18      Nippon Sheet Glass\n",
       "19                Withings"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.json_normalize(result['results'])\n",
    "df[['target-concept.name']]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.2. Discovery: topics most associated with an entity (at a certain time)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Example use case:\n",
    "- What are currently the my brand is most associated with?\n",
    "\n",
    "How: using the Affinity end points, rank topics by their proximity score to a certain organisation, also retrieve the scores to generate a visualisation\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Example**: top 10 topics associated to `Tesla, Inc.` in the last month"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "## Use the entities end point as per the getting started notebook to search for the ids \n",
    "##  but we provide them here for convineince\n",
    "entities = {\n",
    "    'Tesla, Inc.': '11cab8df-4be1-470f-8f49-8f7f0863ec95',\n",
    "    'General Motors': 'a9cf01c5-751f-4fe5-a529-12e0d297cb63', \n",
    "    'Mercedes-Benz': '8d9ee12f-f4f2-3dc7-a8b9-673a07bd7747', \n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "entity_name = 'Tesla, Inc.'\n",
    "entity_id = entities[entity_name]\n",
    "response = requests.post(\n",
    "    # call the entities endpoint\n",
    "    'https://api.signal-ai.com/affinity',\n",
    "    json={\n",
    "        'source-concept': {\n",
    "            'id': entity_id\n",
    "        },\n",
    "        'relationship': {\n",
    "            'type': 'proximity',\n",
    "            'date': {\n",
    "                'start': last_month,\n",
    "                'end': last_month\n",
    "            },\n",
    "            'interval': 'month',\n",
    "            'limit-per-interval': 10\n",
    "        }\n",
    "    },\n",
    "    # include the access token in the header\n",
    "    headers={\n",
    "        \"Authorization\": f'Bearer {TEMP_ACCESS_TOKEN}',\n",
    "        \"Content-Type\": \"application/json\",\n",
    "    }\n",
    ")\n",
    "\n",
    "result = response.json()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>target-concept.name</th>\n",
       "      <th>relationship.proximity-score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Electric Vehicles (EV)</td>\n",
       "      <td>0.660930</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Futures of Transport</td>\n",
       "      <td>0.656709</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Cleantech</td>\n",
       "      <td>0.607097</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Automotive Industry</td>\n",
       "      <td>0.601650</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Lidar</td>\n",
       "      <td>0.584335</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Touch Free Technology</td>\n",
       "      <td>0.529220</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Corporate Controversy</td>\n",
       "      <td>0.492055</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Accounting Irregularities</td>\n",
       "      <td>0.483460</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Transport</td>\n",
       "      <td>0.470326</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Autonomous Vehicles</td>\n",
       "      <td>0.466495</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         target-concept.name  relationship.proximity-score\n",
       "0     Electric Vehicles (EV)                      0.660930\n",
       "1       Futures of Transport                      0.656709\n",
       "2                  Cleantech                      0.607097\n",
       "3        Automotive Industry                      0.601650\n",
       "4                      Lidar                      0.584335\n",
       "5      Touch Free Technology                      0.529220\n",
       "6      Corporate Controversy                      0.492055\n",
       "7  Accounting Irregularities                      0.483460\n",
       "8                  Transport                      0.470326\n",
       "9        Autonomous Vehicles                      0.466495"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.json_normalize(result['results'])\n",
    "df[['target-concept.name', 'relationship.proximity-score']]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Top Articles driving the associations**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "we can also use the search endpoint to identify the top headlines around the coverage for these topics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Top articles on Tesla, Inc. and Electric Vehicles (EV) in 2022-03\n",
      "1: (SINA) Panasonic announced the start of mass production of Tesla 4680 batteries in fiscal year 2023 https://auto.sina.cn/zz/hy/2022-03-01/detail-imcwiwss3421611.d.html\n",
      "2: (TMT Post) Musk's super factory \"Hydrangea”, to whom is it thrown? https://www.tmtpost.com/6022700.html\n",
      "3: (News Explorer) Honda S2000 Swaps Screaming VTEC For Deadly Tesla Electric Motor https://newsexplorer.net/honda-s2000-swaps-screaming-vtec-for-deadly-tesla-electric-motor-s1041239.html\n",
      "\n",
      "Top articles on Tesla, Inc. and Futures of Transport in 2022-03\n",
      "1: (Eetop) Tesla launches fully autonomous driving test version in Canada https://auto.163.com/22/0301/07/H1BUCJTA000884MM.html\n",
      "2: (SINA) Panasonic announced the start of mass production of Tesla 4680 batteries in fiscal year 2023 https://auto.sina.cn/zz/hy/2022-03-01/detail-imcwiwss3421611.d.html\n",
      "3: (TMT Post) Musk's super factory \"Hydrangea”, to whom is it thrown? https://www.tmtpost.com/6022700.html\n",
      "\n",
      "Top articles on Tesla, Inc. and Cleantech in 2022-03\n",
      "1: (SINA) Panasonic announced the start of mass production of Tesla 4680 batteries in fiscal year 2023 https://auto.sina.cn/zz/hy/2022-03-01/detail-imcwiwss3421611.d.html\n",
      "2: (TMT Post) Musk's super factory \"Hydrangea”, to whom is it thrown? https://www.tmtpost.com/6022700.html\n",
      "3: (Tech Register) Tesla shares rally on expectations German factory will open in March https://www.techregister.co.uk/tesla-shares-rally-on-expectations-german-factory-will-open-in-march/\n",
      "\n"
     ]
    }
   ],
   "source": [
    "## most relevant articles for the top 3 topics\n",
    "## change depth to do more than 3\n",
    "depth = 3\n",
    "\n",
    "last_day_prev_month = pd.to_datetime(\"today\").replace(day=1)- datetime.timedelta(days=1)\n",
    "first_day_prev_month = last_day_prev_month.replace(day=1)\n",
    "\n",
    "for i in range(depth):    \n",
    "    query = {\n",
    "        'where': {\n",
    "            \"published-at\": {\n",
    "                \"gte\": first_day_prev_month.strftime('%Y-%m-%d'), \n",
    "                \"lte\": last_day_prev_month.strftime('%Y-%m-%d')\n",
    "            },\n",
    "            'topics': {\n",
    "                'id': {\n",
    "                    'eq': df.iloc[i]['target-concept.id']\n",
    "                },\n",
    "            },\n",
    "            'entities': {\n",
    "                'id': {\n",
    "                    'eq': entities['Tesla, Inc.'] \n",
    "                },\n",
    "                'salient-only':True,\n",
    "            }\n",
    "        },\n",
    "        'sort': [['score', 'desc']],\n",
    "        'size': 3\n",
    "    }\n",
    "    response = requests.post(\n",
    "        'https://api.signal-ai.com/search',\n",
    "        json=query,\n",
    "        headers={\n",
    "            \"Authorization\": f'Bearer {TEMP_ACCESS_TOKEN}',\n",
    "            \"Content-Type\": \"application/json\",\n",
    "        },\n",
    "    )\n",
    "    documents = response.json()['documents']\n",
    "    print('Top articles on {} and {} in {}'.format(\n",
    "        entity_name,\n",
    "        df.iloc[i]['target-concept.name'],\n",
    "        df.iloc[i]['relationship.date']        \n",
    "    ))\n",
    "    for i, doc in enumerate(documents):\n",
    "        print('{}: ({}) {} {}'.format(i+1, doc['source']['name'], doc['title'], doc.get('url')))\n",
    "    print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "### 1.3. Comparison: proximity of different entities to a certain topic"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Example use case:\n",
    "- What is my association to a certain like compared to my competitors? Whose association to X is stronger/weaker?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Example**: How do these car manufacturers: `Mercedez-Benz`, `General Motors`, `Tesla, Inc.` compare  with regards to their association to the `Product Recall`topic in the last month?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [],
   "source": [
    "topic_id = '734a342f-a053-4823-9bda-8abb687182ba' ##  ID for Product Recall\n",
    "response = requests.post(\n",
    "    # call the entities endpoint\n",
    "    'https://api.signal-ai.com/affinity',\n",
    "    json={\n",
    "        'source-concept': {\n",
    "            'id': topic_id\n",
    "        },\n",
    "        'relationship': {\n",
    "            'type': 'proximity',\n",
    "            'date': {\n",
    "                'start': last_month,\n",
    "                'end': last_month\n",
    "            },\n",
    "            'interval': 'month'\n",
    "        },\n",
    "        'target-concepts': {\n",
    "            'ids': [\n",
    "                entities['Mercedes-Benz'],\n",
    "                entities['General Motors'],\n",
    "                entities['Tesla, Inc.'],\n",
    "            ]\n",
    "        }\n",
    "    },\n",
    "    # include the access token in the header\n",
    "    headers={\n",
    "        \"Authorization\": f'Bearer {TEMP_ACCESS_TOKEN}',\n",
    "        \"Content-Type\": \"application/json\",\n",
    "    }\n",
    ")\n",
    "\n",
    "result = response.json()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.json_normalize(result['results'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>target-concept.name</th>\n",
       "      <th>relationship.proximity-score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>General Motors</td>\n",
       "      <td>0.527692</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Mercedes-Benz</td>\n",
       "      <td>0.422225</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Tesla, Inc.</td>\n",
       "      <td>0.380587</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  target-concept.name  relationship.proximity-score\n",
       "1      General Motors                      0.527692\n",
       "0       Mercedes-Benz                      0.422225\n",
       "2         Tesla, Inc.                      0.380587"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.sort_values(by='relationship.proximity-score', ascending=False)[['target-concept.name','relationship.proximity-score']]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The proximity scores retrieved from Affinity allow us to order these organisations by their association to the topics. However, it should be noted that currently it is not possible to interpret the differences as the scale for proximity scores is not linear. Alternatively one can use the metrics API to examine the main drivers for the proximity scores in a particular month: (number of salient co-mentions of the entity with the topic, total number of salient mentions of the entity and the total number of mentions of the topic)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.4. Comparison: proximity of an entities to a topic over time"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Example use case:\n",
    "- How is my/their association with certain topics changing over time? Is my association now stronger/weaker"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Example**: How does `Tesla, Inc.`'s association to the `Product Recall` topic changing over the past year?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [],
   "source": [
    "twelve_months_ago = (\n",
    "    pd.to_datetime(\"today\").replace(day=1)- datetime.timedelta(days=365)\n",
    ").strftime('%Y-%m')\n",
    "response = requests.post(\n",
    "    # call the entities endpoint\n",
    "    'https://api.signal-ai.com/affinity',\n",
    "    json={\n",
    "        'source-concept': {\n",
    "            'id': topic_id\n",
    "        },\n",
    "        'relationship': {\n",
    "            'type': 'proximity',\n",
    "            'date': {\n",
    "                'start': twelve_months_ago,\n",
    "                'end': last_month\n",
    "            },\n",
    "            'interval': 'month'\n",
    "        },\n",
    "        'target-concepts': {\n",
    "            'ids': [\n",
    "                entities['Tesla, Inc.'],\n",
    "            ]\n",
    "        }\n",
    "    },\n",
    "    # include the access token in the header\n",
    "    headers={\n",
    "        \"Authorization\": f'Bearer {TEMP_ACCESS_TOKEN}',\n",
    "        \"Content-Type\": \"application/json\",\n",
    "    }\n",
    ")\n",
    "\n",
    "result = response.json()\n",
    "df = pd.json_normalize(result['results'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>relationship.date</th>\n",
       "      <th>relationship.proximity-score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2021-04</td>\n",
       "      <td>0.266264</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2021-05</td>\n",
       "      <td>0.302764</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2021-06</td>\n",
       "      <td>0.635515</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2021-07</td>\n",
       "      <td>0.262505</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2021-08</td>\n",
       "      <td>0.234020</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>2021-09</td>\n",
       "      <td>0.233576</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>2021-10</td>\n",
       "      <td>0.375716</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>2021-11</td>\n",
       "      <td>0.479871</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>2021-12</td>\n",
       "      <td>0.619636</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>2022-01</td>\n",
       "      <td>0.464340</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>2022-02</td>\n",
       "      <td>0.686569</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>2022-03</td>\n",
       "      <td>0.380587</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   relationship.date  relationship.proximity-score\n",
       "0            2021-04                      0.266264\n",
       "1            2021-05                      0.302764\n",
       "2            2021-06                      0.635515\n",
       "3            2021-07                      0.262505\n",
       "4            2021-08                      0.234020\n",
       "5            2021-09                      0.233576\n",
       "6            2021-10                      0.375716\n",
       "7            2021-11                      0.479871\n",
       "8            2021-12                      0.619636\n",
       "9            2022-01                      0.464340\n",
       "10           2022-02                      0.686569\n",
       "11           2022-03                      0.380587"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[['relationship.date', 'relationship.proximity-score']]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Like before, the proximity scores retrieved from Affinity allow us to compare the associations to the topic month-on-month. However, it should be noted that currently it is not possible to interpret the differences as the scale for proximity scores is not linear. Alternatively one can use the metrics API to examine the main drivers for the proximity scores in a particular month: (number of salient co-mentions of the entity with the topic, total number of salient mentions of the entity and the total number of mentions of the topic)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}