tools/6_recognise_instructions_detect_follow-up_actions.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Extracts verbalised instruction such as \"connect Mount Basel to Montreux\", and pairs them with the follow-up action that may *match* (e.g. if the other connects Basel to Montreux) or *mismatch* (e.g. if the other connects Basel to Neuchatel) with the instruction."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook produces an annotated corpus with one tab-separated text file per team at [annotated_corpus/](../annotated_corpus/).\n",
    "\n",
    "In particular, the columns are:\n",
    "\n",
    "* *team_no*: The number of the team that the event belongs to\n",
    "* *attempt_no*: The attempt number that the event belongs to, starting from 1. An attempt is the duration of the team constructing a solution and submitting it together.\n",
    "* *turn_no*: The turn number of the event, starting from 1. A turn is the duration where one of the participants is in the figurative view, and the other is in the abstract view.\n",
    "* *event_no*: The event number of the event, starting from 1\n",
    "* *start*: The start timestamp of the event (in seconds), from the beginning of the activity\n",
    "* *end*: The end timestamp of the event (in seconds)\n",
    "* *subject*: The subject that the event is executed by (*A*, *B*: the participants; *R*: the robot; *T*: the team)\n",
    "* *verb*: The verb that describes the event (e.g. \"presses\", \"adds\", \"removes\")\n",
    "* *object*: The object that is acted on by the subject performing the verb (e.g. \"submit (enabled)\" for subject i.e. participant: *A*, verb i.e. the action: \"presses\")\n",
    "* *instructions*: the list of instructions that are inferred for this event\n",
    "* *pending_instructions*: the list of instructions that are pending/cached to be matched with an action\n",
    "* *matching*: the result matching of the pending instructions with the action (if this event is an action)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pickle\n",
    "\n",
    "import pathlib as pl\n",
    "import pandas as pd\n",
    "\n",
    "import spacy\n",
    "from spacy import displacy\n",
    "from spacy.pipeline import EntityRuler\n",
    "\n",
    "from read_utils import read_tables, read_network\n",
    "from match_utils import \\\n",
    "    Instruction, Do, \\\n",
    "    Match, Mismatch, Nonmatch, \\\n",
    "    make_edit_action"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Define paths."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Created ../processed_data/annotated_corpus\n"
     ]
    }
   ],
   "source": [
    "# Inputs.\n",
    "data_dir = pl.Path('../data')\n",
    "network_file = data_dir.joinpath('metadata/network.json')\n",
    "\n",
    "processed_data_dir = pl.Path('../processed_data')\n",
    "corpus_dir = processed_data_dir.joinpath('corpus')\n",
    "\n",
    "# # Outputs.\n",
    "annot_corpus_dir = processed_data_dir.joinpath('annotated_corpus')\n",
    "annot_corpus_pickle_file = annot_corpus_dir.joinpath(\n",
    "    'justhing19_annotated_corpus.pickle')\n",
    "\n",
    "for d in [annot_corpus_dir]:\n",
    "    if not d.exists():\n",
    "        d.mkdir()\n",
    "        print('Created {}'.format(d))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load corpus tables (logs with transcripts)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Reading transcript files from ../processed_data/corpus.\n",
      "transcript 10 files found.\n",
      "File justhink19_corpus_07 belongs to team  7\n",
      "File justhink19_corpus_08 belongs to team  8\n",
      "File justhink19_corpus_09 belongs to team  9\n",
      "File justhink19_corpus_10 belongs to team 10\n",
      "File justhink19_corpus_11 belongs to team 11\n",
      "File justhink19_corpus_17 belongs to team 17\n",
      "File justhink19_corpus_18 belongs to team 18\n",
      "File justhink19_corpus_20 belongs to team 20\n",
      "File justhink19_corpus_28 belongs to team 28\n",
      "File justhink19_corpus_47 belongs to team 47\n",
      "Transcript of  7 has 1059 utterances\n",
      "Transcript of  8 has  932 utterances\n",
      "Transcript of  9 has 1076 utterances\n",
      "Transcript of 10 has  769 utterances\n",
      "Transcript of 11 has  910 utterances\n",
      "Transcript of 17 has  451 utterances\n",
      "Transcript of 18 has  506 utterances\n",
      "Transcript of 20 has  653 utterances\n",
      "Transcript of 28 has  490 utterances\n",
      "Transcript of 47 has  642 utterances\n"
     ]
    }
   ],
   "source": [
    "corpus_dfs = read_tables(corpus_dir, form='transcript')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load the background network."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Network read from : 10 nodes, 20 edges\n"
     ]
    }
   ],
   "source": [
    "network = read_network(network_file)\n",
    "print('Network read from {}: {} nodes, {} edges'.format(\n",
    "    network, network.number_of_nodes(), network.number_of_edges()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prepare parsers and rulers."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Parser for edge objects in the extended transcripts."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[2, 8]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def parse_edge_object(obj, names=False):\n",
    "    ''''Parses edit event object string to its node ids\n",
    "    e.g. Zurich-Gallen (2-8)' to [2, 8]'''\n",
    "    if names:  # Parse for names.\n",
    "        (u, v) = obj.split()[0].split('-')\n",
    "    else:  # Parse for node indices.\n",
    "        (u, v) = obj.split()[1].strip('(').strip(')').split('-')\n",
    "        u = int(u)\n",
    "        v = int(v)\n",
    "    return [u, v]\n",
    "\n",
    "\n",
    "# Try\n",
    "obj = 'Zurich-Gallen (2-8)'\n",
    "parse_edge_object(obj)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Define keywords to detect instuction entities."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ADD {'build', 'do', 'go', 'connect', 'add', 'put'}\n",
      "REMOVE {'delete', 'away', 'rub', 'cut', 'remove', 'erase'}\n"
     ]
    }
   ],
   "source": [
    "entity_keywords = {\n",
    "    'ADD': {\n",
    "        'add',    # \"adding zurich to bern .\"\n",
    "        'do',\n",
    "        'go',\n",
    "        'put',    # \"putting that one there\"\n",
    "        'connect',\n",
    "        'build',  # \"i'll build mount luzern to zermatt\"\n",
    "    },\n",
    "    'REMOVE': {\n",
    "        'remove',\n",
    "        \"delete\",  # \"okay so delete that .\"\n",
    "        'erase',\n",
    "        'cut',    # 'yeah then cut out mount basel to mount interlaken .'\n",
    "        'away',   # 'take away',\n",
    "        'rub',    # 'rub that out',\n",
    "        # as in \"it's 3 francs rub that out\" ; \n",
    "        # \"no wait let me rub that out again .\" ; \n",
    "        # \"oh then rub that out\"\n",
    "    },\n",
    "}\n",
    "\n",
    "for k, words in entity_keywords.items():\n",
    "    print(k, words)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Define function to recognise instructions from an utterance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[ADD(9,?)]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<span class=\"tex2jax_ignore\"><div class=\"entities\" style=\"line-height: 2.5; direction: ltr\">no lets \n",
       "<mark class=\"entity\" style=\"background: #ddd; padding: 0.45em 0.6em; margin: 0 0.25em; line-height: 1; border-radius: 0.35em;\">\n",
       "    do\n",
       "    <span style=\"font-size: 0.8em; font-weight: bold; line-height: 1; border-radius: 0.35em; text-transform: uppercase; vertical-align: middle; margin-left: 0.5rem\">ADD</span>\n",
       "</mark>\n",
       " mount \n",
       "<mark class=\"entity\" style=\"background: #ddd; padding: 0.45em 0.6em; margin: 0 0.25em; line-height: 1; border-radius: 0.35em;\">\n",
       "    davos\n",
       "    <span style=\"font-size: 0.8em; font-weight: bold; line-height: 1; border-radius: 0.35em; text-transform: uppercase; vertical-align: middle; margin-left: 0.5rem\">NODE</span>\n",
       "</mark>\n",
       " to , where \n",
       "<mark class=\"entity\" style=\"background: #ddd; padding: 0.45em 0.6em; margin: 0 0.25em; line-height: 1; border-radius: 0.35em;\">\n",
       "    do\n",
       "    <span style=\"font-size: 0.8em; font-weight: bold; line-height: 1; border-radius: 0.35em; text-transform: uppercase; vertical-align: middle; margin-left: 0.5rem\">ADD</span>\n",
       "</mark>\n",
       " you wanna \n",
       "<mark class=\"entity\" style=\"background: #ddd; padding: 0.45em 0.6em; margin: 0 0.25em; line-height: 1; border-radius: 0.35em;\">\n",
       "    go\n",
       "    <span style=\"font-size: 0.8em; font-weight: bold; line-height: 1; border-radius: 0.35em; text-transform: uppercase; vertical-align: middle; margin-left: 0.5rem\">ADD</span>\n",
       "</mark>\n",
       " ?</div></span>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def prepare_ruler(network, entity_keywords):\n",
    "    nlp = spacy.load(\"en_core_web_sm\", disable=[\"ner\"])\n",
    "    ruler = EntityRuler(nlp)\n",
    "\n",
    "    node_ids = list()\n",
    "    node_patterns = list()\n",
    "    for u, d in network.nodes(data=True):\n",
    "        word = d['label'].split()[-1]\n",
    "        identifier = str(u)\n",
    "        pattern = {'id': identifier, 'label': 'NODE', \"pattern\": [\n",
    "            {'LOWER': word.lower()}]}\n",
    "        node_patterns.append(pattern)\n",
    "        node_ids.append(identifier)\n",
    "\n",
    "    entity_ids = list()\n",
    "    entity_patterns = list()\n",
    "    for label, words in entity_keywords.items():\n",
    "        for word in words:\n",
    "            identifier = str(label) \n",
    "            pattern = {'id': identifier, 'label': label, \"pattern\": [\n",
    "                {'LOWER': word.lower()}]}\n",
    "            entity_patterns.append(pattern)\n",
    "            entity_ids.append(identifier)\n",
    "\n",
    "    patterns = [\n",
    "        *node_patterns,\n",
    "        *entity_patterns,\n",
    "    ]\n",
    "    ruler.add_patterns(patterns)\n",
    "    nlp.add_pipe(ruler)\n",
    "\n",
    "    return nlp, node_ids, entity_ids\n",
    "\n",
    "\n",
    "def get_node_ids(text, doc):\n",
    "    node_ids = [int(ent.ent_id_) for ent in doc.ents if ent.label_ == 'NODE']\n",
    "    return node_ids\n",
    "\n",
    "\n",
    "def recognise_instructions(text, node_ids, entity_ids, nlp,\n",
    "                           default_entity_id='ADD'):\n",
    "    '''Possible intended actions as entities.'''\n",
    "    doc = nlp(text)\n",
    "\n",
    "    instructions = list()\n",
    "    # an instruction template [add/remove, [node1, node2]]\n",
    "    template = [None, [None, None]]  \n",
    "    for ent in doc.ents:\n",
    "        ent_id = ent.ent_id_\n",
    "\n",
    "        if ent_id in entity_ids:\n",
    "            if template[0] is None:\n",
    "                template[0] = ent_id\n",
    "            else: # Begin inferring a new instruction.\n",
    "                # Add the currently inferred instruction to the list.\n",
    "                if template[1][0] is not None:\n",
    "                    instruction = make_edit_action(template[0], template[1])\n",
    "                    if instruction is not None:\n",
    "                        instructions.append(instruction)\n",
    "                # Create a new instruction.\n",
    "                template = [ent_id, [None, None]]\n",
    "\n",
    "        elif ent_id in node_ids:\n",
    "            if template[1][0] is None:\n",
    "                template[1][0] = ent_id\n",
    "            elif template[1][1] is None:\n",
    "                if ent_id != template[1][0]:\n",
    "                    template[1][1] = ent_id\n",
    "                # Start a new entity if has a verb.\n",
    "                # Default to add or the previous\n",
    "                if template[0] is None:\n",
    "                    if len(instructions) > 0:\n",
    "                        template[0] = instructions[-1].name\n",
    "                    else:\n",
    "                        template[0] = default_entity_id\n",
    "\n",
    "                instruction = make_edit_action(template[0], template[1])\n",
    "                if instruction is not None:\n",
    "                    instructions.append(instruction)\n",
    "                template = [None, [None, None]]\n",
    "\n",
    "    if template[1][0] is not None:\n",
    "        if template[0] is None:  # assume adds if otherwise detected\n",
    "            template[0] = 'ADD'\n",
    "        instruction = make_edit_action(template[0], template[1])\n",
    "        if instruction is not None:\n",
    "            instructions.append(instruction)\n",
    "\n",
    "    return instructions\n",
    "\n",
    "\n",
    "# Try\n",
    "text = \"go from basel to zurich and then from zurich to saint gallen .\"\n",
    "text = \"then rub that out and then go , interlaken to mount bern .\"\n",
    "text = \"okay rub it out and go bern to interlaken .\"\n",
    "text = 'is that how much that ?'\n",
    "text = \"how do i get off this screen ?\"\n",
    "text = \"go from basel to zurich and then from zurich to saint gallen .\"\n",
    "text = 'to mount davos .'\n",
    "text = \"then rub that out and then go , interlaken .\"\n",
    "text = \"no lets do mount davos to , where do you wanna go ?\"\n",
    "nlp, node_ids, entity_ids = prepare_ruler(network, entity_keywords)\n",
    "\n",
    "\n",
    "entities = recognise_instructions(text, node_ids, entity_ids, nlp)\n",
    "display(entities)\n",
    "\n",
    "doc = nlp(text)\n",
    "displacy.render(doc, style=\"ent\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Define function to process a row or a table to recognise instructions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>team_no</th>\n",
       "      <th>attempt_no</th>\n",
       "      <th>turn_no</th>\n",
       "      <th>utterance_no</th>\n",
       "      <th>start</th>\n",
       "      <th>end</th>\n",
       "      <th>subject</th>\n",
       "      <th>verb</th>\n",
       "      <th>object</th>\n",
       "      <th>nodes</th>\n",
       "      <th>instructions</th>\n",
       "      <th>matching</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>28</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>-1</td>\n",
       "      <td>0.296</td>\n",
       "      <td>0.296</td>\n",
       "      <td>R</td>\n",
       "      <td>shows</td>\n",
       "      <td>observe gesture</td>\n",
       "      <td>[]</td>\n",
       "      <td>[]</td>\n",
       "      <td>[]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>28</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>-1</td>\n",
       "      <td>0.365</td>\n",
       "      <td>0.365</td>\n",
       "      <td>R</td>\n",
       "      <td>says</td>\n",
       "      <td>so, ann and bob, let's start building the trac...</td>\n",
       "      <td>[]</td>\n",
       "      <td>[]</td>\n",
       "      <td>[]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>28</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>-1</td>\n",
       "      <td>33.409</td>\n",
       "      <td>33.409</td>\n",
       "      <td>A</td>\n",
       "      <td>presses</td>\n",
       "      <td>help (enabled)</td>\n",
       "      <td>[]</td>\n",
       "      <td>[]</td>\n",
       "      <td>[]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>28</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>40.000</td>\n",
       "      <td>41.161</td>\n",
       "      <td>A</td>\n",
       "      <td>says</td>\n",
       "      <td>okay , so</td>\n",
       "      <td>[]</td>\n",
       "      <td>[]</td>\n",
       "      <td>[]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>28</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>40.580</td>\n",
       "      <td>45.036</td>\n",
       "      <td>B</td>\n",
       "      <td>says</td>\n",
       "      <td>so we have to connect all the places with trac...</td>\n",
       "      <td>[]</td>\n",
       "      <td>[]</td>\n",
       "      <td>[]</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   team_no  attempt_no  turn_no  utterance_no   start     end subject  \\\n",
       "0       28           1        1            -1   0.296   0.296       R   \n",
       "1       28           1        1            -1   0.365   0.365       R   \n",
       "2       28           1        1            -1  33.409  33.409       A   \n",
       "3       28           1        1             0  40.000  41.161       A   \n",
       "4       28           1        1             1  40.580  45.036       B   \n",
       "\n",
       "      verb                                             object nodes  \\\n",
       "0    shows                                    observe gesture    []   \n",
       "1     says  so, ann and bob, let's start building the trac...    []   \n",
       "2  presses                                     help (enabled)    []   \n",
       "3     says                                          okay , so    []   \n",
       "4     says  so we have to connect all the places with trac...    []   \n",
       "\n",
       "  instructions matching  \n",
       "0           []       []  \n",
       "1           []       []  \n",
       "2           []       []  \n",
       "3           []       []  \n",
       "4           []       []  "
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def recognise_instructions_for_row(sbj, verb, obj, nlp, node_ids, entity_ids):\n",
    "    '''make an instruct act or an edit act'''\n",
    "\n",
    "    # Make an node list and instruction list.\n",
    "    if verb == 'says' and sbj in ['A', 'B']:\n",
    "        text = obj\n",
    "        nodes = get_node_ids(text, nlp(text))\n",
    "        instructions = recognise_instructions(text, node_ids, entity_ids, nlp)\n",
    "        if instructions is None:\n",
    "            instructions = []\n",
    "    else:\n",
    "        nodes = []\n",
    "        instructions = []\n",
    "\n",
    "    # Make an act list.\n",
    "    acts = list()\n",
    "    for instruction in instructions:\n",
    "        act = Instruction(instruction, agent=sbj)\n",
    "        acts.append(act)\n",
    "\n",
    "    if verb == 'adds':\n",
    "        act_verb = 'ADD'\n",
    "    elif verb == 'removes':\n",
    "        act_verb = 'REMOVE'\n",
    "    else:\n",
    "        act_verb = None\n",
    "    if act_verb is not None:\n",
    "        action = make_edit_action(act_verb, parse_edge_object(obj))\n",
    "        act = Do(action, sbj)\n",
    "        acts.append(act)\n",
    "\n",
    "    return nodes, instructions, acts\n",
    "\n",
    "\n",
    "def recognise_instructions_for_table(df, network, entity_keywords, inplace=False):\n",
    "    if not inplace:\n",
    "        df = df.copy()\n",
    "\n",
    "    nlp, node_ids, entity_ids = prepare_ruler(network, entity_keywords)\n",
    "\n",
    "    node_lists, instruction_lists, act_lists = list(), list(), list()\n",
    "    for i, row in df.iterrows():\n",
    "        nodes, instructions, acts = recognise_instructions_for_row(\n",
    "            row['subject'], row['verb'], row['object'],\n",
    "            nlp, node_ids, entity_ids)\n",
    "\n",
    "        node_lists.append(nodes)\n",
    "        instruction_lists.append(instructions)\n",
    "        act_lists.append(acts)\n",
    "\n",
    "    df['nodes'] = node_lists\n",
    "    df['instructions'] = instruction_lists\n",
    "    df['matching'] = act_lists\n",
    "\n",
    "    return df\n",
    "\n",
    "\n",
    "# Try:\n",
    "team_no = 28\n",
    "df = corpus_dfs[team_no].copy()\n",
    "recognise_instructions_for_table(df, network, entity_keywords).head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Define function to match instructions and actions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>team_no</th>\n",
       "      <th>attempt_no</th>\n",
       "      <th>turn_no</th>\n",
       "      <th>utterance_no</th>\n",
       "      <th>start</th>\n",
       "      <th>end</th>\n",
       "      <th>subject</th>\n",
       "      <th>verb</th>\n",
       "      <th>object</th>\n",
       "      <th>nodes</th>\n",
       "      <th>instructions</th>\n",
       "      <th>matching</th>\n",
       "      <th>pending_instructions</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>-1</td>\n",
       "      <td>0.204</td>\n",
       "      <td>0.204</td>\n",
       "      <td>R</td>\n",
       "      <td>shows</td>\n",
       "      <td>observe gesture</td>\n",
       "      <td>[]</td>\n",
       "      <td>[]</td>\n",
       "      <td>[]</td>\n",
       "      <td>[]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>-1</td>\n",
       "      <td>0.334</td>\n",
       "      <td>0.334</td>\n",
       "      <td>R</td>\n",
       "      <td>says</td>\n",
       "      <td>so, ann and bob, let's start building the trac...</td>\n",
       "      <td>[]</td>\n",
       "      <td>[]</td>\n",
       "      <td>[]</td>\n",
       "      <td>[]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>-1</td>\n",
       "      <td>53.525</td>\n",
       "      <td>53.525</td>\n",
       "      <td>R</td>\n",
       "      <td>shows</td>\n",
       "      <td>thinking gesture</td>\n",
       "      <td>[]</td>\n",
       "      <td>[]</td>\n",
       "      <td>[]</td>\n",
       "      <td>[]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>-1</td>\n",
       "      <td>53.544</td>\n",
       "      <td>53.544</td>\n",
       "      <td>R</td>\n",
       "      <td>says</td>\n",
       "      <td>hmm, i see.</td>\n",
       "      <td>[]</td>\n",
       "      <td>[]</td>\n",
       "      <td>[]</td>\n",
       "      <td>[]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>-1</td>\n",
       "      <td>56.192</td>\n",
       "      <td>56.192</td>\n",
       "      <td>A</td>\n",
       "      <td>adds</td>\n",
       "      <td>Zermatt-Davos (4-9)</td>\n",
       "      <td>[]</td>\n",
       "      <td>[]</td>\n",
       "      <td>[DO_A(ADD(4,9)), NONMATCH_A(DO_A(ADD(4,9)))]</td>\n",
       "      <td>[]</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   team_no  attempt_no  turn_no  utterance_no   start     end subject   verb  \\\n",
       "0       10           1        1            -1   0.204   0.204       R  shows   \n",
       "1       10           1        1            -1   0.334   0.334       R   says   \n",
       "2       10           1        1            -1  53.525  53.525       R  shows   \n",
       "3       10           1        1            -1  53.544  53.544       R   says   \n",
       "4       10           1        1            -1  56.192  56.192       A   adds   \n",
       "\n",
       "                                              object nodes instructions  \\\n",
       "0                                    observe gesture    []           []   \n",
       "1  so, ann and bob, let's start building the trac...    []           []   \n",
       "2                                   thinking gesture    []           []   \n",
       "3                                        hmm, i see.    []           []   \n",
       "4                                Zermatt-Davos (4-9)    []           []   \n",
       "\n",
       "                                       matching pending_instructions  \n",
       "0                                            []                   []  \n",
       "1                                            []                   []  \n",
       "2                                            []                   []  \n",
       "3                                            []                   []  \n",
       "4  [DO_A(ADD(4,9)), NONMATCH_A(DO_A(ADD(4,9)))]                   []  "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "135 15 36 108 108 57\n",
      "36\n"
     ]
    }
   ],
   "source": [
    "def match_instructions_and_actions(df, inplace=False, verbose=False):\n",
    "    if not inplace:\n",
    "        df = df.copy()\n",
    "\n",
    "    pending_instructions_list = list()\n",
    "    pending_instructions = list()\n",
    "    turn_no = 1\n",
    "    attempt_no = 1\n",
    "    for i, row in df.iterrows():\n",
    "        pending_instructions = list(pending_instructions)\n",
    "        act_list = row['matching']\n",
    "        current_turn_no = row['turn_no']\n",
    "        current_attempt_no = row['attempt_no']\n",
    "        \n",
    "        # clear at every turn change\n",
    "        if current_turn_no != -1 and current_turn_no == turn_no + 1:\n",
    "            if verbose:\n",
    "                print('Cleared at {} at row {}'.format(current_turn_no, i))\n",
    "            pending_instructions = list()\n",
    "            turn_no = current_turn_no\n",
    "        # clear at every attempt change\n",
    "        if current_attempt_no != -1 and current_attempt_no == attempt_no + 1:\n",
    "            if verbose:\n",
    "                print('Cleared at {} at row {}'.format(current_attempt_no, i))\n",
    "            pending_instructions = list()\n",
    "            attempt_no = current_attempt_no\n",
    "\n",
    "        instructions = [act for act in act_list if isinstance(act, Instruction)]\n",
    "        edit_acts = [act for act in act_list if isinstance(act, Do)]\n",
    "        assert len(edit_acts) <= 1, 'more than one edit act? at {}'.format(row)\n",
    "\n",
    "        pending_instructions = pending_instructions + instructions\n",
    "\n",
    "        if len(edit_acts) > 0:\n",
    "            edit_act = edit_acts[0]\n",
    "\n",
    "            if verbose and len(pending_instructions) > 0:\n",
    "                print()\n",
    "                print('Matching {} to {}'.format(\n",
    "                    pending_instructions, edit_acts))\n",
    "\n",
    "            # check with its only item in this trivial case\n",
    "            # get instructs by the other speaker.\n",
    "            others_acts = [\n",
    "                a for a in pending_instructions if a.agent != row['subject']]\n",
    "            if len(others_acts) > 0:\n",
    "                new_act = None\n",
    "\n",
    "                for instruction in others_acts:\n",
    "                    if instruction.action.partial_equals(edit_act.action):\n",
    "                        new_act = Match(instruction, agent=edit_act.agent)\n",
    "                        if verbose:\n",
    "                            print('Matched {} to {}'.format(\n",
    "                                edit_act, instruction))\n",
    "\n",
    "                instruction = others_acts[-1]\n",
    "                if new_act is None:\n",
    "                    new_act = Mismatch(instruction, agent=edit_acts[0].agent)\n",
    "                    if verbose:\n",
    "                        print('No match {}: Create {}'.format(\n",
    "                            instruction, new_act))\n",
    "\n",
    "                if new_act is not None:\n",
    "                    # remove all instructions that match with the action.\n",
    "                    l = list(pending_instructions)\n",
    "                    for s in pending_instructions:\n",
    "                        if s.action.partial_equals(edit_act.action):\n",
    "                            l.remove(s)\n",
    "                    pending_instructions = l\n",
    "                    act_list.append(new_act)\n",
    "                    row['matching'] = act_list\n",
    "\n",
    "            else:\n",
    "                act = Nonmatch(action=edit_act, agent=row['subject'])\n",
    "                act_list.append(act)\n",
    "                row['matching'] = act_list\n",
    "\n",
    "        pending_instructions_list.append(pending_instructions)\n",
    "\n",
    "    df['pending_instructions'] = pending_instructions_list\n",
    "    return df\n",
    "\n",
    "\n",
    "# Try.\n",
    "task_index = 10 #28\n",
    "df = corpus_dfs[task_index].copy()\n",
    "df = recognise_instructions_for_table(df, network, entity_keywords)\n",
    "df = match_instructions_and_actions(df) #,verbose=True)\n",
    "display(df.head())\n",
    "\n",
    "n_instructions = df['instructions'].apply(len).sum()\n",
    "n_matches = df['matching'].apply(lambda l: len(\n",
    "    [act for act in l if isinstance(act, Mismatch)])).sum()\n",
    "n_mismatches = df['matching'].apply(lambda l: len(\n",
    "    [act for act in l if isinstance(act, Match)])).sum()\n",
    "n_edits = df['matching'].apply(lambda l: len(\n",
    "    [act for act in l if isinstance(act, Do)])).sum()\n",
    "n_nonmatches = df['matching'].apply(lambda l: len(\n",
    "    [act for act in l if isinstance(act, Nonmatch)])).sum()\n",
    "\n",
    "c = len(df[df.verb.isin(['adds', 'removes'])])\n",
    "print(n_instructions, n_matches, n_mismatches, n_edits, c, n_nonmatches)\n",
    "\n",
    "# e.g. print number of accept acts i.e. matches.\n",
    "print(df['matching'].apply(lambda l: len(\n",
    "    [act for act in l if isinstance(act, Match)])).sum())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Annotate the corpus tables with instructions and follow-up actions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Processing team  7 ...\n",
      "Processing team  8 ...\n",
      "Processing team  9 ...\n",
      "Processing team 10 ...\n",
      "Processing team 11 ...\n",
      "Processing team 17 ...\n",
      "Processing team 18 ...\n",
      "Processing team 20 ...\n",
      "Processing team 28 ...\n",
      "Processing team 47 ...\n",
      "Done!\n"
     ]
    }
   ],
   "source": [
    "annotated_dfs = dict()\n",
    "for team_no in sorted(corpus_dfs):\n",
    "    print('Processing team {:2d} ...'.format(team_no))\n",
    "    df = corpus_dfs[team_no].copy()\n",
    "\n",
    "    recognise_instructions_for_table(\n",
    "        df, network, entity_keywords, inplace=True)\n",
    "    match_instructions_and_actions(df, inplace=True)\n",
    "\n",
    "    annotated_dfs[team_no] = df\n",
    "\n",
    "print('Done!')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Export to text files and a pickle file (to easily load match objects etc. later)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Save team  7 to ../processed_data/annotated_corpus/justhink19_annotated_corpus_07.csv\n",
      "Save team  8 to ../processed_data/annotated_corpus/justhink19_annotated_corpus_08.csv\n",
      "Save team  9 to ../processed_data/annotated_corpus/justhink19_annotated_corpus_09.csv\n",
      "Save team 10 to ../processed_data/annotated_corpus/justhink19_annotated_corpus_10.csv\n",
      "Save team 11 to ../processed_data/annotated_corpus/justhink19_annotated_corpus_11.csv\n",
      "Save team 17 to ../processed_data/annotated_corpus/justhink19_annotated_corpus_17.csv\n",
      "Save team 18 to ../processed_data/annotated_corpus/justhink19_annotated_corpus_18.csv\n",
      "Save team 20 to ../processed_data/annotated_corpus/justhink19_annotated_corpus_20.csv\n",
      "Save team 28 to ../processed_data/annotated_corpus/justhink19_annotated_corpus_28.csv\n",
      "Save team 47 to ../processed_data/annotated_corpus/justhink19_annotated_corpus_47.csv\n"
     ]
    }
   ],
   "source": [
    "for team_no in sorted(annotated_dfs):\n",
    "    df = annotated_dfs[team_no].copy()\n",
    "\n",
    "    # Make filename.\n",
    "    file = annot_corpus_dir.joinpath(\n",
    "        'justhink19_annotated_corpus_{:02d}.csv'.format(team_no))\n",
    "    print('Save team {:2d} to {}'.format(team_no, file))\n",
    "\n",
    "    cols = ['team_no', 'attempt_no', 'turn_no', 'utterance_no',\n",
    "            'start', 'end',\n",
    "            'subject', 'verb', 'object',\n",
    "            'instructions', 'pending_instructions', \n",
    "            'matching',\n",
    "            ]\n",
    "    df = df.loc[:, cols]\n",
    "    \n",
    "    # Export to file.\n",
    "    df.to_csv(file, sep='\\t', float_format='%.3f', index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Export to a pickle file (to easily load in another notebook, preserving data types e.g. matches)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Saved all teams to ../processed_data/annotated_corpus/justhing19_annotated_corpus.pickle\n"
     ]
    }
   ],
   "source": [
    "with annot_corpus_pickle_file.open('wb') as handle:\n",
    "    pickle.dump(annotated_dfs, handle, protocol=pickle.HIGHEST_PROTOCOL)\n",
    "\n",
    "print('Saved all teams to {}'.format(annot_corpus_pickle_file))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python (JUSThink-alignment)",
   "language": "python",
   "name": "justhink-alignment-env"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": false,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {
    "height": "calc(100% - 180px)",
    "left": "10px",
    "top": "150px",
    "width": "274px"
   },
   "toc_section_display": true,
   "toc_window_display": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}