hidden_features.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "toc": true
   },
   "source": [
    "<h1>Table of Contents<span class=\"tocSkip\"></span></h1>\n",
    "<div class=\"toc\"><ul class=\"toc-item\"><li><span><a href=\"#Constructing-Typical-Input-for-a-Classifier\" data-toc-modified-id=\"Constructing-Typical-Input-for-a-Classifier-1\"><span class=\"toc-item-num\">1&nbsp;&nbsp;</span>Constructing Typical Input for a Classifier</a></span><ul class=\"toc-item\"><li><span><a href=\"#Activation-Maximization\" data-toc-modified-id=\"Activation-Maximization-1.1\"><span class=\"toc-item-num\">1.1&nbsp;&nbsp;</span>Activation Maximization</a></span></li><li><span><a href=\"#Introucing-Typicality\" data-toc-modified-id=\"Introucing-Typicality-1.2\"><span class=\"toc-item-num\">1.2&nbsp;&nbsp;</span>Introucing Typicality</a></span></li></ul></li></ul></div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Constructing Typical Input for a Classifier\n",
    "\n",
    "A classifier learns to extract certain information from input. Therefore, it builds hidden features. In this notebook, we want to examine what pattern these features are sensitive to.\n",
    "\n",
    "## Activation Maximization\n",
    "One way to do it is to define a target pattern $p$ for a layer $l$ and find input $X$ that maximally activates the dot product of it's activation and the pattern $ l(X)^T p $. This can be done via gradient descent.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Files already downloaded and verified\n",
      "Files already downloaded and verified\n",
      "Accuracy: 0.5268\n"
     ]
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "from cifar10_utils import *\n",
    "net = Net()\n",
    "net.load(\"cifar10net\")\n",
    "accuracy = net.accuracy(testloader)\n",
    "print(\"Accuracy:\", accuracy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since we want to visualize typical patterns for hidden features, we need a visualization method at some point.\n",
    "So we might as well visualize some images now"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAB6CAYAAACvHqiXAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJztvXmQXtd1H/i779vX/npf0AAaAMEFXCWREk1KsixZEiXLkcsTe+RxJXJGU5yaSmqsmdSM5XFNOZrJTOzxJLZTcewosUfylCLJIykxrdiWLMqyJFOiSFEkCBLEQuyNRu/Lty/v3fnjnPvOaQDdAAEKze7cXxWqP9z3vvfu9r13zvmdxVhr4eHh4eGx/RFsdQc8PDw8PF4f+Ae6h4eHxw6Bf6B7eHh47BD4B7qHh4fHDoF/oHt4eHjsEPgHuoeHh8cOgX+ge3h4eOwQ3NQD3RjzmDHmmDHmpDHmE69Xpzw8PDw8XjvMjQYWGWMSAI4DeC+ACwCeAfAL1tqXX7/ueXh4eHhcL5I38d23AjhprT0FAMaYzwP4MIANH+j5fN5WKpWbuKWHh4fHf36YmZlZsNYOX+u8m3mg7wJwXv3/AoC3bfaFSqWCxx9//CZu6eHh4fGfHz75yU+evZ7zbsaGbq7SdoX9xhjzuDHmWWPMs41G4yZu5+Hh4eGxGW7mgX4BwG71/0kAFy8/yVr7KWvtg9baB/P5/E3czsPDw8NjM9zMA/0ZAAeNMfuMMWkAHwHwxOvTLQ8PDw+P14obtqFba3vGmH8E4KsAEgD+yFr70mu9zvzDHwUAmCgdt5mwDQBIhK24be/YCAAgmwkBAOfOnoqPZVIZAMDoYDFum+LPd4wOxW19hr7ba9UBAM8feTE+Njo4CAA4deJk3Patp/4WANBoianon/36/079KBK5+yv/9H+Lj5144XkAwAff++647Zc//nH60Itk0Jbfo2y0skYsVVFIn21X2sIO9TsTyHL9k9+W+wLAr3zif4o/dztV6s/x78ZtzeY0AKCYk3mOQrpelKR7RZA+GmdQU0a0RDLhjkrfojY39QAAAWQNAuM0sk7cZkHn90I9Zr5Egu7fszLfPdvk78n5vZDub63aMzZL42zUXc/iY9k0re0dB34sbhsd2g8A+Gf/52/hchx4y7uo/4HIO+7+xlwpA9nIHZO2gNe0XBSttFQoUP87bek3z2UqRf2PEnL9Dq+H9kSzHZrnlGrLpGl+yyW6Ri4na9DudAEAtWo9biMHNUAND8asH18QGHXsatbVgPufiFu+9PnPrjujtl9+o+0u9Wnp0lzcduLlHwIAkqm+uG24nz7X1+i8ysCe+Fi+jwwCQ+ODcVtjeQUAUE7KXlhYukDHenQsXxZHjESuDADoJWRdTIGO1xvyvElHqwCA0alxAECu0B8fGzD0LCoFco0Xz34fAHD2pDxTCgn6feXz9N2FFVmDe990L11rYCBum5+fBwCkjl5tvq8PN0OKwlr75wD+/Gau4eHh4eHx+uCmHuivB1IJkgx6vV7cFhl629ZCeVM9e5bemMkUSRxrKyL19dr05susyBu2NEdtkxdX4rZilaTU7z3xBQDArrF98bH/7mO/BAB450/ujdtmF5cBAL/5W78Rtw0P0Rv1rnvfAgD43Bf+OD62f4S0gfe85yfitlaT+mkiEYeSsZTHYzcydstCctBTEhJLQ9ZubCFLplJyDUtSUzor53dZDA5NXX2JzrNByL0RCT1iCdAoSY0F0XUSY2RZ6+G/Bk3pN0t2VkvoEfXz4vla3LZwiTSKSh9JPIMjufhYOs/9tiLVhtyRMJK+uesioGPdblfG2XZzI+NLpUWiuxw1XjMtpW4Wr2F40dTpMDwf6aSsQSpB891pyz5NsjZaICUTiZRIvKZLxzodmb+wRZ8jpfChTFKnSbPGFcj5hudKX6PL2lFCi+g8voDXLJGQR0PAWoOW1J1mEQSydy/HQF687NZoibEcyj2LRfotpTMioefyvPa9Np9TUv2ge44PTMZtew+R1rW6tBC3jffo+FJjFgDQrKs9maTrm6xct8F9yqZl32VTpE1FPZqXQkqO5bt0/q6SaP+9ibsBAJVQzrO8zgXWEHaNidY4OULaRrsl+7o6R5rpAAq4UfjQfw8PD48dAv9A9/Dw8Ngh2HKTS69DKk2rKaaAMEn6p8lmpK1GphPDumb/gKg7KwtkjumuimoFJobmRLPHxRm6x3MvkemlW7ojPvb5bxCp0VsUz8szr5wGANz20GNx26e//HUAwPI//x0AwOiUqH8f/1//DwDA7tvvj9taTGgmlD4esurv1PJkQlQxx6wlFPlm2NRitJp9OTTnah25KOpwO6K56SnzB9g0Y/leYaT6wUgkEle0afNDj+/V4XUxynxkeFzWivmj0yJTx7kLYgo79sIMndel7z741rviY/sOjnBXxUzRC90Y9JzycUPjDFS/3Ue7bpKuGFaMdo/NR1cjQLW5yY2Z5zlhZf6CmNAs4XJoc9DSHI19oUokYDIt/V6urQEAej25Llso0VqTjT05dRsAoFIi1T5IyTjd+mlzSbtDc5VQJHtg15tVkkk1ltCuOwaIOUoTx5ejsyz96NZpjip5IQGTe2gvtNqyZ/rY5DI+SOs+vltMoPky/eb7cvLbL6ZpfnND2bjt1RkyXRQLZPKpL0v8Y47J0+qqPG/GJoh4HRkbj9uyORpXxObIVl2Zc9mBI6jLb+mBvfSb35OTa7RqtEbFAXoWTS/Is2WoQONbbi/Fbba22Q/8+uAldA8PD48dgi2X0HM5IkRqjfm4rVGnt2deSWC5TJ7PW+MWkWT6+sgtKLq0Frclzx0HAKyF8tYrMGmUz9FbvVkVF6qv/AeS0E8cPhK3PfrYLwEADimSc2mV3tQLJ48BAN79X/6P8bHmGGU++IuTq3FbxtBbPFRTbS2RdJUMja+cF2losI8kjf6EjL3IkmsmuFJavhqc1BQoF7iIRdJEUhOJTiojTSiZ0O6I0bq/dD26f7MlmlCXyezuVc5POndIpSl0WnTP1RW5xiq7c+WZEOy0RBpyhGNGSa7xXCqJsetcMC391RxmJkPjS6XUGmwioreZQLzqOarJjdXNLXrS7zJLyamUIqbdXgxFQm/ViC1cWaW2SPVxtU2SXag0hTxrramKXKPHBGarRdfKlYRUS6RImsxkpG21Tt/t9fT4aA86UjRQE+gkcy2hp5JJ/ruxi13eKHfEMSJuEwXRuk/PkqZcrYm2Uc7S/n/ozQ8AAJZWRYItDdL1xvpFQg/XuN9Wrnto8hC1MT950op7odOqBkfG4raBYfpcrohrYqtD+3OpxZaBpuzrXaOjAIB0TdZqnAngYlnI9maS9nW6THu3oMaeZwvCKz8UN8cSWyYgPOlrhpfQPTw8PHYI/APdw8PDY4dgy00uy6ukJkYJ3RVS0avLQpzlC6SipAIyV0RN5ZvLJonqmkQYzh97GgBQ46gxAJiYIN/PYpJUzunD35Y7sn/vvXfcGbeNDJKqNHvyqbht6SwFw07d9xAAoDN6KD721KtkNirmRdUsGSZo0mJW2TtCxFCzTW0X1kRVP/83ZPq5//YDcduuCSJeqw1F+m4CpyxHSqOO+N3dUhGoLsIxwcSPJuuqa2S+ainzR54jHcNQxpJiM0k6Q+uigiARsGnJqG1WKuR5THKN44dfAQAMDJDauv/ASHwsnaF1SSQV2Youj1OZjzj6MYyy8VlxP4xjRbV5YGNTQavLJperWVyuQop2nU92JPPXl6a2TEbiA0J7pR/62jyZ/S6dWwQA9AJRyzscj9FRXFm7y98NZS9EU2QqGOV9VagI8eiiRvM58Y9OGDIJtpVvurmMPLXKdOZmKpGUdezxd3vhxo+Qew6Kc0A/R4C21en5IVrvUJHJaNMG6u+jaNDpSzPxob6AzW8ZmedMmda7Pi/7o5Kl+Uhl2Ny0R66/WqVnyp0H5XcLNmUuLsnzplKkvR7kaL0VR4wKm2YCHeHNpOl4RXzvW+zr3kzQb6l/SNbgyJGjAMTEBACT/Ow59pxE2L5WeAndw8PDY4dgyyX02XmSTHJlFYlVos+Lyuewy+RHpZ/IlYySFnL8kotCcXEKOpzHwVbjtrNHngMAtJYvAQBWVoWIfcvb3gcA2HNAXObqSycAAI01eXOvLZwDAOydeifde2hUzmfJy9RFTD2wn97mw/0yvm5IfSoOE7mz9IpEub3w1PcAAKdeFRengQcoGm6tq97muBxK8rYuOlC5LbI2EFmRvFqc0yaVcNGB8n4v5ElCqZRF2suwxBNp90ZD0lKKJdLASB9t5MhLuacTcPN3CknXrZG201yh88oFcf3KZWkMra6QY07zCFXkrJPW3RjSKhI0kyStIFhHKm9Miladi5qO/HSEoPqaI027TtOJZN1LoySpBYrQXGPyb2lJuapFJOlOsDZorZKaA5ZE1bo4qRNKku8bpj3YZQ2g1hQNIJWja+gcPmX+vLq6HLeF7MJ4NQLUQbso5liyDHGlq6vD/ffcHX+u1khTtQXpR3qINVnl1prjNTJd2gt3Tt0u/R4iiXdQXSOZJQ2onJF9d/xFclNM8l4sDUlum+lTpGGPV+R3m2TXx9qS7NNylvZMjvc3MkKs9hq0zqlQtK8y3yOTkL71OI9OOUHupGuzi/Gx9hL99t75yDvittlleh55Cd3Dw8PDwz/QPTw8PHYKttzkAhetpiIpU1lq0z7TbY7KajrWLS8qZ4bVv5TyTU9nKT3lvvvujdv2T5MJ5Wv/9v+iazVEXdx9/5sBAI988MNx22SFrjt/SUwz5+Y/CQCoL1Pin92DomJ1UqRazc1IFNrRw2cAAH8zcyJu2/cAmRhmzpFqfOLJP4uPzS1Qpal5pe8vnaRItnRJDC1XmlwUHPmntOFsgsaSzYqpY75Oqn8qcCYDNZY6JwQzskWKbO5qt2V8TTYzdVN0T8W9xaaWlibf2ARhFAm+fz+lsi3nyGR2YEoI4U6PSMOFNTFnNNiPOxGom7G2nE3Th6Ty50bkom/FXhJg46i8kOe+15UJdFG066MlXYQtXSutfM77+5k4U/u6xYmYmopoDpnk74DGkgmkX2k2udRrEl8RuuRPygQwf46Iwx6blgpl8acu9brcN0VM52mdjTK/1Tj2I8VJ3nRkszNVOX9+QKJXu+HGTtP9ZZVkivdkW0Wx9nPUpvaHT7CpJcs+2UNTB6UfSepbSpm26j0yYxUGZS+UR8j88e2/egYAMLlb6vC88uKrAIBjzx+P29757p8FAOSL8qvKB2QObbJZsk9t7OUZ+t1MVOT8ygj9NsKWIkq7tB71BSKwV6dl3e+7g55Lja6K3E5eX5zJZvASuoeHh8cOwTUldGPMHwH4EIA5a+093DYA4AsApgCcAfDz1trlja6xGQplepsmcyIRJNnVq39QJI35NhGHnR690VIQaWGxyWknS+IuuMouZPMqrezu+x8FAHzoH/wjAMBnf++fx8curJIUdKoh77jFHrv6hZJQ/+63fxAA8PSX/wAA0FiW2q1rXbrX6opIEHeO0/iCJSF4V374FwCAczMk+c+dFQJ0z31EytaNymNTp6lND0zgehCTom15+7sUoklFWhbz7AqadMUKFPnG0mykUtSGERexSMicprk4QYJd7Noqos4RsEFKSWqxhCv3Wq2StrO6RGs2NizjHB8jKSiVE+lmbpVy7DSUj2RMhrIbpXbJC3tN/ntlzpyrYYVzverUwcKQ6ghKzvfB8zKgUr3mCvS5pYjpLkdyoirrHaxStGSyRxLgfFX6vcxEorsWACSafP9IiHrrohmH6PfSUSlZa+zzmMrKWMrsfjqkJOhqnfai5XnpqTUOXNpkdd2Mi4RNynmXI1AujQM8hrWe9NutC5QbrEtrVGLyN6VIxiaTjNm89LvFGoKLkgWAkTE6vns37c1vfe0v5frsInnkpaNxWzZHabTf8z4hYC+do7kP8vR7aTfl97u2QmN44C7R/nv8XGqrvDRZdrVOcvGVZUVCnzpLrrrnl2Uv3P9jb8HN4nok9E8DeOyytk8AeNJaexDAk/x/Dw8PD48txDUldGvtt4wxU5c1fxjAu/jzZwB8E8Cv3EgHavzmK6myWU6qMUq6Hhgn97n4Za5MoAFLh4EVG1SZpXVnYwOAaoekj+LtbwcA7H3o+/Gxs6dJ0j64IPbh1gC9WRPKGH3wENn0XvgaSQGvvng4Pnb7m+m61TkJZmou01t56cLpuG3pItk8l9r0Pj349vfGx84deQEAMHvi5bht6mE6XtwltuXN4Gy7WRW00OzQXOr8LpUKzZErVqCzIkYFzkGzrpgF5xvpyrZJcuJ/15JRWQ4jl3kxoWzXXGbOQiSZTJ7udfoo2TVPl8WlrFwgqaVcFpfUFhd+qLfEvcvwPVwREO0umEzzOq4LXttYQnf2944ukuGOKXdZ54PptJkRVW4uxzlUoItwONt/TwLgylm2RXNOnERKpPFsZZAvofLB1CkoKKsSfkyOkTZX4QCWbkskwU7oslVKv7M8vsG+ctx2YYa4ih7nm9HOiG4eynnZT25+a82Ng90yqixclzM8ZpQLa45LQiaUfdpRau6bCaUR5VgbSAZy3VyWnhv1jkjoEZctfOvbSNt9+dmn42NHD5Nk3l+Rsbx0hJ4Dd98nEvJkP/3WqpfIXn720vPxsQceJEk+kVLFX3htdcbSgHM1ucFcVJlcn3mR7vmWdz4oY8ndPKV5ozb0UWvtDADw35FrnO/h4eHh8SPGj5wUNcY8box51hjzbKPRuPYXPDw8PDxuCDcq488aY8attTPGmHEAcxudaK39FIBPAcDExMQV4XlJJl9aLVFvu478SOrINFIr2y1STSNVNT7L7lRdo9SdNLtfJUStXGWXojbIfHPboz8XH3v6K1Qb9Pzz34rbDr3jA3RvRUpdZNew/offBQA4/LW/jo8Ncy6S8QG559nDFJ1qFcnZNHS9Ua5Lmtn7QHxs7utUc/v0c3LdHuciyU/JeVdAV5xnk0uxKGasTshEkjKr2LhuKOdBUW5ssTvaumhJl5pWzCoBbyHDdSwDbV5hNi00ynTBKnek+lHmfBztFpnHDr/wXHxsdIBczqayUv29mCOTTCYlKmzIuU26LvHJOrc7FwWpa2hubHKp1akf2ayqD8l/dW1O11pg81F/VuYlzx8bymxj2a1Wk8TNOplJQja1VIYkFwiYqK+tSGRpKiKTYEnlBhrbxaaZFPfNSvrmdECmjpT2iDP028iXxOnARdY2m9115wDiyqhdNmsc9RpuYroySdkLSTY9ZSK5LqdPWidVumjkJNcT1imdk0yQ6hxF6RTtna4q3GIKbFot02/13R98V3zs+Rd/AAColOQ33TdE5714+Jtx290/RabVk0fJVBoqF9apSSoo0lHb2i19qSxmNxcF/MIrdM9vP/PV+NiDj9wDALj9LjGjVhWxe6O4UQn9CQAf5c8fBfCnN90TDw8PD4+bwvW4LX4ORIAOGWMuAPh1AL8B4E+MMR8DcA7Az218hc3R6bgybNIW8H9CJYW7rH8BEyJRT+cpIWnSpoQscW/UpnIjclJHh88rTkpmxQce/XEAwHN/+2TctsKS0W33PyL3qlAy/DvfTETl+QtS3uor/+Y3AACVMQk4COv01l2Ym43b9t1HhTCGcpzl7amvxceWj1PC+1RKxj64hySwRHLjYBiDK13s1hckYFJU5eOwoctFQvPXi2SuInY56+hACZZ+AxXA1etx/pB2mvsoxwosrfQiXXGe7tXtqYAUDszZv5dIrIUZcW27OE25c0bHhChNFUkqSwYi7TWYLHRjSiiRNApcZkCdU2aTbIucC6Whsls6MnQdKcoSepmTCQ0O7ZI+cnV5q0j5iPdwUo29xEUPkpxdsKrMkuNMymdHJTBmaZ72UU4V/Gixe22OtdFkpIqBNOj+vY7K78JkuSOLAaDIroyrjWU1MoKrTN9pyFhyLElfLeeLg9XlDp3mpvqW4DnqqbtFLviLyfueIs8ty5+pREadT/dPJUXryXG2U+c2+cCj74uPvfMxcn74w3/1B3Hb+95Lv9fJsUrc9o0nPw8AaHNeph//iR+Pj9XXqE+pouoH/z0z/Wrc9v1n/xYAcPoskf2PvFtI1wcfomdAW2mvzXDjubxeXI+Xyy9scOg9N313Dw8PD4/XDT5S1MPDw2OHYMtzubhalLm0kBRpVrdSiuQ0EauMHAGaU+lRQ66N2VNkXYLV8WxaqWxMyPUMqVHpvBAYdz5M6XD7RiQq9Omvkylk+tjvxm3DbHKJOOdErSoqVo5rlgY1IaUWZk7z32npL6u/tXki9TIlId8yg6Q2Ly1K28kfkK/77gfFR34zxLGYyrziTAVaqWtzno8e1z3VOTWaa/TdpYvKz5hT8A5VhPTttJz/LV2/pwojrC2Q+UATiSGr0Npfd3iY5vyug0QQXSqeiY9ZNtH0umIyyDA5GyprWpdNdwkesybNHUEaF7rA5qaCDJtQIsW+tdlc0lOmvhT73CfLnGNEkagh91GvAXjdqytiUipXeN9HpL6vLYppbrDPmcnkt7G0SnPZVeRi0KU1Ghyl38RgVu6ZGaU+GeW77dK6JJTf88go5S6Z5YIzqw3Za12OsC1kVfGNyOXp2TiXS1v5ysc1alUxi4AdBbpW7tXjsfQxSZxQMmcY56VRxU54H2kzICyNud2hMf/lt56NDx0+SmbUct9tcdtXnvgGAOCbX/+mXIP9yv+bX/pvAQAXzktE+Heeo+sdvEdMtjMXyayypKKAb7uTTDk/819QXOb4qJgNm10yXxllJk4FG0fdXi+8hO7h4eGxQ7DlErrLTtZblrd0qUBv88Vzkm/h4jFKTF9foZwuGSVdl1zV7l374rbBccrgV+gTN7A6l6rrsLTXU1LLKhNgxT4hNN/7i5TzZXlFvDK7a5Skvsyk1IvfEUlvaDdd37ZFQn/lBLngpXLy9q0tUYGNOZaID7zrA/Gxhx6lt3n9VYmCnJ6jPBu5XUK64YRErgHry6VFTPg5qRUAWg2WcK2Wmji6MkXSiOLNcOk8rcfqrFyjL0Punkttyf7nBNB0jqStTluu3+LPPZVDxUnwmqDsLxMZ1eNI3kDJGfMssV6cFglpLNHP56scIKyB5LgfVhFtzjV2vVS+cYGLxRUiBgs5FfnJ+82ofofsklhgkTenIiMDlwlSSdIuz82paSE+z8/wHHEqJNuWkmupfaQJNdZUkYwVt3dlX+dyRAg2DN0zoYjsMY6iTqjcLEnnrqrcgkslLi/IEq+WvHNF6kerq3MDcdSrGvPl6KgcLXWWSINIR6zSPRMQYrDd5r3I0mpeXT/rsoKuWzrOQ2REO7q4SP384pe+DgD48hNfl+uvUj/uOHCfXKFLe336/Iuq7/Rb/uo3KA/MT//0z8bHlrkoz8WTsiddgO873vdo3DY6Rft6IMX5nFTUcDsiTSgVqchgnfPoBuEldA8PD48dAv9A9/Dw8Ngh2HKTS5ELJOwtiL7fF5D549wxidp84U/+PQAgcsUV9KuIiYVCWdLnDo1N0d/9b4rbBvdTjcNUiVSgsyr17annieioL0rdv5FJik488LAkz9p1FyXTGSmzX3JLzp87Sdd46agk4nJ1NUMVVmbY1LF0kXysV774R/GxzHeJGDx038Nx2/676XOzvrGZINJ0J5PDYffKAiG5jDJFsL2k06Xz586LaenEUSJx05BoQpQ4PbAiNItM6KY4IZL23XaEYFul8XXRmo26mB3mZymZWSpJam67KccsmzimL5yJ2xKcSjeVU77mHEXoeKVIkbMRq/Q6IRg2qYUZ8T0XlyXJVZoJ0GJWzDBZJubLnIY2l1f94dqgytKBTIaO794jNVMz7C+/uEz7aHVO5uqlo2QC0D7nw0Pkk54uSv8XQjqe5ujH/pSYS9KrtMcyHVnHKMUJu1SirDRHVuc4UVZQl364GJBOq3nF+XYTmbCjooFX2/TdclacDmp1uv+Ro+JYcOo0xXXMXmDysiBmiPvupd/v+Kgkaqs1qJ9HX5Xf8tM/oGIyX3+SIjStMvmNDZHZUCeucyR4V5mI9uyle1SGBvk+sicr/fSc2aMKZxSHaS/0D4oJpdYk02SFi3VEyuc8Mi5MVpHsqY3NV9cLL6F7eHh47BBsuYR+10EmsS4JCVhbpDflQw9JdNZLXyVyYu4iEY6BKicWsTuTLnhwnomLsy/8MG5zhFmKC2h0lYTSaRBJ0WuKZDJ3hN53p/5WIjkPvonKx937NpKaLy6Im5lLYJ+tCGE1sZekp6VZOc+ya1YzQe5rXeUu2JkhUuzw2p9Lv39I6T/v/8mfj9sgwgFDxuJyqKQzItll8+zGmREpxFpa/hWWCo+9dC4+tsQuh8P9ovWs8PhyWVXgoktztMJEoiYey2Ui03RU7+gIuW5Z1bbIxGe+SNedV+mHnSDdVRJ3f42k5KxyX4sSLP1w7o9uT9YxsCRhRlZHHW4Ml7skrdI3t1jzWFySvCp558ZniKwOknJPy/fqhZKfo8dE2IGDUsAjye6VPSatO5mx+NiLr9BeGBuXZKZjU5S6tadksSQTieEq7adWRtaswcRxOxTpMGmuzNPjJNY8k7/NhpCzPSbZ+8qSG8gRwp1N3BZ7qpxemyOKv/XUC3HbE5/7TwCAZ576btzm9uTsLM1zuy1aweQUbfo7b/+xuG1hiY4vVyWKtcGkcLFEpGQuLau9vEAOBudDcVyYnae2Zke5qfJWOXrsJADg6e9Lv3fvJocLo9Sv0Sbt66BPyNnRfdQWJmht21akfJfzqN4SF9Z05sp0za8VXkL38PDw2CHwD3QPDw+PHYItN7lMZIkImF4Vk8QzLxAZ1D8o5FErJviYjFGEi9OfIxWFlk66BFWi9rm6mlzIG1dxZ4UO7Euwmt9eXYjbXvg2pcB89Tj5xU/cJnUI+4ZIxSsOiYoccLpdFzkIAM0aXa9lqCN2VVSxFEfEtlWl9+YikZXPP/nluO2dvyR+sQBgFMkXcVKkICkmhnSeIyi1HzqbS6YvkPp5/qyYExKG+t3tqvnjqjApVVVnjpOO9VhH7VNVcCL2+x/qF0Ju9ySbG5T549irRGKd46pOc7OS8Czk+49NiI1KLyweAAAgAElEQVSpPEKmjrIiHE2Wo1JDrrSkIkVTPJZIkbnrHPcvQ4f9rdOKpHLmo25LOeuz2SERMiEbytidFS1phASsLtEeWGvK2u7bTSTdvfdQutbVNTG5TOwj1X546lDcdvgVikhcuSRmqdu4Ms7aGZ6/ZZnv1ARdYzwQEjAXUL9TymzpzEyDg9SfZEIT+2zGUilk25z7Vic5u1w6tJHc83vfoQo9//p3Ph231RdpzZJG9tPIKMWBFPL02zh3/mR8bJarByXT8qxIpug3Z4yYmZjjRB/HN7jYEQCo8m+5uqRiKVzt26QQ3qfOUMSnM0XpqOF8ia536tyZuK3BC54uD8hYJmgsnbRLGS1moS7vTxetDQDd10G89hK6h4eHxw7BlkvorSpJfQOTQiRmjtLbq7kskrGkYHUS45USln47WZbGIiVBOBeruGl9eOUVV3Wfra7+zjep81t/QUmT2T4mydT5HXazNCoaLsmSXypDktTIHZJXopglN62WyqUxc47uMbx3PzaCUdKWc8/rqNqV1QZJ4em09O3CeZJ0jhwhl6+wJyJvgetj6uuW+4gUazalb0uLJDUVOFqx09W5c+hvpLSHKqe5Tas8IiHPzSUmpxoNVdE+pAlvtuWeTSbKikYkwDaTlk2O0i2oyvCWNRUdsYpNcrk045qlQnImmUEcyIjUvneM8p8c5L9pI9J7kzWhUBFhq/M0V6dPSB3al88VuI+0d7IdkRzrNfquOSXupD2e+3JDcoa0uLpCNsu1QlUfgxUiN3s12WNJdu8NVK6kdMrlT6L1DlQq5XqVc9A0RbtLcWGJdLDxPD7/tLgjfunfU7RmdVE0vnvvIJfiTiTEp6sFPMjOCR1FGi4uU4R1SuVnGhjmKNaqXLfCGmGDi4csLImE7jRZXYhlgJ0Y2qoYieHnTIrzM+lqa30VkvxDRdT/4LkjAICXj1+K26ZnaS++/f0UwZ7vU0Q2uxZ3tWvnqrjJ3ii8hO7h4eGxQ3A9BS52A/hjAGMg8fhT1trfNcYMAPgCgCkAZwD8vLX2Nb9ijk+fAQAM9qs37DjZWdtLIpXFFeSdK9RVRGmr/LC6KufGFdjMZ+2qp6kvOKm+5wJ1RKpNcBGJVaVZRFWWSFUyuBYLRrc/RIn373/fR+Jj/fz2tyqxypkTZDctDqtcLotSyZzvrrpIfaypggRrXFYt3ZPzZuZIclnhDHsFlR+nj7MAZpTrXp7dPrsdkahcUEaDMyquqaySXdYUcg3l/heRpBOo0mKWbboJdi8r9Yt07fJbjE0ILzHAwSHJlEg3PZC7WJrLnilhCwHPh7aDbiKgx0FJkRV5xwVHza2JxLirnzSWUoX6GKgMoAnOS5NPinvcvjHux4KsbchSdVQkqbKQFft3p07732V1BIAhzko6kRROoY8l7aDA9uQhyeqXyNNYypH0O+jRPUK1tiHnLmk2aW2rVcXh8IZNF+T8yM2p+s3pKncA8NX/KIGB9SW6xkifuGxadhNMFFROmQHW9Kp03agrknEx43L9yFiiLmk9oTJApyzt4wTzUatL8/GxMkvJk7slZ9MAS/TVmqxLNkPrkuH1OX9BXHqbHGA1c0m0pJlLLrOozNvxU/RbaAWUyfV9Py0lJPvYpbjTlt9ooKPQbhDXI6H3APxja+1dAB4G8A+NMYcAfALAk9bagwCe5P97eHh4eGwRrvlAt9bOWGuf489VAEcB7ALwYQCf4dM+A+BnflSd9PDw8PC4Nl4TKWqMmQLwJgBPAxi11s4A9NA3xoxs8tUNscBE2Mq8qFFzs0TkdNYkyi7gnAfO+89c6bW4mSea6/+6/9trfcF9T52WDNbXO00HMoUp9nlcWVAqHufqaKgI1OIIkZt3v+OnAQC9oqh/a5zS12SF+Bm6m1zfdErYK0wumvzl03RuipBds5ptRVDWSHXsspteRuUp6fboWEJH07Ja21C5Vppcb7LGdTiXVYX6BU47vGtS3E8X2WVzeUnMUo02rXP/ABFcu8bFZDDKZqb+Adlerh7p8iVxX+vxWLNcZCKTVcUswClkdQGB61x7B1cgJFKBkTMzpHJfuESqdd+QEPtdLq7gUrMCwHiR9sWe+8TFrpwrc/9pDo6vqUIe+yh3SaVPzDBnT58BADQU6dvHbnGXWrQHnnlZojzvvo36cUCZuBo8D0FCSOUk5+IZGqS9tnfvVHzs+BnnIinXiHg/dZQZ63KDwfnT4vqY49wlM4tCGtaYIG1DzBQDj1CtzUUmkF9ll1YAmNxF+6i9Knus6n4mgUSxZpI0v3H0cCRmjXyOjun9sXtqCgBw5qzMm+3Qd4rstrhHpa7usDl0dkZcR9dWaZ2LZckz0+Pfy/e/Q/Nwz11SU3T4LupHGjIWlx/nZnDdpKgxpgjgSwA+bq1du9b56nuPG2OeNcY8q5liDw8PD4/XF9cloRtjUqCH+WettS66ZdYYM87S+TiAuat911r7KQCfAoCJiYkrxKI1DiaprwqZ1mLXtmSg3v4ZDoxxJJJyA+x0Xpu09Vqh5XpX4qzLgTTzSkos81vcqiCikLML5ioidb75fRQUlJ+gElb1phqLK6mlxu7c/uympIn00rArWVa5r6W5CIMmIwsFkj7aLEXWVD6MYoHIt1JegmW6CyRVLy2INtVlMm2N3SzXVECUYbIzrxI2jpXJxa+qslQ6zWBsFxFme/YL4Tc8TFKvVQRlo0H30CW7MlxQIsMFEdJJkcByGZLekgmZD4tNWNGrSO9OucvlRaodGKSxJHmeOz3RZuJgnJr4CfRqRKwFDWlrNIjIrLXpu+cWlcYXkVbSX5P9cXHeBdeIcFTj30Q1w5kBE5JPZH6O7mlr4l6b4zntRlLl3jIR3NdHkuPYmAQ4nZ6mPd5QLqlOWQwSsi6Xl2eYmxcJdmWJftOLi6KZuXleq0vb3D6Swl15wVCpRM4FudUQzT3F5GV5UO4eRvz8YEeKvZMS7NOL6twf0RQM6HdYKsu8XTpHz6Nhzp44PCTXcMF0y2osLkCo3Cfr0l+k+c2y6+2ZI5Kv6uAoBZKNDA/FbQs16dON4poSuiE7xR8COGqt/Rfq0BMAPsqfPwrgT2+6Nx4eHh4eN4zrkdAfBfD3ALxojHF1z/4XAL8B4E+MMR8DcA7Az/1ouujh4eHhcT245gPdWvsdYEP99D0324HGEqmfTVUJfeb8MQDA8LCohMwtour8y1UF95i1XKcqX6k2Xy8JejlMUlT7OLVJfE9VQ5OjR20oxFZykMwI9zzyWNxW2Uv5X2odUgkTaammnki4KuaqQANfrhdd7um7QX957Al9OudO0fU6R0dofif3kdp3XpFClRFOUasqvb9ylPzhV2tC9AVsUqq54gcqAi/DBR/2HhSy8K5DRAjfea/Uf81mSa3NFehe2g89V2ASWqXDzXGKUtsWE0omyW0uba2KwMuxr3wqpbf7JsqpywOjmHdHKjsTEwAMDpNJJF+i+euqmpuFBO2LZlNIr1qL+rF3/yNxW7tABU2ef46idettMY2k5mlOTy6J2as4cAcA4ECf7LECr9HIKJn8Dt0nxNziiafoPi0xpyXjDSV7N8kEaZXT4U6fFxPo0jKZMGxKSPOIpy+V2nhPVuvarMYFNApyfoMjYROqtmmt5swprk0ePVP7yBQ3Py/+30tMvCdUzd5KkfaC4dxO4yNCmJ6f5jxK1VCdT98d6pf9dOo4mfUKfXnul+x5Z14J1TOmxEVzOk0xByUtkeDlBO2T1WkheOcu0HyPjMjvoKBMZTcKHynq4eHhsUOw5blceiwN1WpCJqyt0tuxsSpv+GxM6tE7qKcKJChnqvjT60GTugx0qYwQYS5KLGCWLFKE7Moc9XtgXKLhDr3lJwAA/bslSmyRecMQJA0pBQBBwJkSVTbCTIILcyQ2jn69WuSjK/cGAAGL6x0tlWXp+FvfTlLfA287oPpB850IRGopDdP9FxZE6qyvUn+XZ2j9Wi3p98gu0gAeepuUAewbYmk5KXNqmGhutElLC1Ii5ViOxHXRpADghO9UQrSHHrhAREQH00oaDzh/TaRyl2y2Q1wJulC5bDouuatCUBc4j41L6ZHOqP50SKJT3qpYrdNYqm3RQOZ5ul6eoTF3ZlUBjSp9eVY5lWWXOFo3LRGoJdbwojRJgBlV0GHlAmm7fRlx+xzeTwSfSYvkv8JE87e+S+6wR145Fh9zS2qhBhO4aFr1CLlMPKzXpd+TnHmw1ZA5qq2Rdt7XJ/MxOkra3MoKjS9fkH1SqVB/TUIVoTlGUnCrLXsm4vw5EWfeLCsts8vrokvnJfl3Ve6TvV5nSbvJqVk7XdGInPuzzuUyPETMf1MR2FXOPdOpE9FbX5b998Kz9BwZHhGpvDwi63Gj8BK6h4eHxw6Bf6B7eHh47BBsucnFEQu6CneOTRz1ZVGjykNEPOU4Mq2qigQ48i/SbrLOBnGDRCggZp0wFELE8PWc2tVpq8T3BerjgUMSEXbb3RT5NltTUZgteo86X/NOXQgrE7i0nfKuNSlS95LZjU0u6wlfmpBA+V2nc3T/lPKRzxXpvESKi4Go13ub61Qmle97eYj8dXWSqzUm7o4+Q6TezEUxGdx1N/vajkqRhw54TdOifroYA1eUxKpalCbF86H60WNV1yhivBkyCQlSt7sqbrHHKXU7oa5/uYnJxaVeViYXl7ArUNG3Lx99GQAwzCTc+9//3vhYms1BLauKgSzTfnrpP0nSqipH7ha53zmVgjfRoHuOKJtcockRqGtidmhy6mKUSLWfn5djPU7w5VITA0CL4xrm5sWP+s/+nGrYHj56FABgk7J3TIbNAqouaUxZqt8cLita32lLPxpcXKbT0fEj9OVsUUwiDz70ZgDAS0deAQD0lWX+Wi2ao2xazBQuXiJQNsc6Pxsyzlc+rZ4tRSIvaw3ZCwW+hw1kbXuW+r7EBTHSKblns83pldXvdpQdOBLKhHLhPJmU9t9BvxsLSWo3y8TupTlZl/yAN7l4eHh4eDC2XEI3HNFXVqXLZpiUCpQ02QW9xQfHiDTpRRKF5tKMWiWix/lXFFl4PcK6zvfiKqAr7iOuQm7ZPW9wt0R6vemdRHw+8Mib47byCPX30mGRhtLckTwnz08VREKxEX1eq8vbvM2FGVLZzZZLEcJ8/YYqSNBiaShXEmnPSeFhyBJ6pFKhslSoC4S02S0vlxcppMRuXbkCnTc6Kq6mk7s5h4uaUxu59La6wjnJFXHZu0iV5WLSOWXSl50tOVoA0WhClrI0/+kqEwaqGMMmcaIx2aUldMuSuVUSOjhy8jvf+Q4A4PSrEgn49h8naX0oqSS7HkmKzZbSvrisW7nIY1G5gQJOP5xURHarSnOzIsIh2kyauopvqw2l4bBWsrAoWubfMPH5zDHJtfLKCeq74Zw1CRVRbFhT1cS7042MVhovk9AzWSE763XWPKxoVS63TqulokFZOxobo8hq7dIY7+uG/DYadfpuoU/y4/R4r/d4j5X6RMofGKXrduZUpm8mldMFLclT39wjKJWTwSU5TXJSRSPX1uj3nUpJWPSh++6je+6aont25Z4Xz1MEb0el+W52NWl/Y/ASuoeHh8cOgX+ge3h4eOwQbLnJZZGr5gznFQHFqn3UU4mk+khVGuCq8dmKJMuZPkEESnVW8oOJD7Z+Z7lqR1faYwyfp00ujqjtWVGFTJ7Uswfe8XYAwL57xHd76iDVbRyakMRGx18ltXZlQfyGAzYfhRGp45GuHMMpjoxKRpVnn+rcJq9fo0g+x1mGylwS8ph7qr5niv1zE+xLrPKBxfMQKXtTfx+ZU5Kqv132pR/gaLxGS0jR1TUa88KcdDydp5sUSxK9lwhJhbXOThZKv3tMxHVDIdgC3heB6odl9ddVqkqodXSbXBPHFhsTzFHgHK/l/CDeA/I9zrmEJjtqu0haALg4Qyr45KCYAmZPUAX7hbMSMejsQS1HxPbknkWuPTtcETV+jc1kyw0VOcvXyBlO3KXS4maLtJ+eff5M3Pa96T+jW+eEqE9zgjHDHgb6d+D2lrHqN8oRs8oz/Qpki2KObKxQQqu0MuVkuQJRtS1E8FqVTEN9Ra6Ypc4fHqI9c/a8mI8sm3BSyrTVYjK2x771y1VtLqE9ky+LmfPiDPmLVxviaGHZhNfl+YhUzEOxXOR7yvgGxoj4P3lCoq3vuo8cIi6co9qqLkUyILEcK8tihklPSwTsjcJL6B4eHh47BFsuoXebTPwUpCvGuaWpKLSI38RLq0SIaDfHLJN08v4Tcmdy78G4zdV8bHBei6RKB7rA0r2ucl9dozd2qF57d7yNyK6pBz4IABi/TYo3NJiMefG4qvReoy8XiuK6127QPVp1OmY1s5TkYhMqv0uSJZhw+fryyQdchCOfFym4abm6fCCEUoKl2qwlqaWnXMqcXJRRKXi7nMMiUnlVsnn67oG7pmhMqrZjmnOodHU07QpJJG3loVWp0NwE7D6ZVC5zIROkUagJb65nqTS4TovmsMfqSSavXDbZ5THQGpnZREJnMlIXNjGxhC6NTutxiYZMUfbkGkcaHj0pZHiCXeUKZSGOR1K8P7g7KVXkoM0kv45IHOB9cd+ISO1DOdr/bR7ftFrH0xyqutIR4rHCl8sHOh+SWf9XIdZnlcYSn7YJu5wqyZ53eUzKWRmf88YM1f7oMDGYKdGYxpTLa4mfEa2OSOgDI3S8pIqAFFjyP3uCtP+lBVmD2++mPDdKAca5acqfc3Fe8teETMY2WEMsqchjJ6zvGZWI8J/6WUqJ/fW/FJfUH/yA8ujs4bTQrkgPAExO7gEApJQL5twllVr4BuEldA8PD48dgi2X0BP8/tdBCG2WnDOquEKQIunH2cd01sCECzBRElWGAw5GDtwZt+UqZPNqNOkN368CdZ5/it6sjRWxaUXss5TrEymhMjIFAFir0s0WXpak9CmuyF6uyFs3YHc77abX5hJWGQ78KagcIIYDGhLKrlhdpuCD2poEIaiKWK63cs84OEnEp6DL+SeUTbLD7mgpFlcyKngnxcEkoZLK6iztaY3CuYomuPRbQUllLqAjNSCSYJ7zZSzMyjyfPkPjGhwkF889+6QkXyrhbP8iYSZclXsVgFTOkAZSdhXf1c5OBFn+3mbOioLQ2eHXubmya+e6/DjsKnml6R+GHfsSVuV3adL8NhvCY4yyLfdOzmHi3EABoMHFHVYWhRvK8/2nVAGKFLu/Hl0hjfJ8VeZ2liXAssqX4qTPCJpToM8uy+e6vKUuqEoFcjltx9iN53RiSjIJLi2cAQCsLoiNOQmnkcl1nX26zVxPQrnIOj6g2RJdPGTtcnBUuIqRQeLX6lXKDVRflvkbqNBvtNoQTitTousmWzJHaXZbhKE1mBiX69dXaXaGhpT2UKb1e/9jH4rbjrzwIgDg7rtoHkp9yoWV91ipT7S12QUvoXt4eHh4MPwD3cPDw2OH4JomF2NMFsC3AGT4/C9aa3/dGLMPwOcBDAB4DsDfs9Zu5sV0VWTTpG41VW6WiN8z6bxEeLnanI4gUp6E61LYOqQLpCKtqCi01QUiSdIcMRgWRb3ddYBIiplTMoTQkNq163Yx29RapM6Wm+SGlVE5HpKsc0eLcs+lBeeuJWplq0rqXo6rxAcqumzmNKUtNV2dP4bGXK2KSyB2ibsknxV/cjVIe8qprMtFNxpdCTHstmjeLFdMrxSlH03OUdNU+V+5jOo6wtGZX3psHrDKNGKdH2RBuZ/mQ/4r5rRTq6QaP/e9w9SvuizuvtvITBaq6FGX30Vnbu2xmS7HJG5KRVxal9fFKjdH9flyuN5G2s2Ro3Whovkivqdh4izoqVwgvBe6oSJRmYSsK9PF92bJBHG+QWaBt+6Zio8dGiJX3f4+cY9LOle/QNbl26fIHfLFOTJFtHOSKjfFnwMj+7THa9ZWP1eXxiR239WyHpuWNEmc4KjihGYXL5MP+4ZkPx28n6KoD39XfufVBRqzi8gGgDybXFbYfdEqc2SBzRPlPiH7Z5dp7/QPibkkX6bPu9nkM92TPd9pk2NBSps5mYgOlemnNEz7M8nr1+tKv/sG6Pq7pvbEbSFX/OjrF3fqB99GOZ1suMr9lz23tkD9qOl6wr2bl6+v5wptAO+21t4P4AEAjxljHgbwmwB+21p7EMAygI/ddG88PDw8PG4Y11OCzgJwr7gU/7MA3g3gv+L2zwD4JwB+/7V2IMf5SRamRfoM2J8pmb2ysEQsLaTkDdtqkvSZyMmbuzw8yteSt26nRm/zJEv7TREMcN8999D3svKOa2dIQxiaEGl45iz1s36Jcsn0psUNMAjozZ1VLnatGp0/Nipl2FbnZ/l8KqmVKk3JPVcoSKpX0/PBgT9aRMJ6CT1SZJ0rPq8DoroRSWPdUPobsizaAs3zakeI2B7X2gsg8+fIt3X5TJwU23ESrEjoQ0wmZ1NyfqNOa5BXssShg1N8kOZt+uyr8bF0giSZ8V2qLF2aFy6UPdBxhVKatFXTVvZOMsn7QknlZhMJHdZpgdLv0JHaytU1ajsJncv7KfdCFyCk3S0jtx7KBa7JmsTRRZLiLiy8GB97KkOufhMDipQfIin11LyQi8fWSDJPcfBdSf0OXCoUo/Ic9SIae0cFCiX4uCM+zVX8EbUjguNC1wdorZcPm3UpK+mIxwEVdLc0e4H7KNpDhwPIOsxIlwZl7AP8my4UlGbNOXDCQPZuyAUwRnaTdtKsidS8ysFD+ZL0Y61KbTkVaDU8Qe7ITc75kkrKfOzeS7/bsupbj+eh1hLNusX3atTp9x4lxOJQKJD2Ul2T3yOi6yPtN8N1yfjGmAQXiJ4D8FcAXgWwYm1cVucCgCv8Lvi7jxtjnjXGPNtoXJ8ftYeHh4fHa8d1PdCttaG19gEAkwDeCuCuq522wXc/Za190Fr7oLaVeXh4eHi8vnhNfujW2hVjzDcBPAygYoxJspQ+CeCGEhHML5Pa1Va+pS6PiSpniUKGVJpmnaT8Rk/VneQaioMjQgaVR4hISqtIrFKSo9BY3c4lVF1BjvK8bXJ/3DbN0YfNBSFEhjt0/0rKKSdKLbdsbrCiidgcJ/GP5BphndS4M6+QSpqZPRMfC9h3drwipGGe/Wqrayrl52XQJhenBIcqNWfI5gNttUk7MogVraW6+OtmOFozm1KFDti5O1L1XDsdUhPbbJTr1uUGvSZ9Dhti3og6tB6BzjeSIzV//yFSqfsWxbwCjtps9FThB2cGUqlYXXbdbkhz327JGmRSbpyKsIWuzLAesZlEzZ8zw6zLwewiVrlJpxq2zryictC4AhFJlcY3y/EVSTbX1NvSx6OccvZEezZuK67V+V7St2SOVP+MK3SRVHl9khxpm9D7lFMYq/G56Nh4F2myk8el8wU5819i3Tyuf5xEikiscVxFqSJrO7mLfqNZ5TCwyHWEyxNk1iiNCCGc4wjboQGJzk6UuE9JmbdLi5Sa9tGHKN/S/LyY8OpNinkYGZdnxdGX6dFVmRBf8zyT9g1LG6uvLL/HAwdup2Oh8stPutrI8lzqOBMmk+G1mphj9vbTXq8uyhzNTcs63yiuKaEbY4aNMRX+nAPwkwCOAvhrAH+XT/sogD+96d54eHh4eNwwrkdCHwfwGUP1vgIAf2Kt/Yox5mUAnzfG/FMAPwTwhzfSAcMkpC7ekInoLV7KKhGdhbxsiYiwTEplI+QIzUReSLKBEkcHKte9sV30lu1wkYBWTaS+xhK94YsFkSAMk60HB5UbGJczCwJ629ZU8YHMAEnSvaSQH0vsknfk+OG4LbLOLZPIq9Ul6UeBox+7SvKpure+9tW8DFpwNCy5JtTy5lJM2Cp3PudSV6uzu1tbpNokR1c6Ag0AorgQhrpxj+7lSv719cvBXkQSf6OhwzbpT7Mt7lpN/kqbNab8sJjmApbebELmucE5MRKqAEqa90PEeUwaNRlLwG6ZkSoBpt0rL4fLhWOVOnM1e2Ls3shjX3e+S42itMCAydCkmtOwR3s8Yg0qrcaUMS46VUX8ct9yOdE8Iy6+4IrFBIp0TbrfUKBIczcanT2R1YwgvueV7og622J8dJPo20pBaRGc72appvo2To4CfUUhcZ0k7DJ6FkvyW1pZJWl2aFR+j5kyXS/Mye9lcZmk/F2sAcwd3Bsfe/VV2usjKhfO9Bnap1O7JDfLMmtzA2wmGC0JsZpgd+a2IkC5VgfqbWmb5/xQxQKNPa3yMzXW6LnQ7agcRebm3Ravx8vlMIA3XaX9FMie7uHh4eHxBoCPFPXw8PDYIdjy5FwFTpCV6hPzygBHmHWT0r1FTmVbYB/bbFrU20qRyIzCoCS6CUNSfYYror5PDtHn7x1/AQCwb3wqPtbmuqSvHD0bt/U4MdCBMSFLdt9DiaMWV4lcWVlTBGGOIzrros43uXBBS9sp2HSSTrg0sGJ+WGGVrX5Rkgf1unSeroB+P9YjUDYXF70X9ES1DzpcBzEt8xy1+L5snihlhPhJ8jV6KpLNspFBp61NBa4wCJm2KkMqipSJ465KouUIRNsT84fzCY8CmmfFZyKfJ9OPVSRdlgtz5HMyH67Wa8jJsLIqmjUwXFBEW6w2qS/rCmGsK4jhPuvvOXMDk5yRNj+wzcUqotQEVxbmSHGKYbC63emIidCZYaBUcVfII1ImFHBbgqNkkzlR7RPO/KJrc8b9vaK7YioKtCnFsb4ygcnkVcZ8GVJGCMIhNodmB8XUcWKRyEgdbzIxTmaPFpuehlRxjyyPqx2KiTITkCmkvyhxHllOetfmiOxMVp4BuTTt8aJKXDc2Qr/z2/dLUrjZOj1vamx6nJq8PT7WV+DfiTKxuURnuaLsyf4hjmewXIcYgmyG+pQvST8mCjSGUy+8gBuFl9A9PDw8dgiMlkJ+1JiYmLCPP/74Lbufh4eHx07AJz/5yR9Yax+81nleQvfw8PDYIfAPdA8PD48dAv9A9/Dw8Ngh8G6+SaUAAAUwSURBVA90Dw8Pjx2CW0qKGmPmAdQB3HzxvK3FELb3GLZ7/4HtP4bt3n9g+49hO/V/r7V2+Fon3dIHOgAYY569Hrb2jYztPobt3n9g+49hu/cf2P5j2O79vxq8ycXDw8Njh8A/0D08PDx2CLbigf6pLbjn643tPobt3n9g+49hu/cf2P5j2O79vwK33Ibu4eHh4fGjgTe5eHh4eOwQ3NIHujHmMWPMMWPMSWPMJ27lvW8Expjdxpi/NsYcNca8ZIz5ZW4fMMb8lTHmBP/tv9a1thJc5PuHxpiv8P/3GWOe5v5/wRiTvtY1thLGmIox5ovGmFd4LX5sG67B/8B76Igx5nPGmOwbeR2MMX9kjJkzxhxRbVedc0P4l/y7PmyMefPW9VywwRh+i/fRYWPMf3DV2PjYr/IYjhlj3r81vb453LIHOlc8+j0AHwBwCMAvGGMO3ar73yB6AP6xtfYuUB3Vf8h9/gSAJ621BwE8yf9/I+OXQWUDHX4TwG9z/5cBfGxLenX9+F0Af2mtvROUOfgottEaGGN2AfjvATxorb0HVLfpI3hjr8OnATx2WdtGc/4BAAf53+MAfv8W9fFa+DSuHMNfAbjHWnsfgOMAfhUA+Hf9EQB383f+NT+zthVupYT+VgAnrbWnrLUdAJ8H8OFbeP/XDGvtjLX2Of5cBT1IdoH6/Rk+7TMAfmZrenhtGGMmAfwUgH/H/zcA3g3gi3zKG73/ZQDvBJc4tNZ2rLUr2EZrwEgCyBljkgDyAGbwBl4Ha+23ACxd1rzRnH8YwB9bwvdABeTHscW42histV/jwvYA8D1QgXuAxvB5a23bWnsawElsw4pst/KBvgvAefX/C9y2LWCMmQKV4nsawKi1dgaghz6AkY2/ueX4HQD/MwBXZWIQwIra1G/0ddgPYB7A/8Nmo39njClgG62BtXYawP8N4BzoQb4K4AfYXusAbDzn2/W3/V8D+Av+vF3HsA638oF+tdIm28LFxhhTBPAlAB+31q5tdX+uF8aYDwGYs9b+QDdf5dQ38jokAbwZwO9ba98ESh3xhjWvXA1sa/4wgH0AJgAUQGaKy/FGXofNsN32FIwxvwYyqX7WNV3ltDf0GK6GW/lAvwBgt/r/JICLt/D+NwRjTAr0MP+stfbL3DzrVEr+O7dV/bsGHgXwd4wxZ0AmrneDJPYKq/7AG38dLgC4YK19mv//RdADfrusAQD8JIDT1tp5a20XwJcBPILttQ7AxnO+rX7bxpiPAvgQgF+04re9rcawEW7lA/0ZAAeZ2U+DCIgnbuH9XzPY3vyHAI5aa/+FOvQEgI/y548C+NNb3bfrgbX2V621k9baKdB8f8Na+4sA/hrA3+XT3rD9BwBr7SUA540xd3DTewC8jG2yBoxzAB42xuR5T7kxbJt1YGw0508A+Pvs7fIwgFVnmnmjwRjzGIBfAfB3rLWqei2eAPARY0zGGLMPRPB+fyv6eFOw1t6yfwA+CGKWXwXwa7fy3jfY37eD1K7DAJ7nfx8E2aGfBHCC/w5sdV+vYyzvAvAV/rwftFlPAvj/AGS2un/X6PsDAJ7ldfiPAPq32xoA+CSAVwAcAfD/Asi8kdcBwOdA9v4uSHr92EZzDjJX/B7/rl8EefO8UcdwEmQrd7/nP1Dn/xqP4RiAD2x1/2/kn48U9fDw8Ngh8JGiHh4eHjsE/oHu4eHhsUPgH+geHh4eOwT+ge7h4eGxQ+Af6B4eHh47BP6B7uHh4bFD4B/oHh4eHjsE/oHu4eHhsUPw/wOI+Mo5PuMu9gAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " deer  bird  ship  deer\n"
     ]
    }
   ],
   "source": [
    "# functions to show an image\n",
    "def imshow(X):\n",
    "    img = torchvision.utils.make_grid(X)\n",
    "    img = img / 2 + 0.5     # unnormalize\n",
    "    npimg = img.numpy()\n",
    "    plt.imshow(np.transpose(npimg, (1, 2, 0)))\n",
    "    plt.show()\n",
    "\n",
    "# get some training images\n",
    "dataiter = iter(trainloader)\n",
    "images, labels = dataiter.next()\n",
    "images, labels = images[:4], labels[:4]\n",
    "\n",
    "# show images\n",
    "imshow(images)\n",
    "print(' '.join('%5s' % classes[labels[j]] for j in range(4)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As for the LRP implementation, it is again convenient to layerize the network. That gives us an easy way to inspect features of different layers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0 3.8736910820007324\n",
      "1000 0.6775739192962646\n",
      "2000 0.3981252610683441\n",
      "3000 0.29655250906944275\n",
      "4000 0.2449469417333603\n",
      "5000 0.21303337812423706\n",
      "6000 0.19146105647087097\n",
      "7000 0.17538194358348846\n",
      "8000 0.16269393265247345\n",
      "9000 0.1523885875940323\n",
      "10000 0.14385616779327393\n",
      "11000 0.13661091029644012\n",
      "12000 0.1303723156452179\n",
      "13000 0.12492857128381729\n",
      "14000 0.12011914700269699\n",
      "15000 0.11583078652620316\n",
      "16000 0.11194639652967453\n",
      "17000 0.10839983820915222\n",
      "18000 0.10514813661575317\n",
      "19000 0.10214952379465103\n",
      "20000 0.0993882492184639\n",
      "21000 0.09683166444301605\n",
      "22000 0.09444396197795868\n",
      "23000 0.09221692383289337\n",
      "24000 0.09012703597545624\n",
      "25000 0.08817078173160553\n",
      "26000 0.08633121848106384\n",
      "27000 0.08459774404764175\n",
      "28000 0.08296918123960495\n",
      "29000 0.08142450451850891\n",
      "30000 0.07994592189788818\n",
      "31000 0.07853511720895767\n",
      "32000 0.07719370722770691\n",
      "33000 0.07591249793767929\n",
      "34000 0.07469099014997482\n",
      "35000 0.07352511584758759\n",
      "36000 0.07240469008684158\n",
      "37000 0.07133468985557556\n",
      "38000 0.07031258195638657\n",
      "39000 0.06933289021253586\n",
      "40000 0.06839229166507721\n",
      "41000 0.06748846173286438\n",
      "42000 0.06662146002054214\n",
      "43000 0.06578493863344193\n",
      "44000 0.06497761607170105\n",
      "45000 0.06420092284679413\n",
      "46000 0.06345275044441223\n",
      "47000 0.06272865831851959\n",
      "48000 0.062026575207710266\n",
      "49000 0.061345044523477554\n",
      "50000 0.06068610027432442\n",
      "51000 0.06004726514220238\n",
      "52000 0.05943002551794052\n",
      "53000 0.05883055925369263\n",
      "54000 0.05824805423617363\n",
      "55000 0.05768175795674324\n",
      "56000 0.057131048291921616\n",
      "57000 0.05659656226634979\n",
      "58000 0.05607617273926735\n",
      "59000 0.05556536093354225\n",
      "60000 0.05506730079650879\n",
      "61000 0.05458199232816696\n",
      "62000 0.054108913987874985\n",
      "63000 0.05364890396595001\n",
      "64000 0.05320132151246071\n",
      "65000 0.0527658574283123\n",
      "66000 0.052342213690280914\n",
      "67000 0.05192993953824043\n",
      "68000 0.051528677344322205\n",
      "69000 0.05113711208105087\n",
      "70000 0.05075375735759735\n",
      "71000 0.05038023740053177\n",
      "72000 0.05001489818096161\n",
      "73000 0.049656614661216736\n",
      "74000 0.04930398613214493\n",
      "75000 0.048959698528051376\n",
      "76000 0.0486229732632637\n",
      "77000 0.04829320311546326\n",
      "78000 0.04797012358903885\n",
      "79000 0.04765351489186287\n",
      "80000 0.047342799603939056\n",
      "81000 0.04703674837946892\n",
      "82000 0.04673757776618004\n",
      "83000 0.04644353315234184\n",
      "84000 0.046156078577041626\n",
      "85000 0.04587491229176521\n",
      "86000 0.045599378645420074\n",
      "87000 0.04532903432846069\n",
      "88000 0.04506521299481392\n",
      "89000 0.044807422906160355\n",
      "90000 0.044554952532052994\n",
      "91000 0.04430672153830528\n",
      "92000 0.04406316205859184\n",
      "93000 0.043824050575494766\n",
      "94000 0.04358942061662674\n",
      "95000 0.043359026312828064\n",
      "96000 0.04313206300139427\n",
      "97000 0.04290908947587013\n",
      "98000 0.042689546942710876\n",
      "99000 0.042473696172237396\n",
      "100000 0.04226198047399521\n",
      "101000 0.04205334559082985\n",
      "102000 0.04184792935848236\n",
      "103000 0.04164500907063484\n",
      "104000 0.04144449532032013\n",
      "105000 0.041247375309467316\n",
      "106000 0.04105297848582268\n",
      "107000 0.04086211696267128\n",
      "108000 0.04067465290427208\n",
      "109000 0.040490489453077316\n",
      "110000 0.040309906005859375\n",
      "111000 0.04013228416442871\n",
      "112000 0.03995772823691368\n",
      "113000 0.03978567570447922\n",
      "114000 0.03961645066738129\n",
      "115000 0.03945019841194153\n",
      "116000 0.03928709775209427\n",
      "117000 0.03912616893649101\n",
      "118000 0.038967739790678024\n",
      "119000 0.03881218656897545\n",
      "120000 0.038659196346998215\n",
      "121000 0.03850821033120155\n",
      "122000 0.03835972398519516\n",
      "123000 0.03821331635117531\n",
      "124000 0.03806803375482559\n",
      "125000 0.03792473301291466\n",
      "126000 0.037783313542604446\n",
      "127000 0.037643998861312866\n",
      "128000 0.03750693425536156\n",
      "129000 0.0373719222843647\n",
      "130000 0.037238746881484985\n",
      "131000 0.03710734099149704\n",
      "132000 0.03697754815220833\n",
      "133000 0.03684935346245766\n",
      "134000 0.036722857505083084\n",
      "135000 0.036597225815057755\n",
      "136000 0.036473192274570465\n",
      "137000 0.03635040298104286\n",
      "138000 0.03622943162918091\n",
      "139000 0.036110516637563705\n",
      "140000 0.03599325194954872\n",
      "141000 0.03587742894887924\n",
      "142000 0.03576269373297691\n",
      "143000 0.03564915060997009\n",
      "144000 0.035536959767341614\n",
      "145000 0.03542624041438103\n",
      "146000 0.03531660884618759\n",
      "147000 0.03520812466740608\n",
      "148000 0.035101037472486496\n",
      "149000 0.03499531373381615\n",
      "150000 0.03489099442958832\n",
      "151000 0.03478766977787018\n",
      "152000 0.03468543663620949\n",
      "153000 0.03458434343338013\n",
      "154000 0.03448450565338135\n",
      "155000 0.03438597172498703\n",
      "156000 0.03428848460316658\n",
      "157000 0.034192003309726715\n",
      "158000 0.03409690037369728\n",
      "159000 0.03400309011340141\n",
      "160000 0.03391050547361374\n",
      "161000 0.03381885588169098\n",
      "162000 0.03372829407453537\n",
      "163000 0.03363867476582527\n",
      "164000 0.033550042659044266\n",
      "165000 0.033462509512901306\n",
      "166000 0.033375997096300125\n",
      "167000 0.03329043462872505\n",
      "168000 0.033205799758434296\n",
      "169000 0.03312204033136368\n",
      "170000 0.03303899988532066\n",
      "171000 0.032956693321466446\n",
      "172000 0.03287512809038162\n",
      "173000 0.03279375657439232\n",
      "174000 0.032713111490011215\n",
      "175000 0.0326334685087204\n",
      "176000 0.03255471587181091\n",
      "177000 0.03247661143541336\n",
      "178000 0.032399389892816544\n",
      "179000 0.03232307359576225\n",
      "180000 0.032247524708509445\n",
      "181000 0.03217281773686409\n",
      "182000 0.032098669558763504\n",
      "183000 0.032025035470724106\n",
      "184000 0.03195224702358246\n",
      "185000 0.03188043832778931\n",
      "186000 0.03180953487753868\n",
      "187000 0.03173937648534775\n",
      "188000 0.03167000040411949\n",
      "189000 0.03160140663385391\n",
      "190000 0.03153359889984131\n",
      "191000 0.03146650269627571\n",
      "192000 0.03140006959438324\n",
      "193000 0.03133425489068031\n",
      "194000 0.03126898780465126\n",
      "195000 0.031204285100102425\n",
      "196000 0.031140102073550224\n",
      "197000 0.031076576560735703\n",
      "198000 0.031013594940304756\n",
      "199000 0.030951261520385742\n",
      "200000 0.030889317393302917\n",
      "201000 0.030827876180410385\n",
      "202000 0.030766913667321205\n",
      "203000 0.03070656582713127\n",
      "204000 0.03064683824777603\n",
      "205000 0.030587662011384964\n",
      "206000 0.0305289588868618\n",
      "207000 0.030470451340079308\n",
      "208000 0.030411602929234505\n",
      "209000 0.030353471636772156\n",
      "210000 0.030296077951788902\n",
      "211000 0.030239207670092583\n",
      "212000 0.030182987451553345\n",
      "213000 0.03012746013700962\n",
      "214000 0.030072564259171486\n",
      "215000 0.03001786395907402\n",
      "216000 0.02996370568871498\n",
      "217000 0.0299101322889328\n",
      "218000 0.029857084155082703\n",
      "219000 0.029804592952132225\n",
      "220000 0.029752569273114204\n",
      "221000 0.029701068997383118\n",
      "222000 0.029650049284100533\n",
      "223000 0.02959963120520115\n",
      "224000 0.029549654573202133\n",
      "225000 0.029500173404812813\n",
      "226000 0.029451116919517517\n",
      "227000 0.02940240502357483\n",
      "228000 0.02935410849750042\n",
      "229000 0.029306188225746155\n",
      "230000 0.029258687049150467\n",
      "231000 0.029211657121777534\n",
      "232000 0.029165048152208328\n",
      "233000 0.029118800535798073\n",
      "234000 0.02907288447022438\n",
      "235000 0.0290273018181324\n",
      "236000 0.028982212767004967\n",
      "237000 0.028937552124261856\n",
      "238000 0.02889329008758068\n",
      "239000 0.028849398717284203\n",
      "240000 0.0288058053702116\n",
      "241000 0.028762537986040115\n",
      "242000 0.028719618916511536\n",
      "243000 0.028677038848400116\n",
      "244000 0.028634848073124886\n",
      "245000 0.028592782095074654\n",
      "246000 0.028550993651151657\n",
      "247000 0.028509465977549553\n",
      "248000 0.028468213975429535\n",
      "249000 0.028426796197891235\n",
      "250000 0.028385592624545097\n",
      "251000 0.028344623744487762\n",
      "252000 0.028304120525717735\n",
      "253000 0.028264129534363747\n",
      "254000 0.028224488720297813\n",
      "255000 0.028185222297906876\n",
      "256000 0.028146274387836456\n",
      "257000 0.02810765989124775\n",
      "258000 0.028069306164979935\n",
      "259000 0.02803098037838936\n",
      "260000 0.027992768213152885\n",
      "261000 0.027954889461398125\n",
      "262000 0.02791721373796463\n",
      "263000 0.027879860252141953\n",
      "264000 0.02784276008605957\n",
      "265000 0.027805907651782036\n",
      "266000 0.02776934951543808\n",
      "267000 0.02773306705057621\n",
      "268000 0.027696998789906502\n",
      "269000 0.02766118012368679\n",
      "270000 0.027625663205981255\n",
      "271000 0.027590451762080193\n",
      "272000 0.027555376291275024\n",
      "273000 0.027520442381501198\n",
      "274000 0.027485765516757965\n",
      "275000 0.02745130844414234\n",
      "276000 0.027417132630944252\n",
      "277000 0.027383143082261086\n",
      "278000 0.027349477633833885\n",
      "279000 0.027315853163599968\n",
      "280000 0.027282023802399635\n",
      "281000 0.027248751372098923\n",
      "282000 0.027215808629989624\n",
      "283000 0.02718314528465271\n",
      "284000 0.027150727808475494\n",
      "285000 0.027118606492877007\n",
      "286000 0.02708672173321247\n",
      "287000 0.027055008336901665\n",
      "288000 0.027023546397686005\n",
      "289000 0.026992352679371834\n",
      "290000 0.026961423456668854\n",
      "291000 0.02693076804280281\n",
      "292000 0.02690032310783863\n",
      "293000 0.026870040223002434\n",
      "294000 0.026839932426810265\n",
      "295000 0.02681000344455242\n",
      "296000 0.026780232787132263\n",
      "297000 0.02675069123506546\n",
      "298000 0.026721304282546043\n",
      "299000 0.026691999286413193\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAB6CAYAAACvHqiXAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzsXXVcVdn3XcfuQMVWDOxOFAMTG7u7u7t42N3dYmM3JhgoqGBio4gt5qjY3t8f63k2OjrOOPOzvnd9PvMZ3Oe++26ce9/ee629jzIMAyZMmDBh4tdHhB99ACZMmDBh4r+B+UI3YcKEid8E5gvdhAkTJn4TmC90EyZMmPhNYL7QTZgwYeI3gflCN2HChInfBOYL3YQJEyZ+E/yrF7pSqpxS6oJS6rJSqu9/dVAmTJgwYeKfQ31rYZFSKiKAiwDKALgB4CiAeoZhnP3vDs+ECRMmTPxdRPoXny0A4LJhGFcAQCm1EoALgC++0GPEiGHEixfvX3ylCRMmTPzv4fbt2/cNw0j0te3+zQs9OYDr4f59A0DBv/pAvHjx0Lp163/xlSZMmDDxvwc3N7drf2e7f5NDV5+x/Sl/o5RqrZQ6ppQ6FhYW9i++zoQJEyZM/BX+zQv9BoCU4f6dAsCtTzcyDGOOYRj5DMPIFyNGjH/xdSZMmDBh4q/wb17oRwHYK6XSKKWiAKgLYNN/c1gmTJgwYeKf4ptz6IZhvFVKdQSwA0BEAAsMwwj8p/uxXDgCAFAHt4lxkPXvR7u1aUHfZwCA/WgDAMjufl+Plcq5HQDwyqGUthUI6gUACHi4RtvO9c8GAGi0sSIAoEXrhXqsu2N0AMCO+q+17VjKhAAAu7t1tc2IshIAsOYVs0u2XTPqsdivqdzctreZnMtQ/m/xGMlQlTx6BgCQKvZaAEDd8qv12MoYz6wfuKptg5z52UU7omhbc9d+CI9Zbrn031Ny8bzqnNiobR6DeT1iDBmnbQdPbAAAjNpRkp/bVkCPdfrjAQDgfqR72hbrPY+j0Khs2nZzWj0AQOiG/gCAsyFynq6d+f++lyUTt8R6Cl2Py3aHNu/leQbG57k9q6/H+gfPAQAsd8utbadirgcAjO7QUNsC1nL8UdA+AMC+nmX0WOv1fgCAO8OjalveCK8AAJaXFnyK9Klom/KkgrYl6lYNAJAfR7TNbV4j/tGqOAAgWhs5z2LVx3Ao8TltK5jTGQCwSNXTtsHWLGVfuAAARpXpqcfuxCgGAEhySOZTp368NlPnLtC2mWc5f4r1fAsAyDr+sB47hJwAgMKX+mvb9Tce/O4sa7Vt08TZAIAHd3hOx48O0mPPW9QCADj2kWNzcRkIAFjlI4/8KJesCI/GY9Pqv0+GOQAAauGytr0zugAAUt4S2/bkkQEAHhn53JYKSKPHVsZcxvOFPOeGM+/zg+UrtS3WH1m43SW+3mIm76LH3Kpxrt8IcNS2vbFaAAA2JpDjGMPpj6JR6KOmKP9Kj73Z7QUAOHs9urYF9dgKAMg0W+73q6uc12GleU73JgrFaEx/AwBQbRtrG7YtAgC42o3Ht+LfkKIwDGMbgG1f3dCECRMmTPy/41+90P8LVChLzypFJfGWb5zlL58xYpG27X9jDwAY2PsEACDfdJE/5vRLBQBY4lFF284mqwEAmHQru7ZV28jtennTMx0zd5Qea+NUGgAQL4p4LXtC6T0NC5igbZnK0hPodIke5umby/WY6zR6FWqmeHEIoXfTM1wyavl4f55fPrqwSUuJR5VgCX/9028VDzbL5TwAgMkZRSF0ErcRHnericdRpxm9+61NxSu7OOQPAEDXfuJFtnLnd2RM1w4A0DmHeBcVVzACSjdfjiPrzVgAgJbhrlu89gcBAKMWdQQArE1US4+lLsXf+rI7hmpbImMPAKD84XXadqQXjy3owGQAQJGoEgFEHFMUAOB2U7Yfdose7v0OEh1VeZ+e5zQyGgCgQg+JYAqsWwwAuJVykra1vM7zSm5xxad4+ow362h38VWMLdbrUG+ftjle41xZ0O4FAGBlw2p6rEUxztfAdz7a9k7tAgAMyi73YFBpRhmZJiTj9/Qco8fcJ/L/nd+LN75iSh0AQMhVD21raGFUaRvK8KdL9SA9VnetGwCggZ+nto1lIITS60Wk0Ko7I4Rrpen9dos4RY95NygEAIiGd9q22Jn7jb1OoocB+Bhp90rk2ceBnug7Q+7Z0Mr8/6CxObXtkcHne7EnvVu3mLLXfc8uAACMqhfkOHakBgCsLSPHYV9sPwAg5FhXAMDqWhv0WHCBuAAAh+ES8aUNGwsAqD5Y9ptuGc+18y3O6wNLn+qxLZ1mAgBcKl2Sk61wHgBQ8sF+bYrdfhUAYPR1Uos1cw+W4z5DLzy+p0SNlaJ8eKdJdPRPYZb+mzBhwsRvAvOFbsKECRO/CX54ymVbM4acDVUhbVvai2F7acNJ24ZZDzW1D9MmHfwm6zF1jmRDxsztta1olgwAgCgjU2nb67kMoXOcZgpji58cx/KCPI4NJSTMLja4AwBgXKMZ2tYPdgCAChlIHtWZckiOIwdTKUv3CJlbZxRD7wteQu64BXfnH4FXAADlvBLrsfX3XgIA8ldIrW1RVQkAQDW01baTrh+nCgwl4bM6SaLoYnUHbXsW8BwAMHbEFW2LZCWYzyuGkMnryf4bvXUHANQOyaxtb+KVBQA0uVZJ26bVYaj7sBrTDhNm7tRjqUZzu/u2Epomfu0NANjgmk/b1jmGAAB6zqOs9f5rIbhaRuR3likm9zaaR3UAwOyRc7St6U4eb6OS1pROETmOitkDAABpbvhqWynwGm3DUnyKjFV5/2Z3dtG2VUOYJtkWbs6cnEfSckQ+kl4rZr7RY49r8n7X3tRb2x68ItMWfaSkg4ZGYIohtuIxTlhfVY/1sPLXte33aNuaFkyJNFUSlmdUPKjBORcBAGocl/0XzcbUwsAzQqifGEDy9M0pSXMGDeR18B/I8xxQf5ke83YlOXz+qaSbttpsAQB0WDFW2+D18ZzMNiocYQqSkWswTdtab+a53hySQtsuuzL9dg2cp6drNNdjZWPNBwAs85f0iq8Hn9fAUZ21bec4pj98/Kmq7jK+sB67PpPPdMFDTbXNvlV+AECtOJIK86x7CgAQpzjvaYG48g4oXIbzqIzPW2172J3f1aBkcW3rs4Rpq73dSA77DKupxw404nYPz8n9Prc6FACwKtwl/acwPXQTJkyY+E3wwz30ZIgDADgEKTpSY+hV90BXbYs3nkSZWx6Sl02VeLwGKLdTyzaLrSbJqB0d/LVt8n5GAXO2U6bk3Fm8kLu+JN0OthRSI3AMvfw0MzJpW9Gq/HUuPpDex4jON/TYcoNey6rMB7TtansSjhm9xQvPNY6exgEHEkuLVH49NnY+vdOO4SKWaJ4kUCZdFvkVQvEx1gqhgxoRAQDd1j7TphXVywMASqURcqxpMD3y3GVIIOcoINLHeI70glZMc5LjPk9S0Wa2kM+RA+gxTslQGwDg07KNHmu8lFFMw8MiL5y6vBMAIG4RuS+Od3kv53tTjliozRA9lt6Px32khMyFOJl5v/dmk0jh1jBGKFUG8/wCveS4m6xhBLK/7E1tuzjA3noCfyZFh6ZidOSsxFVqndMCADCch2vbgXPcrkcmemojt4nMsdNyRoH2r7ZqW/0E/PtArHnalk/x/K7l5TEW3ixk3fCLlM2+cOulbeN9+LwUzCoe45Fs9K5bOTQFAEQt20qPxWzBazreRojskFb06BMFy/122XsXABB8mwTyzIUH9ZihRgMAVAHxPvfwsNEeIgqwuIabgwDSrJdosDn4DNWP1FHbus0g8XkusUSSeQzOp75L+GysDpL56rCGpP2TKiI6GJ+Nx51miDz7ezLR9vwAPe7rhnxn3UI8h5ORtmjbY9dZ3K6mvA57BJMU3TKgB/c1Q+bf+uwkt/PHLSEnW59E+uhMLbWpC0YAABYc4L5u3xXBgO1J6/3rlETbupbk81UonIz5n8L00E2YMGHiN4H5QjdhwoSJ3wQ/POUyFdQZp4kibWAm7qXmt1tJ0W0uaMGw5WF86q99T0rIeWslta3HHibUtptL3gMA5mW5o22rTjI0PWUlqlbOlRByZGUSWwNn9dC2t52sJOrAR9rW4wkJ2Ka7GwAAag4RIsxFMe3g0EsqNP2j8e/gvA+0rWl5EoJ+a48CAJ7sc9NjG2eQqHKpJKF6lXI5AABlT4km12Ptx6kCNais/vsyLylOSmSPUukZyj6JLseR+ipDxyrDGXK2r5NMj80azeuwr7lU/1WJfREAUHq66K3HNabGfPgThvFjrGkWALD4Mz2RvbPojFP5kDytKlwh+vVkCqreG6anBgQIWTc9I0P124klbG6VnbrraIbc7yrTmWbaWoKpl4RFZT6V6cP9bk8j55J6JfXelxs9x6dwXUMSbUcLSXutHs9UR2DjP7RtZwOmCe/483j3VNilx2xiUhefbYRot5ee4LnXPWCnbdf6kkx7/ZJVpENqCNmZzpo5iVRCUij3I1FnH62vzLEl3Xn9GjVg+iOaTGuU8mKa6UTKbtqW8TxJ+eodZS6cvEZi2tmLaaZIvfLqsdoTgwEA9sIB4lrQMQDAmLRCbuMTTf90yTAgdT+mY5Y9F+JxdjQ7AEC9iUW0rW0AdeKTRzAl9j63VIRX8JzK4298Xtu21iKpeHerTPasb5kuatGewomks6Sj995HvAdVIuyV7/SxVsdmeqxtA5cxndJmBlO3ta/Y67FkbXhNF45sqm3FR/J+3/CXWpEjiXkNl2zke0eVLq3HGt1javWEv9STlD2+AgDwtPy3v5ZND92ECRMmfhP8cA+9Jljh+IeXeDexHVk91bWGVBgmK0AP9H4cejDbQqXHSOZ59NBrlZBKx+fN+Cua/YJ4+Q9KU+Y2Jitdh/dzwlV57iEZFNR/sba5+7IF8eS5UmkW8QFlV+uSU/ZmP/uUHtuzmT1lxtwVgqZsKD2pEpvE4w5LSwmX7WqSTbP7ikQx2l16FQ/fCWHVcTC9MsMrfHdiC8LjxbCJsg8/ehprQ6VvS6vL7EczL6XIu+Z3536bHGUlrEtcIQ23bKPU8GqoXL9iYyjR9F0tXuqWjCSwF91LDgC4vkfuS7mD9EL6HIymbdcd6Ml7lhWPampSei65StO76VpW7sG1DccBAI+eyHH3asjvWDppurbVekyPsWVzEmbNu3vpsTBrC42az9Jr2zTdTyVcGGNF9iOMhA49FCI7ky3lbsHFnbRtmz8/a8nZFABQprKQggvvBQMANhYVgqt+H5K9y3aKjPNEVc77HUN4jd4VkkfyYUTOU4fD87Vt41JGa1luSXvsqWsYfXWyRksuxaRqeNAlztf6eUT+l+4NiddrY4X03byYBPr8IpxHXWvb6LFa8axE32npsdM6B+dACXE6UVi+FgDgdsVb/500L4ndhHtFtphiAM/hRLdgbbuxnFHUqtL87ImNIn0cXIK9ZNxby1wouJHhSMyL8mz0H8n5fCA7I2vVQubCAEcS13mXyHX2PM6/Wzsu0rZJzxi1zq3HexZ7wgs9ZsnK4zi7Rgjy2y4kTV/WEWnsMgvFBnFdGC32MKrrsRUlGW24p+ykbbkr89lw/RfdVEwP3YQJEyZ+E5gvdBMmTJj4TfDDUy6D7pNUfFxY2p1O78Gqv7hqtrbVykLCZcEfTMeM6X1Mjz2ZRp32izSiIS/x1gIAmOYv1VmRizHE7DuYIb2No+i/C60hmda8tqRL5udluN/rlITqZ/xPAwBqrmEDpAlJh+mxN+9p6xA4UNuaXWFIdeW8EHJTs1Kf2/IttcGlB57UYx19WJnZppxUOrZqwVBaPZNKTlcP0a8CQPRS0pJ1z4ukPJ7yQiR2XBwTAJC6rISmufaTLIw0m+dU109+32sGkqiNY2khx9Ga6YH9rSRNsWSCLQBgyAimKR5Wlpapqa+x+rF8I2m7+siPJNCJFkKUGvVJ0rUpRW1wPsh9j9GX+uw2Lie0LWv0JgCAlKNluwrrmT4r/JIpuXv+QlbXLkVCa+37mdp2dTDJyMr4sw691iPmEbxWiSZ8e1sKrzMuF5Iuyz4e7+0X1FH3cJdwu6O1iddy3+7aNjE2r8euIUKyF2jJ+oehl5nW6N1UHsn160nO3swhDb7UURLB/vuEBA+5b73fo1gNemxqZT12vS7J0GEP12vbtGcct50gK0jm9WCq5cIspun2jZdcSr6SJH0TvaiobXUacs6+XbVC2+AoZCUAzH8pJOM6kIRc3EHSZAWGcS72GChVwMnqcZ6O7sMq3W495Tzr3+MzYXSWVOzY3TyXtYFCYM9KzON1bsv3yPs/ZumxoAVMd03YLPfxzlmmGt9mFrJ/9VbOo7QJ2VQstru01X5cz9oMUHXQti4tmJqcPlfeQQcsTgCALKN4zy7XknRTv9h8b+RNIceBqB/eEfKc/1OYHroJEyZM/Cb4qoeulFoAoBKAe4ZhZLPabACsAmAHIBhAbcMwHn1pH3+FtFtItG0KtybE1b1sEbqghLu2tVnCirFHL8sBAK6vkxarMwfQE610WSrOlEEyo9Y2Oay+ia1ytPL0JhcVkmrFTqXpOb4ZHVfbouci0TdtvTTUt5u9BADQOpARQkhwST2WYwmlc4mKiRwsgiOrOxfdEWIwdwfuo0UDesYOg0Ri1zkrF/ZuGyqeycXU9Jq2RJY2u0f7f+yhI6X07yiVhcdUOIsQpT41KMvcebqPtpVPTo/OOEed49TmUpV3qTL3n/GQRCeLgukF1Rko5HOkDfRS0zXioh1XT0t0MjwxidiEE4Qs7DiU3se9ruXl2POz8rT6DO5j7l0hj/bno3wyak2JCsY1oaezpv1LbUthS4Lq6RurRM1TFki/cYrelTFVpGe7xnNu+fSUc/6A6ffpzY6IIj2Pp1zlBPVqL9LOa/nocY8axrEuN6TVsP0YtlLekzW2tk0NtC6KEk6+lusNr4cqzrmz85J4uSXy8vpleC5zfd0NViZfTiqk+d47fIb8+zIS8m0j1boXbtDzPxdPpIENDtHj9/OShVue76K8d35DyvU8uu3QY6kG87zWSUCEEmU4Z7w9M2ib6/6PPfRaOaWKel8gF50oW0gWefC6yO9a5SqywoSKVZ6vCjN6vZBMFl0Zup7bLckgc2HGEZ7znk1SnT2wAoUF6wekAwCMu+esxzauYnRe8KZUL+ceQ4I3cIiIAqIcYKVv6jaUHN5P/USPOfvPBQDsLCrXueAFPtOOsUSCvD4rJZXusZhVaLNmiR7b1o3XOyCJLJLhdoqk83t8O/6Oh74IQLlPbH0B7DEMwx7AHuu/TZgwYcLED4QyDOPrGyllB2BLOA/9AgAnwzBuK6WSAvA2jHCd67+AZMmSGa1bt/7INjMqRf0lEklOfGUryn0WB47WtidzmO9+PZkFRc9GiTzOkoYefZdX4rWExmGHuopekuOLGIm6qpWveaiWJLLk1FRrrBJ0Xn67Bj3k752qKrmvgYmYq66/gtKiK3bi+dS6R2nWrjuSEz+dir91CTylF0TOx8yrv77P4+1iG6LHJqSwAABeXpbCqQd/0FNM9ljyj08nyfkDgLebeAZl0/Cz/a9IbjJLGt7nZMHymcr3eEzFG1Oilr6/SCtPNWGzfU9HycdGCWGOOEI7yY2m8aI/4RuD9yDaJMkjp2pHOeab+Iu0LfQive+YOaVXSNIrjF7SJqD3uzifLPd1bSXve2Ml8tBs6ehV590l+f0ScenFpq/MqGveacnRX39HruBMCilY6v+aBT9V4nzsVQKA10tey/2j5NnYvYOeZa6Rx7Wt+zR6ebOP8DoOSX9aj9VYzfMr986ibZXqca6PWZxc2/qeJmdSMS7P+Uh+4XWKvWZ+/U4ekSiG7qAEb0NG6e/SfAPv3/Xi/P4zd4XHGFKVHmn38+JhvslDLqGRh1zTqwso0awEyhsvz8uhxzzyci7k2yOLygyuwPvtvUQKY7yifOxb2rlJRDRsFXPSl4PsZIMdzC1vfnhXm/4oSlvRJowoW4dJtDunJZ+XCBmEj6pspY4aPBF5qO0YevUv+jLKiFZLev48rMV5kT6+yKR3JeY1dTwt0UDEuHy+lkxkQVmJYZJCSBLCCOCAIUV0diUZId/JI/c2aYeRAIC4Kfgeq/Moph4Ls/I5SVtJ19iUvgkAAM2zf/xsA4Cbm5u/YRj5/jTwCb41h57YMIzbAGD9v+037seECRMmTPxH+H8nRZVSrZVSx5RSx8LCwr7+ARMmTJgw8U34VtniXaVU0nAplz/HCFYYhjEHwByAKZdPx+/1J+m20tGibWOrMsUw6IC0UT1mXRQidAUrtrbXlcbwaizJvTFbRdrWYh/lh6erSdomKDt/v8rGIzHomVVkfVEXs0prwmtJO0xNRNKjgr+QgLHjkeh5eJ0heNV7QqpdS0dC8EIW2UfZlTzO2MsWaVvaOOyJElMx3CpdTypW9+QluROwXKo8R65ntemErau0zTXFx5e8eWK5tE6VrH0+ukjoPeo9Q16/bHKNuiZiGL7+NdNCzqNF4lkmG1t5VngrTNixfezZmy5rsLa970jp1v0+TJtkeJROjzl2YErrdLDIzK7YkYD6I7H4Eq+i8u9387nfHE5CLp4uyzTC0wWSCkuVmvu1nBQp2atFJL/jbmfqILBsRD22Ozr7tSSsX1vbZoPpna2u0iflA/aPpm18IUkPpizLdFdjZ0mJVJtD2V/2jkxPJHgtaR6fKOz1M3SbpL3yLeKcWXRa1g2dlJ1h/opl3O6Ol6xdebsVU1Z7n0uKq2AaplpaH5O1cn0qUBJY6hRTAAPXyRq4C3tTImt7RVJ+ri4PAQABBSWdkbkM5Xw3/bwBAElvSqotgxfnafAyIRcrj2Ufm7pbpJK4zRYLwqPpvgb672rBTEOqs0Lm9x1FQv1CiKTOymdkqrTeWhLlD22CZf/VKRSotUWegyohTIeeOSS9WRr2oXTw7WH2AVriLlWvz+pTEPHyqsydO76s2gwtKXN3dwRKnMvPZAXtwQTSarvHDb5v7vQSkUKirUzjRqgq82lxX6b/TjXg8ziwmVzvXpeZWspZSsjZWPWt13Lun6W0fxff6qFvAtDE+ncTABv/YlsTJkyYMPEd8FVSVCm1AoATgIQA7gJwBbABgAeAVABCANQyDOPh177sc6RoVQsJqFyQ7m4VfektlKgoMiL7+FxxPtoderCxI8iv9IpSLKxIt14kcy+68LwiXxivbRFP0KuYNZ2pn5E7ReoXJTolfuPiCuk1/zK9mw5PpIdF0ziUyh3Mx1/9lJG267EMfvTUDheSJdReJichU6yLrF02ojS9zbIB7DtSYowESs5vuPzZ+JtCNkUN4Ll0nC2eq+WTJegsFiFFbxynB7jkdh5tu52M1+tyD1m2rdww7q+mdbdPCgnpunsgf69zPJGijC3PeO6pH0nRzskAevz1B7FAYvdpkao9TkeiMo+bXD/XjZRoekwRQvrqNcq6smYkUXkum/SyWP6I0jrHm+LZ2frzsxm7SJQyq40TAKCGQQK0mbOQhpZLjOoyTZ2rbaOceP2KjxUCVh+jGz3BCUYNbdureD9ehNvO7SIJzWF3GR3s2DpVj9W0JyGWOZlEPS7WSzO2uSy5dmYQo6guxgIAQOGXIhOdlDo+ACDiJSEjax8m0ffAIY62bX/O6/f2EedJnI0SNaavzLnw1mGttvUeQO9+eM8F2pbOh4U8iZuw4CXhJjs9lj0N566RXnrbnG/IIrrM40WK5xrt43fJxihytY4f5HFc2CY+ZGmQ49uUVuShuS08pkGTSLBWSy6ed6qbJEq7N5LIOlkaazdEhCOkQ3nd6m8jyZgpicg+I0Shl1zztex3gRPv9/O1IqBI/46yTI/8fKazd5JOp6sGeQAAhpyV6H/jdV6H5M1Fcls4Gp/9t5tYELW+u9zbg7Z8R1zMLMeRoTALlupVmYRP8XdJ0a+mXAzDqPeFoVJfsJswYcKEiR8As1LUhAkTJn4T/PBeLp0VwzQF0ZbecGOIFPmB9N7ItIDpiYvJqJA0uktF58BO1D7Pm5dF287MZsgbfboQULVisuprYH+GXQ/zB+mxaYEMNUc3lPUK58cgidX+khCJF94wLMo4g/rlN2mL6bFzrbnK97l90qo01xQSj5s7yooOXUryOKIFM+wqM1JCsYXeXIcwhRLd62CD2wsd92cMDNdy82BuVmFmmyDXI2I6Xt/xjaSyNXMmnp93Buria4bIupNT25AEmptJwspd91jx2aCupIMqh7ESMvd1htcLust3Rh/KNFbEMkKcLd9Jzf3e29J6NOQlr9G259xXQJyUemzIYZKbV1wlZRB5PgnNY6OEdMvem/d55QqmOLYOkdawPt1YzehcSdZ7LGcw3Vb8kzbEAOBikEDeVFGIQXeDeal0kAVQUo/nHNiRnsR3w1EX9VjblCTHwnbLcY/qzf3FbyUpgJajeM2fp2UacNd2SWcdSMo73s9H2jf7NmAtxbtNUgH9Ig/PuXAOkvHFiwuxGvc0e8p0yyopq7ZODPPPe0vVa+KK1KtHiMDURZ7tkrIyinPOF8gvqY6jtrxH/SJbtA39P04DOr2Xe7x5G5+54tNlLU+jA1vj3uy2Rtsc/VlBWa4T5+nQ5lKP0WY7r1/Ta5Iq7TqOev+wx5KmcKvKzxSPxCTChKdyPQJDWaX7Nrekc7fe50IzU2NKGutgYs7xxnGZqhxaeYoeq5WA9QGv04hW307xXZVktrwPwipb15CNyxRQsUtSCzC/HP+e11yI9PTZmNbx/hevZdNDN2HChInfBD/cQ99fx8rI9RFCJYk3PZ1BU8RDagsSVBt3sCJsTySRLZYbyb4csV7E17ZR1yijy3ZIpFZdHCnGuV6Ep53EQyo6N/Wg5+/hK302Xr4lB+F5UirZorTnr65TcVaErVwu8rGSL0jSzdgqHsGEEfzVbxJTKIe9zemt+9dmhdqhDkLG3IrGv6tsl0UvLmSnHG3waeltE8Ei0QUAZLwh+7BNQdnaiyWykELkHbwOE8ZL7437fvScU81ml7mpK8QLKZyHhM6VQ9JRbnITHlOBvlJxWbYOPeE1yXktY3aRTnHJo5CAPdxQKgh/fwMcAAAgAElEQVQzbuN9SZBJiLtU9+hthiQlWZjuuVQwRk3E6rokueV6nKlD7zFJEvG4h2ehV/0kBufMzJZCkPtl5FxR5YV4LzpLyOxPMSKUctVt2+pomwvoMS4rXlTbIgziAhT7nEn0XzOkonNTb/ZXSdY4hrY1ys7eL08qCxn5rjrPz2YcF2o4HUOI7za3GM3YD5YFRRwW0HNsuUSigVRjuDZcxgn0mj0OSB+b+g1IVmeKJT18kuXi9auwUOb/qJaUJiaczIjhRiPpA3TA+nebvEI8zuhL6V6NcNW0zT6JdrIM7KL/npOV96daoIgDpigSvLENiUCmT+EcjJ6F+80+WCTDMzfynvoEVtA2t8r0cDMlE2957B4+r4dduBBG2cdL9VjyQF6bV6Eyx0qlZQ+otUHh+rv48LrNHMk5WTNXqB6zKcd7msZZoqnN970BAHeWStV3ESfO2VijGGk16STPaC2D75s1V4UQnjH3w/y04FtheugmTJgw8ZvAfKGbMGHCxG+CH55yqfyGZMyZPFJ1dXoD29z6JpNqyQozSJY492QawXurVJHG3kVNuO8SWdSwRklqptdUkRDWdQy12iu6MkSNU0c055blDM8u2klY1HID0yXrPYUYdD/CUG1qZ4bstjWkuVNsT+rLD1UVm7GQ1WdL8kv1nlNjhrwlhjB8r99M0hp3IzBlcdPpmbYpR+peW56R319pAUSUmiLVqa2dmZrpOHa3tpWLznRUyE4hpXruYErp5CtWubW3SDvhBedIopaxl7RXjmdMKWya2Fjbno7kogdRi1MrfTuhpLhSLWVomr6shPt16pAQnO9+VM4vBvXNXnF4vfP7iL78QEE2eDp7QJacL7iYevmTtSX0PtGDJNfaWiR4Z+WX+37yIM+z9UW5zisU51NJVyEoPyD7CwsAoPpNSSf0U6xm9B3kom3eu7goS+A7pjpW9JHUSLdMbP5UZoE0dnsYwQ4AUOmSLHqx/DZ11KMiMBXWuPA6PVbgGNN/lkCZC3mceR/7lpSFFK5UZnqsc3USn/X8nfTY4WwkZS8duqptAWc4F/dclbRN6tOc99Wms05gcgVpRzs0LlNnHVsL0dzSujBI9iNynbH1Y1K01T2pN2yVhVW1q5SkP+adYRvcVxWkhWx9kOgeEJE1FAN2yQIQgwewfXOXyJIui5iOuu/q0+Xa90zD9I7PMT57G22k5UjRmEx3xY0iunLf89SJv80gFdsJm/OYzu3k/Bg8R8QE048wheezRUpv2ndmJe7wTFKN/KodxQ7Bh0nYbnwjaazV1jbBvQ1ZNKT9SL6DXL+9UNT00E2YMGHid8EP99BXWHuFuD+QPgcX/SiZc0/4Tts25aHn9Sobf5Ej3ZMqtMp3SNCUXyakRpMaJMwOJJVeJClq0LuPFZFeiE0KkUo6vON2MYvIwguprlMS1Wa0kDv1q7HaMPpoejApj0rV35MsJDoi7BXv5mpP/tIXbiEkbtGElCb2CKQEbkIqOW7MJkE4zzPcAhDd6KVGmBNuu9hSgQgAyfOIt9AriL/6twqJF3e9J8+vS3vxSHPM5DV3duU1dfUX772tHaVhIRDyKOcoekjHFkgfjGMLWCEXsTMrOseGyHHv70gPad8z8XSb3+X1cg+T1c7bvqOMrsAVRlW+jcXLX3yUntTsSSO1bcQaEmvd42TStqO9SLyemsN7sNzeW4/dfsnv91PSvjRRD5E8fgrjeH8AQJWS4gnu7kSCq1EkWXE+iXWRiddR6CG/2FpLj6VaS2/Pv5t46DGs1ZoVLsnSb3FDOReab+a8ip9XjjHoNStPq9wQ2eeyEJ5n3gFC8Fa6SZneo9L8bOR+/fVYRBd60M1sxXd7f4sywfWtxENP/4Ty0DQtSban7SGEut0U9vrxGiiu47XkXJzCbqhF26rn+bhStHYh6R+zehqPe8HVM9qWdjdJ2dplimvbhGV8JvrFoIdu31qqbyclpHftt2qEtvmG0NOdNV22S+TK+RkpHSWHcY5Ji1p/q2TT97TMp3EOjF4uzheZ5VMvVqVmqcQ54J3PW49VrcooLc5TmR+b8vM7ioWrxL5Sg9ejXUlGRE0iy7KSSS5QWBBHyfzATut987HgW2F66CZMmDDxm8B8oZswYcLEb4K/tWLRf4XPNedyu2YBAPiNkrCroHsaAEBA8cXalqEg0zBvIjMNs6exhPZHY7K6s34qWeMvYiR2Quo6Q87vbF+Gav0CGGreiyxEUZJkJFafDSyhbXdaMERKFm5F8djx2Io1aWamRra9FQIvVzS2OY0YrtHYo7NcB3F/rXDrknowFVLmAtMxUc5a9Jj3eRKUQzJJXei8etTfHsgiIW+mt0LOAUA3N9G+P/ZnaNr+mjQma2ThuG1CWdnFtRu/t9gtNneKs0MaC6V8zcrTkMLB2harDc8ldH6gtiUL4nEeeU39bScla0be8aNu+fBwIUBvbmY4HKW0VOWl9WgKALgQlcRWxKtJ9djuiWywtN5H7sGYIKYuCr2Utqh+CZlSqh7Dm8d1XFafKeBIvX+cCUKczarHcPxCPNE5f0DWc6ygrFVF1q6MeJ/3tkELaW30ZgDPr2Jtkq6+xeX6HWvNOTnhumisj5fi/HyUWloBR57Je7QiOe/B9uSD9JhHdJK+xgQhzhLHZ8pqaB/Rc3fazNRgaFGS8bffpNFjBWdTYJA3RGj0pit4bL0HSX3FQk+mu3w8uQLXHw+kavhYGFN9HuHW230yPBgAEHxKGtEVzmRBeNzvIqmi00eYwgloIQubbTnDdNCuJtJEzmETddlbm7MxXr7CMoeztWQVcpYWktKcaEfhxKEtV7TN1p2pi6RHeYxpG4o2fE8pvlMcpguZe6M/G9HVfyypvu6bmY71LsvnZfKMJnrMPydb+2YuI2vaJj/NY4qxU1ZeuzaDIoNtKVhDs3qsVMQm7E7dfPMkktJUnZnmTGnV+IfH//eKRSZMmDBh4ifDD/fQX7qRGClWReQ+p/qzcmvXynba1tue0q07Bem9DdkinnTfXWxB+jKeeDedK/DX7v0BkQuevEi52OBhrFJ0CBBP0NaH1+FY6Vja5nKP3uHO5OEWUrhLb69ICI93fD1pA9tlCfdR2Ea8suGZKa8seFXO+7Y9KzNv3qAnb/dcerkkjMueEFUuSlvSdRHYQ+XeUvFuInyibXobWQiuTPMZvWw8IORRuv30YB3HeGhb0FR6TQmKUhpWaFUqPfZHO3rEfVpJlWfS3fQK7WzFM0npQM9/exp6tS6zhKydEYneZHN78eIcDV7z3vZC/jklIUFknGDUYXNVvObsmyiHXNxTJKZO5yjPe/dK7t+OePR+1QPev+6GrC/rvYAEeue0sv6r6k4v3zW29Ar5ALfDlL8au2WOYSirRtP0v6VNl8aSlK03ncfYoIG0DnYYQY9uvSHkr19h7vf1xITaNqIu5+79vKxEbLZGFiXpmoKe9PmdIt2zLUH5XIrC4i2fasZIIfUuOwBAnGdSsTq6B9sDL+8m0scN1+kxXugu5Kz7RYoIFmVgVHJsh3iJAYkoP1wYTkTQNRmJUs/94l272kllLQBYLOJJH0lCMnLZQom0UoUx2skTU4QLQQ8oD529i7baKSXqjtGOz1WqgrJgiltORoul3BZpW0MXygQ9K/C5iZ5SejHFykd5tBIFJnZF57kXCheslbcjWdmyFaWGyddJlXbh2oys3WKIHHL9Ul7T2/ekX0vwY/bR8fOfDwAo2lnkzBkSce7sHimR1vKBjCpbvrbgU5geugkTJkz8j+GrskWlVEoA7gCSAHgPYI5hGJOVUjYAVgGwAxAMoLZhGI++tJ8voaEvf1OeOMgiCOcH03vKlnWHtjm35q9stTLMpZfpKt6QrTv3ETe6dIi7uZteSvNM4q0siMzCEvuNlJlFvykem90NeoCe8SW3u/sUPaRqodLHYbUNf52veLFwJM978WTCztPjXpZfPOkiW+hVnLMReVK0JOzpkScbC3QO3JTOivXc2R+iRT3p8bC9L/dXqM8wbXOG9JEAALsg8a6jXWUOMW10yWdHX8NrWjlQlkSrdJJeU/5r9N4Su4gXt/QBZXc5ezTVtkdO9O57+UhhzNL6zEvXHUUv0qf6cz3W5iFz+aEPRCo58zDvVekC87XtSQpOm8IejBgsb0RSWNu6yMmrnOJl5bzNgqWZL0UmmHYLuy0+7cMCndk2kg+dvo33oG4lmTPlx1vvqQV/Qo70lMAdSi/T2fEE87cz90vhT+/pzN+6rmBnxZubeuqxQQfYATRCGbke0fpRulrMQaRtV8cyClxjVcBVqSVSzPVbWIRTaZHs43kPer2Gu3Q+xGrK7a7fZc79cEK579P70ZttPVqiwIz92Gcme0KZu5WOMK+fdx6lhilaSeFNLr/cAIDHkGjmwk16xkt8xOu8bIeP4L9Per/sKE/Owj6pSPcObuX1zXtAettszEw+wG0KI6zMtc7qsTqj2ckw3krZ75DdlB8+nyKyxWWRuI/jNZg7dxkv8/oPT57XsqXSVbWIL69zpggSfV3czAK/evtZVPi8gXQRLTtoEQAgp01TbSudigViDe18te1lAV7TOWWZc3cvL9zTjDDO3Q0vZIEV7zFWWXJXuVf/FH/HQ38LoIdhGJkBOADooJTKAqAvgD2GYdgD2GP9twkTJkyY+EH46gvdMIzbhmEEWP9+CuAcWHnuAuCDDGUxgKqf34MJEyZMmPge+EeVokopOwC5AfgBSGwYxm2AL32llO23HMBWB4beMR810rb1a0hGrVgvhNLkyQzLqnZie81i9+X3Y355EmyVNsjvU/sGTH8kKyDpjLYOJOmS+FKSFzGHhHPPlzEEy/le9hGxBkPSE7PstS3fHvISj3cEAwAibBZiKe0rpgLepRSi+cVLVt4V2eGtbV7N+XdIF6YTWqYSAvRwbrb3XHFeUhLoxDSCr420hHUO+5gUbbxQ2rr6F2ZKKVJ0qdRLGUbSpstx6Y/TeBxJm5mlud8kLX30WKMalNHdPygh4eH8vEduNSRVlaAnUzlXivEaFdgtayTGyMq00KhLkkYo3JiEdKntTtoW6w2nYeXC1gVFAqWt6+3GTE81eiry0LWJSS5GnCD9XdLPIFF7oC5TDFVqyT1Qa0hCW5ZLSuT5yg/k7cdEHgCcbMvU3Jz7Ii9sUIJpoKoZZb/DczJlsPQPHn/smxf0WLx9lDkmey/H/XQ4Uy3xfCRV5F+X88duERm5+BXkMYrszF4uqW2kBe+ypEz5xKku5PbJEwzlexdkOqFgqMgFN7Tkd/X0kV4/2TOTOK5UX0jzvD15jdyCWQm7aY/Mv1urSMTGeSUpiavZ+Ww+7iLpEtwXAQIA5C7upf92LMW5+PK63LMZjViZ7BRZFmfxCWTKYkUGplDu3BHScMVmptXShUmr3Iwx6FP23RNb2yI4kvHs787zvJBF7mPSSEyVVq9eRdtSlCKxvzeytMJ2TMC5sj815//NTnIcUZcxP/Z+TLC2jWzFttBOu6USNqgt0zReHfldq3ZL5fbN/Ly3tTfLXN9zh+nk8Te+vZnL3yZFlVKxAKwF0NUwjD++tn24z7VWSh1TSh0LCwv7+gdMmDBhwsQ34W/JFpVSkQFsAbDDMIwJVtsFAE5W7zwpAG/DMDL+1X4+J1u09LF6K6uEOHtq5Q6OlpBimehelP+p7CwGGr0/tx7Ldpa/wH8MkMUVYlck6WWTVDyNOIv4y/14Are7fk6Wi/pjNUm0OHeFAG1WnWTXGcfM2uabiz0eHqSlrCq7fbggZza9xKw3C2vTwG78lV7pIUuRHX9L7/dZGEmbo12ExKrShN5QaBNZcb7VRMoxb00V0nf2449/xd0sci7vrF6nbw3pnjjZl/uIPVV+VEtVZ+e+LEXpebWqId9Z/CCjmbTFhTwKeMrjzuMjUsYo1oUcIrZhxHSuoSwfd+A9u1seChGP/na+WQCAuRGkWCZ6Gy7Zd7kmFys4nEYKeur14nnmqSHR2k5besbPc8h0S3CfhSXDc1K+meyu9O8oupWR2KpDQkqN7ktv7EVUIdg+wGJhtJFosSzCsa4A58CKLOHut8EukrF2k6BMYJdODyWtS0+0jL9EOCsOMXoZM14kldPL00P3m0s5X6vocm133aM0cehw6Tuy+DS3O5FEJIGZuzJ68EzHa5rWSbr6PY7HiO/AQ/nOLA+DAQA5G0gRTOIOfIYeRmLBTYtuUvCycT/n3ftGsrBJfMWCnzKVhOzv7yrPDgDYJ9+s/57ahtGAzwtZfGNwECOKRkFSIObmSe83f2bOk2x95BgTpaP8dEwJkXE+zswoOnl28fyzTOHcrViX2+9ZKIVFAS953wObybNRyZN9iDIkFenvg2jc360VLGRMkFREB/WjkkTN1lyik1hv+Z7p6i6dSDv14TsqwyJ63sUriDyz+3QeU+bWwdqWszE7m1ZzFSnoB/xnskWllAIwH8C5Dy9zKzYB+FA+1QTAxk8/a8KECRMmvh/+Tg7dEUAjAKeVUh+0Y/0BjALgoZRqASAEQK0vfN6ECRMmTHwH/PBKUbczDCHHzWylbZnLMdxqO0nWv4z6lmRXxYkM/9q/W6bHfLNYteMvpcru5Gu2ho39TtqAZs1EUuLtIaYa9qcWshMGw9vSUaT68FIGVoG+d5XQPmA1Q+j060hivYekGOLa0ZYnTPrMXNzJXhDBr6QFb9UMTH8MGkYi80AFaQXcqCdDtnibDmnbFA8SYOPKSdj3rKAQjQBwd6tF/z1TMfA6Ho6czb+WRN/iKUKE+V3jAg0dt5JkCnolPUAeFqLe3iODkL7J31HTn7mxXOc4FUhaporKcP9mcLAem3uK/Xlsp0vwlj8Cyatt8YQEL7/X2u63Ds+vpKcsVOJyk9embkGprpywmXryZS1l1fpTA0j2Rt9MTXbsmtP12PjXTEflmi36+cehnFtNwy068AHJ3JgqavNYCN5CcZleqWxZpW1NczOkTlqVmv0QCLmd9wrD/hY9wtU6lOYiEhkuSdprshO1xxVeM02WvLuQl4tzczubcXIc1Vx4rUI6Nde2hLedAABb07Oi+cIzmX9bZnIerQ2TlMvBUazviFFHtOxP55CYzOZGzfvD3JJeaVmf86l2GW3C8e6cR0XqSzVtKV9JUQFA5HvSWjr6DKbRrrVOom31IrBS1Gm81Jvsdue9P76Lz+itxDIns99hrcjN4nu0LXFCpjf3rpK6jCwOrLg8c5ZzIWoB6aOU9wifZdvacn6xfPle8N0t7XO3FWZap01OppaOhspCL40XMx2zLJGQqGs68JloNVp6UoW5sOI97BwJ1pj1hVSuXoXbB82RNFWj+NT0u56R5/wDzEpREyZMmPgfww/30MtbPZ6CC2VJuYhN2Cely0vpnzAhNyVLI5dRFrQvTAiGBg1YGRbNKZq2bYvH7VKGIzPiBpBwehOV/SKSLxeZ3hkPet5J1goJczQlXZIcL8P1gnhi7ep2gOTlu6LS+6VRenreCZ6KFzfvGcml0zZCrO7KRo8gsIW1YnC5SCuHV6NHXKC3LFixbzTPK4GSStEnrvsQHoV2imQu4SF+9shA6blia0vPJ+oEkWoaWegd3E/A/b6MLt3gHt6k1Cq4oCx6kf8Mo5Gjd8XLCinOhR/ahLGibpWLRA5titFb2e4nHvqBlfRafCcK2TpwISVcya8z+noVpZse232DXku0wEraljsvPSSnUCG8i7SkN1ihKAnCrekD9Fi17vS8ym0SMq3NFEY9lmVClH6AxY1zrWWwm7YNfccosK+ndCHMHY8Zy9SnSOaeriTeeOn2nE9nSogXtzzWIgDA+Ividd67Qa/6RDXOq5s5xItLv55e4cttQngHWbUAqRKLbNEhKoUC50uwo2JosMgcsx/hMUa+ItfjdgNGSY+cRFgQ5TH7n+TuSKlhqUBxBpsnpSdvO0DI/ohdub82USV6WNBTvFgA8AoRmWjLKrxujaLE17ZBmfjZVMekMvLsbd7bsJjcv3FM5LhR6lKQkKimLHxzvCPneP+JMv/92vGZjBe3LAAg5LA8e1Fz8ll7/UzqIF+s5XhiFyFPN7XhfO6zmsSt4xARNUwPICH9dtNNbWsWldLpvv0ksg7rSEK/9TRWbl+sJj1o2m5mRiDfIVmSr0gBvovjuVrwKUwP3YQJEyb+x2C+0E2YMGHiN8EPX1O0syKR6NdvtLbt2Ui9s9ddCR29G1Ej3e8QybQk46S26fA+hpAzD0pl5J5p3N+aOtW0rewL6tDvvmBYeaWcVAleveUNAIjUTbSoed0ZClacIY2eNo9kiuHKaxK3kzPe0WOZHlvXR20t7WIvVWP7zwGBks5IdowNivpaubENsWU1+tTbWOE6oJU04C87huH7YJTVNgUnhEeJPaKn3tSbrYPb15cqtAyHmV7qESwpEaMSz6vxIxKPKYrJ6uu9ojEEz/xM0gMJwph6KlxB1n/tsJtpmLf7mF6JW1FIxuO9WCWbvJ2d7DcuSere0yT1k24Kw8+T5flZ+w2STihYiPrzF8VEd53iHNNinQoK2XVsFteijOnGkF75O+ixWztICDb1ktTF9fmscYgkWTqNMSe4/xvecv3iPWDzuFyOsjL81tU89+jjmJa6307SYO868dFK3kGavdVvzGu54KposW37MFXw5CFJvUqL5R5PzULSvn7WHtoWJ4xVvQdfy/xPPZha7Eot2NBtXF4h/GpfJYnq51Ba2140ZjWmShJR2wLqMSUyJpBa+gyGVFceLEet+dwOojn32s3nyzWHpNNSQ+oHAKDddGmy1uktaz8WjhXCdocnUzJRN0j64+A6HseGzUx/ZLUVMUH2ejyXHeWkWZnLGaaqrjvKc/7GhSnE0wP5gCVPIQ3BXhSk7VVzqQ9425vndfSB6L83uDKF0tmf6ZvmFlnP1+EgidJcKSSF0qMF3wflL0pK7tRApoYeJOR5FrAICd29Ouek53MRfvi2ZXWxJH3/OUwP3YQJEyZ+E/xwD32jQdJjpBKv7FQvStRKKSEch9Uh0ec5g7+EDovq6rHstUl2dhkulWnOfSltezNB2lm6v6F3cLYiico7b8WbzBKD3pgxSioMs4XS4wp2k9W9L/lxHzb9KTvqvkNIr/xV6SElyyBk06uiJErXdBDZ2NVh9CDm9aNEctFi4TrKe9AjTZBGlm3DDVbSZfWUNqBnfS0IjyjRpf2wy0BGIBcmCWmYYRslhznGSa+JzQVIKDXNTS+xVBTxSF/04/m9qyVSv4Bh9LhqVRTya20Detybu1KeOddDPJSqOen+Tg/y1javxiSpXwRLVWW1UrwfNw16uqGzRQqXug7JKCPdQW17nIxS13I9b8uxuVHStmIwJW3TtolX2+IJ58r6ZELquY+it3TH8udeLq4hjLAS7xPvsGcaXiPHodILZ1ebuQAA/6LcLuJGIc/PW/vMBHSRdsVJN/IcqnaW7zrSmed6vAIrF1vECdZjT2IzEkrySmSi0Z4ysshQUSK4aPO5lNzRR6wofTlRiMpxDrxGtadG1bbMCynd6z5Urke8voxG54DE3ORhspxeqWbsl1Kro0TAK9ZZ29U+l2UiXcd9XIpSYYJ473WdGNlOfC5RYLG1/K6rTYQUTd2envacbJSArj8lWskBaxh5FKwgC86cbUMCe5KnCADGL+Zn7g2n7DjyIlkSMvpsjkVOJ1LkBy94TxPGF2J13WsSsIntSHymvSM9eeYsZj1lw9oijd2sKBRw6CxLRz64zSjj9hsu1PNykZznzGMUfiQtIdu/Hsx3xbd3cjE9dBMmTJj4bWC+0E2YMGHiN8EPT7nsXscUQMxxshblslzUoN6aJBWUk/KQ0GwfhamU1CdEH23pw7CrXxOpuupZkprjlamDtW1ALq7oMm0rQ9j1g6RBT8HFDNWfP5OKRPekJEleD5Zq0yguXDMwirV9roO/7OPpARKD16dc1Ta1i6Hd1ALStGfMHZJAEZez2s45QMJbm9wMgx9OlVDspTW6jmbE0zZXXwl/AWDVKFlhJrg0w9ZRraTGIE1Mpj+ONi2pbdEWkJw7n54tRZNWFzJt1niGvFcmSNrhzSSmAE42E8IsYySmtDo+YNVf1yDRO8/OwhRYWHTRI7/eyxRAsVNyv/dUp+78rTerH08Nl3A43SSGpnejCzkbeTibmY1wlCrWToOZnmvlRGLuyRIJn+0cWYUc82UBbUs/l/dKaDvBVlumEUqWF9KrXS2mlEIC7mpbw+5cn3JEQd6X0pmlvXGrGdSkZ3o+U9u2JCQB9vimaN/zJyK5WeAt6wSWJJMWqxHK8rv8A+VaRXVgmudoPCFbH/szpK94jIR+vQqi3X5akXMh7hGpHZjejWm9Q0/kOufJzHvlkIvHcW6g1Is4X+PfbesKIdxuDVNVQ1u6aNvbSBaER1zI9budkPNk0UFJU0z0YrppaH3ReDtfZj3KlYOczwk2StrGvgjTaBfHeMv59d4NAGhiyFqedQ4zXblrFAUJa+NKuulwLj4H5a7O0rb4ttbmfjGlIVmizpxP8csy5fJ8hyw4Os2b75HKm6ROIVN57s99g4gqStpTdJAmgMdRZpjMVxtrZvd1tinaZrhxflosFfCtMD10EyZMmPhN8MM99IbF2QpVWYQYLNuDnnnQDiF+ultXWL+2iySTMVPkgo1y0gNc3kKqR7MfZXXWsVn9tW1uL37mRD+SjPkmP9Zj10pTOui4QdYgPbScpGmjSbJd0jf0kKpu4naruogMK30SEkWe68do27RDJE19Qndp25ta9GKLTmZEMaO5VJcZfbmAwZlk8ssd7QU9qvz55XpAus8CAOrkkv4nbZ1Z+WZskzUg679lhDNwqqwWH1qRnt0O65qYKdJK1drrCPTogseKtK1/VNI180rN1bbnZ+mJPupJ3+DpRJFhrXTnOaRoJ0Tinbz0QKM2WKRtgZNJ3j5qRm8sd3FZwACxeaIlWwhRumQzPdliHz8AABXeSURBVLra/UXWmjmIUs165UlmrXoXWY8ti0lSdFxPOY5CWTlXdrr+mYIaY0MiuGRB+U4nD8ombZ9L756DFUjOdjlIDyxTftl/mRmMvnL5C9lv84Qk3euFibXt1hXejyMNuF1sxNRjo7LymbhSVNbttLlmlWW2aqdt6//gdcs7ht5kgo0iadz9jvfd4Zh4sFtqMeLMWEOI5mft6bnGjMfIN01JqUoeu449Wt5DZIu973PODpRACJasH19Lo3JO/XfQaVaxrskq+2jTk+R97bWy2ES1boyeay3leW6/JhGAGsj5dOn4ZW3Lv4bb37CX9VEbTmE16ObHXJQiU0I5yHTefCZ2XhYxQ+LB9OjjzhM55OkIjG7z3qEXvmOrRBHle/NcKgSJiGB7MUYjG92EPG3QjAIASypKJbfNFy9fdWTls/cOib6OKGsfH9f1+FaYHroJEyZM/Cb44b1cLMEWAMBRZ8mVDa1H+c59SLHMnCMU9h/cTYliu3rJ9dglC3OBbXO30LZ4MenNFkgvtqeDmTOu7cIim3U+ks/zLRHM7eeJt5CnFvPYQeEkgnHv8xd+T2z+IscKkb4S76yFA6eXyvbtQA8pZRGRko3NQPlSnI70ht7NkZxZ4GlGCidDJEce3Z1esvFSPAKLq6w+DwA2x1frv0tX4ir3lx6F61pYgnnVWZfkcw99WWxS0IbnsCaeyC2LHuH1C+0sHfMyVbF6b8nknOc5c7zeK96X3Wva6LFiabnoRUBR6ZER7Q6900Rhk7Rt125+V5lj5BZWlpC+PusK8BhDTkgR0eIplMBtPScSuMWP6JV6WPuw9DREklroDXmO1rWlG+J1ZyYx390RL/IDOkSkpHLmIOkIeeQx87eruotn97aXta/PWEZrAZGlD4vPHhYFHXCQzort09J/SuQrnQGHDec+ssajF56nZBM9dq0NpXg3jHraFuspIw/7IJkzLwvwMxMK0stedkOko/Um8/luH0Hy33fTcB/v7ouXn2gWI4USnXmvbLpKFLFoG89h1CQ7Ob/e5GIGjB2vbcU/iXYqHbLov0+4sPjPI4ZEBU2LMhp5P0w6R96dxA6accsyOrk0X/Lwj25Zi8wySZ76wCNe0/n3pagrVhnOo5zdGGmtPC2Ffpdv0pMP7PxI2ypOYmT9bKDwEsUWMv+99wr3f7O0dJLMnYrzY18diWgvennzeBb207aWfiwSizuAz0TAFllA5s1EXrcYJWSZPo/xLMSb9FQKzz7A7OViwoQJE/9jMF/oJkyYMPGb4KukqFIqGoD9AKJat19jGIarUioNgJUAbAAEAGhkGMbrL+/p8/BPTgLvRX1ZM3JTPaZaNmWTSrYId0kqJjy2CABg2EklZd1VrMqLMlZIrGb9merwLikhstNsEjmDbzLl0SqWkG+2GxkuqpEi5bq2nGRK1NxCUkz0ZQjoXJJE7LNDEpa38KSsb3JhkU8uLsjqQe+yUvFWGta0zkgSvUOKynGcLkIi5eoCIS/tclBiZwjHCWzGR6hYWiovM81nimF3NyFonKyRcY0h0k+icgOmlKIbTXlOA4TQXO1IGdbVQiJbnNCRvVD2lhcSd9oTyjxb9eV3RbGX+zK4NPt39I0qJGr555STLn0vvVkqL2V4u2YJt0sSScip3i+58MLFaNJ7o4czQ27nwkIgznvBlNKtUrxWlhIiwXQNoswxeWNJI2ScxirdYsJJaTwPZqpj8HLZf736lJ3GWiD7rbyOctPrzXgzjk+XuXarGudYxsgiaz2alZLNOGkkzG5Yi1F01Wsk12M0uaHHJo1gmieo2GFt82zA69A2qyykMKMpiWaHolyEo1dFSZMN79sSAGA3Vda/3JyKctxWrUUWHM2e96XQSVYBe0wVWfCLudY1TW9JzxXHiUyn7fAQueyhTzoR55ksz837ojy/kfMl5RcnEuWqlWNJemyLD1NJhSMxrXe5tCxmcS4B0yrey4UUHZKbKcqAJiILvlmO5OP4FDy2nhFl/u14wDl8MEjkp5bznMNhSvocbSlIwnvkRKavvMoKyX4/Jq/3wMXS5rlSLJLI4x5JyqxJCb4PVr3jw+fVR1oHqxt8mO92ECljzs58bzRb8XEb4n+Cv+OhvwJQ0jCMnAByASinlHIAMBrARMMw7AE8AtDiL/ZhwoQJEyb+n/FVD90ga/qhiUhk638GgJIAPjQQWQzAAmDmp5//GvIepHc62RAyckYqEhd2IdK9rcobkiWx5/CXfg7k1zHeZf4Cvo0qBM2lDVw8osD2Xtp2zY9EzsE0JEUHDpbue06R6BmPCxU3OGkhFhqctBdPwz4rPcGHISxJ6dhOFqc4XJ/kVb+M4i1kJhcED0jHt2JD6SXvX83tsvUVj/fyAfbqKFpf+rY8CPUGAOSdK7LFKkk+JqDidxKC99ITRjZedcQTvBGZvWSubpfowfMUieU6Vehtld4s0raE3SklXDhSPO6nL1lA435NelIsPU2v3mM8veZM0eQ4down4djrphT01C1IqWEyG+lsV8eP5FX7/PQiX++S6zdwI69DxkvSc2VQenptto1FBub2ip5t0dX0pOYa0mPkHugFx0wnEsxbU62k+j4hKD+gXEcSptvcxJtM40vibl8l6Qqati2Jz/2XSWAnXumkxwJqkoi7mlG85aF1bAEAkyvJuRRryqihvYWktk0BWSQjdhtGqhmLCEm88W02AMCcEYW0zccgCX5+Mcm9vbGFVNs3k+e+97UULI0cy0gie5D0+mkBkqY1c3CeFEllo8caKUZTAZDr/c6G+0tYa4O2HQq0IDwmZpEOiBW7s+hqhov0EhrWg+d1tqac8wgnRjtZPBn9RKwrBXZBNoyqbtSVtepP+fAapVgnfXTaXuFcmPiIz+qg/bK0XJN11qK0tkKyz6pB0vdxUyH7bwxhceCz6SRDXR4J2R8LXDJvZZCIDrr0YaQwfpR48j3TNAUAhFkVy8MjiRQ5Xz029JmTV6LR7GEfIsL/Xw8dSqmI1gWi7wHYBSAIwGPDMN5aN7kBIPkXPttaKXVMKXUsLCzsc5uYMGHChIn/AH/rhW4YxjvDMHIBSAGgAIDMn9vsC5+dYxhGPsMw8sWIEeNzm5gwYcKEif8A/6hS1DCMx0opbwAOAOIppSJZvfQUAG795Ye/AFWHIcreWqLpbLUxKwBg55xwnTbaMFXQ9T7bpK4NknaZ7x1JNpx6Lb0YcrlZe3/ck9DRKQ+JpNAl3P/SZqI7ze7CtvIuzqLn7riWOt0GSSVcdV1OgiVRZ1ZXNvaRHiPuGVZYj3W3ti1OTx11UBZZ4KJOO56L8YLVgeOmiD4VXfh3n0PSa6KXvTVUa5la2ywWfIQEkApGz1VM6XTwkYrVJaGsjEw9Utq55hvKY6+/idd0zSpJNzUawrBv5NlF2nayEAnj54ckVH+fm/sdUIfnVGSm6HCnObHC9rR1hXgAuD/bmg5YJT1f3MeRlOrdiH1bjj4QItG2KbN6k1Yn0Da/zUzzRDksLYnLu7DKbodBsiuSMVKPFQIJq2UDhPAu8CFTtu/PlaLnKzJzuGCkpKxeNGEV5hsneWRSDGC6sH593qvDxWV7v/okhIv2kfNM60Ttc4nqQghPHcvUzD23LDzuLBLab8lMYn/U6djatiApifpXqaRmoIg9Ccw+CVhVO8NPwvjkA5lKTFlItN4NI1Oz394i6ZJ2JSwAgP2K6c7og6WiM7/B+3daySPe5IGVpI4lhK2rtOwBALyrLinCTBPo750tuU3bEgzg8WZuLqmtWL34DPX3Yaoqf1HR27sdDwYAPAznOtZ05LN2zpC6huuK69XOLmGd/0OklfLUsiQ+2w6UlK192sHcPq5Ug3qV4P0+XYzpy6yj5XkcWoVzTM3207Za55k2TVRdFquxlqCgXFGm/14YMtf+sF6aV55S4brVqkn4qtj8L/BVD10plUgpFc/6d3QApQGcA+AF4INGoAmAjZ/fgwkTJkyY+B74Ox56UgCLlVIRwR8AD8MwtiilzgJYqZQaBuA4gPl/tZMv4gi9zpKrpRrOaED51aF64mnYe/MXMJJB4nNfG1l4wWk2SbSzE6XyrZiFCsqDA6QScOcsevJLntIzjj9C5FJLL5P8C2wv5GVtF3oyBx6JS7DxPXswbN9Mr9JmpqxGvzU/Pek5hYK1racT/99kuzT/bxFK+VreEfS8/CEk4H6D1Yzb7hzRtt53WIVpVA7nyUvPfgDA9Gty/Zzd6eWETRcZ4jDrGh01fOQzZ62LUXTMQzlYjwBpEFOrjrWqMrt4cfUOsarxvkWikmwZSAjWrE2vc3V6Oz12LJBT4qqryAVX3aDnX2eEdLWcP41kZZ3uVs+1q1SANqxBIrtOR1lCLdptLi9XvaeQ2r7WQsFCBj3XkxNF7hZpK+dRVjk05HS2Xi/hFjXS3+JiKhe3Dta2CP7uAIApthZtK9WJ0cmZ2fSqky0QQiyBLc/JpoZ4mBuq8e+0FnnsAs6R1Eszkx59pQdSGTn5IUm3Snmkr8qpa7ynfvkKapv7MAoFyjYnkefqKuTehvo8l7xTRaKYshfJ5y7VpcvnxA08h27WSs2snaSie79PUwCAMkTz4FeVPVQKdL2ibRYrWfgBfcLF7GOn0Pdbc1vu+/KJvQEAVcZLdPnqDiPvScUoB85sE07q95pzwHeHREJezrymByaIB53PcP9wQACAy+EqfjuVo+ffaaSttpVsxohwXm853vdpGWl65udSievWygM3YRMrOu/kFmL1QSp66/3WiSjgYgfSigGHKWU8Fa4SO3HzYADASgfx8lHFumjPv1jh4u+oXE4ByP0Z+xUwn27ChAkTJn4CmJWiJkyYMPGb4Ie3z0VOpgdGd74vtmVMuSzZ+VCbbp8n8bOqOJmGKZWF6JiYjoTZiDfSoGe/i7Uh1BiL7DeutQpvHrWl6qlUbxofuLyPEkcMy6c0FrLQ4zK1xx/Wv7SXNSdwbjh1te2miV7Xy/rRue6y6IBLaepp/ReTGel8XELVpIqVY5sg5MruDdzOe0sf+bK8oucFgNZVRXsc7RrJ4eIPhcwtB17njQMk9K5q7Qp8/gRTOcUCwylPD5JAdLwhoen9JSR2y0qXYkS9TZL4vqIGek9q0R4vHMb0wJ7yU7Xt+nbeA9VR1oQ17K2EXTfqcJN0k5apd55xHyqWEM1DweuwbGpLbVsbxw4AkLwEGyKdTCt6cc/DTGskrZhN28Z0Yn3Coc90KnVQ3K4ZJM2zODqvZZcCQpylj8B7VfcsKxcbLRPicWs+7riIIfc2TxGmBw7lE3J7JEjUj27PFOKqUe56rIl1nd0uZ2Tt1I05eY1aFZRjq3iPZPKVUkyDZDgmBGE9N1Jb1w2pl5imSHImSrVK226EMF00sDrTgC0iyHy9VGQRzwXSlrfgZaZaYuyXNF2va6IZB4Aul4L135M9WetQvJy0GEZnas6j+YRbmKYqydgs8ZkGqd/KWY8Zi63p0Fnihzrd5XkVqSCpH2XVkLd6ZQcAqHFESPbSc9ng7lrDcE37qjP1muaRNMSrvIdzZvOJpgAAfztpq51xqgUAkHiIVFGPduB18Hwl93ZjWaq6N1iXhH2/VBIdNbpQiBBhr2jwvUbx3Me+kOvxT2F66CZMmDDxm+CHt881YcKECRN/DbN9rgkTJkz8j8F8oZswYcLEbwLzhW7ChAkTvwnMF7oJEyZM/Cb4rqSoUioUwHMA97+27U+OhPi1z+FXP37g1z+HX/34gV//HH6l409tGMafFxv9BN/1hQ4ASqljf4et/Znxq5/Dr378wK9/Dr/68QO//jn86sf/OZgpFxMmTJj4TWC+0E2YMGHiN8GPeKHP+QHf+V/jVz+HX/34gV//HH714wd+/XP41Y//T/juOXQTJkyYMPH/AzPlYsKECRO/Cb7rC10pVU4pdUEpdVkp1fd7fve3QCmVUinlpZQ6p5QKVEp1sdptlFK7lFKXrP+P/7V9/UhYF/k+rpTaYv13GqWUn/X4VymlonxtHz8SSql4Sqk1Sqnz1ntR6Be8B92sc+iMUmqFUiraz3wflFILlFL3lFJnwtk+e80VMcX6XJ9SSuX5cUcu+MI5jLXOo1NKqfUfVmOzjvWznsMFpZTz5/f6c+O7vdCtKx5NB1AeQBYA9ZRSWb7X938j3gLoYRhGZnAd1Q7WY+4LYI9hGPYA9lj//TOjC7hs4AeMBjDRevyPALT47Kd+HkwG4GkYRiYAOcFz+WXugVIqOYDOAPIZhpENQEQAdfFz34dFAMp9YvvSNS8PwN76X2sAM/FzYBH+fA67AGQzDCMHgIsA+gGA9bmuCyCr9TMzrO+sXwrf00MvAOCyYRhXDMN4DWAlAJevfOaHwjCM24ZhBFj/fgq+SJKDx73YutliAFV/zBF+HUqpFAAqAphn/bcCUBLAh1Wrf/bjjwOgGKyd6g3DeG0YxmP8QvfAikgAoiulIgGIAeA2fuL7YBjGfgAPPzF/6Zq7AHA3CF9wAfmk3+dIv4zPnYNhGDutC9sDgC+4wD3Ac1hpGMYrwzCuAriMX3BFtu/5Qk8O4Hq4f9+w2n4JKKXswKX4/AAkNgzjNsCXPgDbL3/yh2MSgN4A3lv/nQDA43CT+me/D2kBhAJYaE0bzVNKxcQvdA8Mw7gJYByAEPBF/gSAP36t+wB8+Zr/qs92cwDW1XZ/2XP4CN/zha4+Y/slJDZKqVgA1gLoahjGH1/b/meBUqoSgHuGYfiHN39m05/5PkQCkAfATMMwcoOtI37a9MrnYM01uwBIAyAZgJhgmuJT/Mz34a/wq80pKKUGgCnVZR9Mn9nspz6Hz+F7vtBvAEgZ7t8pANz6wrY/DZRSkcGX+TLDMNZZzXc/hJTW/9/7Ucf3FTgCqKKUCgZTXCVBjz2eNfQHfv77cAPADcMwPiyPvgZ8wf8q9wAASgO4ahhGqGEYbwCsA1AYv9Z9AL58zX+pZ1sp1QRAJQANDNFt/1Ln8CV8zxf6UQD2VmY/CkhAbPqO3/+PYc03zwdwzjCMCeGGNgFoYv27CYCN3/vY/g4Mw+hnGEYKwzDswOu91zCMBgC8ANS0bvbTHj8AGIZxB8B1pVRGq6kUgLP4Re6BFSEAHJRSMaxz6sM5/DL3wYovXfNNABpb1S4OAJ58SM38bFBKlQPQB0AVwzDCwg1tAlBXKRVVKZUGJHiP/Ihj/FcwDOO7/QegAsgsBwEY8D2/+xuPtwgYdp0C8H/t2yEOwkAQRuHnquEInIAjYOEaHAPFIUjwCAQGgeUGCBIQEHoTDGKWBEMgmC2T9yWbVlTMdrt/mkl7LGNC9KH3wK0c+7Vr/WIuI2BXzgfEw9oCG6CpXd+H2ofAoazDFuj92xoAc+ACnIEV0HR5HYA10e+/E2+v03f3nGhXLMq+PhFf83R1Di3RK3/u5+XL9bMyhyswrl3/L8M/RSUpCf8UlaQkDHRJSsJAl6QkDHRJSsJAl6QkDHRJSsJAl6QkDHRJSuIBUtYgBBewZpAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from layerized_net import *\n",
    "from torch import nn\n",
    "from torch.nn import functional as F\n",
    "import numpy as np\n",
    "\n",
    "na = None\n",
    "\n",
    "class AMLayer(Layer):\n",
    "    def compute_stats(self, X):\n",
    "        pass\n",
    "   \n",
    "class Relu():\n",
    "    def forward(self, x):\n",
    "        self.a = super().forward(x)\n",
    "        return F.relu(self.a)\n",
    "    \n",
    "class Linear(AMLayer, nn.Linear):\n",
    "    \"\"\"\n",
    "    For linear layers without Relu activation, usually the last\n",
    "    \"\"\"\n",
    "    def forward(self, x):\n",
    "        D = np.prod(x.shape[1:])\n",
    "        x = x.view(-1, D)\n",
    "        return super().forward(x)\n",
    "  \n",
    "class LinearRelu(Relu, Linear):\n",
    "    pass\n",
    "\n",
    "class ConvRelu(AMLayer, Relu, nn.Conv2d):\n",
    "    pass\n",
    "\n",
    "class MaxPool(AMLayer, Relu, nn.MaxPool2d):\n",
    "    pass\n",
    "\n",
    "\n",
    "class AMNet(LayerizedNet):\n",
    "    def forward_until(self, layer, x, a=False):\n",
    "        # forward until specified layer\n",
    "        for layer in self.layers[:layer+1]:\n",
    "            x = layer.forward(x)\n",
    "        return layer.a if a else x\n",
    "    \n",
    "    def zeros_like(self, layer, samples):\n",
    "        return self.forward_until(2, samples[0][na,...])*0\n",
    "            \n",
    "    def activation_maximization(self, layer, pattern, sample, steps=2000, lr=0.1):\n",
    "        # generate some initial noise\n",
    "        am = torch.from_numpy(np.random.normal(0*sample)).float()\n",
    "        am.requires_grad_(True)\n",
    "                \n",
    "        # define the loss function and the optimizer\n",
    "        criterion = torch.nn.MSELoss()\n",
    "        optimizer = optim.SGD([am], lr=lr, momentum=0.5)\n",
    "\n",
    "        def step():\n",
    "            outputs = self.forward_until(layer, am, a=True)\n",
    "            optimizer.zero_grad()\n",
    "            loss = criterion(outputs, pattern.detach())\n",
    "            loss.backward()\n",
    "            optimizer.step()\n",
    "            return loss\n",
    "        \n",
    "        train_loss = []\n",
    "        for i in range(steps):\n",
    "            train_loss += [float(step())]\n",
    "            if i % 1000 == 0:\n",
    "                print(i, train_loss[-1])\n",
    "            if i > 100 and np.mean(train_loss[-20:-10]) <= np.mean(train_loss[-10:]):\n",
    "                break\n",
    "\n",
    "        return am\n",
    "    \n",
    "    \n",
    "amnet = AMNet.from_model(\n",
    "    model=net,\n",
    "    layer_names=['conv1', 'pool', 'conv2', 'pool', 'fc1', 'fc2', 'fc3'],\n",
    "    layer_base_type=Relu, last_layer_type=Linear)\n",
    "\n",
    "pattern = amnet.zeros_like(2, images)\n",
    "c = 0\n",
    "pattern[0,c, 5, 5] = 1\n",
    "am = amnet.activation_maximization(layer=2, pattern=pattern, sample=images*0, steps=300000)\n",
    "imshow(am.detach())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor(0.0802, grad_fn=<SelectBackward>)"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "amnet.forward_until(2, am)[0,c, 5, 5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introucing Typicality\n",
    "This process has no notion of typicality. Since we have not learned the pdf of $X$, this is hard - but maybe we can introduce a regularization that finds input, that also activates the hidden activations in a typical manner. We estimate the typicality of an activation by a Gaussian distribution with learned mean and variance.\n",
    "\n",
    "Therefore, we need to collect these static for each neuron."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  },
  "latex_envs": {
   "LaTeX_envs_menu_present": true,
   "autoclose": false,
   "autocomplete": true,
   "bibliofile": "biblio.bib",
   "cite_by": "apalike",
   "current_citInitial": 1,
   "eqLabelWithNumbers": true,
   "eqNumInitial": 1,
   "hotkeys": {
    "equation": "Ctrl-E",
    "itemize": "Ctrl-I"
   },
   "labels_anchors": false,
   "latex_user_defs": false,
   "report_style_numbering": false,
   "user_envs_cfg": false
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": true,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}