FS3 - Information Gain For Classification.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "742d1d84",
   "metadata": {},
   "source": [
    "# Information Gain\n",
    "Information gain is a concept used in decision tree algorithms for classification tasks. It measures the amount of information provided by a feature in reducing the uncertainty about the class labels of the samples.\n",
    "\n",
    "In classification problems, we aim to assign class labels to instances based on their feature values. Decision trees are a popular method for performing this task, where each internal node represents a feature, each branch represents a possible value of that feature, and each leaf node represents a class label.\n",
    "\n",
    "To construct a decision tree, we need to determine the order in which features are evaluated. Information gain helps us in this process by quantifying the usefulness of each feature for classification. It measures the reduction in entropy or impurity that a feature provides when it is used to split the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "7a82a708",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "8e0d764a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Wine</th>\n",
       "      <th>Alcohol</th>\n",
       "      <th>Malic.acid</th>\n",
       "      <th>Ash</th>\n",
       "      <th>Acl</th>\n",
       "      <th>Mg</th>\n",
       "      <th>Phenols</th>\n",
       "      <th>Flavanoids</th>\n",
       "      <th>Nonflavanoid.phenols</th>\n",
       "      <th>Proanth</th>\n",
       "      <th>Color.int</th>\n",
       "      <th>Hue</th>\n",
       "      <th>OD</th>\n",
       "      <th>Proline</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>14.23</td>\n",
       "      <td>1.71</td>\n",
       "      <td>2.43</td>\n",
       "      <td>15.6</td>\n",
       "      <td>127</td>\n",
       "      <td>2.80</td>\n",
       "      <td>3.06</td>\n",
       "      <td>0.28</td>\n",
       "      <td>2.29</td>\n",
       "      <td>5.64</td>\n",
       "      <td>1.04</td>\n",
       "      <td>3.92</td>\n",
       "      <td>1065</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>13.20</td>\n",
       "      <td>1.78</td>\n",
       "      <td>2.14</td>\n",
       "      <td>11.2</td>\n",
       "      <td>100</td>\n",
       "      <td>2.65</td>\n",
       "      <td>2.76</td>\n",
       "      <td>0.26</td>\n",
       "      <td>1.28</td>\n",
       "      <td>4.38</td>\n",
       "      <td>1.05</td>\n",
       "      <td>3.40</td>\n",
       "      <td>1050</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>13.16</td>\n",
       "      <td>2.36</td>\n",
       "      <td>2.67</td>\n",
       "      <td>18.6</td>\n",
       "      <td>101</td>\n",
       "      <td>2.80</td>\n",
       "      <td>3.24</td>\n",
       "      <td>0.30</td>\n",
       "      <td>2.81</td>\n",
       "      <td>5.68</td>\n",
       "      <td>1.03</td>\n",
       "      <td>3.17</td>\n",
       "      <td>1185</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>14.37</td>\n",
       "      <td>1.95</td>\n",
       "      <td>2.50</td>\n",
       "      <td>16.8</td>\n",
       "      <td>113</td>\n",
       "      <td>3.85</td>\n",
       "      <td>3.49</td>\n",
       "      <td>0.24</td>\n",
       "      <td>2.18</td>\n",
       "      <td>7.80</td>\n",
       "      <td>0.86</td>\n",
       "      <td>3.45</td>\n",
       "      <td>1480</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>13.24</td>\n",
       "      <td>2.59</td>\n",
       "      <td>2.87</td>\n",
       "      <td>21.0</td>\n",
       "      <td>118</td>\n",
       "      <td>2.80</td>\n",
       "      <td>2.69</td>\n",
       "      <td>0.39</td>\n",
       "      <td>1.82</td>\n",
       "      <td>4.32</td>\n",
       "      <td>1.04</td>\n",
       "      <td>2.93</td>\n",
       "      <td>735</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Wine  Alcohol  Malic.acid   Ash   Acl   Mg  Phenols  Flavanoids  \\\n",
       "0     1    14.23        1.71  2.43  15.6  127     2.80        3.06   \n",
       "1     1    13.20        1.78  2.14  11.2  100     2.65        2.76   \n",
       "2     1    13.16        2.36  2.67  18.6  101     2.80        3.24   \n",
       "3     1    14.37        1.95  2.50  16.8  113     3.85        3.49   \n",
       "4     1    13.24        2.59  2.87  21.0  118     2.80        2.69   \n",
       "\n",
       "   Nonflavanoid.phenols  Proanth  Color.int   Hue    OD  Proline  \n",
       "0                  0.28     2.29       5.64  1.04  3.92     1065  \n",
       "1                  0.26     1.28       4.38  1.05  3.40     1050  \n",
       "2                  0.30     2.81       5.68  1.03  3.17     1185  \n",
       "3                  0.24     2.18       7.80  0.86  3.45     1480  \n",
       "4                  0.39     1.82       4.32  1.04  2.93      735  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df=pd.read_csv('https://gist.githubusercontent.com/tijptjik/9408623/raw/b237fa5848349a14a14e5d4107dc7897c21951f5/wine.csv')\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "2cb7df1e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1, 2, 3], dtype=int64)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['Wine'].unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "cdfa9534",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 178 entries, 0 to 177\n",
      "Data columns (total 14 columns):\n",
      " #   Column                Non-Null Count  Dtype  \n",
      "---  ------                --------------  -----  \n",
      " 0   Wine                  178 non-null    int64  \n",
      " 1   Alcohol               178 non-null    float64\n",
      " 2   Malic.acid            178 non-null    float64\n",
      " 3   Ash                   178 non-null    float64\n",
      " 4   Acl                   178 non-null    float64\n",
      " 5   Mg                    178 non-null    int64  \n",
      " 6   Phenols               178 non-null    float64\n",
      " 7   Flavanoids            178 non-null    float64\n",
      " 8   Nonflavanoid.phenols  178 non-null    float64\n",
      " 9   Proanth               178 non-null    float64\n",
      " 10  Color.int             178 non-null    float64\n",
      " 11  Hue                   178 non-null    float64\n",
      " 12  OD                    178 non-null    float64\n",
      " 13  Proline               178 non-null    int64  \n",
      "dtypes: float64(11), int64(3)\n",
      "memory usage: 19.6 KB\n"
     ]
    }
   ],
   "source": [
    "df.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "be7871fc",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Wine</th>\n",
       "      <th>Alcohol</th>\n",
       "      <th>Malic.acid</th>\n",
       "      <th>Ash</th>\n",
       "      <th>Acl</th>\n",
       "      <th>Mg</th>\n",
       "      <th>Phenols</th>\n",
       "      <th>Flavanoids</th>\n",
       "      <th>Nonflavanoid.phenols</th>\n",
       "      <th>Proanth</th>\n",
       "      <th>Color.int</th>\n",
       "      <th>Hue</th>\n",
       "      <th>OD</th>\n",
       "      <th>Proline</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>178.000000</td>\n",
       "      <td>178.000000</td>\n",
       "      <td>178.000000</td>\n",
       "      <td>178.000000</td>\n",
       "      <td>178.000000</td>\n",
       "      <td>178.000000</td>\n",
       "      <td>178.000000</td>\n",
       "      <td>178.000000</td>\n",
       "      <td>178.000000</td>\n",
       "      <td>178.000000</td>\n",
       "      <td>178.000000</td>\n",
       "      <td>178.000000</td>\n",
       "      <td>178.000000</td>\n",
       "      <td>178.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>1.938202</td>\n",
       "      <td>13.000618</td>\n",
       "      <td>2.336348</td>\n",
       "      <td>2.366517</td>\n",
       "      <td>19.494944</td>\n",
       "      <td>99.741573</td>\n",
       "      <td>2.295112</td>\n",
       "      <td>2.029270</td>\n",
       "      <td>0.361854</td>\n",
       "      <td>1.590899</td>\n",
       "      <td>5.058090</td>\n",
       "      <td>0.957449</td>\n",
       "      <td>2.611685</td>\n",
       "      <td>746.893258</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.775035</td>\n",
       "      <td>0.811827</td>\n",
       "      <td>1.117146</td>\n",
       "      <td>0.274344</td>\n",
       "      <td>3.339564</td>\n",
       "      <td>14.282484</td>\n",
       "      <td>0.625851</td>\n",
       "      <td>0.998859</td>\n",
       "      <td>0.124453</td>\n",
       "      <td>0.572359</td>\n",
       "      <td>2.318286</td>\n",
       "      <td>0.228572</td>\n",
       "      <td>0.709990</td>\n",
       "      <td>314.907474</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>11.030000</td>\n",
       "      <td>0.740000</td>\n",
       "      <td>1.360000</td>\n",
       "      <td>10.600000</td>\n",
       "      <td>70.000000</td>\n",
       "      <td>0.980000</td>\n",
       "      <td>0.340000</td>\n",
       "      <td>0.130000</td>\n",
       "      <td>0.410000</td>\n",
       "      <td>1.280000</td>\n",
       "      <td>0.480000</td>\n",
       "      <td>1.270000</td>\n",
       "      <td>278.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>12.362500</td>\n",
       "      <td>1.602500</td>\n",
       "      <td>2.210000</td>\n",
       "      <td>17.200000</td>\n",
       "      <td>88.000000</td>\n",
       "      <td>1.742500</td>\n",
       "      <td>1.205000</td>\n",
       "      <td>0.270000</td>\n",
       "      <td>1.250000</td>\n",
       "      <td>3.220000</td>\n",
       "      <td>0.782500</td>\n",
       "      <td>1.937500</td>\n",
       "      <td>500.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>2.000000</td>\n",
       "      <td>13.050000</td>\n",
       "      <td>1.865000</td>\n",
       "      <td>2.360000</td>\n",
       "      <td>19.500000</td>\n",
       "      <td>98.000000</td>\n",
       "      <td>2.355000</td>\n",
       "      <td>2.135000</td>\n",
       "      <td>0.340000</td>\n",
       "      <td>1.555000</td>\n",
       "      <td>4.690000</td>\n",
       "      <td>0.965000</td>\n",
       "      <td>2.780000</td>\n",
       "      <td>673.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>3.000000</td>\n",
       "      <td>13.677500</td>\n",
       "      <td>3.082500</td>\n",
       "      <td>2.557500</td>\n",
       "      <td>21.500000</td>\n",
       "      <td>107.000000</td>\n",
       "      <td>2.800000</td>\n",
       "      <td>2.875000</td>\n",
       "      <td>0.437500</td>\n",
       "      <td>1.950000</td>\n",
       "      <td>6.200000</td>\n",
       "      <td>1.120000</td>\n",
       "      <td>3.170000</td>\n",
       "      <td>985.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>3.000000</td>\n",
       "      <td>14.830000</td>\n",
       "      <td>5.800000</td>\n",
       "      <td>3.230000</td>\n",
       "      <td>30.000000</td>\n",
       "      <td>162.000000</td>\n",
       "      <td>3.880000</td>\n",
       "      <td>5.080000</td>\n",
       "      <td>0.660000</td>\n",
       "      <td>3.580000</td>\n",
       "      <td>13.000000</td>\n",
       "      <td>1.710000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>1680.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             Wine     Alcohol  Malic.acid         Ash         Acl          Mg  \\\n",
       "count  178.000000  178.000000  178.000000  178.000000  178.000000  178.000000   \n",
       "mean     1.938202   13.000618    2.336348    2.366517   19.494944   99.741573   \n",
       "std      0.775035    0.811827    1.117146    0.274344    3.339564   14.282484   \n",
       "min      1.000000   11.030000    0.740000    1.360000   10.600000   70.000000   \n",
       "25%      1.000000   12.362500    1.602500    2.210000   17.200000   88.000000   \n",
       "50%      2.000000   13.050000    1.865000    2.360000   19.500000   98.000000   \n",
       "75%      3.000000   13.677500    3.082500    2.557500   21.500000  107.000000   \n",
       "max      3.000000   14.830000    5.800000    3.230000   30.000000  162.000000   \n",
       "\n",
       "          Phenols  Flavanoids  Nonflavanoid.phenols     Proanth   Color.int  \\\n",
       "count  178.000000  178.000000            178.000000  178.000000  178.000000   \n",
       "mean     2.295112    2.029270              0.361854    1.590899    5.058090   \n",
       "std      0.625851    0.998859              0.124453    0.572359    2.318286   \n",
       "min      0.980000    0.340000              0.130000    0.410000    1.280000   \n",
       "25%      1.742500    1.205000              0.270000    1.250000    3.220000   \n",
       "50%      2.355000    2.135000              0.340000    1.555000    4.690000   \n",
       "75%      2.800000    2.875000              0.437500    1.950000    6.200000   \n",
       "max      3.880000    5.080000              0.660000    3.580000   13.000000   \n",
       "\n",
       "              Hue          OD      Proline  \n",
       "count  178.000000  178.000000   178.000000  \n",
       "mean     0.957449    2.611685   746.893258  \n",
       "std      0.228572    0.709990   314.907474  \n",
       "min      0.480000    1.270000   278.000000  \n",
       "25%      0.782500    1.937500   500.500000  \n",
       "50%      0.965000    2.780000   673.500000  \n",
       "75%      1.120000    3.170000   985.000000  \n",
       "max      1.710000    4.000000  1680.000000  "
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "6e67385e",
   "metadata": {},
   "outputs": [],
   "source": [
    "x = df.drop('Wine', axis = 1)\n",
    "y = df[['Wine']]\n",
    "\n",
    "#Train test split\n",
    "from sklearn.model_selection import train_test_split as tts\n",
    "X_train, X_test, y_train, y_test = tts(x, y, test_size=0.3, random_state=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "a8a77807",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Alcohol</th>\n",
       "      <th>Malic.acid</th>\n",
       "      <th>Ash</th>\n",
       "      <th>Acl</th>\n",
       "      <th>Mg</th>\n",
       "      <th>Phenols</th>\n",
       "      <th>Flavanoids</th>\n",
       "      <th>Nonflavanoid.phenols</th>\n",
       "      <th>Proanth</th>\n",
       "      <th>Color.int</th>\n",
       "      <th>Hue</th>\n",
       "      <th>OD</th>\n",
       "      <th>Proline</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>13.71</td>\n",
       "      <td>1.86</td>\n",
       "      <td>2.36</td>\n",
       "      <td>16.6</td>\n",
       "      <td>101</td>\n",
       "      <td>2.61</td>\n",
       "      <td>2.88</td>\n",
       "      <td>0.27</td>\n",
       "      <td>1.69</td>\n",
       "      <td>3.80</td>\n",
       "      <td>1.11</td>\n",
       "      <td>4.00</td>\n",
       "      <td>1035</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>108</th>\n",
       "      <td>12.22</td>\n",
       "      <td>1.29</td>\n",
       "      <td>1.94</td>\n",
       "      <td>19.0</td>\n",
       "      <td>92</td>\n",
       "      <td>2.36</td>\n",
       "      <td>2.04</td>\n",
       "      <td>0.39</td>\n",
       "      <td>2.08</td>\n",
       "      <td>2.70</td>\n",
       "      <td>0.86</td>\n",
       "      <td>3.02</td>\n",
       "      <td>312</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>175</th>\n",
       "      <td>13.27</td>\n",
       "      <td>4.28</td>\n",
       "      <td>2.26</td>\n",
       "      <td>20.0</td>\n",
       "      <td>120</td>\n",
       "      <td>1.59</td>\n",
       "      <td>0.69</td>\n",
       "      <td>0.43</td>\n",
       "      <td>1.35</td>\n",
       "      <td>10.20</td>\n",
       "      <td>0.59</td>\n",
       "      <td>1.56</td>\n",
       "      <td>835</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>145</th>\n",
       "      <td>13.16</td>\n",
       "      <td>3.57</td>\n",
       "      <td>2.15</td>\n",
       "      <td>21.0</td>\n",
       "      <td>102</td>\n",
       "      <td>1.50</td>\n",
       "      <td>0.55</td>\n",
       "      <td>0.43</td>\n",
       "      <td>1.30</td>\n",
       "      <td>4.00</td>\n",
       "      <td>0.60</td>\n",
       "      <td>1.68</td>\n",
       "      <td>830</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>71</th>\n",
       "      <td>13.86</td>\n",
       "      <td>1.51</td>\n",
       "      <td>2.67</td>\n",
       "      <td>25.0</td>\n",
       "      <td>86</td>\n",
       "      <td>2.95</td>\n",
       "      <td>2.86</td>\n",
       "      <td>0.21</td>\n",
       "      <td>1.87</td>\n",
       "      <td>3.38</td>\n",
       "      <td>1.36</td>\n",
       "      <td>3.16</td>\n",
       "      <td>410</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     Alcohol  Malic.acid   Ash   Acl   Mg  Phenols  Flavanoids  \\\n",
       "22     13.71        1.86  2.36  16.6  101     2.61        2.88   \n",
       "108    12.22        1.29  1.94  19.0   92     2.36        2.04   \n",
       "175    13.27        4.28  2.26  20.0  120     1.59        0.69   \n",
       "145    13.16        3.57  2.15  21.0  102     1.50        0.55   \n",
       "71     13.86        1.51  2.67  25.0   86     2.95        2.86   \n",
       "\n",
       "     Nonflavanoid.phenols  Proanth  Color.int   Hue    OD  Proline  \n",
       "22                   0.27     1.69       3.80  1.11  4.00     1035  \n",
       "108                  0.39     2.08       2.70  0.86  3.02      312  \n",
       "175                  0.43     1.35      10.20  0.59  1.56      835  \n",
       "145                  0.43     1.30       4.00  0.60  1.68      830  \n",
       "71                   0.21     1.87       3.38  1.36  3.16      410  "
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "1916e8ea",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.42307209, 0.30374536, 0.15180604, 0.26832608, 0.20497099,\n",
       "       0.49171715, 0.7246538 , 0.09689377, 0.26236895, 0.62800268,\n",
       "       0.53656758, 0.55705506, 0.53671122])"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "\n",
    "from sklearn.feature_selection import mutual_info_classif as ms\n",
    "# determine the mutual information\n",
    "mutual_info = ms(X_train, y_train)\n",
    "mutual_info"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "dcb61c34",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Flavanoids              0.724654\n",
       "Color.int               0.628003\n",
       "OD                      0.557055\n",
       "Proline                 0.536711\n",
       "Hue                     0.536568\n",
       "Phenols                 0.491717\n",
       "Alcohol                 0.423072\n",
       "Malic.acid              0.303745\n",
       "Acl                     0.268326\n",
       "Proanth                 0.262369\n",
       "Mg                      0.204971\n",
       "Ash                     0.151806\n",
       "Nonflavanoid.phenols    0.096894\n",
       "dtype: float64"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mutual_info = pd.Series(mutual_info)\n",
    "mutual_info.index = X_train.columns\n",
    "mutual_info.sort_values(ascending = False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "de61522d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Axes: >"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABL4AAAMcCAYAAABekWxnAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABgoklEQVR4nO39eXSV1d0/7r8ThgDKZJFBjASLCigKhcIXtXVCsfo4thUVBaPytCpKiVrh81RwqigqUpWKAzi1DNahlWqpGqdasVQQpwoOIGA1CKWAQgVJ8vvDn6kRUAInOWRzXWudtWTf90leuVeEc15n33vnlJeXlwcAAAAAJCY32wEAAAAAoDoovgAAAABIkuILAAAAgCQpvgAAAABIkuILAAAAgCQpvgAAAABIkuILAAAAgCQpvgAAAABIUt1sB9gcZWVl8cEHH0Tjxo0jJycn23EAAAAAyJLy8vL4+OOPY5dddonc3K+f01Uriq8PPvgg8vPzsx0DAAAAgG3E4sWLY9ddd/3ac2pF8dW4ceOI+PwHatKkSZbTAAAAAJAtq1ativz8/Iq+6OvUiuLri9sbmzRpovgCAAAAYLOWw7K4PQAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkKS62Q6QbQXDHs12hCp775qjsx0BAAAAYJtnxhcAAAAASVJ8AQAAAJAkxRcAAAAASVJ8AQAAAJAkxRcAAAAASVJ8AQAAAJAkxRcAAAAASVJ8AQAAAJAkxRcAAAAASVJ8AQAAAJAkxRcAAAAASVJ8AQAAAJAkxRcAAAAASVJ8AQAAAJAkxRcAAAAASdqi4mvcuHFRUFAQDRo0iF69esXMmTM3ee7BBx8cOTk5GzyOPvroLQ4NAAAAAN+kysXX1KlTo6ioKEaOHBmzZ8+O/fbbL/r27RsfffTRRs9/6KGH4sMPP6x4vP7661GnTp348Y9/vNXhAQAAAGBTqlx8jRkzJgYNGhSFhYXRuXPnGD9+fDRq1CgmTpy40fN32mmnaN26dcXjiSeeiEaNGim+AAAAAKhWVSq+1q1bF7NmzYo+ffr89wvk5kafPn1ixowZm/U1JkyYECeffHLssMMOmzxn7dq1sWrVqkoPAAAAAKiKKhVfy5Yti9LS0mjVqlWl8VatWkVJSck3Pn/mzJnx+uuvx9lnn/21540aNSqaNm1a8cjPz69KTAAAAACo2V0dJ0yYEF26dImePXt+7XnDhw+PlStXVjwWL15cQwkBAAAASEXdqpzcokWLqFOnTixZsqTS+JIlS6J169Zf+9zVq1fHlClT4oorrvjG75OXlxd5eXlViQYAAAAAlVRpxlf9+vWje/fuUVxcXDFWVlYWxcXF0bt376997u9+97tYu3ZtnHbaaVuWFAAAAACqoEozviIiioqKYuDAgdGjR4/o2bNnjB07NlavXh2FhYURETFgwIBo27ZtjBo1qtLzJkyYEMcff3x861vfykxyAAAAAPgaVS6++vXrF0uXLo0RI0ZESUlJdO3aNaZPn16x4P2iRYsiN7fyRLJ58+bF888/H48//nhmUgMAAADAN8gpLy8vz3aIb7Jq1apo2rRprFy5Mpo0aZLRr10w7NGMfr2a8N41R2c7AgAAAEBWVKUnqtFdHQEAAACgpii+AAAAAEiS4gsAAACAJCm+AAAAAEiS4gsAAACAJCm+AAAAAEiS4gsAAACAJCm+AAAAAEiS4gsAAACAJCm+AAAAAEiS4gsAAACAJCm+AAAAAEiS4gsAAACAJCm+AAAAAEiS4gsAAACAJCm+AAAAAEiS4gsAAACAJCm+AAAAAEiS4gsAAACAJCm+AAAAAEiS4gsAAACAJCm+AAAAAEiS4gsAAACAJCm+AAAAAEiS4gsAAACAJCm+AAAAAEiS4gsAAACAJCm+AAAAAEiS4gsAAACAJCm+AAAAAEiS4gsAAACAJCm+AAAAAEiS4gsAAACAJCm+AAAAAEiS4gsAAACAJCm+AAAAAEiS4gsAAACAJCm+AAAAAEiS4gsAAACAJCm+AAAAAEiS4gsAAACAJCm+AAAAAEiS4gsAAACAJCm+AAAAAEiS4gsAAACAJCm+AAAAAEiS4gsAAACAJCm+AAAAAEiS4gsAAACAJCm+AAAAAEiS4gsAAACAJNXNdgC2DwXDHs12hCp575qjsx0BAAAA2EpmfAEAAACQJMUXAAAAAElSfAEAAACQJMUXAAAAAElSfAEAAACQJMUXAAAAAElSfAEAAACQJMUXAAAAAElSfAEAAACQJMUXAAAAAElSfAEAAACQJMUXAAAAAElSfAEAAACQJMUXAAAAAElSfAEAAACQJMUXAAAAAElSfAEAAACQJMUXAAAAAEnaouJr3LhxUVBQEA0aNIhevXrFzJkzv/b8FStWxHnnnRdt2rSJvLy82HPPPeOxxx7bosAAAAAAsDnqVvUJU6dOjaKiohg/fnz06tUrxo4dG3379o158+ZFy5YtNzh/3bp1cfjhh0fLli3jgQceiLZt28bChQujWbNmmcgPAAAAABtV5eJrzJgxMWjQoCgsLIyIiPHjx8ejjz4aEydOjGHDhm1w/sSJE2P58uXxwgsvRL169SIioqCgYOtSAwAAAMA3qNKtjuvWrYtZs2ZFnz59/vsFcnOjT58+MWPGjI0+55FHHonevXvHeeedF61atYp99tknrr766igtLd3k91m7dm2sWrWq0gMAAAAAqqJKxdeyZcuitLQ0WrVqVWm8VatWUVJSstHnzJ8/Px544IEoLS2Nxx57LC699NK44YYb4qqrrtrk9xk1alQ0bdq04pGfn1+VmAAAAABQ/bs6lpWVRcuWLeP222+P7t27R79+/eL//u//Yvz48Zt8zvDhw2PlypUVj8WLF1d3TAAAAAASU6U1vlq0aBF16tSJJUuWVBpfsmRJtG7deqPPadOmTdSrVy/q1KlTMdapU6coKSmJdevWRf369Td4Tl5eXuTl5VUlGgAAAABUUqUZX/Xr14/u3btHcXFxxVhZWVkUFxdH7969N/qcAw44IN55550oKyurGHvrrbeiTZs2Gy29AAAAACATqnyrY1FRUdxxxx1xzz33xJtvvhnnnHNOrF69umKXxwEDBsTw4cMrzj/nnHNi+fLlMWTIkHjrrbfi0UcfjauvvjrOO++8zP0UAAAAAPAVVbrVMSKiX79+sXTp0hgxYkSUlJRE165dY/r06RUL3i9atChyc//bp+Xn58ef//znGDp0aOy7777Rtm3bGDJkSFxyySWZ+ykAAAAA4CuqXHxFRAwePDgGDx680WPPPPPMBmO9e/eOF198cUu+FQAAAABskWrf1REAAAAAskHxBQAAAECSFF8AAAAAJEnxBQAAAECSFF8AAAAAJEnxBQAAAECSFF8AAAAAJEnxBQAAAECSFF8AAAAAJEnxBQAAAECSFF8AAAAAJEnxBQAAAECSFF8AAAAAJEnxBQAAAECSFF8AAAAAJEnxBQAAAECSFF8AAAAAJEnxBQAAAECSFF8AAAAAJEnxBQAAAECSFF8AAAAAJEnxBQAAAECSFF8AAAAAJEnxBQAAAECSFF8AAAAAJEnxBQAAAECSFF8AAAAAJEnxBQAAAECS6mY7AJAZBcMezXaEKnvvmqOzHQEAAICEmfEFAAAAQJIUXwAAAAAkSfEFAAAAQJIUXwAAAAAkSfEFAAAAQJIUXwAAAAAkSfEFAAAAQJIUXwAAAAAkSfEFAAAAQJIUXwAAAAAkSfEFAAAAQJIUXwAAAAAkSfEFAAAAQJIUXwAAAAAkSfEFAAAAQJLqZjsAQG1RMOzRbEeosveuOTrbEaqstl3n2niNAQBge2HGFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJ2qLia9y4cVFQUBANGjSIXr16xcyZMzd57t133x05OTmVHg0aNNjiwAAAAACwOapcfE2dOjWKiopi5MiRMXv27Nhvv/2ib9++8dFHH23yOU2aNIkPP/yw4rFw4cKtCg0AAAAA36TKxdeYMWNi0KBBUVhYGJ07d47x48dHo0aNYuLEiZt8Tk5OTrRu3bri0apVq60KDQAAAADfpErF17p162LWrFnRp0+f/36B3Nzo06dPzJgxY5PP++STT6Jdu3aRn58fxx13XLzxxhtf+33Wrl0bq1atqvQAAAAAgKqoUvG1bNmyKC0t3WDGVqtWraKkpGSjz9lrr71i4sSJ8Yc//CF+85vfRFlZWey///7x/vvvb/L7jBo1Kpo2bVrxyM/Pr0pMAAAAAKj+XR179+4dAwYMiK5du8ZBBx0UDz30UOy8885x2223bfI5w4cPj5UrV1Y8Fi9eXN0xAQAAAEhM3aqc3KJFi6hTp04sWbKk0viSJUuidevWm/U16tWrF926dYt33nlnk+fk5eVFXl5eVaIBAAAAQCVVmvFVv3796N69exQXF1eMlZWVRXFxcfTu3XuzvkZpaWm89tpr0aZNm6olBQAAAIAqqNKMr4iIoqKiGDhwYPTo0SN69uwZY8eOjdWrV0dhYWFERAwYMCDatm0bo0aNioiIK664Iv6//+//iw4dOsSKFSviuuuui4ULF8bZZ5+d2Z8EAAAAAL6kysVXv379YunSpTFixIgoKSmJrl27xvTp0ysWvF+0aFHk5v53Itm///3vGDRoUJSUlETz5s2je/fu8cILL0Tnzp0z91MAAAAAwFdUufiKiBg8eHAMHjx4o8eeeeaZSn++8cYb48Ybb9ySbwMAVIOCYY9mO0KVvXfN0dmOAABALVTtuzoCAAAAQDYovgAAAABIkuILAAAAgCQpvgAAAABIkuILAAAAgCQpvgAAAABIkuILAAAAgCQpvgAAAABIkuILAAAAgCQpvgAAAABIkuILAAAAgCQpvgAAAABIkuILAAAAgCQpvgAAAABIkuILAAAAgCQpvgAAAABIkuILAAAAgCQpvgAAAABIkuILAAAAgCQpvgAAAABIkuILAAAAgCQpvgAAAABIkuILAAAAgCQpvgAAAABIkuILAAAAgCQpvgAAAABIkuILAAAAgCQpvgAAAABIkuILAAAAgCQpvgAAAABIkuILAAAAgCQpvgAAAABIkuILAAAAgCQpvgAAAABIkuILAAAAgCQpvgAAAABIkuILAAAAgCQpvgAAAABIkuILAAAAgCQpvgAAAABIkuILAAAAgCQpvgAAAABIkuILAAAAgCQpvgAAAABIkuILAAAAgCQpvgAAAABIkuILAAAAgCQpvgAAAABIkuILAAAAgCQpvgAAAABIkuILAAAAgCQpvgAAAABIkuILAAAAgCQpvgAAAABIkuILAAAAgCTVzXYAAIAUFQx7NNsRquS9a47OdgQAgIwz4wsAAACAJCm+AAAAAEiS4gsAAACAJCm+AAAAAEiS4gsAAACAJCm+AAAAAEiS4gsAAACAJCm+AAAAAEiS4gsAAACAJCm+AAAAAEiS4gsAAACAJCm+AAAAAEiS4gsAAACAJCm+AAAAAEjSFhVf48aNi4KCgmjQoEH06tUrZs6cuVnPmzJlSuTk5MTxxx+/Jd8WAAAAADZblYuvqVOnRlFRUYwcOTJmz54d++23X/Tt2zc++uijr33ee++9FxdddFF873vf2+KwAAAAALC5qlx8jRkzJgYNGhSFhYXRuXPnGD9+fDRq1CgmTpy4yeeUlpZG//794/LLL4/dd999qwIDAAAAwOaoUvG1bt26mDVrVvTp0+e/XyA3N/r06RMzZszY5POuuOKKaNmyZZx11lmb9X3Wrl0bq1atqvQAAAAAgKqoUvG1bNmyKC0tjVatWlUab9WqVZSUlGz0Oc8//3xMmDAh7rjjjs3+PqNGjYqmTZtWPPLz86sSEwAAAACqd1fHjz/+OE4//fS44447okWLFpv9vOHDh8fKlSsrHosXL67GlAAAAACkqG5VTm7RokXUqVMnlixZUml8yZIl0bp16w3Of/fdd+O9996LY445pmKsrKzs829ct27Mmzcvvv3tb2/wvLy8vMjLy6tKNAAAAACopEozvurXrx/du3eP4uLiirGysrIoLi6O3r17b3B+x44d47XXXos5c+ZUPI499tg45JBDYs6cOW5hBAAAAKDaVGnGV0REUVFRDBw4MHr06BE9e/aMsWPHxurVq6OwsDAiIgYMGBBt27aNUaNGRYMGDWKfffap9PxmzZpFRGwwDgAAAACZVOXiq1+/frF06dIYMWJElJSURNeuXWP69OkVC94vWrQocnOrdekwAAAAAPhGVS6+IiIGDx4cgwcP3uixZ5555mufe/fdd2/JtwQAAACAKjE1CwAAAIAkKb4AAAAASJLiCwAAAIAkKb4AAAAASJLiCwAAAIAkKb4AAAAASJLiCwAAAIAkKb4AAAAASJLiCwAAAIAkKb4AAAAASJLiCwAAAIAkKb4AAAAASJLiCwAAAIAkKb4AAAAASJLiCwAAAIAkKb4AAAAASJLiCwAAAIAkKb4AAAAASJLiCwAAAIAkKb4AAAAASJLiCwAAAIAkKb4AAAAASJLiCwAAAIAkKb4AAAAASJLiCwAAAIAkKb4AAAAASJLiCwAAAIAkKb4AAAAASJLiCwAAAIAkKb4AAAAASJLiCwAAAIAkKb4AAAAASJLiCwAAAIAkKb4AAAAASJLiCwAAAIAkKb4AAAAASJLiCwAAAIAkKb4AAAAASJLiCwAAAIAkKb4AAAAASJLiCwAAAIAkKb4AAAAASJLiCwAAAIAkKb4AAAAASJLiCwAAAIAkKb4AAAAASJLiCwAAAIAkKb4AAAAASJLiCwAAAIAkKb4AAAAASJLiCwAAAIAkKb4AAAAASJLiCwAAAIAkKb4AAAAASJLiCwAAAIAkKb4AAAAASJLiCwAAAIAkKb4AAAAASJLiCwAAAIAkKb4AAAAASJLiCwAAAIAkKb4AAAAASJLiCwAAAIAkKb4AAAAASJLiCwAAAIAkKb4AAAAASJLiCwAAAIAkKb4AAAAASJLiCwAAAIAkKb4AAAAASNIWFV/jxo2LgoKCaNCgQfTq1Stmzpy5yXMfeuih6NGjRzRr1ix22GGH6Nq1a9x3331bHBgAAAAANkfdqj5h6tSpUVRUFOPHj49evXrF2LFjo2/fvjFv3rxo2bLlBufvtNNO8X//93/RsWPHqF+/fvzxj3+MwsLCaNmyZfTt2zcjPwQAANufgmGPZjtClb13zdHZjgAA25Uqz/gaM2ZMDBo0KAoLC6Nz584xfvz4aNSoUUycOHGj5x988MFxwgknRKdOneLb3/52DBkyJPbdd994/vnntzo8AAAAAGxKlYqvdevWxaxZs6JPnz7//QK5udGnT5+YMWPGNz6/vLw8iouLY968efH9739/k+etXbs2Vq1aVekBAAAAAFVRpeJr2bJlUVpaGq1atao03qpVqygpKdnk81auXBk77rhj1K9fP44++ui4+eab4/DDD9/k+aNGjYqmTZtWPPLz86sSEwAAAABqZlfHxo0bx5w5c+Lvf/97/PKXv4yioqJ45plnNnn+8OHDY+XKlRWPxYsX10RMAAAAABJSpcXtW7RoEXXq1IklS5ZUGl+yZEm0bt16k8/Lzc2NDh06RERE165d480334xRo0bFwQcfvNHz8/LyIi8vryrRAAAAAKCSKs34ql+/fnTv3j2Ki4srxsrKyqK4uDh69+692V+nrKws1q5dW5VvDQAAAABVUqUZXxERRUVFMXDgwOjRo0f07Nkzxo4dG6tXr47CwsKIiBgwYEC0bds2Ro0aFRGfr9fVo0eP+Pa3vx1r166Nxx57LO6777649dZbM/uTAAAAAMCXVLn46tevXyxdujRGjBgRJSUl0bVr15g+fXrFgveLFi2K3Nz/TiRbvXp1nHvuufH+++9Hw4YNo2PHjvGb3/wm+vXrl7mfAgAAAAC+osrFV0TE4MGDY/DgwRs99tVF66+66qq46qqrtuTbAAAAAMAWq5FdHQEAAACgpim+AAAAAEiS4gsAAACAJCm+AAAAAEiS4gsAAACAJCm+AAAAAEiS4gsAAACAJCm+AAAAAEiS4gsAAACAJCm+AAAAAEiS4gsAAACAJCm+AAAAAEiS4gsAAACAJCm+AAAAAEiS4gsAAACAJCm+AAAAAEiS4gsAAACAJCm+AAAAAEhS3WwHAAAAtk0Fwx7NdoQqe++ao7MdAYBtiBlfAAAAACRJ8QUAAABAkhRfAAAAACRJ8QUAAABAkixuDwAAkEU2EQCoPmZ8AQAAAJAkxRcAAAAASVJ8AQAAAJAkxRcAAAAASVJ8AQAAAJAkxRcAAAAASVJ8AQAAAJAkxRcAAAAASVJ8AQAAAJAkxRcAAAAASVJ8AQAAAJAkxRcAAAAASVJ8AQAAAJAkxRcAAAAASVJ8AQAAAJAkxRcAAAAASVJ8AQAAAJAkxRcAAAAASVJ8AQAAAJAkxRcAAAAASVJ8AQAAAJAkxRcAAAAASVJ8AQAAAJAkxRcAAAAASVJ8AQAAAJAkxRcAAAAASVJ8AQAAAJAkxRcAAAAASVJ8AQAAAJAkxRcAAAAASVJ8AQAAAJAkxRcAAAAASVJ8AQAAAJAkxRcAAAAASVJ8AQAAAJAkxRcAAAAASVJ8AQAAAJAkxRcAAAAASVJ8AQAAAJAkxRcAAAAASVJ8AQAAAJAkxRcAAAAASVJ8AQAAAJAkxRcAAAAASVJ8AQAAAJAkxRcAAAAASaq7JU8aN25cXHfddVFSUhL77bdf3HzzzdGzZ8+NnnvHHXfEvffeG6+//npERHTv3j2uvvrqTZ4PAAAAmVQw7NFsR6iy9645OtsRIAlVnvE1derUKCoqipEjR8bs2bNjv/32i759+8ZHH3200fOfeeaZOOWUU+Lpp5+OGTNmRH5+fhxxxBHxz3/+c6vDAwAAAMCmVLn4GjNmTAwaNCgKCwujc+fOMX78+GjUqFFMnDhxo+f/9re/jXPPPTe6du0aHTt2jDvvvDPKysqiuLh4q8MDAAAAwKZUqfhat25dzJo1K/r06fPfL5CbG3369IkZM2Zs1tdYs2ZNfPbZZ7HTTjtt8py1a9fGqlWrKj0AAAAAoCqqVHwtW7YsSktLo1WrVpXGW7VqFSUlJZv1NS655JLYZZddKpVnXzVq1Kho2rRpxSM/P78qMQEAAACgZnd1vOaaa2LKlCnx8MMPR4MGDTZ53vDhw2PlypUVj8WLF9dgSgAAAABSUKVdHVu0aBF16tSJJUuWVBpfsmRJtG7d+mufe/3118c111wTTz75ZOy7775fe25eXl7k5eVVJRoAAAAAVFKlGV/169eP7t27V1qY/ouF6nv37r3J540ePTquvPLKmD59evTo0WPL0wIAAADAZqrSjK+IiKKiohg4cGD06NEjevbsGWPHjo3Vq1dHYWFhREQMGDAg2rZtG6NGjYqIiGuvvTZGjBgRkyZNioKCgoq1wHbcccfYcccdM/ijAAAAAMB/Vbn46tevXyxdujRGjBgRJSUl0bVr15g+fXrFgveLFi2K3Nz/TiS79dZbY926dfGjH/2o0tcZOXJkXHbZZVuXHgAAAAA2ocrFV0TE4MGDY/DgwRs99swzz1T683vvvbcl3wIAAAAAtkqN7uoIAAAAADVF8QUAAABAkhRfAAAAACRJ8QUAAABAkhRfAAAAACRJ8QUAAABAkhRfAAAAACRJ8QUAAABAkhRfAAAAACRJ8QUAAABAkhRfAAAAACRJ8QUAAABAkhRfAAAAACRJ8QUAAABAkhRfAAAAACRJ8QUAAABAkhRfAAAAACRJ8QUAAABAkhRfAAAAACRJ8QUAAABAkhRfAAAAACRJ8QUAAABAkhRfAAAAACRJ8QUAAABAkhRfAAAAACRJ8QUAAABAkhRfAAAAACRJ8QUAAABAkhRfAAAAACRJ8QUAAABAkhRfAAAAACRJ8QUAAABAkupmOwAAAABQ+xUMezTbEarkvWuOznYEaoAZXwAAAAAkSfEFAAAAQJIUXwAAAAAkSfEFAAAAQJIUXwAAAAAkSfEFAAAAQJIUXwAAAAAkSfEFAAAAQJIUXwAAAAAkSfEFAAAAQJIUXwAAAAAkSfEFAAAAQJIUXwAAAAAkSfEFAAAAQJIUXwAAAAAkSfEFAAAAQJIUXwAAAAAkSfEFAAAAQJIUXwAAAAAkSfEFAAAAQJIUXwAAAAAkSfEFAAAAQJIUXwAAAAAkSfEFAAAAQJIUXwAAAAAkSfEFAAAAQJIUXwAAAAAkSfEFAAAAQJIUXwAAAAAkSfEFAAAAQJIUXwAAAAAkSfEFAAAAQJIUXwAAAAAkSfEFAAAAQJIUXwAAAAAkSfEFAAAAQJIUXwAAAAAkaYuKr3HjxkVBQUE0aNAgevXqFTNnztzkuW+88Ub88Ic/jIKCgsjJyYmxY8duaVYAAAAA2GxVLr6mTp0aRUVFMXLkyJg9e3bst99+0bdv3/joo482ev6aNWti9913j2uuuSZat2691YEBAAAAYHNUufgaM2ZMDBo0KAoLC6Nz584xfvz4aNSoUUycOHGj53/3u9+N6667Lk4++eTIy8vb6sAAAAAAsDmqVHytW7cuZs2aFX369PnvF8jNjT59+sSMGTMyFmrt2rWxatWqSg8AAAAAqIoqFV/Lli2L0tLSaNWqVaXxVq1aRUlJScZCjRo1Kpo2bVrxyM/Pz9jXBgAAAGD7sE3u6jh8+PBYuXJlxWPx4sXZjgQAAABALVO3Kie3aNEi6tSpE0uWLKk0vmTJkowuXJ+Xl2c9MAAAAIAvKRj2aLYjVNl71xyd1e9fpRlf9evXj+7du0dxcXHFWFlZWRQXF0fv3r0zHg4AAAAAtlSVZnxFRBQVFcXAgQOjR48e0bNnzxg7dmysXr06CgsLIyJiwIAB0bZt2xg1alREfL4g/j/+8Y+K//7nP/8Zc+bMiR133DE6dOiQwR8FAAAAAP6rysVXv379YunSpTFixIgoKSmJrl27xvTp0ysWvF+0aFHk5v53ItkHH3wQ3bp1q/jz9ddfH9dff30cdNBB8cwzz2z9TwAAAAAAG1Hl4isiYvDgwTF48OCNHvtqmVVQUBDl5eVb8m0AAAAAYIttk7s6AgAAAMDWUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkCTFFwAAAABJUnwBAAAAkKQtKr7GjRsXBQUF0aBBg+jVq1fMnDnza8//3e9+Fx07dowGDRpEly5d4rHHHtuisAAAAACwuapcfE2dOjWKiopi5MiRMXv27Nhvv/2ib9++8dFHH230/BdeeCFOOeWUOOuss+Lll1+O448/Po4//vh4/fXXtzo8AAAAAGxKlYuvMWPGxKBBg6KwsDA6d+4c48ePj0aNGsXEiRM3ev6vfvWrOPLII+Piiy+OTp06xZVXXhnf+c534pZbbtnq8AAAAACwKXWrcvK6deti1qxZMXz48Iqx3Nzc6NOnT8yYMWOjz5kxY0YUFRVVGuvbt2/8/ve/3+T3Wbt2baxdu7bizytXroyIiFWrVlUl7mYpW7sm41+zulXHdahute06u8Y1o7ZdZ9e4ZtS26+wa1wzXufq5xjWjtl1n17hmuM7VzzWuGbXtOrvGNaM6rvMXX7O8vPwbz61S8bVs2bIoLS2NVq1aVRpv1apVzJ07d6PPKSkp2ej5JSUlm/w+o0aNissvv3yD8fz8/KrETVbTsdlOkD7XuGa4ztXPNa5+rnHNcJ2rn2tcM1zn6uca1wzXufq5xtXPNa4Z1XmdP/7442jatOnXnlOl4qumDB8+vNIssbKysli+fHl861vfipycnCwm2zyrVq2K/Pz8WLx4cTRp0iTbcZLlOlc/17hmuM7VzzWuGa5z9XONq59rXDNc5+rnGtcM17n6ucY1o7Zd5/Ly8vj4449jl112+cZzq1R8tWjRIurUqRNLliypNL5kyZJo3br1Rp/TunXrKp0fEZGXlxd5eXmVxpo1a1aVqNuEJk2a1IpfmNrOda5+rnHNcJ2rn2tcM1zn6ucaVz/XuGa4ztXPNa4ZrnP1c41rRm26zt800+sLVVrcvn79+tG9e/coLi6uGCsrK4vi4uLo3bv3Rp/Tu3fvSudHRDzxxBObPB8AAAAAMqHKtzoWFRXFwIEDo0ePHtGzZ88YO3ZsrF69OgoLCyMiYsCAAdG2bdsYNWpUREQMGTIkDjrooLjhhhvi6KOPjilTpsRLL70Ut99+e2Z/EgAAAAD4kioXX/369YulS5fGiBEjoqSkJLp27RrTp0+vWMB+0aJFkZv734lk+++/f0yaNCl+8YtfxP/7f/8v9thjj/j9738f++yzT+Z+im1MXl5ejBw5coPbNcks17n6ucY1w3Wufq5xzXCdq59rXP1c45rhOlc/17hmuM7VzzWuGSlf55zyzdn7EQAAAABqmSqt8QUAAAAAtYXiCwAAAIAkKb4AAAAASJLiCwAAAIAkKb6oVZ577rlYv379BuPr16+P5557LguJAAAAgG2VXR2pVerUqRMffvhhtGzZstL4v/71r2jZsmWUlpZmKVk6nnrqqXjooYfivffei5ycnGjfvn386Ec/iu9///vZjgawXXjkkUc2+9xjjz22GpPA1rnppps2+9wLLrigGpMAtU1ZWVm888478dFHH0VZWVmlY96XVK8VK1ZEs2bNsh0joxRfGTR9+vTYcccd48ADD4yIiHHjxsUdd9wRnTt3jnHjxkXz5s2znLD2y83NjSVLlsTOO+9cafytt96KHj16xKpVq7KULA0//elP4/bbb4/mzZvHnnvuGeXl5fH222/HihUr4txzz42bb7452xGTsn79+njmmWfi3XffjVNPPTUaN24cH3zwQTRp0iR23HHHbMdLxn333Rfjx4+PBQsWxIwZM6Jdu3YxduzYaN++fRx33HHZjpec0tLSeO2116Jdu3b+3dtCubmbNyE/JyfHBz5bqSqvG5o0aVKNSdLUvn37zTovJycn5s+fX81pti/FxcVRXFy80dJg4sSJWUoFm+fFF1+MU089NRYuXBhfrSv825dZ1157bRQUFES/fv0iIuKkk06KBx98MFq3bh2PPfZY7LfffllOmBl1sx0gJRdffHFce+21ERHx2muvxYUXXhhFRUXx9NNPR1FRUdx1111ZTlh7nXjiiRHx+V90Z5xxRuTl5VUcKy0tjVdffTX233//bMVLwsMPPxx33XVXTJw4MQYOHBg5OTkR8fmnLXfffXecc845cfjhh5tdkCELFy6MI488MhYtWhRr166Nww8/PBo3bhzXXnttrF27NsaPH5/tiEm49dZbY8SIEfGzn/0sfvnLX1a8UGrWrFmMHTtW8ZUBP/vZz6JLly5x1llnRWlpaRx00EHxwgsvRKNGjeKPf/xjHHzwwdmOWOt89U0q1adZs2YV/959E2+0qm7BggXZjrBduvzyy+OKK66IHj16RJs2bTb7d5zNV1RUtNHxnJycaNCgQXTo0CGOO+642GmnnWo4WRp++tOfRo8ePeLRRx/1O1zNxo8fH7/97W8jIuKJJ56IJ554Iv70pz/F/fffHxdffHE8/vjjWU6YGWZ8ZdCOO+4Yr7/+ehQUFMRll10Wr7/+ejzwwAMxe/bsOOqoo6KkpCTbEWutwsLCiIi455574qSTToqGDRtWHKtfv34UFBTEoEGDokWLFtmKWOsde+yxsffee8eoUaM2evySSy6JuXPnxh/+8IcaTpam448/Pho3bhwTJkyIb33rW/HKK6/E7rvvHs8880wMGjQo3n777WxHTELnzp3j6quvrrjeX1zn119/PQ4++OBYtmxZtiPWervuumv8/ve/jx49esTvf//7OO+88+Lpp5+O++67L5566qn461//mu2IsEnPPvtsxX+/9957MWzYsDjjjDOid+/eERExY8aMuOeee2LUqFExcODAbMWEKmnTpk2MHj06Tj/99GxHSdYhhxwSs2fPjtLS0thrr70i4vM7UOrUqRMdO3aMefPmRU5OTjz//PPRuXPnLKetfXbYYYd45ZVXokOHDtmOkryGDRvGW2+9Ffn5+TFkyJD49NNP47bbbou33norevXqFf/+97+zHTEjzPjKoPr168eaNWsiIuLJJ5+MAQMGRETETjvt5Ba8rfTFbLmCgoK46KKLYocddshyovTMnj07fvGLX2zy+Iknnhg//OEPazBR2v7yl7/ECy+8EPXr1680XlBQEP/85z+zlCo9CxYsiG7dum0wnpeXF6tXr85CovQsW7YsWrduHRERjz32WPz4xz+OPffcM84888z41a9+leV0aXj22Wfj+uuvjzfffDMiPi90L7744vje976X5WS130EHHVTx31dccUWMGTMmTjnllIqxY489Nrp06RK333674msr/fCHP4yePXvGJZdcUml89OjR8fe//z1+97vfZSlZetatW+dOiGr2xWyuu+66q+I26JUrV8bZZ58dBx54YAwaNChOPfXUGDp0aPz5z3/Octrap1evXvHOO+8ovmpA8+bNY/HixZGfnx/Tp0+Pq666KiIiysvLk5rpbFfHDDrwwAOjqKgorrzyypg5c2YcffTREfF5+7/rrrtmOV0aRo4cqfSqJsuWLfva39Ndd901/vWvf9VgorSVlZVt9B+T999/Pxo3bpyFRGlq3759zJkzZ4Px6dOnR6dOnWo+UIJatWoV//jHP6K0tDSmT58ehx9+eERErFmzJurUqZPldLXfb37zm+jTp080atQoLrjggrjggguiYcOGcdhhh8WkSZOyHS8pM2bMiB49emww3qNHj5g5c2YWEqXlueeei6OOOmqD8R/84Ad25s6ws88+298P1ey6666LK6+8stLaf02bNo3LLrssRo8eHY0aNYoRI0bErFmzspiydnn11VcrHueff35ceOGFcffdd8esWbMqHXv11VezHTUpJ554Ypx66qlx+OGHx7/+9a/4wQ9+EBERL7/8clLFoxlfGXTLLbfEueeeGw888EDceuut0bZt24iI+NOf/hRHHnlkltOlYcmSJXHRRRdVLNb51Tt1U2qla9q6deuiXr16mzxet27dWLduXQ0mStsRRxwRY8eOjdtvvz0iPl8T4pNPPomRI0du9I0BW6aoqCjOO++8+PTTT6O8vDxmzpwZkydPjlGjRsWdd96Z7XhJKCwsjJNOOqliDY4+ffpERMTf/va36NixY5bT1X6//OUvY/To0TF06NCKsQsuuCDGjBkTV155ZZx66qlZTJeW/Pz8uOOOO2L06NGVxu+8887Iz8/PUqp0fPLJJxvMco6IqFevnjsjMuDLa06VlZXF7bffHk8++WTsu+++G7y+GzNmTE3HS87KlSvjo48+2uA2xqVLl1b8Pjdr1sxr5yro2rVr5OTkVHp/d+aZZ1b89xfHLG6fWTfeeGMUFBTE4sWLY/To0RUbbH344Ydx7rnnZjld5ljji1rlBz/4QSxatCgGDx680YUOLVS95XJzc+N///d/o1GjRhs9vmbNmrjjjjv8Q5Mh77//fvTt27di58wePXrE22+/HS1atIjnnnsuWrZsme2Iyfjtb38bl112Wbz77rsREbHLLrvE5ZdfHmeddVaWk6XjgQceiMWLF8ePf/zjipmj99xzTzRr1szfy1spLy8v3njjjQ0+dX3nnXdin332iU8//TRLydLz2GOPxQ9/+MPo0KFD9OrVKyIiZs6cGW+//XY8+OCDPpTYSj179oz/+Z//iREjRlQav+yyy2LatGlmxmylQw45ZLPPffrpp6sxyfahf//+MWPGjLjhhhviu9/9bkRE/P3vf4+LLroo9t9//7jvvvtiypQpcf3118dLL72U5bS1w8KFCzf73Hbt2lVjElKk+NpKtsGuWY0bN46//OUv0bVr12xHSc7BBx+8WTumeLGUOevXr48pU6bEq6++Gp988kl85zvfif79+1favIHMWbNmTXzyySdKRWqVDh06xMUXXxw/+clPKo2PHz8+brjhBhthZNjixYvj1ltvjblz50ZERKdOneKnP/2pGV8ZMG3atIpbag499NCIiCguLo5JkybFAw88EMcff3x2A0IVfPLJJzF06NC49957Y/369VFeXh716tWLgQMHxpgxY2LHHXesWGrB+xa2NY888shmn3vsscdWY5Kao/jaSrm5ubbBrkGdO3eO3/72txtdrJrM+mK3OztlAl910003bfa5F1xwQTUmSd+tt94aP/vZz+LMM8+sWKz6r3/9a9x9993xq1/9aoNCDLZljz76aFx99dUxZ86caNiwYey3334xcuTI2GmnnWKfffbJdrxkfLG5yFfXDF29enWcf/75MXHixCwlS88nn3wS8+fPj4iI3XffveI2MbbOPffcEy1atKhYM/vnP/953H777dG5c+eYPHmyGV9bKTd385Z6T+m2UsXXVrINds16/PHH44YbbojbbrstCgoKsh0nOStWrIj/+7//i6lTp1ZsXdu8efM4+eST46qrropmzZplN2Bi3n777Xj66afjo48+irKyskrHvnorCFumffv2X/vhxBcvVqma9u3bb9Z5OTk5rnEGPPzww3HDDTdU7OrYqVOnuPjii91GmgGvvvpq7LPPPpGbm/uNCybvu+++NZRq+7Bq1aqYPHlyTJgwIWbNmpXMm6ttQZ06deLDDz/cYIbzF7vwrl+/PkvJar8vrzn1dZSLW2evvfaKW2+9NQ499NCYMWNGHHbYYTF27Nj44x//GHXr1o2HHnoo2xGpZRRfGXTYYYfF2WefXWkb7IiISZMmxe233x7PPPNMdoIlpHnz5rFmzZpYv359NGrUaIPFOpcvX56lZLXf8uXLo3fv3vHPf/4z+vfvX7Hj3T/+8Y+YNGlS5OfnxwsvvBDNmzfPctI03HHHHXHOOedEixYtonXr1pXKmZycnJg9e3YW06XjV7/6VaU/f/bZZ/Hyyy/H9OnT4+KLL45hw4ZlKRmwLcjNzY2SkpJo2bJlxSz+jb00TulT72x77rnnYsKECfHggw/GLrvsEieeeGL88Ic/rFgniS23atWqKC8vj+bNm8fbb78dO++8c8Wx0tLSmDZtWgwbNiw++OCDLKas3XJzc6Ndu3bRrVu3jf5d8YWHH364BlOlp1GjRjF37tzYbbfd4pJLLokPP/ww7r333njjjTfi4IMPjqVLl2Y7IrWMXR0zaMaMGTF+/PgNxnv06BFnn312FhKlZ+zYsdmOkKwrrrgi6tevH++++260atVqg2NHHHFEXHHFFXHjjTdmKWFarrrqqvjlL38Zl1xySbajJG3IkCEbHR83bpzFZqvBF28CNncJADbfunXrNjo7dLfddstSojQsWLCgohxYsGBBltOkq6SkJO6+++6YMGFCrFq1Kk466aRYu3Zt/P73v99gVzy2XLNmzSInJydycnJizz333OB4Tk5OXH755VlIlo5zzjknJk+eHAsWLIjCwsI47bTTYqeddsp2rOTsuOOO8a9//St22223ePzxxyt2LW3QoEH85z//yXK69Dz77LNx/fXXV8ws79y5c1x88cXxve99L8vJMseMrwzaa6+94rjjjttgG+yf//zn8Yc//CHmzZuXpWTwzQoKCuK2226Lvn37bvT49OnT46c//Wm89957NRssUU2aNIk5c+bE7rvvnu0o26X58+dH165dq7RBCZt27733xnXXXVex0Pqee+4ZF198cZx++ulZTlb7vf3223HmmWfGCy+8UGnclu7UFsccc0w899xzcfTRR0f//v3jyCOPjDp16kS9evXilVdeUXxl0LPPPhvl5eVx6KGHxoMPPlipkKlfv360a9cudtlllywmTMPatWvjoYceiokTJ8YLL7wQRx99dJx11llxxBFH+OAnQ/r37x9z586Nbt26xeTJk2PRokXxrW99Kx555JEYPnx4vPHGG9mOmIzf/OY3UVhYGCeeeGIccMABEfH5WqIPP/xw3H333XHqqadmOWFmKL4yyDbY1WPVqlUVO2J+05tUO2duuby8vHj33Xdj11133ejx999/Pzp06BCffvppDSdL01lnnRXf/e5346c//Wm2o2yXRo8eHb/+9a8VuRkwZsyYuPTSS2Pw4MEVL5ief/75GDduXFx11VUxdOjQLCes3Q444ICoW7duDBs2LNq0abPBm6r99tsvS8nSM2rUqGjVqtUGa/hMnDgxli5daobuFqpbt25ccMEFcc4558Qee+xRMa74qj4LFy6M/Pz8zV7Ami23cOHCuPvuuyt2d3zjjTcscJ8BK1asiF/84hexePHiOOecc+LII4+MiIiRI0dGvXr14he/+EWWE6ajU6dO8b//+78bvF4bM2ZM3HHHHRWzwGo7tzpm0FFHHRVvvfVWpW2wjznmGNtgb6XmzZtXLND5xRTur/LJ99Zr0aJFvPfee5ssvhYsWGAqdwZ16NAhLr300njxxRejS5cuG6xXZye8zOjWrVulvzPKy8ujpKQkli5dGr/+9a+zmCwdN998c9x6660xYMCAirFjjz029t5777jssssUX1tpzpw5MWvWrOjYsWO2oyTvtttui0mTJm0wvvfee8fJJ5+s+NpCzz//fEyYMCG6d+8enTp1itNPPz1OPvnkbMdKWrt27WLFihUxc+bMjd4i/eW/r9k6X14b0PuQzGnWrFnccsstFX/++OOPY/LkyfGnP/0pZs2apfjKoPnz58cxxxyzwfixxx4b/+///b8sJKoeZnyxzXv22WcrPvH+8i6aG3PQQQfVUKr0nHnmmfHuu+/GE088EfXr1690bO3atdG3b9/Yfffd7VKTIV+3K56d8DLnsssuq1R85ebmxs477xwHH3ywIiFDGjRoEK+//np06NCh0vjbb78dXbp0MUt0K333u9+NG2+8MQ488MBsR0legwYN4s0339zg7+f58+dH586d/S5vpdWrV8fUqVNj4sSJMXPmzCgtLY0xY8bEmWeeGY0bN852vKRMmzYt+vfvH5988kk0adJkgw10bAa1db58q+Pzzz8f//M//xOFhYVx5JFHmmWXYTbDqH4dOnSIiy++OH7yk59UGh8/fnzccMMNFctY1HaKr61kG2xS8f7770ePHj0iLy8vzjvvvOjYsWOUl5fHm2++Gb/+9a9j7dq18dJLL5m9SK2wuWt3uT166+2zzz5x6qmnbvCp4FVXXRVTp06N1157LUvJaq8v//6+9NJL8Ytf/CKuvvrqjc4O9TucOXvssUeMHDkyTjvttErj9913X4wcOdIHEhk0b968mDBhQtx3332xYsWKOPzww+ORRx7Jdqxk7LnnnnHUUUfF1VdfHY0aNcp2nKSce+65MWXKlMjPz48zzzwz+vfvHy1atMh2rKRsbDOM8ePHuzW6mtx6663xs5/9LM4888zYf//9I+LzNb7uvvvu+NWvfrVBIVZbKb62km2wScmCBQvi3HPPjccff7zS7myHH3543HLLLRvM6IBt1Rd/H2+K26Mz58EHH4x+/fpFnz59Ki2KWlxcHPfff3+ccMIJWU5Y+3z19/eL39cv8zuceaNHj47Ro0fHddddF4ceemhERBQXF8fPf/7zuPDCC2P48OFZTpie0tLSmDZtWkycOFHxlUE77LBDvPbaazbQqQa5ubmx2267bbCUwlc99NBDNZgqHTbDyI6HH344brjhhor1vDp16hQXX3xxHHfccVlOljmKr620cOHC2G233SInJycWLlz4tee2a9euhlJtfzp16hRvvfWWNwAZ8u9//7tiWmuHDh2s7ZUhRUVFceWVV8YOO+xQsS3zpowZM6aGUqXpy7dFl5eXx1FHHRV33nlntG3bttJ5bo/OjFmzZsWNN95Y6QXThRdeGN26dctystrpm27r/zK/w5lTXl4ew4YNi5tuuinWrVsXEZ/f/njJJZfEiBEjspwONt+JJ54YJ598cpx00knZjpKcM844Y7N2brzrrrtqIE16bIZBdVF8UWusX78+Jk2aFH379o1WrVpVOvb73/8+Vq5cGQMHDsxSOvhmhxxySDz88MPRrFmzOOSQQzZ5Xk5OTjz11FM1mCx9jRs3jldeecWn38A3+uSTT+LNN9+Mhg0bxh577BF5eXnZjgRVMmHChLjiiiuisLBwo7dIH3vssVlKBl/vxRdfjAkTJsTUqVMrbYbRpk0bxVc1W7du3UY3w9htt92ylCizFF8Z9u6778bYsWMrPvXu3LlzDBkyJL797W9nOVkaGjVqFG+++abZc0CVKL6qV1lZWbzzzjsbfcH0/e9/P0up0rFixYqYMGFCxWuLvffeO84888xo2rRplpMB26KvW2DdLdLUBjbDqDlvv/12nHnmmfHCCy9UGk9tSQXFVwb9+c9/jmOPPTa6du1aaZ2TV155JaZNmxaHH354lhPWfgcffHAMHTo0qfuNgeqn+Ko+L774Ypx66qmxcOHCDda4TOkFU7a89NJL0bdv32jYsGH07NkzIiL+/ve/x3/+8594/PHH4zvf+U6WE6blpZdeivvvvz8WLVpUcbvjF6zZA1DzbIZRvQ444ICoW7duDBs2LNq0abPBrbz77bdflpJlluIrg7p16xZ9+/aNa665ptL4sGHD4vHHH4/Zs2dnKVk67r///hg+fHgMHTo0unfvHjvssEOl43bOZFt24oknbva53mBlVuPGjePVV1+N9u3bZztKcrp27Rp77rlnXH755Rt9wWRW0tb53ve+Fx06dIg77rgj6tatGxGf3/p/9tlnx/z58+O5557LcsJ0TJkyJQYMGBB9+/aNxx9/PI444oh46623YsmSJXHCCSdYswcgi2yGUT122GGHmDVrVnTs2DHbUaqV4iuDGjRoEK+99lqlhfgiIt56663Yd99949NPP81SsnRsbOr2FztpmlnAtq6wsHCzz/UGa+t8tWScNm1aHHrooRuU5QrGrbfDDjvEK6+8YtfXatKwYcN4+eWXN3hB+o9//CN69OgRa9asyVKy9Oy7777xk5/8JM4777yKWaLt27ePn/zkJ9GmTZu4/PLLsx0RNtvq1avj2Wef3ejsxQsuuCBLqYBtzXe/+9248cYb48ADD8x2lGpVN9sBUrLzzjvHnDlzNii+5syZEy1btsxSqrQsWLAg2xFgiymzas5XZxmddtppWUqSvl69esU777yj+KomTZo0iUWLFm1QfC1evNg6Jxn27rvvxtFHHx0REfXr14/Vq1dHTk5ODB06NA499FDFF7XGyy+/HEcddVSsWbMmVq9eHTvttFMsW7YsGjVqFC1btlR8ARWuvfba+PnPfx5XX331RjfDaNKkSZaSZZbiK4MGDRoU//u//xvz58+P/fffPyI+X+Pr2muvjaKioiynS4NF7UnN0qVLY968eRERsddee8XOO++c5URpUDLWnPPPPz8uvPDCKCkp2egLJregb51+/frFWWedFddff32l1xYXX3xxnHLKKVlOl5bmzZvHxx9/HBERbdu2jddffz26dOkSK1asMLOOWmXo0KFxzDHHxPjx46Np06bx4osvRr169eK0006LIUOGZDsesA3p06dPREQcdthhlcZTu6PKrY4ZVF5eHmPHjo0bbrghPvjgg4iI2GWXXeLiiy+OCy64YIN1T9gyds4kBatXr47zzz8/7r333opd8OrUqRMDBgyIm2++ORo1apTlhLB53IJevdatWxcXX3xxjB8/PtavXx8REfXq1YtzzjknrrnmmsjLy8tywnSceuqp0aNHjygqKoorr7wybr755jjuuOPiiSeeiO985ztujabWaNasWfztb3+LvfbaK5o1axYzZsyITp06xd/+9rcYOHBgzJ07N9sRgW3Es88++7XHDzrooBpKUr0UX9Xki08M3YaQWXbOJBU/+clP4sknn4xbbrml4nf5+eefjwsuuCAOP/zwuPXWW7OcEDbPwoULv/a4mbqZsWbNmnj33XcjIuLb3/62crwaLF++PD799NPYZZddoqysLEaPHh0vvPBC7LHHHvGLX/wimjdvnu2IsFl23nnnit/dPffcM26++ebo27dvzJ07N7p37x6rV6/OdkSAGqX4olaxcyapaNGiRTzwwANx8MEHVxp/+umn46STToqlS5dmJxiwTVm5cmWUlpbGTjvtVGl8+fLlUbdu3WTW3gAy54gjjogzzjgjTj311Bg0aFC8+uqrccEFF8R9990X//73v+Nvf/tbtiMC25C//OUvcdttt8X8+fPjd7/7XbRt2zbuu+++aN++fTKL3m94fwJbbMmSJXH66afHLrvsEnXr1o06depUerD13nzzzTjrrLM2GD/zzDPjH//4RxYSwZZZs2ZNtGrVaoPxli1bWkuGWue+++6LAw44IHbZZZeKGWBjx46NP/zhD1lOVvudfPLJMWXKlA3G77///jj55JOzkAjY1l199dXRpk2biIj45S9/Gc2bN49zzjknli5dGrfffnuW0wHbkgcffDD69u0bDRs2jNmzZ8fatWsj4vMP3q6++uosp8scxVcGnXHGGTF79uy49NJL44EHHoiHHnqo0oOt98XOmV9l50xqm969e8fIkSPj008/rRj7z3/+E5dffnn07t07i8mgam699dYoKiqKo446KlasWFGxplezZs1i7Nix2Q2XgL/97W9xyCGHbDB+8MEHm7VRQ/r06RO77757tmPAZuvRo0fF3xstW7aM6dOnx6pVq2LWrFmx3377ZTkdsC256qqrYvz48XHHHXdU2qDogAMOSOpuKrs6ZtDzzz8ff/nLX6Jr167ZjpIsO2eSirFjx8aRRx4Zu+66a8WL0FdeeSUaNGgQf/7zn7OcDjbfzTffHHfccUccf/zxlW5D79GjR1x00UVZTJaGtWvXVixq/2WfffZZ/Oc//8lCou3PCSecEMuWLct2DKiyL+8c3bFjx2jRokWWEwHbmnnz5sX3v//9DcabNm0aK1asqPlA1UTxlUH5+flhybTqdemll0bjxo3jhhtuiOHDh0fE5ztnXnbZZXHBBRdkOR1svi5dusTbb78dv/3tbyt2VzrllFOif//+0bBhwyyng823YMGC6Nat2wbjeXl5FlDOgJ49e8btt98eN998c6Xx8ePHR/fu3bOUavty3nnnZTsCVMkXO0ffd999FbNw7RwNbEzr1q3jnXfeiYKCgkrjzz//fFKznRVfGTR27NgYNmxY3HbbbRv84pAZOTk5MXTo0Bg6dKidM6m1Pvvss+jYsWP88Y9/jEGDBmU7DmyV9u3bx5w5czbYvXH69OnRqVOnLKVKx1VXXRV9+vSJV155JQ477LCIiCguLo6///3v8fjjj2c5XVpsJEAqioqK4tlnn41HHnlkg52jL7zwQjtHAxUGDRoUQ4YMiYkTJ0ZOTk588MEHMWPGjLjooovi0ksvzXa8jFF8ZVC/fv1izZo1FduMf/ke2YjPXziROQovaqt69epVWtsLarOioqI477zz4tNPP43y8vKYOXNmTJ48OUaNGhV33nlntuPVegcccEDMmDEjrrvuurj//vujYcOGse+++8aECRNijz32yHa8pJx88slxzDHHxLnnnltp/P77749HHnkkHnvssSwlg6p58MEHN9g5+qijjoqGDRvGSSedpPgCKgwbNizKysrisMMOizVr1sT3v//9yMvLi4suuijOP//8bMfLmJxy9+ZlzD333PO1xwcOHFhDSdLSrVu3yMnJ2axzU1qAj7RdffXV8dZbb8Wdd94Zdev6DILa7be//W1cdtll8e6770bE57egX3755RvdhRe2VTvttFP89a9/3WCm4ty5c+OAAw6If/3rX1lKBlXTqFGjmDVr1ga/y2+88Ub07NnTbejABtatWxfvvPNOfPLJJ9G5c+fYcccdsx0poxRfbPMuv/zyzT535MiR1ZgEMueEE06I4uLi2HHHHaNLly6xww47VDpuJ1hqozVr1sQnn3xil92ttGrVqs0+1+13mbPDDjvEiy++GF26dKk0/tprr0WvXr1izZo1WUoGVXPYYYfFt771rbj33nujQYMGEfH5ztEDBw6M5cuXx5NPPpnlhAA1S/FVTT799NNYt25dpTEvToEvFBYWfu3xu+66q4aSANua3Nzcb5zpXF5eHjk5ORULV7P1DjnkkNhnn3022EjgvPPOi1dffTX+8pe/ZCkZVM1rr70WRx55ZKxdu3ajO0fvvffeWU4IbCtWr14d11xzTRQXF8dHH30UZWVllY7Pnz8/S8kyy/01GbR69eq45JJL4v7779/odHgvTjNn1qxZ8eabb0ZExN57773RHcVgW1RWVhbXXXddvPXWW7Fu3bo49NBD47LLLrOTI7XWkiVL4qKLLqp4wfTVz9P821d1Tz/99Gad99prr1Vzku2LjQRIhZ2jgc119tlnx7PPPhunn356tGnTZrOXGKptzPjKoPPOOy+efvrpuPLKK+P000+PcePGxT//+c+47bbb4pprron+/ftnO2Kt99FHH8XJJ58czzzzTDRr1iwiIlasWBGHHHJITJkyJXbeeefsBoRvcOWVV8Zll10Wffr0iYYNG8af//znOOWUU2LixInZjgZb5Ac/+EEsWrQoBg8evNEXTMcdd1yWkqXp448/jsmTJ8edd94Zs2bNUixm2Jw5c+K6666LOXPmVGwkMHz4cBsJUGt8eedoO+sC36RZs2bx6KOPVuwAmyrFVwbttttuce+998bBBx8cTZo0idmzZ0eHDh3ivvvui8mTJ9sNKAP69esX8+fPj3vvvbfiH/N//OMfMXDgwOjQoUNMnjw5ywnh6+2xxx5x0UUXxU9+8pOIiHjyySfj6KOPjv/85z+Rm5ub5XRQdY0bN46//OUv0bVr12xHSdpzzz0XEyZMiAcffDB22WWXOPHEE+OHP/xhfPe73812NGAb07Zt23jyyScVX8A3at++fTz22GPJ/33hXVYGLV++PHbfffeI+Hw9r+XLl0dExIEHHhjPPfdcNqMlY/r06fHrX/+60v+YnTt3jnHjxsWf/vSnLCaDzbNo0aI46qijKv7cp0+fyMnJiQ8++CCLqWDL5efnb3B7I5lRUlIS11xzTeyxxx7x4x//OJo0aRJr166N3//+93HNNdcovTLgyxsJrFq16msfUFucd955ce2118b69euzHQXYxl155ZUxYsSI5DdwscZXBu2+++6xYMGC2G233aJjx45x//33R8+ePWPatGkVt+WxdcrKyqJevXobjNerV2+DhfhgW7R+/fqKHZa+UK9evfjss8+ylAi2ztixY2PYsGFx2223RUFBQbbjJOOYY46J5557Lo4++ugYO3ZsHHnkkVGnTp0YP358tqMlpXnz5vHhhx9Gy5Yto1mzZhtd28RGAtQ2f//736O4uDgef/xxO0cDG+jWrVulf+/eeeedaNWqVRQUFGzwXnv27Nk1Ha9aKL4yqLCwMF555ZU46KCDYtiwYXHMMcfELbfcEp999lmMGTMm2/GScOihh8aQIUNi8uTJscsuu0RExD//+c8YOnRoxUK0sC0rLy+PM844I/Ly8irGPv300/jpT39a6YWpF6Vsy5o3b17pBdPq1avj29/+djRq1GiDF0xfzH6mav70pz/FBRdcEOecc471parRU089FTvttFNEbP6mArCta9asWfzwhz/MdgxgG3X88cdnO0KNs8ZXNVq4cGHMmjUrOnToEPvuu2+24yRh8eLFceyxx8Ybb7wR+fn5FWP77LNPPPLII7HrrrtmOSF8vcLCws0676677qrmJLDl7rnnns0+d+DAgdWYJF0vvvhiTJgwIaZOnRqdOnWK008/PU4++eRo06ZNvPLKK9G5c+dsRwS2MV/sHP3II4/YORrgSxRfGbR48eKKMobqU15eHk8++WTF9sydOnWKPn36ZDkVwPaltLQ0rr/++oo3WIcddliMHDnSG6wMW716dUydOjUmTpwYM2fOjNLS0hgzZkyceeaZ0bhx42zHq/VeffXVzT7Xh5hs6+wcDWypl156Kd58882I+HwN7e7du2c5UWYpvjKoTp06ceCBB8Zpp50WP/rRj6J58+bZjpSMp556KgYPHhwvvvhiNGnSpNKxlStXxv777x/jx4+P733ve1lKCLB98Qar5s2bNy8mTJgQ9913X6xYsSIOP/zweOSRR7Idq1bLzc2NnJycb9ygwRpf1AZ2jgaq6v33349TTjkl/vrXv1asS75ixYrYf//9Y8qUKcncUaX4yqCXX345Jk2aFFOmTImlS5fGkUceGaeddlocc8wxldbzoeqOPfbYOOSQQ2Lo0KEbPX7TTTfF008/HQ8//HANJwPYPnmDlT2lpaUxbdq0mDhxouJrKy1cuHCzz23Xrl01JoGtl5eXF++8806lO1AaNGgQ77zzTjJvXoHMOvLII2PFihVxzz33xF577RURn3/QVlhYGE2aNInp06dnOWFmKL6qQXl5eTzzzDMxadKkePDBB6OsrCxOPPFEn4JvhXbt2sX06dOjU6dOGz0+d+7cOOKII2LRokU1nAxg++QNFsC2pU6dOlFSUhI777xzxVjjxo3j1Vdfjfbt22cxGbCtatiwYbzwwgvRrVu3SuOzZs2K733ve7FmzZosJcssuzpWg5ycnDjkkEPikEMOiXPOOSfOOuusuOeeexRfW2HJkiUb7BT2ZXXr1o2lS5fWYCKA7dv69eujQYMGlcbq1asXn332WZYSQWb84x//iEWLFsW6desqjR977LFZSgSbx87RQFXl5+dv9LVbaWlp7LLLLllIVD0UX9Xg/fffj0mTJsWkSZPi9ddfj969e8e4ceOyHatWa9u2bbz++uvRoUOHjR5/9dVXo02bNjWcCmD75Q0WqZk/f36ccMIJ8dprr1Va9ysnJyciwhpfbPM2tovuaaedloUkQG1x3XXXxfnnnx/jxo2LHj16RMTnC90PGTIkrr/++iynyxy3OmbQbbfdFpMmTYq//vWv0bFjx+jfv3+ceuqp1oTIgPPPPz+eeeaZ+Pvf/77BDIP//Oc/0bNnzzjkkEPipptuylJCgO1LYWHhZp131113VXMSyIxjjjkm6tSpE3feeWe0b98+Zs6cGf/617/iwgsvjOuvv94GOgAkp3nz5rFmzZpYv3591K37+byoL/77yx9kRkQsX748GxEzQvGVQfn5+XHKKadE//79Y7/99st2nKQsWbIkvvOd70SdOnVi8ODBFQvvzZ07N8aNGxelpaUxe/bsaNWqVZaTAgC1UYsWLeKpp56KfffdN5o2bRozZ86MvfbaK5566qm48MIL4+WXX852RADIqHvuuWezz93YrNLawq2OGbRo0aKK6fBkVqtWreKFF16Ic845J4YPH17p9oO+ffvGuHHjlF4AwBYrLS2Nxo0bR8TnJdgHH3wQe+21V7Rr1y7mzZuX5XQAkHm1ucyqCsVXBn1Req1Zs2aji6Luu+++2YiVjHbt2sVjjz0W//73v+Odd96J8vLy2GOPPaJ58+bZjgYA1HL77LNPvPLKK9G+ffvo1atXjB49OurXrx+333577L777tmOBwBsIbc6ZtDSpUvjjDPOiOnTp2/0uEVRAQC2TX/+859j9erVceKJJ8bbb78dxxxzTLz11lvxrW99K6ZMmRKHHXZYtiMCQI3o06dPzJ8/P+bPn5/tKBlhxlcG/exnP4uVK1fG3/72tzj44IPj4YcfjiVLlsRVV10VN9xwQ7bjAQCwCX379q347z322CPmzp0by5cvj+bNm1vKAoDtygknnBDLli3LdoyMMeMrg9q0aRN/+MMfomfPntGkSZN46aWXYs8994xHHnkkRo8eHc8//3y2IwIA8CVnnnnmZp03ceLEak4CAFQHM74yaPXq1dGyZcuI+Hxb0KVLl8aee+4ZXbp0idmzZ2c5HQAAX3X33XdHu3btolu3buHzYABIj+Irg/baa6+YN29eFBQUxH777Re33XZbFBQUxPjx46NNmzbZjgcAwFecc845MXny5FiwYEEUFhbGaaedFjvttFO2YwFAtTjxxBM3+9yHHnqoGpPUnNxsB0jJkCFD4sMPP4yIiJEjR8af/vSn2G233eKmm26Kq6++OsvpAAD4qnHjxsWHH34YP//5z2PatGmRn58fJ510Uvz5z382AwyA5DRt2rTi0aRJkyguLo6XXnqp4visWbOiuLg4mjZtmsWUmWWNr2q0Zs2amDt3buy2227RokWLbMcBAOAbLFy4MO6+++649957Y/369fHGG2/EjjvumO1YAJBxl1xySSxfvjzGjx8fderUiYiI0tLSOPfcc6NJkyZx3XXXZTlhZii+AADg/2/x4sVx1113xd133x3r1q2LuXPnKr4ASNLOO+8czz//fOy1116VxufNmxf7779//Otf/8pSssyyxtdWKioq2uxzx4wZU41JAADYEmvXro2HHnooJk6cGM8//3z8z//8T9xyyy1x5JFHRm6ulUEASNP69etj7ty5GxRfc+fOjbKysiylyjzF11Z6+eWXN+u8nJycak4CAEBVnXvuuTFlypTIz8+PM888MyZPnmyJCgC2C4WFhXHWWWfFu+++Gz179oyIiL/97W9xzTXXRGFhYZbTZY5bHTNg/vz5UVBQ4BNBAIBaJjc3N3bbbbfo1q3b135QmcrOVgDwhbKysrj++uvjV7/6VcVGfW3atIkhQ4bEhRdeWLHuV22n+MqAOnXqxIcffhgtW7aMiIh+/frFTTfdFK1atcpyMgAAvs4ZZ5yxWTPz77rrrhpIAwDZsWrVqoiIaNKkSZaTZJ7iKwNyc3OjpKSkovhq3LhxvPLKK7H77rtnORkAAADA9ssaXwAAAADbge985ztRXFwczZs3/8bb/GfPnl2DyaqP4isDcnJyNvhlsZg9AAAAsC057rjjIi8vLyIijj/++OyGqSFudcyA3Nzc+MEPflDxyzNt2rQ49NBDY4cddqh0nkVRAQAAAGqOGV8ZMHDgwEp/Pu2007KUBAAAAGDzzZo1K958882IiNh7772jW7duWU6UWWZ8AQAAAGxnPvroozj55JPjmWeeiWbNmkVExIoVK+KQQw6JKVOmxM4775zdgBmSm+0AAAAAANSs888/Pz7++ON44403Yvny5bF8+fJ4/fXXY9WqVXHBBRdkO17GmPEFAAAAsJ1p2rRpPPnkk/Hd73630vjMmTPjiCOOiBUrVmQnWIaZ8QUAAACwnSkrK4t69eptMF6vXr0oKyvLQqLqofgCAAAA2M4ceuihMWTIkPjggw8qxv75z3/G0KFD47DDDstissxyqyMAAADAdmbx4sVx7LHHxhtvvBH5+fkVY/vss0888sgjseuuu2Y5YWYovgAAAAC2Q+Xl5fHkk0/G3LlzIyKiU6dO0adPnyynyizFFwAAAABJqpvtAAAAAADUvOLi4iguLo6PPvpogwXtJ06cmKVUmaX4AgAAANjOXH755XHFFVdEjx49ok2bNpGTk5PtSNXCrY4AAAAA25k2bdrE6NGj4/TTT892lGqVm+0AAAAAANSsdevWxf7775/tGNVO8QUAAACwnTn77LNj0qRJ2Y5R7azxBQAAALCd+fTTT+P222+PJ598Mvbdd9+oV69epeNjxozJUrLMssYXAAAAwHbmkEMO2eSxnJyceOqpp2owTfVRfAEAAACQJGt8AQAAAJAka3wBAAAAbIdeeumluP/++2PRokWxbt26SsceeuihLKXKLDO+AAAAALYzU6ZMif333z/efPPNePjhh+Ozzz6LN954I5566qlo2rRptuNljOILAAAAYDtz9dVXx4033hjTpk2L+vXrx69+9auYO3dunHTSSbHbbrtlO17GKL4AAAAAtjPvvvtuHH300RERUb9+/Vi9enXk5OTE0KFD4/bbb89yusxRfAEAAABsZ5o3bx4ff/xxRES0bds2Xn/99YiIWLFiRaxZsyab0TLK4vYAAAAA25nvf//78cQTT0SXLl3ixz/+cQwZMiSeeuqpeOKJJ+Kwww7LdryMySkvLy/PdggAAAAAas7y5cvj008/jV122SXKyspi9OjR8cILL8Qee+wRv/jFL6J58+bZjpgRii8AAAAAkmSNLwAAAIDtTJ8+feLuu++OVatWZTtKtVJ8AQAAAGxn9t577xg+fHi0bt06fvzjH8cf/vCH+Oyzz7IdK+Pc6ggAAACwHSorK4snn3wyJk2aFA8//HDUqVMnfvSjH0X//v3joIMOyna8jFB8AQAAAGznPv3005g2bVr88pe/jNdeey1KS0uzHSkj6mY7AAAAAADZU1JSElOmTInf/OY38eqrr0bPnj2zHSljrPEFAAAAsJ1ZtWpV3HXXXXH44YdHfn5+3HrrrXHsscfG22+/HS+++GK242WMWx0BAAAAtjMNGzaM5s2bR79+/aJ///7Ro0ePbEeqFoovAAAAgO3ME088EYcddljk5qZ9M6DiCwAAAIAkWdweAAAAYDv0wAMPxP333x+LFi2KdevWVTo2e/bsLKXKrLTnswEAAACwgZtuuikKCwujVatW8fLLL0fPnj3jW9/6VsyfPz9+8IMfZDtexrjVEQAAAGA707Fjxxg5cmSccsop0bhx43jllVdi9913jxEjRsTy5cvjlltuyXbEjDDjCwAAAGA7s2jRoth///0j4vMdHj/++OOIiDj99NNj8uTJ2YyWUYovAAAAgO1M69atY/ny5RERsdtuu8WLL74YERELFiyIlG4OVHwBAAAAbGcOPfTQeOSRRyIiorCwMIYOHRqHH3549OvXL0444YQsp8sca3wBAAAAbGfKysqirKws6tatGxERU6ZMiRdeeCH22GOP+MlPfhL169fPcsLMUHwBAAAAkKS62Q4AAAAAQPV79dVXN/vcfffdtxqT1BwzvgAAAAC2A7m5uZGTk/ONi9fn5OREaWlpDaWqXmZ8AQAAAGwHFixYkO0INc6MLwAAAIDtwHe+850oLi6O5s2bxxVXXBEXXXRRNGrUKNuxqpXiCwAAAGA70LBhw3j77bdj1113jTp16sSHH34YLVu2zHasauVWRwAAAIDtQNeuXaOwsDAOPPDAKC8vj+uvvz523HHHjZ47YsSIGk5XPcz4AgAAANgOzJs3L0aOHBnvvvtuzJ49Ozp37hx16244JyonJydmz56dhYSZp/gCAAAA2M7k5uZGSUlJ8rc6Kr4AAAAASJI1vgAAAAC2Q2+//XY8/fTT8dFHH0VZWVmlY9b4AgAAAKBWuuOOO+Kcc86JFi1aROvWrSMnJ6fimDW+AAAAAKi12rVrF+eee25ccskl2Y5SrRRfAAAAANuZJk2axJw5c2L33XfPdpRqlZvtAAAAAADUrB//+Mfx+OOPZztGtbO4PQAAAMB2pkOHDnHppZfGiy++GF26dIl69epVOn7BBRdkKVlmudURAAAAYDvTvn37TR7LycmJ+fPn12Ca6qP4AgAAACBJ1vgCAAAA2I6Vl5dHqvOiFF8AAAAA26F77703unTpEg0bNoyGDRvGvvvuG/fdd1+2Y2WUxe0BAAAAtjNjxoyJSy+9NAYPHhwHHHBAREQ8//zz8dOf/jSWLVsWQ4cOzXLCzLDGFwAAAMB2pn379nH55ZfHgAEDKo3fc889cdlll8WCBQuylCyz3OoIAAAAsJ358MMPY//9999gfP/9948PP/wwC4mqh+ILAAAAYDvToUOHuP/++zcYnzp1auyxxx5ZSFQ9rPEFAAAAsJ25/PLLo1+/fvHcc89VrPH117/+NYqLizdaiNVW1vgCAAAA2A7NmjUrxowZE3Pnzo2IiE6dOsWFF14Y3bp1y3KyzFF8AQAAAJAktzoCAAAAbCdyc3MjJyfna8/JycmJ9evX11Ci6qX4AgAAANhOPPzww5s8NmPGjLjpppuirKysBhNVL7c6AgAAAGzH5s2bF8OGDYtp06ZF//7944orroh27dplO1ZG5GY7AAAAAAA174MPPohBgwZFly5dYv369TFnzpy45557kim9IhRfAAAAANuVlStXxiWXXBIdOnSIN954I4qLi2PatGmxzz77ZDtaxlnjCwAAAGA7MXr06Lj22mujdevWMXny5DjuuOOyHalaWeMLAAAAYDuRm5sbDRs2jD59+kSdOnU2ed5DDz1Ug6mqjxlfAAAAANuJAQMGRE5OTrZj1BgzvgAAAABIksXtAQAAAEiS4gsAAACAJCm+AAAAAEiS4gsAAACAJCm+AAAAAEiS4gsAAACAJCm+AAAAAEiS4gsAAACAJP3/AO9GARjvjS43AAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 1500x800 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "mutual_info.sort_values(ascending  = False).plot.bar (figsize = (15,8))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e30fdbbf",
   "metadata": {},
   "source": [
    "# Mutual Gain"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "b12463a2",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.feature_selection import SelectKBest"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "cba578ed",
   "metadata": {},
   "outputs": [],
   "source": [
    "#k is the number of features we are selecting from the datatset\n",
    "selected_col = SelectKBest(ms, k=5)\n",
    "selected_col.fit(X_train, y_train)\n",
    "final_col = X_train.columns[selected_col.get_support()]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "45383016",
   "metadata": {},
   "source": [
    "### Difference Between Information Gain And Mutual Information\n",
    "\n",
    "I(X ; Y) = H(X) – H(X | Y) and IG(S, a) = H(S) – H(S | a)\n",
    "\n",
    "As such, mutual information is sometimes used as a synonym for information gain. Technically, they calculate the same quantity if applied to the same data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "f7ada794",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['Flavanoids', 'Color.int', 'Hue', 'OD', 'Proline'], dtype='object')"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "final_col"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "dec42d45",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Flavanoids</th>\n",
       "      <th>Color.int</th>\n",
       "      <th>Hue</th>\n",
       "      <th>OD</th>\n",
       "      <th>Proline</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>3.06</td>\n",
       "      <td>5.64</td>\n",
       "      <td>1.04</td>\n",
       "      <td>3.92</td>\n",
       "      <td>1065</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2.76</td>\n",
       "      <td>4.38</td>\n",
       "      <td>1.05</td>\n",
       "      <td>3.40</td>\n",
       "      <td>1050</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3.24</td>\n",
       "      <td>5.68</td>\n",
       "      <td>1.03</td>\n",
       "      <td>3.17</td>\n",
       "      <td>1185</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3.49</td>\n",
       "      <td>7.80</td>\n",
       "      <td>0.86</td>\n",
       "      <td>3.45</td>\n",
       "      <td>1480</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2.69</td>\n",
       "      <td>4.32</td>\n",
       "      <td>1.04</td>\n",
       "      <td>2.93</td>\n",
       "      <td>735</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Flavanoids  Color.int   Hue    OD  Proline\n",
       "0        3.06       5.64  1.04  3.92     1065\n",
       "1        2.76       4.38  1.05  3.40     1050\n",
       "2        3.24       5.68  1.03  3.17     1185\n",
       "3        3.49       7.80  0.86  3.45     1480\n",
       "4        2.69       4.32  1.04  2.93      735"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[final_col].head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "89139d48",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Wine</th>\n",
       "      <th>Alcohol</th>\n",
       "      <th>Malic.acid</th>\n",
       "      <th>Ash</th>\n",
       "      <th>Acl</th>\n",
       "      <th>Mg</th>\n",
       "      <th>Phenols</th>\n",
       "      <th>Flavanoids</th>\n",
       "      <th>Nonflavanoid.phenols</th>\n",
       "      <th>Proanth</th>\n",
       "      <th>Color.int</th>\n",
       "      <th>Hue</th>\n",
       "      <th>OD</th>\n",
       "      <th>Proline</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>14.23</td>\n",
       "      <td>1.71</td>\n",
       "      <td>2.43</td>\n",
       "      <td>15.6</td>\n",
       "      <td>127</td>\n",
       "      <td>2.80</td>\n",
       "      <td>3.06</td>\n",
       "      <td>0.28</td>\n",
       "      <td>2.29</td>\n",
       "      <td>5.64</td>\n",
       "      <td>1.04</td>\n",
       "      <td>3.92</td>\n",
       "      <td>1065</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>13.20</td>\n",
       "      <td>1.78</td>\n",
       "      <td>2.14</td>\n",
       "      <td>11.2</td>\n",
       "      <td>100</td>\n",
       "      <td>2.65</td>\n",
       "      <td>2.76</td>\n",
       "      <td>0.26</td>\n",
       "      <td>1.28</td>\n",
       "      <td>4.38</td>\n",
       "      <td>1.05</td>\n",
       "      <td>3.40</td>\n",
       "      <td>1050</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>13.16</td>\n",
       "      <td>2.36</td>\n",
       "      <td>2.67</td>\n",
       "      <td>18.6</td>\n",
       "      <td>101</td>\n",
       "      <td>2.80</td>\n",
       "      <td>3.24</td>\n",
       "      <td>0.30</td>\n",
       "      <td>2.81</td>\n",
       "      <td>5.68</td>\n",
       "      <td>1.03</td>\n",
       "      <td>3.17</td>\n",
       "      <td>1185</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>14.37</td>\n",
       "      <td>1.95</td>\n",
       "      <td>2.50</td>\n",
       "      <td>16.8</td>\n",
       "      <td>113</td>\n",
       "      <td>3.85</td>\n",
       "      <td>3.49</td>\n",
       "      <td>0.24</td>\n",
       "      <td>2.18</td>\n",
       "      <td>7.80</td>\n",
       "      <td>0.86</td>\n",
       "      <td>3.45</td>\n",
       "      <td>1480</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>13.24</td>\n",
       "      <td>2.59</td>\n",
       "      <td>2.87</td>\n",
       "      <td>21.0</td>\n",
       "      <td>118</td>\n",
       "      <td>2.80</td>\n",
       "      <td>2.69</td>\n",
       "      <td>0.39</td>\n",
       "      <td>1.82</td>\n",
       "      <td>4.32</td>\n",
       "      <td>1.04</td>\n",
       "      <td>2.93</td>\n",
       "      <td>735</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Wine  Alcohol  Malic.acid   Ash   Acl   Mg  Phenols  Flavanoids  \\\n",
       "0     1    14.23        1.71  2.43  15.6  127     2.80        3.06   \n",
       "1     1    13.20        1.78  2.14  11.2  100     2.65        2.76   \n",
       "2     1    13.16        2.36  2.67  18.6  101     2.80        3.24   \n",
       "3     1    14.37        1.95  2.50  16.8  113     3.85        3.49   \n",
       "4     1    13.24        2.59  2.87  21.0  118     2.80        2.69   \n",
       "\n",
       "   Nonflavanoid.phenols  Proanth  Color.int   Hue    OD  Proline  \n",
       "0                  0.28     2.29       5.64  1.04  3.92     1065  \n",
       "1                  0.26     1.28       4.38  1.05  3.40     1050  \n",
       "2                  0.30     2.81       5.68  1.03  3.17     1185  \n",
       "3                  0.24     2.18       7.80  0.86  3.45     1480  \n",
       "4                  0.39     1.82       4.32  1.04  2.93      735  "
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7a495dcc",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}