fraud_ml_easy_ensemble.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## import libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# import necessary libraries\n",
    "\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import missingno as msno \n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt \n",
    "\n",
    "#sklearn\n",
    "\n",
    "from sklearn.model_selection import train_test_split, cross_val_score, GridSearchCV, RepeatedStratifiedKFold, RandomizedSearchCV\n",
    "from sklearn.linear_model import LogisticRegression, SGDClassifier\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "from sklearn.metrics import f1_score, fbeta_score\n",
    "from sklearn.metrics import precision_score, recall_score, roc_auc_score, roc_curve, make_scorer\n",
    "from sklearn.metrics import confusion_matrix, classification_report\n",
    "from sklearn.metrics import accuracy_score, balanced_accuracy_score\n",
    "\n",
    "#imbalanced learning\n",
    "from imblearn.ensemble import EasyEnsembleClassifier\n",
    "\n",
    "# import custom functions\n",
    "from custom_functions import get_data_summary, our_metrics, eval_metrics, class_metrics_var_threshold#, evaluate_model\n",
    "\n",
    "# import custom scorers\n",
    "from custom_functions import get_f15, get_f2, get_f3\n",
    "\n",
    "\n",
    "\n",
    "# import warnings\n",
    "# warnings.filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# for reproducibility sake\n",
    "RSEED = 42"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Make scorer Fbeta"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# make the Fbeta scorers needed for the grid search\n",
    "f15_scorer = get_f15()\n",
    "ftwo_scorer = get_f2()\n",
    "fthree_scorer = get_f3()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Import test and train data, target"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# # Don't use storemagic (%store -r <filename>) for lare Dataframes /Arrays, it will keep crashing!\n",
    "# # %store shows all stored variables. \n",
    "# # %store -z deletes all stored variables from memory"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# reading in the whole data set\n",
    "X_train_tree = pd.read_csv('data/X_train_tree.csv', delimiter=',')\n",
    "X_test_tree = pd.read_csv('data/X_test_tree.csv', delimiter=',')\n",
    "y_test_tree = pd.read_csv('data/y_test_tree.csv', delimiter=',')\n",
    "y_train_tree = pd.read_csv('data/y_train_tree.csv', delimiter=',')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# reading in the partial data sets\n",
    "X_train_tree_large = pd.read_csv('data/X_train_tree_large.csv', delimiter=',')#\n",
    "y_train_tree_large = pd.read_csv('data/y_train_tree_large.csv', delimiter=',')#\n",
    "X_train_tree_small = pd.read_csv('data/X_train_tree_small.csv', delimiter=',')#\n",
    "y_train_tree_small = pd.read_csv('data/y_train_tree_small.csv', delimiter=',')#"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# change format of y from dataframe to array\n",
    "y_test_tree = np.ravel(y_test_tree)\n",
    "y_train_tree = np.ravel(y_train_tree)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 0, 0, ..., 0, 0, 0])"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_test_tree"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## EasyEnsembleClassifier\n",
    "\n",
    "### What is the EasyEnsembleClassifier?\n",
    "\n",
    "* Machine Learning Model for imbalanced data\n",
    "* Bag of balanced boosted learners.\n",
    "* Ensemble of AdaBoost learners trained on different balanced bootstrap samples. The balancing is achieved by random under-sampling.\n",
    "\n",
    "### AdaBoostClassifier\n",
    "\n",
    "[scikit-learn documentation](https://scikit-learn.org/stable/modules/ensemble.html#adaboost)\n",
    "The core principle of AdaBoost is to fit a sequence of weak learners (i.e., models that are only slightly better than random guessing, such as small decision trees) on repeatedly modified versions of the data. The predictions from all of them are then combined through a weighted majority vote (or sum) to produce the final prediction. The data modifications at each so-called boosting iteration consist of applying weights , , …,  to each of the training samples. Initially, those weights are all set to , so that the first step simply trains a weak learner on the original data. For each successive iteration, the sample weights are individually modified and the learning algorithm is reapplied to the reweighted data. At a given step, those training examples that were incorrectly predicted by the boosted model induced at the previous step have their weights increased, whereas the weights are decreased for those that were predicted correctly. As iterations proceed, examples that are difficult to predict receive ever-increasing influence. Each subsequent weak learner is thereby forced to concentrate on the examples that are missed by the previous ones in the sequence.\n",
    "\n",
    " By default, weak learners are decision stumps. Different weak learners can be specified through the estimator parameter. The main parameters to tune to obtain good results are n_estimators and the complexity of the base estimators (e.g., its depth max_depth or minimum required number of samples to consider a split min_samples_split).\n",
    "\n",
    "The individual learners can be weak, but as long as the performance of each one is slightly better than random guessing, the final model can be proven to converge to a strong learner."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Single EnsembleClassifier\n",
    "\n",
    "training an EasyEnsemble Classifier with the whole X_train with default setting, without hyperparameter tuning."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# initiate EasyEnsembleClassifier\n",
    "eec = EasyEnsembleClassifier(random_state=RSEED)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# no more NaN values since cleaned out in cleaning NB\n",
    "# X_train_tree.isnull().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>member_years</th>\n",
       "      <th>index_change_month</th>\n",
       "      <th>quantity_counters</th>\n",
       "      <th>client_catg_11</th>\n",
       "      <th>client_catg_12</th>\n",
       "      <th>client_catg_51</th>\n",
       "      <th>region_101</th>\n",
       "      <th>region_103</th>\n",
       "      <th>region_104</th>\n",
       "      <th>region_105</th>\n",
       "      <th>...</th>\n",
       "      <th>counter_statue_2.0</th>\n",
       "      <th>counter_statue_3.0</th>\n",
       "      <th>counter_statue_4.0</th>\n",
       "      <th>counter_statue_5.0</th>\n",
       "      <th>reading_remarque_6</th>\n",
       "      <th>reading_remarque_7</th>\n",
       "      <th>reading_remarque_8</th>\n",
       "      <th>reading_remarque_9</th>\n",
       "      <th>counter_type_ELEC</th>\n",
       "      <th>counter_type_GAZ</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>0 rows × 53 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "Empty DataFrame\n",
       "Columns: [member_years, index_change_month, quantity_counters, client_catg_11, client_catg_12, client_catg_51, region_101, region_103, region_104, region_105, region_106, region_107, region_199, region_206, region_301, region_302, region_303, region_304, region_305, region_306, region_307, region_308, region_309, region_310, region_311, region_312, region_313, region_371, region_372, region_379, region_399, tarif_type_0, tarif_type_10, tarif_type_11, tarif_type_12, tarif_type_13, tarif_type_14, tarif_type_15, tarif_type_29, tarif_type_40, tarif_type_45, counter_statue_0.0, counter_statue_1.0, counter_statue_2.0, counter_statue_3.0, counter_statue_4.0, counter_statue_5.0, reading_remarque_6, reading_remarque_7, reading_remarque_8, reading_remarque_9, counter_type_ELEC, counter_type_GAZ]\n",
       "Index: []\n",
       "\n",
       "[0 rows x 53 columns]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# X_train_tree.loc[X_train_tree.isnull().any(axis=1)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "# fit eec to train data\n",
    "eec.fit(X_train_tree, y_train_tree)\n",
    "# make prediction on test data\n",
    "y_pred_eec = eec.predict(X_test_tree)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred_train_eec = eec.predict(X_train_tree)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Performance on Train Set\n",
      "Model Metrics and Normalized Confusion Matrix\n",
      "_____________________\n",
      "_____________________\n",
      "Accuracy: 0.6084\n",
      "_____________________\n",
      "F1-score: 0.2033\n",
      "_____________________\n",
      "Fbeta_score with beta=1.5: 0.2752\n",
      "_____________________\n",
      "Fbeta_score with beta=2: 0.3431\n",
      "_____________________\n",
      "Fbeta_score with beta=3: 0.4451\n",
      "_____________________\n",
      "Recall 0.6335\n",
      "_____________________\n",
      "Specificity 0.6063\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgwAAAGdCAYAAAB+VCt0AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAApXElEQVR4nO3df3xU1Z3/8fdMSCaAEgIxP8DwW34JBAwQg1JLjQZtEdyquKuAqUKlLF91FDVWEgFrRBRTVtxolALVKitb0X61kTYVLTWCBoNIEUR+RIWZkPAjEDGBzOwfrmPn3ISb0QmTpa/n43Efj3Lm3DNnbh/Iez7n3DsOv9/vFwAAwCk4Iz0BAADQ9hEYAACALQIDAACwRWAAAAC2CAwAAMAWgQEAANgiMAAAAFsEBgAAYIvAAAAAbLWL9AS+0XvEI5GeAtDmVGxIi/QUgDYpLia7Vcdv3+NfwzbW8coXwjZWJLWZwAAAQFvhcFCAN3FFAACALSoMAAAYHHyftiAwAABgYEnCisAAAICBwGDFFQEAALaoMAAAYHA4HJGeQptDYAAAwIICvIkrAgAAbFFhAADAwKZHKwIDAAAGAoMVVwQAANiiwgAAgIEnPVoRGAAAMLAkYcUVAQAAtqgwAABgoMJgRWAAAMBAYLAiMAAAYHCIR0ObiFAAAMAWFQYAAAwsSVgRGAAAMBAYrLgiAADAFhUGAAAMVBisCAwAAFgQGExcEQAAYIsKAwAABpYkrAgMAAAYCAxWXBEAAGCLCgMAAAYH36ctCAwAABhYkrAiMAAAYHA4+PEpExEKAADYosIAAICBJQkrAgMAAAY2PVpxRQAAgC0qDAAAGFiSsCIwAABgIDBYcUUAAIAtKgwAABjY9GhFYAAAwMSShAVXBAAA2CIwAABgcDicYTtCtXTpUvXq1UuxsbHKyMjQxo0bT9n/8OHDmjVrllJSUuRyudS/f3+9/vrr32vMphAYAAAwOByOsB2hWLVqldxut/Lz87Vp0yalpaUpOztbVVVVTfZvaGjQZZddpj179mj16tXavn27iouL1b179+88ZrPXxO/3+0M6o5X0HvFIpKcAtDkVG9IiPQWgTYqLyW7V8c8b+R9hG+uT92e3uG9GRoZGjRqlJ554QpLk8/mUmpqq2bNn695777X0Lyoq0qJFi/Txxx8rOjo6LGM2hwoDAACtqL6+XrW1tUFHfX29pV9DQ4PKy8uVlZUVaHM6ncrKylJZWVmTY7/66qvKzMzUrFmzlJSUpCFDhuihhx5SY2Pjdx6zOQQGAAAM4dzDUFBQoLi4uKCjoKDA8p7V1dVqbGxUUlJSUHtSUpI8Hk+T89y1a5dWr16txsZGvf7665o7d64ee+wxPfjgg995zOZwWyUAAKYQ9x6cSm5urtxud1Cby+UKy9g+n0+JiYl6+umnFRUVpfT0dH3xxRdatGiR8vPzw/Ie3yAwAADQilwuV4sCQkJCgqKiouT1eoPavV6vkpOTmzwnJSVF0dHRioqKCrQNGjRIHo9HDQ0N32nM5rAkAQCAyRnGo4ViYmKUnp6u0tLSQJvP51NpaakyMzObPOeiiy7Szp075fP5Am07duxQSkqKYmJivtOYzSEwAABgcjjCd4TA7XaruLhYK1as0LZt2zRz5kzV1dUpJydHkjR16lTl5uYG+s+cOVMHDx7Ubbfdph07dui1117TQw89pFmzZrV4zJZiSQIAgDZi8uTJOnDggPLy8uTxeDR8+HCVlJQENi1WVlbK6fz2u35qaqreeOMN3XHHHRo2bJi6d++u2267Tffcc0+Lx2wpnsMAtGE8hwFoWms/h6H/mKKwjbXjnVvDNlYkUWEAAMDEgr0FlwQAANiiwgAAgMEfxucwnCkIDAAAmMgLFgQGAABMThKDiT0MAADAFhUGAABM7GGwIDAAAGAiL1iwJAEAAGxRYQAAwMSmRwsCAwAAJvYwWLAkAQAAbFFhAADARIHBgsAAAICJPQwWLEkAAABbVBgAADBRYLAgMAAAYODXKq0IDAAAmNjDYMEeBgAAYIsKAwAAJgoMFgQGAABM7GGwYEkCAADYosIAAICJTY8WBAYAAEzkBQuWJAAAgC0qDAAAmNj0aEFgAADARGCwYEkCAADYosIAAICJr9MWBAYAAEwsSVgQGAAAMJEXLCi6AAAAW1QYAAAw+HnSowWB4Qw35boRmjFttM7p2lHbdlTpgYV/1uatnmb7n32WS3P+fayyf9RfcXGx2re/VvMf/YvWrd8lSRp9wbmaMXW0hgxOVtI5Z2nGHb/Xn9btPF0fBwibl154W88t/4tqqmt13oDuuiv3Gp0/tGeTfd/882b9pnitPv+sWidPNiq1xzm6Ydo4XTlhdKBPTXWtnnj8VW0o+1hHjx7XiPS+uiv3GvXomXi6PhLCiT0MFgSGM9iPLx+oX945Tvf/aq0qPtqvn/3bSK148jpdOukZ1Rz60tI/up1Tvy26TjUHv9Qv5rwiT9VRndstTrVHvwr0ad8+Wtt2VOm/XtmipxZffTo/DhA2fyrZpMJFL+veuZN1/rCeevG3b+n//fxJvfSH+9Wl69mW/p3iOihnxuXq1TtJ0dFRWv/WVi2Y+zvFdzlbmRcNkt/v15zbnlG7dlF6dMl0dewYq9+tfFP/Pn2pVq25T+07uCLwKYHwYg/DGeyWG0dq1e8/1OpXP9LOXTX65a/e0PGvTujaSUOb7H/tpGHq3ClWP3e/rPLNX+iL/bXaUP6Ztu04EOjz1t9267En12vtm5+cro8BhN3vVr6pST8dowlXX6g+fVN0b951im0foz+8/G6T/dNHnadxl6apd59knZt6jq6/8Yfq17+bNm/6uvJWufeAPvpwj+6Ze50GD+mpnr2TdM/c61Rff0Jv/LH8dH40hIsjjMcZgsBwhopu59SQQclav2FPoM3vl/62Ya8uGNatyXOyLumrDz7cp/n3Xqb3/jxLJS/l6Bc/u1BO1vJwBjlx4qQ+/vtnGnXhgECb0+nUqAsHaMvm3bbn+/1+bXx3u/buqdKI9L5fj9lwUpLkcn1btHU6nYqObhcIFfg/xukI33GGCHlJorq6WsuWLVNZWZk8nq/XwpOTkzVmzBjddNNNOuecc8I+SYQuPr6D2rVzqvpg8NJDdU2d+vbq0uQ5Pbp31rmj4rTmj39XzuzV6pkarwW5l6ldO6eWPP3O6Zg20OoOH6pTY6PPsvTQpevZ2rvb2+x5x44e148vnauGEycV5XTq7vuvVcaYgZKkXr2TlJwSr6WFf1Bu3vVq3yFGv1v5pqq8h1VdXduqnwc4XUIKDO+9956ys7PVoUMHZWVlqX///pIkr9erJUuW6OGHH9Ybb7yhkSNHnnKc+vp61dfXB7X5fSflcLKlIpKcToeqD36p+xa8IZ/Pr4+2eZWceJZmTB1NYMA/vQ4dXXpu9T06/mW93tuwQ4WL1qj7uQlKH3We2kVHaeHjN+vB/BeUdfG9iopyatSF/TXm4sHy+/2Rnjq+CzY9WoT0L/Ts2bN17bXXqqioSA7jYvr9ft16662aPXu2ysrKTjlOQUGB5s2bF9QWl5Sl+JTLQpkOTuHQoS918qRPCV06BLUndO2oAzV1TZ5TVV2nEycb5fN9+x+4nbtrlHjOWYpu59SJk75WnTNwOnSO76ioKKcO1hwNaj9Yc1Rdm9jw+A2n06nUHl9XUPsPPFe7d3m0/Jk/KX3UeZKkQef30POr79Gxo8d14sRJxXc5Wzn/9pgGDU5tvQ+D1kNesAhpD8PmzZt1xx13WMKCJDkcDt1xxx2qqKiwHSc3N1dHjhwJOjonjQtlKrBx4qRPH23z6KKMb28TczikMaN7atOH+5o85/2Kz9UrNT4oWPfu0UXeA8cICzhjREe308DBqXpvw45Am8/n0/vvbtfQtN4tHsfv8wf2Lvyjs85ur/guZ6tyb5W2ba3UD37U9CZj4P+akCoMycnJ2rhxowYOHNjk6xs3blRSUpLtOC6XSy5X8G1GLEeE3zPPva/H5l+pD//u0eb/va2yQ/torX5liyTpsQVXylN1TIv+421J0vMvVWjq5AuUf/elWvHCJvXqEa9ZN1+o5S98u8u7Q/to9UyND/w5tXtnDeqfqCO1x7XPE/yNDWir/m3qOM375XMadH6qzh/aUy/+dp2OH2/QTyZlSJLy7/utEhPjNOv2qyRJy59Zq0GDe+jc1AQ1nDipd/76d73+/9/TPfdfFxjzz298oPguZyk5OV47P9mnxQt/r0t+NEwXjhkUiY+I7+sM2qwYLiH9K33XXXdpxowZKi8v16WXXhoIB16vV6WlpSouLtajjz7aKhNF6F5b+7G6xreXe+bFSujaUdu2V+mmWS8FNkJ2S+4UtPyw33tU02a9pLl3/kh//K8ceaqO6je/K1fR8g2BPkMHJ+vFZ/418Oe5d/1IkrT61S2ak//H0/TJgO/nsvEX6NDBY3p66euqqa5V/4Hn6tdFM9U1oZMkybv/kJz/UGo7/mWDHvnVS6ryHpbLFa2evRM1v2CqLht/QaBPTXWtChe9rIM1R5VwTiddOWG0br41+7R/NoQJgcHC4Q9xR86qVav0+OOPq7y8XI2NjZKkqKgopaeny+1267rrrrMZoWm9Rzzync4DzmQVG9IiPQWgTYqLad0w1ueWl8I21q5nrg3bWJEU8jrA5MmTNXnyZJ04cULV1dWSpISEBEVHR4d9cgAAoG34zhsHoqOjlZKSEs65AADQNrAkYcFOQwAATDyHwYJHQwMAAFtUGAAAMLEkYUFgAADARP3dgksCAABsUWEAAMDEpkcLAgMAACb2MFiwJAEAAGxRYQAAwOBnScKCwAAAgIn6uwWBAQAAE3sYLMhQAADAFoEBAACTwxG+I0RLly5Vr169FBsbq4yMDG3cuLHZvsuXL5fD4Qg6YmNjg/rcdNNNlj7jx48PeV4sSQAAYIrQksSqVavkdrtVVFSkjIwMFRYWKjs7W9u3b1diYmKT53Tq1Enbt28P/NnRREgZP368fvOb3wT+7HK5Qp4bFQYAANqIxYsXa/r06crJydHgwYNVVFSkDh06aNmyZc2e43A4lJycHDiSkpIsfVwuV1Cf+Pj4kOdGYAAAwOQI31FfX6/a2tqgo76+3vKWDQ0NKi8vV1ZWVqDN6XQqKytLZWVlzU712LFj6tmzp1JTUzVx4kRt3brV0mfdunVKTEzUgAEDNHPmTNXU1IR8SQgMAAAY/E5H2I6CggLFxcUFHQUFBZb3rK6uVmNjo6VCkJSUJI/H0+Q8BwwYoGXLlumVV17Rc889J5/PpzFjxujzzz8P9Bk/frxWrlyp0tJSLVy4UG+99ZauuOIKNTY2hnRN2MMAAEArys3NldvtDmr7LnsImpKZmanMzMzAn8eMGaNBgwbpqaee0oIFCyRJ119/feD1oUOHatiwYerbt6/WrVunSy+9tMXvRWAAAMAUxk2PLperRQEhISFBUVFR8nq9Qe1er1fJyckteq/o6GiNGDFCO3fubLZPnz59lJCQoJ07d4YUGFiSAADAFIHbKmNiYpSenq7S0tJAm8/nU2lpaVAV4VQaGxu1ZcsWpaSkNNvn888/V01NzSn7NIXAAABAG+F2u1VcXKwVK1Zo27Ztmjlzpurq6pSTkyNJmjp1qnJzcwP958+fr7Vr12rXrl3atGmTbrzxRu3du1e33HKLpK83RM6ZM0fvvvuu9uzZo9LSUk2cOFH9+vVTdnZ2SHNjSQIAAFOEvk5PnjxZBw4cUF5enjwej4YPH66SkpLARsjKyko5nd9O7tChQ5o+fbo8Ho/i4+OVnp6ud955R4MHD5YkRUVF6cMPP9SKFSt0+PBhdevWTZdffrkWLFgQ8j4Kh9/v94fvo353vUc8EukpAG1OxYa0SE8BaJPiYkL7dhyqXvklYRtrz7zQn6rYFlFhAADAxI9PWbCHAQAA2KLCAACAiQqDBYEBAACD/zv8yuSZjiUJAABgiwoDAAAmvk5bEBgAADCxJGFBhgIAALaoMAAAYOIuCQsCAwAAJgKDBUsSAADAFhUGAABMFBgsCAwAABj8LElYEBgAADBxW6UFexgAAIAtKgwAAJhYkrAgMAAAYCIvWLAkAQAAbFFhAADA4OTrtAWBAQAAAzdJWJGhAACALSoMAAAYqDBYERgAADA4SAwWBAYAAAzkBSv2MAAAAFtUGAAAMFBhsCIwAABgcFB/t+CSAAAAW1QYAAAwsCRhRWAAAMDAj1VasSQBAABsUWEAAMDAkoQVgQEAAAOBwYolCQAAYIsKAwAABn5LworAAACAgQc3WREYAAAwUGCwIkMBAABbVBgAADBQYbAiMAAAYCAwWLEkAQAAbFFhAADAwG9JWBEYAAAwsCRhxZIEAACwRYUBAAADFQYrAgMAAAYHmxgsWJIAAAC2qDAAAGBgScKKwAAAgIHAYEVgAADAQGCwYg8DAACwRYUBAAADN0lYERgAADCwJGHFkgQAALBFhQEAAIODr9MWBAYAAAwsSViRoQAAgC0qDAAAGByUGCyoMAAAYHA4wneEaunSperVq5diY2OVkZGhjRs3Ntt3+fLlcjgcQUdsbGxQH7/fr7y8PKWkpKh9+/bKysrSJ598EvK8CAwAALQRq1atktvtVn5+vjZt2qS0tDRlZ2erqqqq2XM6deqk/fv3B469e/cGvf7II49oyZIlKioq0oYNG9SxY0dlZ2frq6++CmluBAYAAAyRqjAsXrxY06dPV05OjgYPHqyioiJ16NBBy5YtO8VcHUpOTg4cSUlJgdf8fr8KCwt1//33a+LEiRo2bJhWrlypffv2ac2aNSHNjcAAAIAhnIGhvr5etbW1QUd9fb3lPRsaGlReXq6srKxAm9PpVFZWlsrKypqd67Fjx9SzZ0+lpqZq4sSJ2rp1a+C13bt3y+PxBI0ZFxenjIyMU47ZlDaz6XH3B5MiPQWgzWnfIz/SUwDapOOV2a06fjgfDV1QUKB58+YFteXn5+uBBx4IaquurlZjY2NQhUCSkpKS9PHHHzc59oABA7Rs2TINGzZMR44c0aOPPqoxY8Zo69atOvfcc+XxeAJjmGN+81pLtZnAAADAmSg3N1dutzuozeVyhWXszMxMZWZmBv48ZswYDRo0SE899ZQWLFgQlvf4BoEBAABDOCsMLperRQEhISFBUVFR8nq9Qe1er1fJyckteq/o6GiNGDFCO3fulKTAeV6vVykpKUFjDh8+vIWf4GvsYQAAwOB0+MN2tFRMTIzS09NVWloaaPP5fCotLQ2qIpxKY2OjtmzZEggHvXv3VnJyctCYtbW12rBhQ4vH/AYVBgAA2gi3261p06Zp5MiRGj16tAoLC1VXV6ecnBxJ0tSpU9W9e3cVFBRIkubPn68LL7xQ/fr10+HDh7Vo0SLt3btXt9xyi6Sv76C4/fbb9eCDD+q8885T7969NXfuXHXr1k2TJk0KaW4EBgAADOFckgjF5MmTdeDAAeXl5cnj8Wj48OEqKSkJbFqsrKyU0/nt4sChQ4c0ffp0eTwexcfHKz09Xe+8844GDx4c6HP33Xerrq5OM2bM0OHDh3XxxRerpKTE8oAnOw6/39/yekmr2hHpCQBtDndJAE07XvlCq47/47XrwzbWa5dfHLaxIok9DAAAwBZLEgAAGELZrPjPgsAAAIAhUnsY2jKWJAAAgC0qDAAAGPg2bUVgAADAwJKEFYEBAACDg02PFlRdAACALSoMAAAYWJKwIjAAAGCg/G7FNQEAALaoMAAAYOBJj1YEBgAADOxhsGJJAgAA2KLCAACAgW/TVgQGAAAMLElYEaIAAIAtKgwAABi4S8KKwAAAgIElCSsCAwAABtbrrbgmAADAFhUGAAAM7GGwIjAAAGBgD4MVSxIAAMAWFQYAAAxUGKwIDAAAGCi/W3FNAACALSoMAAAYuEvCisAAAICBPQxWLEkAAABbVBgAADDwbdqKwAAAgIElCSsCAwAABgebHi2ougAAAFtUGAAAMLAkYUVgAADAQPndimsCAABsUWEAAMDAkx6tCAwAABjYw2DFkgQAALBFhQEAAAMVBisCAwAAhqhIT6ANYkkCAADYosIAAICBuySsCAwAABjYw2BFYAAAwEBgsGIPAwAAsEWFAQAAQxQVBgsCAwAABpYkrFiSAAAAtqgwAABg4LZKKwIDAAAGliSsWJIAAAC2qDAAAGDgtySsCAwAABhYkrBiSQIAANiiwgAAgIG7JKwIDAAAGHjSoxWBAQAAA3sYrNjDAABAG7J06VL16tVLsbGxysjI0MaNG1t03osvviiHw6FJkyYFtd90001yOBxBx/jx40OeF4EBAACD0xG+IxSrVq2S2+1Wfn6+Nm3apLS0NGVnZ6uqquqU5+3Zs0d33XWXxo4d2+Tr48eP1/79+wPHCy+8ENrERGAAAMAiUoFh8eLFmj59unJycjR48GAVFRWpQ4cOWrZsWbPnNDY26oYbbtC8efPUp0+fJvu4XC4lJycHjvj4+NAmJgIDAACtqr6+XrW1tUFHfX29pV9DQ4PKy8uVlZUVaHM6ncrKylJZWVmz48+fP1+JiYm6+eabm+2zbt06JSYmasCAAZo5c6ZqampC/hwEBgAADFEOf9iOgoICxcXFBR0FBQWW96yurlZjY6OSkpKC2pOSkuTxeJqc5/r16/Xss8+quLi42c8yfvx4rVy5UqWlpVq4cKHeeustXXHFFWpsbAzpmnCXBAAAhnB+m87NzZXb7Q5qc7lc33vco0ePasqUKSouLlZCQkKz/a6//vrA/x46dKiGDRumvn37at26dbr00ktb/H4EBgAAWpHL5WpRQEhISFBUVJS8Xm9Qu9frVXJysqX/p59+qj179mjChAmBNp/PJ0lq166dtm/frr59+1rO69OnjxISErRz586QAgNLEgAAGCKx6TEmJkbp6ekqLS0NtPl8PpWWliozM9PSf+DAgdqyZYsqKioCx1VXXaVx48apoqJCqampTb7P559/rpqaGqWkpIR0TagwAABgiNSDm9xut6ZNm6aRI0dq9OjRKiwsVF1dnXJyciRJU6dOVffu3VVQUKDY2FgNGTIk6PzOnTtLUqD92LFjmjdvnn76058qOTlZn376qe6++27169dP2dnZIc2NwAAAQBsxefJkHThwQHl5efJ4PBo+fLhKSkoCGyErKyvldLZ8cSAqKkoffvihVqxYocOHD6tbt266/PLLtWDBgpD3UTj8fn8b+YWNHZGeANDmtO+RH+kpAG3S8crQHzwUipf3/DFsY13d64qwjRVJVBgAADDwWxJWBAYAAAwEBivukgAAALaoMAAAYKDCYEVgAADAEEVgsGBJAgAA2KLCAACAweloI08caEMIDAAAGCi/W3FNAACALSoMAAAYuEvCisBwhnv++df07LO/14EDhzRwYG/NnftzDRvWv8m+a9e+o6Kil1RZuV8nT55Uz57dlJMzSZMm/SjQZ8CACU2eO2dOjm655V9a5TMAreHnUy/THT+foKRz4rRlW6Xcecv1/uZPm+0f16mDHpgzWROvGKUucWep8otqzZm3Um+8WSFJmn5jlqZPuUw9z02QJG3b8bke+vXvtXbd5tPxcRBm3CVhRWA4g73++l9VUPCM5s2bpbS0/lqx4lXdfHOeSkqK1LVrZ0v/uLizNXPmderT51xFR7fTm2++p/vu+7W6du2ssWMvkCStX78y6Jy33y7XL3+5RNnZY07HRwLC4poJF2rh3Cmafd+zeq9ip/795iv06nP3Ku2Hd+pATa2lf3R0lF57/j5VVdfqhlsL9YXnoHp0P0dHausCfb7wHNTch1/Qzt0eORzSjdf8QC89c5cuvDJX23Z8fjo/HtAq+PGpM9i1196poUPPU17erZK+/l31Sy7J0ZQpP9GMGde2aIyrr75Nl1wySrfffmOTr//iFw+qru64Vqz4VdjmjW/x41Ot4+1XFqh886e6I2+5JMnhcGjnhif0n8vf0KNPvmrpf8uNWbrj5z9R2rg7dfJkY4vf54sPi3Xfr57XilXrwjRzfKO1f3zqr57XwjbW2OQfh22sSGLT4xmqoeGEtm7dqTFj0gJtTqdTY8YM1wcfbLc93+/3q6xss3bv/kKjRp3fZJ/q6kN66633dc01l4Vt3kBri46O0oihvfWX9R8F2vx+v/6y/iONvuC8Js/5cdYF2lD+iQofzNGe8iK9/6dHNGfWRDmbWeh2Oh26dkKmOrZ3acOmT1rlc6B1OR3hO84ULEmcoQ4dqlVjo09du8YHtXft2lm7djVfHj16tE4/+MFNamg4IafTqfz8mbroohFN9n355b+oY8f2uvxyliPwf0dCl05q1y5KVdVHgtqrqo9oQN9uTZ7Tu0eifjjmHL245m+6+qaF6tsrSYUP/kzR0e30UOF/B/qdPyBV69bMV6wrWsfqvtLkGYv18SdftOrnQes4k/6hD5ewB4bPPvtM+fn5WrZsWbN96uvrVV9fH9TmcjXI5YoJ93QQoo4d22vNml/ryy+/UlnZZj388LNKTU1WRsZQS9///u8/acKEH/L/G854TqdTB2pqNeveYvl8fn2wZbe6JXXR7bf+JCgw7Ni1Txnj71Vcpw66+soMFS+eqcuvm09owBkh7EsSBw8e1IoVK07Zp6CgQHFxcUFHQcFT4Z7KP7X4+E6KinKqpuZQUHtNzWElJMQ3c9bX/2Hs2bObBg3qo5/97GplZ4/R00+/ZOn3/vtbtXv3F7r22svDPnegNVUfrNXJk41KTIgLak9MiJPnwOEmz/FUHdYnu/fL5/t2y9fHO79QSmK8oqOjAm0nTjRq116vPtiyW3kLX9SWbXs162fjW+VzoHU5w3icKUKuMLz6qnVD0D/atWuX7Ri5ublyu91BbS5XZahTwSnExETr/PP7qazsQ2VlZUr6etNjWdlm3Xhjyzfg+Hx+NTScsLSvXr1W55/fTwMH9g7bnIHT4cSJRn2wZbfGXTREf1j7vqSvNz2Ou+h8Fa1Y2+Q5Ze9v1+SJF8nhcOibfeLn9UnRfu8hnTjR/CZIp8MpV0x0+D8EWp2DJQmLkAPDpEmTgv7SNMVhc6VdLpdcLpfRSlk73HJyJumeex7XkCH9NGxYf61Y8YqOH/9K//IvWZKku+9erKSkrrrzzmmSpKeeeklDhvRTjx4pamg4obfeel+vvvqmHnhgZtC4x459qZKSv+mee24+7Z8JCIclz7ym4sdmqnzLLr3/v7dVdujg0sr/ekuS9MzjM7XPc0h5C1+UJBX/9k+6ddrleuyBaXpyeYn69U7RnFmT9ORvSgJjzr/ner3xZoU+21etszu21+RJF+kHmYM0YcrDEfmMQLiFHBhSUlL05JNPauLEiU2+XlFRofT09O89MXx/V145VgcPHtGSJc/rwIFDGjSoj555Zl5gSWL//gNBu7y//PIrzZv3n/J4ahQbG6M+fc7VokV36sorxwaN+9prb8vv9+snP/nBaf08QLis/sO7SujSSXnua5R0Tmd9+Pe9mjjl4cBGyNRuCUHLD5/vP6irpjysR/Km6L03Fmqf95CWLvujHvvPbyuu53TtpGcf/4WSEzvryNEv9dHHlZow5WH95a9bTvvnw/dHgcEq5OcwXHXVVRo+fLjmz5/f5OubN2/WiBEj5PP5QpwKz2EATDyHAWhaaz+H4f3q8D2HYWTCmfEchpArDHPmzFFdXV2zr/fr109vvvnm95oUAABoW0IODGPHjj3l6x07dtQll1zynScEAECknUl3N4QLD24CAMDgcLSRX01oQwhRAADAFhUGAAAM3CVhRWAAAMDAg5usCAwAABjIC1bsYQAAALaoMAAAYODnra0IDAAAGMgLVixJAAAAW1QYAAAwcJeEFYEBAAADecGKJQkAAGCLCgMAAAYqDFYEBgAADNxWacWSBAAAsEWFAQAAAwUGKwIDAAAGh8Mf6Sm0OQQGAAAMVBis2MMAAABsUWEAAMDAkx6tCAwAABgov1txTQAAgC0qDAAAGFiSsCIwAABgIC9YsSQBAABsUWEAAMDAkoQVgQEAAAN5wYolCQAAYIsKAwAABn7e2orAAACAgbxgRWAAAMDAr1VasYcBAADYosIAAICBJQkrAgMAAAaew2DFkgQAALBFYAAAwOAI4xGqpUuXqlevXoqNjVVGRoY2btzYovNefPFFORwOTZo0Kajd7/crLy9PKSkpat++vbKysvTJJ5+EPC8CAwAABmcYj1CsWrVKbrdb+fn52rRpk9LS0pSdna2qqqpTnrdnzx7dddddGjt2rOW1Rx55REuWLFFRUZE2bNigjh07Kjs7W1999VVIcyMwAADQRixevFjTp09XTk6OBg8erKKiInXo0EHLli1r9pzGxkbdcMMNmjdvnvr06RP0mt/vV2Fhoe6//35NnDhRw4YN08qVK7Vv3z6tWbMmpLkRGAAAMDgc4Tvq6+tVW1sbdNTX11ves6GhQeXl5crKygq0OZ1OZWVlqaysrNm5zp8/X4mJibr55pstr+3evVsejydozLi4OGVkZJxyzKYQGAAAsAjfLoaCggLFxcUFHQUFBZZ3rK6uVmNjo5KSkoLak5KS5PF4mpzl+vXr9eyzz6q4uLjJ1785L5Qxm8NtlQAAtKLc3Fy53e6gNpfL9b3HPXr0qKZMmaLi4mIlJCR87/HsEBgAADA4wvjoJpfL1aKAkJCQoKioKHm93qB2r9er5ORkS/9PP/1Ue/bs0YQJEwJtPp9PktSuXTtt3749cJ7X61VKSkrQmMOHDw/pc7AkAQCAweFwhu1oqZiYGKWnp6u0tDTQ5vP5VFpaqszMTEv/gQMHasuWLaqoqAgcV111lcaNG6eKigqlpqaqd+/eSk5ODhqztrZWGzZsaHLMU6HCAACARWQe9eh2uzVt2jSNHDlSo0ePVmFhoerq6pSTkyNJmjp1qrp3766CggLFxsZqyJAhQed37txZkoLab7/9dj344IM677zz1Lt3b82dO1fdunWzPK/BDoEBAIA2YvLkyTpw4IDy8vLk8Xg0fPhwlZSUBDYtVlZWyukMbXHg7rvvVl1dnWbMmKHDhw/r4osvVklJiWJjY0Max+H3+9vIb3juiPQEgDanfY/8SE8BaJOOV77QquMfaSgJ21hxMePDNlYkUWEAAMCCX58ysekRAADYosIAAIAhlLsb/lkQGAAAsGBJwkSEAgAAtqgwAABgCOeTHs8UBAYAAAwEBiuWJAAAgC0qDAAAWPB92kRgAADA4HCwJGEiMAAAYEFgMFFzAQAAtqgwAABg4C4JKwIDAAAWFOBNXBEAAGCLCgMAAAaWJKwIDAAAGLit0oolCQAAYIsKAwAAFlQYTAQGAAAMDgrwFlwRAABgiwoDAAAWLEmYCAwAABi4S8KKwAAAgAWBwcQeBgAAYIsKAwAABu6SsCIwAABgwZKEiQgFAABsUWEAAMDAj09ZERgAADBwW6UVSxIAAMAWFQYAACz4Pm0iMAAAYGAPgxURCgAA2KLCAACABRUGE4EBAAADd0lYERgAALBgxd7EFQEAALaoMAAAYOAuCSuH3+/3R3oSaDvq6+tVUFCg3NxcuVyuSE8HaBP4ewEQGGCora1VXFycjhw5ok6dOkV6OkCbwN8LgD0MAACgBQgMAADAFoEBAADYIjAgiMvlUn5+Phu7gH/A3wuATY8AAKAFqDAAAABbBAYAAGCLwAAAAGwRGAAAgC0CAwKWLl2qXr16KTY2VhkZGdq4cWOkpwRE1Ntvv60JEyaoW7ducjgcWrNmTaSnBEQMgQGSpFWrVsntdis/P1+bNm1SWlqasrOzVVVVFempARFTV1entLQ0LV26NNJTASKO2yohScrIyNCoUaP0xBNPSJJ8Pp9SU1M1e/Zs3XvvvRGeHRB5DodDL7/8siZNmhTpqQARQYUBamhoUHl5ubKysgJtTqdTWVlZKisri+DMAABtBYEBqq6uVmNjo5KSkoLak5KS5PF4IjQrAEBbQmAAAAC2CAxQQkKCoqKi5PV6g9q9Xq+Sk5MjNCsAQFtCYIBiYmKUnp6u0tLSQJvP51NpaakyMzMjODMAQFvRLtITQNvgdrs1bdo0jRw5UqNHj1ZhYaHq6uqUk5MT6akBEXPs2DHt3Lkz8Ofdu3eroqJCXbp0UY8ePSI4M+D047ZKBDzxxBNatGiRPB6Phg8friVLligjIyPS0wIiZt26dRo3bpylfdq0aVq+fPnpnxAQQQQGAABgiz0MAADAFoEBAADYIjAAAABbBAYAAGCLwAAAAGwRGAAAgC0CAwAAsEVgAAAAtggMAADAFoEBAADYIjAAAABbBAYAAGDrfwAa/gCxMwkTvQAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "print('Performance on Train Set')\n",
    "our_metrics(y_train_tree, y_pred_train_eec)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model Metrics and Normalized Confusion Matrix\n",
      "_____________________\n",
      "_____________________\n",
      "Accuracy: 0.6089\n",
      "_____________________\n",
      "F1-score: 0.204\n",
      "_____________________\n",
      "Fbeta_score with beta=1.5: 0.2761\n",
      "_____________________\n",
      "Fbeta_score with beta=2: 0.3442\n",
      "_____________________\n",
      "Fbeta_score with beta=3: 0.4465\n",
      "_____________________\n",
      "Recall 0.6353\n",
      "_____________________\n",
      "Specificity 0.6067\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgwAAAGdCAYAAAB+VCt0AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAph0lEQVR4nO3de3hU1b3/8c9MSCYESwhEkoCBEFAEuYQGiEFti0SD7UE4tYg/L0CKWCmllRHR9EgilxoVxRx+4olGEahWOdJTtD9tvKRivUSiQS5aBJFLqjATEi6BiAlk5veHx7GzdsLO6MBE+n49z34eWbP2mjX7eZDPfNfaexx+v98vAACAk3BGegIAAKD9IzAAAABbBAYAAGCLwAAAAGwRGAAAgC0CAwAAsEVgAAAAtggMAADAFoEBAADY6hDpCXwlfdj9kZ4C0O68v35QpKcAtEvxMWNP6fgde/2fsI11rPrpsI0VSe0mMAAA0F44HBTgTVwRAABgiwoDAAAGB9+nLQgMAAAYWJKwIjAAAGAgMFhxRQAAgC0qDAAAGBwOR6Sn0O4QGAAAsKAAb+KKAAAAW1QYAAAwsOnRisAAAICBwGDFFQEAALaoMAAAYOBJj1YEBgAADCxJWHFFAACALSoMAAAYqDBYERgAADAQGKwIDAAAGBzi0dAmIhQAALBFhQEAAANLElYEBgAADAQGK64IAACwRYUBAAADFQYrAgMAABYEBhNXBAAA2KLCAACAgSUJKwIDAAAGAoMVVwQAANiiwgAAgMHB92kLAgMAAAaWJKwIDAAAGBwOfnzKRIQCAAC2qDAAAGBgScKKwAAAgIFNj1ZcEQAAYIsKAwAABpYkrAgMAAAYCAxWXBEAAGCLCgMAAAY2PVoRGAAAMLEkYcEVAQAAtqgwAABgYNOjFYEBAAADvyVhRYQCAMDgkDNsR6iWLVumtLQ0xcbGKisrS5WVlSftf+jQIc2cOVMpKSlyuVw677zz9OKLL36rMVtCYAAAoJ1YvXq13G63CgsLtWHDBg0dOlS5ubmqqalpsX9TU5Muu+wy7d69W2vWrNG2bdtUWlqqnj17fuMxW+Pw+/3+b/XpwiR92P2RngLQ7ry/flCkpwC0S/ExY0/p+OeNWBa2sba/O7PNfbOysjRixAg99NBDkiSfz6fU1FTNmjVLd9xxh6V/SUmJFi9erI8++kjR0dFhGbM1VBgAADA5HGE7GhsbVV9fH3Q0NjZa3rKpqUlVVVXKyckJtDmdTuXk5KiioqLFaT7//PPKzs7WzJkzlZSUpEGDBunuu+9Wc3PzNx6zNQQGAABOoaKiIsXHxwcdRUVFln61tbVqbm5WUlJSUHtSUpI8Hk+LY+/cuVNr1qxRc3OzXnzxRc2bN08PPPCAFi1a9I3HbA13SQAAYArj1+n8/Hy53e6gNpfLFZaxfT6funfvrkcffVRRUVHKzMzUZ599psWLF6uwsDAs7/EVAgMAAKYw3lbpcrnaFBASExMVFRUlr9cb1O71epWcnNziOSkpKYqOjlZUVFSgbcCAAfJ4PGpqavpGY7aGJQkAANqBmJgYZWZmqry8PNDm8/lUXl6u7OzsFs+56KKLtGPHDvl8vkDb9u3blZKSopiYmG80ZmsIDAAAmMK46TEUbrdbpaWlWrlypbZu3aoZM2aooaFBeXl5kqTJkycrPz8/0H/GjBk6cOCAfvOb32j79u164YUXdPfdd2vmzJltHrOtWJIAAMAUoa/TkyZN0v79+1VQUCCPx6OMjAyVlZUFNi1WV1fL6fx6cqmpqXrppZc0e/ZsDRkyRD179tRvfvMb3X777W0es614DgPQjvEcBqBlp/w5DBeXhG2s7W/eHLaxIokKAwAABj+/JWFBYAAAwEResCAwAABgcpIYTNwlAQAAbFFhAADAxB4GCwIDAAAm8oIFSxIAAMAWFQYAAExserQgMAAAYGIPgwVLEgAAwBYVBgAATBQYLAgMAACY2MNgwZIEAACwRYUBAAATBQYLAgMAAAZ+rdKKwAAAgIk9DBbsYQAAALaoMAAAYKLAYEFgAADAxB4GC5YkAACALSoMAACY2PRoQWAAAMBEXrBgSQIAANiiwgAAgIlNjxYEBgAATAQGC5YkAACALSoMAACY+DptQWAAAMDEkoQFgQEAABN5wYKiCwAAsEWFAQAAg58nPVoQGM5wN1ydoelTRujsbp20dft+3XVvuTZ/6Gm1//fOcmnOry5W7qXnKj4+Vnv31Wvh/a9p3Zu7JEkjvn+Obpo8QoMGJinp7LP0i9lr9cq6Hafr4wBh8+zTb+jJFX9VXW29zu3fU3Pyr9IFg3u32Pe1VzfpidJX9Ok/anXiRLNSe52t66aM1o/HjQj0qaut10MP/lnrKz7SkSPHNCyzr+bkX6Vevbufro+EcGIPgwWB4Qz2k8v767e3/kjzfveqNn6wT3nXfl8rH/6ZciYsV93Bzy39ozs49fuSiao78Llm3va8PDVH1bNHZ9UfaQz0iesYra3ba/Tsc1tUsmTCafw0QPi8UrZBxYv/pDvmXa0LhqTpmd+v069/8V969s//oa7dvmfp3zk+Tnk3Xaa0PkmKju6gN1//QAvn/UEJXc9S9kUD5Pf7ddtvHleHDlG6f+mN6tQpVn9YtU6/mv6wVq/NV8c41+n/kECYsYfhDDbt+uFa/T9btOb5D7RjZ53u/N0rOvbFcU2cMKjF/hMnDFZ851j9wr1WVZv26rN99aqs+lQfbd8f6PP6W7u05OG39PJrVBXw3fWHVes04apRGvfvFyq9b7LuKLhasR1j9Oc/vdNi/8wR52r0mKHqk56sc1ITdc31P1K/83po04adkqTqPfv1webdun3eRA0c1Fu9+yTp9nkT1dh4XC/9ZcPp/GgIF0cYjzMEgeEMFd3BqUEDkvTW+j2BNr9femt9tYYN6dHiOTk/7Kv3N+/V/DvGqPLVGfrLs1P1y59nyclaHs4gx4+f0Ed//4dGXHheoM3pdGrEhedpy6bdtuf7/X5VvrNNe3bXaFhm3y/HbDohSXK5ooPGjI7uEAgV+I5xOsJ3nCFCXpKora3V8uXLVVFRIY/ny7Xw5ORkjRo1SlOnTtXZZ58d9kkidAkJHdWhg1O1BxqC2mvrGtQ3rWuL56T2jFf2iF567i9b9fNZ/6PeqV20ID9HHTo4tfTRitMxbeCUO3SwQc3NPsvSQ9du39OeXTWtnnf0yDH9ZEyBmo6fUJTTqbl3TlTWqPMlSWl9kpSckqBlxX9WfsEkdYyL0R9WrVON95Bqa+tP6ecBTpeQAsO7776r3NxcxcXFKScnR+ed92VC93q9Wrp0qe655x699NJLGj58+EnHaWxsVGNjY1Cb33dCDidbKiLJ6XSo7sDn+u3Cl+Xz+fXBVq+Su5+l6ZNHEBjwLy+uk0tPrpmrY5836t3121W8eK16ntNNmSPOVYfoKN374DQtKnxaORfnKyrqy4rFqIsHyO+P9MzxjbDp0SKkf6FnzZqliRMnqqSkRA7jYvr9ft18882aNWuWKipO/o9LUVGR5s+fH9TWJekyJaRcHsp0cBIHDx7TiRM+JXbtFNSe2K2T9tc1tHhOTW2DTpzwyef7+v9wO3YdUPezz1J0B6eOn/Cd0jkDp0OXhE6KinLqQN2RoPYDdUfUrYUNj19xOp1K7fVlBfW888/Rrp1erXjsVWWOOFeSNOCCVD21Zq6OHjmm48ebldD1LOVdu0QDBqaeug+DU4e8YBHSHoZNmzZp9uzZlrAgSQ6HQ7Nnz9bGjRttx8nPz9fhw4eDji5Jl4YyFdg4fsKnD7Z6NSqrV6DN4ZBGjeyl9zfvbfGcqo2fqXdql6Bg3adXgrz7jxIWcMaIju6g8wem6t312wNtPp9P772zXYOHprV5HL/PH9i78M/O+l5HJXQ9S9V7arT1w2r94NLB4Zg2EHEhVRiSk5NVWVmp888/v8XXKysrlZSUZDuOy+WSyxV8mxHLEeH3+JPv6f4FV2jL373a9ME+5V2bqbiO0Vrz3AeSpPsXXiFvzVEt/r9vSJKeenaTbpg0TAVzL9Wqp99XWq8E/XJallY8/fUu77iO0eqd2iXw59Se8Rpw3tk6XP+F9nqCv7EB7dW1k3+k+f/xlAZc0EsXDO6lZ37/uo4da9K/TciSJBX+9kl17x6vmbeMkySteOwVDRiYqnNSE9V0/ITefuPvevH/vavb77w6MOarL72vhK5nKTk5QTs+3qcl9/6PfnjpYF04quX/X6KdO4M2K4ZLSP9Kz5kzRzfddJOqqqo0ZsyYQDjwer0qLy9XaWmp7r///lMyUYTuhZe3qWtCnGbPuEiJ3eK0ddt+TZ25RrUHvnwGQ4/kzkHLD/u8RzR15hrdeetovfjfU+SpOaoVf9igkhWVgT6DBybr6ccmBf5855zRkqQ1z3+guYVlp+mTAd/OZWO/r4MHjurRZS+qrrZe551/jv6z5GZ1S+wsSfLuOyjnP5Xajn3epPt+96xqvIflckWrd5/uWlB0gy4b+/1An7raehUvXqsDdUeUeHZn/XjcCE27Ofe0fzaECYHBwuH3h7YlZ/Xq1XrwwQdVVVWl5uZmSVJUVJQyMzPldrt19dVX24zQsvRhBA3A9P76lp+ZAfyri48Ze0rHT7/x2bCNtfOxiWEbK5JCXgeYNGmSJk2apOPHj6u2tlaSlJiYqOjoaJszAQDAd9U33jgQHR2tlJSUcM4FAID2gSUJC3YaAgBg4jkMFjwaGgAA2KLCAACAiSUJCwIDAAAm6u8WXBIAAGCLCgMAACY2PVoQGAAAMLGHwYIlCQAAYIsKAwAABj9LEhYEBgAATNTfLQgMAACY2MNgQYYCAAC2qDAAAGBiD4MFFQYAAExOR/iOEC1btkxpaWmKjY1VVlaWKisrW+27YsUKORyOoCM2Njaoz9SpUy19xo4dG/K8qDAAANBOrF69Wm63WyUlJcrKylJxcbFyc3O1bds2de/evcVzOnfurG3btgX+7GihOjJ27Fg98cQTgT+7XK6Q50aFAQAAkyOMRwiWLFmi6dOnKy8vTwMHDlRJSYni4uK0fPny1qfqcCg5OTlwJCUlWfq4XK6gPgkJCaFNTAQGAAAs/E5H2I7GxkbV19cHHY2NjZb3bGpqUlVVlXJycgJtTqdTOTk5qqioaHWuR48eVe/evZWamqrx48frww8/tPRZt26dunfvrv79+2vGjBmqq6sL+ZoQGAAAOIWKiooUHx8fdBQVFVn61dbWqrm52VIhSEpKksfjaXHs/v37a/ny5Xruuef05JNPyufzadSoUfr0008DfcaOHatVq1apvLxc9957r15//XVdccUVam5uDulzsIcBAABTGJ/DkJ+fL7fbHdT2TfYQtCQ7O1vZ2dmBP48aNUoDBgzQI488ooULF0qSrrnmmsDrgwcP1pAhQ9S3b1+tW7dOY8aMafN7ERgAADCF8bZKl8vVpoCQmJioqKgoeb3eoHav16vk5OQ2vVd0dLSGDRumHTt2tNonPT1diYmJ2rFjR0iBgSUJAADagZiYGGVmZqq8vDzQ5vP5VF5eHlRFOJnm5mZt2bJFKSkprfb59NNPVVdXd9I+LSEwAABgcobxCIHb7VZpaalWrlyprVu3asaMGWpoaFBeXp4kafLkycrPzw/0X7BggV5++WXt3LlTGzZs0PXXX689e/boxhtvlPTlhsjbbrtN77zzjnbv3q3y8nKNHz9e/fr1U25ubkhzY0kCAABThJ70OGnSJO3fv18FBQXyeDzKyMhQWVlZYCNkdXW1nM6vU8jBgwc1ffp0eTweJSQkKDMzU2+//bYGDhwoSYqKitLmzZu1cuVKHTp0SD169NDll1+uhQsXhryPwuH3+/3h+6jfXPqw+yM9BaDdeX/9oEhPAWiX4mNCf1JhKNLmvxS2sXYXhvZNvr1iSQIAANhiSQIAABM/b21BYAAAwODn1yotWJIAAAC2qDAAAGDi67QFgQEAABNLEhZkKAAAYIsKAwAAJu6SsCAwAABgIjBYsCQBAABsUWEAAMBEgcGCwAAAgMHPkoQFgQEAABO3VVqwhwEAANiiwgAAgIklCQsCAwAAJvKCBUsSAADAFhUGAAAMTr5OWxAYAAAwcJOEFRkKAADYosIAAICBCoMVgQEAAIODxGBBYAAAwEBesGIPAwAAsEWFAQAAAxUGKwIDAAAGB/V3Cy4JAACwRYUBAAADSxJWBAYAAAz8WKUVSxIAAMAWFQYAAAwsSVgRGAAAMBAYrFiSAAAAtqgwAABg4LckrAgMAAAYeHCTFYEBAAADBQYrMhQAALBFhQEAAAMVBisCAwAABgKDFUsSAADAFhUGAAAM/JaEFYEBAAADSxJWLEkAAABbVBgAADBQYbAiMAAAYHCwicGCJQkAAGCLCgMAAAaWJKwIDAAAGAgMVgQGAAAMBAYr9jAAAABbVBgAADBwk4QVgQEAAANLElYsSQAAAFtUGAAAMDj4Om1BYAAAwMCShBUZCgAA2KLCAACAwUGJwYIKAwAABocjfEeoli1bprS0NMXGxiorK0uVlZWt9l2xYoUcDkfQERsbG9TH7/eroKBAKSkp6tixo3JycvTxxx+HPC8CAwAA7cTq1avldrtVWFioDRs2aOjQocrNzVVNTU2r53Tu3Fn79u0LHHv27Al6/b777tPSpUtVUlKi9evXq1OnTsrNzdUXX3wR0twIDAAAGCJVYViyZImmT5+uvLw8DRw4UCUlJYqLi9Py5ctPMleHkpOTA0dSUlLgNb/fr+LiYt15550aP368hgwZolWrVmnv3r1au3ZtSHMjMAAAYAhnYGhsbFR9fX3Q0djYaHnPpqYmVVVVKScnJ9DmdDqVk5OjioqKVud69OhR9e7dW6mpqRo/frw+/PDDwGu7du2Sx+MJGjM+Pl5ZWVknHbMl7WbT4873r4z0FIB2p2OvwkhPAWiXjlWPPaXjh/PR0EVFRZo/f35QW2Fhoe66666gttraWjU3NwdVCCQpKSlJH330UYtj9+/fX8uXL9eQIUN0+PBh3X///Ro1apQ+/PBDnXPOOfJ4PIExzDG/eq2t2k1gAADgTJSfny+32x3U5nK5wjJ2dna2srOzA38eNWqUBgwYoEceeUQLFy4My3t8hcAAAIAhnBUGl8vVpoCQmJioqKgoeb3eoHav16vk5OQ2vVd0dLSGDRumHTt2SFLgPK/Xq5SUlKAxMzIy2vgJvsQeBgAADE6HP2xHW8XExCgzM1Pl5eWBNp/Pp/Ly8qAqwsk0Nzdry5YtgXDQp08fJScnB41ZX1+v9evXt3nMr1BhAACgnXC73ZoyZYqGDx+ukSNHqri4WA0NDcrLy5MkTZ48WT179lRRUZEkacGCBbrwwgvVr18/HTp0SIsXL9aePXt04403SvryDopbbrlFixYt0rnnnqs+ffpo3rx56tGjhyZMmBDS3AgMAAAYwrkkEYpJkyZp//79KigokMfjUUZGhsrKygKbFqurq+V0fr04cPDgQU2fPl0ej0cJCQnKzMzU22+/rYEDBwb6zJ07Vw0NDbrpppt06NAhXXzxxSorK7M84MmOw+/3t71eckptj/QEgHaHuySAlh2rfvqUjv+Tl98M21gvXH5x2MaKJPYwAAAAWyxJAABgCGWz4r8KAgMAAIZI7WFoz1iSAAAAtqgwAABg4Nu0FYEBAAADSxJWBAYAAAwONj1aUHUBAAC2qDAAAGBgScKKwAAAgIHyuxXXBAAA2KLCAACAgSc9WhEYAAAwsIfBiiUJAABgiwoDAAAGvk1bERgAADCwJGFFiAIAALaoMAAAYOAuCSsCAwAABpYkrAgMAAAYWK+34poAAABbVBgAADCwh8GKwAAAgIE9DFYsSQAAAFtUGAAAMFBhsCIwAABgoPxuxTUBAAC2qDAAAGDgLgkrAgMAAAb2MFixJAEAAGxRYQAAwMC3aSsCAwAABpYkrAgMAAAYHGx6tKDqAgAAbFFhAADAwJKEFYEBAAAD5XcrrgkAALBFhQEAAANPerQiMAAAYGAPgxVLEgAAwBYVBgAADFQYrAgMAAAYoiI9gXaIJQkAAGCLCgMAAAbukrAiMAAAYGAPgxWBAQAAA4HBij0MAADAFhUGAAAMUVQYLAgMAAAYWJKwYkkCAADYosIAAICB2yqtCAwAABhYkrBiSQIAANiiwgAAgIHfkrAiMAAAYGBJwoolCQAAYIsKAwAABu6SsCIwAABg4EmPVixJAABgcDrCd4Rq2bJlSktLU2xsrLKyslRZWdmm85555hk5HA5NmDAhqH3q1KlyOBxBx9ixY0OeF4EBAIB2YvXq1XK73SosLNSGDRs0dOhQ5ebmqqam5qTn7d69W3PmzNEll1zS4utjx47Vvn37AsfTTz8d8twIDAAAGCJVYViyZImmT5+uvLw8DRw4UCUlJYqLi9Py5ctbPae5uVnXXXed5s+fr/T09Bb7uFwuJScnB46EhITQJiYCAwAAFuEMDI2Njaqvrw86GhsbLe/Z1NSkqqoq5eTkfD0Pp1M5OTmqqKhoda4LFixQ9+7dNW3atFb7rFu3Tt27d1f//v01Y8YM1dXVhX5NQj4DAAC0WVFRkeLj44OOoqIiS7/a2lo1NzcrKSkpqD0pKUkej6fFsd988009/vjjKi0tbfX9x44dq1WrVqm8vFz33nuvXn/9dV1xxRVqbm4O6XNwlwQAAIaoMN5WmZ+fL7fbHdTmcrm+9bhHjhzRDTfcoNLSUiUmJrba75prrgn89+DBgzVkyBD17dtX69at05gxY9r8fgQGAAAM4Sy/u1yuNgWExMRERUVFyev1BrV7vV4lJydb+n/yySfavXu3xo0bF2jz+XySpA4dOmjbtm3q27ev5bz09HQlJiZqx44dIQUGliQAAGgHYmJilJmZqfLy8kCbz+dTeXm5srOzLf3PP/98bdmyRRs3bgwcV155pUaPHq2NGzcqNTW1xff59NNPVVdXp5SUlJDmR4UBAABDpH5Lwu12a8qUKRo+fLhGjhyp4uJiNTQ0KC8vT5I0efJk9ezZU0VFRYqNjdWgQYOCzu/SpYskBdqPHj2q+fPn66qrrlJycrI++eQTzZ07V/369VNubm5IcyMwAABgiFRgmDRpkvbv36+CggJ5PB5lZGSorKwssBGyurpaTmfbFweioqK0efNmrVy5UocOHVKPHj10+eWXa+HChSHvo3D4/f528sDs7ZGeANDudOxVGOkpAO3SserQHzwUijW7ysI21s/6hP5UxfaICgMAAIZw3iVxpiAwAABgiNSSRHtGYAAAwEBgsOK2SgAAYIsKAwAABioMVgQGAAAMUQQGC5YkAACALSoMAAAYnNxWaUFgAADAQPndimsCAABsUWEAAMDAXRJWVBjOcE899YIuvXSaBg/+qSZOvFWbN7f+mx0vv/y2fvrT2Ro+/BplZPxM48f/WmvX/tXS75NP/qGbb16ozMxJysj4ma66arb27q05lR8DCLtfTL5MH721VAe3r9Tfnluo4UP7nrR/fOc4PbgwTzvfe1iHPl6lzeuWKHd0Rot95/zySh2rflqLCyefgpnjdIhyhO84U1BhOIO9+OIbKip6TPPnz9TQoedp5crnNW1agcrKStStWxdL//j472nGjKuVnn6OoqM76LXX3tVvf/uf6tatiy655PuSpOrqfbr22tt11VWX6de/vlZnnRWnjz+ulssVc5o/HfDN/Wzchbp33g2a9dvH9e7GHfrVtCv0/JN3aOiPbtX+unpL/+joKL3w1G9VU1uv624u1meeA+rV82wdrm+w9M0ckq5p147R5r/vOR0fBThtqDCcwZ54Yq2uvjpXV12Vo379emn+/F8qNtalP/7xlRb7Z2UN1mWXZatv31T16pWiKVOuVP/+aaqq+nugz4MP/l4/+EGm5s7N08CBfdWrV4rGjMlqMYAA7dWvb/yJnnj6r/r9s6/ro48/06z8x3XsWJOmTPpRi/2nTBqthC5n6erpD6jive2q/rRWb67fqi1bq4P6dYpz6Ymlv9Iv7yjVocPWMIHvDqfDH7bjTEFgOEM1NR3Xhx/u0KhRQwNtTqdTo0Zl6P33t9me7/f7VVGxSbt2faYRIy6QJPl8Pq1b957S0npq2rQCZWdfr4kTb9Wrr1acss8BhFt0dJSGDe6jv775QaDN7/frr29+oJHfP7fFc36S832tr/pYxYvytLuqRO+9cp9umzleTmOhu3jRz1X21/f12j+Nje8mpyN8x5mCJYkz1MGD9Wpu9qlbt4Sg9m7dumjnzk9bPe/IkQb94AdT1dR0XE6nU4WFM3TRRcMkSXV1h/X558dUWrpGt9xyvebMmao33qjSr35VpFWrfqeRIwef0s8EhENi187q0CFKNbWHg9prag+rf98eLZ7Tp1d3/WjU2Xpm7Vv696n3qm9akooX/VzR0R10d/EfJUkTx2UrY1CaLh535yn/DDj1zqR/6MMl7IHhH//4hwoLC7V8+fJW+zQ2NqqxsTGozeVqYh28HejUqaPWrv1Pff75F6qo2KR77nlcqanJysoaLJ/PJ0kaMyZLU6dOkCQNGJCuDRs+0jPPlBEYcMZyOp3aX1evmXeUyufz6/0tu9QjqatuufnfdHfxH3VOSlctvmuK/u26u9XYeDzS0wVOibAvSRw4cEArV648aZ+ioiLFx8cHHUVFj4R7Kv/SEhI6KyrKqbq6g0HtdXWHlJiY0MpZX/6PsXfvHhowIF0///m/Kzd3lB599NnAmB06RKlv315B5/Ttm6q9e/eH/0MAp0DtgXqdONGs7onxQe3dE+Pl2X+oxXM8NYf08a598vm+Xo/+aMdnSume8L9LHOlKOjteFS/erSM7n9SRnU/qB9kD9cu8XB3Z+aRl6QLtnzOMx5ki5ArD888/f9LXd+7caTtGfn6+3G53UJvLVd1Kb3wTMTHRuuCCfqqo2KycnGxJX+5BqKjYpOuv/0mbx/H5/GpqOh4Yc/Dgc7VrV/CSxu7dn6lnz7PDN3ngFDp+vFnvb9ml0RcN0p9ffk+S5HA4NPqiC1Sy8uUWz6l4b5smjb9IDodDfv+XoeHc9BTt8x7U8ePNeu2tD5SZc1vQOY8+cLO2fbJXDzz8fFDQwHeDg4xnEXJgmDBhQtBfmpY4bK60y+WSy+UyWlmOCLe8vAm6/fYHNWhQPw0Zcp5WrnxOx459oZ/+NEeSNHfuEiUlddOtt06RJD3yyLMaNKifevVKUVPTcb3++nt6/vnXdNddMwJjTpv2U82efZ9GjBikrKzBeuONDXrttUqtWnV3RD4j8E0sfewFlT4wQ1Vbduq9/72tMi7OpVX//bok6bEHZ2iv56AK7n1GklT6+1d085TL9cBdU/TwijL165Oi22ZO0MNPlEmSjjZ8ob9vDw7SDZ836sDBo5Z24Lsq5MCQkpKihx9+WOPHj2/x9Y0bNyozM/NbTwzf3o9/fIkOHDispUuf0v79BzVgQLoee2x+YEli3779QaXSzz//QvPn/5c8njrFxsYoPf0cLV58q37840sCfS67LFt33fVLPfros1q06FH16dNTS5fma/jwC0775wO+qTV/fkeJXTurwP0zJZ3dRZv/vkfjb7gnsBEytUdiUFXg030HdOUN9+i+ghv07kv3aq/3oJYt/4se+K+TV1zx3UWBwcrhP1mpoAVXXnmlMjIytGDBghZf37Rpk4YNGxbYINd2rT+BEPhX1bFXYaSnALRLx6qfPqXjv1f7QtjGGp7Y9mXg9izkCsNtt92mhobWH0jSr18/vfbaa99qUgAAoH0JOTBccsklJ329U6dO+uEPf/iNJwQAQKSdSXc3hAsPbgIAwOA4gx7pHC6EKAAAYIsKAwAABu6SsCIwAABg4MFNVgQGAAAM5AUr9jAAAABbVBgAADDwe2FWBAYAAAzkBSuWJAAAgC0qDAAAGLhLworAAACAgbxgxZIEAACwRYUBAAADFQYrAgMAAAZuq7RiSQIAANiiwgAAgIECgxWBAQAAg8Phj/QU2h0CAwAABioMVuxhAAAAtqgwAABg4EmPVgQGAAAMlN+tuCYAAMAWFQYAAAwsSVgRGAAAMJAXrFiSAAAAtqgwAABgYEnCisAAAICBvGDFkgQAALBFhQEAAAM/b21FYAAAwEBesCIwAABg4NcqrdjDAAAAbFFhAADAwJKEFRUGAAAMDkf4jlAtW7ZMaWlpio2NVVZWliorK9t03jPPPCOHw6EJEyYEtfv9fhUUFCglJUUdO3ZUTk6OPv7445DnRWAAAKCdWL16tdxutwoLC7VhwwYNHTpUubm5qqmpOel5u3fv1pw5c3TJJZdYXrvvvvu0dOlSlZSUaP369erUqZNyc3P1xRdfhDQ3AgMAAAZHGI9QLFmyRNOnT1deXp4GDhyokpISxcXFafny5a2e09zcrOuuu07z589Xenp60Gt+v1/FxcW68847NX78eA0ZMkSrVq3S3r17tXbt2pDmRmAAAMDgDOPR2Nio+vr6oKOxsdHynk1NTaqqqlJOTs7X83A6lZOTo4qKilbnumDBAnXv3l3Tpk2zvLZr1y55PJ6gMePj45WVlXXSMVtCYAAA4BQqKipSfHx80FFUVGTpV1tbq+bmZiUlJQW1JyUlyePxtDj2m2++qccff1ylpaUtvv7VeaGM2RrukgAAwBDOH5/Kz8+X2+0OanO5XN963CNHjuiGG25QaWmpEhMTv/V4dggMAABYhC8xuFyuNgWExMRERUVFyev1BrV7vV4lJydb+n/yySfavXu3xo0bF2jz+XySpA4dOmjbtm2B87xer1JSUoLGzMjICOlzsCQBAEA7EBMTo8zMTJWXlwfafD6fysvLlZ2dbel//vnna8uWLdq4cWPguPLKKzV69Ght3LhRqamp6tOnj5KTk4PGrK+v1/r161sc82SoMAAAYHBE6NFNbrdbU6ZM0fDhwzVy5EgVFxeroaFBeXl5kqTJkyerZ8+eKioqUmxsrAYNGhR0fpcuXSQpqP2WW27RokWLdO6556pPnz6aN2+eevToYXlegx0CAwAABocjMgX4SZMmaf/+/SooKJDH41FGRobKysoCmxarq6vldIY2t7lz56qhoUE33XSTDh06pIsvvlhlZWWKjY0NaRyH3+9vJ7+wsT3SEwDanY69CiM9BaBdOlb99Ckd/1DTX8I2VpeYK8I2ViSxhwEAANhiSQIAAEOk9jC0ZwQGAAAsCAwmliQAAIAtKgwAABgidZdEe0ZgAADAgiUJExEKAADYosIAAICBuySsCAwAABgIDFYsSQAAAFtUGAAAsOD7tInAAACAweFgScJEYAAAwILAYKLmAgAAbFFhAADAwF0SVgQGAAAsKMCbuCIAAMAWFQYAAAwsSVgRGAAAMHBbpRVLEgAAwBYVBgAALKgwmAgMAAAYHBTgLbgiAADAFhUGAAAsWJIwERgAADBwl4QVgQEAAAsCg4k9DAAAwBYVBgAADNwlYUVgAADAgiUJExEKAADYosIAAICBH5+yIjAAAGDgtkorliQAAIAtKgwAAFjwfdpEYAAAwMAeBisiFAAAsEWFAQAACyoMJgIDAAAG7pKwIjAAAGDBir2JKwIAAGxRYQAAwMBdElYOv9/vj/Qk0H40NjaqqKhI+fn5crlckZ4O0C7w9wIgMMBQX1+v+Ph4HT58WJ07d470dIB2gb8XAHsYAABAGxAYAACALQIDAACwRWBAEJfLpcLCQjZ2Af+EvxcAmx4BAEAbUGEAAAC2CAwAAMAWgQEAANgiMAAAAFsEBgQsW7ZMaWlpio2NVVZWliorKyM9JSCi/va3v2ncuHHq0aOHHA6H1q5dG+kpARFDYIAkafXq1XK73SosLNSGDRs0dOhQ5ebmqqamJtJTAyKmoaFBQ4cO1bJlyyI9FSDiuK0SkqSsrCyNGDFCDz30kCTJ5/MpNTVVs2bN0h133BHh2QGR53A49Kc//UkTJkyI9FSAiKDCADU1Namqqko5OTmBNqfTqZycHFVUVERwZgCA9oLAANXW1qq5uVlJSUlB7UlJSfJ4PBGaFQCgPSEwAAAAWwQGKDExUVFRUfJ6vUHtXq9XycnJEZoVAKA9ITBAMTExyszMVHl5eaDN5/OpvLxc2dnZEZwZAKC96BDpCaB9cLvdmjJlioYPH66RI0equLhYDQ0NysvLi/TUgIg5evSoduzYEfjzrl27tHHjRnXt2lW9evWK4MyA04/bKhHw0EMPafHixfJ4PMrIyNDSpUuVlZUV6WkBEbNu3TqNHj3a0j5lyhStWLHi9E8IiCACAwAAsMUeBgAAYIvAAAAAbBEYAACALQIDAACwRWAAAAC2CAwAAMAWgQEAANgiMAAAAFsEBgAAYIvAAAAAbBEYAACALQIDAACw9f8Bxr0P1c/Iw7YAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# get performance metrics\n",
    "print('Performance on Test Set')\n",
    "our_metrics(y_test_tree, y_pred_eec)\n",
    "# print confusion matrix\n",
    "# print(confusion_matrix(y_test, y_pred))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of AdaBoostClassifiers of the Model:10\n"
     ]
    }
   ],
   "source": [
    "# print(f'Number of AdaBoostClassifiers of the Model:{len(eec.estimators_)}')\n",
    "# n_adas = []\n",
    "\n",
    "# for ind_ada in eec.estimators_:\n",
    "#     n_adas.append(len(ind_ada.estimators_))\n",
    "#     # for ind_tree in ind_ada.estimators_:\n",
    "#     #     n_nodes.append(ind_tree.tree_.node_count)\n",
    "#     #     max_depths.append(ind_tree.tree_.max_depth)\n",
    "#     # print(f'Average number of nodes {int(np.mean(n_nodes))}')\n",
    "#     # print(f'Average maximum depth {int(np.mean(max_depths))}')\n",
    "#     print(f'Average number of DecisionTreeClassifiers per AdaBoostClassifier {int(np.mean(n_adas))}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Hyperparameter Tuning - RandomSearch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "# to see the list of hyperparameters that can be tuned\n",
    "#eec_rs.get_params()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Hyperparameters of the EasyEnsembleClassifier\n",
    "\n",
    "* 'base_estimator': 'deprecated', now 'estimator'\n",
    "* 'estimator': None, default=AdaBoostClassifier()\n",
    "* 'n_estimators': default=10, Number of AdaBoost learners in the ensemble.\n",
    "* 'n_jobs': default=None, Number of CPU cores used during the cross-validation loop. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors.\n",
    "* 'random_state': default=None, If None, the random number generator is the RandomState instance used by np.random.\n",
    "* 'replacement': default=False, Whether or not to sample randomly with replacement or not.\n",
    "* 'sampling_strategy': default='auto'equivalent to 'not minority', Sampling information to sample the data set. float, str, dict, callable, default=’auto’. When float, it corresponds to the desired ratio of the number of samples in the minority class over the number of samples in the majority class after resampling. Therefore, the ratio is expressed as $N_m/N_{rM}$,\n",
    " where $N_m$ is the number of samples in the minority class and $N_{rM}$ is the number of samples in the majority class after resampling.\n",
    "* 'verbose': default=0, Controls the verbosity of the building process.\n",
    "* 'warm_start': default=False. When set to True, reuse the solution of the previous call to fit and add more estimators to the ensemble, otherwise, just fit a whole new ensemble."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 123,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define parameter grid for randomized search\n",
    "# sampling_strategy=0.5 yields very poor results, is removed. score 0.147-0.162\n",
    "# sampling_strategy=1.5 yields error, probably because this ratio cannot be achieved by the dataset, is removed.\n",
    "# sampling_strategy=1.2 yields error, probably because this ratio cannot be achieved by the dataset, is removed.\n",
    "# n_estimators 14 give no gain in scores, take a long time. can be removed\n",
    "param_grid = { \"sampling_strategy\" : ['auto', 0.8, 1],\n",
    "            \"replacement\" : [False, True],\n",
    "                \"n_estimators\" : [5, 6, 8, 10, 12]}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "# # Instantiate stratified CV. if to use, use cv=cv in RandomizedSearch\n",
    "# cv = RepeatedStratifiedKFold(n_splits=5, n_repeats=3, random_state=RSEED)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Hyperparameter tuning using the F2 scorer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Instantiate random search and define the metric to optimize \n",
    "rs = RandomizedSearchCV(EasyEnsembleClassifier(random_state=RSEED), param_grid, scoring=ftwo_scorer,\n",
    "                  cv=5, verbose=5, n_jobs=1, n_iter=10, random_state=RSEED) #cv=5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 5 folds for each of 10 candidates, totalling 50 fits\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 1/5] END n_estimators=6, replacement=False, sampling_strategy=auto;, score=0.341 total time=  29.0s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 2/5] END n_estimators=6, replacement=False, sampling_strategy=auto;, score=0.342 total time=  28.9s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 3/5] END n_estimators=6, replacement=False, sampling_strategy=auto;, score=0.347 total time=  28.6s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 4/5] END n_estimators=6, replacement=False, sampling_strategy=auto;, score=0.341 total time=  29.1s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 5/5] END n_estimators=6, replacement=False, sampling_strategy=auto;, score=0.347 total time=  28.0s\n",
      "[CV 1/5] END n_estimators=6, replacement=False, sampling_strategy=1.5;, score=nan total time=   0.1s\n",
      "[CV 2/5] END n_estimators=6, replacement=False, sampling_strategy=1.5;, score=nan total time=   0.1s\n",
      "[CV 3/5] END n_estimators=6, replacement=False, sampling_strategy=1.5;, score=nan total time=   0.1s\n",
      "[CV 4/5] END n_estimators=6, replacement=False, sampling_strategy=1.5;, score=nan total time=   0.1s\n",
      "[CV 5/5] END n_estimators=6, replacement=False, sampling_strategy=1.5;, score=nan total time=   0.1s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 1/5] END n_estimators=12, replacement=False, sampling_strategy=auto;, score=0.341 total time=  55.6s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 2/5] END n_estimators=12, replacement=False, sampling_strategy=auto;, score=0.342 total time=  56.7s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 3/5] END n_estimators=12, replacement=False, sampling_strategy=auto;, score=0.346 total time=  57.2s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 4/5] END n_estimators=12, replacement=False, sampling_strategy=auto;, score=0.340 total time=  57.3s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 5/5] END n_estimators=12, replacement=False, sampling_strategy=auto;, score=0.347 total time=  56.9s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 1/5] END n_estimators=12, replacement=True, sampling_strategy=1;, score=0.341 total time=  57.5s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 2/5] END n_estimators=12, replacement=True, sampling_strategy=1;, score=0.341 total time=  58.0s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 3/5] END n_estimators=12, replacement=True, sampling_strategy=1;, score=0.347 total time=  57.5s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 4/5] END n_estimators=12, replacement=True, sampling_strategy=1;, score=0.341 total time=  57.3s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 5/5] END n_estimators=12, replacement=True, sampling_strategy=1;, score=0.348 total time=  56.9s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 1/5] END n_estimators=8, replacement=False, sampling_strategy=0.5;, score=0.147 total time=  56.4s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 2/5] END n_estimators=8, replacement=False, sampling_strategy=0.5;, score=0.157 total time=  56.6s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 3/5] END n_estimators=8, replacement=False, sampling_strategy=0.5;, score=0.158 total time=  56.1s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 4/5] END n_estimators=8, replacement=False, sampling_strategy=0.5;, score=0.159 total time=  56.7s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 5/5] END n_estimators=8, replacement=False, sampling_strategy=0.5;, score=0.162 total time=  57.9s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 1/5] END n_estimators=14, replacement=True, sampling_strategy=auto;, score=0.340 total time= 1.1min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 2/5] END n_estimators=14, replacement=True, sampling_strategy=auto;, score=0.341 total time= 1.1min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 3/5] END n_estimators=14, replacement=True, sampling_strategy=auto;, score=0.347 total time= 1.1min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 4/5] END n_estimators=14, replacement=True, sampling_strategy=auto;, score=0.341 total time= 1.1min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 5/5] END n_estimators=14, replacement=True, sampling_strategy=auto;, score=0.348 total time= 1.1min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 1/5] END n_estimators=10, replacement=True, sampling_strategy=1;, score=0.341 total time=  46.7s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 2/5] END n_estimators=10, replacement=True, sampling_strategy=1;, score=0.341 total time=  47.1s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 3/5] END n_estimators=10, replacement=True, sampling_strategy=1;, score=0.346 total time=  47.0s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 4/5] END n_estimators=10, replacement=True, sampling_strategy=1;, score=0.341 total time=  46.8s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 5/5] END n_estimators=10, replacement=True, sampling_strategy=1;, score=0.348 total time=  46.6s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 1/5] END n_estimators=14, replacement=False, sampling_strategy=auto;, score=0.341 total time= 1.1min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 2/5] END n_estimators=14, replacement=False, sampling_strategy=auto;, score=0.342 total time= 1.1min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 3/5] END n_estimators=14, replacement=False, sampling_strategy=auto;, score=0.346 total time= 6.0min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 4/5] END n_estimators=14, replacement=False, sampling_strategy=auto;, score=0.340 total time= 5.4min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 5/5] END n_estimators=14, replacement=False, sampling_strategy=auto;, score=0.347 total time= 1.1min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 1/5] END n_estimators=8, replacement=False, sampling_strategy=auto;, score=0.341 total time=  37.6s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 2/5] END n_estimators=8, replacement=False, sampling_strategy=auto;, score=0.342 total time=  37.1s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 3/5] END n_estimators=8, replacement=False, sampling_strategy=auto;, score=0.347 total time=  37.1s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 4/5] END n_estimators=8, replacement=False, sampling_strategy=auto;, score=0.340 total time=  37.1s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 5/5] END n_estimators=8, replacement=False, sampling_strategy=auto;, score=0.347 total time=  37.1s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 1/5] END n_estimators=14, replacement=True, sampling_strategy=1;, score=0.340 total time= 1.1min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 2/5] END n_estimators=14, replacement=True, sampling_strategy=1;, score=0.341 total time= 1.1min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 3/5] END n_estimators=14, replacement=True, sampling_strategy=1;, score=0.347 total time= 1.1min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 4/5] END n_estimators=14, replacement=True, sampling_strategy=1;, score=0.341 total time= 1.1min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 5/5] END n_estimators=14, replacement=True, sampling_strategy=1;, score=0.348 total time= 1.1min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/model_selection/_validation.py:378: FitFailedWarning: \n",
      "5 fits failed out of a total of 50.\n",
      "The score on these train-test partitions for these parameters will be set to nan.\n",
      "If these failures are not expected, you can try to debug them by setting error_score='raise'.\n",
      "\n",
      "Below are more details about the failures:\n",
      "--------------------------------------------------------------------------------\n",
      "5 fits failed with the following error:\n",
      "Traceback (most recent call last):\n",
      "  File \"/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/model_selection/_validation.py\", line 686, in _fit_and_score\n",
      "    estimator.fit(X_train, y_train, **fit_params)\n",
      "  File \"/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/imblearn/utils/fixes.py\", line 78, in wrapper\n",
      "    estimator._validate_params()\n",
      "  File \"/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/imblearn/base.py\", line 42, in _validate_params\n",
      "    validate_parameter_constraints(\n",
      "  File \"/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/imblearn/utils/_param_validation.py\", line 107, in validate_parameter_constraints\n",
      "    raise InvalidParameterError(\n",
      "imblearn.utils._param_validation.InvalidParameterError: The 'sampling_strategy' parameter of EasyEnsembleClassifier must be a float in the range (0.0, 1.0], a str among {'majority', 'auto', 'not majority', 'not minority', 'all'}, an instance of 'dict' or a callable. Got 1.5 instead.\n",
      "\n",
      "  warnings.warn(some_fits_failed_message, FitFailedWarning)\n",
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/model_selection/_search.py:952: UserWarning: One or more of the test scores are non-finite: [0.34368584        nan 0.34323132 0.3435133  0.15682134 0.34336035\n",
      " 0.34331153 0.34298221 0.34339131 0.34336035]\n",
      "  warnings.warn(\n",
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-1 {color: black;background-color: white;}#sk-container-id-1 pre{padding: 0;}#sk-container-id-1 div.sk-toggleable {background-color: white;}#sk-container-id-1 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-1 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-1 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-1 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-1 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-1 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-1 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-1 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-1 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-1 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-1 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-1 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-1 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-1 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-1 div.sk-item {position: relative;z-index: 1;}#sk-container-id-1 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-1 div.sk-item::before, #sk-container-id-1 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-1 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-1 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-1 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-1 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-1 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-1 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-1 div.sk-label-container {text-align: center;}#sk-container-id-1 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-1 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-1\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>RandomizedSearchCV(cv=5, estimator=EasyEnsembleClassifier(random_state=42),\n",
       "                   n_jobs=1,\n",
       "                   param_distributions={&#x27;n_estimators&#x27;: [6, 8, 10, 12, 14],\n",
       "                                        &#x27;replacement&#x27;: [False, True],\n",
       "                                        &#x27;sampling_strategy&#x27;: [&#x27;auto&#x27;, 0.5, 0.8,\n",
       "                                                              1, 1.2, 1.5]},\n",
       "                   random_state=42, scoring=make_scorer(fbeta_score, beta=2),\n",
       "                   verbose=5)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-1\" type=\"checkbox\" ><label for=\"sk-estimator-id-1\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">RandomizedSearchCV</label><div class=\"sk-toggleable__content\"><pre>RandomizedSearchCV(cv=5, estimator=EasyEnsembleClassifier(random_state=42),\n",
       "                   n_jobs=1,\n",
       "                   param_distributions={&#x27;n_estimators&#x27;: [6, 8, 10, 12, 14],\n",
       "                                        &#x27;replacement&#x27;: [False, True],\n",
       "                                        &#x27;sampling_strategy&#x27;: [&#x27;auto&#x27;, 0.5, 0.8,\n",
       "                                                              1, 1.2, 1.5]},\n",
       "                   random_state=42, scoring=make_scorer(fbeta_score, beta=2),\n",
       "                   verbose=5)</pre></div></div></div><div class=\"sk-parallel\"><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-2\" type=\"checkbox\" ><label for=\"sk-estimator-id-2\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">estimator: EasyEnsembleClassifier</label><div class=\"sk-toggleable__content\"><pre>EasyEnsembleClassifier(random_state=42)</pre></div></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-3\" type=\"checkbox\" ><label for=\"sk-estimator-id-3\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">EasyEnsembleClassifier</label><div class=\"sk-toggleable__content\"><pre>EasyEnsembleClassifier(random_state=42)</pre></div></div></div></div></div></div></div></div></div></div>"
      ],
      "text/plain": [
       "RandomizedSearchCV(cv=5, estimator=EasyEnsembleClassifier(random_state=42),\n",
       "                   n_jobs=1,\n",
       "                   param_distributions={'n_estimators': [6, 8, 10, 12, 14],\n",
       "                                        'replacement': [False, True],\n",
       "                                        'sampling_strategy': ['auto', 0.5, 0.8,\n",
       "                                                              1, 1.2, 1.5]},\n",
       "                   random_state=42, scoring=make_scorer(fbeta_score, beta=2),\n",
       "                   verbose=5)"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Fit randomized search object to data\n",
    "rs.fit(X_train_tree_small, y_train_tree_small)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best score (F2): 0.344\n",
      "Best parameters: {'sampling_strategy': 'auto', 'replacement': False, 'n_estimators': 6}\n"
     ]
    }
   ],
   "source": [
    "# Best score\n",
    "print('Best score (F2):', round(rs.best_score_, 3))\n",
    "\n",
    "# Best parameters\n",
    "print('Best parameters:', rs.best_params_)\n",
    "\n",
    "\n",
    "# Assigning the fitted EasyEnsemble model with best parameter combination to a new variable sgd_best_rs\n",
    "eec_best_rs = rs.best_estimator_\n",
    "\n",
    "# Best score (F2): 0.344\n",
    "# Best parameters: {'sampling_strategy': 'auto', 'replacement': False, 'n_estimators': 6}\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>mean_fit_time</th>\n",
       "      <th>std_fit_time</th>\n",
       "      <th>mean_score_time</th>\n",
       "      <th>std_score_time</th>\n",
       "      <th>param_sampling_strategy</th>\n",
       "      <th>param_replacement</th>\n",
       "      <th>param_n_estimators</th>\n",
       "      <th>params</th>\n",
       "      <th>split0_test_score</th>\n",
       "      <th>split1_test_score</th>\n",
       "      <th>split2_test_score</th>\n",
       "      <th>split3_test_score</th>\n",
       "      <th>split4_test_score</th>\n",
       "      <th>mean_test_score</th>\n",
       "      <th>std_test_score</th>\n",
       "      <th>rank_test_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>24.674976</td>\n",
       "      <td>0.385618</td>\n",
       "      <td>4.055783</td>\n",
       "      <td>0.031895</td>\n",
       "      <td>auto</td>\n",
       "      <td>False</td>\n",
       "      <td>6</td>\n",
       "      <td>{'sampling_strategy': 'auto', 'replacement': F...</td>\n",
       "      <td>0.341466</td>\n",
       "      <td>0.341968</td>\n",
       "      <td>0.347311</td>\n",
       "      <td>0.340704</td>\n",
       "      <td>0.346980</td>\n",
       "      <td>0.343686</td>\n",
       "      <td>0.002855</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.079664</td>\n",
       "      <td>0.020227</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.5</td>\n",
       "      <td>False</td>\n",
       "      <td>6</td>\n",
       "      <td>{'sampling_strategy': 1.5, 'replacement': Fals...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>48.627287</td>\n",
       "      <td>0.512083</td>\n",
       "      <td>8.120380</td>\n",
       "      <td>0.102189</td>\n",
       "      <td>auto</td>\n",
       "      <td>False</td>\n",
       "      <td>12</td>\n",
       "      <td>{'sampling_strategy': 'auto', 'replacement': F...</td>\n",
       "      <td>0.340702</td>\n",
       "      <td>0.342124</td>\n",
       "      <td>0.346242</td>\n",
       "      <td>0.340185</td>\n",
       "      <td>0.346903</td>\n",
       "      <td>0.343231</td>\n",
       "      <td>0.002809</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>49.398040</td>\n",
       "      <td>0.321447</td>\n",
       "      <td>8.054350</td>\n",
       "      <td>0.052197</td>\n",
       "      <td>1</td>\n",
       "      <td>True</td>\n",
       "      <td>12</td>\n",
       "      <td>{'sampling_strategy': 1, 'replacement': True, ...</td>\n",
       "      <td>0.340619</td>\n",
       "      <td>0.341292</td>\n",
       "      <td>0.346818</td>\n",
       "      <td>0.341045</td>\n",
       "      <td>0.347792</td>\n",
       "      <td>0.343513</td>\n",
       "      <td>0.003119</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>51.326847</td>\n",
       "      <td>0.499438</td>\n",
       "      <td>5.426844</td>\n",
       "      <td>0.125236</td>\n",
       "      <td>0.5</td>\n",
       "      <td>False</td>\n",
       "      <td>8</td>\n",
       "      <td>{'sampling_strategy': 0.5, 'replacement': Fals...</td>\n",
       "      <td>0.147146</td>\n",
       "      <td>0.157463</td>\n",
       "      <td>0.158150</td>\n",
       "      <td>0.159184</td>\n",
       "      <td>0.162164</td>\n",
       "      <td>0.156821</td>\n",
       "      <td>0.005097</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>56.734713</td>\n",
       "      <td>1.205055</td>\n",
       "      <td>9.451854</td>\n",
       "      <td>0.231032</td>\n",
       "      <td>auto</td>\n",
       "      <td>True</td>\n",
       "      <td>14</td>\n",
       "      <td>{'sampling_strategy': 'auto', 'replacement': T...</td>\n",
       "      <td>0.340484</td>\n",
       "      <td>0.341219</td>\n",
       "      <td>0.346562</td>\n",
       "      <td>0.340896</td>\n",
       "      <td>0.347641</td>\n",
       "      <td>0.343360</td>\n",
       "      <td>0.003082</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>40.159334</td>\n",
       "      <td>0.173078</td>\n",
       "      <td>6.685404</td>\n",
       "      <td>0.013534</td>\n",
       "      <td>1</td>\n",
       "      <td>True</td>\n",
       "      <td>10</td>\n",
       "      <td>{'sampling_strategy': 1, 'replacement': True, ...</td>\n",
       "      <td>0.340611</td>\n",
       "      <td>0.341294</td>\n",
       "      <td>0.346121</td>\n",
       "      <td>0.340795</td>\n",
       "      <td>0.347737</td>\n",
       "      <td>0.343312</td>\n",
       "      <td>0.003006</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>166.867892</td>\n",
       "      <td>137.161199</td>\n",
       "      <td>9.326243</td>\n",
       "      <td>0.106383</td>\n",
       "      <td>auto</td>\n",
       "      <td>False</td>\n",
       "      <td>14</td>\n",
       "      <td>{'sampling_strategy': 'auto', 'replacement': F...</td>\n",
       "      <td>0.340533</td>\n",
       "      <td>0.341783</td>\n",
       "      <td>0.346189</td>\n",
       "      <td>0.339783</td>\n",
       "      <td>0.346624</td>\n",
       "      <td>0.342982</td>\n",
       "      <td>0.002871</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>31.824622</td>\n",
       "      <td>0.224274</td>\n",
       "      <td>5.358602</td>\n",
       "      <td>0.014662</td>\n",
       "      <td>auto</td>\n",
       "      <td>False</td>\n",
       "      <td>8</td>\n",
       "      <td>{'sampling_strategy': 'auto', 'replacement': F...</td>\n",
       "      <td>0.340703</td>\n",
       "      <td>0.342032</td>\n",
       "      <td>0.347001</td>\n",
       "      <td>0.340425</td>\n",
       "      <td>0.346795</td>\n",
       "      <td>0.343391</td>\n",
       "      <td>0.002915</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>55.303939</td>\n",
       "      <td>0.208569</td>\n",
       "      <td>9.302041</td>\n",
       "      <td>0.030709</td>\n",
       "      <td>1</td>\n",
       "      <td>True</td>\n",
       "      <td>14</td>\n",
       "      <td>{'sampling_strategy': 1, 'replacement': True, ...</td>\n",
       "      <td>0.340484</td>\n",
       "      <td>0.341219</td>\n",
       "      <td>0.346562</td>\n",
       "      <td>0.340896</td>\n",
       "      <td>0.347641</td>\n",
       "      <td>0.343360</td>\n",
       "      <td>0.003082</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   mean_fit_time  std_fit_time  mean_score_time  std_score_time   \n",
       "0      24.674976      0.385618         4.055783        0.031895  \\\n",
       "1       0.079664      0.020227         0.000000        0.000000   \n",
       "2      48.627287      0.512083         8.120380        0.102189   \n",
       "3      49.398040      0.321447         8.054350        0.052197   \n",
       "4      51.326847      0.499438         5.426844        0.125236   \n",
       "5      56.734713      1.205055         9.451854        0.231032   \n",
       "6      40.159334      0.173078         6.685404        0.013534   \n",
       "7     166.867892    137.161199         9.326243        0.106383   \n",
       "8      31.824622      0.224274         5.358602        0.014662   \n",
       "9      55.303939      0.208569         9.302041        0.030709   \n",
       "\n",
       "  param_sampling_strategy param_replacement param_n_estimators   \n",
       "0                    auto             False                  6  \\\n",
       "1                     1.5             False                  6   \n",
       "2                    auto             False                 12   \n",
       "3                       1              True                 12   \n",
       "4                     0.5             False                  8   \n",
       "5                    auto              True                 14   \n",
       "6                       1              True                 10   \n",
       "7                    auto             False                 14   \n",
       "8                    auto             False                  8   \n",
       "9                       1              True                 14   \n",
       "\n",
       "                                              params  split0_test_score   \n",
       "0  {'sampling_strategy': 'auto', 'replacement': F...           0.341466  \\\n",
       "1  {'sampling_strategy': 1.5, 'replacement': Fals...                NaN   \n",
       "2  {'sampling_strategy': 'auto', 'replacement': F...           0.340702   \n",
       "3  {'sampling_strategy': 1, 'replacement': True, ...           0.340619   \n",
       "4  {'sampling_strategy': 0.5, 'replacement': Fals...           0.147146   \n",
       "5  {'sampling_strategy': 'auto', 'replacement': T...           0.340484   \n",
       "6  {'sampling_strategy': 1, 'replacement': True, ...           0.340611   \n",
       "7  {'sampling_strategy': 'auto', 'replacement': F...           0.340533   \n",
       "8  {'sampling_strategy': 'auto', 'replacement': F...           0.340703   \n",
       "9  {'sampling_strategy': 1, 'replacement': True, ...           0.340484   \n",
       "\n",
       "   split1_test_score  split2_test_score  split3_test_score  split4_test_score   \n",
       "0           0.341968           0.347311           0.340704           0.346980  \\\n",
       "1                NaN                NaN                NaN                NaN   \n",
       "2           0.342124           0.346242           0.340185           0.346903   \n",
       "3           0.341292           0.346818           0.341045           0.347792   \n",
       "4           0.157463           0.158150           0.159184           0.162164   \n",
       "5           0.341219           0.346562           0.340896           0.347641   \n",
       "6           0.341294           0.346121           0.340795           0.347737   \n",
       "7           0.341783           0.346189           0.339783           0.346624   \n",
       "8           0.342032           0.347001           0.340425           0.346795   \n",
       "9           0.341219           0.346562           0.340896           0.347641   \n",
       "\n",
       "   mean_test_score  std_test_score  rank_test_score  \n",
       "0         0.343686        0.002855                1  \n",
       "1              NaN             NaN               10  \n",
       "2         0.343231        0.002809                7  \n",
       "3         0.343513        0.003119                2  \n",
       "4         0.156821        0.005097                9  \n",
       "5         0.343360        0.003082                4  \n",
       "6         0.343312        0.003006                6  \n",
       "7         0.342982        0.002871                8  \n",
       "8         0.343391        0.002915                3  \n",
       "9         0.343360        0.003082                4  "
      ]
     },
     "execution_count": 110,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# pd.DataFrame(rs.cv_results_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Making predictions on the test set\n",
    "y_pred_test_eec_rs = eec_best_rs.predict(X_test_tree)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Training an EasyEnsembleClassifier with the best Hyperparameters\n",
    "\n",
    "We will now train an EEC on the whole data set with the hyperparameters determined in the RandomizedSearch with the partial data set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# initiate EasyEnsembleClassifier with the best hyperparameters\n",
    "eec_best = EasyEnsembleClassifier(random_state=RSEED, sampling_strategy='auto', replacement=False, n_estimators=6)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-1 {color: black;background-color: white;}#sk-container-id-1 pre{padding: 0;}#sk-container-id-1 div.sk-toggleable {background-color: white;}#sk-container-id-1 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-1 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-1 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-1 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-1 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-1 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-1 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-1 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-1 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-1 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-1 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-1 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-1 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-1 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-1 div.sk-item {position: relative;z-index: 1;}#sk-container-id-1 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-1 div.sk-item::before, #sk-container-id-1 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-1 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-1 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-1 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-1 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-1 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-1 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-1 div.sk-label-container {text-align: center;}#sk-container-id-1 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-1 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-1\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>EasyEnsembleClassifier(n_estimators=6, random_state=42)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-1\" type=\"checkbox\" checked><label for=\"sk-estimator-id-1\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">EasyEnsembleClassifier</label><div class=\"sk-toggleable__content\"><pre>EasyEnsembleClassifier(n_estimators=6, random_state=42)</pre></div></div></div></div></div>"
      ],
      "text/plain": [
       "EasyEnsembleClassifier(n_estimators=6, random_state=42)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# fit eec to train data\n",
    "eec_best.fit(X_train_tree, y_train_tree)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# make prediction on test data\n",
    "y_pred_eec_best = eec_best.predict(X_test_tree)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# make prediction on train data\n",
    "y_pred_train_eec_best = eec_best.predict(X_train_tree)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Performance on Train Set\n",
      "Model Metrics and Normalized Confusion Matrix\n",
      "_____________________\n",
      "_____________________\n",
      "Accuracy: 0.6089\n",
      "_____________________\n",
      "F1-score: 0.2034\n",
      "_____________________\n",
      "Fbeta_score with beta=1.5: 0.2752\n",
      "_____________________\n",
      "Fbeta_score with beta=2: 0.3431\n",
      "_____________________\n",
      "Fbeta_score with beta=3: 0.445\n",
      "_____________________\n",
      "Recall 0.6329\n",
      "_____________________\n",
      "Specificity 0.6069\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgwAAAGdCAYAAAB+VCt0AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAApZ0lEQVR4nO3de3hU1b3/8c9MSCZAJQRCLmAgXJSrEBskBkstNRL0iNCe2tijgCmCUsqjTEWNlUTAY1AUKUfa2CgCtVWqreg5eoL9pWJrTUGCAeQgiFyiyEwuXAIRE8jM7w+PY2fthJ3RCZND36/n2c9T1qy9svbug3zyXWvvcfj9fr8AAADOwhnpCQAAgI6PwAAAAGwRGAAAgC0CAwAAsEVgAAAAtggMAADAFoEBAADYIjAAAABbBAYAAGCrU6Qn8IW09CWRngLQ4Wx/JyPSUwA6pG7RV7fr+J37/ihsY52qei5sY0VShwkMAAB0FA4HBXgTdwQAANiiwgAAgMHB79MWBAYAAAwsSVgRGAAAMBAYrLgjAADAFhUGAAAMDocj0lPocAgMAABYUIA3cUcAAIAtKgwAABjY9GhFYAAAwEBgsOKOAAAAW1QYAAAw8KZHKwIDAAAGliSsuCMAAMAWFQYAAAxUGKwIDAAAGAgMVgQGAAAMDvFqaBMRCgAA2KLCAACAgSUJKwIDAAAGAoMVdwQAANiiwgAAgIEKgxWBAQAACwKDiTsCAABsUWEAAMDAkoQVgQEAAAOBwYo7AgAAbFFhAADA4OD3aQsCAwAABpYkrAgMAAAYHA6+fMpEhAIAALaoMAAAYGBJworAAACAgU2PVtwRAABgiwoDAAAGliSsCAwAABgIDFbcEQAAYIsKAwAABjY9WhEYAAAwsSRhwR0BAKADWblypdLS0hQbG6vMzExt3rz5rP2PHTumOXPmKCUlRS6XSxdffLFee+21rzVmSwgMAAAYHA5n2I5QrFu3Tm63W4WFhdq6datGjRqlnJwcVVdXt9i/qalJV199tQ4cOKAXX3xRu3fvVklJifr06fOVx2z1nvj9fn9IZ7STtPQlkZ4C0OFsfycj0lMAOqRu0Ve36/iDRv8ibGPt3XJHm/tmZmbqsssu0xNPPCFJ8vl8Sk1N1dy5c3Xvvfda+hcXF2vp0qV6//33FR0dHZYxW0OFAQAAg0POsB2NjY2qr68POhobGy0/s6mpSRUVFcrOzg60OZ1OZWdnq7y8vMV5vvLKK8rKytKcOXOUlJSkESNG6KGHHlJzc/NXHrM1BAYAANpRUVGR4uLigo6ioiJLv9raWjU3NyspKSmoPSkpSR6Pp8Wx9+3bpxdffFHNzc167bXXtGDBAj322GN68MEHv/KYreEpCQAADOF8cVN+fr7cbndQm8vlCsvYPp9PiYmJ+vWvf62oqChlZGTo0KFDWrp0qQoLC8PyM75AYAAAwORwhG0ol8vVpoCQkJCgqKgoeb3eoHav16vk5OQWz0lJSVF0dLSioqICbUOHDpXH41FTU9NXGrM1LEkAANABxMTEKCMjQ2VlZYE2n8+nsrIyZWVltXjOFVdcob1798rn8wXa9uzZo5SUFMXExHylMVtDYAAAwOQM4xECt9utkpISrVmzRrt27dLs2bPV0NCgvLw8SdK0adOUn58f6D979mwdOXJEd9xxh/bs2aNXX31VDz30kObMmdPmMduKJQkAAExhXJIIRW5urmpqalRQUCCPx6P09HSVlpYGNi1WVVXJ6fwyhaSmpmrDhg2aN2+eRo4cqT59+uiOO+7QPffc0+Yx24r3MAAdGO9hAFrW3u9huDjrV2Eba0/57LCNFUlUGAAAMEWowtCRERgAADCxw8+CWwIAAGxRYQAAwOBnScKCwAAAgIm8YEFgAADA5CQxmNjDAAAAbFFhAADAxB4GCwIDAAAm8oIFSxIAAMAWFQYAAExserQgMAAAYGIPgwVLEgAAwBYVBgAATBQYLAgMAACY2MNgwZIEAACwRYUBAAATBQYLAgMAAAa+rdKKwAAAgIk9DBbsYQAAALaoMAAAYKLAYEFgAADAxB4GC5YkAACALSoMAACY2PRoQWAAAMBEXrBgSQIAANiiwgAAgIlNjxYEBgAATAQGC5YkAACALSoMAACY+HXagsAAAICJJQkLAgMAACbyggVFFwAAYIsKAwAABj9verQgMJznpuZ+U7dNz1Svnl21a0+1Ch/+k7a9d7jV/t0ucOmun35bE787WHFxsTp0uF6Llv4/bXxrnyRpzDdTNWt6pi4ZmqSkxAs0a94f9PobH5yrywHC5vfPvalnnylTXW29LhrcR/Pvu0HDL0lrse+f/1Sp1SUb9NFHtTpzplmpfXvp5ulX6drrxwT61NXW6z8ef1mb3t6lEydO6dKMQZp/3w3q2y/xHF0Rwoo9DBYEhvPYdROG6P6ffVf3//sGvbvjE/34psu09pe5+u7kX6vu6KeW/tGdnPpN8Y2qO9Kg2fNfkrf6pPqkdFP9icZAny6do7Vrj1cvrN+uJx///rm8HCBsXv/vCi1/5CXdW5CrESPT9Nxv3tDc21bqxf8sUI+eF1j6x8V1Ud6siUrrn6To6Cj99c33tGjBs4rv+Q1lXTFMfr9f8+/4tTp1itKjK25T12/E6ndr/6w5t/6Hfv/y/ercxRWBqwTCiz0M57Fbp47R83/cphde3qG9++r08wdLdeqz0/rhlJEt9v/hlJHq3i1Ws+b9URWVh/TxJ8e1qeIj7dpTHeiz8W/79NjKv2rDG3vO1WUAYfe7tX/WlB+M1fXfy9KAgSnKL7hRsbExeuWl8hb7Z4y5WOOzR6n/wGRd2LeXfjR1vAZd3FuVWz+vvFUdrNaObQd0z4IbNfySfkrrn6R7F+SqsfG0NrxWcS4vDeHiCONxniAwnKeiOzk1Ymiy/rbpQKDN75f+tumAvjmyT4vnZH/nIm3dfkiL8ifonbK52vDiDP1kRpacrOXhPHL69Bm9/z8faczlgwNtTqdTYy4frB3b9tue7/f7tfnvu3XwQLW+mTHw8zGbzkiSXDFfFm2dTqeiozup8t0Pw3wFOCecjvAd54mQlyRqa2u1atUqlZeXy+PxSJKSk5M1duxY3XLLLerVq1fYJ4nQxcd3UadOTtXWNQS119Q1aGBazxbP6dunu8Ze1k/rX9upvJ/+Xmmp8Vp8X46iOzn1iyf/di6mDbS7Y0dPqrnZZ1l66NGzmw7s97Z63skTp3Ttd3+uptNnFOV06p77c5U5dqgkKa1/spJT4rXyF68ov+BH6twlRr9b+4aqvcdUV3O8Xa8HOFdCCgzvvPOOcnJy1KVLF2VnZ+viiy+WJHm9Xq1YsUJLlizRhg0bNHr06LOO09jYqMbGxqA2v++MHE62VESSw+lQ7ZEG5S8ulc/n13u7vEpKvEC3Tc8kMOCfXpeuLv32D/n69NNGvfP33Xp86R/V58KeyhhzsTpFR+mR5TO1uOC3uuqKuxUV5dRllw/W2HHD5PdHeub4Stj0aBHSv9Bz587VDTfcoOLiYjmMm+n3+3X77bdr7ty5Ki9veR3wC0VFRVq4cGFQW1zSVeqenB3KdHAWR49+qjNnfEro2TWovVfPrqqpbWjxnJqakzp9xief78v/wn24v06Jvb6h6E5OnT7ja9c5A+dC9/hvKCrKqSN1J4Laj9TVq2dCt1bPczqdSu37eQV18JALdWCfR6ufel0ZYz7/xWno8L763R/ydfLEKZ0+fUbxPS7QLT9aqqHD+7bfxaD9kBcsQtrDsG3bNs2bN88SFiTJ4XBo3rx5qqystB0nPz9fx48fDzriEr8TylRg4/QZn97b5dHYMWmBNodDGjumn7ZuP9TiOVu2fay0vvFBwbp/vx7yVp8gLOC8ER3dSUOGpeqdTbsDbT6fT+9s2qNLRvVv8zg+n19N/7t34R9944LOiu9xgaoOVmvXzipdOb7lTcbA/zUhVRiSk5O1efNmDRkypMXPN2/erKSkJNtxXC6XXK7gx4xYjgi/p36zWY8tvk47/uewKt87rBk3jVaXzjF64eXtkqTHFl8nb/UJPfIfb0qSnv39u5qWm6HCu6/Wmue2KK1fD/1kRpZWP7clMGaXztFK6xsf+HNqn+4aNjhRx45/pk889ef2AoGv6N+mfVcLf/4bDR3eV8NHpOm5Z9/QqVONmjTlcklSYf5a9UqM00/nTZYkPVOyQcOG91Wf1F463XRGf/vrTr32X5t17/03Bsb8fxu2Kj7+G0pK6aEPP/hEjy15UVd+d6Quv2JoRK4RX9N5tFkxXEL6V/quu+7SrFmzVFFRoauuuioQDrxer8rKylRSUqJHH320XSaK0P3X6++rR3wXzZs9Tr0SumrX7mpN/8k61R75/B0MfVK6yf8PC6yHvSc0/SfrtOCuq1T6wgx5qk/omd9tUfEzfw/0GTk8Rc8/9W+BPy+46ypJ0ouv7NBdBa+eoysDvp4J12To2NGTevKJV1VXe0IXD+mjFcVzAksSnsNH5PiHfzA+O9Wkhx/8vaq9x+RyRatf/yQtKpquCddkBPrU1tTr8Uf+qCN1J5TQq5uuvT5Tt94+8ZxfG8KEwGDh8PtD25Kzbt06Pf7446qoqFBzc7MkKSoqShkZGXK73frhD3/4lSaSlr7kK50HnM+2v5Nh3wn4J9Qt+up2HX/ArS+Ebax9T90QtrEiKeR1gNzcXOXm5ur06dOqra2VJCUkJCg6OjrskwMAAB3DV944EB0drZSUlHDOBQCAjoElCQt2GgIAYOI9DBa8GhoAANiiwgAAgIklCQsCAwAAJurvFtwSAABgiwoDAAAmNj1aEBgAADCxh8GCJQkAAGCLCgMAAAY/SxIWBAYAAEzU3y0IDAAAmNjDYEGGAgAAtggMAACYHI7wHSFauXKl0tLSFBsbq8zMTG3evLnVvqtXr5bD4Qg6YmNjg/rccsstlj4TJ04MeV4sSQAAYIrQksS6devkdrtVXFyszMxMLV++XDk5Odq9e7cSExNbPKdbt27avXt34M+OFkLKxIkT9cwzzwT+7HK5Qp4bFQYAADqIZcuWaebMmcrLy9OwYcNUXFysLl26aNWqVa2e43A4lJycHDiSkpIsfVwuV1Cf+Pj4kOdGYAAAwOQI39HY2Kj6+vqgo7Gx0fIjm5qaVFFRoezs7ECb0+lUdna2ysvLW53qyZMn1a9fP6Wmpmry5MnauXOnpc/GjRuVmJiowYMHa/bs2aqrqwv5lhAYAAAw+J2OsB1FRUWKi4sLOoqKiiw/s7a2Vs3NzZYKQVJSkjweT4vzHDx4sFatWqWXX35Zzz77rHw+n8aOHauPP/440GfixIlau3atysrK9PDDD+vNN9/UNddco+bm5pDuCXsYAABoR/n5+XK73UFtX2UPQUuysrKUlZUV+PPYsWM1dOhQPfnkk1q8eLEk6cYbbwx8fskll2jkyJEaOHCgNm7cqKuuuqrNP4vAAACAKYybHl0uV5sCQkJCgqKiouT1eoPavV6vkpOT2/SzoqOjdemll2rv3r2t9hkwYIASEhK0d+/ekAIDSxIAAJgi8FhlTEyMMjIyVFZWFmjz+XwqKysLqiKcTXNzs3bs2KGUlJRW+3z88ceqq6s7a5+WEBgAAOgg3G63SkpKtGbNGu3atUuzZ89WQ0OD8vLyJEnTpk1Tfn5+oP+iRYv0+uuva9++fdq6datuvvlmHTx4ULfeequkzzdEzp8/X3//+9914MABlZWVafLkyRo0aJBycnJCmhtLEgAAmCL063Rubq5qampUUFAgj8ej9PR0lZaWBjZCVlVVyen8cnJHjx7VzJkz5fF4FB8fr4yMDL399tsaNmyYJCkqKkrbt2/XmjVrdOzYMfXu3VsTJkzQ4sWLQ95H4fD7/f7wXepXl5a+JNJTADqc7e9kRHoKQIfULfrqdh0/rbA0bGMdWBj6WxU7IioMAACY+PIpC/YwAAAAW1QYAAAwUWGwIDAAAGDwf4VvmTzfsSQBAABsUWEAAMDEr9MWBAYAAEwsSViQoQAAgC0qDAAAmHhKwoLAAACAicBgwZIEAACwRYUBAAATBQYLAgMAAAY/SxIWBAYAAEw8VmnBHgYAAGCLCgMAACaWJCwIDAAAmMgLFixJAAAAW1QYAAAwOPl12oLAAACAgYckrMhQAADAFhUGAAAMVBisCAwAABgcJAYLAgMAAAbyghV7GAAAgC0qDAAAGKgwWBEYAAAwOKi/W3BLAACALSoMAAAYWJKwIjAAAGDgyyqtWJIAAAC2qDAAAGBgScKKwAAAgIHAYMWSBAAAsEWFAQAAA98lYUVgAADAwIubrAgMAAAYKDBYkaEAAIAtKgwAABioMFgRGAAAMBAYrFiSAAAAtqgwAABg4LskrAgMAAAYWJKwYkkCAADYosIAAICBCoMVgQEAAIODTQwWLEkAAABbVBgAADCwJGFFYAAAwEBgsCIwAABgIDBYsYcBAADYosIAAICBhySsCAwAABhYkrBiSQIAANiiwgAAgMHBr9MWBAYAAAwsSViRoQAAgC0qDAAAGByUGCyoMAAAYHA4wneEauXKlUpLS1NsbKwyMzO1efPmVvuuXr1aDocj6IiNjQ3q4/f7VVBQoJSUFHXu3FnZ2dn64IMPQp4XgQEAgA5i3bp1crvdKiws1NatWzVq1Cjl5OSourq61XO6deumw4cPB46DBw8Gff7II49oxYoVKi4u1qZNm9S1a1fl5OTos88+C2luBAYAAAyRqjAsW7ZMM2fOVF5enoYNG6bi4mJ16dJFq1atOstcHUpOTg4cSUlJgc/8fr+WL1+u+++/X5MnT9bIkSO1du1affLJJ1q/fn1IcyMwAABgCGdgaGxsVH19fdDR2Nho+ZlNTU2qqKhQdnZ2oM3pdCo7O1vl5eWtzvXkyZPq16+fUlNTNXnyZO3cuTPw2f79++XxeILGjIuLU2Zm5lnHbEmH2fR4oPL7kZ4C0OF07lsY6SkAHdKpqqvbdfxwvhq6qKhICxcuDGorLCzUAw88ENRWW1ur5ubmoAqBJCUlJen9999vcezBgwdr1apVGjlypI4fP65HH31UY8eO1c6dO3XhhRfK4/EExjDH/OKztuowgQEAgPNRfn6+3G53UJvL5QrL2FlZWcrKygr8eezYsRo6dKiefPJJLV68OCw/4wsEBgAADOGsMLhcrjYFhISEBEVFRcnr9Qa1e71eJScnt+lnRUdH69JLL9XevXslKXCe1+tVSkpK0Jjp6eltvILPsYcBAACD0+EP29FWMTExysjIUFlZWaDN5/OprKwsqIpwNs3NzdqxY0cgHPTv31/JyclBY9bX12vTpk1tHvMLVBgAAOgg3G63pk+frtGjR2vMmDFavny5GhoalJeXJ0maNm2a+vTpo6KiIknSokWLdPnll2vQoEE6duyYli5dqoMHD+rWW2+V9PkTFHfeeacefPBBXXTRRerfv78WLFig3r17a8qUKSHNjcAAAIAhnEsSocjNzVVNTY0KCgrk8XiUnp6u0tLSwKbFqqoqOZ1fLg4cPXpUM2fOlMfjUXx8vDIyMvT2229r2LBhgT533323GhoaNGvWLB07dkzf+ta3VFpaannBkx2H3+9ve72kXe2J9ASADoenJICWnap6rl3H/5fX3wrbWK9O+FbYxook9jAAAABbLEkAAGAIZbPiPwsCAwAAhkjtYejIWJIAAAC2qDAAAGDgt2krAgMAAAaWJKwIDAAAGBxserSg6gIAAGxRYQAAwMCShBWBAQAAA+V3K+4JAACwRYUBAAADb3q0IjAAAGBgD4MVSxIAAMAWFQYAAAz8Nm1FYAAAwMCShBUhCgAA2KLCAACAgackrAgMAAAYWJKwIjAAAGBgvd6KewIAAGxRYQAAwMAeBisCAwAABvYwWLEkAQAAbFFhAADAQIXBisAAAICB8rsV9wQAANiiwgAAgIGnJKwIDAAAGNjDYMWSBAAAsEWFAQAAA79NWxEYAAAwsCRhRWAAAMDgYNOjBVUXAABgiwoDAAAGliSsCAwAABgov1txTwAAgC0qDAAAGHjToxWBAQAAA3sYrFiSAAAAtqgwAABgoMJgRWAAAMAQFekJdEAsSQAAAFtUGAAAMPCUhBWBAQAAA3sYrAgMAAAYCAxW7GEAAAC2qDAAAGCIosJgQWAAAMDAkoQVSxIAAMAWFQYAAAw8VmlFYAAAwMCShBVLEgAAwBYVBgAADHyXhBWBAQAAA0sSVixJAAAAW1QYAAAw8JSEFYEBAAADb3q0IjAAAGBgD4MVexgAAOhAVq5cqbS0NMXGxiozM1ObN29u03nPP/+8HA6HpkyZEtR+yy23yOFwBB0TJ04MeV4EBgAADE5H+I5QrFu3Tm63W4WFhdq6datGjRqlnJwcVVdXn/W8AwcO6K677tK4ceNa/HzixIk6fPhw4HjuuedCm5gIDAAAWEQqMCxbtkwzZ85UXl6ehg0bpuLiYnXp0kWrVq1q9Zzm5mbddNNNWrhwoQYMGNBiH5fLpeTk5MARHx8f2sREYAAAoF01Njaqvr4+6GhsbLT0a2pqUkVFhbKzswNtTqdT2dnZKi8vb3X8RYsWKTExUTNmzGi1z8aNG5WYmKjBgwdr9uzZqqurC/k6CAwAABiiHP6wHUVFRYqLiws6ioqKLD+ztrZWzc3NSkpKCmpPSkqSx+NpcZ5vvfWWnn76aZWUlLR6LRMnTtTatWtVVlamhx9+WG+++aauueYaNTc3h3RPeEoCAABDOH+bzs/Pl9vtDmpzuVxfe9wTJ05o6tSpKikpUUJCQqv9brzxxsD/vuSSSzRy5EgNHDhQGzdu1FVXXdXmn0dgAACgHblcrjYFhISEBEVFRcnr9Qa1e71eJScnW/p/+OGHOnDggCZNmhRo8/l8kqROnTpp9+7dGjhwoOW8AQMGKCEhQXv37g0pMLAkAQCAIRKbHmNiYpSRkaGysrJAm8/nU1lZmbKysiz9hwwZoh07dqiysjJwXH/99Ro/frwqKyuVmpra4s/5+OOPVVdXp5SUlJDuCRUGAAAMkXpxk9vt1vTp0zV69GiNGTNGy5cvV0NDg/Ly8iRJ06ZNU58+fVRUVKTY2FiNGDEi6Pzu3btLUqD95MmTWrhwof71X/9VycnJ+vDDD3X33Xdr0KBBysnJCWluBAYAADqI3Nxc1dTUqKCgQB6PR+np6SotLQ1shKyqqpLT2fbFgaioKG3fvl1r1qzRsWPH1Lt3b02YMEGLFy8OeR+Fw+/3d5Bv2NgT6QkAHU7nvoWRngLQIZ2qCv3FQ6F46cB/h22s76VdE7axIokKAwAABr5LworAAACAgcBgxVMSAADAFhUGAAAMVBisCAwAABiiCAwWLEkAAABbVBgAADA4HR3kjQMdCIEBAAAD5Xcr7gkAALBFhQEAAANPSVgRGM5zv/3tq3r66T+qpuaohgzprwULbtPIkRe32Pf1199WcfELqqo6rDNnzqhfv97Ky5uiKVO+G+gzePCkFs+dPz9Pt976/Xa5BqA93Dbtas27bZKSesVpx64quQtWa8u2D1vtH9etix6Yn6vJ11ymHnHfUNWhWs1fuFYb3qiUJM28OVszp16tfhcmSJJ27flYD/3ij3p947ZzcTkIM56SsCIwnMdee+2vKip6SgsXztGoURdrzZpXNGNGgUpLi9WzZ3dL/7i4CzR79g81YMCFio7upDfeeEf33fcL9ezZXePGfVOS9NZba4PO+ctfKvTzn69QTs7Yc3FJQFj8YNLlenjBVM2972m9U7lXP51xjV559l6N+s7PVFNXb+kfHR2lV397n6pr63XT7ct1yHNEffv00vH6hkCfQ54jWrDkOe3d75HDId38g2/rhafu0uXX5mvXno/P5eUB7YIvnzqP3XDDz3TJJRepoOB2SZ9/r/qVV+Zp6tTrNGvWDW0a43vfu0NXXnmZ7rzz5hY//8lPHlRDwymtWfPvYZs3vsSXT7WPv7y8WBXbPtS8gtWSJIfDob2bntCvVm/Qo798xdL/1puzNe+26zRq/M905kxzm3/Ooe0luu/ff6s16zaGaeb4Qnt/+dRfPa+Gbaxxyf8StrEiiU2P56mmptPauXOvxo4dFWhzOp0aOzZd77672/Z8v9+v8vJt2r//kC67bHiLfWprj+rNN7foBz+4OmzzBtpbdHSULr2kv/781nuBNr/frz+/9Z7GfPOiFs/5l+xvalPFB1r+YJ4OVBRry58e0fw5k+VsZaHb6XTohklZ6trZpU1bP2iX60D7cjrCd5wvWJI4Tx09Wq/mZp969owPau/Zs7v27Wu9PHriRIO+/e1b1NR0Wk6nU4WFs3XFFZe22Pell/6srl07a8IEliPwf0dCj27q1ClK1bXHg9qra49r8MDeLZ7Tv2+ivjO2l55f/zd975aHNTAtScsf/LGiozvpoeV/CPQbPjhVG9cvUqwrWicbPlPurGV6/4ND7Xo9aB/n0z/04RL2wPDRRx+psLBQq1atarVPY2OjGhsbg9pcria5XDHhng5C1LVrZ61f/wt9+ulnKi/fpiVLnlZqarIyMy+x9P3DH/6kSZO+w/9vOO85nU7V1NVrzr0l8vn8enfHfvVO6qE7b78uKDDs2feJMifeq7huXfS9azNVsmy2JvxwEaEB54WwL0kcOXJEa9asOWufoqIixcXFBR1FRU+Geyr/1OLjuykqyqm6uqNB7XV1x5SQEN/KWZ//h7Ffv94aOnSAfvzj7yknZ6x+/esXLP22bNmp/fsP6YYbJoR97kB7qj1SrzNnmpWYEBfUnpgQJ0/NsRbP8VQf0wf7D8vn+3LL1/t7DyklMV7R0VGBttOnm7XvoFfv7tivgoef145dBzXnxxPb5TrQvpxhPM4XIVcYXnnFuiHoH+3bt892jPz8fLnd7qA2l6sq1KngLGJiojV8+CCVl29XdnaWpM83PZaXb9PNN7d9A47P51dT02lL+4svvq7hwwdpyJD+YZszcC6cPt2sd3fs1/grRug/X98i6fNNj+OvGK7iNa+3eE75lt3KnXyFHA6HvtgnftGAFB32HtXp061vgnQ6nHLFRIf/ItDuHCxJWIQcGKZMmRL0l6YlDps77XK55HK5jFbK2uGWlzdF99zzuEaMGKSRIy/WmjUv69Spz/T972dLku6+e5mSknrqZz+bLkl68skXNGLEIPXtm6KmptN6880teuWVN/TAA7ODxj158lOVlv5N99wz45xfExAOK556VSWPzVbFjn3a8r+PVXbp4tLa378pSXrq8dn6xHNUBQ8/L0kq+c2fdPv0CXrsgen65epSDeqfovlzpuiXz5QGxlx0z43a8EalPvqkVhd07azcKVfo21lDNWnqkohcIxBuIQeGlJQU/fKXv9TkyZNb/LyyslIZGRlfe2L4+q69dpyOHDmuFSt+q5qaoxo6dICeemphYEni8OGaoF3en376mRYu/JU8njrFxsZowIALtXTpz3TtteOCxn311b/I7/fruuu+fU6vBwiXF//z70ro0U0F7h8oqVd3bf+fg5o8dUlgI2Rq74Sg5YePDx/R9VOX6JGCqXpnw8P6xHtUK1f9tx771ZcV1149u+npx3+i5MTuOn7iU733fpUmTV2iP/91xzm/Pnx9FBisQn4Pw/XXX6/09HQtWrSoxc+3bdumSy+9VD6fL8Sp8B4GwMR7GICWtfd7GLbUhu89DKMTzo/3MIRcYZg/f74aGhpa/XzQoEF64403vtakAABAxxJyYBg3btxZP+/atauuvPLKrzwhAAAi7Xx6uiFceHETAAAGh6ODfGtCB0KIAgAAtqgwAABg4CkJKwIDAAAGXtxkRWAAAMBAXrBiDwMAALBFhQEAAANfb21FYAAAwEBesGJJAgAA2KLCAACAgackrAgMAAAYyAtWLEkAAABbVBgAADBQYbAiMAAAYOCxSiuWJAAAgC0qDAAAGCgwWBEYAAAwOBz+SE+hwyEwAABgoMJgxR4GAABgiwoDAAAG3vRoRWAAAMBA+d2KewIAAGxRYQAAwMCShBWBAQAAA3nBiiUJAABgiwoDAAAGliSsCAwAABjIC1YsSQAAAFtUGAAAMPD11lYEBgAADOQFKwIDAAAGvq3Sij0MAADAFhUGAAAMLElYERgAADDwHgYrliQAAIAtAgMAAAZHGI9QrVy5UmlpaYqNjVVmZqY2b97cpvOef/55ORwOTZkyJajd7/eroKBAKSkp6ty5s7Kzs/XBBx+EPC8CAwAABmcYj1CsW7dObrdbhYWF2rp1q0aNGqWcnBxVV1ef9bwDBw7orrvu0rhx4yyfPfLII1qxYoWKi4u1adMmde3aVTk5Ofrss89CmhuBAQCADmLZsmWaOXOm8vLyNGzYMBUXF6tLly5atWpVq+c0Nzfrpptu0sKFCzVgwICgz/x+v5YvX677779fkydP1siRI7V27Vp98sknWr9+fUhzIzAAAGBwOMJ3NDY2qr6+PuhobGy0/MympiZVVFQoOzs70OZ0OpWdna3y8vJW57po0SIlJiZqxowZls/2798vj8cTNGZcXJwyMzPPOmZLCAwAAFiEbxdDUVGR4uLigo6ioiLLT6ytrVVzc7OSkpKC2pOSkuTxeFqc5VtvvaWnn35aJSUlLX7+xXmhjNkaHqsEAKAd5efny+12B7W5XK6vPe6JEyc0depUlZSUKCEh4WuPZ4fAAACAwRHGVze5XK42BYSEhARFRUXJ6/UGtXu9XiUnJ1v6f/jhhzpw4IAmTZoUaPP5fJKkTp06affu3YHzvF6vUlJSgsZMT08P6TpYkgAAwOBwOMN2tFVMTIwyMjJUVlYWaPP5fCorK1NWVpal/5AhQ7Rjxw5VVlYGjuuvv17jx49XZWWlUlNT1b9/fyUnJweNWV9fr02bNrU45tlQYQAAwCIyr3p0u92aPn26Ro8erTFjxmj58uVqaGhQXl6eJGnatGnq06ePioqKFBsbqxEjRgSd3717d0kKar/zzjv14IMP6qKLLlL//v21YMEC9e7d2/K+BjsEBgAAOojc3FzV1NSooKBAHo9H6enpKi0tDWxarKqqktMZ2uLA3XffrYaGBs2aNUvHjh3Tt771LZWWlio2NjakcRx+v7+DfIfnnkhPAOhwOvctjPQUgA7pVNVz7Tr+8abSsI0VFzMxbGNFEhUGAAAs+PYpE5seAQCALSoMAAAYQnm64Z8FgQEAAAuWJExEKAAAYIsKAwAAhnC+6fF8QWAAAMBAYLBiSQIAANiiwgAAgAW/T5sIDAAAGBwOliRMBAYAACwIDCZqLgAAwBYVBgAADDwlYUVgAADAggK8iTsCAABsUWEAAMDAkoQVgQEAAAOPVVqxJAEAAGxRYQAAwIIKg4nAAACAwUEB3oI7AgAAbFFhAADAgiUJE4EBAAADT0lYERgAALAgMJjYwwAAAGxRYQAAwMBTElYEBgAALFiSMBGhAACALSoMAAAY+PIpKwIDAAAGHqu0YkkCAADYosIAAIAFv0+bCAwAABjYw2BFhAIAALaoMAAAYEGFwURgAADAwFMSVgQGAAAsWLE3cUcAAIAtKgwAABh4SsLK4ff7/ZGeBDqOxsZGFRUVKT8/Xy6XK9LTAToE/l4ABAYY6uvrFRcXp+PHj6tbt26Rng7QIfD3AmAPAwAAaAMCAwAAsEVgAAAAtggMCOJyuVRYWMjGLuAf8PcCYNMjAABoAyoMAADAFoEBAADYIjAAAABbBAYAAGCLwICAlStXKi0tTbGxscrMzNTmzZsjPSUgov7yl79o0qRJ6t27txwOh9avXx/pKQERQ2CAJGndunVyu90qLCzU1q1bNWrUKOXk5Ki6ujrSUwMipqGhQaNGjdLKlSsjPRUg4nisEpKkzMxMXXbZZXriiSckST6fT6mpqZo7d67uvffeCM8OiDyHw6GXXnpJU6ZMifRUgIigwgA1NTWpoqJC2dnZgTan06ns7GyVl5dHcGYAgI6CwADV1taqublZSUlJQe1JSUnyeDwRmhUAoCMhMAAAAFsEBighIUFRUVHyer1B7V6vV8nJyRGaFQCgIyEwQDExMcrIyFBZWVmgzefzqaysTFlZWRGcGQCgo+gU6QmgY3C73Zo+fbpGjx6tMWPGaPny5WpoaFBeXl6kpwZEzMmTJ7V3797An/fv36/Kykr16NFDffv2jeDMgHOPxyoR8MQTT2jp0qXyeDxKT0/XihUrlJmZGelpARGzceNGjR8/3tI+ffp0rV69+txPCIggAgMAALDFHgYAAGCLwAAAAGwRGAAAgC0CAwAAsEVgAAAAtggMAADAFoEBAADYIjAAAABbBAYAAGCLwAAAAGwRGAAAgC0CAwAAsPX/Abiy/2uuobUrAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "print('Performance on Train Set')\n",
    "our_metrics(y_train_tree, y_pred_train_eec_best)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Performance on Test Set\n",
      "Model Metrics and Normalized Confusion Matrix\n",
      "_____________________\n",
      "_____________________\n",
      "Accuracy: 0.6094\n",
      "_____________________\n",
      "F1-score: 0.2041\n",
      "_____________________\n",
      "Fbeta_score with beta=1.5: 0.2762\n",
      "_____________________\n",
      "Fbeta_score with beta=2: 0.3443\n",
      "_____________________\n",
      "Fbeta_score with beta=3: 0.4465\n",
      "_____________________\n",
      "Recall 0.6349\n",
      "_____________________\n",
      "Specificity 0.6072\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgwAAAGdCAYAAAB+VCt0AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAApaElEQVR4nO3de3hU1b3/8c9MSCZAJQRiLmCAAHITCBggBqWWGg3aIvRUjb+jgKlCRQ4/dRQ1KomANSqKKUc8sVEK1FY5eiranzbQRtFSU9BgECkXI5dUYRISLoGICWTm94fHsbN2ws7ohMnhvF/Ps5+nrFl7Ze3dB/nku9be4/D5fD4BAACchjPcEwAAAB0fgQEAANgiMAAAAFsEBgAAYIvAAAAAbBEYAACALQIDAACwRWAAAAC2CAwAAMBWp3BP4Gspox8P9xSADqdiY2q4pwB0SDFRWe06fuc+/ydkY52oejFkY4VThwkMAAB0FA4HBXgTdwQAANiiwgAAgMHB79MWBAYAAAwsSVgRGAAAMBAYrLgjAADAFhUGAAAMDocj3FPocAgMAABYUIA3cUcAAIAtKgwAABjY9GhFYAAAwEBgsOKOAAAAW1QYAAAw8KZHKwIDAAAGliSsuCMAAMAWFQYAAAxUGKwIDAAAGAgMVgQGAAAMDvFqaBMRCgAA2KLCAACAgSUJKwIDAAAGAoMVdwQAANiiwgAAgIEKgxWBAQAACwKDiTsCAABsUWEAAMDAkoQVgQEAAAOBwYo7AgAAbFFhAADA4OD3aQsCAwAABpYkrAgMAAAYHA6+fMpEhAIAALaoMAAAYGBJworAAACAgU2PVtwRAABgiwoDAAAGliSsCAwAABgIDFbcEQAAYIsKAwAABjY9WhEYAAAwsSRhwR0BAAC2qDAAAGBg06MVdwQAAIPD4QjZEaxly5apX79+io6OVnp6ujZt2nTa/keOHNGcOXOUlJQkl8ulQYMG6c033/xOY7aEwAAAgMEhZ8iOYKxevVput1v5+fnavHmzUlNTlZWVpZqamhb7NzU16fLLL9fevXv1yiuvaOfOnSouLlbv3r2/9Zit3hOfz+cL6ox2kjL68XBPAehwKjamhnsKQIcUE5XVruOfP+bfQzbWJx/MbXPf9PR0jR07Vk8//bQkyev1Kjk5WXPnztV9991n6V9UVKTFixdrx44dioyMDMmYraHCAACAweFwhuxoq6amJpWXlyszM9Pf5nQ6lZmZqbKyshbPef3115WRkaE5c+YoISFBw4cP1yOPPKLm5uZvPWZr2PQIAIDpW+w9aE1jY6MaGxsD2lwul1wuV0BbbW2tmpublZCQENCekJCgHTt2tDj27t279dZbb+mGG27Qm2++qcrKSt122206efKk8vPzv9WYraHCAABAOyooKFBMTEzAUVBQEJKxvV6v4uPj9atf/UppaWnKzs7WAw88oKKiopCM/8+oMAAAYArhr9O5ublyu90BbWZ1QZLi4uIUERGh6urqgPbq6molJia2OHZSUpIiIyMVERHhbxs6dKg8Ho+ampq+1ZitocIAAIDJ4QjZ4XK51K1bt4CjpcAQFRWltLQ0lZaW+tu8Xq9KS0uVkZHR4jQvvvhiVVZWyuv1+tt27dqlpKQkRUVFfasxW0NgAACgg3C73SouLtbKlSu1fft2zZ49Ww0NDcrJyZEkTZ8+Xbm5uf7+s2fP1qFDh3T77bdr165deuONN/TII49ozpw5bR6zrViSAADAFMJNj8HIzs7WwYMHlZeXJ4/Ho1GjRqmkpMS/abGqqkpO5ze/6ycnJ2vt2rW68847NXLkSPXu3Vu333677r333jaP2Va8hwHowHgPA9Cy9n4Pw6BLQrdpcNeGW0M2VjixJAEAAGyxJAEAgMEXpiWJjozAAACAibxgQWAAAMDkJDGY2MMAAABsUWEAAMDEHgYLAgMAACbyggVLEgAAwBYVBgAATGx6tCAwAABgYg+DBUsSAADAFhUGAABMFBgsCAwAAJjYw2DBkgQAALBFhQEAABMFBgsCAwAABr6t0orAAACAiT0MFuxhAAAAtqgwAABgosBgQWAAAMDEHgYLliQAAIAtKgwAAJjY9GhBYAAAwEResGBJAgAA2KLCAACAiU2PFgQGAABMBAYLliQAAIAtKgwAAJj4ddqCwAAAgIklCQsCAwAAJvKCBUUXAABgiwoDAAAGH296tCAwnOWmXTdas2aM07k9u2r7rho99NiftWWbp9X+53zPpXn/NkFZPxykmJho7T9Qr4VPvKX1G3ZLksZdeJ5mTR+n4cMSlXDu9zTrzt/rT+srz9TlACHz8ovv6oUVb6mutl7nD+6tu3Ov0QUj+rbY9+0/b9Gvi9fps3/U6tSpZiX3OVc3zJioqyaP8/epq63X00+9ro1lO3Ts2AmNThugu3OvUZ++8WfqkhBK7GGwIDCcxX50xRA9cNdEPfiLdar4+IB+9q9jtPKZ63TZ1OdUd/gLS//ITk79pug61R36QrfNe02emmM6r1eM6o996e/TuXOktu+q0X++tlXPLvnJmbwcIGT+VLJZhYtf1X3zs3XByL566Tfv6P/+/Bm9/IcH1aPnOZb+3WK6KGfWFeqXkqDIyAhteGebFs3/nWJ7nKOMi4fK5/Np3u3PqVOnCD2xdKa6do3W71a9rX+buUyr19yvzl1cYbhKILTYw3AWu+XGMVr9+4/0yusfq3J3nR74xVqd+PKkrp06osX+104dqe7dovVz96sq3/K5Pj9Qr43l/9D2XQf9fd756x49+cwGrXv7kzN1GUDI/W7V25r60/Ga/JOL1H9Aku7Lu07RnaP0h1f/1mL/tLHna+JlqUrpn6jzks/V9Tf+QAMH9dKWzV9V3qr2HdTHH+3VvfOv07DhfdU3JUH3zr9OjY0ntfaP5Wfy0hAqjhAeZwkCw1kqspNTw4cmasPGvf42n0/668Z9unBkrxbPybx0gD78aL8W3ne53v/zHJW8nKPbfnaRnKzl4Sxy8uQp7fj7PzT2osH+NqfTqbEXDdbWLXtsz/f5fNr0t53at7dGo9MGfDVm0ylJksv1TdHW6XQqMrKTP1TgfxinI3THWSLoJYna2lotX75cZWVl8ni+WgtPTEzU+PHjddNNN+ncc88N+SQRvNjYLurUyanaQ4FLD7V1DRrQr0eL5/Tp3V3njY3Rmj/+XTlzX1Hf5Fgtyr1cnTo5tfRX752JaQPt7sjhBjU3ey1LDz16nqN9e6pbPe/4sRP60WXz1XTylCKcTt3z4LVKHz9EktQvJUGJSbFaVvgH5eZdr85dovS7VW+rpvqIamvr2/V6gDMlqMDw/vvvKysrS126dFFmZqYGDRokSaqurtbSpUv16KOPau3atRozZsxpx2lsbFRjY2NAm897Sg4nWyrCyel0qPbQF7p/0Vp5vT59vL1aifHf06zp4wgM+F+vS1eXXnjlXp34olHvb9ylwsVr1Pu8OKWNPV+dIiP02FM36+H8F5V5yX2KiHBq7EWDNP6SYfL5fOGeOr4NNj1aBPUv9Ny5c3XttdeqqKhIDuNm+nw+3XrrrZo7d67KyspOO05BQYEWLFgQ0BaTkKnYpMuDmQ5O4/DhL3TqlFdxPboEtMf17KqDdQ0tnlNT26CTp5rl9X7zH7jKPXWKP/d7iuzk1MlT3nadM3AmdI/tqogIpw7VHQtoP1R3TD1b2PD4NafTqeQ+X1VQBw05T3t2e7TiuT8pbez5kqShF/TRb1+5V8ePndDJk6cU2+Mc5fzrkxo6LLn9Lgbth7xgEdQehi1btujOO++0hAVJcjgcuvPOO1VRUWE7Tm5uro4ePRpwdE+YGMxUYOPkKa8+3u7RxenfPCbmcEjjx/XV5o/2t3jOBxWfqV9ybECwTunTQ9UHjxMWcNaIjOykIcOS9f7GXf42r9erD/62UyNSU9o8js/r8+9d+GffO6ezYnuco6p9Ndq+rUrf/2HLm4yB/2mCqjAkJiZq06ZNGjJkSIufb9q0SQkJCbbjuFwuuVyBjxmxHBF6z73wgZ5ceJU++rtHW/77scounSP1ymtbJUlPLrpKnprjWvzv70qSfvtyhaZnX6j8ey7Tyhc3q1+fWM25+SKtePGbXd5dOkeqb3Ks/8/Jvbtr6KB4Ha0/of2ewN/YgI7qX6dP1IIHXtDQC5J1wYi+euk363XiRJN+PDVdkpR//28UHx+jOXdcLUla8dw6DR3WR+clx6np5Cm995e/683/977uffA6/5h/XvuhYnt8T4mJsar8ZL+WPPZ7XfrDkbpo/NBwXCK+q7Nos2KoBPWv9N13361Zs2apvLxcl112mT8cVFdXq7S0VMXFxXriiSfaZaII3hvrdqhnbGe5Z1+iuJ5dtX1njW6a87J/I2SvxG4Byw8Hqo9pxpyXNf+uH+qP/5kjT80x/fp35SpasdHfZ8SwRL303P/x/3n+3T+UJL3y+lbNy//jGboy4Lu5fNKFOnzouH617E3V1dZr0JDz9Mui2eoZ102SVH3gsJz/VGo78UWTHv/Fy6qpPiKXK1J9U+K1sGC6Lp90ob9PXW29Che/qkN1xxR3bjddNXmcbr4164xfG0KEwGDh8AW5I2f16tV66qmnVF5erubmZklSRESE0tLS5Ha7dd1119mM0LKU0Y9/q/OAs1nFxtRwTwHokGKi2jeM9b/l5ZCNtfu5a0M2VjgFvQ6QnZ2t7OxsnTx5UrW1tZKkuLg4RUZGhnxyAACgY/jWGwciIyOVlJQUyrkAANAxsCRhwU5DAABMvIfBgldDAwAAW1QYAAAwsSRhQWAAAMBE/d2CWwIAAGxRYQAAwMSmRwsCAwAAJvYwWLAkAQAAbFFhAADA4GNJwoLAAACAifq7BYEBAAATexgsyFAAAMAWFQYAAEzsYbCgwgAAgMnpCN0RpGXLlqlfv36Kjo5Wenq6Nm3a1GrfFStWyOFwBBzR0dEBfW666SZLn0mTJgU9LyoMAAB0EKtXr5bb7VZRUZHS09NVWFiorKws7dy5U/Hx8S2e061bN+3cudP/Z0cL1ZFJkybp17/+tf/PLpcr6LlRYQAAwOQI4RGEJUuWaObMmcrJydGwYcNUVFSkLl26aPny5a1P1eFQYmKi/0hISLD0cblcAX1iY2ODm5gIDAAAWPicjpAdjY2Nqq+vDzgaGxstP7OpqUnl5eXKzMz0tzmdTmVmZqqsrKzVuR4/flx9+/ZVcnKypkyZom3btln6rF+/XvHx8Ro8eLBmz56turq6oO8JgQEAgHZUUFCgmJiYgKOgoMDSr7a2Vs3NzZYKQUJCgjweT4tjDx48WMuXL9drr72mF154QV6vV+PHj9dnn33m7zNp0iStWrVKpaWleuyxx/TOO+/oyiuvVHNzc1DXwR4GAABMIXwPQ25urtxud0Dbt9lD0JKMjAxlZGT4/zx+/HgNHTpUzz77rBYtWiRJuv766/2fjxgxQiNHjtSAAQO0fv16XXbZZW3+WQQGAABMIXys0uVytSkgxMXFKSIiQtXV1QHt1dXVSkxMbNPPioyM1OjRo1VZWdlqn/79+ysuLk6VlZVBBQaWJAAA6ACioqKUlpam0tJSf5vX61VpaWlAFeF0mpubtXXrViUlJbXa57PPPlNdXd1p+7SEwAAAgMkZwiMIbrdbxcXFWrlypbZv367Zs2eroaFBOTk5kqTp06crNzfX33/hwoVat26ddu/erc2bN+vGG2/Uvn37dMstt0j6akPkvHnz9Le//U179+5VaWmppkyZooEDByorKyuoubEkAQCAKUxveszOztbBgweVl5cnj8ejUaNGqaSkxL8RsqqqSk7nNynk8OHDmjlzpjwej2JjY5WWlqb33ntPw4YNkyRFREToo48+0sqVK3XkyBH16tVLV1xxhRYtWhT0PgqHz+fzhe5Sv72U0Y+HewpAh1OxMTXcUwA6pJio4H47Dla/BWtDNtbe/Pad65nCkgQAALDFkgQAACa+3tqCwAAAgMHHt1VasCQBAABsUWEAAMDEr9MWBAYAAEwsSViQoQAAgC0qDAAAmHhKwoLAAACAicBgwZIEAACwRYUBAAATBQYLAgMAAAYfSxIWBAYAAEw8VmnBHgYAAGCLCgMAACaWJCwIDAAAmMgLFixJAAAAW1QYAAAwOPl12oLAAACAgYckrMhQAADAFhUGAAAMVBisCAwAABgcJAYLAgMAAAbyghV7GAAAgC0qDAAAGKgwWBEYAAAwOKi/W3BLAACALSoMAAAYWJKwIjAAAGDgyyqtWJIAAAC2qDAAAGBgScKKwAAAgIHAYMWSBAAAsEWFAQAAA98lYUVgAADAwIubrAgMAAAYKDBYkaEAAIAtKgwAABioMFgRGAAAMBAYrFiSAAAAtqgwAABg4LskrAgMAAAYWJKwYkkCAADYosIAAICBCoMVgQEAAIODTQwWLEkAAABbVBgAADCwJGFFYAAAwEBgsCIwAABgIDBYsYcBAADYosIAAICBhySsCAwAABhYkrBiSQIAANiiwgAAgMHBr9MWBAYAAAwsSViRoQAAgC0qDAAAGByUGCyoMAAAYHA4QncEa9myZerXr5+io6OVnp6uTZs2tdp3xYoVcjgcAUd0dHRAH5/Pp7y8PCUlJalz587KzMzUJ598EvS8CAwAAHQQq1evltvtVn5+vjZv3qzU1FRlZWWppqam1XO6deumAwcO+I99+/YFfP74449r6dKlKioq0saNG9W1a1dlZWXpyy+/DGpuBAYAAAzhqjAsWbJEM2fOVE5OjoYNG6aioiJ16dJFy5cvP81cHUpMTPQfCQkJ/s98Pp8KCwv14IMPasqUKRo5cqRWrVql/fv3a82aNUHNjcAAAIAhlIGhsbFR9fX1AUdjY6PlZzY1Nam8vFyZmZn+NqfTqczMTJWVlbU61+PHj6tv375KTk7WlClTtG3bNv9ne/bskcfjCRgzJiZG6enppx2zJR1m0+OeD6eGewpAh9O5T364pwB0SCeqstp1/FC+GrqgoEALFiwIaMvPz9dDDz0U0FZbW6vm5uaACoEkJSQkaMeOHS2OPXjwYC1fvlwjR47U0aNH9cQTT2j8+PHatm2bzjvvPHk8Hv8Y5phff9ZWHSYwAABwNsrNzZXb7Q5oc7lcIRk7IyNDGRkZ/j+PHz9eQ4cO1bPPPqtFixaF5Gd8jcAAAIAhlBUGl8vVpoAQFxeniIgIVVdXB7RXV1crMTGxTT8rMjJSo0ePVmVlpST5z6uurlZSUlLAmKNGjWrjFXyFPQwAABicDl/IjraKiopSWlqaSktL/W1er1elpaUBVYTTaW5u1tatW/3hICUlRYmJiQFj1tfXa+PGjW0e82tUGAAA6CDcbrdmzJihMWPGaNy4cSosLFRDQ4NycnIkSdOnT1fv3r1VUFAgSVq4cKEuuugiDRw4UEeOHNHixYu1b98+3XLLLZK+eoLijjvu0MMPP6zzzz9fKSkpmj9/vnr16qWpU6cGNTcCAwAAhlAuSQQjOztbBw8eVF5enjwej0aNGqWSkhL/psWqqio5nd8sDhw+fFgzZ86Ux+NRbGys0tLS9N5772nYsGH+Pvfcc48aGho0a9YsHTlyRJdccolKSkosL3iy4/D5fG2vl7SrXeGeANDh8JQE0LITVS+26/g/WrchZGO9ccUlIRsrnNjDAAAAbLEkAQCAIZjNiv9bEBgAADCEaw9DR8aSBAAAsEWFAQAAA79NWxEYAAAwsCRhRWAAAMDgYNOjBVUXAABgiwoDAAAGliSsCAwAABgov1txTwAAgC0qDAAAGHjToxWBAQAAA3sYrFiSAAAAtqgwAABg4LdpKwIDAAAGliSsCFEAAMAWFQYAAAw8JWFFYAAAwMCShBWBAQAAA+v1VtwTAABgiwoDAAAG9jBYERgAADCwh8GKJQkAAGCLCgMAAAYqDFYEBgAADJTfrbgnAADAFhUGAAAMPCVhRWAAAMDAHgYrliQAAIAtKgwAABj4bdqKwAAAgIElCSsCAwAABgebHi2ougAAAFtUGAAAMLAkYUVgAADAQPndinsCAABsUWEAAMDAmx6tCAwAABjYw2DFkgQAALBFhQEAAAMVBisCAwAAhohwT6ADYkkCAADYosIAAICBpySsCAwAABjYw2BFYAAAwEBgsGIPAwAAsEWFAQAAQwQVBgsCAwAABpYkrFiSAAAAtqgwAABg4LFKKwIDAAAGliSsWJIAAAC2qDAAAGDguySsCAwAABhYkrBiSQIAANiiwgAAgIGnJKwIDAAAGHjToxVLEgAAGJyO0B3BWrZsmfr166fo6Gilp6dr06ZNbTrvpZdeksPh0NSpUwPab7rpJjkcjoBj0qRJQc+LwAAAQAexevVqud1u5efna/PmzUpNTVVWVpZqampOe97evXt19913a8KECS1+PmnSJB04cMB/vPjii0HPjcAAAIAhXBWGJUuWaObMmcrJydGwYcNUVFSkLl26aPny5a2e09zcrBtuuEELFixQ//79W+zjcrmUmJjoP2JjY4ObmAgMAABYhDIwNDY2qr6+PuBobGy0/MympiaVl5crMzPzm3k4ncrMzFRZWVmrc124cKHi4+N18803t9pn/fr1io+P1+DBgzV79mzV1dUFf0+CPgMAALRZQUGBYmJiAo6CggJLv9raWjU3NyshISGgPSEhQR6Pp8WxN2zYoOeff17FxcWt/vxJkyZp1apVKi0t1WOPPaZ33nlHV155pZqbm4O6Dp6SAADAEBHCxypzc3PldrsD2lwu13ce99ixY5o2bZqKi4sVFxfXar/rr7/e/79HjBihkSNHasCAAVq/fr0uu+yyNv88AgMAAIZQlt9dLlebAkJcXJwiIiJUXV0d0F5dXa3ExERL/08//VR79+7V5MmT/W1er1eS1KlTJ+3cuVMDBgywnNe/f3/FxcWpsrIyqMDAkgQAAB1AVFSU0tLSVFpa6m/zer0qLS1VRkaGpf+QIUO0detWVVRU+I+rr75aEydOVEVFhZKTk1v8OZ999pnq6uqUlJQU1PyoMAAAYAjXd0m43W7NmDFDY8aM0bhx41RYWKiGhgbl5ORIkqZPn67evXuroKBA0dHRGj58eMD53bt3lyR/+/Hjx7VgwQL99Kc/VWJioj799FPdc889GjhwoLKysoKaG4EBAABDuAJDdna2Dh48qLy8PHk8Ho0aNUolJSX+jZBVVVVyOtu+OBAREaGPPvpIK1eu1JEjR9SrVy9dccUVWrRoUdD7KBw+n6+DvDB7V7gnAHQ4nfvkh3sKQId0oir4Fw8F45U9JSEb65qU4N+q2BFRYQAAwBDKpyTOFgQGAAAM4VqS6MgIDAAAGAgMVjxWCQAAbFFhAADAQIXBisAAAIAhgsBgwZIEAACwRYUBAACDk8cqLQgMAAAYKL9bcU8AAIAtKgwAABh4SsKKwHCW++1v39Dzz/9eBw8e1pAhKZo//+caOXJQi33XrXtPRUUvq6rqgE6dOqW+fXspJ2eqpk79ob/P4MGTWzx33rwc3XLLv7TLNQDt4efTL9edP5+shHNjtHV7ldx5K/TBlk9b7R/TrYsempetKVeOVY+Y76nq81rNW7BKa9+ukCTNvDFTM6ddrr7nxUmStu/6TI/88vdat37LmbgchBhPSVgRGM5ib775FxUUPKcFC+YoNXWQVq58XTffnKeSkiL17Nnd0j8m5hzNnn2d+vc/T5GRnfT22+/r/vt/qZ49u2vChAslSRs2rAo45913y/XAA0uVlTX+TFwSEBLXTL5Ij82fprn3P6/3Kyr1bzdfqddfuE+pP7hLB+vqLf0jIyP0xm/vV01tvW64tVCfew6pT+9zdbS+wd/nc88hzX/0RVXu8cjhkG685vt6+bm7ddFVudq+67MzeXlAu+DbKs9i1157l0aMOF95ebdKkrxery69NEfTpv1Ys2Zd26YxfvKT23XppWN1xx03tvj5bbc9rIaGE1q58hchmze+wbdVto93X1uk8i2f6s68FZIkh8Ohyo1P6z9WrNUTz7xu6X/LjZm68+c/VurEu3TqVHObf87nHxXr/l/8VitXrw/RzPG19v62yr943gjZWBMSfxSyscKJTY9nqaamk9q2rVLjx6f625xOp8aPH6UPP9xpe77P51NZ2Rbt2fO5xo69oMU+tbWH9c47H+iaay4P2byB9hYZGaHRI1L01oaP/W0+n09vbfhY4y48v8VzfpR5oTaWf6LCh3O0t7xIH/zpcc2bM0XOVha6nU6Hrp2coa6dXdq4+ZN2uQ60L6cjdMfZgiWJs9Thw/VqbvaqZ8/YgPaePbtr9+7Wy6PHjjXo+9+/SU1NJ+V0OpWfP1sXXzy6xb6vvvqWunbtrCuuYDkC/3PE9eimTp0iVFN7NKC9pvaoBg/o1eI5KX3i9YPx5+qlNX/VT256TAP6Jajw4Z8pMrKTHin8L3+/CwYna/2ahYp2Rep4w5fKnrVEOz75vF2vB+3jbPqHPlRCHhj+8Y9/KD8/X8uXL2+1T2NjoxobGwPaXK4muVxRoZ4OgtS1a2etWfNLffHFlyor26JHH31eycmJSk8fYen7X//1J02e/AP+f8NZz+l06mBdvebcVyyv16cPt+5Rr4QeuuPWHwcEhl279yt90n2K6dZFP7kqXcVLZuuK6xYSGnBWCPmSxKFDh7Ry5crT9ikoKFBMTEzAUVDwbKin8r9abGw3RUQ4VVd3OKC9ru6I4uJiWznrq/8w9u3bS0OH9tfPfvYTZWWN169+9bKl3wcfbNOePZ/r2muvCPncgfZUe6hep041Kz4uJqA9Pi5GnoNHWjzHU3NEn+w5IK/3my1fOyo/V1J8rCIjI/xtJ082a/e+an24dY/yHntJW7fv05yfTWqX60D7cobwOFsEXWF4/XXrhqB/tnv3btsxcnNz5Xa7A9pcrqpgp4LTiIqK1AUXDFRZ2UfKzMyQ9NWmx7KyLbrxxrZvwPF6fWpqOmlpf+WVdbrggoEaMiQlZHMGzoSTJ5v14dY9mnjxcP1h3QeSvtr0OPHiC1S0cl2L55R9sFPZUy6Ww+HQ1/vEz++fpAPVh3XyZOubIJ0Op1xRkaG/CLQ7B0sSFkEHhqlTpwb8pWmJw+ZOu1wuuVwuo5Wydqjl5EzVvfc+peHDB2rkyEFaufI1nTjxpf7lXzIlSffcs0QJCT11110zJEnPPvuyhg8fqD59ktTUdFLvvPOBXn/9bT300OyAcY8f/0IlJX/VvffefMavCQiFpc+9oeInZ6t862598N+PVXbp4tKq/3xHkvTcU7O133NYeY+9JEkq/s2fdOuMK/TkQzP0zIoSDUxJ0rw5U/XMr0v8Yy6893qtfbtC/9hfq3O6dlb21Iv1/Yyhmjzt0bBcIxBqQQeGpKQkPfPMM5oyZUqLn1dUVCgtLe07Twzf3VVXTdChQ0e1dOlvdfDgYQ0d2l/PPbfAvyRx4MDBgF3eX3zxpRYs+A95PHWKjo5S//7nafHiu3TVVRMCxn3jjXfl8/n04x9//4xeDxAqr/zhb4rr0U157muUcG53ffT3fZoy7VH/RsjkXnEByw+fHTikq6c9qsfzpun9tY9pf/VhLVv+Rz35H99UXM/t2U3PP3WbEuO76+ixL/TxjipNnvao3vrL1jN+ffjuKDBYBf0ehquvvlqjRo3SwoULW/x8y5YtGj16tLxeb5BT4T0MgIn3MAAta+/3MHxQG7r3MIyJOzvewxB0hWHevHlqaGho9fOBAwfq7bff/k6TAgAAHUvQgWHChAmn/bxr16669NJLv/WEAAAIt7Pp6YZQ4cVNAAAYHI4O8q0JHQghCgAA2KLCAACAgackrAgMAAAYeHGTFYEBAAADecGKPQwAAMAWFQYAAAx8vbUVgQEAAAN5wYolCQAAYIsKAwAABp6SsCIwAABgIC9YsSQBAABsUWEAAMBAhcGKwAAAgIHHKq1YkgAAALaoMAAAYKDAYEVgAADA4HD4wj2FDofAAACAgQqDFXsYAACALSoMAAAYeNOjFYEBAAAD5Xcr7gkAALBFhQEAAANLElYEBgAADOQFK5YkAACALSoMAAAYWJKwIjAAAGAgL1ixJAEAAGxRYQAAwMDXW1sRGAAAMJAXrAgMAAAY+LZKK/YwAAAAW1QYAAAwsCRhRYUBAACDwxG6I1jLli1Tv379FB0drfT0dG3atKlN57300ktyOByaOnVqQLvP51NeXp6SkpLUuXNnZWZm6pNPPgl6XgQGAAA6iNWrV8vtdis/P1+bN29WamqqsrKyVFNTc9rz9u7dq7vvvlsTJkywfPb4449r6dKlKioq0saNG9W1a1dlZWXpyy+/DGpuBAYAAAyOEB7BWLJkiWbOnKmcnBwNGzZMRUVF6tKli5YvX97qOc3Nzbrhhhu0YMEC9e/fP+Azn8+nwsJCPfjgg5oyZYpGjhypVatWaf/+/VqzZk1QcyMwAABgcIbwaGxsVH19fcDR2Nho+ZlNTU0qLy9XZmbmN/NwOpWZmamysrJW57pw4ULFx8fr5ptvtny2Z88eeTyegDFjYmKUnp5+2jFbQmAAAKAdFRQUKCYmJuAoKCiw9KutrVVzc7MSEhIC2hMSEuTxeFoce8OGDXr++edVXFzc4udfnxfMmK3hKQkAAAyh/PKp3Nxcud3ugDaXy/Wdxz127JimTZum4uJixcXFfefx7BAYAACwCF1icLlcbQoIcXFxioiIUHV1dUB7dXW1EhMTLf0//fRT7d27V5MnT/a3eb1eSVKnTp20c+dO/3nV1dVKSkoKGHPUqFFBXQdLEgAAdABRUVFKS0tTaWmpv83r9aq0tFQZGRmW/kOGDNHWrVtVUVHhP66++mpNnDhRFRUVSk5OVkpKihITEwPGrK+v18aNG1sc83SoMAAAYHCE6dVNbrdbM2bM0JgxYzRu3DgVFhaqoaFBOTk5kqTp06erd+/eKigoUHR0tIYPHx5wfvfu3SUpoP2OO+7Qww8/rPPPP18pKSmaP3++evXqZXlfgx0CAwAABocjPAX47OxsHTx4UHl5efJ4PBo1apRKSkr8mxarqqrkdAY3t3vuuUcNDQ2aNWuWjhw5oksuuUQlJSWKjo4OahyHz+frIN+wsSvcEwA6nM598sM9BaBDOlH1YruOf6TpjyEbq3vUlSEbK5zYwwAAAGyxJAEAgCFcexg6MgIDAAAWBAYTSxIAAMAWFQYAAAzhekqiIyMwAABgwZKEiQgFAABsUWEAAMDAUxJWBAYAAAwEBiuWJAAAgC0qDAAAWPD7tInAAACAweFgScJEYAAAwILAYKLmAgAAbFFhAADAwFMSVgQGAAAsKMCbuCMAAMAWFQYAAAwsSVgRGAAAMPBYpRVLEgAAwBYVBgAALKgwmAgMAAAYHBTgLbgjAADAFhUGAAAsWJIwERgAADDwlIQVgQEAAAsCg4k9DAAAwBYVBgAADDwlYUVgAADAgiUJExEKAADYosIAAICBL5+yIjAAAGDgsUorliQAAIAtKgwAAFjw+7SJwAAAgIE9DFZEKAAAYIsKAwAAFlQYTAQGAAAMPCVhRWAAAMCCFXsTdwQAANiiwgAAgIGnJKwcPp/PF+5JoONobGxUQUGBcnNz5XK5wj0doEPg7wVAYIChvr5eMTExOnr0qLp16xbu6QAdAn8vAPYwAACANiAwAAAAWwQGAABgi8CAAC6XS/n5+WzsAv4Jfy8ANj0CAIA2oMIAAABsERgAAIAtAgMAALBFYAAAALYIDPBbtmyZ+vXrp+joaKWnp2vTpk3hnhIQVu+++64mT56sXr16yeFwaM2aNeGeEhA2BAZIklavXi232638/Hxt3rxZqampysrKUk1NTbinBoRNQ0ODUlNTtWzZsnBPBQg7HquEJCk9PV1jx47V008/LUnyer1KTk7W3Llzdd9994V5dkD4ORwOvfrqq5o6dWq4pwKEBRUGqKmpSeXl5crMzPS3OZ1OZWZmqqysLIwzAwB0FAQGqLa2Vs3NzUpISAhoT0hIkMfjCdOsAAAdCYEBAADYIjBAcXFxioiIUHV1dUB7dXW1EhMTwzQrAEBHQmCAoqKilJaWptLSUn+b1+tVaWmpMjIywjgzAEBH0SncE0DH4Ha7NWPGDI0ZM0bjxo1TYWGhGhoalJOTE+6pAWFz/PhxVVZW+v+8Z88eVVRUqEePHurTp08YZwaceTxWCb+nn35aixcvlsfj0ahRo7R06VKlp6eHe1pA2Kxfv14TJ060tM+YMUMrVqw48xMCwojAAAAAbLGHAQAA2CIwAAAAWwQGAABgi8AAAABsERgAAIAtAgMAALBFYAAAALYIDAAAwBaBAQAA2CIwAAAAWwQGAABgi8AAAABs/X8gPeRcl2NbNgAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# get performance metrics\n",
    "print('Performance on Test Set')\n",
    "our_metrics(y_test_tree, y_pred_eec_best)\n",
    "# print confusion matrix\n",
    "# print(confusion_matrix(y_test, y_pred))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The model with the tuned hyperparameters performs slightly better (w.r.t. all the decision metrics) than the EEC with the default settings."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test ROC AUC Score: 0.6210741899986002\n"
     ]
    }
   ],
   "source": [
    "# roc-auc-score of train (small subsample)\n",
    "print(f'Test ROC AUC Score: {roc_auc_score(y_test_tree, y_pred_eec_best)}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Hyperparameter tuning with the F1.5 Scorer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Instantiate random search and define the metric to optimize \n",
    "rs_f15 = RandomizedSearchCV(EasyEnsembleClassifier(random_state=RSEED), param_grid, scoring=f15_scorer,\n",
    "                  cv=5, verbose=5, n_jobs=1, n_iter=10, random_state=RSEED) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 5 folds for each of 10 candidates, totalling 50 fits\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 1/5] END n_estimators=14, replacement=True, sampling_strategy=auto;, score=0.273 total time= 1.1min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 2/5] END n_estimators=14, replacement=True, sampling_strategy=auto;, score=0.274 total time= 1.1min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 3/5] END n_estimators=14, replacement=True, sampling_strategy=auto;, score=0.278 total time= 1.1min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 4/5] END n_estimators=14, replacement=True, sampling_strategy=auto;, score=0.274 total time= 1.1min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 5/5] END n_estimators=14, replacement=True, sampling_strategy=auto;, score=0.279 total time= 1.1min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 1/5] END n_estimators=10, replacement=True, sampling_strategy=auto;, score=0.273 total time=  47.9s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 2/5] END n_estimators=10, replacement=True, sampling_strategy=auto;, score=0.274 total time=  49.7s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 3/5] END n_estimators=10, replacement=True, sampling_strategy=auto;, score=0.277 total time=  48.6s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 4/5] END n_estimators=10, replacement=True, sampling_strategy=auto;, score=0.274 total time=  48.1s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 5/5] END n_estimators=10, replacement=True, sampling_strategy=auto;, score=0.279 total time=  48.0s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 1/5] END n_estimators=12, replacement=True, sampling_strategy=1;, score=0.273 total time=  57.0s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 2/5] END n_estimators=12, replacement=True, sampling_strategy=1;, score=0.274 total time=  57.1s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 3/5] END n_estimators=12, replacement=True, sampling_strategy=1;, score=0.278 total time=  57.1s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 4/5] END n_estimators=12, replacement=True, sampling_strategy=1;, score=0.274 total time=  56.8s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 5/5] END n_estimators=12, replacement=True, sampling_strategy=1;, score=0.279 total time=  56.9s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 1/5] END n_estimators=10, replacement=True, sampling_strategy=1;, score=0.273 total time=  47.8s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 2/5] END n_estimators=10, replacement=True, sampling_strategy=1;, score=0.274 total time=  48.0s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 3/5] END n_estimators=10, replacement=True, sampling_strategy=1;, score=0.277 total time=  47.4s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 4/5] END n_estimators=10, replacement=True, sampling_strategy=1;, score=0.274 total time=  47.3s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 5/5] END n_estimators=10, replacement=True, sampling_strategy=1;, score=0.279 total time=  46.9s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 1/5] END n_estimators=8, replacement=False, sampling_strategy=1;, score=0.274 total time=  37.9s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 2/5] END n_estimators=8, replacement=False, sampling_strategy=1;, score=0.274 total time=  37.8s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 3/5] END n_estimators=8, replacement=False, sampling_strategy=1;, score=0.278 total time=  37.9s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 4/5] END n_estimators=8, replacement=False, sampling_strategy=1;, score=0.273 total time=  37.9s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 5/5] END n_estimators=8, replacement=False, sampling_strategy=1;, score=0.278 total time=  38.2s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 1/5] END n_estimators=8, replacement=True, sampling_strategy=auto;, score=0.274 total time=  38.0s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 2/5] END n_estimators=8, replacement=True, sampling_strategy=auto;, score=0.274 total time=  38.6s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 3/5] END n_estimators=8, replacement=True, sampling_strategy=auto;, score=0.277 total time=  38.2s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 4/5] END n_estimators=8, replacement=True, sampling_strategy=auto;, score=0.274 total time=  38.3s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 5/5] END n_estimators=8, replacement=True, sampling_strategy=auto;, score=0.279 total time=  38.1s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 1/5] END n_estimators=14, replacement=True, sampling_strategy=0.8;, score=0.270 total time= 1.2min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 2/5] END n_estimators=14, replacement=True, sampling_strategy=0.8;, score=0.272 total time= 1.2min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 3/5] END n_estimators=14, replacement=True, sampling_strategy=0.8;, score=0.273 total time= 1.2min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 4/5] END n_estimators=14, replacement=True, sampling_strategy=0.8;, score=0.269 total time= 1.2min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 5/5] END n_estimators=14, replacement=True, sampling_strategy=0.8;, score=0.274 total time= 1.2min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 1/5] END n_estimators=14, replacement=False, sampling_strategy=auto;, score=0.273 total time= 1.1min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 2/5] END n_estimators=14, replacement=False, sampling_strategy=auto;, score=0.274 total time= 1.1min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 3/5] END n_estimators=14, replacement=False, sampling_strategy=auto;, score=0.277 total time= 1.1min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 4/5] END n_estimators=14, replacement=False, sampling_strategy=auto;, score=0.273 total time= 1.1min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 5/5] END n_estimators=14, replacement=False, sampling_strategy=auto;, score=0.278 total time= 1.1min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 1/5] END n_estimators=10, replacement=False, sampling_strategy=auto;, score=0.274 total time=  47.8s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 2/5] END n_estimators=10, replacement=False, sampling_strategy=auto;, score=0.274 total time=  47.7s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 3/5] END n_estimators=10, replacement=False, sampling_strategy=auto;, score=0.278 total time=  47.4s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 4/5] END n_estimators=10, replacement=False, sampling_strategy=auto;, score=0.274 total time=  47.3s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 5/5] END n_estimators=10, replacement=False, sampling_strategy=auto;, score=0.278 total time=  48.0s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 1/5] END n_estimators=6, replacement=False, sampling_strategy=auto;, score=0.274 total time=  28.7s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 2/5] END n_estimators=6, replacement=False, sampling_strategy=auto;, score=0.274 total time=  28.6s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 3/5] END n_estimators=6, replacement=False, sampling_strategy=auto;, score=0.278 total time=  28.8s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 4/5] END n_estimators=6, replacement=False, sampling_strategy=auto;, score=0.274 total time=  28.8s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 5/5] END n_estimators=6, replacement=False, sampling_strategy=auto;, score=0.278 total time=  29.0s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lanacasselmann/neuefische/fraud_detection_electricity_company/.venv/lib/python3.11/site-packages/sklearn/ensemble/_bagging.py:802: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  y = column_or_1d(y, warn=True)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-3 {color: black;background-color: white;}#sk-container-id-3 pre{padding: 0;}#sk-container-id-3 div.sk-toggleable {background-color: white;}#sk-container-id-3 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-3 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-3 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-3 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-3 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-3 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-3 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-3 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-3 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-3 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-3 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-3 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-3 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-3 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-3 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-3 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-3 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-3 div.sk-item {position: relative;z-index: 1;}#sk-container-id-3 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-3 div.sk-item::before, #sk-container-id-3 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-3 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-3 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-3 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-3 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-3 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-3 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-3 div.sk-label-container {text-align: center;}#sk-container-id-3 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-3 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-3\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>RandomizedSearchCV(cv=5, estimator=EasyEnsembleClassifier(random_state=42),\n",
       "                   n_jobs=1,\n",
       "                   param_distributions={&#x27;n_estimators&#x27;: [6, 8, 10, 12, 14],\n",
       "                                        &#x27;replacement&#x27;: [False, True],\n",
       "                                        &#x27;sampling_strategy&#x27;: [&#x27;auto&#x27;, 0.8, 1]},\n",
       "                   random_state=42, scoring=make_scorer(fbeta_score, beta=1.5),\n",
       "                   verbose=5)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-5\" type=\"checkbox\" ><label for=\"sk-estimator-id-5\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">RandomizedSearchCV</label><div class=\"sk-toggleable__content\"><pre>RandomizedSearchCV(cv=5, estimator=EasyEnsembleClassifier(random_state=42),\n",
       "                   n_jobs=1,\n",
       "                   param_distributions={&#x27;n_estimators&#x27;: [6, 8, 10, 12, 14],\n",
       "                                        &#x27;replacement&#x27;: [False, True],\n",
       "                                        &#x27;sampling_strategy&#x27;: [&#x27;auto&#x27;, 0.8, 1]},\n",
       "                   random_state=42, scoring=make_scorer(fbeta_score, beta=1.5),\n",
       "                   verbose=5)</pre></div></div></div><div class=\"sk-parallel\"><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-6\" type=\"checkbox\" ><label for=\"sk-estimator-id-6\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">estimator: EasyEnsembleClassifier</label><div class=\"sk-toggleable__content\"><pre>EasyEnsembleClassifier(random_state=42)</pre></div></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-7\" type=\"checkbox\" ><label for=\"sk-estimator-id-7\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">EasyEnsembleClassifier</label><div class=\"sk-toggleable__content\"><pre>EasyEnsembleClassifier(random_state=42)</pre></div></div></div></div></div></div></div></div></div></div>"
      ],
      "text/plain": [
       "RandomizedSearchCV(cv=5, estimator=EasyEnsembleClassifier(random_state=42),\n",
       "                   n_jobs=1,\n",
       "                   param_distributions={'n_estimators': [6, 8, 10, 12, 14],\n",
       "                                        'replacement': [False, True],\n",
       "                                        'sampling_strategy': ['auto', 0.8, 1]},\n",
       "                   random_state=42, scoring=make_scorer(fbeta_score, beta=1.5),\n",
       "                   verbose=5)"
      ]
     },
     "execution_count": 125,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Fit randomized search object to data\n",
    "rs_f15.fit(X_train_tree_small, y_train_tree_small)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 126,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best score (F1.5): 0.276\n",
      "Best parameters (F1.5): {'sampling_strategy': 'auto', 'replacement': False, 'n_estimators': 6}\n"
     ]
    }
   ],
   "source": [
    "# Best score\n",
    "print('Best score (F1.5):', round(rs_f15.best_score_, 3))\n",
    "\n",
    "# Best parameters\n",
    "print('Best parameters (F1.5):', rs_f15.best_params_)\n",
    "\n",
    "# Best score (F2): 0.344\n",
    "# Best parameters: {'sampling_strategy': 'auto', 'replacement': False, 'n_estimators': 6}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The best hyper parameters are the same as with F2 scorer: {'sampling_strategy': 'auto', 'replacement': False, 'n_estimators': 6}. Hence, there is no need to train a new model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 127,
   "metadata": {},
   "outputs": [],
   "source": [
    "# # Assigning the fitted EasyEnsemble model with best parameter combination to a new variable sgd_best_rs\n",
    "# eec_best_rs_15 = rs_f15.best_estimator_\n",
    "# # Making predictions on the test set\n",
    "# y_pred_test_eec_rs_f15 = eec_best_rs_15.predict(X_test_tree)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "metadata": {},
   "outputs": [],
   "source": [
    "# pd.DataFrame(rs_f15.cv_results_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training an EasyEnsembleClassifier with the best Hyperparameters\n",
    "\n",
    "There is no need to train a new model since the hyperparameters are the same, we would obtain the same result."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 129,
   "metadata": {},
   "outputs": [],
   "source": [
    "# # train an EEC on the whole data set with the hyperparameters determined in the F1.5-scored RandomizedSearch \n",
    "# # with the partial data set.\n",
    "\n",
    "# # initiate EasyEnsembleClassifier with the best hyperparameters\n",
    "# eec_best_f15 = EasyEnsembleClassifier(random_state=RSEED, sampling_strategy='auto', replacement=False, n_estimators=6)\n",
    "# # fit eec to train data\n",
    "# eec_best_f15.fit(X_train_tree, y_train_tree)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# # make prediction on test data\n",
    "# y_pred_eec_best_f15 = eec_best_f15.predict(X_test_tree)\n",
    "# # make prediction on train data\n",
    "# y_pred_train_eec_best_f15 = eec_best_f15.predict(X_train_tree)\n",
    "# print('Performance on Train Set')\n",
    "# our_metrics(y_train_tree, y_pred_train_eec_best_f15)\n",
    "# # get performance metrics\n",
    "# print('Performance on Test Set')\n",
    "# our_metrics(y_test_tree, y_pred_eec_best_f15)\n",
    "# # print confusion matrix\n",
    "# # print(confusion_matrix(y_test, y_pred))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## GridSearch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#eec_rs = EasyEnsembleClassifier(random_state=RSEED)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# GridSearch\n",
    "# gs = GridSearchCV(SGDClassifier(random_state=RSEED), param_grid, scoring='accuracy',\n",
    "#                   cv=5, verbose=5, n_jobs=-1)\n",
    "\n",
    "# # Fit gridsearch object to data.\n",
    "# gs.fit(X_train_preprocessed, y_train)\n",
    "\n",
    "# # Best score\n",
    "# print('Best score:', round(gs.best_score_, 3))\n",
    "\n",
    "# # Best parameters\n",
    "# print('Best parameters:', gs.best_params_)\n",
    "\n",
    "# # Assigning the fitted SGDClassifier model with best parameter combination to a new variable sgd_best\n",
    "# sgd_best = gs.best_estimator_\n",
    "\n",
    "# # Making predictions on the test set\n",
    "# y_pred_test = sgd_best.predict(X_test_preprocessed)\n",
    "# # Let us print out the performance of our model on the test set.\n",
    "# sgd_accuracy = print_pretty_summary('SGDClassifier model', sgd_best, y_test, y_pred_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Error Analysis for the trained Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# eec_best: EEC with the best hyperparameters, trained on X_train_tree, y_train_tree\n",
    "# y_pred_eec_best predictions of eec_best on X_test_tree\n",
    "# y_pres_train_eec_best: predictions of eec_best on y_train_tree"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# plot roc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "____________________\n",
      "Confusion Matrix :\n",
      "[[  19064 1011832]\n",
      " [    153   88124]]\n",
      "____________________\n",
      "Accuracy Score : 0.0957742904805602\n",
      "____________________\n",
      "Report : \n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.99      0.02      0.04   1030896\n",
      "           1       0.08      1.00      0.15     88277\n",
      "\n",
      "    accuracy                           0.10   1119173\n",
      "   macro avg       0.54      0.51      0.09   1119173\n",
      "weighted avg       0.92      0.10      0.05   1119173\n",
      "\n",
      "____________________\n"
     ]
    }
   ],
   "source": [
    "test_proba_49 = class_metrics_var_threshold(eec_best, X_test_tree, y_test_tree, threshold=0.49)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Changing the threshold slightly does not improve the model:\n",
    "* changing the threshold to 0.6 is equivalent to predicting no one commits fraud\n",
    "* changing the threshold to 0.49 yields a lot of true positives, hardly any false negatives. However, this comes at the price of a lot of false positive predictions. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "____________________\n",
      "Confusion Matrix :\n",
      "[[625964 404932]\n",
      " [ 32226  56051]]\n",
      "____________________\n",
      "Accuracy Score : 0.6093919349376727\n",
      "____________________\n",
      "Report : \n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.95      0.61      0.74   1030896\n",
      "           1       0.12      0.63      0.20     88277\n",
      "\n",
      "    accuracy                           0.61   1119173\n",
      "   macro avg       0.54      0.62      0.47   1119173\n",
      "weighted avg       0.89      0.61      0.70   1119173\n",
      "\n",
      "____________________\n"
     ]
    }
   ],
   "source": [
    "# get scorers for different thresholds\n",
    "test_proba = class_metrics_var_threshold(eec_best, X_test_tree, y_test_tree, threshold=0.5)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Error Analysis on Test Set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "numpy.ndarray"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(y_test_tree)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "# turn test_proba into DataFrame\n",
    "df_proba = pd.DataFrame(data=test_proba, columns=['no_fraud_proba','fraud_proba'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_y_test = pd.DataFrame(data=y_test_tree, columns=['target'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_y_test_pred = pd.DataFrame(data=y_pred_eec_best, columns=['fraud_pred'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "# make DataFrame containing the test data and predictions of it\n",
    "df_test = pd.concat([X_test_tree, df_y_test, df_y_test_pred, df_proba], axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>member_years</th>\n",
       "      <th>index_change_month</th>\n",
       "      <th>quantity_counters</th>\n",
       "      <th>client_catg_11</th>\n",
       "      <th>client_catg_12</th>\n",
       "      <th>client_catg_51</th>\n",
       "      <th>region_101</th>\n",
       "      <th>region_103</th>\n",
       "      <th>region_104</th>\n",
       "      <th>region_105</th>\n",
       "      <th>...</th>\n",
       "      <th>reading_remarque_6</th>\n",
       "      <th>reading_remarque_7</th>\n",
       "      <th>reading_remarque_8</th>\n",
       "      <th>reading_remarque_9</th>\n",
       "      <th>counter_type_ELEC</th>\n",
       "      <th>counter_type_GAZ</th>\n",
       "      <th>target</th>\n",
       "      <th>fraud_pred</th>\n",
       "      <th>no_fraud_proba</th>\n",
       "      <th>fraud_proba</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>11</td>\n",
       "      <td>17.000</td>\n",
       "      <td>58</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.499381</td>\n",
       "      <td>0.500619</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5</td>\n",
       "      <td>20.375</td>\n",
       "      <td>68</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.496718</td>\n",
       "      <td>0.503282</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4</td>\n",
       "      <td>0.000</td>\n",
       "      <td>48</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.498331</td>\n",
       "      <td>0.501669</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>28</td>\n",
       "      <td>415.500</td>\n",
       "      <td>62</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.499155</td>\n",
       "      <td>0.500845</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>20</td>\n",
       "      <td>5.750</td>\n",
       "      <td>60</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.495342</td>\n",
       "      <td>0.504658</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 57 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   member_years  index_change_month  quantity_counters  client_catg_11   \n",
       "0            11              17.000                 58               1  \\\n",
       "1             5              20.375                 68               1   \n",
       "2             4               0.000                 48               0   \n",
       "3            28             415.500                 62               1   \n",
       "4            20               5.750                 60               1   \n",
       "\n",
       "   client_catg_12  client_catg_51  region_101  region_103  region_104   \n",
       "0               0               0           0           0           0  \\\n",
       "1               0               0           0           1           0   \n",
       "2               0               1           0           0           1   \n",
       "3               0               0           1           0           0   \n",
       "4               0               0           0           1           0   \n",
       "\n",
       "   region_105  ...  reading_remarque_6  reading_remarque_7   \n",
       "0           0  ...                   1                   0  \\\n",
       "1           0  ...                   1                   0   \n",
       "2           0  ...                   1                   0   \n",
       "3           0  ...                   1                   0   \n",
       "4           0  ...                   0                   0   \n",
       "\n",
       "   reading_remarque_8  reading_remarque_9  counter_type_ELEC   \n",
       "0                   0                   0                  0  \\\n",
       "1                   0                   0                  0   \n",
       "2                   0                   0                  0   \n",
       "3                   0                   0                  1   \n",
       "4                   1                   0                  0   \n",
       "\n",
       "   counter_type_GAZ  target  fraud_pred  no_fraud_proba  fraud_proba  \n",
       "0                 1       0           1        0.499381     0.500619  \n",
       "1                 1       0           1        0.496718     0.503282  \n",
       "2                 1       0           1        0.498331     0.501669  \n",
       "3                 0       0           1        0.499155     0.500845  \n",
       "4                 1       0           1        0.495342     0.504658  \n",
       "\n",
       "[5 rows x 57 columns]"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_test.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_test['pred_corr']= (df_test['target'] == df_test['fraud_pred'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_test['pred_corr']= (df_test['target'] == df_test['fraud_pred'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>member_years</th>\n",
       "      <th>index_change_month</th>\n",
       "      <th>quantity_counters</th>\n",
       "      <th>client_catg_11</th>\n",
       "      <th>client_catg_12</th>\n",
       "      <th>client_catg_51</th>\n",
       "      <th>region_101</th>\n",
       "      <th>region_103</th>\n",
       "      <th>region_104</th>\n",
       "      <th>region_105</th>\n",
       "      <th>...</th>\n",
       "      <th>reading_remarque_7</th>\n",
       "      <th>reading_remarque_8</th>\n",
       "      <th>reading_remarque_9</th>\n",
       "      <th>counter_type_ELEC</th>\n",
       "      <th>counter_type_GAZ</th>\n",
       "      <th>target</th>\n",
       "      <th>fraud_pred</th>\n",
       "      <th>no_fraud_proba</th>\n",
       "      <th>fraud_proba</th>\n",
       "      <th>pred_corr</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>11</td>\n",
       "      <td>17.000</td>\n",
       "      <td>58</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.499381</td>\n",
       "      <td>0.500619</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5</td>\n",
       "      <td>20.375</td>\n",
       "      <td>68</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.496718</td>\n",
       "      <td>0.503282</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4</td>\n",
       "      <td>0.000</td>\n",
       "      <td>48</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.498331</td>\n",
       "      <td>0.501669</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>28</td>\n",
       "      <td>415.500</td>\n",
       "      <td>62</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.499155</td>\n",
       "      <td>0.500845</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>20</td>\n",
       "      <td>5.750</td>\n",
       "      <td>60</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.495342</td>\n",
       "      <td>0.504658</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 58 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   member_years  index_change_month  quantity_counters  client_catg_11   \n",
       "0            11              17.000                 58               1  \\\n",
       "1             5              20.375                 68               1   \n",
       "2             4               0.000                 48               0   \n",
       "3            28             415.500                 62               1   \n",
       "4            20               5.750                 60               1   \n",
       "\n",
       "   client_catg_12  client_catg_51  region_101  region_103  region_104   \n",
       "0               0               0           0           0           0  \\\n",
       "1               0               0           0           1           0   \n",
       "2               0               1           0           0           1   \n",
       "3               0               0           1           0           0   \n",
       "4               0               0           0           1           0   \n",
       "\n",
       "   region_105  ...  reading_remarque_7  reading_remarque_8   \n",
       "0           0  ...                   0                   0  \\\n",
       "1           0  ...                   0                   0   \n",
       "2           0  ...                   0                   0   \n",
       "3           0  ...                   0                   0   \n",
       "4           0  ...                   0                   1   \n",
       "\n",
       "   reading_remarque_9  counter_type_ELEC  counter_type_GAZ  target   \n",
       "0                   0                  0                 1       0  \\\n",
       "1                   0                  0                 1       0   \n",
       "2                   0                  0                 1       0   \n",
       "3                   0                  1                 0       0   \n",
       "4                   0                  0                 1       0   \n",
       "\n",
       "   fraud_pred  no_fraud_proba  fraud_proba  pred_corr  \n",
       "0           1        0.499381     0.500619      False  \n",
       "1           1        0.496718     0.503282      False  \n",
       "2           1        0.498331     0.501669      False  \n",
       "3           1        0.499155     0.500845      False  \n",
       "4           1        0.495342     0.504658      False  \n",
       "\n",
       "[5 rows x 58 columns]"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_test.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_false_pred = df_test.query('target != fraud_pred')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pd.set_option('display.max_columns', None)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# regions not of relevance\n",
    "# 305      94276\n",
    "# 307      85531\n",
    "# 313      76924\n",
    "# 105      53131\n",
    "# 371      53021\n",
    "# 308      28699\n",
    "# 106      24212\n",
    "# 372      17013\n",
    "# 379      13255\n",
    "# 399       1566\n",
    "# 206       1004\n",
    "# 199          2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>target</th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">member_years</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>17.286337</td>\n",
       "      <td>12.452057</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>10.136763</td>\n",
       "      <td>9.806865</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>8.000000</td>\n",
       "      <td>5.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>17.000000</td>\n",
       "      <td>10.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>25.000000</td>\n",
       "      <td>19.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>42.000000</td>\n",
       "      <td>42.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">index_change_month</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>200.954188</td>\n",
       "      <td>114.497452</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>3408.063564</td>\n",
       "      <td>544.278680</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>-996608.000000</td>\n",
       "      <td>-24876.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>20.750000</td>\n",
       "      <td>20.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>71.500000</td>\n",
       "      <td>78.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>176.281250</td>\n",
       "      <td>148.750000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>74169.000000</td>\n",
       "      <td>10483.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">quantity_counters</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>64.313225</td>\n",
       "      <td>48.759759</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>30.269453</td>\n",
       "      <td>21.452984</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>6.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>42.000000</td>\n",
       "      <td>32.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>62.000000</td>\n",
       "      <td>47.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>76.000000</td>\n",
       "      <td>67.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>272.000000</td>\n",
       "      <td>171.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">client_catg_11</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.948831</td>\n",
       "      <td>0.976013</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.220343</td>\n",
       "      <td>0.153010</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">client_catg_12</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.007110</td>\n",
       "      <td>0.019208</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.084020</td>\n",
       "      <td>0.137258</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">client_catg_51</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.044059</td>\n",
       "      <td>0.004779</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.205227</td>\n",
       "      <td>0.068964</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">region_101</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.050557</td>\n",
       "      <td>0.340098</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.219091</td>\n",
       "      <td>0.473749</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">region_103</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.144523</td>\n",
       "      <td>0.007230</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.351620</td>\n",
       "      <td>0.084724</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">region_104</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.087459</td>\n",
       "      <td>0.091913</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.282507</td>\n",
       "      <td>0.288908</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">region_105</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.013916</td>\n",
       "      <td>0.010985</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.117142</td>\n",
       "      <td>0.104233</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">region_106</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.007461</td>\n",
       "      <td>0.005834</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.086052</td>\n",
       "      <td>0.076157</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">region_107</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.094964</td>\n",
       "      <td>0.071712</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.293166</td>\n",
       "      <td>0.258015</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">region_199</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">region_206</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.000025</td>\n",
       "      <td>0.000279</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.004969</td>\n",
       "      <td>0.016710</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">region_301</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.007201</td>\n",
       "      <td>0.095885</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.084554</td>\n",
       "      <td>0.294438</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">region_302</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.039836</td>\n",
       "      <td>0.028859</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.195575</td>\n",
       "      <td>0.167412</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">region_303</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.051238</td>\n",
       "      <td>0.052008</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.220484</td>\n",
       "      <td>0.222046</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">region_304</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.046035</td>\n",
       "      <td>0.024266</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.209561</td>\n",
       "      <td>0.153877</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">region_305</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.025340</td>\n",
       "      <td>0.018991</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.157156</td>\n",
       "      <td>0.136495</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">region_306</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.038767</td>\n",
       "      <td>0.052877</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.193039</td>\n",
       "      <td>0.223790</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">region_307</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.022996</td>\n",
       "      <td>0.018805</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.149892</td>\n",
       "      <td>0.135837</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">region_308</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.007863</td>\n",
       "      <td>0.004468</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.088325</td>\n",
       "      <td>0.066698</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">region_309</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.008063</td>\n",
       "      <td>0.049308</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.089432</td>\n",
       "      <td>0.216514</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">region_310</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.028797</td>\n",
       "      <td>0.044933</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.167237</td>\n",
       "      <td>0.207159</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">region_311</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.245787</td>\n",
       "      <td>0.017843</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.430553</td>\n",
       "      <td>0.132382</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">region_312</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.022816</td>\n",
       "      <td>0.046546</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.149317</td>\n",
       "      <td>0.210668</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">region_313</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.024083</td>\n",
       "      <td>0.010675</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.153307</td>\n",
       "      <td>0.102767</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">region_371</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.019922</td>\n",
       "      <td>0.005430</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.139732</td>\n",
       "      <td>0.073492</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">region_372</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.007404</td>\n",
       "      <td>0.000186</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.085726</td>\n",
       "      <td>0.013644</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">region_379</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.004816</td>\n",
       "      <td>0.000621</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.069227</td>\n",
       "      <td>0.024905</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">region_399</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.000131</td>\n",
       "      <td>0.000248</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.011440</td>\n",
       "      <td>0.015754</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">tarif_type_0</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.001847</td>\n",
       "      <td>0.000621</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.042940</td>\n",
       "      <td>0.024905</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">tarif_type_10</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.074738</td>\n",
       "      <td>0.067399</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.262969</td>\n",
       "      <td>0.250716</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">tarif_type_11</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.523261</td>\n",
       "      <td>0.616521</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.499459</td>\n",
       "      <td>0.486241</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">tarif_type_12</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.004791</td>\n",
       "      <td>0.002141</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.069051</td>\n",
       "      <td>0.046223</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">tarif_type_13</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.005077</td>\n",
       "      <td>0.001552</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.071075</td>\n",
       "      <td>0.039360</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">tarif_type_14</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.004959</td>\n",
       "      <td>0.001769</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.070244</td>\n",
       "      <td>0.042020</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">tarif_type_15</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.028084</td>\n",
       "      <td>0.009309</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.165212</td>\n",
       "      <td>0.096036</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">tarif_type_29</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.003080</td>\n",
       "      <td>0.002451</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.055408</td>\n",
       "      <td>0.049452</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">tarif_type_40</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.347752</td>\n",
       "      <td>0.295972</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.476257</td>\n",
       "      <td>0.456486</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">tarif_type_45</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.006411</td>\n",
       "      <td>0.002265</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.079811</td>\n",
       "      <td>0.047541</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">counter_statue_0.0</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.979263</td>\n",
       "      <td>0.977161</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.142502</td>\n",
       "      <td>0.149391</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">counter_statue_1.0</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.013400</td>\n",
       "      <td>0.018091</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.114979</td>\n",
       "      <td>0.133283</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">counter_statue_2.0</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.000005</td>\n",
       "      <td>0.000031</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.002222</td>\n",
       "      <td>0.005571</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">counter_statue_3.0</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.000047</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.006850</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">counter_statue_4.0</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.000467</td>\n",
       "      <td>0.000714</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.021599</td>\n",
       "      <td>0.026706</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">counter_statue_5.0</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.006818</td>\n",
       "      <td>0.004003</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.082292</td>\n",
       "      <td>0.063143</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">reading_remarque_6</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.490547</td>\n",
       "      <td>0.507478</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.499911</td>\n",
       "      <td>0.499952</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">reading_remarque_7</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.000002</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.001571</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">reading_remarque_8</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.207042</td>\n",
       "      <td>0.178583</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.405187</td>\n",
       "      <td>0.383008</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">reading_remarque_9</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.302409</td>\n",
       "      <td>0.313939</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.459302</td>\n",
       "      <td>0.464099</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">counter_type_ELEC</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.645834</td>\n",
       "      <td>0.701763</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.478260</td>\n",
       "      <td>0.457491</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">counter_type_GAZ</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.354166</td>\n",
       "      <td>0.298237</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.478260</td>\n",
       "      <td>0.457491</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">fraud_pred</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">no_fraud_proba</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.498032</td>\n",
       "      <td>0.502038</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.001675</td>\n",
       "      <td>0.001747</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.484773</td>\n",
       "      <td>0.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.497216</td>\n",
       "      <td>0.500779</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.498450</td>\n",
       "      <td>0.501613</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.499264</td>\n",
       "      <td>0.502848</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.517550</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">fraud_proba</th>\n",
       "      <th>count</th>\n",
       "      <td>404932.000000</td>\n",
       "      <td>32226.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.501968</td>\n",
       "      <td>0.497962</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.001675</td>\n",
       "      <td>0.001747</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.482450</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.500736</td>\n",
       "      <td>0.497152</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.501550</td>\n",
       "      <td>0.498387</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.502784</td>\n",
       "      <td>0.499221</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>0.515227</td>\n",
       "      <td>0.500000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "target                                0             1\n",
       "member_years       count  404932.000000  32226.000000\n",
       "                   mean       17.286337     12.452057\n",
       "                   std        10.136763      9.806865\n",
       "                   min         0.000000      0.000000\n",
       "                   25%         8.000000      5.000000\n",
       "                   50%        17.000000     10.000000\n",
       "                   75%        25.000000     19.000000\n",
       "                   max        42.000000     42.000000\n",
       "index_change_month count  404932.000000  32226.000000\n",
       "                   mean      200.954188    114.497452\n",
       "                   std      3408.063564    544.278680\n",
       "                   min   -996608.000000 -24876.500000\n",
       "                   25%        20.750000     20.500000\n",
       "                   50%        71.500000     78.000000\n",
       "                   75%       176.281250    148.750000\n",
       "                   max     74169.000000  10483.000000\n",
       "quantity_counters  count  404932.000000  32226.000000\n",
       "                   mean       64.313225     48.759759\n",
       "                   std        30.269453     21.452984\n",
       "                   min         6.000000      1.000000\n",
       "                   25%        42.000000     32.000000\n",
       "                   50%        62.000000     47.000000\n",
       "                   75%        76.000000     67.000000\n",
       "                   max       272.000000    171.000000\n",
       "client_catg_11     count  404932.000000  32226.000000\n",
       "                   mean        0.948831      0.976013\n",
       "                   std         0.220343      0.153010\n",
       "                   min         0.000000      0.000000\n",
       "                   25%         1.000000      1.000000\n",
       "                   50%         1.000000      1.000000\n",
       "                   75%         1.000000      1.000000\n",
       "                   max         1.000000      1.000000\n",
       "client_catg_12     count  404932.000000  32226.000000\n",
       "                   mean        0.007110      0.019208\n",
       "                   std         0.084020      0.137258\n",
       "                   min         0.000000      0.000000\n",
       "                   25%         0.000000      0.000000\n",
       "                   50%         0.000000      0.000000\n",
       "                   75%         0.000000      0.000000\n",
       "                   max         1.000000      1.000000\n",
       "client_catg_51     count  404932.000000  32226.000000\n",
       "                   mean        0.044059      0.004779\n",
       "                   std         0.205227      0.068964\n",
       "                   min         0.000000      0.000000\n",
       "                   25%         0.000000      0.000000\n",
       "                   50%         0.000000      0.000000\n",
       "                   75%         0.000000      0.000000\n",
       "                   max         1.000000      1.000000\n",
       "region_101         count  404932.000000  32226.000000\n",
       "                   mean        0.050557      0.340098\n",
       "                   std         0.219091      0.473749\n",
       "                   min         0.000000      0.000000\n",
       "                   25%         0.000000      0.000000\n",
       "                   50%         0.000000      0.000000\n",
       "                   75%         0.000000      1.000000\n",
       "                   max         1.000000      1.000000\n",
       "region_103         count  404932.000000  32226.000000\n",
       "                   mean        0.144523      0.007230\n",
       "                   std         0.351620      0.084724\n",
       "                   min         0.000000      0.000000\n",
       "                   25%         0.000000      0.000000\n",
       "                   50%         0.000000      0.000000\n",
       "                   75%         0.000000      0.000000\n",
       "                   max         1.000000      1.000000\n",
       "region_104         count  404932.000000  32226.000000\n",
       "                   mean        0.087459      0.091913\n",
       "                   std         0.282507      0.288908\n",
       "                   min         0.000000      0.000000\n",
       "                   25%         0.000000      0.000000\n",
       "                   50%         0.000000      0.000000\n",
       "                   75%         0.000000      0.000000\n",
       "                   max         1.000000      1.000000\n",
       "region_105         count  404932.000000  32226.000000\n",
       "                   mean        0.013916      0.010985\n",
       "                   std         0.117142      0.104233\n",
       "                   min         0.000000      0.000000\n",
       "                   25%         0.000000      0.000000\n",
       "                   50%         0.000000      0.000000\n",
       "                   75%         0.000000      0.000000\n",
       "                   max         1.000000      1.000000\n",
       "region_106         count  404932.000000  32226.000000\n",
       "                   mean        0.007461      0.005834\n",
       "                   std         0.086052      0.076157\n",
       "                   min         0.000000      0.000000\n",
       "                   25%         0.000000      0.000000\n",
       "                   50%         0.000000      0.000000\n",
       "                   75%         0.000000      0.000000\n",
       "                   max         1.000000      1.000000\n",
       "region_107         count  404932.000000  32226.000000\n",
       "                   mean        0.094964      0.071712\n",
       "                   std         0.293166      0.258015\n",
       "                   min         0.000000      0.000000\n",
       "                   25%         0.000000      0.000000\n",
       "                   50%         0.000000      0.000000\n",
       "                   75%         0.000000      0.000000\n",
       "                   max         1.000000      1.000000\n",
       "region_199         count  404932.000000  32226.000000\n",
       "                   mean        0.000000      0.000000\n",
       "                   std         0.000000      0.000000\n",
       "                   min         0.000000      0.000000\n",
       "                   25%         0.000000      0.000000\n",
       "                   50%         0.000000      0.000000\n",
       "                   75%         0.000000      0.000000\n",
       "                   max         0.000000      0.000000\n",
       "region_206         count  404932.000000  32226.000000\n",
       "                   mean        0.000025      0.000279\n",
       "                   std         0.004969      0.016710\n",
       "                   min         0.000000      0.000000\n",
       "                   25%         0.000000      0.000000\n",
       "                   50%         0.000000      0.000000\n",
       "                   75%         0.000000      0.000000\n",
       "                   max         1.000000      1.000000\n",
       "region_301         count  404932.000000  32226.000000\n",
       "                   mean        0.007201      0.095885\n",
       "                   std         0.084554      0.294438\n",
       "                   min         0.000000      0.000000\n",
       "                   25%         0.000000      0.000000\n",
       "                   50%         0.000000      0.000000\n",
       "                   75%         0.000000      0.000000\n",
       "                   max         1.000000      1.000000\n",
       "region_302         count  404932.000000  32226.000000\n",
       "                   mean        0.039836      0.028859\n",
       "                   std         0.195575      0.167412\n",
       "                   min         0.000000      0.000000\n",
       "                   25%         0.000000      0.000000\n",
       "                   50%         0.000000      0.000000\n",
       "                   75%         0.000000      0.000000\n",
       "                   max         1.000000      1.000000\n",
       "region_303         count  404932.000000  32226.000000\n",
       "                   mean        0.051238      0.052008\n",
       "                   std         0.220484      0.222046\n",
       "                   min         0.000000      0.000000\n",
       "                   25%         0.000000      0.000000\n",
       "                   50%         0.000000      0.000000\n",
       "                   75%         0.000000      0.000000\n",
       "                   max         1.000000      1.000000\n",
       "region_304         count  404932.000000  32226.000000\n",
       "                   mean        0.046035      0.024266\n",
       "                   std         0.209561      0.153877\n",
       "                   min         0.000000      0.000000\n",
       "                   25%         0.000000      0.000000\n",
       "                   50%         0.000000      0.000000\n",
       "                   75%         0.000000      0.000000\n",
       "                   max         1.000000      1.000000\n",
       "region_305         count  404932.000000  32226.000000\n",
       "                   mean        0.025340      0.018991\n",
       "                   std         0.157156      0.136495\n",
       "                   min         0.000000      0.000000\n",
       "                   25%         0.000000      0.000000\n",
       "                   50%         0.000000      0.000000\n",
       "                   75%         0.000000      0.000000\n",
       "                   max         1.000000      1.000000\n",
       "region_306         count  404932.000000  32226.000000\n",
       "                   mean        0.038767      0.052877\n",
       "                   std         0.193039      0.223790\n",
       "                   min         0.000000      0.000000\n",
       "                   25%         0.000000      0.000000\n",
       "                   50%         0.000000      0.000000\n",
       "                   75%         0.000000      0.000000\n",
       "                   max         1.000000      1.000000\n",
       "region_307         count  404932.000000  32226.000000\n",
       "                   mean        0.022996      0.018805\n",
       "                   std         0.149892      0.135837\n",
       "                   min         0.000000      0.000000\n",
       "                   25%         0.000000      0.000000\n",
       "                   50%         0.000000      0.000000\n",
       "                   75%         0.000000      0.000000\n",
       "                   max         1.000000      1.000000\n",
       "region_308         count  404932.000000  32226.000000\n",
       "                   mean        0.007863      0.004468\n",
       "                   std         0.088325      0.066698\n",
       "                   min         0.000000      0.000000\n",
       "                   25%         0.000000      0.000000\n",
       "                   50%         0.000000      0.000000\n",
       "                   75%         0.000000      0.000000\n",
       "                   max         1.000000      1.000000\n",
       "region_309         count  404932.000000  32226.000000\n",
       "                   mean        0.008063      0.049308\n",
       "                   std         0.089432      0.216514\n",
       "                   min         0.000000      0.000000\n",
       "                   25%         0.000000      0.000000\n",
       "                   50%         0.000000      0.000000\n",
       "                   75%         0.000000      0.000000\n",
       "                   max         1.000000      1.000000\n",
       "region_310         count  404932.000000  32226.000000\n",
       "                   mean        0.028797      0.044933\n",
       "                   std         0.167237      0.207159\n",
       "                   min         0.000000      0.000000\n",
       "                   25%         0.000000      0.000000\n",
       "                   50%         0.000000      0.000000\n",
       "                   75%         0.000000      0.000000\n",
       "                   max         1.000000      1.000000\n",
       "region_311         count  404932.000000  32226.000000\n",
       "                   mean        0.245787      0.017843\n",
       "                   std         0.430553      0.132382\n",
       "                   min         0.000000      0.000000\n",
       "                   25%         0.000000      0.000000\n",
       "                   50%         0.000000      0.000000\n",
       "                   75%         0.000000      0.000000\n",
       "                   max         1.000000      1.000000\n",
       "region_312         count  404932.000000  32226.000000\n",
       "                   mean        0.022816      0.046546\n",
       "                   std         0.149317      0.210668\n",
       "                   min         0.000000      0.000000\n",
       "                   25%         0.000000      0.000000\n",
       "                   50%         0.000000      0.000000\n",
       "                   75%         0.000000      0.000000\n",
       "                   max         1.000000      1.000000\n",
       "region_313         count  404932.000000  32226.000000\n",
       "                   mean        0.024083      0.010675\n",
       "                   std         0.153307      0.102767\n",
       "                   min         0.000000      0.000000\n",
       "                   25%         0.000000      0.000000\n",
       "                   50%         0.000000      0.000000\n",
       "                   75%         0.000000      0.000000\n",
       "                   max         1.000000      1.000000\n",
       "region_371         count  404932.000000  32226.000000\n",
       "                   mean        0.019922      0.005430\n",
       "                   std         0.139732      0.073492\n",
       "                   min         0.000000      0.000000\n",
       "                   25%         0.000000      0.000000\n",
       "                   50%         0.000000      0.000000\n",
       "                   75%         0.000000      0.000000\n",
       "                   max         1.000000      1.000000\n",
       "region_372         count  404932.000000  32226.000000\n",
       "                   mean        0.007404      0.000186\n",
       "                   std         0.085726      0.013644\n",
       "                   min         0.000000      0.000000\n",
       "                   25%         0.000000      0.000000\n",
       "                   50%         0.000000      0.000000\n",
       "                   75%         0.000000      0.000000\n",
       "                   max         1.000000      1.000000\n",
       "region_379         count  404932.000000  32226.000000\n",
       "                   mean        0.004816      0.000621\n",
       "                   std         0.069227      0.024905\n",
       "                   min         0.000000      0.000000\n",
       "                   25%         0.000000      0.000000\n",
       "                   50%         0.000000      0.000000\n",
       "                   75%         0.000000      0.000000\n",
       "                   max         1.000000      1.000000\n",
       "region_399         count  404932.000000  32226.000000\n",
       "                   mean        0.000131      0.000248\n",
       "                   std         0.011440      0.015754\n",
       "                   min         0.000000      0.000000\n",
       "                   25%         0.000000      0.000000\n",
       "                   50%         0.000000      0.000000\n",
       "                   75%         0.000000      0.000000\n",
       "                   max         1.000000      1.000000\n",
       "tarif_type_0       count  404932.000000  32226.000000\n",
       "                   mean        0.001847      0.000621\n",
       "                   std         0.042940      0.024905\n",
       "                   min         0.000000      0.000000\n",
       "                   25%         0.000000      0.000000\n",
       "                   50%         0.000000      0.000000\n",
       "                   75%         0.000000      0.000000\n",
       "                   max         1.000000      1.000000\n",
       "tarif_type_10      count  404932.000000  32226.000000\n",
       "                   mean        0.074738      0.067399\n",
       "                   std         0.262969      0.250716\n",
       "                   min         0.000000      0.000000\n",
       "                   25%         0.000000      0.000000\n",
       "                   50%         0.000000      0.000000\n",
       "                   75%         0.000000      0.000000\n",
       "                   max         1.000000      1.000000\n",
       "tarif_type_11      count  404932.000000  32226.000000\n",
       "                   mean        0.523261      0.616521\n",
       "                   std         0.499459      0.486241\n",
       "                   min         0.000000      0.000000\n",
       "                   25%         0.000000      0.000000\n",
       "                   50%         1.000000      1.000000\n",
       "                   75%         1.000000      1.000000\n",
       "                   max         1.000000      1.000000\n",
       "tarif_type_12      count  404932.000000  32226.000000\n",
       "                   mean        0.004791      0.002141\n",
       "                   std         0.069051      0.046223\n",
       "                   min         0.000000      0.000000\n",
       "                   25%         0.000000      0.000000\n",
       "                   50%         0.000000      0.000000\n",
       "                   75%         0.000000      0.000000\n",
       "                   max         1.000000      1.000000\n",
       "tarif_type_13      count  404932.000000  32226.000000\n",
       "                   mean        0.005077      0.001552\n",
       "                   std         0.071075      0.039360\n",
       "                   min         0.000000      0.000000\n",
       "                   25%         0.000000      0.000000\n",
       "                   50%         0.000000      0.000000\n",
       "                   75%         0.000000      0.000000\n",
       "                   max         1.000000      1.000000\n",
       "tarif_type_14      count  404932.000000  32226.000000\n",
       "                   mean        0.004959      0.001769\n",
       "                   std         0.070244      0.042020\n",
       "                   min         0.000000      0.000000\n",
       "                   25%         0.000000      0.000000\n",
       "                   50%         0.000000      0.000000\n",
       "                   75%         0.000000      0.000000\n",
       "                   max         1.000000      1.000000\n",
       "tarif_type_15      count  404932.000000  32226.000000\n",
       "                   mean        0.028084      0.009309\n",
       "                   std         0.165212      0.096036\n",
       "                   min         0.000000      0.000000\n",
       "                   25%         0.000000      0.000000\n",
       "                   50%         0.000000      0.000000\n",
       "                   75%         0.000000      0.000000\n",
       "                   max         1.000000      1.000000\n",
       "tarif_type_29      count  404932.000000  32226.000000\n",
       "                   mean        0.003080      0.002451\n",
       "                   std         0.055408      0.049452\n",
       "                   min         0.000000      0.000000\n",
       "                   25%         0.000000      0.000000\n",
       "                   50%         0.000000      0.000000\n",
       "                   75%         0.000000      0.000000\n",
       "                   max         1.000000      1.000000\n",
       "tarif_type_40      count  404932.000000  32226.000000\n",
       "                   mean        0.347752      0.295972\n",
       "                   std         0.476257      0.456486\n",
       "                   min         0.000000      0.000000\n",
       "                   25%         0.000000      0.000000\n",
       "                   50%         0.000000      0.000000\n",
       "                   75%         1.000000      1.000000\n",
       "                   max         1.000000      1.000000\n",
       "tarif_type_45      count  404932.000000  32226.000000\n",
       "                   mean        0.006411      0.002265\n",
       "                   std         0.079811      0.047541\n",
       "                   min         0.000000      0.000000\n",
       "                   25%         0.000000      0.000000\n",
       "                   50%         0.000000      0.000000\n",
       "                   75%         0.000000      0.000000\n",
       "                   max         1.000000      1.000000\n",
       "counter_statue_0.0 count  404932.000000  32226.000000\n",
       "                   mean        0.979263      0.977161\n",
       "                   std         0.142502      0.149391\n",
       "                   min         0.000000      0.000000\n",
       "                   25%         1.000000      1.000000\n",
       "                   50%         1.000000      1.000000\n",
       "                   75%         1.000000      1.000000\n",
       "                   max         1.000000      1.000000\n",
       "counter_statue_1.0 count  404932.000000  32226.000000\n",
       "                   mean        0.013400      0.018091\n",
       "                   std         0.114979      0.133283\n",
       "                   min         0.000000      0.000000\n",
       "                   25%         0.000000      0.000000\n",
       "                   50%         0.000000      0.000000\n",
       "                   75%         0.000000      0.000000\n",
       "                   max         1.000000      1.000000\n",
       "counter_statue_2.0 count  404932.000000  32226.000000\n",
       "                   mean        0.000005      0.000031\n",
       "                   std         0.002222      0.005571\n",
       "                   min         0.000000      0.000000\n",
       "                   25%         0.000000      0.000000\n",
       "                   50%         0.000000      0.000000\n",
       "                   75%         0.000000      0.000000\n",
       "                   max         1.000000      1.000000\n",
       "counter_statue_3.0 count  404932.000000  32226.000000\n",
       "                   mean        0.000047      0.000000\n",
       "                   std         0.006850      0.000000\n",
       "                   min         0.000000      0.000000\n",
       "                   25%         0.000000      0.000000\n",
       "                   50%         0.000000      0.000000\n",
       "                   75%         0.000000      0.000000\n",
       "                   max         1.000000      0.000000\n",
       "counter_statue_4.0 count  404932.000000  32226.000000\n",
       "                   mean        0.000467      0.000714\n",
       "                   std         0.021599      0.026706\n",
       "                   min         0.000000      0.000000\n",
       "                   25%         0.000000      0.000000\n",
       "                   50%         0.000000      0.000000\n",
       "                   75%         0.000000      0.000000\n",
       "                   max         1.000000      1.000000\n",
       "counter_statue_5.0 count  404932.000000  32226.000000\n",
       "                   mean        0.006818      0.004003\n",
       "                   std         0.082292      0.063143\n",
       "                   min         0.000000      0.000000\n",
       "                   25%         0.000000      0.000000\n",
       "                   50%         0.000000      0.000000\n",
       "                   75%         0.000000      0.000000\n",
       "                   max         1.000000      1.000000\n",
       "reading_remarque_6 count  404932.000000  32226.000000\n",
       "                   mean        0.490547      0.507478\n",
       "                   std         0.499911      0.499952\n",
       "                   min         0.000000      0.000000\n",
       "                   25%         0.000000      0.000000\n",
       "                   50%         0.000000      1.000000\n",
       "                   75%         1.000000      1.000000\n",
       "                   max         1.000000      1.000000\n",
       "reading_remarque_7 count  404932.000000  32226.000000\n",
       "                   mean        0.000002      0.000000\n",
       "                   std         0.001571      0.000000\n",
       "                   min         0.000000      0.000000\n",
       "                   25%         0.000000      0.000000\n",
       "                   50%         0.000000      0.000000\n",
       "                   75%         0.000000      0.000000\n",
       "                   max         1.000000      0.000000\n",
       "reading_remarque_8 count  404932.000000  32226.000000\n",
       "                   mean        0.207042      0.178583\n",
       "                   std         0.405187      0.383008\n",
       "                   min         0.000000      0.000000\n",
       "                   25%         0.000000      0.000000\n",
       "                   50%         0.000000      0.000000\n",
       "                   75%         0.000000      0.000000\n",
       "                   max         1.000000      1.000000\n",
       "reading_remarque_9 count  404932.000000  32226.000000\n",
       "                   mean        0.302409      0.313939\n",
       "                   std         0.459302      0.464099\n",
       "                   min         0.000000      0.000000\n",
       "                   25%         0.000000      0.000000\n",
       "                   50%         0.000000      0.000000\n",
       "                   75%         1.000000      1.000000\n",
       "                   max         1.000000      1.000000\n",
       "counter_type_ELEC  count  404932.000000  32226.000000\n",
       "                   mean        0.645834      0.701763\n",
       "                   std         0.478260      0.457491\n",
       "                   min         0.000000      0.000000\n",
       "                   25%         0.000000      0.000000\n",
       "                   50%         1.000000      1.000000\n",
       "                   75%         1.000000      1.000000\n",
       "                   max         1.000000      1.000000\n",
       "counter_type_GAZ   count  404932.000000  32226.000000\n",
       "                   mean        0.354166      0.298237\n",
       "                   std         0.478260      0.457491\n",
       "                   min         0.000000      0.000000\n",
       "                   25%         0.000000      0.000000\n",
       "                   50%         0.000000      0.000000\n",
       "                   75%         1.000000      1.000000\n",
       "                   max         1.000000      1.000000\n",
       "fraud_pred         count  404932.000000  32226.000000\n",
       "                   mean        1.000000      0.000000\n",
       "                   std         0.000000      0.000000\n",
       "                   min         1.000000      0.000000\n",
       "                   25%         1.000000      0.000000\n",
       "                   50%         1.000000      0.000000\n",
       "                   75%         1.000000      0.000000\n",
       "                   max         1.000000      0.000000\n",
       "no_fraud_proba     count  404932.000000  32226.000000\n",
       "                   mean        0.498032      0.502038\n",
       "                   std         0.001675      0.001747\n",
       "                   min         0.484773      0.500000\n",
       "                   25%         0.497216      0.500779\n",
       "                   50%         0.498450      0.501613\n",
       "                   75%         0.499264      0.502848\n",
       "                   max         0.500000      0.517550\n",
       "fraud_proba        count  404932.000000  32226.000000\n",
       "                   mean        0.501968      0.497962\n",
       "                   std         0.001675      0.001747\n",
       "                   min         0.500000      0.482450\n",
       "                   25%         0.500736      0.497152\n",
       "                   50%         0.501550      0.498387\n",
       "                   75%         0.502784      0.499221\n",
       "                   max         0.515227      0.500000"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "pd.set_option('display.max_rows', None)\n",
    "display(df_false_pred.groupby('target').describe().T)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pred_corr</th>\n",
       "      <th>False</th>\n",
       "      <th>True</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">member_years</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>16.929968</td>\n",
       "      <td>11.614654</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>10.191395</td>\n",
       "      <td>9.620570</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>8.000000</td>\n",
       "      <td>4.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>16.000000</td>\n",
       "      <td>9.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>25.000000</td>\n",
       "      <td>17.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>42.000000</td>\n",
       "      <td>42.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">index_change_month</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>194.580852</td>\n",
       "      <td>135.039329</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>3283.447417</td>\n",
       "      <td>901.874224</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>-996608.000000</td>\n",
       "      <td>-237774.250000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>20.750000</td>\n",
       "      <td>19.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>72.000000</td>\n",
       "      <td>79.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>173.500000</td>\n",
       "      <td>150.750000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>74169.000000</td>\n",
       "      <td>116645.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">quantity_counters</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>63.166670</td>\n",
       "      <td>47.469597</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>29.985686</td>\n",
       "      <td>28.455375</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>41.000000</td>\n",
       "      <td>28.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>62.000000</td>\n",
       "      <td>43.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>76.000000</td>\n",
       "      <td>66.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>272.000000</td>\n",
       "      <td>439.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">client_catg_11</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.950835</td>\n",
       "      <td>0.968719</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.216213</td>\n",
       "      <td>0.174076</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">client_catg_12</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.008002</td>\n",
       "      <td>0.019555</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.089094</td>\n",
       "      <td>0.138466</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">client_catg_51</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.041164</td>\n",
       "      <td>0.011726</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.198669</td>\n",
       "      <td>0.107648</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">region_101</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.071901</td>\n",
       "      <td>0.346809</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.258324</td>\n",
       "      <td>0.475955</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">region_103</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.134402</td>\n",
       "      <td>0.022017</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.341084</td>\n",
       "      <td>0.146739</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">region_104</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.087787</td>\n",
       "      <td>0.090728</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.282986</td>\n",
       "      <td>0.287222</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">region_105</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.013700</td>\n",
       "      <td>0.010630</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.116242</td>\n",
       "      <td>0.102554</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">region_106</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.007341</td>\n",
       "      <td>0.004157</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.085362</td>\n",
       "      <td>0.064339</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">region_107</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.093250</td>\n",
       "      <td>0.051332</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.290783</td>\n",
       "      <td>0.220674</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">region_199</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">region_206</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.000043</td>\n",
       "      <td>0.000317</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.006592</td>\n",
       "      <td>0.017794</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">region_301</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.013739</td>\n",
       "      <td>0.106709</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.116405</td>\n",
       "      <td>0.308743</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">region_302</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.039027</td>\n",
       "      <td>0.027938</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.193660</td>\n",
       "      <td>0.164795</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">region_303</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.051295</td>\n",
       "      <td>0.039767</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.220599</td>\n",
       "      <td>0.195413</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">region_304</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.044430</td>\n",
       "      <td>0.024692</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.206049</td>\n",
       "      <td>0.155183</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">region_305</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.024872</td>\n",
       "      <td>0.018893</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.155735</td>\n",
       "      <td>0.136146</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">region_306</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.039807</td>\n",
       "      <td>0.045876</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.195506</td>\n",
       "      <td>0.209216</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">region_307</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.022687</td>\n",
       "      <td>0.016506</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.148905</td>\n",
       "      <td>0.127409</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">region_308</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.007613</td>\n",
       "      <td>0.005611</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.086919</td>\n",
       "      <td>0.074698</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">region_309</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.011104</td>\n",
       "      <td>0.040053</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.104787</td>\n",
       "      <td>0.196085</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">region_310</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.029987</td>\n",
       "      <td>0.041508</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.170551</td>\n",
       "      <td>0.199462</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">region_311</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.228984</td>\n",
       "      <td>0.039806</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.420179</td>\n",
       "      <td>0.195502</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">region_312</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.024565</td>\n",
       "      <td>0.041438</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.154797</td>\n",
       "      <td>0.199300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">region_313</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.023095</td>\n",
       "      <td>0.013554</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.150204</td>\n",
       "      <td>0.115630</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">region_371</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.018854</td>\n",
       "      <td>0.007262</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.136008</td>\n",
       "      <td>0.084909</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">region_372</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.006872</td>\n",
       "      <td>0.001887</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.082610</td>\n",
       "      <td>0.043399</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">region_379</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.004506</td>\n",
       "      <td>0.002056</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.066978</td>\n",
       "      <td>0.045293</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">region_399</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.000140</td>\n",
       "      <td>0.000456</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.011812</td>\n",
       "      <td>0.021349</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">tarif_type_0</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.001757</td>\n",
       "      <td>0.001132</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.041877</td>\n",
       "      <td>0.033625</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">tarif_type_10</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.074197</td>\n",
       "      <td>0.053452</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.262092</td>\n",
       "      <td>0.224933</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">tarif_type_11</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.530136</td>\n",
       "      <td>0.642998</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.499092</td>\n",
       "      <td>0.479116</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">tarif_type_12</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.004596</td>\n",
       "      <td>0.001243</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.067635</td>\n",
       "      <td>0.035240</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">tarif_type_13</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.004817</td>\n",
       "      <td>0.001152</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.069241</td>\n",
       "      <td>0.033928</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">tarif_type_14</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.004724</td>\n",
       "      <td>0.001230</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.068567</td>\n",
       "      <td>0.035052</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">tarif_type_15</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.026700</td>\n",
       "      <td>0.009498</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.161205</td>\n",
       "      <td>0.096995</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">tarif_type_29</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.003033</td>\n",
       "      <td>0.001758</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.054991</td>\n",
       "      <td>0.041892</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">tarif_type_40</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.343935</td>\n",
       "      <td>0.285022</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.475020</td>\n",
       "      <td>0.451425</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">tarif_type_45</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.006105</td>\n",
       "      <td>0.002515</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.077898</td>\n",
       "      <td>0.050083</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">counter_statue_0.0</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.979108</td>\n",
       "      <td>0.977704</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.143022</td>\n",
       "      <td>0.147644</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">counter_statue_1.0</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.013746</td>\n",
       "      <td>0.018073</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.116433</td>\n",
       "      <td>0.133215</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">counter_statue_2.0</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.000007</td>\n",
       "      <td>0.000007</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.002620</td>\n",
       "      <td>0.002708</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">counter_statue_3.0</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.000043</td>\n",
       "      <td>0.000075</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.006592</td>\n",
       "      <td>0.008647</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">counter_statue_4.0</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.000485</td>\n",
       "      <td>0.000723</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.022016</td>\n",
       "      <td>0.026876</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">counter_statue_5.0</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.006611</td>\n",
       "      <td>0.003418</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.081038</td>\n",
       "      <td>0.058362</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">reading_remarque_6</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.491795</td>\n",
       "      <td>0.503517</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.499933</td>\n",
       "      <td>0.499988</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">reading_remarque_7</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.000002</td>\n",
       "      <td>0.000254</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.001512</td>\n",
       "      <td>0.015925</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">reading_remarque_8</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.204944</td>\n",
       "      <td>0.171499</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.403661</td>\n",
       "      <td>0.376945</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">reading_remarque_9</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.303259</td>\n",
       "      <td>0.324730</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.459667</td>\n",
       "      <td>0.468274</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">counter_type_ELEC</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.649957</td>\n",
       "      <td>0.712454</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.476984</td>\n",
       "      <td>0.452619</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">counter_type_GAZ</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.350043</td>\n",
       "      <td>0.287546</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.476984</td>\n",
       "      <td>0.452619</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">target</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.073717</td>\n",
       "      <td>0.082184</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.261310</td>\n",
       "      <td>0.274646</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">fraud_pred</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.926283</td>\n",
       "      <td>0.082184</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.261310</td>\n",
       "      <td>0.274646</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">no_fraud_proba</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.498327</td>\n",
       "      <td>0.502750</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.001980</td>\n",
       "      <td>0.007922</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.484773</td>\n",
       "      <td>0.484687</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.497341</td>\n",
       "      <td>0.500867</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.498597</td>\n",
       "      <td>0.501998</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.499417</td>\n",
       "      <td>0.503775</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>0.517550</td>\n",
       "      <td>0.671550</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">fraud_proba</th>\n",
       "      <th>count</th>\n",
       "      <td>437158.000000</td>\n",
       "      <td>682015.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.501673</td>\n",
       "      <td>0.497250</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.001980</td>\n",
       "      <td>0.007922</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.482450</td>\n",
       "      <td>0.328450</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.500583</td>\n",
       "      <td>0.496225</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.501403</td>\n",
       "      <td>0.498002</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.502659</td>\n",
       "      <td>0.499133</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>0.515227</td>\n",
       "      <td>0.515313</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "pred_corr                         False          True \n",
       "member_years       count  437158.000000  682015.000000\n",
       "                   mean       16.929968      11.614654\n",
       "                   std        10.191395       9.620570\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         8.000000       4.000000\n",
       "                   50%        16.000000       9.000000\n",
       "                   75%        25.000000      17.000000\n",
       "                   max        42.000000      42.000000\n",
       "index_change_month count  437158.000000  682015.000000\n",
       "                   mean      194.580852     135.039329\n",
       "                   std      3283.447417     901.874224\n",
       "                   min   -996608.000000 -237774.250000\n",
       "                   25%        20.750000      19.000000\n",
       "                   50%        72.000000      79.000000\n",
       "                   75%       173.500000     150.750000\n",
       "                   max     74169.000000  116645.000000\n",
       "quantity_counters  count  437158.000000  682015.000000\n",
       "                   mean       63.166670      47.469597\n",
       "                   std        29.985686      28.455375\n",
       "                   min         1.000000       1.000000\n",
       "                   25%        41.000000      28.000000\n",
       "                   50%        62.000000      43.000000\n",
       "                   75%        76.000000      66.000000\n",
       "                   max       272.000000     439.000000\n",
       "client_catg_11     count  437158.000000  682015.000000\n",
       "                   mean        0.950835       0.968719\n",
       "                   std         0.216213       0.174076\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         1.000000       1.000000\n",
       "                   50%         1.000000       1.000000\n",
       "                   75%         1.000000       1.000000\n",
       "                   max         1.000000       1.000000\n",
       "client_catg_12     count  437158.000000  682015.000000\n",
       "                   mean        0.008002       0.019555\n",
       "                   std         0.089094       0.138466\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         0.000000       0.000000\n",
       "                   50%         0.000000       0.000000\n",
       "                   75%         0.000000       0.000000\n",
       "                   max         1.000000       1.000000\n",
       "client_catg_51     count  437158.000000  682015.000000\n",
       "                   mean        0.041164       0.011726\n",
       "                   std         0.198669       0.107648\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         0.000000       0.000000\n",
       "                   50%         0.000000       0.000000\n",
       "                   75%         0.000000       0.000000\n",
       "                   max         1.000000       1.000000\n",
       "region_101         count  437158.000000  682015.000000\n",
       "                   mean        0.071901       0.346809\n",
       "                   std         0.258324       0.475955\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         0.000000       0.000000\n",
       "                   50%         0.000000       0.000000\n",
       "                   75%         0.000000       1.000000\n",
       "                   max         1.000000       1.000000\n",
       "region_103         count  437158.000000  682015.000000\n",
       "                   mean        0.134402       0.022017\n",
       "                   std         0.341084       0.146739\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         0.000000       0.000000\n",
       "                   50%         0.000000       0.000000\n",
       "                   75%         0.000000       0.000000\n",
       "                   max         1.000000       1.000000\n",
       "region_104         count  437158.000000  682015.000000\n",
       "                   mean        0.087787       0.090728\n",
       "                   std         0.282986       0.287222\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         0.000000       0.000000\n",
       "                   50%         0.000000       0.000000\n",
       "                   75%         0.000000       0.000000\n",
       "                   max         1.000000       1.000000\n",
       "region_105         count  437158.000000  682015.000000\n",
       "                   mean        0.013700       0.010630\n",
       "                   std         0.116242       0.102554\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         0.000000       0.000000\n",
       "                   50%         0.000000       0.000000\n",
       "                   75%         0.000000       0.000000\n",
       "                   max         1.000000       1.000000\n",
       "region_106         count  437158.000000  682015.000000\n",
       "                   mean        0.007341       0.004157\n",
       "                   std         0.085362       0.064339\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         0.000000       0.000000\n",
       "                   50%         0.000000       0.000000\n",
       "                   75%         0.000000       0.000000\n",
       "                   max         1.000000       1.000000\n",
       "region_107         count  437158.000000  682015.000000\n",
       "                   mean        0.093250       0.051332\n",
       "                   std         0.290783       0.220674\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         0.000000       0.000000\n",
       "                   50%         0.000000       0.000000\n",
       "                   75%         0.000000       0.000000\n",
       "                   max         1.000000       1.000000\n",
       "region_199         count  437158.000000  682015.000000\n",
       "                   mean        0.000000       0.000000\n",
       "                   std         0.000000       0.000000\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         0.000000       0.000000\n",
       "                   50%         0.000000       0.000000\n",
       "                   75%         0.000000       0.000000\n",
       "                   max         0.000000       0.000000\n",
       "region_206         count  437158.000000  682015.000000\n",
       "                   mean        0.000043       0.000317\n",
       "                   std         0.006592       0.017794\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         0.000000       0.000000\n",
       "                   50%         0.000000       0.000000\n",
       "                   75%         0.000000       0.000000\n",
       "                   max         1.000000       1.000000\n",
       "region_301         count  437158.000000  682015.000000\n",
       "                   mean        0.013739       0.106709\n",
       "                   std         0.116405       0.308743\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         0.000000       0.000000\n",
       "                   50%         0.000000       0.000000\n",
       "                   75%         0.000000       0.000000\n",
       "                   max         1.000000       1.000000\n",
       "region_302         count  437158.000000  682015.000000\n",
       "                   mean        0.039027       0.027938\n",
       "                   std         0.193660       0.164795\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         0.000000       0.000000\n",
       "                   50%         0.000000       0.000000\n",
       "                   75%         0.000000       0.000000\n",
       "                   max         1.000000       1.000000\n",
       "region_303         count  437158.000000  682015.000000\n",
       "                   mean        0.051295       0.039767\n",
       "                   std         0.220599       0.195413\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         0.000000       0.000000\n",
       "                   50%         0.000000       0.000000\n",
       "                   75%         0.000000       0.000000\n",
       "                   max         1.000000       1.000000\n",
       "region_304         count  437158.000000  682015.000000\n",
       "                   mean        0.044430       0.024692\n",
       "                   std         0.206049       0.155183\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         0.000000       0.000000\n",
       "                   50%         0.000000       0.000000\n",
       "                   75%         0.000000       0.000000\n",
       "                   max         1.000000       1.000000\n",
       "region_305         count  437158.000000  682015.000000\n",
       "                   mean        0.024872       0.018893\n",
       "                   std         0.155735       0.136146\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         0.000000       0.000000\n",
       "                   50%         0.000000       0.000000\n",
       "                   75%         0.000000       0.000000\n",
       "                   max         1.000000       1.000000\n",
       "region_306         count  437158.000000  682015.000000\n",
       "                   mean        0.039807       0.045876\n",
       "                   std         0.195506       0.209216\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         0.000000       0.000000\n",
       "                   50%         0.000000       0.000000\n",
       "                   75%         0.000000       0.000000\n",
       "                   max         1.000000       1.000000\n",
       "region_307         count  437158.000000  682015.000000\n",
       "                   mean        0.022687       0.016506\n",
       "                   std         0.148905       0.127409\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         0.000000       0.000000\n",
       "                   50%         0.000000       0.000000\n",
       "                   75%         0.000000       0.000000\n",
       "                   max         1.000000       1.000000\n",
       "region_308         count  437158.000000  682015.000000\n",
       "                   mean        0.007613       0.005611\n",
       "                   std         0.086919       0.074698\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         0.000000       0.000000\n",
       "                   50%         0.000000       0.000000\n",
       "                   75%         0.000000       0.000000\n",
       "                   max         1.000000       1.000000\n",
       "region_309         count  437158.000000  682015.000000\n",
       "                   mean        0.011104       0.040053\n",
       "                   std         0.104787       0.196085\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         0.000000       0.000000\n",
       "                   50%         0.000000       0.000000\n",
       "                   75%         0.000000       0.000000\n",
       "                   max         1.000000       1.000000\n",
       "region_310         count  437158.000000  682015.000000\n",
       "                   mean        0.029987       0.041508\n",
       "                   std         0.170551       0.199462\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         0.000000       0.000000\n",
       "                   50%         0.000000       0.000000\n",
       "                   75%         0.000000       0.000000\n",
       "                   max         1.000000       1.000000\n",
       "region_311         count  437158.000000  682015.000000\n",
       "                   mean        0.228984       0.039806\n",
       "                   std         0.420179       0.195502\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         0.000000       0.000000\n",
       "                   50%         0.000000       0.000000\n",
       "                   75%         0.000000       0.000000\n",
       "                   max         1.000000       1.000000\n",
       "region_312         count  437158.000000  682015.000000\n",
       "                   mean        0.024565       0.041438\n",
       "                   std         0.154797       0.199300\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         0.000000       0.000000\n",
       "                   50%         0.000000       0.000000\n",
       "                   75%         0.000000       0.000000\n",
       "                   max         1.000000       1.000000\n",
       "region_313         count  437158.000000  682015.000000\n",
       "                   mean        0.023095       0.013554\n",
       "                   std         0.150204       0.115630\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         0.000000       0.000000\n",
       "                   50%         0.000000       0.000000\n",
       "                   75%         0.000000       0.000000\n",
       "                   max         1.000000       1.000000\n",
       "region_371         count  437158.000000  682015.000000\n",
       "                   mean        0.018854       0.007262\n",
       "                   std         0.136008       0.084909\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         0.000000       0.000000\n",
       "                   50%         0.000000       0.000000\n",
       "                   75%         0.000000       0.000000\n",
       "                   max         1.000000       1.000000\n",
       "region_372         count  437158.000000  682015.000000\n",
       "                   mean        0.006872       0.001887\n",
       "                   std         0.082610       0.043399\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         0.000000       0.000000\n",
       "                   50%         0.000000       0.000000\n",
       "                   75%         0.000000       0.000000\n",
       "                   max         1.000000       1.000000\n",
       "region_379         count  437158.000000  682015.000000\n",
       "                   mean        0.004506       0.002056\n",
       "                   std         0.066978       0.045293\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         0.000000       0.000000\n",
       "                   50%         0.000000       0.000000\n",
       "                   75%         0.000000       0.000000\n",
       "                   max         1.000000       1.000000\n",
       "region_399         count  437158.000000  682015.000000\n",
       "                   mean        0.000140       0.000456\n",
       "                   std         0.011812       0.021349\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         0.000000       0.000000\n",
       "                   50%         0.000000       0.000000\n",
       "                   75%         0.000000       0.000000\n",
       "                   max         1.000000       1.000000\n",
       "tarif_type_0       count  437158.000000  682015.000000\n",
       "                   mean        0.001757       0.001132\n",
       "                   std         0.041877       0.033625\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         0.000000       0.000000\n",
       "                   50%         0.000000       0.000000\n",
       "                   75%         0.000000       0.000000\n",
       "                   max         1.000000       1.000000\n",
       "tarif_type_10      count  437158.000000  682015.000000\n",
       "                   mean        0.074197       0.053452\n",
       "                   std         0.262092       0.224933\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         0.000000       0.000000\n",
       "                   50%         0.000000       0.000000\n",
       "                   75%         0.000000       0.000000\n",
       "                   max         1.000000       1.000000\n",
       "tarif_type_11      count  437158.000000  682015.000000\n",
       "                   mean        0.530136       0.642998\n",
       "                   std         0.499092       0.479116\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         0.000000       0.000000\n",
       "                   50%         1.000000       1.000000\n",
       "                   75%         1.000000       1.000000\n",
       "                   max         1.000000       1.000000\n",
       "tarif_type_12      count  437158.000000  682015.000000\n",
       "                   mean        0.004596       0.001243\n",
       "                   std         0.067635       0.035240\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         0.000000       0.000000\n",
       "                   50%         0.000000       0.000000\n",
       "                   75%         0.000000       0.000000\n",
       "                   max         1.000000       1.000000\n",
       "tarif_type_13      count  437158.000000  682015.000000\n",
       "                   mean        0.004817       0.001152\n",
       "                   std         0.069241       0.033928\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         0.000000       0.000000\n",
       "                   50%         0.000000       0.000000\n",
       "                   75%         0.000000       0.000000\n",
       "                   max         1.000000       1.000000\n",
       "tarif_type_14      count  437158.000000  682015.000000\n",
       "                   mean        0.004724       0.001230\n",
       "                   std         0.068567       0.035052\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         0.000000       0.000000\n",
       "                   50%         0.000000       0.000000\n",
       "                   75%         0.000000       0.000000\n",
       "                   max         1.000000       1.000000\n",
       "tarif_type_15      count  437158.000000  682015.000000\n",
       "                   mean        0.026700       0.009498\n",
       "                   std         0.161205       0.096995\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         0.000000       0.000000\n",
       "                   50%         0.000000       0.000000\n",
       "                   75%         0.000000       0.000000\n",
       "                   max         1.000000       1.000000\n",
       "tarif_type_29      count  437158.000000  682015.000000\n",
       "                   mean        0.003033       0.001758\n",
       "                   std         0.054991       0.041892\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         0.000000       0.000000\n",
       "                   50%         0.000000       0.000000\n",
       "                   75%         0.000000       0.000000\n",
       "                   max         1.000000       1.000000\n",
       "tarif_type_40      count  437158.000000  682015.000000\n",
       "                   mean        0.343935       0.285022\n",
       "                   std         0.475020       0.451425\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         0.000000       0.000000\n",
       "                   50%         0.000000       0.000000\n",
       "                   75%         1.000000       1.000000\n",
       "                   max         1.000000       1.000000\n",
       "tarif_type_45      count  437158.000000  682015.000000\n",
       "                   mean        0.006105       0.002515\n",
       "                   std         0.077898       0.050083\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         0.000000       0.000000\n",
       "                   50%         0.000000       0.000000\n",
       "                   75%         0.000000       0.000000\n",
       "                   max         1.000000       1.000000\n",
       "counter_statue_0.0 count  437158.000000  682015.000000\n",
       "                   mean        0.979108       0.977704\n",
       "                   std         0.143022       0.147644\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         1.000000       1.000000\n",
       "                   50%         1.000000       1.000000\n",
       "                   75%         1.000000       1.000000\n",
       "                   max         1.000000       1.000000\n",
       "counter_statue_1.0 count  437158.000000  682015.000000\n",
       "                   mean        0.013746       0.018073\n",
       "                   std         0.116433       0.133215\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         0.000000       0.000000\n",
       "                   50%         0.000000       0.000000\n",
       "                   75%         0.000000       0.000000\n",
       "                   max         1.000000       1.000000\n",
       "counter_statue_2.0 count  437158.000000  682015.000000\n",
       "                   mean        0.000007       0.000007\n",
       "                   std         0.002620       0.002708\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         0.000000       0.000000\n",
       "                   50%         0.000000       0.000000\n",
       "                   75%         0.000000       0.000000\n",
       "                   max         1.000000       1.000000\n",
       "counter_statue_3.0 count  437158.000000  682015.000000\n",
       "                   mean        0.000043       0.000075\n",
       "                   std         0.006592       0.008647\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         0.000000       0.000000\n",
       "                   50%         0.000000       0.000000\n",
       "                   75%         0.000000       0.000000\n",
       "                   max         1.000000       1.000000\n",
       "counter_statue_4.0 count  437158.000000  682015.000000\n",
       "                   mean        0.000485       0.000723\n",
       "                   std         0.022016       0.026876\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         0.000000       0.000000\n",
       "                   50%         0.000000       0.000000\n",
       "                   75%         0.000000       0.000000\n",
       "                   max         1.000000       1.000000\n",
       "counter_statue_5.0 count  437158.000000  682015.000000\n",
       "                   mean        0.006611       0.003418\n",
       "                   std         0.081038       0.058362\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         0.000000       0.000000\n",
       "                   50%         0.000000       0.000000\n",
       "                   75%         0.000000       0.000000\n",
       "                   max         1.000000       1.000000\n",
       "reading_remarque_6 count  437158.000000  682015.000000\n",
       "                   mean        0.491795       0.503517\n",
       "                   std         0.499933       0.499988\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         0.000000       0.000000\n",
       "                   50%         0.000000       1.000000\n",
       "                   75%         1.000000       1.000000\n",
       "                   max         1.000000       1.000000\n",
       "reading_remarque_7 count  437158.000000  682015.000000\n",
       "                   mean        0.000002       0.000254\n",
       "                   std         0.001512       0.015925\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         0.000000       0.000000\n",
       "                   50%         0.000000       0.000000\n",
       "                   75%         0.000000       0.000000\n",
       "                   max         1.000000       1.000000\n",
       "reading_remarque_8 count  437158.000000  682015.000000\n",
       "                   mean        0.204944       0.171499\n",
       "                   std         0.403661       0.376945\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         0.000000       0.000000\n",
       "                   50%         0.000000       0.000000\n",
       "                   75%         0.000000       0.000000\n",
       "                   max         1.000000       1.000000\n",
       "reading_remarque_9 count  437158.000000  682015.000000\n",
       "                   mean        0.303259       0.324730\n",
       "                   std         0.459667       0.468274\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         0.000000       0.000000\n",
       "                   50%         0.000000       0.000000\n",
       "                   75%         1.000000       1.000000\n",
       "                   max         1.000000       1.000000\n",
       "counter_type_ELEC  count  437158.000000  682015.000000\n",
       "                   mean        0.649957       0.712454\n",
       "                   std         0.476984       0.452619\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         0.000000       0.000000\n",
       "                   50%         1.000000       1.000000\n",
       "                   75%         1.000000       1.000000\n",
       "                   max         1.000000       1.000000\n",
       "counter_type_GAZ   count  437158.000000  682015.000000\n",
       "                   mean        0.350043       0.287546\n",
       "                   std         0.476984       0.452619\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         0.000000       0.000000\n",
       "                   50%         0.000000       0.000000\n",
       "                   75%         1.000000       1.000000\n",
       "                   max         1.000000       1.000000\n",
       "target             count  437158.000000  682015.000000\n",
       "                   mean        0.073717       0.082184\n",
       "                   std         0.261310       0.274646\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         0.000000       0.000000\n",
       "                   50%         0.000000       0.000000\n",
       "                   75%         0.000000       0.000000\n",
       "                   max         1.000000       1.000000\n",
       "fraud_pred         count  437158.000000  682015.000000\n",
       "                   mean        0.926283       0.082184\n",
       "                   std         0.261310       0.274646\n",
       "                   min         0.000000       0.000000\n",
       "                   25%         1.000000       0.000000\n",
       "                   50%         1.000000       0.000000\n",
       "                   75%         1.000000       0.000000\n",
       "                   max         1.000000       1.000000\n",
       "no_fraud_proba     count  437158.000000  682015.000000\n",
       "                   mean        0.498327       0.502750\n",
       "                   std         0.001980       0.007922\n",
       "                   min         0.484773       0.484687\n",
       "                   25%         0.497341       0.500867\n",
       "                   50%         0.498597       0.501998\n",
       "                   75%         0.499417       0.503775\n",
       "                   max         0.517550       0.671550\n",
       "fraud_proba        count  437158.000000  682015.000000\n",
       "                   mean        0.501673       0.497250\n",
       "                   std         0.001980       0.007922\n",
       "                   min         0.482450       0.328450\n",
       "                   25%         0.500583       0.496225\n",
       "                   50%         0.501403       0.498002\n",
       "                   75%         0.502659       0.499133\n",
       "                   max         0.515227       0.515313"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "display(df_test.groupby('pred_corr').describe().T)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['member_years', 'index_change_month', 'quantity_counters',\n",
       "       'client_catg_11', 'client_catg_12', 'client_catg_51', 'region_101',\n",
       "       'region_103', 'region_104', 'region_105', 'region_106', 'region_107',\n",
       "       'region_199', 'region_206', 'region_301', 'region_302', 'region_303',\n",
       "       'region_304', 'region_305', 'region_306', 'region_307', 'region_308',\n",
       "       'region_309', 'region_310', 'region_311', 'region_312', 'region_313',\n",
       "       'region_371', 'region_372', 'region_379', 'region_399', 'tarif_type_0',\n",
       "       'tarif_type_10', 'tarif_type_11', 'tarif_type_12', 'tarif_type_13',\n",
       "       'tarif_type_14', 'tarif_type_15', 'tarif_type_29', 'tarif_type_40',\n",
       "       'tarif_type_45', 'counter_statue_0.0', 'counter_statue_1.0',\n",
       "       'counter_statue_2.0', 'counter_statue_3.0', 'counter_statue_4.0',\n",
       "       'counter_statue_5.0', 'reading_remarque_6', 'reading_remarque_7',\n",
       "       'reading_remarque_8', 'reading_remarque_9', 'counter_type_ELEC',\n",
       "       'counter_type_GAZ', 'target', 'fraud_pred', 'no_fraud_proba',\n",
       "       'fraud_proba', 'pred_corr'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_test.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>member_years</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>counter_type_ELEC</th>\n",
       "      <th>pred_corr</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">0</th>\n",
       "      <th>False</th>\n",
       "      <td>153024</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>True</th>\n",
       "      <td>196111</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">1</th>\n",
       "      <th>False</th>\n",
       "      <td>284134</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>True</th>\n",
       "      <td>485904</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                             member_years\n",
       "counter_type_ELEC pred_corr              \n",
       "0                 False            153024\n",
       "                  True             196111\n",
       "1                 False            284134\n",
       "                  True             485904"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_performance = df_test.groupby(['counter_type_ELEC', 'pred_corr'])[['member_years']].count()\n",
    "df_performance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Percent of gas clients correctly detected:  0.5617053575264582\n",
      "Percent of electricity clients correctly detected:  0.6310130149421197\n"
     ]
    }
   ],
   "source": [
    "gas_false_pred = df_performance.loc[0].loc[False][0]\n",
    "gas_corr_pred = df_performance.loc[0].loc[True][0]\n",
    "gas_all_test = gas_false_pred + gas_corr_pred\n",
    "elec_false_pred = df_performance.loc[1].loc[False][0]\n",
    "elec_corr_pred = df_performance.loc[1].loc[True][0]\n",
    "elec_all_test = elec_false_pred + elec_corr_pred\n",
    "print('Percent of gas clients correctly detected: ', gas_corr_pred/gas_all_test)\n",
    "print('Percent of electricity clients correctly detected: ', elec_corr_pred/elec_all_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "566.397344729248"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# True positive rate: TP/(TP+FN) = 0.63\n",
    "tpr = 0.63\n",
    "# False positive rate: FP/(TN+FP) = 0.39\n",
    "fpr = 0.39\n",
    "# target value count\n",
    "# 0    4123581\n",
    "# 1     353109\n",
    "total_loss = 200000000\n",
    "num_fraud = 353109\n",
    "avg_loss = total_loss / num_fraud\n",
    "cost_falsely_acc = 0\n",
    "gain_per_client = avg_loss * tpr - cost_falsely_acc * fpr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "126000000.0"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# expected gain during next period by detecting fraud in 63% of the fraud cases\n",
    "total_gain = 0.63 * total_loss\n",
    "total_gain"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.3"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}