Redes_Neurais_Mini_Projeto 1.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "SptUxj9bkrNO"
   },
   "source": [
    "# IF702 - REDES NEURAIS\n",
    "\n",
    "**Grupo:**\n",
    "- Adriano Felipe Cabral Filho (afcf)\n",
    "- Felipe Rodrigues de Souza (frs3)\n",
    "- Lucas Vinícius José da Silva (lvjs)\n",
    "- Pedro José Carneiro de Souza (pjcs) \n",
    "- Vitor Sousa Silva (vss2)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "vvKzsWFyttnd"
   },
   "source": [
    "# Reconhecimento de Dígitos com MLPs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "e2BS5wMWm-bL"
   },
   "source": [
    "## Importação de datasets e definição de classes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "973VROBYXAZC",
    "outputId": "20716c5a-4e22-4e1e-bb9e-153589eeae9a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[K     |████████████████████████████████| 308 kB 12.9 MB/s \n",
      "\u001b[K     |████████████████████████████████| 80 kB 7.5 MB/s \n",
      "\u001b[K     |████████████████████████████████| 209 kB 74.4 MB/s \n",
      "\u001b[K     |████████████████████████████████| 75 kB 3.9 MB/s \n",
      "\u001b[K     |████████████████████████████████| 111 kB 73.3 MB/s \n",
      "\u001b[K     |████████████████████████████████| 144 kB 66.0 MB/s \n",
      "\u001b[K     |████████████████████████████████| 49 kB 5.3 MB/s \n",
      "\u001b[?25h  Building wheel for pyperclip (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
      "time: 264 µs (started: 2021-10-20 01:24:40 +00:00)\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "import keras\n",
    "from keras.layers import LSTM\n",
    "import numpy as np\n",
    "from sklearn.metrics import accuracy_score, plot_confusion_matrix, confusion_matrix, classification_report\n",
    "!pip install ipython-autotime --quiet\n",
    "!pip install optuna --quiet\n",
    "\n",
    "%load_ext autotime"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "de6fUm1hXL_T",
    "outputId": "c1c37563-fcbe-4998-e7cd-86fbde691006"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 216 ms (started: 2021-10-20 01:24:41 +00:00)\n"
     ]
    }
   ],
   "source": [
    "# Dataset de amostras de dígitos\n",
    "mnist = tf.keras.datasets.mnist"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "mhfm4UIvHuNi",
    "outputId": "2877555c-9421-484e-b238-11da4aa3eac3"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 1.36 ms (started: 2021-10-20 01:24:42 +00:00)\n"
     ]
    }
   ],
   "source": [
    "# Category\n",
    "classes = [\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\",\"8\",\"9\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "RhXWXPPSmhm2"
   },
   "source": [
    "## Experimento\n",
    "## Especificação:\n",
    "Para esse experimento, fixei alguns parâmetros para focar na mudança de outros\n",
    "Parâmetros fixados:\n",
    "- Taxa de aprendizado = 0.001 (Adam Algorithm - Kingma et al., 2014)\n",
    "- Número de épocas = 30 \n",
    "\n",
    "\n",
    "Parâmetros experimentados\n",
    "- Número de camadas ocultas = 1, 2 e 4\n",
    "- Número de neurônios = 16, 200\n",
    "- Função de ativação = ReLu, sigmoid"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "VLZhAqHMcPeE",
    "outputId": "61303443-0d78-4710-8a1b-35686c1b09d3"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 1.69 ms (started: 2021-10-20 01:24:48 +00:00)\n"
     ]
    }
   ],
   "source": [
    "#seed\n",
    "tf.random.set_seed(42)\n",
    "np.random.seed(42)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "2RhX_PYfVj4C"
   },
   "outputs": [],
   "source": [
    "def mlp1(params):\n",
    "  (x_train, y_train), (x_test, y_test) = mnist.load_data()\n",
    "  model = keras.models.Sequential()\n",
    "  #normalizando valores dos picels para ser entre 0 e 1\n",
    "  x_train, x_test = x_train/255.0, x_test/255.0\n",
    "  model.add(keras.layers.Flatten(input_shape=[28, 28]))\n",
    "  model.add(keras.layers.Dense(params['neurons'], activation=params['activation_func'])) \n",
    "  model.add(keras.layers.Dense(10, activation=\"softmax\"))\n",
    "  model.compile(loss=\"sparse_categorical_crossentropy\",optimizer=\"adam\",metrics=[\"accuracy\"])\n",
    "  history = model.fit(x_train, y_train, epochs=30, verbose=0)\n",
    "  y_pred = np.argmax(model.predict(x_test), axis=-1)\n",
    "  y_true = y_test\n",
    "  print(classification_report(y_true, y_pred))  \n",
    "\n",
    "def mlp2(params):\n",
    "  (x_train, y_train), (x_test, y_test) = mnist.load_data()\n",
    "  model = keras.models.Sequential()\n",
    "  #normalizando valores dos picels para ser entre 0 e 1\n",
    "  x_train, x_test = x_train/255.0, x_test/255.0\n",
    "  model.add(keras.layers.Flatten(input_shape=[28, 28]))\n",
    "  model.add(keras.layers.Dense(params['neurons'], activation=params['activation_func']))\n",
    "  model.add(keras.layers.Dense(params['neurons'], activation=params['activation_func'])) \n",
    "  model.add(keras.layers.Dense(10, activation=\"softmax\"))\n",
    "  model.compile(loss=\"sparse_categorical_crossentropy\",optimizer=\"adam\",metrics=[\"accuracy\"])\n",
    "  history = model.fit(x_train, y_train, epochs=30, verbose=0)\n",
    "  y_pred = np.argmax(model.predict(x_test), axis=-1)\n",
    "  y_true = y_test\n",
    "  print(classification_report(y_true, y_pred))  \n",
    "\n",
    "def mlp4(params):\n",
    "  (x_train, y_train), (x_test, y_test) = mnist.load_data()\n",
    "  model = keras.models.Sequential()\n",
    "  #normalizando valores dos picels para ser entre 0 e 1\n",
    "  x_train, x_test = x_train/255.0, x_test/255.0\n",
    "  model.add(keras.layers.Flatten(input_shape=[28, 28]))\n",
    "  model.add(keras.layers.Dense(params['neurons'], activation=params['activation_func']))\n",
    "  model.add(keras.layers.Dense(params['neurons'], activation=params['activation_func']))\n",
    "  model.add(keras.layers.Dense(params['neurons'], activation=params['activation_func']))\n",
    "  model.add(keras.layers.Dense(params['neurons'], activation=params['activation_func']))  \n",
    "  model.add(keras.layers.Dense(10, activation=\"softmax\"))\n",
    "  model.compile(loss=\"sparse_categorical_crossentropy\",optimizer=\"adam\",metrics=[\"accuracy\"])\n",
    "  history = model.fit(x_train, y_train, epochs=30, verbose=0)\n",
    "  y_pred = np.argmax(model.predict(x_test), axis=-1)\n",
    "  y_true = y_test\n",
    "  print(classification_report(y_true, y_pred))\n",
    "  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "hbzHbAp2YNPx",
    "outputId": "e0213431-14ea-4a3c-b81d-f868174d576e"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.97      0.98      0.97       980\n",
      "           1       0.98      0.98      0.98      1135\n",
      "           2       0.96      0.94      0.95      1032\n",
      "           3       0.92      0.95      0.94      1010\n",
      "           4       0.95      0.96      0.95       982\n",
      "           5       0.94      0.93      0.93       892\n",
      "           6       0.95      0.97      0.96       958\n",
      "           7       0.95      0.95      0.95      1028\n",
      "           8       0.94      0.94      0.94       974\n",
      "           9       0.95      0.92      0.93      1009\n",
      "\n",
      "    accuracy                           0.95     10000\n",
      "   macro avg       0.95      0.95      0.95     10000\n",
      "weighted avg       0.95      0.95      0.95     10000\n",
      "\n",
      "time: 2min 23s (started: 2021-10-17 05:44:06 +00:00)\n"
     ]
    }
   ],
   "source": [
    "#1 camada, 16 neuronios, Sigmoid\n",
    "\n",
    "params = {\n",
    "    \"activation_func\":\"sigmoid\",\n",
    "    \"neurons\": 16\n",
    "}\n",
    "mlp1(params)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "EftKEyRvblJK",
    "outputId": "a489239e-12a2-4027-c516-0b0024536466"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.97      0.98      0.97       980\n",
      "           1       0.97      0.98      0.98      1135\n",
      "           2       0.95      0.92      0.94      1032\n",
      "           3       0.91      0.94      0.93      1010\n",
      "           4       0.95      0.96      0.95       982\n",
      "           5       0.93      0.91      0.92       892\n",
      "           6       0.96      0.96      0.96       958\n",
      "           7       0.96      0.94      0.95      1028\n",
      "           8       0.92      0.94      0.93       974\n",
      "           9       0.96      0.92      0.94      1009\n",
      "\n",
      "    accuracy                           0.95     10000\n",
      "   macro avg       0.95      0.95      0.95     10000\n",
      "weighted avg       0.95      0.95      0.95     10000\n",
      "\n",
      "time: 2min 23s (started: 2021-10-17 05:46:30 +00:00)\n"
     ]
    }
   ],
   "source": [
    "#2 camadas, 16 neuronios, Sigmoid\n",
    "\n",
    "params = {\n",
    "    \"activation_func\":\"sigmoid\",\n",
    "    \"neurons\": 16\n",
    "}\n",
    "mlp2(params)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "YD9dWcoLbfZH",
    "outputId": "e74481c3-65e2-49a7-e7cb-654ffc38696d"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.96      0.98      0.97       980\n",
      "           1       0.98      0.97      0.98      1135\n",
      "           2       0.94      0.93      0.93      1032\n",
      "           3       0.90      0.95      0.92      1010\n",
      "           4       0.96      0.94      0.95       982\n",
      "           5       0.90      0.90      0.90       892\n",
      "           6       0.96      0.95      0.95       958\n",
      "           7       0.94      0.96      0.95      1028\n",
      "           8       0.93      0.91      0.92       974\n",
      "           9       0.95      0.92      0.94      1009\n",
      "\n",
      "    accuracy                           0.94     10000\n",
      "   macro avg       0.94      0.94      0.94     10000\n",
      "weighted avg       0.94      0.94      0.94     10000\n",
      "\n",
      "time: 1min 49s (started: 2021-10-17 05:48:53 +00:00)\n"
     ]
    }
   ],
   "source": [
    "#4 camadas, 16 neuronios, Sigmoid\n",
    "\n",
    "params = {\n",
    "    \"activation_func\":\"sigmoid\",\n",
    "    \"neurons\": 16\n",
    "}\n",
    "mlp4(params)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "r3z6Sc80bdbA",
    "outputId": "f7a0ae25-1c36-4817-fdf7-fba6eb550e04"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.98      0.99      0.98       980\n",
      "           1       0.99      0.99      0.99      1135\n",
      "           2       0.98      0.97      0.98      1032\n",
      "           3       0.97      0.99      0.98      1010\n",
      "           4       0.98      0.99      0.98       982\n",
      "           5       0.98      0.97      0.98       892\n",
      "           6       0.99      0.98      0.98       958\n",
      "           7       0.98      0.98      0.98      1028\n",
      "           8       0.97      0.97      0.97       974\n",
      "           9       0.98      0.97      0.98      1009\n",
      "\n",
      "    accuracy                           0.98     10000\n",
      "   macro avg       0.98      0.98      0.98     10000\n",
      "weighted avg       0.98      0.98      0.98     10000\n",
      "\n",
      "time: 1min 28s (started: 2021-10-17 05:50:43 +00:00)\n"
     ]
    }
   ],
   "source": [
    "#1 camada, 200 neuronios, Sigmoid\n",
    "\n",
    "params = {\n",
    "    \"activation_func\":\"sigmoid\",\n",
    "    \"neurons\": 200\n",
    "}\n",
    "mlp1(params)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "WbBGDMqsWg1p",
    "outputId": "c44440bb-bf03-4348-ac3f-859f2ae2e9ba"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.98      0.99      0.98       980\n",
      "           1       0.99      0.99      0.99      1135\n",
      "           2       0.97      0.98      0.98      1032\n",
      "           3       0.96      0.98      0.97      1010\n",
      "           4       0.99      0.98      0.98       982\n",
      "           5       0.98      0.96      0.97       892\n",
      "           6       0.98      0.98      0.98       958\n",
      "           7       0.98      0.98      0.98      1028\n",
      "           8       0.98      0.97      0.97       974\n",
      "           9       0.98      0.98      0.98      1009\n",
      "\n",
      "    accuracy                           0.98     10000\n",
      "   macro avg       0.98      0.98      0.98     10000\n",
      "weighted avg       0.98      0.98      0.98     10000\n",
      "\n",
      "time: 2min 23s (started: 2021-10-17 05:52:11 +00:00)\n"
     ]
    }
   ],
   "source": [
    "#2 camadas, 200 neuronios, Sigmoid\n",
    "\n",
    "params = {\n",
    "    \"activation_func\":\"sigmoid\",\n",
    "    \"neurons\": 200\n",
    "}\n",
    "mlp2(params)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "5Z4SBAmexciX",
    "outputId": "bb82e5a4-da14-4836-e6db-60fa18abb452"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.98      0.99      0.99       980\n",
      "           1       0.99      0.99      0.99      1135\n",
      "           2       0.98      0.97      0.98      1032\n",
      "           3       0.97      0.98      0.98      1010\n",
      "           4       0.98      0.98      0.98       982\n",
      "           5       0.97      0.98      0.98       892\n",
      "           6       0.99      0.98      0.98       958\n",
      "           7       0.98      0.97      0.98      1028\n",
      "           8       0.97      0.97      0.97       974\n",
      "           9       0.98      0.97      0.97      1009\n",
      "\n",
      "    accuracy                           0.98     10000\n",
      "   macro avg       0.98      0.98      0.98     10000\n",
      "weighted avg       0.98      0.98      0.98     10000\n",
      "\n",
      "time: 1min 53s (started: 2021-10-17 05:54:35 +00:00)\n"
     ]
    }
   ],
   "source": [
    "#4 camadas, 200 neuronios, Sigmoid\n",
    "\n",
    "params = {\n",
    "    \"activation_func\":\"sigmoid\",\n",
    "    \"neurons\": 200\n",
    "}\n",
    "mlp4(params)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "mOSkCeZHxQgH",
    "outputId": "1a59aa71-9e21-4e81-af3a-b16b3ae98eea"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.97      0.97      0.97       980\n",
      "           1       0.98      0.98      0.98      1135\n",
      "           2       0.96      0.94      0.95      1032\n",
      "           3       0.93      0.95      0.94      1010\n",
      "           4       0.94      0.96      0.95       982\n",
      "           5       0.95      0.92      0.94       892\n",
      "           6       0.96      0.95      0.96       958\n",
      "           7       0.94      0.96      0.95      1028\n",
      "           8       0.92      0.93      0.93       974\n",
      "           9       0.96      0.93      0.95      1009\n",
      "\n",
      "    accuracy                           0.95     10000\n",
      "   macro avg       0.95      0.95      0.95     10000\n",
      "weighted avg       0.95      0.95      0.95     10000\n",
      "\n",
      "time: 1min 30s (started: 2021-10-17 05:56:28 +00:00)\n"
     ]
    }
   ],
   "source": [
    "#1 camada, 16 neuronios, Relu\n",
    "\n",
    "params = {\n",
    "    \"activation_func\":\"relu\",\n",
    "    \"neurons\": 16\n",
    "}\n",
    "mlp1(params)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "y5XNwyQKwRxg",
    "outputId": "14e16f2b-b369-4ef0-d258-d8b5416e4374"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.98      0.98      0.98       980\n",
      "           1       0.98      0.98      0.98      1135\n",
      "           2       0.98      0.93      0.95      1032\n",
      "           3       0.93      0.96      0.95      1010\n",
      "           4       0.93      0.97      0.95       982\n",
      "           5       0.94      0.93      0.93       892\n",
      "           6       0.98      0.94      0.96       958\n",
      "           7       0.94      0.97      0.96      1028\n",
      "           8       0.93      0.92      0.93       974\n",
      "           9       0.94      0.94      0.94      1009\n",
      "\n",
      "    accuracy                           0.95     10000\n",
      "   macro avg       0.95      0.95      0.95     10000\n",
      "weighted avg       0.95      0.95      0.95     10000\n",
      "\n",
      "time: 2min 23s (started: 2021-10-17 05:57:59 +00:00)\n"
     ]
    }
   ],
   "source": [
    "#2 camadas, 16 neuronios, ReLu\n",
    "\n",
    "params = {\n",
    "    \"activation_func\":\"relu\",\n",
    "    \"neurons\": 16\n",
    "}\n",
    "mlp2(params)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "4u0gCr6TZgcZ",
    "outputId": "f2ea4655-b361-4448-9041-eb319e75ef3b"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.97      0.97      0.97       980\n",
      "           1       0.98      0.99      0.98      1135\n",
      "           2       0.98      0.92      0.95      1032\n",
      "           3       0.95      0.97      0.96      1010\n",
      "           4       0.96      0.95      0.96       982\n",
      "           5       0.96      0.94      0.95       892\n",
      "           6       0.96      0.96      0.96       958\n",
      "           7       0.95      0.96      0.95      1028\n",
      "           8       0.93      0.96      0.94       974\n",
      "           9       0.95      0.95      0.95      1009\n",
      "\n",
      "    accuracy                           0.96     10000\n",
      "   macro avg       0.96      0.96      0.96     10000\n",
      "weighted avg       0.96      0.96      0.96     10000\n",
      "\n",
      "time: 1min 50s (started: 2021-10-17 06:00:23 +00:00)\n"
     ]
    }
   ],
   "source": [
    "#4 camadas, 16 neuronios, ReLu\n",
    "\n",
    "params = {\n",
    "    \"activation_func\":\"relu\",\n",
    "    \"neurons\": 16\n",
    "}\n",
    "mlp4(params)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "yT55EcbAZLqq",
    "outputId": "8f94fe9b-3419-48fd-f9bf-967c8c9bebdc"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.98      0.99      0.99       980\n",
      "           1       0.99      0.99      0.99      1135\n",
      "           2       0.98      0.98      0.98      1032\n",
      "           3       0.97      0.99      0.98      1010\n",
      "           4       0.97      0.98      0.98       982\n",
      "           5       0.98      0.97      0.97       892\n",
      "           6       0.98      0.98      0.98       958\n",
      "           7       0.98      0.98      0.98      1028\n",
      "           8       0.98      0.97      0.98       974\n",
      "           9       0.98      0.97      0.98      1009\n",
      "\n",
      "    accuracy                           0.98     10000\n",
      "   macro avg       0.98      0.98      0.98     10000\n",
      "weighted avg       0.98      0.98      0.98     10000\n",
      "\n",
      "time: 2min 23s (started: 2021-10-17 06:02:13 +00:00)\n"
     ]
    }
   ],
   "source": [
    "#1 camada, 200 neuronios, Relu\n",
    "\n",
    "params = {\n",
    "    \"activation_func\":\"relu\",\n",
    "    \"neurons\": 200\n",
    "}\n",
    "mlp1(params)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "ewcbK_Y2ZIB1",
    "outputId": "04fa9910-2988-4665-d317-2ccf05cfc872"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.98      0.99      0.99       980\n",
      "           1       0.99      0.99      0.99      1135\n",
      "           2       0.99      0.97      0.98      1032\n",
      "           3       0.97      0.98      0.97      1010\n",
      "           4       0.97      0.98      0.98       982\n",
      "           5       0.98      0.98      0.98       892\n",
      "           6       0.98      0.99      0.98       958\n",
      "           7       0.98      0.98      0.98      1028\n",
      "           8       0.99      0.96      0.97       974\n",
      "           9       0.97      0.97      0.97      1009\n",
      "\n",
      "    accuracy                           0.98     10000\n",
      "   macro avg       0.98      0.98      0.98     10000\n",
      "weighted avg       0.98      0.98      0.98     10000\n",
      "\n",
      "time: 2min 23s (started: 2021-10-17 06:04:36 +00:00)\n"
     ]
    }
   ],
   "source": [
    "#2 camadas, 200 neuronios, ReLu\n",
    "\n",
    "params = {\n",
    "    \"activation_func\":\"relu\",\n",
    "    \"neurons\": 200\n",
    "}\n",
    "mlp2(params)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "fHquwVeiZGXS",
    "outputId": "328dc1b6-ad01-471e-de25-1f10cdd5d64e"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       1.00      0.98      0.99       980\n",
      "           1       0.99      0.99      0.99      1135\n",
      "           2       0.98      0.99      0.98      1032\n",
      "           3       0.98      0.98      0.98      1010\n",
      "           4       0.98      0.97      0.98       982\n",
      "           5       0.98      0.99      0.98       892\n",
      "           6       0.98      0.99      0.98       958\n",
      "           7       0.97      0.99      0.98      1028\n",
      "           8       0.97      0.98      0.98       974\n",
      "           9       0.98      0.97      0.97      1009\n",
      "\n",
      "    accuracy                           0.98     10000\n",
      "   macro avg       0.98      0.98      0.98     10000\n",
      "weighted avg       0.98      0.98      0.98     10000\n",
      "\n",
      "time: 1min 51s (started: 2021-10-17 06:07:00 +00:00)\n"
     ]
    }
   ],
   "source": [
    "#4 camadas, 200 neuronios, ReLu\n",
    "\n",
    "params = {\n",
    "    \"activation_func\":\"relu\",\n",
    "    \"neurons\": 200\n",
    "}\n",
    "mlp4(params)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0cFU52tzqZRN"
   },
   "source": [
    "### Sobre os resultados\n",
    "\n",
    "Analisando os experimentos, vemos que é altamente eficaz aumentar o número de neurônios para aumentar a acurácia.\n",
    "Também podemos perceber que a função de ativação das camadas ocultas não têm muita influência nesse caso quando temos grande quantidade de neurônios."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "527oA3x-jwmE"
   },
   "source": [
    "##Experimento com tuning\n",
    "\n",
    "#########Fazer aqui a especificação do experimento(verificar como foi feito no tópico \"experimento\")#########\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "CnC5kc1NIVyi",
    "outputId": "c99a142d-468a-446c-b307-3b4235992fe6"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 1.7 s (started: 2021-10-20 01:26:06 +00:00)\n"
     ]
    }
   ],
   "source": [
    "import optuna"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "LHHK751kB-G0"
   },
   "source": [
    "### Estudos\n",
    "* Funções de ativação\n",
    "  * 9 utilizadas: Sigmoid, softmax, tanh, relu, softplus, softsign, selu, elu, exponential\n",
    "* Número de neurônios: valor entre 16 e 256\n",
    "* Função de cálculo de perda: sparse_categorical_crossentropy\n",
    "* Otimizadores: SGD, Adam, RMSprop, Adamax"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "VE4WfLJ5i5ip",
    "outputId": "5fd15d3b-55a4-4028-d6c3-333f839d9a56"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 101 ms (started: 2021-10-20 04:41:22 +00:00)\n"
     ]
    }
   ],
   "source": [
    "# BLOCOS DE TUNING\n",
    "def tuned_mlp1(trial):\n",
    "  (x_train, y_train), (x_test, y_test) = mnist.load_data()\n",
    "\n",
    "  #setando conjunto de hiper parametros que vão ser tunados\n",
    "  activation = trial.suggest_categorical('activation', ['sigmoid', 'softmax', 'tanh', 'relu','softplus','softsign','selu','elu','exponential'])\n",
    "  n_neurons = trial.suggest_int('n_neurons', 16,256)\n",
    "  loss = trial.suggest_categorical('loss', ['sparse_categorical_crossentropy'])\n",
    "  optimizer = trial.suggest_categorical('optimizer', ['sgd', 'adam','rmsprop','adamax'])\n",
    "  print(activation,n_neurons,loss,optimizer)\n",
    "  model = keras.models.Sequential()\n",
    "  #normalizando valores dos picels para ser entre 0 e 1\n",
    "  x_train, x_test = x_train/255.0, x_test/255.0\n",
    "  model.add(keras.layers.Flatten(input_shape=[28, 28]))\n",
    "  model.add(keras.layers.Dense(n_neurons, activation=activation)) \n",
    "  model.add(keras.layers.Dense(10, activation=\"softmax\"))\n",
    "  model.compile(loss=loss,optimizer=optimizer,metrics=[\"accuracy\"])\n",
    "  history = model.fit(x_train, y_train, epochs=30, verbose=0)\n",
    "  y_pred = np.argmax(model.predict(x_test), axis=-1)\n",
    "  y_true = y_test\n",
    "  return accuracy_score(y_true, y_pred)\n",
    "\n",
    "def tuned_mlp2(trial):\n",
    "  (x_train, y_train), (x_test, y_test) = mnist.load_data()\n",
    "\n",
    "  #setando conjunto de hiper parametros que vão ser tunados\n",
    "  activation = trial.suggest_categorical('activation', ['sigmoid', 'softmax', 'tanh', 'relu','softplus','softsign','selu','elu','exponential'])\n",
    "  n_neurons = trial.suggest_int('n_neurons', 16,256)\n",
    "  loss = trial.suggest_categorical('loss', ['sparse_categorical_crossentropy'])\n",
    "  optimizer = trial.suggest_categorical('optimizer', ['sgd', 'adam','rmsprop','adamax'])\n",
    "  print(activation,n_neurons,loss,optimizer)\n",
    "\n",
    "\n",
    "  model = keras.models.Sequential()\n",
    "  #normalizando valores dos picels para ser entre 0 e 1\n",
    "  x_train, x_test = x_train/255.0, x_test/255.0\n",
    "  model.add(keras.layers.Flatten(input_shape=[28, 28]))\n",
    "  model.add(keras.layers.Dense(n_neurons, activation=activation)) \n",
    "  model.add(keras.layers.Dense(n_neurons, activation=activation)) \n",
    "  model.add(keras.layers.Dense(10, activation=\"softmax\"))\n",
    "  model.compile(loss=loss,optimizer=optimizer,metrics=[\"accuracy\"])\n",
    "  history = model.fit(x_train, y_train, epochs=30, verbose=0)\n",
    "  y_pred = np.argmax(model.predict(x_test), axis=-1)\n",
    "  y_true = y_test\n",
    "  return accuracy_score(y_true, y_pred)\n",
    "\n",
    "def tuned_mlp4(trial):\n",
    "  (x_train, y_train), (x_test, y_test) = mnist.load_data()\n",
    "\n",
    "  #setando conjunto de hiper parametros que vão ser tunados\n",
    "  activation = trial.suggest_categorical('activation', ['sigmoid', 'softmax', 'tanh', 'relu','softplus','softsign','selu','elu','exponential'])\n",
    "  n_neurons = trial.suggest_int('n_neurons', 16,256)\n",
    "  loss = trial.suggest_categorical('loss', ['sparse_categorical_crossentropy'])\n",
    "  optimizer = trial.suggest_categorical('optimizer', ['sgd', 'adam','rmsprop','adamax'])\n",
    "  print(activation,n_neurons,loss,optimizer)\n",
    "\n",
    "\n",
    "  model = keras.models.Sequential()\n",
    "  #normalizando valores dos picels para ser entre 0 e 1\n",
    "  x_train, x_test = x_train/255.0, x_test/255.0\n",
    "  model.add(keras.layers.Flatten(input_shape=[28, 28]))\n",
    "  model.add(keras.layers.Dense(n_neurons, activation=activation)) \n",
    "  model.add(keras.layers.Dense(n_neurons, activation=activation)) \n",
    "  model.add(keras.layers.Dense(n_neurons, activation=activation)) \n",
    "  model.add(keras.layers.Dense(n_neurons, activation=activation)) \n",
    "  model.add(keras.layers.Dense(10, activation=\"softmax\"))\n",
    "  model.compile(loss=loss,optimizer=optimizer,metrics=[\"accuracy\"])\n",
    "  history = model.fit(x_train, y_train, epochs=30, verbose=0)\n",
    "  y_pred = np.argmax(model.predict(x_test), axis=-1)\n",
    "  y_true = y_test\n",
    "  return accuracy_score(y_true, y_pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "wmC2540rIYGR",
    "outputId": "ba232cae-d5fa-4a71-fe14-9790f64aaf7c"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 02:19:49,641]\u001b[0m A new study created in memory with name: no-name-8448f802-55a0-4afe-bc55-aff4c98f7a99\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "selu 22 sparse_categorical_crossentropy sgd\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 02:21:12,816]\u001b[0m Trial 0 finished with value: 0.9564 and parameters: {'activation': 'selu', 'n_neurons': 22, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'sgd'}. Best is trial 0 with value: 0.9564.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "softplus 18 sparse_categorical_crossentropy adam\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 02:22:35,889]\u001b[0m Trial 1 finished with value: 0.9589 and parameters: {'activation': 'softplus', 'n_neurons': 18, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adam'}. Best is trial 1 with value: 0.9589.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "relu 215 sparse_categorical_crossentropy sgd\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 02:24:14,338]\u001b[0m Trial 2 finished with value: 0.9751 and parameters: {'activation': 'relu', 'n_neurons': 215, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'sgd'}. Best is trial 2 with value: 0.9751.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "softmax 111 sparse_categorical_crossentropy sgd\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 02:25:34,970]\u001b[0m Trial 3 finished with value: 0.3631 and parameters: {'activation': 'softmax', 'n_neurons': 111, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'sgd'}. Best is trial 2 with value: 0.9751.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "softmax 192 sparse_categorical_crossentropy rmsprop\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 02:28:02,231]\u001b[0m Trial 4 finished with value: 0.941 and parameters: {'activation': 'softmax', 'n_neurons': 192, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'rmsprop'}. Best is trial 2 with value: 0.9751.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "relu 31 sparse_categorical_crossentropy sgd\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 02:28:59,886]\u001b[0m Trial 5 finished with value: 0.9591 and parameters: {'activation': 'relu', 'n_neurons': 31, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'sgd'}. Best is trial 2 with value: 0.9751.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "softmax 130 sparse_categorical_crossentropy rmsprop\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 02:31:18,100]\u001b[0m Trial 6 finished with value: 0.9366 and parameters: {'activation': 'softmax', 'n_neurons': 130, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'rmsprop'}. Best is trial 2 with value: 0.9751.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "exponential 214 sparse_categorical_crossentropy sgd\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 02:32:54,418]\u001b[0m Trial 7 finished with value: 0.9738 and parameters: {'activation': 'exponential', 'n_neurons': 214, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'sgd'}. Best is trial 2 with value: 0.9751.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "selu 130 sparse_categorical_crossentropy sgd\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 02:34:22,808]\u001b[0m Trial 8 finished with value: 0.9661 and parameters: {'activation': 'selu', 'n_neurons': 130, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'sgd'}. Best is trial 2 with value: 0.9751.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "relu 125 sparse_categorical_crossentropy rmsprop\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 02:36:35,691]\u001b[0m Trial 9 finished with value: 0.9745 and parameters: {'activation': 'relu', 'n_neurons': 125, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'rmsprop'}. Best is trial 2 with value: 0.9751.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "softsign 250 sparse_categorical_crossentropy adamax\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 02:38:32,846]\u001b[0m Trial 10 finished with value: 0.9801 and parameters: {'activation': 'softsign', 'n_neurons': 250, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adamax'}. Best is trial 10 with value: 0.9801.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "softsign 256 sparse_categorical_crossentropy adamax\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 02:40:27,548]\u001b[0m Trial 11 finished with value: 0.9814 and parameters: {'activation': 'softsign', 'n_neurons': 256, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adamax'}. Best is trial 11 with value: 0.9814.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "softsign 245 sparse_categorical_crossentropy adamax\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 02:42:28,311]\u001b[0m Trial 12 finished with value: 0.9801 and parameters: {'activation': 'softsign', 'n_neurons': 245, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adamax'}. Best is trial 11 with value: 0.9814.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "softsign 251 sparse_categorical_crossentropy adamax\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 02:44:36,325]\u001b[0m Trial 13 finished with value: 0.9805 and parameters: {'activation': 'softsign', 'n_neurons': 251, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adamax'}. Best is trial 11 with value: 0.9814.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "softsign 173 sparse_categorical_crossentropy adamax\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 02:47:00,020]\u001b[0m Trial 14 finished with value: 0.9779 and parameters: {'activation': 'softsign', 'n_neurons': 173, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adamax'}. Best is trial 11 with value: 0.9814.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tanh 80 sparse_categorical_crossentropy adamax\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 02:48:19,783]\u001b[0m Trial 15 finished with value: 0.9768 and parameters: {'activation': 'tanh', 'n_neurons': 80, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adamax'}. Best is trial 11 with value: 0.9814.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "elu 172 sparse_categorical_crossentropy adamax\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 02:50:09,400]\u001b[0m Trial 16 finished with value: 0.9795 and parameters: {'activation': 'elu', 'n_neurons': 172, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adamax'}. Best is trial 11 with value: 0.9814.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "sigmoid 255 sparse_categorical_crossentropy adam\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 02:52:08,207]\u001b[0m Trial 17 finished with value: 0.9815 and parameters: {'activation': 'sigmoid', 'n_neurons': 255, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adam'}. Best is trial 17 with value: 0.9815.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "sigmoid 222 sparse_categorical_crossentropy adam\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 02:53:59,783]\u001b[0m Trial 18 finished with value: 0.9811 and parameters: {'activation': 'sigmoid', 'n_neurons': 222, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adam'}. Best is trial 17 with value: 0.9815.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "sigmoid 164 sparse_categorical_crossentropy adam\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 02:55:40,417]\u001b[0m Trial 19 finished with value: 0.9789 and parameters: {'activation': 'sigmoid', 'n_neurons': 164, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adam'}. Best is trial 17 with value: 0.9815.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "sigmoid 228 sparse_categorical_crossentropy adam\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 02:57:33,313]\u001b[0m Trial 20 finished with value: 0.9816 and parameters: {'activation': 'sigmoid', 'n_neurons': 228, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adam'}. Best is trial 20 with value: 0.9816.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "sigmoid 227 sparse_categorical_crossentropy adam\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 02:59:57,018]\u001b[0m Trial 21 finished with value: 0.9806 and parameters: {'activation': 'sigmoid', 'n_neurons': 227, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adam'}. Best is trial 20 with value: 0.9816.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "sigmoid 194 sparse_categorical_crossentropy adam\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 03:01:44,059]\u001b[0m Trial 22 finished with value: 0.9798 and parameters: {'activation': 'sigmoid', 'n_neurons': 194, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adam'}. Best is trial 20 with value: 0.9816.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "sigmoid 235 sparse_categorical_crossentropy adam\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 03:04:08,113]\u001b[0m Trial 23 finished with value: 0.9806 and parameters: {'activation': 'sigmoid', 'n_neurons': 235, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adam'}. Best is trial 20 with value: 0.9816.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "elu 255 sparse_categorical_crossentropy adam\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 03:06:02,586]\u001b[0m Trial 24 finished with value: 0.9754 and parameters: {'activation': 'elu', 'n_neurons': 255, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adam'}. Best is trial 20 with value: 0.9816.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "exponential 201 sparse_categorical_crossentropy adam\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 03:07:48,528]\u001b[0m Trial 25 finished with value: 0.098 and parameters: {'activation': 'exponential', 'n_neurons': 201, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adam'}. Best is trial 20 with value: 0.9816.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "softplus 155 sparse_categorical_crossentropy adam\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 03:10:11,686]\u001b[0m Trial 26 finished with value: 0.9816 and parameters: {'activation': 'softplus', 'n_neurons': 155, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adam'}. Best is trial 20 with value: 0.9816.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "softplus 156 sparse_categorical_crossentropy adam\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 03:12:34,857]\u001b[0m Trial 27 finished with value: 0.98 and parameters: {'activation': 'softplus', 'n_neurons': 156, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adam'}. Best is trial 20 with value: 0.9816.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "softplus 59 sparse_categorical_crossentropy adam\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 03:13:57,989]\u001b[0m Trial 28 finished with value: 0.9761 and parameters: {'activation': 'softplus', 'n_neurons': 59, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adam'}. Best is trial 20 with value: 0.9816.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "selu 90 sparse_categorical_crossentropy adam\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 03:15:32,083]\u001b[0m Trial 29 finished with value: 0.9757 and parameters: {'activation': 'selu', 'n_neurons': 90, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adam'}. Best is trial 20 with value: 0.9816.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tanh 185 sparse_categorical_crossentropy adam\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 03:17:14,109]\u001b[0m Trial 30 finished with value: 0.9814 and parameters: {'activation': 'tanh', 'n_neurons': 185, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adam'}. Best is trial 20 with value: 0.9816.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tanh 151 sparse_categorical_crossentropy adam\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 03:19:37,250]\u001b[0m Trial 31 finished with value: 0.9752 and parameters: {'activation': 'tanh', 'n_neurons': 151, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adam'}. Best is trial 20 with value: 0.9816.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 59min 47s (started: 2021-10-20 02:19:49 +00:00)\n"
     ]
    }
   ],
   "source": [
    "study_1 = optuna.create_study(direction='maximize')\n",
    "study_1.optimize(tuned_mlp1,n_trials=32)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "oLXvwH_KT3Gp",
    "outputId": "b53995ca-8ff1-4d7d-89f6-2b26dc510c2b"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 03:24:23,011]\u001b[0m A new study created in memory with name: no-name-008273fd-c44d-40f6-aa94-e302fb6c8474\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tanh 51 sparse_categorical_crossentropy rmsprop\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 03:25:44,642]\u001b[0m Trial 0 finished with value: 0.9702 and parameters: {'activation': 'tanh', 'n_neurons': 51, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'rmsprop'}. Best is trial 0 with value: 0.9702.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "softmax 191 sparse_categorical_crossentropy sgd\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 03:28:07,857]\u001b[0m Trial 1 finished with value: 0.1135 and parameters: {'activation': 'softmax', 'n_neurons': 191, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'sgd'}. Best is trial 0 with value: 0.9702.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tanh 79 sparse_categorical_crossentropy sgd\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 03:29:22,036]\u001b[0m Trial 2 finished with value: 0.9728 and parameters: {'activation': 'tanh', 'n_neurons': 79, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'sgd'}. Best is trial 2 with value: 0.9728.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "relu 250 sparse_categorical_crossentropy adam\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 03:31:52,833]\u001b[0m Trial 3 finished with value: 0.9817 and parameters: {'activation': 'relu', 'n_neurons': 250, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adam'}. Best is trial 3 with value: 0.9817.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "relu 44 sparse_categorical_crossentropy rmsprop\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 03:33:16,142]\u001b[0m Trial 4 finished with value: 0.9708 and parameters: {'activation': 'relu', 'n_neurons': 44, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'rmsprop'}. Best is trial 3 with value: 0.9817.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "softmax 220 sparse_categorical_crossentropy rmsprop\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 03:36:39,975]\u001b[0m Trial 5 finished with value: 0.9455 and parameters: {'activation': 'softmax', 'n_neurons': 220, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'rmsprop'}. Best is trial 3 with value: 0.9817.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "exponential 231 sparse_categorical_crossentropy adamax\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 03:39:05,976]\u001b[0m Trial 6 finished with value: 0.9773 and parameters: {'activation': 'exponential', 'n_neurons': 231, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adamax'}. Best is trial 3 with value: 0.9817.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tanh 219 sparse_categorical_crossentropy adam\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 03:41:29,247]\u001b[0m Trial 7 finished with value: 0.9768 and parameters: {'activation': 'tanh', 'n_neurons': 219, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adam'}. Best is trial 3 with value: 0.9817.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "selu 43 sparse_categorical_crossentropy adamax\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 03:42:52,918]\u001b[0m Trial 8 finished with value: 0.9713 and parameters: {'activation': 'selu', 'n_neurons': 43, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adamax'}. Best is trial 3 with value: 0.9817.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "softmax 48 sparse_categorical_crossentropy sgd\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 03:44:02,461]\u001b[0m Trial 9 finished with value: 0.1135 and parameters: {'activation': 'softmax', 'n_neurons': 48, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'sgd'}. Best is trial 3 with value: 0.9817.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "sigmoid 146 sparse_categorical_crossentropy adam\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 03:45:55,508]\u001b[0m Trial 10 finished with value: 0.9796 and parameters: {'activation': 'sigmoid', 'n_neurons': 146, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adam'}. Best is trial 3 with value: 0.9817.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "sigmoid 138 sparse_categorical_crossentropy adam\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 03:48:18,724]\u001b[0m Trial 11 finished with value: 0.9758 and parameters: {'activation': 'sigmoid', 'n_neurons': 138, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adam'}. Best is trial 3 with value: 0.9817.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "softplus 155 sparse_categorical_crossentropy adam\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 03:50:26,218]\u001b[0m Trial 12 finished with value: 0.9809 and parameters: {'activation': 'softplus', 'n_neurons': 155, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adam'}. Best is trial 3 with value: 0.9817.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "softplus 174 sparse_categorical_crossentropy adam\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 03:52:40,631]\u001b[0m Trial 13 finished with value: 0.9801 and parameters: {'activation': 'softplus', 'n_neurons': 174, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adam'}. Best is trial 3 with value: 0.9817.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "softsign 256 sparse_categorical_crossentropy adam\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 03:56:04,172]\u001b[0m Trial 14 finished with value: 0.979 and parameters: {'activation': 'softsign', 'n_neurons': 256, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adam'}. Best is trial 3 with value: 0.9817.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "softplus 110 sparse_categorical_crossentropy adam\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 03:58:27,362]\u001b[0m Trial 15 finished with value: 0.9774 and parameters: {'activation': 'softplus', 'n_neurons': 110, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adam'}. Best is trial 3 with value: 0.9817.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "relu 180 sparse_categorical_crossentropy adam\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 04:00:36,459]\u001b[0m Trial 16 finished with value: 0.9805 and parameters: {'activation': 'relu', 'n_neurons': 180, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adam'}. Best is trial 3 with value: 0.9817.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "elu 92 sparse_categorical_crossentropy adam\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 04:02:28,563]\u001b[0m Trial 17 finished with value: 0.9771 and parameters: {'activation': 'elu', 'n_neurons': 92, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adam'}. Best is trial 3 with value: 0.9817.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "softplus 252 sparse_categorical_crossentropy adamax\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 04:05:33,366]\u001b[0m Trial 18 finished with value: 0.9797 and parameters: {'activation': 'softplus', 'n_neurons': 252, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adamax'}. Best is trial 3 with value: 0.9817.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "relu 158 sparse_categorical_crossentropy adam\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 04:07:56,590]\u001b[0m Trial 19 finished with value: 0.9818 and parameters: {'activation': 'relu', 'n_neurons': 158, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adam'}. Best is trial 19 with value: 0.9818.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "relu 105 sparse_categorical_crossentropy adam\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 04:10:19,788]\u001b[0m Trial 20 finished with value: 0.9784 and parameters: {'activation': 'relu', 'n_neurons': 105, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adam'}. Best is trial 19 with value: 0.9818.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "relu 159 sparse_categorical_crossentropy adam\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 04:12:43,134]\u001b[0m Trial 21 finished with value: 0.9802 and parameters: {'activation': 'relu', 'n_neurons': 159, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adam'}. Best is trial 19 with value: 0.9818.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "relu 203 sparse_categorical_crossentropy adam\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 04:14:56,272]\u001b[0m Trial 22 finished with value: 0.982 and parameters: {'activation': 'relu', 'n_neurons': 203, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adam'}. Best is trial 22 with value: 0.982.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "relu 207 sparse_categorical_crossentropy adam\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 04:17:19,538]\u001b[0m Trial 23 finished with value: 0.9822 and parameters: {'activation': 'relu', 'n_neurons': 207, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adam'}. Best is trial 23 with value: 0.9822.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "relu 207 sparse_categorical_crossentropy adam\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 04:19:43,570]\u001b[0m Trial 24 finished with value: 0.9789 and parameters: {'activation': 'relu', 'n_neurons': 207, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adam'}. Best is trial 23 with value: 0.9822.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "relu 198 sparse_categorical_crossentropy adam\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 04:21:55,156]\u001b[0m Trial 25 finished with value: 0.9803 and parameters: {'activation': 'relu', 'n_neurons': 198, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adam'}. Best is trial 23 with value: 0.9822.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "softsign 127 sparse_categorical_crossentropy sgd\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 04:23:37,264]\u001b[0m Trial 26 finished with value: 0.9692 and parameters: {'activation': 'softsign', 'n_neurons': 127, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'sgd'}. Best is trial 23 with value: 0.9822.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "exponential 171 sparse_categorical_crossentropy adamax\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 04:25:53,110]\u001b[0m Trial 27 finished with value: 0.9753 and parameters: {'activation': 'exponential', 'n_neurons': 171, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adamax'}. Best is trial 23 with value: 0.9822.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "elu 194 sparse_categorical_crossentropy rmsprop\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 04:29:16,741]\u001b[0m Trial 28 finished with value: 0.9811 and parameters: {'activation': 'elu', 'n_neurons': 194, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'rmsprop'}. Best is trial 23 with value: 0.9822.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "relu 229 sparse_categorical_crossentropy rmsprop\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 04:32:32,276]\u001b[0m Trial 29 finished with value: 0.9767 and parameters: {'activation': 'relu', 'n_neurons': 229, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'rmsprop'}. Best is trial 23 with value: 0.9822.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "selu 125 sparse_categorical_crossentropy adam\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 04:34:55,478]\u001b[0m Trial 30 finished with value: 0.9805 and parameters: {'activation': 'selu', 'n_neurons': 125, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adam'}. Best is trial 23 with value: 0.9822.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "relu 238 sparse_categorical_crossentropy adam\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 04:37:15,408]\u001b[0m Trial 31 finished with value: 0.9814 and parameters: {'activation': 'relu', 'n_neurons': 238, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adam'}. Best is trial 23 with value: 0.9822.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 1h 12min 52s (started: 2021-10-20 03:24:23 +00:00)\n"
     ]
    }
   ],
   "source": [
    "study_2 = optuna.create_study(direction='maximize')\n",
    "study_2.optimize(tuned_mlp2,n_trials=32)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "SZ8nvzSST3kl",
    "outputId": "ccd40fc7-bcc9-42c9-a8ca-0b133d624ef3"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 04:41:34,192]\u001b[0m A new study created in memory with name: no-name-350ca4a3-1a6f-443c-a5ed-8136e4cdb700\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "softplus 17 sparse_categorical_crossentropy adam\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 04:42:57,438]\u001b[0m Trial 0 finished with value: 0.9639 and parameters: {'activation': 'softplus', 'n_neurons': 17, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adam'}. Best is trial 0 with value: 0.9639.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tanh 209 sparse_categorical_crossentropy adamax\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 04:46:14,949]\u001b[0m Trial 1 finished with value: 0.9806 and parameters: {'activation': 'tanh', 'n_neurons': 209, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adamax'}. Best is trial 1 with value: 0.9806.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tanh 139 sparse_categorical_crossentropy rmsprop\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 04:49:17,385]\u001b[0m Trial 2 finished with value: 0.9774 and parameters: {'activation': 'tanh', 'n_neurons': 139, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'rmsprop'}. Best is trial 1 with value: 0.9806.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "softsign 25 sparse_categorical_crossentropy adam\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 04:50:41,766]\u001b[0m Trial 3 finished with value: 0.9643 and parameters: {'activation': 'softsign', 'n_neurons': 25, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adam'}. Best is trial 1 with value: 0.9806.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "sigmoid 113 sparse_categorical_crossentropy sgd\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 04:52:32,059]\u001b[0m Trial 4 finished with value: 0.8731 and parameters: {'activation': 'sigmoid', 'n_neurons': 113, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'sgd'}. Best is trial 1 with value: 0.9806.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "exponential 222 sparse_categorical_crossentropy rmsprop\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 04:56:45,930]\u001b[0m Trial 5 finished with value: 0.098 and parameters: {'activation': 'exponential', 'n_neurons': 222, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'rmsprop'}. Best is trial 1 with value: 0.9806.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "elu 135 sparse_categorical_crossentropy adamax\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 05:00:10,341]\u001b[0m Trial 6 finished with value: 0.9798 and parameters: {'activation': 'elu', 'n_neurons': 135, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adamax'}. Best is trial 1 with value: 0.9806.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "softplus 115 sparse_categorical_crossentropy sgd\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 05:02:14,881]\u001b[0m Trial 7 finished with value: 0.9708 and parameters: {'activation': 'softplus', 'n_neurons': 115, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'sgd'}. Best is trial 1 with value: 0.9806.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "relu 157 sparse_categorical_crossentropy rmsprop\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 05:05:33,301]\u001b[0m Trial 8 finished with value: 0.975 and parameters: {'activation': 'relu', 'n_neurons': 157, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'rmsprop'}. Best is trial 1 with value: 0.9806.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "elu 52 sparse_categorical_crossentropy rmsprop\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 05:07:56,715]\u001b[0m Trial 9 finished with value: 0.9687 and parameters: {'activation': 'elu', 'n_neurons': 52, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'rmsprop'}. Best is trial 1 with value: 0.9806.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "softmax 243 sparse_categorical_crossentropy adamax\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 05:11:50,085]\u001b[0m Trial 10 finished with value: 0.1135 and parameters: {'activation': 'softmax', 'n_neurons': 243, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adamax'}. Best is trial 1 with value: 0.9806.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "elu 193 sparse_categorical_crossentropy adamax\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 05:14:57,845]\u001b[0m Trial 11 finished with value: 0.9816 and parameters: {'activation': 'elu', 'n_neurons': 193, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adamax'}. Best is trial 11 with value: 0.9816.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tanh 198 sparse_categorical_crossentropy adamax\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 05:18:00,358]\u001b[0m Trial 12 finished with value: 0.9816 and parameters: {'activation': 'tanh', 'n_neurons': 198, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adamax'}. Best is trial 11 with value: 0.9816.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "selu 189 sparse_categorical_crossentropy adamax\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 05:21:04,667]\u001b[0m Trial 13 finished with value: 0.9816 and parameters: {'activation': 'selu', 'n_neurons': 189, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adamax'}. Best is trial 11 with value: 0.9816.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "elu 180 sparse_categorical_crossentropy adamax\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 05:24:01,762]\u001b[0m Trial 14 finished with value: 0.9808 and parameters: {'activation': 'elu', 'n_neurons': 180, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adamax'}. Best is trial 11 with value: 0.9816.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tanh 255 sparse_categorical_crossentropy adamax\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 05:28:25,246]\u001b[0m Trial 15 finished with value: 0.9824 and parameters: {'activation': 'tanh', 'n_neurons': 255, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adamax'}. Best is trial 15 with value: 0.9824.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "sigmoid 253 sparse_categorical_crossentropy adamax\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 05:32:20,920]\u001b[0m Trial 16 finished with value: 0.9782 and parameters: {'activation': 'sigmoid', 'n_neurons': 253, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adamax'}. Best is trial 15 with value: 0.9824.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "selu 236 sparse_categorical_crossentropy adamax\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 05:36:44,532]\u001b[0m Trial 17 finished with value: 0.9828 and parameters: {'activation': 'selu', 'n_neurons': 236, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adamax'}. Best is trial 17 with value: 0.9828.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "selu 224 sparse_categorical_crossentropy sgd\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 05:39:46,025]\u001b[0m Trial 18 finished with value: 0.9785 and parameters: {'activation': 'selu', 'n_neurons': 224, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'sgd'}. Best is trial 17 with value: 0.9828.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "selu 255 sparse_categorical_crossentropy adam\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 05:43:41,691]\u001b[0m Trial 19 finished with value: 0.9781 and parameters: {'activation': 'selu', 'n_neurons': 255, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adam'}. Best is trial 17 with value: 0.9828.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "exponential 79 sparse_categorical_crossentropy adamax\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 05:45:28,082]\u001b[0m Trial 20 finished with value: 0.098 and parameters: {'activation': 'exponential', 'n_neurons': 79, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adamax'}. Best is trial 17 with value: 0.9828.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "selu 229 sparse_categorical_crossentropy adamax\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 05:49:10,239]\u001b[0m Trial 21 finished with value: 0.98 and parameters: {'activation': 'selu', 'n_neurons': 229, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adamax'}. Best is trial 17 with value: 0.9828.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "selu 174 sparse_categorical_crossentropy adamax\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 05:52:01,437]\u001b[0m Trial 22 finished with value: 0.9812 and parameters: {'activation': 'selu', 'n_neurons': 174, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adamax'}. Best is trial 17 with value: 0.9828.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "softsign 215 sparse_categorical_crossentropy adamax\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 05:55:22,924]\u001b[0m Trial 23 finished with value: 0.9801 and parameters: {'activation': 'softsign', 'n_neurons': 215, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adamax'}. Best is trial 17 with value: 0.9828.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "relu 236 sparse_categorical_crossentropy adamax\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 05:59:46,532]\u001b[0m Trial 24 finished with value: 0.9828 and parameters: {'activation': 'relu', 'n_neurons': 236, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adamax'}. Best is trial 17 with value: 0.9828.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "relu 242 sparse_categorical_crossentropy adamax\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 06:04:10,062]\u001b[0m Trial 25 finished with value: 0.9756 and parameters: {'activation': 'relu', 'n_neurons': 242, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adamax'}. Best is trial 17 with value: 0.9828.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "relu 238 sparse_categorical_crossentropy adamax\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 06:07:50,357]\u001b[0m Trial 26 finished with value: 0.9788 and parameters: {'activation': 'relu', 'n_neurons': 238, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adamax'}. Best is trial 17 with value: 0.9828.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tanh 208 sparse_categorical_crossentropy adam\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 06:10:53,165]\u001b[0m Trial 27 finished with value: 0.9776 and parameters: {'activation': 'tanh', 'n_neurons': 208, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adam'}. Best is trial 17 with value: 0.9828.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "softmax 165 sparse_categorical_crossentropy sgd\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 06:13:25,557]\u001b[0m Trial 28 finished with value: 0.1135 and parameters: {'activation': 'softmax', 'n_neurons': 165, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'sgd'}. Best is trial 17 with value: 0.9828.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "softplus 254 sparse_categorical_crossentropy adam\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 06:17:41,691]\u001b[0m Trial 29 finished with value: 0.9816 and parameters: {'activation': 'softplus', 'n_neurons': 254, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adam'}. Best is trial 17 with value: 0.9828.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "relu 233 sparse_categorical_crossentropy adamax\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 06:21:20,465]\u001b[0m Trial 30 finished with value: 0.9804 and parameters: {'activation': 'relu', 'n_neurons': 233, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adamax'}. Best is trial 17 with value: 0.9828.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "softplus 252 sparse_categorical_crossentropy adam\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-10-20 06:25:39,390]\u001b[0m Trial 31 finished with value: 0.9819 and parameters: {'activation': 'softplus', 'n_neurons': 252, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adam'}. Best is trial 17 with value: 0.9828.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 1h 44min 5s (started: 2021-10-20 04:41:34 +00:00)\n"
     ]
    }
   ],
   "source": [
    "study_4 = optuna.create_study(direction='maximize')\n",
    "study_4.optimize(tuned_mlp4,n_trials=32)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "RbHKRYi-YR9f",
    "outputId": "1d13db3a-d1b4-4e56-ecba-8bbcfda27adb"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy: 0.9816\n",
      "Best hyperparameters: {'activation': 'sigmoid', 'n_neurons': 228, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adam'}\n",
      "FrozenTrial(number=20, values=[0.9816], datetime_start=datetime.datetime(2021, 10, 20, 2, 55, 40, 419629), datetime_complete=datetime.datetime(2021, 10, 20, 2, 57, 33, 313460), params={'activation': 'sigmoid', 'n_neurons': 228, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adam'}, distributions={'activation': CategoricalDistribution(choices=('sigmoid', 'softmax', 'tanh', 'relu', 'softplus', 'softsign', 'selu', 'elu', 'exponential')), 'n_neurons': IntUniformDistribution(high=256, low=16, step=1), 'loss': CategoricalDistribution(choices=('sparse_categorical_crossentropy',)), 'optimizer': CategoricalDistribution(choices=('sgd', 'adam', 'rmsprop', 'adamax'))}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=20, state=TrialState.COMPLETE, value=None)\n",
      "time: 4.85 ms (started: 2021-10-20 06:26:53 +00:00)\n"
     ]
    }
   ],
   "source": [
    "trial = study_1.best_trial\n",
    "print(f'Accuracy: {trial.value}')\n",
    "print(f'Best hyperparameters: {trial.params}')\n",
    "print(trial)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "oyMn95qxYato",
    "outputId": "8d143742-2ec3-4a32-81d0-c17ba11c5572"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy: 0.9822\n",
      "Best hyperparameters: {'activation': 'relu', 'n_neurons': 207, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adam'}\n",
      "FrozenTrial(number=23, values=[0.9822], datetime_start=datetime.datetime(2021, 10, 20, 4, 14, 56, 274775), datetime_complete=datetime.datetime(2021, 10, 20, 4, 17, 19, 538143), params={'activation': 'relu', 'n_neurons': 207, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adam'}, distributions={'activation': CategoricalDistribution(choices=('sigmoid', 'softmax', 'tanh', 'relu', 'softplus', 'softsign', 'selu', 'elu', 'exponential')), 'n_neurons': IntUniformDistribution(high=256, low=16, step=1), 'loss': CategoricalDistribution(choices=('sparse_categorical_crossentropy',)), 'optimizer': CategoricalDistribution(choices=('sgd', 'adam', 'rmsprop', 'adamax'))}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=23, state=TrialState.COMPLETE, value=None)\n",
      "time: 2.64 ms (started: 2021-10-20 06:27:17 +00:00)\n"
     ]
    }
   ],
   "source": [
    "trial = study_2.best_trial\n",
    "print(f'Accuracy: {trial.value}')\n",
    "print(f'Best hyperparameters: {trial.params}')\n",
    "print(trial)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "CEv69NibYgJR",
    "outputId": "092a5840-3b22-4930-c488-70dacc55a923"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy: 0.9828\n",
      "Best hyperparameters: {'activation': 'selu', 'n_neurons': 236, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adamax'}\n",
      "FrozenTrial(number=17, values=[0.9828], datetime_start=datetime.datetime(2021, 10, 20, 5, 32, 20, 921783), datetime_complete=datetime.datetime(2021, 10, 20, 5, 36, 44, 532400), params={'activation': 'selu', 'n_neurons': 236, 'loss': 'sparse_categorical_crossentropy', 'optimizer': 'adamax'}, distributions={'activation': CategoricalDistribution(choices=('sigmoid', 'softmax', 'tanh', 'relu', 'softplus', 'softsign', 'selu', 'elu', 'exponential')), 'n_neurons': IntUniformDistribution(high=256, low=16, step=1), 'loss': CategoricalDistribution(choices=('sparse_categorical_crossentropy',)), 'optimizer': CategoricalDistribution(choices=('sgd', 'adam', 'rmsprop', 'adamax'))}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=17, state=TrialState.COMPLETE, value=None)\n",
      "time: 2.14 ms (started: 2021-10-20 06:32:08 +00:00)\n"
     ]
    }
   ],
   "source": [
    "trial = study_4.best_trial\n",
    "print(f'Accuracy: {trial.value}')\n",
    "print(f'Best hyperparameters: {trial.params}')\n",
    "print(trial)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "yAyIr6RXRB5C"
   },
   "source": [
    "Primeiro estudo\n",
    "* Acurácia: 98.16%\n",
    "* Melhor configuração de hiperparâmetros:\n",
    "* Função de ativação: Sigmoide\n",
    "* Número de neurônios: 228\n",
    "* Otimizador: Adam\n",
    "\n",
    "Segundo estudo\n",
    "* Acurácia: 98.22%\n",
    "* Melhor configuração de hiperparâmetros:\n",
    "* Função de ativação: Relu\n",
    "* Número de neurônios: 207\n",
    "* Otimizador: Adam\n",
    "\n",
    "Terceiro estudo\n",
    "* Acurácia: 98.28%\n",
    "* Melhor configuração de hiperparâmetros:\n",
    "* Função de ativação: Selu\n",
    "* Número de neurônios: 236\n",
    "* Otimizador: Adamax\n",
    "\n",
    "Concluímos que nos três estudos obtivemos resultados bastante semelhantes, dado que a variação da acurácia entre o melhor e o pior resultado foi de apenas 0.12%. Também percebemos que a função de ativação foi distinta nos três casos, e que o otimizador Adam foi o melhor em 2 dos nossos estudos."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ZXZMtjImC2pc"
   },
   "source": [
    "### Resultados"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 558
    },
    "id": "Eml3P_CTYhsi",
    "outputId": "266a27b0-ec10-43c4-a7d0-f6c0b951cea7"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>\n",
       "            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax) {MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>\n",
       "                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-latest.min.js\"></script>    \n",
       "            <div id=\"6fc01c8d-fbe1-40cc-a85a-38f7b6ab093b\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>\n",
       "            <script type=\"text/javascript\">\n",
       "                \n",
       "                    window.PLOTLYENV=window.PLOTLYENV || {};\n",
       "                    \n",
       "                if (document.getElementById(\"6fc01c8d-fbe1-40cc-a85a-38f7b6ab093b\")) {\n",
       "                    Plotly.newPlot(\n",
       "                        '6fc01c8d-fbe1-40cc-a85a-38f7b6ab093b',\n",
       "                        [{\"mode\": \"markers\", \"name\": \"Objective Value\", \"type\": \"scatter\", \"x\": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31], \"y\": [0.9564, 0.9589, 0.9751, 0.3631, 0.941, 0.9591, 0.9366, 0.9738, 0.9661, 0.9745, 0.9801, 0.9814, 0.9801, 0.9805, 0.9779, 0.9768, 0.9795, 0.9815, 0.9811, 0.9789, 0.9816, 0.9806, 0.9798, 0.9806, 0.9754, 0.098, 0.9816, 0.98, 0.9761, 0.9757, 0.9814, 0.9752]}, {\"name\": \"Best Value\", \"type\": \"scatter\", \"x\": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31], \"y\": [0.9564, 0.9589, 0.9751, 0.9751, 0.9751, 0.9751, 0.9751, 0.9751, 0.9751, 0.9751, 0.9801, 0.9814, 0.9814, 0.9814, 0.9814, 0.9814, 0.9814, 0.9815, 0.9815, 0.9815, 0.9816, 0.9816, 0.9816, 0.9816, 0.9816, 0.9816, 0.9816, 0.9816, 0.9816, 0.9816, 0.9816, 0.9816]}],\n",
       "                        {\"template\": {\"data\": {\"bar\": [{\"error_x\": {\"color\": \"#2a3f5f\"}, \"error_y\": {\"color\": \"#2a3f5f\"}, \"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"bar\"}], \"barpolar\": [{\"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"barpolar\"}], \"carpet\": [{\"aaxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"baxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"type\": \"carpet\"}], \"choropleth\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"choropleth\"}], \"contour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"contour\"}], \"contourcarpet\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"contourcarpet\"}], \"heatmap\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmap\"}], \"heatmapgl\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmapgl\"}], \"histogram\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"histogram\"}], \"histogram2d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2d\"}], \"histogram2dcontour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2dcontour\"}], \"mesh3d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"mesh3d\"}], \"parcoords\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"parcoords\"}], \"pie\": [{\"automargin\": true, \"type\": \"pie\"}], \"scatter\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter\"}], \"scatter3d\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter3d\"}], \"scattercarpet\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattercarpet\"}], \"scattergeo\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergeo\"}], \"scattergl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergl\"}], \"scattermapbox\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattermapbox\"}], \"scatterpolar\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolar\"}], \"scatterpolargl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolargl\"}], \"scatterternary\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterternary\"}], \"surface\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"surface\"}], \"table\": [{\"cells\": {\"fill\": {\"color\": \"#EBF0F8\"}, \"line\": {\"color\": \"white\"}}, \"header\": {\"fill\": {\"color\": \"#C8D4E3\"}, \"line\": {\"color\": \"white\"}}, \"type\": \"table\"}]}, \"layout\": {\"annotationdefaults\": {\"arrowcolor\": \"#2a3f5f\", \"arrowhead\": 0, \"arrowwidth\": 1}, \"coloraxis\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"colorscale\": {\"diverging\": [[0, \"#8e0152\"], [0.1, \"#c51b7d\"], [0.2, \"#de77ae\"], [0.3, \"#f1b6da\"], [0.4, \"#fde0ef\"], [0.5, \"#f7f7f7\"], [0.6, \"#e6f5d0\"], [0.7, \"#b8e186\"], [0.8, \"#7fbc41\"], [0.9, \"#4d9221\"], [1, \"#276419\"]], \"sequential\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"sequentialminus\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]]}, \"colorway\": [\"#636efa\", \"#EF553B\", \"#00cc96\", \"#ab63fa\", \"#FFA15A\", \"#19d3f3\", \"#FF6692\", \"#B6E880\", \"#FF97FF\", \"#FECB52\"], \"font\": {\"color\": \"#2a3f5f\"}, \"geo\": {\"bgcolor\": \"white\", \"lakecolor\": \"white\", \"landcolor\": \"#E5ECF6\", \"showlakes\": true, \"showland\": true, \"subunitcolor\": \"white\"}, \"hoverlabel\": {\"align\": \"left\"}, \"hovermode\": \"closest\", \"mapbox\": {\"style\": \"light\"}, \"paper_bgcolor\": \"white\", \"plot_bgcolor\": \"#E5ECF6\", \"polar\": {\"angularaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"radialaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"scene\": {\"xaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"yaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"zaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}}, \"shapedefaults\": {\"line\": {\"color\": \"#2a3f5f\"}}, \"ternary\": {\"aaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"baxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"caxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"title\": {\"x\": 0.05}, \"xaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"title\": {\"standoff\": 15}, \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}, \"yaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"title\": {\"standoff\": 15}, \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}}}, \"title\": {\"text\": \"Optimization History Plot\"}, \"xaxis\": {\"title\": {\"text\": \"#Trials\"}}, \"yaxis\": {\"title\": {\"text\": \"Objective Value\"}}},\n",
       "                        {\"responsive\": true}\n",
       "                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('6fc01c8d-fbe1-40cc-a85a-38f7b6ab093b');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })\n",
       "                };\n",
       "                \n",
       "            </script>\n",
       "        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 1.34 s (started: 2021-10-20 06:27:46 +00:00)\n"
     ]
    }
   ],
   "source": [
    "optuna.visualization.plot_optimization_history(study_1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "v7-XpIg_DDHi"
   },
   "source": [
    "### Estudo 1\n",
    "Houve alguns outliers no testes 3, mas no geral, todos os outros testes atingiram o melhor valor possível."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 558
    },
    "id": "7yLG5SWKYoiM",
    "outputId": "e4b64ffd-8c97-49b7-fa67-86dfe5b88b37"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>\n",
       "            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax) {MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>\n",
       "                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-latest.min.js\"></script>    \n",
       "            <div id=\"52a2045b-4f97-43c2-b7fb-4bf0656c17c8\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>\n",
       "            <script type=\"text/javascript\">\n",
       "                \n",
       "                    window.PLOTLYENV=window.PLOTLYENV || {};\n",
       "                    \n",
       "                if (document.getElementById(\"52a2045b-4f97-43c2-b7fb-4bf0656c17c8\")) {\n",
       "                    Plotly.newPlot(\n",
       "                        '52a2045b-4f97-43c2-b7fb-4bf0656c17c8',\n",
       "                        [{\"mode\": \"markers\", \"name\": \"Objective Value\", \"type\": \"scatter\", \"x\": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31], \"y\": [0.9702, 0.1135, 0.9728, 0.9817, 0.9708, 0.9455, 0.9773, 0.9768, 0.9713, 0.1135, 0.9796, 0.9758, 0.9809, 0.9801, 0.979, 0.9774, 0.9805, 0.9771, 0.9797, 0.9818, 0.9784, 0.9802, 0.982, 0.9822, 0.9789, 0.9803, 0.9692, 0.9753, 0.9811, 0.9767, 0.9805, 0.9814]}, {\"name\": \"Best Value\", \"type\": \"scatter\", \"x\": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31], \"y\": [0.9702, 0.9702, 0.9728, 0.9817, 0.9817, 0.9817, 0.9817, 0.9817, 0.9817, 0.9817, 0.9817, 0.9817, 0.9817, 0.9817, 0.9817, 0.9817, 0.9817, 0.9817, 0.9817, 0.9818, 0.9818, 0.9818, 0.982, 0.9822, 0.9822, 0.9822, 0.9822, 0.9822, 0.9822, 0.9822, 0.9822, 0.9822]}],\n",
       "                        {\"template\": {\"data\": {\"bar\": [{\"error_x\": {\"color\": \"#2a3f5f\"}, \"error_y\": {\"color\": \"#2a3f5f\"}, \"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"bar\"}], \"barpolar\": [{\"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"barpolar\"}], \"carpet\": [{\"aaxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"baxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"type\": \"carpet\"}], \"choropleth\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"choropleth\"}], \"contour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"contour\"}], \"contourcarpet\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"contourcarpet\"}], \"heatmap\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmap\"}], \"heatmapgl\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmapgl\"}], \"histogram\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"histogram\"}], \"histogram2d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2d\"}], \"histogram2dcontour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2dcontour\"}], \"mesh3d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"mesh3d\"}], \"parcoords\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"parcoords\"}], \"pie\": [{\"automargin\": true, \"type\": \"pie\"}], \"scatter\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter\"}], \"scatter3d\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter3d\"}], \"scattercarpet\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattercarpet\"}], \"scattergeo\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergeo\"}], \"scattergl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergl\"}], \"scattermapbox\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattermapbox\"}], \"scatterpolar\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolar\"}], \"scatterpolargl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolargl\"}], \"scatterternary\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterternary\"}], \"surface\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"surface\"}], \"table\": [{\"cells\": {\"fill\": {\"color\": \"#EBF0F8\"}, \"line\": {\"color\": \"white\"}}, \"header\": {\"fill\": {\"color\": \"#C8D4E3\"}, \"line\": {\"color\": \"white\"}}, \"type\": \"table\"}]}, \"layout\": {\"annotationdefaults\": {\"arrowcolor\": \"#2a3f5f\", \"arrowhead\": 0, \"arrowwidth\": 1}, \"coloraxis\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"colorscale\": {\"diverging\": [[0, \"#8e0152\"], [0.1, \"#c51b7d\"], [0.2, \"#de77ae\"], [0.3, \"#f1b6da\"], [0.4, \"#fde0ef\"], [0.5, \"#f7f7f7\"], [0.6, \"#e6f5d0\"], [0.7, \"#b8e186\"], [0.8, \"#7fbc41\"], [0.9, \"#4d9221\"], [1, \"#276419\"]], \"sequential\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"sequentialminus\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]]}, \"colorway\": [\"#636efa\", \"#EF553B\", \"#00cc96\", \"#ab63fa\", \"#FFA15A\", \"#19d3f3\", \"#FF6692\", \"#B6E880\", \"#FF97FF\", \"#FECB52\"], \"font\": {\"color\": \"#2a3f5f\"}, \"geo\": {\"bgcolor\": \"white\", \"lakecolor\": \"white\", \"landcolor\": \"#E5ECF6\", \"showlakes\": true, \"showland\": true, \"subunitcolor\": \"white\"}, \"hoverlabel\": {\"align\": \"left\"}, \"hovermode\": \"closest\", \"mapbox\": {\"style\": \"light\"}, \"paper_bgcolor\": \"white\", \"plot_bgcolor\": \"#E5ECF6\", \"polar\": {\"angularaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"radialaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"scene\": {\"xaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"yaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"zaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}}, \"shapedefaults\": {\"line\": {\"color\": \"#2a3f5f\"}}, \"ternary\": {\"aaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"baxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"caxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"title\": {\"x\": 0.05}, \"xaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"title\": {\"standoff\": 15}, \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}, \"yaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"title\": {\"standoff\": 15}, \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}}}, \"title\": {\"text\": \"Optimization History Plot\"}, \"xaxis\": {\"title\": {\"text\": \"#Trials\"}}, \"yaxis\": {\"title\": {\"text\": \"Objective Value\"}}},\n",
       "                        {\"responsive\": true}\n",
       "                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('52a2045b-4f97-43c2-b7fb-4bf0656c17c8');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })\n",
       "                };\n",
       "                \n",
       "            </script>\n",
       "        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 176 ms (started: 2021-10-20 06:28:04 +00:00)\n"
     ]
    }
   ],
   "source": [
    "optuna.visualization.plot_optimization_history(study_2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6hbdaU7oDeHH"
   },
   "source": [
    "### Estudo 2\n",
    "Houveram apenas 2 outliers distoantes, todos os outros valores foram consistentes em atingir o melhor valor possível."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 558
    },
    "id": "jKFJt_hZYyla",
    "outputId": "8c31311c-b037-45ae-f903-c24eb1310b4b"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>\n",
       "            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax) {MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>\n",
       "                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-latest.min.js\"></script>    \n",
       "            <div id=\"269eb7d1-96b8-4e96-b146-0743d353a35f\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>\n",
       "            <script type=\"text/javascript\">\n",
       "                \n",
       "                    window.PLOTLYENV=window.PLOTLYENV || {};\n",
       "                    \n",
       "                if (document.getElementById(\"269eb7d1-96b8-4e96-b146-0743d353a35f\")) {\n",
       "                    Plotly.newPlot(\n",
       "                        '269eb7d1-96b8-4e96-b146-0743d353a35f',\n",
       "                        [{\"mode\": \"markers\", \"name\": \"Objective Value\", \"type\": \"scatter\", \"x\": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31], \"y\": [0.9639, 0.9806, 0.9774, 0.9643, 0.8731, 0.098, 0.9798, 0.9708, 0.975, 0.9687, 0.1135, 0.9816, 0.9816, 0.9816, 0.9808, 0.9824, 0.9782, 0.9828, 0.9785, 0.9781, 0.098, 0.98, 0.9812, 0.9801, 0.9828, 0.9756, 0.9788, 0.9776, 0.1135, 0.9816, 0.9804, 0.9819]}, {\"name\": \"Best Value\", \"type\": \"scatter\", \"x\": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31], \"y\": [0.9639, 0.9806, 0.9806, 0.9806, 0.9806, 0.9806, 0.9806, 0.9806, 0.9806, 0.9806, 0.9806, 0.9816, 0.9816, 0.9816, 0.9816, 0.9824, 0.9824, 0.9828, 0.9828, 0.9828, 0.9828, 0.9828, 0.9828, 0.9828, 0.9828, 0.9828, 0.9828, 0.9828, 0.9828, 0.9828, 0.9828, 0.9828]}],\n",
       "                        {\"template\": {\"data\": {\"bar\": [{\"error_x\": {\"color\": \"#2a3f5f\"}, \"error_y\": {\"color\": \"#2a3f5f\"}, \"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"bar\"}], \"barpolar\": [{\"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"barpolar\"}], \"carpet\": [{\"aaxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"baxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"type\": \"carpet\"}], \"choropleth\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"choropleth\"}], \"contour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"contour\"}], \"contourcarpet\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"contourcarpet\"}], \"heatmap\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmap\"}], \"heatmapgl\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmapgl\"}], \"histogram\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"histogram\"}], \"histogram2d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2d\"}], \"histogram2dcontour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2dcontour\"}], \"mesh3d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"mesh3d\"}], \"parcoords\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"parcoords\"}], \"pie\": [{\"automargin\": true, \"type\": \"pie\"}], \"scatter\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter\"}], \"scatter3d\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter3d\"}], \"scattercarpet\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattercarpet\"}], \"scattergeo\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergeo\"}], \"scattergl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergl\"}], \"scattermapbox\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattermapbox\"}], \"scatterpolar\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolar\"}], \"scatterpolargl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolargl\"}], \"scatterternary\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterternary\"}], \"surface\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"surface\"}], \"table\": [{\"cells\": {\"fill\": {\"color\": \"#EBF0F8\"}, \"line\": {\"color\": \"white\"}}, \"header\": {\"fill\": {\"color\": \"#C8D4E3\"}, \"line\": {\"color\": \"white\"}}, \"type\": \"table\"}]}, \"layout\": {\"annotationdefaults\": {\"arrowcolor\": \"#2a3f5f\", \"arrowhead\": 0, \"arrowwidth\": 1}, \"coloraxis\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"colorscale\": {\"diverging\": [[0, \"#8e0152\"], [0.1, \"#c51b7d\"], [0.2, \"#de77ae\"], [0.3, \"#f1b6da\"], [0.4, \"#fde0ef\"], [0.5, \"#f7f7f7\"], [0.6, \"#e6f5d0\"], [0.7, \"#b8e186\"], [0.8, \"#7fbc41\"], [0.9, \"#4d9221\"], [1, \"#276419\"]], \"sequential\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"sequentialminus\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]]}, \"colorway\": [\"#636efa\", \"#EF553B\", \"#00cc96\", \"#ab63fa\", \"#FFA15A\", \"#19d3f3\", \"#FF6692\", \"#B6E880\", \"#FF97FF\", \"#FECB52\"], \"font\": {\"color\": \"#2a3f5f\"}, \"geo\": {\"bgcolor\": \"white\", \"lakecolor\": \"white\", \"landcolor\": \"#E5ECF6\", \"showlakes\": true, \"showland\": true, \"subunitcolor\": \"white\"}, \"hoverlabel\": {\"align\": \"left\"}, \"hovermode\": \"closest\", \"mapbox\": {\"style\": \"light\"}, \"paper_bgcolor\": \"white\", \"plot_bgcolor\": \"#E5ECF6\", \"polar\": {\"angularaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"radialaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"scene\": {\"xaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"yaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"zaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}}, \"shapedefaults\": {\"line\": {\"color\": \"#2a3f5f\"}}, \"ternary\": {\"aaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"baxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"caxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"title\": {\"x\": 0.05}, \"xaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"title\": {\"standoff\": 15}, \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}, \"yaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"title\": {\"standoff\": 15}, \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}}}, \"title\": {\"text\": \"Optimization History Plot\"}, \"xaxis\": {\"title\": {\"text\": \"#Trials\"}}, \"yaxis\": {\"title\": {\"text\": \"Objective Value\"}}},\n",
       "                        {\"responsive\": true}\n",
       "                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('269eb7d1-96b8-4e96-b146-0743d353a35f');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })\n",
       "                };\n",
       "                \n",
       "            </script>\n",
       "        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 187 ms (started: 2021-10-20 06:28:43 +00:00)\n"
     ]
    }
   ],
   "source": [
    "optuna.visualization.plot_optimization_history(study_4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "QWG-2prWDqGc"
   },
   "source": [
    "### Estudo 3\n",
    "Este estudo teve 5 pontos como outliers, sendo este o pior de nossos cenários."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 558
    },
    "id": "SRGjSAHoY023",
    "outputId": "a0e8956a-4f4b-4223-ea5d-574bc1838ab1"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>\n",
       "            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax) {MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>\n",
       "                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-latest.min.js\"></script>    \n",
       "            <div id=\"e60281fa-6736-4115-b4c0-9602855d051c\" class=\"plotly-graph-div\" style=\"height:525px; width:1200px;\"></div>\n",
       "            <script type=\"text/javascript\">\n",
       "                \n",
       "                    window.PLOTLYENV=window.PLOTLYENV || {};\n",
       "                    \n",
       "                if (document.getElementById(\"e60281fa-6736-4115-b4c0-9602855d051c\")) {\n",
       "                    Plotly.newPlot(\n",
       "                        'e60281fa-6736-4115-b4c0-9602855d051c',\n",
       "                        [{\"marker\": {\"color\": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31], \"colorbar\": {\"title\": {\"text\": \"#Trials\"}, \"x\": 1.0, \"xpad\": 40}, \"colorscale\": [[0.0, \"rgb(247,251,255)\"], [0.125, \"rgb(222,235,247)\"], [0.25, \"rgb(198,219,239)\"], [0.375, \"rgb(158,202,225)\"], [0.5, \"rgb(107,174,214)\"], [0.625, \"rgb(66,146,198)\"], [0.75, \"rgb(33,113,181)\"], [0.875, \"rgb(8,81,156)\"], [1.0, \"rgb(8,48,107)\"]], \"line\": {\"color\": \"Grey\", \"width\": 0.5}, \"showscale\": true}, \"mode\": \"markers\", \"showlegend\": false, \"type\": \"scatter\", \"x\": [\"selu\", \"softplus\", \"relu\", \"softmax\", \"softmax\", \"relu\", \"softmax\", \"exponential\", \"selu\", \"relu\", \"softsign\", \"softsign\", \"softsign\", \"softsign\", \"softsign\", \"tanh\", \"elu\", \"sigmoid\", \"sigmoid\", \"sigmoid\", \"sigmoid\", \"sigmoid\", \"sigmoid\", \"sigmoid\", \"elu\", \"exponential\", \"softplus\", \"softplus\", \"softplus\", \"selu\", \"tanh\", \"tanh\"], \"xaxis\": \"x\", \"y\": [0.9564, 0.9589, 0.9751, 0.3631, 0.941, 0.9591, 0.9366, 0.9738, 0.9661, 0.9745, 0.9801, 0.9814, 0.9801, 0.9805, 0.9779, 0.9768, 0.9795, 0.9815, 0.9811, 0.9789, 0.9816, 0.9806, 0.9798, 0.9806, 0.9754, 0.098, 0.9816, 0.98, 0.9761, 0.9757, 0.9814, 0.9752], \"yaxis\": \"y\"}, {\"marker\": {\"color\": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31], \"colorbar\": {\"title\": {\"text\": \"#Trials\"}, \"x\": 1.0, \"xpad\": 40}, \"colorscale\": [[0.0, \"rgb(247,251,255)\"], [0.125, \"rgb(222,235,247)\"], [0.25, \"rgb(198,219,239)\"], [0.375, \"rgb(158,202,225)\"], [0.5, \"rgb(107,174,214)\"], [0.625, \"rgb(66,146,198)\"], [0.75, \"rgb(33,113,181)\"], [0.875, \"rgb(8,81,156)\"], [1.0, \"rgb(8,48,107)\"]], \"line\": {\"color\": \"Grey\", \"width\": 0.5}, \"showscale\": false}, \"mode\": \"markers\", \"showlegend\": false, \"type\": \"scatter\", \"x\": [\"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\"], \"xaxis\": \"x2\", \"y\": [0.9564, 0.9589, 0.9751, 0.3631, 0.941, 0.9591, 0.9366, 0.9738, 0.9661, 0.9745, 0.9801, 0.9814, 0.9801, 0.9805, 0.9779, 0.9768, 0.9795, 0.9815, 0.9811, 0.9789, 0.9816, 0.9806, 0.9798, 0.9806, 0.9754, 0.098, 0.9816, 0.98, 0.9761, 0.9757, 0.9814, 0.9752], \"yaxis\": \"y2\"}, {\"marker\": {\"color\": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31], \"colorbar\": {\"title\": {\"text\": \"#Trials\"}, \"x\": 1.0, \"xpad\": 40}, \"colorscale\": [[0.0, \"rgb(247,251,255)\"], [0.125, \"rgb(222,235,247)\"], [0.25, \"rgb(198,219,239)\"], [0.375, \"rgb(158,202,225)\"], [0.5, \"rgb(107,174,214)\"], [0.625, \"rgb(66,146,198)\"], [0.75, \"rgb(33,113,181)\"], [0.875, \"rgb(8,81,156)\"], [1.0, \"rgb(8,48,107)\"]], \"line\": {\"color\": \"Grey\", \"width\": 0.5}, \"showscale\": false}, \"mode\": \"markers\", \"showlegend\": false, \"type\": \"scatter\", \"x\": [22, 18, 215, 111, 192, 31, 130, 214, 130, 125, 250, 256, 245, 251, 173, 80, 172, 255, 222, 164, 228, 227, 194, 235, 255, 201, 155, 156, 59, 90, 185, 151], \"xaxis\": \"x3\", \"y\": [0.9564, 0.9589, 0.9751, 0.3631, 0.941, 0.9591, 0.9366, 0.9738, 0.9661, 0.9745, 0.9801, 0.9814, 0.9801, 0.9805, 0.9779, 0.9768, 0.9795, 0.9815, 0.9811, 0.9789, 0.9816, 0.9806, 0.9798, 0.9806, 0.9754, 0.098, 0.9816, 0.98, 0.9761, 0.9757, 0.9814, 0.9752], \"yaxis\": \"y3\"}, {\"marker\": {\"color\": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31], \"colorbar\": {\"title\": {\"text\": \"#Trials\"}, \"x\": 1.0, \"xpad\": 40}, \"colorscale\": [[0.0, \"rgb(247,251,255)\"], [0.125, \"rgb(222,235,247)\"], [0.25, \"rgb(198,219,239)\"], [0.375, \"rgb(158,202,225)\"], [0.5, \"rgb(107,174,214)\"], [0.625, \"rgb(66,146,198)\"], [0.75, \"rgb(33,113,181)\"], [0.875, \"rgb(8,81,156)\"], [1.0, \"rgb(8,48,107)\"]], \"line\": {\"color\": \"Grey\", \"width\": 0.5}, \"showscale\": false}, \"mode\": \"markers\", \"showlegend\": false, \"type\": \"scatter\", \"x\": [\"sgd\", \"adam\", \"sgd\", \"sgd\", \"rmsprop\", \"sgd\", \"rmsprop\", \"sgd\", \"sgd\", \"rmsprop\", \"adamax\", \"adamax\", \"adamax\", \"adamax\", \"adamax\", \"adamax\", \"adamax\", \"adam\", \"adam\", \"adam\", \"adam\", \"adam\", \"adam\", \"adam\", \"adam\", \"adam\", \"adam\", \"adam\", \"adam\", \"adam\", \"adam\", \"adam\"], \"xaxis\": \"x4\", \"y\": [0.9564, 0.9589, 0.9751, 0.3631, 0.941, 0.9591, 0.9366, 0.9738, 0.9661, 0.9745, 0.9801, 0.9814, 0.9801, 0.9805, 0.9779, 0.9768, 0.9795, 0.9815, 0.9811, 0.9789, 0.9816, 0.9806, 0.9798, 0.9806, 0.9754, 0.098, 0.9816, 0.98, 0.9761, 0.9757, 0.9814, 0.9752], \"yaxis\": \"y4\"}],\n",
       "                        {\"template\": {\"data\": {\"bar\": [{\"error_x\": {\"color\": \"#2a3f5f\"}, \"error_y\": {\"color\": \"#2a3f5f\"}, \"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"bar\"}], \"barpolar\": [{\"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"barpolar\"}], \"carpet\": [{\"aaxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"baxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"type\": \"carpet\"}], \"choropleth\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"choropleth\"}], \"contour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"contour\"}], \"contourcarpet\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"contourcarpet\"}], \"heatmap\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmap\"}], \"heatmapgl\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmapgl\"}], \"histogram\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"histogram\"}], \"histogram2d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2d\"}], \"histogram2dcontour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2dcontour\"}], \"mesh3d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"mesh3d\"}], \"parcoords\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"parcoords\"}], \"pie\": [{\"automargin\": true, \"type\": \"pie\"}], \"scatter\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter\"}], \"scatter3d\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter3d\"}], \"scattercarpet\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattercarpet\"}], \"scattergeo\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergeo\"}], \"scattergl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergl\"}], \"scattermapbox\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattermapbox\"}], \"scatterpolar\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolar\"}], \"scatterpolargl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolargl\"}], \"scatterternary\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterternary\"}], \"surface\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"surface\"}], \"table\": [{\"cells\": {\"fill\": {\"color\": \"#EBF0F8\"}, \"line\": {\"color\": \"white\"}}, \"header\": {\"fill\": {\"color\": \"#C8D4E3\"}, \"line\": {\"color\": \"white\"}}, \"type\": \"table\"}]}, \"layout\": {\"annotationdefaults\": {\"arrowcolor\": \"#2a3f5f\", \"arrowhead\": 0, \"arrowwidth\": 1}, \"coloraxis\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"colorscale\": {\"diverging\": [[0, \"#8e0152\"], [0.1, \"#c51b7d\"], [0.2, \"#de77ae\"], [0.3, \"#f1b6da\"], [0.4, \"#fde0ef\"], [0.5, \"#f7f7f7\"], [0.6, \"#e6f5d0\"], [0.7, \"#b8e186\"], [0.8, \"#7fbc41\"], [0.9, \"#4d9221\"], [1, \"#276419\"]], \"sequential\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"sequentialminus\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]]}, \"colorway\": [\"#636efa\", \"#EF553B\", \"#00cc96\", \"#ab63fa\", \"#FFA15A\", \"#19d3f3\", \"#FF6692\", \"#B6E880\", \"#FF97FF\", \"#FECB52\"], \"font\": {\"color\": \"#2a3f5f\"}, \"geo\": {\"bgcolor\": \"white\", \"lakecolor\": \"white\", \"landcolor\": \"#E5ECF6\", \"showlakes\": true, \"showland\": true, \"subunitcolor\": \"white\"}, \"hoverlabel\": {\"align\": \"left\"}, \"hovermode\": \"closest\", \"mapbox\": {\"style\": \"light\"}, \"paper_bgcolor\": \"white\", \"plot_bgcolor\": \"#E5ECF6\", \"polar\": {\"angularaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"radialaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"scene\": {\"xaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"yaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"zaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}}, \"shapedefaults\": {\"line\": {\"color\": \"#2a3f5f\"}}, \"ternary\": {\"aaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"baxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"caxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"title\": {\"x\": 0.05}, \"xaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"title\": {\"standoff\": 15}, \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}, \"yaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"title\": {\"standoff\": 15}, \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}}}, \"title\": {\"text\": \"Slice Plot\"}, \"width\": 1200, \"xaxis\": {\"anchor\": \"y\", \"domain\": [0.0, 0.2125], \"title\": {\"text\": \"activation\"}}, \"xaxis2\": {\"anchor\": \"y2\", \"domain\": [0.2625, 0.475], \"title\": {\"text\": \"loss\"}}, \"xaxis3\": {\"anchor\": \"y3\", \"domain\": [0.525, 0.7375], \"title\": {\"text\": \"n_neurons\"}}, \"xaxis4\": {\"anchor\": \"y4\", \"domain\": [0.7875, 1.0], \"title\": {\"text\": \"optimizer\"}}, \"yaxis\": {\"anchor\": \"x\", \"domain\": [0.0, 1.0], \"title\": {\"text\": \"Objective Value\"}}, \"yaxis2\": {\"anchor\": \"x2\", \"domain\": [0.0, 1.0], \"matches\": \"y\", \"showticklabels\": false}, \"yaxis3\": {\"anchor\": \"x3\", \"domain\": [0.0, 1.0], \"matches\": \"y\", \"showticklabels\": false}, \"yaxis4\": {\"anchor\": \"x4\", \"domain\": [0.0, 1.0], \"matches\": \"y\", \"showticklabels\": false}},\n",
       "                        {\"responsive\": true}\n",
       "                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('e60281fa-6736-4115-b4c0-9602855d051c');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })\n",
       "                };\n",
       "                \n",
       "            </script>\n",
       "        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 525 ms (started: 2021-10-20 06:29:11 +00:00)\n"
     ]
    }
   ],
   "source": [
    "optuna.visualization.plot_slice(study_1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "hTwScVXfEOMG"
   },
   "source": [
    "### Avaliação dos parâmetros do estudo 1\n",
    "* Para as funções de ativação, todas elas demonstraram estar no mesmo espectro, houve um outlier na exponencial e na softmax. <br>\n",
    "  * Softsign, tan, elu e sigmoid bastaram ser bastante consistentes em suas pontuações. \n",
    "  * Selu, softplus, relu e softmax foram mais dispersas.\n",
    "* Para quantidade de neurônios houveram apenas 2 outliers. Os valores mais a esquerda (com menos neurônios) se provaram um pouco menos eficientes que os mais a direita (com mais neurônios).\n",
    "* Quanto ao otimizador, houveram outliers na SGD e no Adam, sendo o Adamax a mais consistente em apresentar bons resultados."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 558
    },
    "id": "0AfQ3bUcY7pg",
    "outputId": "c540227c-4d29-4aed-e510-ee1814fd7117"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>\n",
       "            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax) {MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>\n",
       "                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-latest.min.js\"></script>    \n",
       "            <div id=\"6aab2e57-083f-43d6-800d-a0b4de49be59\" class=\"plotly-graph-div\" style=\"height:525px; width:1200px;\"></div>\n",
       "            <script type=\"text/javascript\">\n",
       "                \n",
       "                    window.PLOTLYENV=window.PLOTLYENV || {};\n",
       "                    \n",
       "                if (document.getElementById(\"6aab2e57-083f-43d6-800d-a0b4de49be59\")) {\n",
       "                    Plotly.newPlot(\n",
       "                        '6aab2e57-083f-43d6-800d-a0b4de49be59',\n",
       "                        [{\"marker\": {\"color\": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31], \"colorbar\": {\"title\": {\"text\": \"#Trials\"}, \"x\": 1.0, \"xpad\": 40}, \"colorscale\": [[0.0, \"rgb(247,251,255)\"], [0.125, \"rgb(222,235,247)\"], [0.25, \"rgb(198,219,239)\"], [0.375, \"rgb(158,202,225)\"], [0.5, \"rgb(107,174,214)\"], [0.625, \"rgb(66,146,198)\"], [0.75, \"rgb(33,113,181)\"], [0.875, \"rgb(8,81,156)\"], [1.0, \"rgb(8,48,107)\"]], \"line\": {\"color\": \"Grey\", \"width\": 0.5}, \"showscale\": true}, \"mode\": \"markers\", \"showlegend\": false, \"type\": \"scatter\", \"x\": [\"tanh\", \"softmax\", \"tanh\", \"relu\", \"relu\", \"softmax\", \"exponential\", \"tanh\", \"selu\", \"softmax\", \"sigmoid\", \"sigmoid\", \"softplus\", \"softplus\", \"softsign\", \"softplus\", \"relu\", \"elu\", \"softplus\", \"relu\", \"relu\", \"relu\", \"relu\", \"relu\", \"relu\", \"relu\", \"softsign\", \"exponential\", \"elu\", \"relu\", \"selu\", \"relu\"], \"xaxis\": \"x\", \"y\": [0.9702, 0.1135, 0.9728, 0.9817, 0.9708, 0.9455, 0.9773, 0.9768, 0.9713, 0.1135, 0.9796, 0.9758, 0.9809, 0.9801, 0.979, 0.9774, 0.9805, 0.9771, 0.9797, 0.9818, 0.9784, 0.9802, 0.982, 0.9822, 0.9789, 0.9803, 0.9692, 0.9753, 0.9811, 0.9767, 0.9805, 0.9814], \"yaxis\": \"y\"}, {\"marker\": {\"color\": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31], \"colorbar\": {\"title\": {\"text\": \"#Trials\"}, \"x\": 1.0, \"xpad\": 40}, \"colorscale\": [[0.0, \"rgb(247,251,255)\"], [0.125, \"rgb(222,235,247)\"], [0.25, \"rgb(198,219,239)\"], [0.375, \"rgb(158,202,225)\"], [0.5, \"rgb(107,174,214)\"], [0.625, \"rgb(66,146,198)\"], [0.75, \"rgb(33,113,181)\"], [0.875, \"rgb(8,81,156)\"], [1.0, \"rgb(8,48,107)\"]], \"line\": {\"color\": \"Grey\", \"width\": 0.5}, \"showscale\": false}, \"mode\": \"markers\", \"showlegend\": false, \"type\": \"scatter\", \"x\": [\"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\"], \"xaxis\": \"x2\", \"y\": [0.9702, 0.1135, 0.9728, 0.9817, 0.9708, 0.9455, 0.9773, 0.9768, 0.9713, 0.1135, 0.9796, 0.9758, 0.9809, 0.9801, 0.979, 0.9774, 0.9805, 0.9771, 0.9797, 0.9818, 0.9784, 0.9802, 0.982, 0.9822, 0.9789, 0.9803, 0.9692, 0.9753, 0.9811, 0.9767, 0.9805, 0.9814], \"yaxis\": \"y2\"}, {\"marker\": {\"color\": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31], \"colorbar\": {\"title\": {\"text\": \"#Trials\"}, \"x\": 1.0, \"xpad\": 40}, \"colorscale\": [[0.0, \"rgb(247,251,255)\"], [0.125, \"rgb(222,235,247)\"], [0.25, \"rgb(198,219,239)\"], [0.375, \"rgb(158,202,225)\"], [0.5, \"rgb(107,174,214)\"], [0.625, \"rgb(66,146,198)\"], [0.75, \"rgb(33,113,181)\"], [0.875, \"rgb(8,81,156)\"], [1.0, \"rgb(8,48,107)\"]], \"line\": {\"color\": \"Grey\", \"width\": 0.5}, \"showscale\": false}, \"mode\": \"markers\", \"showlegend\": false, \"type\": \"scatter\", \"x\": [51, 191, 79, 250, 44, 220, 231, 219, 43, 48, 146, 138, 155, 174, 256, 110, 180, 92, 252, 158, 105, 159, 203, 207, 207, 198, 127, 171, 194, 229, 125, 238], \"xaxis\": \"x3\", \"y\": [0.9702, 0.1135, 0.9728, 0.9817, 0.9708, 0.9455, 0.9773, 0.9768, 0.9713, 0.1135, 0.9796, 0.9758, 0.9809, 0.9801, 0.979, 0.9774, 0.9805, 0.9771, 0.9797, 0.9818, 0.9784, 0.9802, 0.982, 0.9822, 0.9789, 0.9803, 0.9692, 0.9753, 0.9811, 0.9767, 0.9805, 0.9814], \"yaxis\": \"y3\"}, {\"marker\": {\"color\": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31], \"colorbar\": {\"title\": {\"text\": \"#Trials\"}, \"x\": 1.0, \"xpad\": 40}, \"colorscale\": [[0.0, \"rgb(247,251,255)\"], [0.125, \"rgb(222,235,247)\"], [0.25, \"rgb(198,219,239)\"], [0.375, \"rgb(158,202,225)\"], [0.5, \"rgb(107,174,214)\"], [0.625, \"rgb(66,146,198)\"], [0.75, \"rgb(33,113,181)\"], [0.875, \"rgb(8,81,156)\"], [1.0, \"rgb(8,48,107)\"]], \"line\": {\"color\": \"Grey\", \"width\": 0.5}, \"showscale\": false}, \"mode\": \"markers\", \"showlegend\": false, \"type\": \"scatter\", \"x\": [\"rmsprop\", \"sgd\", \"sgd\", \"adam\", \"rmsprop\", \"rmsprop\", \"adamax\", \"adam\", \"adamax\", \"sgd\", \"adam\", \"adam\", \"adam\", \"adam\", \"adam\", \"adam\", \"adam\", \"adam\", \"adamax\", \"adam\", \"adam\", \"adam\", \"adam\", \"adam\", \"adam\", \"adam\", \"sgd\", \"adamax\", \"rmsprop\", \"rmsprop\", \"adam\", \"adam\"], \"xaxis\": \"x4\", \"y\": [0.9702, 0.1135, 0.9728, 0.9817, 0.9708, 0.9455, 0.9773, 0.9768, 0.9713, 0.1135, 0.9796, 0.9758, 0.9809, 0.9801, 0.979, 0.9774, 0.9805, 0.9771, 0.9797, 0.9818, 0.9784, 0.9802, 0.982, 0.9822, 0.9789, 0.9803, 0.9692, 0.9753, 0.9811, 0.9767, 0.9805, 0.9814], \"yaxis\": \"y4\"}],\n",
       "                        {\"template\": {\"data\": {\"bar\": [{\"error_x\": {\"color\": \"#2a3f5f\"}, \"error_y\": {\"color\": \"#2a3f5f\"}, \"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"bar\"}], \"barpolar\": [{\"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"barpolar\"}], \"carpet\": [{\"aaxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"baxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"type\": \"carpet\"}], \"choropleth\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"choropleth\"}], \"contour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"contour\"}], \"contourcarpet\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"contourcarpet\"}], \"heatmap\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmap\"}], \"heatmapgl\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmapgl\"}], \"histogram\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"histogram\"}], \"histogram2d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2d\"}], \"histogram2dcontour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2dcontour\"}], \"mesh3d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"mesh3d\"}], \"parcoords\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"parcoords\"}], \"pie\": [{\"automargin\": true, \"type\": \"pie\"}], \"scatter\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter\"}], \"scatter3d\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter3d\"}], \"scattercarpet\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattercarpet\"}], \"scattergeo\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergeo\"}], \"scattergl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergl\"}], \"scattermapbox\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattermapbox\"}], \"scatterpolar\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolar\"}], \"scatterpolargl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolargl\"}], \"scatterternary\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterternary\"}], \"surface\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"surface\"}], \"table\": [{\"cells\": {\"fill\": {\"color\": \"#EBF0F8\"}, \"line\": {\"color\": \"white\"}}, \"header\": {\"fill\": {\"color\": \"#C8D4E3\"}, \"line\": {\"color\": \"white\"}}, \"type\": \"table\"}]}, \"layout\": {\"annotationdefaults\": {\"arrowcolor\": \"#2a3f5f\", \"arrowhead\": 0, \"arrowwidth\": 1}, \"coloraxis\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"colorscale\": {\"diverging\": [[0, \"#8e0152\"], [0.1, \"#c51b7d\"], [0.2, \"#de77ae\"], [0.3, \"#f1b6da\"], [0.4, \"#fde0ef\"], [0.5, \"#f7f7f7\"], [0.6, \"#e6f5d0\"], [0.7, \"#b8e186\"], [0.8, \"#7fbc41\"], [0.9, \"#4d9221\"], [1, \"#276419\"]], \"sequential\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"sequentialminus\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]]}, \"colorway\": [\"#636efa\", \"#EF553B\", \"#00cc96\", \"#ab63fa\", \"#FFA15A\", \"#19d3f3\", \"#FF6692\", \"#B6E880\", \"#FF97FF\", \"#FECB52\"], \"font\": {\"color\": \"#2a3f5f\"}, \"geo\": {\"bgcolor\": \"white\", \"lakecolor\": \"white\", \"landcolor\": \"#E5ECF6\", \"showlakes\": true, \"showland\": true, \"subunitcolor\": \"white\"}, \"hoverlabel\": {\"align\": \"left\"}, \"hovermode\": \"closest\", \"mapbox\": {\"style\": \"light\"}, \"paper_bgcolor\": \"white\", \"plot_bgcolor\": \"#E5ECF6\", \"polar\": {\"angularaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"radialaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"scene\": {\"xaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"yaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"zaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}}, \"shapedefaults\": {\"line\": {\"color\": \"#2a3f5f\"}}, \"ternary\": {\"aaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"baxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"caxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"title\": {\"x\": 0.05}, \"xaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"title\": {\"standoff\": 15}, \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}, \"yaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"title\": {\"standoff\": 15}, \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}}}, \"title\": {\"text\": \"Slice Plot\"}, \"width\": 1200, \"xaxis\": {\"anchor\": \"y\", \"domain\": [0.0, 0.2125], \"title\": {\"text\": \"activation\"}}, \"xaxis2\": {\"anchor\": \"y2\", \"domain\": [0.2625, 0.475], \"title\": {\"text\": \"loss\"}}, \"xaxis3\": {\"anchor\": \"y3\", \"domain\": [0.525, 0.7375], \"title\": {\"text\": \"n_neurons\"}}, \"xaxis4\": {\"anchor\": \"y4\", \"domain\": [0.7875, 1.0], \"title\": {\"text\": \"optimizer\"}}, \"yaxis\": {\"anchor\": \"x\", \"domain\": [0.0, 1.0], \"title\": {\"text\": \"Objective Value\"}}, \"yaxis2\": {\"anchor\": \"x2\", \"domain\": [0.0, 1.0], \"matches\": \"y\", \"showticklabels\": false}, \"yaxis3\": {\"anchor\": \"x3\", \"domain\": [0.0, 1.0], \"matches\": \"y\", \"showticklabels\": false}, \"yaxis4\": {\"anchor\": \"x4\", \"domain\": [0.0, 1.0], \"matches\": \"y\", \"showticklabels\": false}},\n",
       "                        {\"responsive\": true}\n",
       "                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('6aab2e57-083f-43d6-800d-a0b4de49be59');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })\n",
       "                };\n",
       "                \n",
       "            </script>\n",
       "        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 283 ms (started: 2021-10-20 06:29:19 +00:00)\n"
     ]
    }
   ],
   "source": [
    "optuna.visualization.plot_slice(study_2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "KCqj78RuG3BH"
   },
   "source": [
    "### Avaliação dos parâmetros do estudo 2\n",
    "\n",
    "* Mais uma vez, a função de ativação softmax se provou um outlier, enquanto as outras mantiveram bons resultados.\n",
    "* Houveram 2 outliers no número de neurônios, e, desta vez, não é tão perceptível a evolução com aumento do número de neurônios vista no estudo 1.\n",
    "* SGD foi o único outlier dentre os otimizadores, os outros apresentaram consistência em bons resultados."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 558
    },
    "id": "5cKOhPmLY9kf",
    "outputId": "fd08d608-389d-4bed-e4e6-2526e87e69cf"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>\n",
       "            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax) {MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>\n",
       "                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-latest.min.js\"></script>    \n",
       "            <div id=\"f92c8081-32da-4806-b97c-13d9bde11703\" class=\"plotly-graph-div\" style=\"height:525px; width:1200px;\"></div>\n",
       "            <script type=\"text/javascript\">\n",
       "                \n",
       "                    window.PLOTLYENV=window.PLOTLYENV || {};\n",
       "                    \n",
       "                if (document.getElementById(\"f92c8081-32da-4806-b97c-13d9bde11703\")) {\n",
       "                    Plotly.newPlot(\n",
       "                        'f92c8081-32da-4806-b97c-13d9bde11703',\n",
       "                        [{\"marker\": {\"color\": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31], \"colorbar\": {\"title\": {\"text\": \"#Trials\"}, \"x\": 1.0, \"xpad\": 40}, \"colorscale\": [[0.0, \"rgb(247,251,255)\"], [0.125, \"rgb(222,235,247)\"], [0.25, \"rgb(198,219,239)\"], [0.375, \"rgb(158,202,225)\"], [0.5, \"rgb(107,174,214)\"], [0.625, \"rgb(66,146,198)\"], [0.75, \"rgb(33,113,181)\"], [0.875, \"rgb(8,81,156)\"], [1.0, \"rgb(8,48,107)\"]], \"line\": {\"color\": \"Grey\", \"width\": 0.5}, \"showscale\": true}, \"mode\": \"markers\", \"showlegend\": false, \"type\": \"scatter\", \"x\": [\"softplus\", \"tanh\", \"tanh\", \"softsign\", \"sigmoid\", \"exponential\", \"elu\", \"softplus\", \"relu\", \"elu\", \"softmax\", \"elu\", \"tanh\", \"selu\", \"elu\", \"tanh\", \"sigmoid\", \"selu\", \"selu\", \"selu\", \"exponential\", \"selu\", \"selu\", \"softsign\", \"relu\", \"relu\", \"relu\", \"tanh\", \"softmax\", \"softplus\", \"relu\", \"softplus\"], \"xaxis\": \"x\", \"y\": [0.9639, 0.9806, 0.9774, 0.9643, 0.8731, 0.098, 0.9798, 0.9708, 0.975, 0.9687, 0.1135, 0.9816, 0.9816, 0.9816, 0.9808, 0.9824, 0.9782, 0.9828, 0.9785, 0.9781, 0.098, 0.98, 0.9812, 0.9801, 0.9828, 0.9756, 0.9788, 0.9776, 0.1135, 0.9816, 0.9804, 0.9819], \"yaxis\": \"y\"}, {\"marker\": {\"color\": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31], \"colorbar\": {\"title\": {\"text\": \"#Trials\"}, \"x\": 1.0, \"xpad\": 40}, \"colorscale\": [[0.0, \"rgb(247,251,255)\"], [0.125, \"rgb(222,235,247)\"], [0.25, \"rgb(198,219,239)\"], [0.375, \"rgb(158,202,225)\"], [0.5, \"rgb(107,174,214)\"], [0.625, \"rgb(66,146,198)\"], [0.75, \"rgb(33,113,181)\"], [0.875, \"rgb(8,81,156)\"], [1.0, \"rgb(8,48,107)\"]], \"line\": {\"color\": \"Grey\", \"width\": 0.5}, \"showscale\": false}, \"mode\": \"markers\", \"showlegend\": false, \"type\": \"scatter\", \"x\": [\"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\", \"sparse_categorical_crossentropy\"], \"xaxis\": \"x2\", \"y\": [0.9639, 0.9806, 0.9774, 0.9643, 0.8731, 0.098, 0.9798, 0.9708, 0.975, 0.9687, 0.1135, 0.9816, 0.9816, 0.9816, 0.9808, 0.9824, 0.9782, 0.9828, 0.9785, 0.9781, 0.098, 0.98, 0.9812, 0.9801, 0.9828, 0.9756, 0.9788, 0.9776, 0.1135, 0.9816, 0.9804, 0.9819], \"yaxis\": \"y2\"}, {\"marker\": {\"color\": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31], \"colorbar\": {\"title\": {\"text\": \"#Trials\"}, \"x\": 1.0, \"xpad\": 40}, \"colorscale\": [[0.0, \"rgb(247,251,255)\"], [0.125, \"rgb(222,235,247)\"], [0.25, \"rgb(198,219,239)\"], [0.375, \"rgb(158,202,225)\"], [0.5, \"rgb(107,174,214)\"], [0.625, \"rgb(66,146,198)\"], [0.75, \"rgb(33,113,181)\"], [0.875, \"rgb(8,81,156)\"], [1.0, \"rgb(8,48,107)\"]], \"line\": {\"color\": \"Grey\", \"width\": 0.5}, \"showscale\": false}, \"mode\": \"markers\", \"showlegend\": false, \"type\": \"scatter\", \"x\": [17, 209, 139, 25, 113, 222, 135, 115, 157, 52, 243, 193, 198, 189, 180, 255, 253, 236, 224, 255, 79, 229, 174, 215, 236, 242, 238, 208, 165, 254, 233, 252], \"xaxis\": \"x3\", \"y\": [0.9639, 0.9806, 0.9774, 0.9643, 0.8731, 0.098, 0.9798, 0.9708, 0.975, 0.9687, 0.1135, 0.9816, 0.9816, 0.9816, 0.9808, 0.9824, 0.9782, 0.9828, 0.9785, 0.9781, 0.098, 0.98, 0.9812, 0.9801, 0.9828, 0.9756, 0.9788, 0.9776, 0.1135, 0.9816, 0.9804, 0.9819], \"yaxis\": \"y3\"}, {\"marker\": {\"color\": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31], \"colorbar\": {\"title\": {\"text\": \"#Trials\"}, \"x\": 1.0, \"xpad\": 40}, \"colorscale\": [[0.0, \"rgb(247,251,255)\"], [0.125, \"rgb(222,235,247)\"], [0.25, \"rgb(198,219,239)\"], [0.375, \"rgb(158,202,225)\"], [0.5, \"rgb(107,174,214)\"], [0.625, \"rgb(66,146,198)\"], [0.75, \"rgb(33,113,181)\"], [0.875, \"rgb(8,81,156)\"], [1.0, \"rgb(8,48,107)\"]], \"line\": {\"color\": \"Grey\", \"width\": 0.5}, \"showscale\": false}, \"mode\": \"markers\", \"showlegend\": false, \"type\": \"scatter\", \"x\": [\"adam\", \"adamax\", \"rmsprop\", \"adam\", \"sgd\", \"rmsprop\", \"adamax\", \"sgd\", \"rmsprop\", \"rmsprop\", \"adamax\", \"adamax\", \"adamax\", \"adamax\", \"adamax\", \"adamax\", \"adamax\", \"adamax\", \"sgd\", \"adam\", \"adamax\", \"adamax\", \"adamax\", \"adamax\", \"adamax\", \"adamax\", \"adamax\", \"adam\", \"sgd\", \"adam\", \"adamax\", \"adam\"], \"xaxis\": \"x4\", \"y\": [0.9639, 0.9806, 0.9774, 0.9643, 0.8731, 0.098, 0.9798, 0.9708, 0.975, 0.9687, 0.1135, 0.9816, 0.9816, 0.9816, 0.9808, 0.9824, 0.9782, 0.9828, 0.9785, 0.9781, 0.098, 0.98, 0.9812, 0.9801, 0.9828, 0.9756, 0.9788, 0.9776, 0.1135, 0.9816, 0.9804, 0.9819], \"yaxis\": \"y4\"}],\n",
       "                        {\"template\": {\"data\": {\"bar\": [{\"error_x\": {\"color\": \"#2a3f5f\"}, \"error_y\": {\"color\": \"#2a3f5f\"}, \"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"bar\"}], \"barpolar\": [{\"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"barpolar\"}], \"carpet\": [{\"aaxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"baxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"type\": \"carpet\"}], \"choropleth\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"choropleth\"}], \"contour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"contour\"}], \"contourcarpet\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"contourcarpet\"}], \"heatmap\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmap\"}], \"heatmapgl\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmapgl\"}], \"histogram\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"histogram\"}], \"histogram2d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2d\"}], \"histogram2dcontour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2dcontour\"}], \"mesh3d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"mesh3d\"}], \"parcoords\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"parcoords\"}], \"pie\": [{\"automargin\": true, \"type\": \"pie\"}], \"scatter\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter\"}], \"scatter3d\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter3d\"}], \"scattercarpet\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattercarpet\"}], \"scattergeo\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergeo\"}], \"scattergl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergl\"}], \"scattermapbox\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattermapbox\"}], \"scatterpolar\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolar\"}], \"scatterpolargl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolargl\"}], \"scatterternary\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterternary\"}], \"surface\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"surface\"}], \"table\": [{\"cells\": {\"fill\": {\"color\": \"#EBF0F8\"}, \"line\": {\"color\": \"white\"}}, \"header\": {\"fill\": {\"color\": \"#C8D4E3\"}, \"line\": {\"color\": \"white\"}}, \"type\": \"table\"}]}, \"layout\": {\"annotationdefaults\": {\"arrowcolor\": \"#2a3f5f\", \"arrowhead\": 0, \"arrowwidth\": 1}, \"coloraxis\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"colorscale\": {\"diverging\": [[0, \"#8e0152\"], [0.1, \"#c51b7d\"], [0.2, \"#de77ae\"], [0.3, \"#f1b6da\"], [0.4, \"#fde0ef\"], [0.5, \"#f7f7f7\"], [0.6, \"#e6f5d0\"], [0.7, \"#b8e186\"], [0.8, \"#7fbc41\"], [0.9, \"#4d9221\"], [1, \"#276419\"]], \"sequential\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"sequentialminus\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]]}, \"colorway\": [\"#636efa\", \"#EF553B\", \"#00cc96\", \"#ab63fa\", \"#FFA15A\", \"#19d3f3\", \"#FF6692\", \"#B6E880\", \"#FF97FF\", \"#FECB52\"], \"font\": {\"color\": \"#2a3f5f\"}, \"geo\": {\"bgcolor\": \"white\", \"lakecolor\": \"white\", \"landcolor\": \"#E5ECF6\", \"showlakes\": true, \"showland\": true, \"subunitcolor\": \"white\"}, \"hoverlabel\": {\"align\": \"left\"}, \"hovermode\": \"closest\", \"mapbox\": {\"style\": \"light\"}, \"paper_bgcolor\": \"white\", \"plot_bgcolor\": \"#E5ECF6\", \"polar\": {\"angularaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"radialaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"scene\": {\"xaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"yaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"zaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}}, \"shapedefaults\": {\"line\": {\"color\": \"#2a3f5f\"}}, \"ternary\": {\"aaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"baxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"caxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"title\": {\"x\": 0.05}, \"xaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"title\": {\"standoff\": 15}, \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}, \"yaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"title\": {\"standoff\": 15}, \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}}}, \"title\": {\"text\": \"Slice Plot\"}, \"width\": 1200, \"xaxis\": {\"anchor\": \"y\", \"domain\": [0.0, 0.2125], \"title\": {\"text\": \"activation\"}}, \"xaxis2\": {\"anchor\": \"y2\", \"domain\": [0.2625, 0.475], \"title\": {\"text\": \"loss\"}}, \"xaxis3\": {\"anchor\": \"y3\", \"domain\": [0.525, 0.7375], \"title\": {\"text\": \"n_neurons\"}}, \"xaxis4\": {\"anchor\": \"y4\", \"domain\": [0.7875, 1.0], \"title\": {\"text\": \"optimizer\"}}, \"yaxis\": {\"anchor\": \"x\", \"domain\": [0.0, 1.0], \"title\": {\"text\": \"Objective Value\"}}, \"yaxis2\": {\"anchor\": \"x2\", \"domain\": [0.0, 1.0], \"matches\": \"y\", \"showticklabels\": false}, \"yaxis3\": {\"anchor\": \"x3\", \"domain\": [0.0, 1.0], \"matches\": \"y\", \"showticklabels\": false}, \"yaxis4\": {\"anchor\": \"x4\", \"domain\": [0.0, 1.0], \"matches\": \"y\", \"showticklabels\": false}},\n",
       "                        {\"responsive\": true}\n",
       "                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('f92c8081-32da-4806-b97c-13d9bde11703');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })\n",
       "                };\n",
       "                \n",
       "            </script>\n",
       "        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 298 ms (started: 2021-10-20 06:29:28 +00:00)\n"
     ]
    }
   ],
   "source": [
    "optuna.visualization.plot_slice(study_4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ypZoWjmkHZq3"
   },
   "source": [
    "### Avaliação dos parâmetros do estudo 3\n",
    "\n",
    "* A função de ativação softmax (como vista nos dois estudos anteriores) continua sendo um outlier, e, de forma semelhante ao que havia acontecido no estudo 1, a função exponencial também foi um outlier, a sigmoid apresentou compartamento um pouco abaixo em um de seus testes. No geral, as outras funções foram consistentes em obter bons resultados.\n",
    "* Surpreendentemente houveram mais outliers quando foram adicionados mais neurônios neste estudo.\n",
    "* SGD, Adamax e RMSprop foram os outlier dentre os otimizadores, apenas a Adam se mostrou consistente nos seus resultados."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "wzTiGhnEF_rg"
   },
   "source": [
    "### Importância dos parâmetros\n",
    "\n",
    "Para todos os nossos estudos, a função de ativação foi o parâmetro que mais importou (sempre com mais de 67% de importância, ou seja mais de 2/3), assim como o otimizador ficou em segundo lugar em todos e o número de neurônios em terceiro."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 558
    },
    "id": "q78ICdPKZGlJ",
    "outputId": "1bff921f-9240-4c3f-a066-c944c4e5fb34"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>\n",
       "            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax) {MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>\n",
       "                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-latest.min.js\"></script>    \n",
       "            <div id=\"9cb92e91-7ab4-4dc8-88ee-c8718803cb33\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>\n",
       "            <script type=\"text/javascript\">\n",
       "                \n",
       "                    window.PLOTLYENV=window.PLOTLYENV || {};\n",
       "                    \n",
       "                if (document.getElementById(\"9cb92e91-7ab4-4dc8-88ee-c8718803cb33\")) {\n",
       "                    Plotly.newPlot(\n",
       "                        '9cb92e91-7ab4-4dc8-88ee-c8718803cb33',\n",
       "                        [{\"cliponaxis\": false, \"hovertemplate\": [\"loss (CategoricalDistribution): 0.0<extra></extra>\", \"n_neurons (IntUniformDistribution): 0.08554847979829752<extra></extra>\", \"optimizer (CategoricalDistribution): 0.23150933715447822<extra></extra>\", \"activation (CategoricalDistribution): 0.6829421830472242<extra></extra>\"], \"marker\": {\"color\": \"rgb(66,146,198)\"}, \"orientation\": \"h\", \"text\": [\"0.0\", \"0.08554847979829752\", \"0.23150933715447822\", \"0.6829421830472242\"], \"textposition\": \"outside\", \"texttemplate\": \"%{text:.2f}\", \"type\": \"bar\", \"x\": [0.0, 0.08554847979829752, 0.23150933715447822, 0.6829421830472242], \"y\": [\"loss\", \"n_neurons\", \"optimizer\", \"activation\"]}],\n",
       "                        {\"showlegend\": false, \"template\": {\"data\": {\"bar\": [{\"error_x\": {\"color\": \"#2a3f5f\"}, \"error_y\": {\"color\": \"#2a3f5f\"}, \"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"bar\"}], \"barpolar\": [{\"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"barpolar\"}], \"carpet\": [{\"aaxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"baxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"type\": \"carpet\"}], \"choropleth\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"choropleth\"}], \"contour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"contour\"}], \"contourcarpet\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"contourcarpet\"}], \"heatmap\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmap\"}], \"heatmapgl\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmapgl\"}], \"histogram\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"histogram\"}], \"histogram2d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2d\"}], \"histogram2dcontour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2dcontour\"}], \"mesh3d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"mesh3d\"}], \"parcoords\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"parcoords\"}], \"pie\": [{\"automargin\": true, \"type\": \"pie\"}], \"scatter\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter\"}], \"scatter3d\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter3d\"}], \"scattercarpet\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattercarpet\"}], \"scattergeo\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergeo\"}], \"scattergl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergl\"}], \"scattermapbox\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattermapbox\"}], \"scatterpolar\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolar\"}], \"scatterpolargl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolargl\"}], \"scatterternary\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterternary\"}], \"surface\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"surface\"}], \"table\": [{\"cells\": {\"fill\": {\"color\": \"#EBF0F8\"}, \"line\": {\"color\": \"white\"}}, \"header\": {\"fill\": {\"color\": \"#C8D4E3\"}, \"line\": {\"color\": \"white\"}}, \"type\": \"table\"}]}, \"layout\": {\"annotationdefaults\": {\"arrowcolor\": \"#2a3f5f\", \"arrowhead\": 0, \"arrowwidth\": 1}, \"coloraxis\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"colorscale\": {\"diverging\": [[0, \"#8e0152\"], [0.1, \"#c51b7d\"], [0.2, \"#de77ae\"], [0.3, \"#f1b6da\"], [0.4, \"#fde0ef\"], [0.5, \"#f7f7f7\"], [0.6, \"#e6f5d0\"], [0.7, \"#b8e186\"], [0.8, \"#7fbc41\"], [0.9, \"#4d9221\"], [1, \"#276419\"]], \"sequential\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"sequentialminus\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]]}, \"colorway\": [\"#636efa\", \"#EF553B\", \"#00cc96\", \"#ab63fa\", \"#FFA15A\", \"#19d3f3\", \"#FF6692\", \"#B6E880\", \"#FF97FF\", \"#FECB52\"], \"font\": {\"color\": \"#2a3f5f\"}, \"geo\": {\"bgcolor\": \"white\", \"lakecolor\": \"white\", \"landcolor\": \"#E5ECF6\", \"showlakes\": true, \"showland\": true, \"subunitcolor\": \"white\"}, \"hoverlabel\": {\"align\": \"left\"}, \"hovermode\": \"closest\", \"mapbox\": {\"style\": \"light\"}, \"paper_bgcolor\": \"white\", \"plot_bgcolor\": \"#E5ECF6\", \"polar\": {\"angularaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"radialaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"scene\": {\"xaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"yaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"zaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}}, \"shapedefaults\": {\"line\": {\"color\": \"#2a3f5f\"}}, \"ternary\": {\"aaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"baxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"caxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"title\": {\"x\": 0.05}, \"xaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"title\": {\"standoff\": 15}, \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}, \"yaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"title\": {\"standoff\": 15}, \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}}}, \"title\": {\"text\": \"Hyperparameter Importances\"}, \"xaxis\": {\"title\": {\"text\": \"Importance for Objective Value\"}}, \"yaxis\": {\"title\": {\"text\": \"Hyperparameter\"}}},\n",
       "                        {\"responsive\": true}\n",
       "                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('9cb92e91-7ab4-4dc8-88ee-c8718803cb33');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })\n",
       "                };\n",
       "                \n",
       "            </script>\n",
       "        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 1.11 s (started: 2021-10-20 06:30:07 +00:00)\n"
     ]
    }
   ],
   "source": [
    "optuna.visualization.plot_param_importances(study_1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 558
    },
    "id": "smMrf-KjZJsY",
    "outputId": "4de64d89-81f1-4700-930d-92f5bc6a4c2c"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>\n",
       "            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax) {MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>\n",
       "                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-latest.min.js\"></script>    \n",
       "            <div id=\"ee4dcb9c-922e-44ea-a27a-94d2f508bd2d\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>\n",
       "            <script type=\"text/javascript\">\n",
       "                \n",
       "                    window.PLOTLYENV=window.PLOTLYENV || {};\n",
       "                    \n",
       "                if (document.getElementById(\"ee4dcb9c-922e-44ea-a27a-94d2f508bd2d\")) {\n",
       "                    Plotly.newPlot(\n",
       "                        'ee4dcb9c-922e-44ea-a27a-94d2f508bd2d',\n",
       "                        [{\"cliponaxis\": false, \"hovertemplate\": [\"loss (CategoricalDistribution): 0.0<extra></extra>\", \"n_neurons (IntUniformDistribution): 0.10189871101732348<extra></extra>\", \"optimizer (CategoricalDistribution): 0.13686889868289004<extra></extra>\", \"activation (CategoricalDistribution): 0.7612323902997866<extra></extra>\"], \"marker\": {\"color\": \"rgb(66,146,198)\"}, \"orientation\": \"h\", \"text\": [\"0.0\", \"0.10189871101732348\", \"0.13686889868289004\", \"0.7612323902997866\"], \"textposition\": \"outside\", \"texttemplate\": \"%{text:.2f}\", \"type\": \"bar\", \"x\": [0.0, 0.10189871101732348, 0.13686889868289004, 0.7612323902997866], \"y\": [\"loss\", \"n_neurons\", \"optimizer\", \"activation\"]}],\n",
       "                        {\"showlegend\": false, \"template\": {\"data\": {\"bar\": [{\"error_x\": {\"color\": \"#2a3f5f\"}, \"error_y\": {\"color\": \"#2a3f5f\"}, \"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"bar\"}], \"barpolar\": [{\"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"barpolar\"}], \"carpet\": [{\"aaxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"baxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"type\": \"carpet\"}], \"choropleth\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"choropleth\"}], \"contour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"contour\"}], \"contourcarpet\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"contourcarpet\"}], \"heatmap\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmap\"}], \"heatmapgl\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmapgl\"}], \"histogram\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"histogram\"}], \"histogram2d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2d\"}], \"histogram2dcontour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2dcontour\"}], \"mesh3d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"mesh3d\"}], \"parcoords\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"parcoords\"}], \"pie\": [{\"automargin\": true, \"type\": \"pie\"}], \"scatter\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter\"}], \"scatter3d\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter3d\"}], \"scattercarpet\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattercarpet\"}], \"scattergeo\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergeo\"}], \"scattergl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergl\"}], \"scattermapbox\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattermapbox\"}], \"scatterpolar\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolar\"}], \"scatterpolargl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolargl\"}], \"scatterternary\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterternary\"}], \"surface\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"surface\"}], \"table\": [{\"cells\": {\"fill\": {\"color\": \"#EBF0F8\"}, \"line\": {\"color\": \"white\"}}, \"header\": {\"fill\": {\"color\": \"#C8D4E3\"}, \"line\": {\"color\": \"white\"}}, \"type\": \"table\"}]}, \"layout\": {\"annotationdefaults\": {\"arrowcolor\": \"#2a3f5f\", \"arrowhead\": 0, \"arrowwidth\": 1}, \"coloraxis\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"colorscale\": {\"diverging\": [[0, \"#8e0152\"], [0.1, \"#c51b7d\"], [0.2, \"#de77ae\"], [0.3, \"#f1b6da\"], [0.4, \"#fde0ef\"], [0.5, \"#f7f7f7\"], [0.6, \"#e6f5d0\"], [0.7, \"#b8e186\"], [0.8, \"#7fbc41\"], [0.9, \"#4d9221\"], [1, \"#276419\"]], \"sequential\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"sequentialminus\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]]}, \"colorway\": [\"#636efa\", \"#EF553B\", \"#00cc96\", \"#ab63fa\", \"#FFA15A\", \"#19d3f3\", \"#FF6692\", \"#B6E880\", \"#FF97FF\", \"#FECB52\"], \"font\": {\"color\": \"#2a3f5f\"}, \"geo\": {\"bgcolor\": \"white\", \"lakecolor\": \"white\", \"landcolor\": \"#E5ECF6\", \"showlakes\": true, \"showland\": true, \"subunitcolor\": \"white\"}, \"hoverlabel\": {\"align\": \"left\"}, \"hovermode\": \"closest\", \"mapbox\": {\"style\": \"light\"}, \"paper_bgcolor\": \"white\", \"plot_bgcolor\": \"#E5ECF6\", \"polar\": {\"angularaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"radialaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"scene\": {\"xaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"yaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"zaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}}, \"shapedefaults\": {\"line\": {\"color\": \"#2a3f5f\"}}, \"ternary\": {\"aaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"baxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"caxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"title\": {\"x\": 0.05}, \"xaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"title\": {\"standoff\": 15}, \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}, \"yaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"title\": {\"standoff\": 15}, \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}}}, \"title\": {\"text\": \"Hyperparameter Importances\"}, \"xaxis\": {\"title\": {\"text\": \"Importance for Objective Value\"}}, \"yaxis\": {\"title\": {\"text\": \"Hyperparameter\"}}},\n",
       "                        {\"responsive\": true}\n",
       "                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('ee4dcb9c-922e-44ea-a27a-94d2f508bd2d');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })\n",
       "                };\n",
       "                \n",
       "            </script>\n",
       "        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 954 ms (started: 2021-10-20 06:30:18 +00:00)\n"
     ]
    }
   ],
   "source": [
    "optuna.visualization.plot_param_importances(study_2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 558
    },
    "id": "6q_StVKHZMRL",
    "outputId": "01606645-9c52-4f9e-d0c6-f65b2eaf76a5"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>\n",
       "            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax) {MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>\n",
       "                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-latest.min.js\"></script>    \n",
       "            <div id=\"a6f8b9ac-9d49-4bff-bf82-909a7245bd63\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>\n",
       "            <script type=\"text/javascript\">\n",
       "                \n",
       "                    window.PLOTLYENV=window.PLOTLYENV || {};\n",
       "                    \n",
       "                if (document.getElementById(\"a6f8b9ac-9d49-4bff-bf82-909a7245bd63\")) {\n",
       "                    Plotly.newPlot(\n",
       "                        'a6f8b9ac-9d49-4bff-bf82-909a7245bd63',\n",
       "                        [{\"cliponaxis\": false, \"hovertemplate\": [\"loss (CategoricalDistribution): 0.0<extra></extra>\", \"n_neurons (IntUniformDistribution): 0.00017334665843778593<extra></extra>\", \"optimizer (CategoricalDistribution): 0.0051594611579057195<extra></extra>\", \"activation (CategoricalDistribution): 0.9946671921836565<extra></extra>\"], \"marker\": {\"color\": \"rgb(66,146,198)\"}, \"orientation\": \"h\", \"text\": [\"0.0\", \"0.00017334665843778593\", \"0.0051594611579057195\", \"0.9946671921836565\"], \"textposition\": \"outside\", \"texttemplate\": \"%{text:.2f}\", \"type\": \"bar\", \"x\": [0.0, 0.00017334665843778593, 0.0051594611579057195, 0.9946671921836565], \"y\": [\"loss\", \"n_neurons\", \"optimizer\", \"activation\"]}],\n",
       "                        {\"showlegend\": false, \"template\": {\"data\": {\"bar\": [{\"error_x\": {\"color\": \"#2a3f5f\"}, \"error_y\": {\"color\": \"#2a3f5f\"}, \"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"bar\"}], \"barpolar\": [{\"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"barpolar\"}], \"carpet\": [{\"aaxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"baxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"type\": \"carpet\"}], \"choropleth\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"choropleth\"}], \"contour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"contour\"}], \"contourcarpet\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"contourcarpet\"}], \"heatmap\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmap\"}], \"heatmapgl\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmapgl\"}], \"histogram\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"histogram\"}], \"histogram2d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2d\"}], \"histogram2dcontour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2dcontour\"}], \"mesh3d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"mesh3d\"}], \"parcoords\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"parcoords\"}], \"pie\": [{\"automargin\": true, \"type\": \"pie\"}], \"scatter\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter\"}], \"scatter3d\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter3d\"}], \"scattercarpet\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattercarpet\"}], \"scattergeo\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergeo\"}], \"scattergl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergl\"}], \"scattermapbox\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattermapbox\"}], \"scatterpolar\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolar\"}], \"scatterpolargl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolargl\"}], \"scatterternary\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterternary\"}], \"surface\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"surface\"}], \"table\": [{\"cells\": {\"fill\": {\"color\": \"#EBF0F8\"}, \"line\": {\"color\": \"white\"}}, \"header\": {\"fill\": {\"color\": \"#C8D4E3\"}, \"line\": {\"color\": \"white\"}}, \"type\": \"table\"}]}, \"layout\": {\"annotationdefaults\": {\"arrowcolor\": \"#2a3f5f\", \"arrowhead\": 0, \"arrowwidth\": 1}, \"coloraxis\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"colorscale\": {\"diverging\": [[0, \"#8e0152\"], [0.1, \"#c51b7d\"], [0.2, \"#de77ae\"], [0.3, \"#f1b6da\"], [0.4, \"#fde0ef\"], [0.5, \"#f7f7f7\"], [0.6, \"#e6f5d0\"], [0.7, \"#b8e186\"], [0.8, \"#7fbc41\"], [0.9, \"#4d9221\"], [1, \"#276419\"]], \"sequential\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"sequentialminus\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]]}, \"colorway\": [\"#636efa\", \"#EF553B\", \"#00cc96\", \"#ab63fa\", \"#FFA15A\", \"#19d3f3\", \"#FF6692\", \"#B6E880\", \"#FF97FF\", \"#FECB52\"], \"font\": {\"color\": \"#2a3f5f\"}, \"geo\": {\"bgcolor\": \"white\", \"lakecolor\": \"white\", \"landcolor\": \"#E5ECF6\", \"showlakes\": true, \"showland\": true, \"subunitcolor\": \"white\"}, \"hoverlabel\": {\"align\": \"left\"}, \"hovermode\": \"closest\", \"mapbox\": {\"style\": \"light\"}, \"paper_bgcolor\": \"white\", \"plot_bgcolor\": \"#E5ECF6\", \"polar\": {\"angularaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"radialaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"scene\": {\"xaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"yaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"zaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}}, \"shapedefaults\": {\"line\": {\"color\": \"#2a3f5f\"}}, \"ternary\": {\"aaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"baxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"caxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"title\": {\"x\": 0.05}, \"xaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"title\": {\"standoff\": 15}, \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}, \"yaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"title\": {\"standoff\": 15}, \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}}}, \"title\": {\"text\": \"Hyperparameter Importances\"}, \"xaxis\": {\"title\": {\"text\": \"Importance for Objective Value\"}}, \"yaxis\": {\"title\": {\"text\": \"Hyperparameter\"}}},\n",
       "                        {\"responsive\": true}\n",
       "                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('a6f8b9ac-9d49-4bff-bf82-909a7245bd63');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })\n",
       "                };\n",
       "                \n",
       "            </script>\n",
       "        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 1.14 s (started: 2021-10-20 06:30:28 +00:00)\n"
     ]
    }
   ],
   "source": [
    "optuna.visualization.plot_param_importances(study_4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "8qh1g-npjiVr"
   },
   "source": [
    "###Sobre os resultados\n",
    "#########Fazer aqui análise comparativa do  tuning aqui#########"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "oRM6RlUju6Vk"
   },
   "source": [
    "#Reconhecimento de Dígitos com CNN"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "machine_shape": "hm",
   "name": "Redes Neurais - Mini Projeto.ipynb",
   "provenance": [],
   "toc_visible": true
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}