Autoassociator Logistic.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "f0e2aaf8-0b8f-4fc1-a771-1867b22350c7",
   "metadata": {},
   "source": [
    "<hr style=\"border-top: 3px solid black\"></hr>\n",
    "<center>\n",
    "<h1><span style=\"color:black\">Coding an Autoassociator from Scratch</span></h1>\n",
    "</center>\n",
    "<hr style=\"border-top: 3px solid black\"></hr>\n",
    "\n",
    "# 1°) With the logistic as activation function (binary inputs)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1a819ee9-0b35-4fc6-840c-02e9dcdb332f",
   "metadata": {},
   "source": [
    "**Activation function and its derivative:**\n",
    "*Logisitc*\n",
    "\n",
    "$ S(x) = \\frac{1}{1 + e^{-x}} $\n",
    "\n",
    "$ S'(x) = S(x) \\cdot (1 - S(x)) $\n",
    "\n",
    "**Error function:** \n",
    "*The Mean Squared Error*\n",
    "\n",
    "$\\text{MSE}(y_{\\text{true}}, y_{\\text{pred}}) = \\frac{1}{n} \\sum_{i=1}^{n} (y_{\\text{pred},i} - y_{\\text{true},i})^2$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "777209f6-b50f-49ce-a80c-6e1dbb3290ab",
   "metadata": {},
   "source": [
    "| Phase             | Formulas |\n",
    "|-------------------|----------|\n",
    "| **Forward pass:** | *Weighted Sum:* $z_1 = p \\cdot w_1 + b_1$ <br> $a_1 = \\text{S}(z_1)$ |\n",
    "| **Backward pass:** | $\\delta z_1 = (a_1 - t) \\cdot \\text{S'}(z_1)$ <br> $\\frac{\\partial w_1}{\\partial \\text{loss}} = \\frac{p^T \\cdot \\delta z_1}{\\text{len}(p)}$ <br> $\\frac{\\partial b_1}{\\partial \\text{loss}} = \\frac{\\text{sum}(\\delta z_1)}{\\text{len}(p)}$ |\n",
    "| **Delta Rule:**   | $w_1 = w_1 - \\text{lr} \\cdot \\frac{\\partial w_1}{\\partial \\text{loss}}$ <br> $b_1 = b_1 - \\text{lr} \\cdot \\frac{\\partial b_1}{\\partial \\text{loss}}$ |\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "05dcd464-8a7d-4982-9791-e1b6aff7c525",
   "metadata": {},
   "source": [
    "In the *backward pass* the input pattern $p$ is transposed to a column vector such that it can be multiplied with the gradient $\\delta z_1$. In linear algebra, the number of columns of the first matrix has to correspond to the number of lines of the second matrix during matrix multiplication. Then, $p$, wich is first defined as a line vector $(1, elements)$, has to change to be a column vector of shape $(elements, 1)$.\n",
    "<br><br>\n",
    "A dot product takes two sequences of the same lenght and outputs a single number. It is the sum of the products of the corresponding elements between the two vectors. Then, here, with p = (0,1) and w = (0.5, 0.2) we compute $z = p \\cdot w = 0 \\times .5 + 1 \\times .2 = .2$\n",
    "<br><br>\n",
    "We divide by $len(p)$ to compute a mean of the gradients. This normalizes the gradients relatively to the number of elements in the input vector."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "0bdccbf9-0c5e-45fe-ab9b-543567425b8a",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "bb07dfc4-2ba2-473c-b5c1-5ab583c0b23e",
   "metadata": {},
   "outputs": [],
   "source": [
    "#define the patterns:\n",
    "p1 = [0,0]\n",
    "p2 = [0,1]\n",
    "p3 = [1,0]\n",
    "p4 = [1,1]\n",
    "p5 = [0.5,0.5]\n",
    "p6 = [0.1,0.5]\n",
    "p_list = [p1,p2,p3,p4,p5,p6]\n",
    "patterns = np.asarray(p_list)\n",
    "\n",
    "t1 = 0.3\n",
    "t2 = 0.3\n",
    "t3 = 0.1\n",
    "t4 = 0.3\n",
    "t5 = 0.7\n",
    "t6 = 0.4\n",
    "targets = np.asarray([t1,t2,t3,t4,t5,t6])\n",
    "\n",
    "input_shape = 2\n",
    "output_shape= 1\n",
    "epochs = 500\n",
    "learning_rate = 0.01\n",
    "lr = learning_rate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "b38482fd-4275-4932-b9d3-d14d177002ad",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch number: 0 MSE_loss: 0.27411583891362784\n",
      "Epoch number: 200 MSE_loss: 0.27184121330768657\n",
      "Epoch number: 400 MSE_loss: 0.2695666135988211\n",
      "Epoch number: 600 MSE_loss: 0.2672917534273627\n",
      "Epoch number: 800 MSE_loss: 0.26501644696760257\n",
      "Epoch number: 1000 MSE_loss: 0.26274060822615736\n",
      "Epoch number: 1200 MSE_loss: 0.2604642501337386\n",
      "Epoch number: 1400 MSE_loss: 0.25818748338338865\n",
      "Epoch number: 1600 MSE_loss: 0.25591051496621653\n",
      "Epoch number: 1800 MSE_loss: 0.253633646355183\n",
      "Epoch number: 2000 MSE_loss: 0.25135727128864443\n",
      "Epoch number: 2200 MSE_loss: 0.24908187310822497\n",
      "Epoch number: 2400 MSE_loss: 0.2468080216101963\n",
      "Epoch number: 2600 MSE_loss: 0.24453636937589682\n",
      "Epoch number: 2800 MSE_loss: 0.2422676475547703\n",
      "Epoch number: 3000 MSE_loss: 0.24000266108325197\n",
      "Epoch number: 3200 MSE_loss: 0.23774228333379535\n",
      "Epoch number: 3400 MSE_loss: 0.23548745020059056\n",
      "Epoch number: 3600 MSE_loss: 0.23323915364166328\n",
      "Epoch number: 3800 MSE_loss: 0.2309984347107001\n",
      "Epoch number: 4000 MSE_loss: 0.22876637612569536\n",
      "Epoch number: 4200 MSE_loss: 0.22654409443490695\n",
      "Epoch number: 4400 MSE_loss: 0.2243327318531544\n",
      "Epoch number: 4600 MSE_loss: 0.22213344785271924\n",
      "Epoch number: 4800 MSE_loss: 0.2199474106025632\n",
      "Epoch number: 5000 MSE_loss: 0.21777578835685188\n",
      "Epoch number: 5200 MSE_loss: 0.21561974089854438\n",
      "Epoch number: 5400 MSE_loss: 0.2134804111458246\n",
      "Epoch number: 5600 MSE_loss: 0.21135891702830034\n",
      "Epoch number: 5800 MSE_loss: 0.2092563437361518\n",
      "Epoch number: 6000 MSE_loss: 0.20717373643889705\n",
      "Epoch number: 6200 MSE_loss: 0.20511209356137103\n",
      "Epoch number: 6400 MSE_loss: 0.20307236069323495\n",
      "Epoch number: 6600 MSE_loss: 0.20105542519524286\n",
      "Epoch number: 6800 MSE_loss: 0.1990621115510918\n",
      "Epoch number: 7000 MSE_loss: 0.19709317749847932\n",
      "Epoch number: 7200 MSE_loss: 0.1951493109575195\n",
      "Epoch number: 7400 MSE_loss: 0.19323112775944573\n",
      "Epoch number: 7600 MSE_loss: 0.1913391701640205\n",
      "Epoch number: 7800 MSE_loss: 0.18947390614069862\n",
      "Epoch number: 8000 MSE_loss: 0.18763572937668652\n",
      "Epoch number: 8200 MSE_loss: 0.1858249599648585\n",
      "Epoch number: 8400 MSE_loss: 0.18404184571619883\n",
      "Epoch number: 8600 MSE_loss: 0.18228656403510624\n",
      "Epoch number: 8800 MSE_loss: 0.1805592242915327\n",
      "Epoch number: 9000 MSE_loss: 0.17885987062144107\n",
      "Epoch number: 9200 MSE_loss: 0.17718848508633248\n",
      "Epoch number: 9400 MSE_loss: 0.17554499112343577\n",
      "Epoch number: 9600 MSE_loss: 0.17392925722034103\n",
      "Epoch number: 9800 MSE_loss: 0.17234110075119147\n",
      "Epoch number: 10000 MSE_loss: 0.17078029191576372\n",
      "Epoch number: 10200 MSE_loss: 0.16924655772765496\n",
      "Epoch number: 10400 MSE_loss: 0.16773958600312602\n",
      "Epoch number: 10600 MSE_loss: 0.16625902930773148\n",
      "Epoch number: 10800 MSE_loss: 0.1648045088235243\n",
      "Epoch number: 11000 MSE_loss: 0.16337561810521078\n",
      "Epoch number: 11200 MSE_loss: 0.16197192669901794\n",
      "Epoch number: 11400 MSE_loss: 0.1605929836031464\n",
      "Epoch number: 11600 MSE_loss: 0.15923832055342357\n",
      "Epoch number: 11800 MSE_loss: 0.15790745512211737\n",
      "Epoch number: 12000 MSE_loss: 0.15659989362177865\n",
      "Epoch number: 12200 MSE_loss: 0.15531513380944567\n",
      "Epoch number: 12400 MSE_loss: 0.15405266738957024\n",
      "Epoch number: 12600 MSE_loss: 0.15281198231661866\n",
      "Epoch number: 12800 MSE_loss: 0.15159256490049158\n",
      "Epoch number: 13000 MSE_loss: 0.15039390171971886\n",
      "Epoch number: 13200 MSE_loss: 0.14921548134884546\n",
      "Epoch number: 13400 MSE_loss: 0.1480567959075745\n",
      "Epoch number: 13600 MSE_loss: 0.14691734244010066\n",
      "Epoch number: 13800 MSE_loss: 0.14579662413368635\n",
      "Epoch number: 14000 MSE_loss: 0.14469415138594066\n",
      "Epoch number: 14200 MSE_loss: 0.1436094427304793\n",
      "Epoch number: 14400 MSE_loss: 0.14254202563071341\n",
      "Epoch number: 14600 MSE_loss: 0.1414914371514483\n",
      "Epoch number: 14800 MSE_loss: 0.1404572245178033\n",
      "Epoch number: 15000 MSE_loss: 0.1394389455707106\n",
      "Epoch number: 15200 MSE_loss: 0.13843616912792608\n",
      "Epoch number: 15400 MSE_loss: 0.13744847525911305\n",
      "Epoch number: 15600 MSE_loss: 0.13647545548314988\n",
      "Epoch number: 15800 MSE_loss: 0.13551671289537767\n",
      "Epoch number: 16000 MSE_loss: 0.13457186223205345\n",
      "Epoch number: 16200 MSE_loss: 0.13364052987881694\n",
      "Epoch number: 16400 MSE_loss: 0.13272235382952416\n",
      "Epoch number: 16600 MSE_loss: 0.13181698360134872\n",
      "Epoch number: 16800 MSE_loss: 0.13092408011161094\n",
      "Epoch number: 17000 MSE_loss: 0.1300433155213693\n",
      "Epoch number: 17200 MSE_loss: 0.1291743730503964\n",
      "Epoch number: 17400 MSE_loss: 0.1283169467677706\n",
      "Epoch number: 17600 MSE_loss: 0.12747074136194156\n",
      "Epoch number: 17800 MSE_loss: 0.1266354718937743\n",
      "Epoch number: 18000 MSE_loss: 0.12581086353574758\n",
      "Epoch number: 18200 MSE_loss: 0.12499665130016924\n",
      "Epoch number: 18400 MSE_loss: 0.12419257975898283\n",
      "Epoch number: 18600 MSE_loss: 0.12339840275747077\n",
      "Epoch number: 18800 MSE_loss: 0.12261388312390797\n",
      "Epoch number: 19000 MSE_loss: 0.12183879237699206\n",
      "Epoch number: 19200 MSE_loss: 0.12107291043265966\n",
      "Epoch number: 19400 MSE_loss: 0.12031602531170842\n",
      "Epoch number: 19600 MSE_loss: 0.11956793284946171\n",
      "Epoch number: 19800 MSE_loss: 0.11882843640855105\n",
      "Epoch number: 20000 MSE_loss: 0.11809734659574482\n",
      "Epoch number: 20200 MSE_loss: 0.11737448098361329\n",
      "Epoch number: 20400 MSE_loss: 0.1166596638377017\n",
      "Epoch number: 20600 MSE_loss: 0.11595272584977012\n",
      "Epoch number: 20800 MSE_loss: 0.11525350387756061\n",
      "Epoch number: 21000 MSE_loss: 0.11456184069146351\n",
      "Epoch number: 21200 MSE_loss: 0.11387758472837332\n",
      "Epoch number: 21400 MSE_loss: 0.11320058985295589\n",
      "Epoch number: 21600 MSE_loss: 0.11253071512648262\n",
      "Epoch number: 21800 MSE_loss: 0.11186782458333357\n",
      "Epoch number: 22000 MSE_loss: 0.11121178701522076\n",
      "Epoch number: 22200 MSE_loss: 0.11056247576313992\n",
      "Epoch number: 22400 MSE_loss: 0.109919768517021\n",
      "Epoch number: 22600 MSE_loss: 0.10928354712301536\n",
      "Epoch number: 22800 MSE_loss: 0.10865369739832743\n",
      "Epoch number: 23000 MSE_loss: 0.10803010895347702\n",
      "Epoch number: 23200 MSE_loss: 0.10741267502185549\n",
      "Epoch number: 23400 MSE_loss: 0.10680129229642299\n",
      "Epoch number: 23600 MSE_loss: 0.10619586077337845\n",
      "Epoch number: 23800 MSE_loss: 0.10559628360262424\n",
      "Epoch number: 24000 MSE_loss: 0.10500246694483568\n",
      "Epoch number: 24200 MSE_loss: 0.10441431983494066\n",
      "Epoch number: 24400 MSE_loss: 0.1038317540518074\n",
      "Epoch number: 24600 MSE_loss: 0.10325468399393639\n",
      "Epoch number: 24800 MSE_loss: 0.10268302656094912\n",
      "Epoch number: 25000 MSE_loss: 0.10211670104066596\n",
      "Epoch number: 25200 MSE_loss: 0.10155562900156534\n",
      "Epoch number: 25400 MSE_loss: 0.10099973419041834\n",
      "Epoch number: 25600 MSE_loss: 0.1004489424348939\n",
      "Epoch number: 25800 MSE_loss: 0.09990318155093325\n",
      "Epoch number: 26000 MSE_loss: 0.09936238125469521\n",
      "Epoch number: 26200 MSE_loss: 0.09882647307887871\n",
      "Epoch number: 26400 MSE_loss: 0.09829539029323135\n",
      "Epoch number: 26600 MSE_loss: 0.0977690678290604\n",
      "Epoch number: 26800 MSE_loss: 0.09724744220756433\n",
      "Epoch number: 27000 MSE_loss: 0.09673045147181131\n",
      "Epoch number: 27200 MSE_loss: 0.09621803512219389\n",
      "Epoch number: 27400 MSE_loss: 0.09571013405519628\n",
      "Epoch number: 27600 MSE_loss: 0.09520669050531566\n",
      "Epoch number: 27800 MSE_loss: 0.09470764798998425\n",
      "Epoch number: 28000 MSE_loss: 0.09421295125734519\n",
      "Epoch number: 28200 MSE_loss: 0.09372254623674026\n",
      "Epoch number: 28400 MSE_loss: 0.09323637999177331\n",
      "Epoch number: 28600 MSE_loss: 0.0927544006758191\n",
      "Epoch number: 28800 MSE_loss: 0.0922765574898516\n",
      "Epoch number: 29000 MSE_loss: 0.09180280064247304\n",
      "Epoch number: 29200 MSE_loss: 0.09133308131202734\n",
      "Epoch number: 29400 MSE_loss: 0.09086735161069028\n",
      "Epoch number: 29600 MSE_loss: 0.09040556455042988\n",
      "Epoch number: 29800 MSE_loss: 0.08994767401073837\n",
      "Epoch number: 30000 MSE_loss: 0.08949363470803993\n",
      "Epoch number: 30200 MSE_loss: 0.08904340216668434\n",
      "Epoch number: 30400 MSE_loss: 0.08859693269143883\n",
      "Epoch number: 30600 MSE_loss: 0.08815418334139741\n",
      "Epoch number: 30800 MSE_loss: 0.08771511190522903\n",
      "Epoch number: 31000 MSE_loss: 0.08727967687769041\n",
      "Epoch number: 31200 MSE_loss: 0.08684783743733353\n",
      "Epoch number: 31400 MSE_loss: 0.08641955342534073\n",
      "Epoch number: 31600 MSE_loss: 0.08599478532542391\n",
      "Epoch number: 31800 MSE_loss: 0.08557349424472888\n",
      "Epoch number: 32000 MSE_loss: 0.08515564189568674\n",
      "Epoch number: 32200 MSE_loss: 0.0847411905787591\n",
      "Epoch number: 32400 MSE_loss: 0.08433010316602689\n",
      "Epoch number: 32600 MSE_loss: 0.08392234308557316\n",
      "Epoch number: 32800 MSE_loss: 0.08351787430661584\n",
      "Epoch number: 33000 MSE_loss: 0.08311666132534676\n",
      "Epoch number: 33200 MSE_loss: 0.0827186691514364\n",
      "Epoch number: 33400 MSE_loss: 0.08232386329516599\n",
      "Epoch number: 33600 MSE_loss: 0.0819322097551511\n",
      "Epoch number: 33800 MSE_loss: 0.08154367500662214\n",
      "Epoch number: 34000 MSE_loss: 0.08115822599023038\n",
      "Epoch number: 34200 MSE_loss: 0.08077583010134802\n",
      "Epoch number: 34400 MSE_loss: 0.0803964551798348\n",
      "Epoch number: 34600 MSE_loss: 0.0800200695002437\n",
      "Epoch number: 34800 MSE_loss: 0.07964664176244056\n",
      "Epoch number: 35000 MSE_loss: 0.07927614108261337\n",
      "Epoch number: 35200 MSE_loss: 0.07890853698464971\n",
      "Epoch number: 35400 MSE_loss: 0.07854379939186008\n",
      "Epoch number: 35600 MSE_loss: 0.07818189861902813\n",
      "Epoch number: 35800 MSE_loss: 0.07782280536476861\n",
      "Epoch number: 36000 MSE_loss: 0.07746649070417583\n",
      "Epoch number: 36200 MSE_loss: 0.07711292608174548\n",
      "Epoch number: 36400 MSE_loss: 0.07676208330455565\n",
      "Epoch number: 36600 MSE_loss: 0.07641393453569094\n",
      "Epoch number: 36800 MSE_loss: 0.07606845228789678\n",
      "Epoch number: 37000 MSE_loss: 0.07572560941745139\n",
      "Epoch number: 37200 MSE_loss: 0.07538537911824252\n",
      "Epoch number: 37400 MSE_loss: 0.07504773491603829\n",
      "Epoch number: 37600 MSE_loss: 0.07471265066294146\n",
      "Epoch number: 37800 MSE_loss: 0.07438010053201684\n",
      "Epoch number: 38000 MSE_loss: 0.074050059012083\n",
      "Epoch number: 38200 MSE_loss: 0.07372250090265882\n",
      "Epoch number: 38400 MSE_loss: 0.07339740130905743\n",
      "Epoch number: 38600 MSE_loss: 0.07307473563761946\n",
      "Epoch number: 38800 MSE_loss: 0.07275447959107802\n",
      "Epoch number: 39000 MSE_loss: 0.0724366091640496\n",
      "Epoch number: 39200 MSE_loss: 0.07212110063864358\n",
      "Epoch number: 39400 MSE_loss: 0.07180793058018481\n",
      "Epoch number: 39600 MSE_loss: 0.07149707583304382\n",
      "Epoch number: 39800 MSE_loss: 0.07118851351656928\n",
      "Epoch number: 40000 MSE_loss: 0.07088222102111756\n",
      "Epoch number: 40200 MSE_loss: 0.07057817600417539\n",
      "Epoch number: 40400 MSE_loss: 0.07027635638657044\n",
      "Epoch number: 40600 MSE_loss: 0.06997674034876726\n",
      "Epoch number: 40800 MSE_loss: 0.06967930632724263\n",
      "Epoch number: 41000 MSE_loss: 0.069384033010939\n",
      "Epoch number: 41200 MSE_loss: 0.06909089933779113\n",
      "Epoch number: 41400 MSE_loss: 0.06879988449132368\n",
      "Epoch number: 41600 MSE_loss: 0.06851096789731617\n",
      "Epoch number: 41800 MSE_loss: 0.06822412922053338\n",
      "Epoch number: 42000 MSE_loss: 0.0679393483615177\n",
      "Epoch number: 42200 MSE_loss: 0.06765660545344164\n",
      "Epoch number: 42400 MSE_loss: 0.06737588085901829\n",
      "Epoch number: 42600 MSE_loss: 0.06709715516746682\n",
      "Epoch number: 42800 MSE_loss: 0.0668204091915321\n",
      "Epoch number: 43000 MSE_loss: 0.06654562396455577\n",
      "Epoch number: 43200 MSE_loss: 0.0662727807375969\n",
      "Epoch number: 43400 MSE_loss: 0.06600186097660199\n",
      "Epoch number: 43600 MSE_loss: 0.06573284635962033\n",
      "Epoch number: 43800 MSE_loss: 0.06546571877406543\n",
      "Epoch number: 44000 MSE_loss: 0.06520046031402014\n",
      "Epoch number: 44200 MSE_loss: 0.06493705327758423\n",
      "Epoch number: 44400 MSE_loss: 0.06467548016426318\n",
      "Epoch number: 44600 MSE_loss: 0.06441572367239744\n",
      "Epoch number: 44800 MSE_loss: 0.06415776669663022\n",
      "Epoch number: 45000 MSE_loss: 0.06390159232541362\n",
      "Epoch number: 45200 MSE_loss: 0.06364718383855197\n",
      "Epoch number: 45400 MSE_loss: 0.06339452470478069\n",
      "Epoch number: 45600 MSE_loss: 0.06314359857938077\n",
      "Epoch number: 45800 MSE_loss: 0.06289438930182734\n",
      "Epoch number: 46000 MSE_loss: 0.06264688089347237\n",
      "Epoch number: 46200 MSE_loss: 0.062401057555259286\n",
      "Epoch number: 46400 MSE_loss: 0.06215690366547063\n",
      "Epoch number: 46600 MSE_loss: 0.061914403777506365\n",
      "Epoch number: 46800 MSE_loss: 0.06167354261769342\n",
      "Epoch number: 47000 MSE_loss: 0.06143430508312527\n",
      "Epoch number: 47200 MSE_loss: 0.061196676239530935\n",
      "Epoch number: 47400 MSE_loss: 0.060960641319173225\n",
      "Epoch number: 47600 MSE_loss: 0.06072618571877502\n",
      "Epoch number: 47800 MSE_loss: 0.06049329499747397\n",
      "Epoch number: 48000 MSE_loss: 0.060261954874804326\n",
      "Epoch number: 48200 MSE_loss: 0.06003215122870561\n",
      "Epoch number: 48400 MSE_loss: 0.05980387009355814\n",
      "Epoch number: 48600 MSE_loss: 0.05957709765824424\n",
      "Epoch number: 48800 MSE_loss: 0.05935182026423527\n",
      "Epoch number: 49000 MSE_loss: 0.05912802440370377\n",
      "Epoch number: 49200 MSE_loss: 0.05890569671766042\n",
      "Epoch number: 49400 MSE_loss: 0.05868482399411534\n",
      "Epoch number: 49600 MSE_loss: 0.05846539316626376\n",
      "Epoch number: 49800 MSE_loss: 0.05824739131069476\n",
      "Epoch number: 50000 MSE_loss: 0.05803080564562407\n",
      "Epoch number: 50200 MSE_loss: 0.05781562352914906\n",
      "Epoch number: 50400 MSE_loss: 0.05760183245752722\n",
      "Epoch number: 50600 MSE_loss: 0.05738942006347627\n",
      "Epoch number: 50800 MSE_loss: 0.057178374114496645\n",
      "Epoch number: 51000 MSE_loss: 0.05696868251121562\n",
      "Epoch number: 51200 MSE_loss: 0.05676033328575273\n",
      "Epoch number: 51400 MSE_loss: 0.05655331460010649\n",
      "Epoch number: 51600 MSE_loss: 0.05634761474456177\n",
      "Epoch number: 51800 MSE_loss: 0.05614322213611789\n",
      "Epoch number: 52000 MSE_loss: 0.055940125316936685\n",
      "Epoch number: 52200 MSE_loss: 0.055738312952811044\n",
      "Epoch number: 52400 MSE_loss: 0.055537773831652475\n",
      "Epoch number: 52600 MSE_loss: 0.0553384968619991\n",
      "Epoch number: 52800 MSE_loss: 0.05514047107154194\n",
      "Epoch number: 53000 MSE_loss: 0.05494368560567072\n",
      "Epoch number: 53200 MSE_loss: 0.05474812972603823\n",
      "Epoch number: 53400 MSE_loss: 0.0545537928091431\n",
      "Epoch number: 53600 MSE_loss: 0.05436066434493114\n",
      "Epoch number: 53800 MSE_loss: 0.05416873393541425\n",
      "Epoch number: 54000 MSE_loss: 0.05397799129330739\n",
      "Epoch number: 54200 MSE_loss: 0.053788426240683106\n",
      "Epoch number: 54400 MSE_loss: 0.05360002870764304\n",
      "Epoch number: 54600 MSE_loss: 0.053412788731007126\n",
      "Epoch number: 54800 MSE_loss: 0.05322669645301927\n",
      "Epoch number: 55000 MSE_loss: 0.053041742120069624\n",
      "Epoch number: 55200 MSE_loss: 0.052857916081433774\n",
      "Epoch number: 55400 MSE_loss: 0.05267520878802771\n",
      "Epoch number: 55600 MSE_loss: 0.05249361079117911\n",
      "Epoch number: 55800 MSE_loss: 0.05231311274141425\n",
      "Epoch number: 56000 MSE_loss: 0.05213370538726101\n",
      "Epoch number: 56200 MSE_loss: 0.05195537957406671\n",
      "Epoch number: 56400 MSE_loss: 0.05177812624283149\n",
      "Epoch number: 56600 MSE_loss: 0.05160193642905698\n",
      "Epoch number: 56800 MSE_loss: 0.0514268012616095\n",
      "Epoch number: 57000 MSE_loss: 0.05125271196159786\n",
      "Epoch number: 57200 MSE_loss: 0.05107965984126602\n",
      "Epoch number: 57400 MSE_loss: 0.05090763630289976\n",
      "Epoch number: 57600 MSE_loss: 0.0507366328377473\n",
      "Epoch number: 57800 MSE_loss: 0.05056664102495471\n",
      "Epoch number: 58000 MSE_loss: 0.050397652530513766\n",
      "Epoch number: 58200 MSE_loss: 0.050229659106224546\n",
      "Epoch number: 58400 MSE_loss: 0.050062652588670764\n",
      "Epoch number: 58600 MSE_loss: 0.0498966248982086\n",
      "Epoch number: 58800 MSE_loss: 0.0497315680379686\n",
      "Epoch number: 59000 MSE_loss: 0.04956747409287038\n",
      "Epoch number: 59200 MSE_loss: 0.04940433522864996\n",
      "Epoch number: 59400 MSE_loss: 0.049242143690899944\n",
      "Epoch number: 59600 MSE_loss: 0.04908089180412186\n",
      "Epoch number: 59800 MSE_loss: 0.048920571970790805\n",
      "Epoch number: 60000 MSE_loss: 0.04876117667043234\n",
      "Epoch number: 60200 MSE_loss: 0.04860269845871101\n",
      "Epoch number: 60400 MSE_loss: 0.04844512996653081\n",
      "Epoch number: 60600 MSE_loss: 0.04828846389914734\n",
      "Epoch number: 60800 MSE_loss: 0.04813269303529113\n",
      "Epoch number: 61000 MSE_loss: 0.04797781022630263\n",
      "Epoch number: 61200 MSE_loss: 0.04782380839527823\n",
      "Epoch number: 61400 MSE_loss: 0.04767068053622716\n",
      "Epoch number: 61600 MSE_loss: 0.04751841971323945\n",
      "Epoch number: 61800 MSE_loss: 0.04736701905966446\n",
      "Epoch number: 62000 MSE_loss: 0.047216471777300296\n",
      "Epoch number: 62200 MSE_loss: 0.04706677113559334\n",
      "Epoch number: 62400 MSE_loss: 0.04691791047084811\n",
      "Epoch number: 62600 MSE_loss: 0.046769883185447614\n",
      "Epoch number: 62800 MSE_loss: 0.04662268274708357\n",
      "Epoch number: 63000 MSE_loss: 0.04647630268799629\n",
      "Epoch number: 63200 MSE_loss: 0.04633073660422471\n",
      "Epoch number: 63400 MSE_loss: 0.046185978154866025\n",
      "Epoch number: 63600 MSE_loss: 0.04604202106134438\n",
      "Epoch number: 63800 MSE_loss: 0.04589885910668981\n",
      "Epoch number: 64000 MSE_loss: 0.04575648613482556\n",
      "Epoch number: 64200 MSE_loss: 0.04561489604986518\n",
      "Epoch number: 64400 MSE_loss: 0.04547408281541818\n",
      "Epoch number: 64600 MSE_loss: 0.04533404045390504\n",
      "Epoch number: 64800 MSE_loss: 0.045194763045880694\n",
      "Epoch number: 65000 MSE_loss: 0.04505624472936671\n",
      "Epoch number: 65200 MSE_loss: 0.04491847969919211\n",
      "Epoch number: 65400 MSE_loss: 0.04478146220634265\n",
      "Epoch number: 65600 MSE_loss: 0.04464518655731834\n",
      "Epoch number: 65800 MSE_loss: 0.04450964711349901\n",
      "Epoch number: 66000 MSE_loss: 0.04437483829051864\n",
      "Epoch number: 66200 MSE_loss: 0.044240754557646596\n",
      "Epoch number: 66400 MSE_loss: 0.04410739043717779\n",
      "Epoch number: 66600 MSE_loss: 0.04397474050383011\n",
      "Epoch number: 66800 MSE_loss: 0.0438427993841495\n",
      "Epoch number: 67000 MSE_loss: 0.0437115617559228\n",
      "Epoch number: 67200 MSE_loss: 0.043581022347598114\n",
      "Epoch number: 67400 MSE_loss: 0.04345117593771205\n",
      "Epoch number: 67600 MSE_loss: 0.04332201735432487\n",
      "Epoch number: 67800 MSE_loss: 0.04319354147446259\n",
      "Epoch number: 68000 MSE_loss: 0.04306574322356577\n",
      "Epoch number: 68200 MSE_loss: 0.04293861757494585\n",
      "Epoch number: 68400 MSE_loss: 0.0428121595492481\n",
      "Epoch number: 68600 MSE_loss: 0.0426863642139213\n",
      "Epoch number: 68800 MSE_loss: 0.04256122668269447\n",
      "Epoch number: 69000 MSE_loss: 0.04243674211505961\n",
      "Epoch number: 69200 MSE_loss: 0.042312905715761434\n",
      "Epoch number: 69400 MSE_loss: 0.04218971273429356\n",
      "Epoch number: 69600 MSE_loss: 0.04206715846440066\n",
      "Epoch number: 69800 MSE_loss: 0.041945238243587195\n",
      "Epoch number: 70000 MSE_loss: 0.041823947452632376\n",
      "Epoch number: 70200 MSE_loss: 0.041703281515110845\n",
      "Epoch number: 70400 MSE_loss: 0.04158323589691989\n",
      "Epoch number: 70600 MSE_loss: 0.041463806105811844\n",
      "Epoch number: 70800 MSE_loss: 0.041344987690933276\n",
      "Epoch number: 71000 MSE_loss: 0.0412267762423692\n",
      "Epoch number: 71200 MSE_loss: 0.04110916739069328\n",
      "Epoch number: 71400 MSE_loss: 0.040992156806523704\n",
      "Epoch number: 71600 MSE_loss: 0.04087574020008433\n",
      "Epoch number: 71800 MSE_loss: 0.040759913320771717\n",
      "Epoch number: 72000 MSE_loss: 0.04064467195672702\n",
      "Epoch number: 72200 MSE_loss: 0.04053001193441374\n",
      "Epoch number: 72400 MSE_loss: 0.040415929118200335\n",
      "Epoch number: 72600 MSE_loss: 0.04030241940994814\n",
      "Epoch number: 72800 MSE_loss: 0.040189478748604464\n",
      "Epoch number: 73000 MSE_loss: 0.04007710310980064\n",
      "Epoch number: 73200 MSE_loss: 0.039965288505455006\n",
      "Epoch number: 73400 MSE_loss: 0.039854030983380855\n",
      "Epoch number: 73600 MSE_loss: 0.0397433266268994\n",
      "Epoch number: 73800 MSE_loss: 0.039633171554456934\n",
      "Epoch number: 74000 MSE_loss: 0.039523561919247434\n",
      "Epoch number: 74200 MSE_loss: 0.03941449390883937\n",
      "Epoch number: 74400 MSE_loss: 0.03930596374480705\n",
      "Epoch number: 74600 MSE_loss: 0.039197967682366885\n",
      "Epoch number: 74800 MSE_loss: 0.039090502010017555\n",
      "Epoch number: 75000 MSE_loss: 0.03898356304918515\n",
      "Epoch number: 75200 MSE_loss: 0.0388771471538721\n",
      "Epoch number: 75400 MSE_loss: 0.03877125071031096\n",
      "Epoch number: 75600 MSE_loss: 0.0386658701366219\n",
      "Epoch number: 75800 MSE_loss: 0.03856100188247464\n",
      "Epoch number: 76000 MSE_loss: 0.038456642428754476\n",
      "Epoch number: 76200 MSE_loss: 0.03835278828723239\n",
      "Epoch number: 76400 MSE_loss: 0.0382494360002389\n",
      "Epoch number: 76600 MSE_loss: 0.03814658214034238\n",
      "Epoch number: 76800 MSE_loss: 0.03804422331003065\n",
      "Epoch number: 77000 MSE_loss: 0.037942356141397085\n",
      "Epoch number: 77200 MSE_loss: 0.03784097729582973\n",
      "Epoch number: 77400 MSE_loss: 0.037740083463705026\n",
      "Epoch number: 77600 MSE_loss: 0.03763967136408434\n",
      "Epoch number: 77800 MSE_loss: 0.037539737744414985\n",
      "Epoch number: 78000 MSE_loss: 0.03744027938023428\n",
      "Epoch number: 78200 MSE_loss: 0.03734129307487757\n",
      "Epoch number: 78400 MSE_loss: 0.03724277565918886\n",
      "Epoch number: 78600 MSE_loss: 0.037144723991236334\n",
      "Epoch number: 78800 MSE_loss: 0.03704713495603001\n",
      "Epoch number: 79000 MSE_loss: 0.036950005465243296\n",
      "Epoch number: 79200 MSE_loss: 0.036853332456938066\n",
      "Epoch number: 79400 MSE_loss: 0.03675711289529246\n",
      "Epoch number: 79600 MSE_loss: 0.03666134377033243\n",
      "Epoch number: 79800 MSE_loss: 0.036566022097666157\n",
      "Epoch number: 80000 MSE_loss: 0.036471144918221794\n",
      "Epoch number: 80200 MSE_loss: 0.03637670929798823\n",
      "Epoch number: 80400 MSE_loss: 0.03628271232775892\n",
      "Epoch number: 80600 MSE_loss: 0.03618915112287869\n",
      "Epoch number: 80800 MSE_loss: 0.03609602282299378\n",
      "Epoch number: 81000 MSE_loss: 0.03600332459180447\n",
      "Epoch number: 81200 MSE_loss: 0.03591105361682078\n",
      "Epoch number: 81400 MSE_loss: 0.03581920710912135\n",
      "Epoch number: 81600 MSE_loss: 0.03572778230311444\n",
      "Epoch number: 81800 MSE_loss: 0.035636776456302505\n",
      "Epoch number: 82000 MSE_loss: 0.03554618684904878\n",
      "Epoch number: 82200 MSE_loss: 0.03545601078434743\n",
      "Epoch number: 82400 MSE_loss: 0.03536624558759551\n",
      "Epoch number: 82600 MSE_loss: 0.035276888606368205\n",
      "Epoch number: 82800 MSE_loss: 0.035187937210196536\n",
      "Epoch number: 83000 MSE_loss: 0.03509938879034738\n",
      "Epoch number: 83200 MSE_loss: 0.03501124075960657\n",
      "Epoch number: 83400 MSE_loss: 0.03492349055206378\n",
      "Epoch number: 83600 MSE_loss: 0.034836135622900555\n",
      "Epoch number: 83800 MSE_loss: 0.034749173448180654\n",
      "Epoch number: 84000 MSE_loss: 0.03466260152464248\n",
      "Epoch number: 84200 MSE_loss: 0.03457641736949422\n",
      "Epoch number: 84400 MSE_loss: 0.03449061852021124\n",
      "Epoch number: 84600 MSE_loss: 0.0344052025343359\n",
      "Epoch number: 84800 MSE_loss: 0.03432016698927931\n",
      "Epoch number: 85000 MSE_loss: 0.034235509482126066\n",
      "Epoch number: 85200 MSE_loss: 0.03415122762944043\n",
      "Epoch number: 85400 MSE_loss: 0.03406731906707512\n",
      "Epoch number: 85600 MSE_loss: 0.03398378144998246\n",
      "Epoch number: 85800 MSE_loss: 0.03390061245202719\n",
      "Epoch number: 86000 MSE_loss: 0.03381780976580184\n",
      "Epoch number: 86200 MSE_loss: 0.03373537110244403\n",
      "Epoch number: 86400 MSE_loss: 0.03365329419145583\n",
      "Epoch number: 86600 MSE_loss: 0.03357157678052512\n",
      "Epoch number: 86800 MSE_loss: 0.03349021663534916\n",
      "Epoch number: 87000 MSE_loss: 0.03340921153945992\n",
      "Epoch number: 87200 MSE_loss: 0.033328559294051624\n",
      "Epoch number: 87400 MSE_loss: 0.03324825771780999\n",
      "Epoch number: 87600 MSE_loss: 0.03316830464674357\n",
      "Epoch number: 87800 MSE_loss: 0.03308869793401691\n",
      "Epoch number: 88000 MSE_loss: 0.0330094354497855\n",
      "Epoch number: 88200 MSE_loss: 0.032930515081032764\n",
      "Epoch number: 88400 MSE_loss: 0.03285193473140887\n",
      "Epoch number: 88600 MSE_loss: 0.0327736923210711\n",
      "Epoch number: 88800 MSE_loss: 0.03269578578652602\n",
      "Epoch number: 89000 MSE_loss: 0.032618213080473864\n",
      "Epoch number: 89200 MSE_loss: 0.032540972171654026\n",
      "Epoch number: 89400 MSE_loss: 0.032464061044692745\n",
      "Epoch number: 89600 MSE_loss: 0.03238747769995223\n",
      "Epoch number: 89800 MSE_loss: 0.032311220153381476\n",
      "Epoch number: 90000 MSE_loss: 0.032235286436368756\n",
      "Epoch number: 90200 MSE_loss: 0.03215967459559584\n",
      "Epoch number: 90400 MSE_loss: 0.03208438269289366\n",
      "Epoch number: 90600 MSE_loss: 0.03200940880509977\n",
      "Epoch number: 90800 MSE_loss: 0.03193475102391705\n",
      "Epoch number: 91000 MSE_loss: 0.03186040745577428\n",
      "Epoch number: 91200 MSE_loss: 0.031786376221688124\n",
      "Epoch number: 91400 MSE_loss: 0.031712655457126565\n",
      "Epoch number: 91600 MSE_loss: 0.03163924331187387\n",
      "Epoch number: 91800 MSE_loss: 0.03156613794989718\n",
      "Epoch number: 92000 MSE_loss: 0.031493337549214184\n",
      "Epoch number: 92200 MSE_loss: 0.031420840301762676\n",
      "Epoch number: 92400 MSE_loss: 0.031348644413271155\n",
      "Epoch number: 92600 MSE_loss: 0.03127674810313116\n",
      "Epoch number: 92800 MSE_loss: 0.03120514960427063\n",
      "Epoch number: 93000 MSE_loss: 0.031133847163029013\n",
      "Epoch number: 93200 MSE_loss: 0.031062839039033435\n",
      "Epoch number: 93400 MSE_loss: 0.030992123505076307\n",
      "Epoch number: 93600 MSE_loss: 0.030921698846994205\n",
      "Epoch number: 93800 MSE_loss: 0.03085156336354818\n",
      "Epoch number: 94000 MSE_loss: 0.030781715366305303\n",
      "Epoch number: 94200 MSE_loss: 0.030712153179521185\n",
      "Epoch number: 94400 MSE_loss: 0.030642875140024423\n",
      "Epoch number: 94600 MSE_loss: 0.03057387959710155\n",
      "Epoch number: 94800 MSE_loss: 0.030505164912383646\n",
      "Epoch number: 95000 MSE_loss: 0.03043672945973407\n",
      "Epoch number: 95200 MSE_loss: 0.030368571625137387\n",
      "Epoch number: 95400 MSE_loss: 0.030300689806589304\n",
      "Epoch number: 95600 MSE_loss: 0.03023308241398814\n",
      "Epoch number: 95800 MSE_loss: 0.030165747869027038\n",
      "Epoch number: 96000 MSE_loss: 0.030098684605087623\n",
      "Epoch number: 96200 MSE_loss: 0.030031891067134678\n",
      "Epoch number: 96400 MSE_loss: 0.029965365711611913\n",
      "Epoch number: 96600 MSE_loss: 0.029899107006338733\n",
      "Epoch number: 96800 MSE_loss: 0.029833113430408453\n",
      "Epoch number: 97000 MSE_loss: 0.029767383474086986\n",
      "Epoch number: 97200 MSE_loss: 0.029701915638713255\n",
      "Epoch number: 97400 MSE_loss: 0.029636708436600167\n",
      "Epoch number: 97600 MSE_loss: 0.029571760390936796\n",
      "Epoch number: 97800 MSE_loss: 0.029507070035691706\n",
      "Epoch number: 98000 MSE_loss: 0.029442635915517018\n",
      "Epoch number: 98200 MSE_loss: 0.029378456585653707\n",
      "Epoch number: 98400 MSE_loss: 0.029314530611837742\n",
      "Epoch number: 98600 MSE_loss: 0.029250856570207224\n",
      "Epoch number: 98800 MSE_loss: 0.029187433047210592\n",
      "Epoch number: 99000 MSE_loss: 0.029124258639515666\n",
      "Epoch number: 99200 MSE_loss: 0.029061331953919623\n",
      "Epoch number: 99400 MSE_loss: 0.028998651607260142\n",
      "Epoch number: 99600 MSE_loss: 0.028936216226326927\n",
      "Epoch number: 99800 MSE_loss: 0.028874024447774798\n",
      "Epoch number: 100000 MSE_loss: 0.02881207491803715\n",
      "Epoch number: 100200 MSE_loss: 0.028750366293240653\n",
      "Epoch number: 100400 MSE_loss: 0.028688897239120725\n",
      "Epoch number: 100600 MSE_loss: 0.02862766643093753\n",
      "Epoch number: 100800 MSE_loss: 0.028566672553393502\n",
      "Epoch number: 101000 MSE_loss: 0.028505914300551122\n",
      "Epoch number: 101200 MSE_loss: 0.028445390375751816\n",
      "Epoch number: 101400 MSE_loss: 0.028385099491535577\n",
      "Epoch number: 101600 MSE_loss: 0.028325040369561558\n",
      "Epoch number: 101800 MSE_loss: 0.02826521174052918\n",
      "Epoch number: 102000 MSE_loss: 0.028205612344100354\n",
      "Epoch number: 102200 MSE_loss: 0.028146240928822352\n",
      "Epoch number: 102400 MSE_loss: 0.028087096252051368\n",
      "Epoch number: 102600 MSE_loss: 0.028028177079876912\n",
      "Epoch number: 102800 MSE_loss: 0.02796948218704723\n",
      "Epoch number: 103000 MSE_loss: 0.02791101035689493\n",
      "Epoch number: 103200 MSE_loss: 0.027852760381263764\n",
      "Epoch number: 103400 MSE_loss: 0.0277947310604362\n",
      "Epoch number: 103600 MSE_loss: 0.027736921203061402\n",
      "Epoch number: 103800 MSE_loss: 0.027679329626084114\n",
      "Epoch number: 104000 MSE_loss: 0.027621955154674176\n",
      "Epoch number: 104200 MSE_loss: 0.027564796622156912\n",
      "Epoch number: 104400 MSE_loss: 0.02750785286994409\n",
      "Epoch number: 104600 MSE_loss: 0.027451122747465417\n",
      "Epoch number: 104800 MSE_loss: 0.027394605112101077\n",
      "Epoch number: 105000 MSE_loss: 0.02733829882911464\n",
      "Epoch number: 105200 MSE_loss: 0.02728220277158665\n",
      "Epoch number: 105400 MSE_loss: 0.0272263158203491\n",
      "Epoch number: 105600 MSE_loss: 0.027170636863920294\n",
      "Epoch number: 105800 MSE_loss: 0.027115164798440422\n",
      "Epoch number: 106000 MSE_loss: 0.027059898527608\n",
      "Epoch number: 106200 MSE_loss: 0.027004836962616602\n",
      "Epoch number: 106400 MSE_loss: 0.026949979022092233\n",
      "Epoch number: 106600 MSE_loss: 0.026895323632031665\n",
      "Epoch number: 106800 MSE_loss: 0.026840869725741078\n",
      "Epoch number: 107000 MSE_loss: 0.02678661624377529\n",
      "Epoch number: 107200 MSE_loss: 0.026732562133877685\n",
      "Epoch number: 107400 MSE_loss: 0.026678706350920817\n",
      "Epoch number: 107600 MSE_loss: 0.02662504785684731\n",
      "Epoch number: 107800 MSE_loss: 0.026571585620611616\n",
      "Epoch number: 108000 MSE_loss: 0.026518318618122122\n",
      "Epoch number: 108200 MSE_loss: 0.02646524583218406\n",
      "Epoch number: 108400 MSE_loss: 0.026412366252442716\n",
      "Epoch number: 108600 MSE_loss: 0.026359678875327346\n",
      "Epoch number: 108800 MSE_loss: 0.026307182703995513\n",
      "Epoch number: 109000 MSE_loss: 0.026254876748278106\n",
      "Epoch number: 109200 MSE_loss: 0.026202760024624818\n",
      "Epoch number: 109400 MSE_loss: 0.026150831556050125\n",
      "Epoch number: 109600 MSE_loss: 0.02609909037207983\n",
      "Epoch number: 109800 MSE_loss: 0.026047535508698025\n",
      "Epoch number: 110000 MSE_loss: 0.025996166008294696\n",
      "Epoch number: 110200 MSE_loss: 0.025944980919613742\n",
      "Epoch number: 110400 MSE_loss: 0.025893979297701552\n",
      "Epoch number: 110600 MSE_loss: 0.025843160203855898\n",
      "Epoch number: 110800 MSE_loss: 0.025792522705575655\n",
      "Epoch number: 111000 MSE_loss: 0.025742065876510595\n",
      "Epoch number: 111200 MSE_loss: 0.02569178879641199\n",
      "Epoch number: 111400 MSE_loss: 0.02564169055108341\n",
      "Epoch number: 111600 MSE_loss: 0.02559177023233226\n",
      "Epoch number: 111800 MSE_loss: 0.025542026937921597\n",
      "Epoch number: 112000 MSE_loss: 0.02549245977152237\n",
      "Epoch number: 112200 MSE_loss: 0.02544306784266631\n",
      "Epoch number: 112400 MSE_loss: 0.025393850266699058\n",
      "Epoch number: 112600 MSE_loss: 0.02534480616473374\n",
      "Epoch number: 112800 MSE_loss: 0.02529593466360529\n",
      "Epoch number: 113000 MSE_loss: 0.02524723489582466\n",
      "Epoch number: 113200 MSE_loss: 0.025198705999533947\n",
      "Epoch number: 113400 MSE_loss: 0.025150347118461702\n",
      "Epoch number: 113600 MSE_loss: 0.025102157401878766\n",
      "Epoch number: 113800 MSE_loss: 0.025054136004554306\n",
      "Epoch number: 114000 MSE_loss: 0.02500628208671271\n",
      "Epoch number: 114200 MSE_loss: 0.024958594813990126\n",
      "Epoch number: 114400 MSE_loss: 0.02491107335739225\n",
      "Epoch number: 114600 MSE_loss: 0.024863716893252074\n",
      "Epoch number: 114800 MSE_loss: 0.02481652460318783\n",
      "Epoch number: 115000 MSE_loss: 0.02476949567406188\n",
      "Epoch number: 115200 MSE_loss: 0.024722629297939364\n",
      "Epoch number: 115400 MSE_loss: 0.024675924672047767\n",
      "Epoch number: 115600 MSE_loss: 0.024629380998736516\n",
      "Epoch number: 115800 MSE_loss: 0.02458299748543703\n",
      "Epoch number: 116000 MSE_loss: 0.02453677334462334\n",
      "Epoch number: 116200 MSE_loss: 0.024490707793772614\n",
      "Epoch number: 116400 MSE_loss: 0.02444480005532658\n",
      "Epoch number: 116600 MSE_loss: 0.02439904935665281\n",
      "Epoch number: 116800 MSE_loss: 0.02435345493000689\n",
      "Epoch number: 117000 MSE_loss: 0.024308016012494427\n",
      "Epoch number: 117200 MSE_loss: 0.024262731846033548\n",
      "Epoch number: 117400 MSE_loss: 0.024217601677317974\n",
      "Epoch number: 117600 MSE_loss: 0.024172624757780304\n",
      "Epoch number: 117800 MSE_loss: 0.02412780034355531\n",
      "Epoch number: 118000 MSE_loss: 0.02408312769544412\n",
      "Epoch number: 118200 MSE_loss: 0.02403860607887832\n",
      "Epoch number: 118400 MSE_loss: 0.02399423476388457\n",
      "Epoch number: 118600 MSE_loss: 0.0239500130250493\n",
      "Epoch number: 118800 MSE_loss: 0.023905940141484228\n",
      "Epoch number: 119000 MSE_loss: 0.02386201539679152\n",
      "Epoch number: 119200 MSE_loss: 0.0238182380790299\n",
      "Epoch number: 119400 MSE_loss: 0.02377460748068046\n",
      "Epoch number: 119600 MSE_loss: 0.02373112289861334\n",
      "Epoch number: 119800 MSE_loss: 0.023687783634054386\n",
      "Epoch number: 120000 MSE_loss: 0.023644588992552163\n",
      "Epoch number: 120200 MSE_loss: 0.02360153828394516\n",
      "Epoch number: 120400 MSE_loss: 0.02355863082232949\n",
      "Epoch number: 120600 MSE_loss: 0.02351586592602691\n",
      "Epoch number: 120800 MSE_loss: 0.02347324291755278\n",
      "Epoch number: 121000 MSE_loss: 0.023430761123584595\n",
      "Epoch number: 121200 MSE_loss: 0.023388419874930854\n",
      "Epoch number: 121400 MSE_loss: 0.023346218506499945\n",
      "Epoch number: 121600 MSE_loss: 0.02330415635726943\n",
      "Epoch number: 121800 MSE_loss: 0.023262232770255676\n",
      "Epoch number: 122000 MSE_loss: 0.023220447092483748\n",
      "Epoch number: 122200 MSE_loss: 0.023178798674957257\n",
      "Epoch number: 122400 MSE_loss: 0.023137286872628986\n",
      "Epoch number: 122600 MSE_loss: 0.023095911044371514\n",
      "Epoch number: 122800 MSE_loss: 0.023054670552947788\n",
      "Epoch number: 123000 MSE_loss: 0.023013564764982647\n",
      "Epoch number: 123200 MSE_loss: 0.022972593050933934\n",
      "Epoch number: 123400 MSE_loss: 0.022931754785064245\n",
      "Epoch number: 123600 MSE_loss: 0.022891049345412823\n",
      "Epoch number: 123800 MSE_loss: 0.022850476113767643\n",
      "Epoch number: 124000 MSE_loss: 0.02281003447563773\n",
      "Epoch number: 124200 MSE_loss: 0.022769723820225962\n",
      "Epoch number: 124400 MSE_loss: 0.02272954354040177\n",
      "Epoch number: 124600 MSE_loss: 0.02268949303267422\n",
      "Epoch number: 124800 MSE_loss: 0.022649571697165385\n",
      "Epoch number: 125000 MSE_loss: 0.02260977893758387\n",
      "Epoch number: 125200 MSE_loss: 0.022570114161198584\n",
      "Epoch number: 125400 MSE_loss: 0.022530576778812775\n",
      "Epoch number: 125600 MSE_loss: 0.022491166204738233\n",
      "Epoch number: 125800 MSE_loss: 0.02245188185676977\n",
      "Epoch number: 126000 MSE_loss: 0.022412723156159817\n",
      "Epoch number: 126200 MSE_loss: 0.02237368952759332\n",
      "Epoch number: 126400 MSE_loss: 0.022334780399162976\n",
      "Epoch number: 126600 MSE_loss: 0.022295995202344394\n",
      "Epoch number: 126800 MSE_loss: 0.022257333371971666\n",
      "Epoch number: 127000 MSE_loss: 0.022218794346213166\n",
      "Epoch number: 127200 MSE_loss: 0.022180377566547255\n",
      "Epoch number: 127400 MSE_loss: 0.022142082477738845\n",
      "Epoch number: 127600 MSE_loss: 0.02210390852781534\n",
      "Epoch number: 127800 MSE_loss: 0.022065855168043463\n",
      "Epoch number: 128000 MSE_loss: 0.02202792185290581\n",
      "Epoch number: 128200 MSE_loss: 0.021990108040077977\n",
      "Epoch number: 128400 MSE_loss: 0.021952413190405473\n",
      "Epoch number: 128600 MSE_loss: 0.021914836767881297\n",
      "Epoch number: 128800 MSE_loss: 0.021877378239623255\n",
      "Epoch number: 129000 MSE_loss: 0.02184003707585187\n",
      "Epoch number: 129200 MSE_loss: 0.02180281274986809\n",
      "Epoch number: 129400 MSE_loss: 0.021765704738031483\n",
      "Epoch number: 129600 MSE_loss: 0.021728712519738475\n",
      "Epoch number: 129800 MSE_loss: 0.021691835577400963\n",
      "Epoch number: 130000 MSE_loss: 0.021655073396424623\n",
      "Epoch number: 130200 MSE_loss: 0.02161842546518801\n",
      "Epoch number: 130400 MSE_loss: 0.021581891275021397\n",
      "Epoch number: 130600 MSE_loss: 0.021545470320185976\n",
      "Epoch number: 130800 MSE_loss: 0.02150916209785328\n",
      "Epoch number: 131000 MSE_loss: 0.021472966108084496\n",
      "Epoch number: 131200 MSE_loss: 0.021436881853810343\n",
      "Epoch number: 131400 MSE_loss: 0.021400908840810844\n",
      "Epoch number: 131600 MSE_loss: 0.021365046577695345\n",
      "Epoch number: 131800 MSE_loss: 0.021329294575882698\n",
      "Epoch number: 132000 MSE_loss: 0.02129365234958148\n",
      "Epoch number: 132200 MSE_loss: 0.021258119415770844\n",
      "Epoch number: 132400 MSE_loss: 0.021222695294180766\n",
      "Epoch number: 132600 MSE_loss: 0.021187379507273174\n",
      "Epoch number: 132800 MSE_loss: 0.021152171580222905\n",
      "Epoch number: 133000 MSE_loss: 0.021117071040898508\n",
      "Epoch number: 133200 MSE_loss: 0.021082077419844057\n",
      "Epoch number: 133400 MSE_loss: 0.02104719025026031\n",
      "Epoch number: 133600 MSE_loss: 0.021012409067986227\n",
      "Epoch number: 133800 MSE_loss: 0.02097773341148089\n",
      "Epoch number: 134000 MSE_loss: 0.020943162821805444\n",
      "Epoch number: 134200 MSE_loss: 0.020908696842604987\n",
      "Epoch number: 134400 MSE_loss: 0.020874335020090826\n",
      "Epoch number: 134600 MSE_loss: 0.020840076903022853\n",
      "Epoch number: 134800 MSE_loss: 0.020805922042691976\n",
      "Epoch number: 135000 MSE_loss: 0.020771869992902748\n",
      "Epoch number: 135200 MSE_loss: 0.020737920309956182\n",
      "Epoch number: 135400 MSE_loss: 0.020704072552632707\n",
      "Epoch number: 135600 MSE_loss: 0.02067032628217502\n",
      "Epoch number: 135800 MSE_loss: 0.02063668106227134\n",
      "Epoch number: 136000 MSE_loss: 0.02060313645903888\n",
      "Epoch number: 136200 MSE_loss: 0.020569692041007146\n",
      "Epoch number: 136400 MSE_loss: 0.02053634737910148\n",
      "Epoch number: 136600 MSE_loss: 0.020503102046626972\n",
      "Epoch number: 136800 MSE_loss: 0.020469955619252057\n",
      "Epoch number: 137000 MSE_loss: 0.0204369076749927\n",
      "Epoch number: 137200 MSE_loss: 0.020403957794196386\n",
      "Epoch number: 137400 MSE_loss: 0.020371105559526356\n",
      "Epoch number: 137600 MSE_loss: 0.02033835055594602\n",
      "Epoch number: 137800 MSE_loss: 0.020305692370703433\n",
      "Epoch number: 138000 MSE_loss: 0.020273130593315813\n",
      "Epoch number: 138200 MSE_loss: 0.020240664815554366\n",
      "Epoch number: 138400 MSE_loss: 0.020208294631429084\n",
      "Epoch number: 138600 MSE_loss: 0.020176019637173737\n",
      "Epoch number: 138800 MSE_loss: 0.020143839431231016\n",
      "Epoch number: 139000 MSE_loss: 0.02011175361423749\n",
      "Epoch number: 139200 MSE_loss: 0.020079761789009384\n",
      "Epoch number: 139400 MSE_loss: 0.020047863560527523\n",
      "Epoch number: 139600 MSE_loss: 0.02001605853592324\n",
      "Epoch number: 139800 MSE_loss: 0.01998434632446391\n",
      "Epoch number: 140000 MSE_loss: 0.019952726537538656\n",
      "Epoch number: 140200 MSE_loss: 0.019921198788644404\n",
      "Epoch number: 140400 MSE_loss: 0.019889762693371782\n",
      "Epoch number: 140600 MSE_loss: 0.019858417869391326\n",
      "Epoch number: 140800 MSE_loss: 0.019827163936439542\n",
      "Epoch number: 141000 MSE_loss: 0.019796000516305357\n",
      "Epoch number: 141200 MSE_loss: 0.019764927232816573\n",
      "Epoch number: 141400 MSE_loss: 0.019733943711826348\n",
      "Epoch number: 141600 MSE_loss: 0.019703049581199825\n",
      "Epoch number: 141800 MSE_loss: 0.01967224447080093\n",
      "Epoch number: 142000 MSE_loss: 0.019641528012479283\n",
      "Epoch number: 142200 MSE_loss: 0.01961089984005702\n",
      "Epoch number: 142400 MSE_loss: 0.01958035958931591\n",
      "Epoch number: 142600 MSE_loss: 0.019549906897984564\n",
      "Epoch number: 142800 MSE_loss: 0.019519541405725656\n",
      "Epoch number: 143000 MSE_loss: 0.019489262754123198\n",
      "Epoch number: 143200 MSE_loss: 0.019459070586670204\n",
      "Epoch number: 143400 MSE_loss: 0.019428964548755947\n",
      "Epoch number: 143600 MSE_loss: 0.019398944287653964\n",
      "Epoch number: 143800 MSE_loss: 0.01936900945250937\n",
      "Epoch number: 144000 MSE_loss: 0.019339159694327077\n",
      "Epoch number: 144200 MSE_loss: 0.01930939466595952\n",
      "Epoch number: 144400 MSE_loss: 0.019279714022094756\n",
      "Epoch number: 144600 MSE_loss: 0.019250117419244483\n",
      "Epoch number: 144800 MSE_loss: 0.0192206045157324\n",
      "Epoch number: 145000 MSE_loss: 0.019191174971682316\n",
      "Epoch number: 145200 MSE_loss: 0.0191618284490067\n",
      "Epoch number: 145400 MSE_loss: 0.01913256461139507\n",
      "Epoch number: 145600 MSE_loss: 0.01910338312430256\n",
      "Epoch number: 145800 MSE_loss: 0.019074283654938514\n",
      "Epoch number: 146000 MSE_loss: 0.019045265872255326\n",
      "Epoch number: 146200 MSE_loss: 0.019016329446937225\n",
      "Epoch number: 146400 MSE_loss: 0.018987474051389182\n",
      "Epoch number: 146600 MSE_loss: 0.018958699359725814\n",
      "Epoch number: 146800 MSE_loss: 0.01893000504776055\n",
      "Epoch number: 147000 MSE_loss: 0.018901390792994757\n",
      "Epoch number: 147200 MSE_loss: 0.018872856274606973\n",
      "Epoch number: 147400 MSE_loss: 0.01884440117344222\n",
      "Epoch number: 147600 MSE_loss: 0.018816025172001396\n",
      "Epoch number: 147800 MSE_loss: 0.018787727954430764\n",
      "Epoch number: 148000 MSE_loss: 0.018759509206511548\n",
      "Epoch number: 148200 MSE_loss: 0.018731368615649443\n",
      "Epoch number: 148400 MSE_loss: 0.018703305870864474\n",
      "Epoch number: 148600 MSE_loss: 0.018675320662780635\n",
      "Epoch number: 148800 MSE_loss: 0.01864741268361592\n",
      "Epoch number: 149000 MSE_loss: 0.018619581627172144\n",
      "Epoch number: 149200 MSE_loss: 0.018591827188824923\n",
      "Epoch number: 149400 MSE_loss: 0.018564149065513937\n",
      "Epoch number: 149600 MSE_loss: 0.018536546955732924\n",
      "Epoch number: 149800 MSE_loss: 0.018509020559520018\n",
      "Epoch number: 150000 MSE_loss: 0.018481569578448025\n",
      "Epoch number: 150200 MSE_loss: 0.018454193715614787\n",
      "Epoch number: 150400 MSE_loss: 0.01842689267563358\n",
      "Epoch number: 150600 MSE_loss: 0.01839966616462381\n",
      "Epoch number: 150800 MSE_loss: 0.018372513890201437\n",
      "Epoch number: 151000 MSE_loss: 0.018345435561469725\n",
      "Epoch number: 151200 MSE_loss: 0.018318430889009956\n",
      "Epoch number: 151400 MSE_loss: 0.01829149958487223\n",
      "Epoch number: 151600 MSE_loss: 0.01826464136256628\n",
      "Epoch number: 151800 MSE_loss: 0.018237855937052457\n",
      "Epoch number: 152000 MSE_loss: 0.018211143024732746\n",
      "Epoch number: 152200 MSE_loss: 0.018184502343441836\n",
      "Epoch number: 152400 MSE_loss: 0.01815793361243816\n",
      "Epoch number: 152600 MSE_loss: 0.01813143655239523\n",
      "Epoch number: 152800 MSE_loss: 0.018105010885392778\n",
      "Epoch number: 153000 MSE_loss: 0.018078656334908155\n",
      "Epoch number: 153200 MSE_loss: 0.01805237262580765\n",
      "Epoch number: 153400 MSE_loss: 0.01802615948433806\n",
      "Epoch number: 153600 MSE_loss: 0.018000016638118146\n",
      "Epoch number: 153800 MSE_loss: 0.017973943816130027\n",
      "Epoch number: 154000 MSE_loss: 0.01794794074871113\n",
      "Epoch number: 154200 MSE_loss: 0.0179220071675457\n",
      "Epoch number: 154400 MSE_loss: 0.017896142805656587\n",
      "Epoch number: 154600 MSE_loss: 0.017870347397397093\n",
      "Epoch number: 154800 MSE_loss: 0.017844620678442783\n",
      "Epoch number: 155000 MSE_loss: 0.017818962385783496\n",
      "Epoch number: 155200 MSE_loss: 0.017793372257715313\n",
      "Epoch number: 155400 MSE_loss: 0.0177678500338325\n",
      "Epoch number: 155600 MSE_loss: 0.01774239545501981\n",
      "Epoch number: 155800 MSE_loss: 0.01771700826344449\n",
      "Epoch number: 156000 MSE_loss: 0.017691688202548628\n",
      "Epoch number: 156200 MSE_loss: 0.017666435017041317\n",
      "Epoch number: 156400 MSE_loss: 0.017641248452891025\n",
      "Epoch number: 156600 MSE_loss: 0.017616128257317964\n",
      "Epoch number: 156800 MSE_loss: 0.017591074178786616\n",
      "Epoch number: 157000 MSE_loss: 0.017566085966998088\n",
      "Epoch number: 157200 MSE_loss: 0.017541163372882754\n",
      "Epoch number: 157400 MSE_loss: 0.017516306148592863\n",
      "Epoch number: 157600 MSE_loss: 0.017491514047495178\n",
      "Epoch number: 157800 MSE_loss: 0.017466786824163662\n",
      "Epoch number: 158000 MSE_loss: 0.017442124234372202\n",
      "Epoch number: 158200 MSE_loss: 0.01741752603508753\n",
      "Epoch number: 158400 MSE_loss: 0.017392991984461936\n",
      "Epoch number: 158600 MSE_loss: 0.017368521841826298\n",
      "Epoch number: 158800 MSE_loss: 0.017344115367683013\n",
      "Epoch number: 159000 MSE_loss: 0.017319772323699\n",
      "Epoch number: 159200 MSE_loss: 0.017295492472698723\n",
      "Epoch number: 159400 MSE_loss: 0.017271275578657323\n",
      "Epoch number: 159600 MSE_loss: 0.01724712140669385\n",
      "Epoch number: 159800 MSE_loss: 0.017223029723064352\n",
      "Epoch number: 160000 MSE_loss: 0.01719900029515514\n",
      "Epoch number: 160200 MSE_loss: 0.017175032891476216\n",
      "Epoch number: 160400 MSE_loss: 0.017151127281654504\n",
      "Epoch number: 160600 MSE_loss: 0.017127283236427194\n",
      "Epoch number: 160800 MSE_loss: 0.01710350052763533\n",
      "Epoch number: 161000 MSE_loss: 0.017079778928217233\n",
      "Epoch number: 161200 MSE_loss: 0.017056118212201977\n",
      "Epoch number: 161400 MSE_loss: 0.01703251815470306\n",
      "Epoch number: 161600 MSE_loss: 0.01700897853191194\n",
      "Epoch number: 161800 MSE_loss: 0.01698549912109173\n",
      "Epoch number: 162000 MSE_loss: 0.01696207970057099\n",
      "Epoch number: 162200 MSE_loss: 0.01693872004973726\n",
      "Epoch number: 162400 MSE_loss: 0.01691541994903115\n",
      "Epoch number: 162600 MSE_loss: 0.01689217917993992\n",
      "Epoch number: 162800 MSE_loss: 0.016868997524991543\n",
      "Epoch number: 163000 MSE_loss: 0.016845874767748563\n",
      "Epoch number: 163200 MSE_loss: 0.01682281069280199\n",
      "Epoch number: 163400 MSE_loss: 0.016799805085765406\n",
      "Epoch number: 163600 MSE_loss: 0.016776857733269007\n",
      "Epoch number: 163800 MSE_loss: 0.01675396842295364\n",
      "Epoch number: 164000 MSE_loss: 0.016731136943464925\n",
      "Epoch number: 164200 MSE_loss: 0.016708363084447547\n",
      "Epoch number: 164400 MSE_loss: 0.016685646636539323\n",
      "Epoch number: 164600 MSE_loss: 0.016662987391365593\n",
      "Epoch number: 164800 MSE_loss: 0.016640385141533296\n",
      "Epoch number: 165000 MSE_loss: 0.016617839680625496\n",
      "Epoch number: 165200 MSE_loss: 0.01659535080319577\n",
      "Epoch number: 165400 MSE_loss: 0.016572918304762346\n",
      "Epoch number: 165600 MSE_loss: 0.016550541981802905\n",
      "Epoch number: 165800 MSE_loss: 0.016528221631748812\n",
      "Epoch number: 166000 MSE_loss: 0.016505957052979695\n",
      "Epoch number: 166200 MSE_loss: 0.016483748044818055\n",
      "Epoch number: 166400 MSE_loss: 0.016461594407523834\n",
      "Epoch number: 166600 MSE_loss: 0.016439495942288964\n",
      "Epoch number: 166800 MSE_loss: 0.01641745245123223\n",
      "Epoch number: 167000 MSE_loss: 0.016395463737393715\n",
      "Epoch number: 167200 MSE_loss: 0.016373529604729747\n",
      "Epoch number: 167400 MSE_loss: 0.01635164985810762\n",
      "Epoch number: 167600 MSE_loss: 0.01632982430330029\n",
      "Epoch number: 167800 MSE_loss: 0.016308052746981397\n",
      "Epoch number: 168000 MSE_loss: 0.01628633499672004\n",
      "Epoch number: 168200 MSE_loss: 0.016264670860975675\n",
      "Epoch number: 168400 MSE_loss: 0.01624306014909312\n",
      "Epoch number: 168600 MSE_loss: 0.016221502671297475\n",
      "Epoch number: 168800 MSE_loss: 0.016199998238689308\n",
      "Epoch number: 169000 MSE_loss: 0.01617854666323941\n",
      "Epoch number: 169200 MSE_loss: 0.016157147757784138\n",
      "Epoch number: 169400 MSE_loss: 0.016135801336020363\n",
      "Epoch number: 169600 MSE_loss: 0.016114507212500736\n",
      "Epoch number: 169800 MSE_loss: 0.016093265202628822\n",
      "Epoch number: 170000 MSE_loss: 0.01607207512265424\n",
      "Epoch number: 170200 MSE_loss: 0.01605093678966804\n",
      "Epoch number: 170400 MSE_loss: 0.016029850021597847\n",
      "Epoch number: 170600 MSE_loss: 0.016008814637203304\n",
      "Epoch number: 170800 MSE_loss: 0.015987830456071276\n",
      "Epoch number: 171000 MSE_loss: 0.015966897298611307\n",
      "Epoch number: 171200 MSE_loss: 0.015946014986051013\n",
      "Epoch number: 171400 MSE_loss: 0.015925183340431442\n",
      "Epoch number: 171600 MSE_loss: 0.01590440218460262\n",
      "Epoch number: 171800 MSE_loss: 0.015883671342219057\n",
      "Epoch number: 172000 MSE_loss: 0.015862990637735\n",
      "Epoch number: 172200 MSE_loss: 0.015842359896400423\n",
      "Epoch number: 172400 MSE_loss: 0.01582177894425633\n",
      "Epoch number: 172600 MSE_loss: 0.015801247608130325\n",
      "Epoch number: 172800 MSE_loss: 0.01578076571563235\n",
      "Epoch number: 173000 MSE_loss: 0.015760333095150434\n",
      "Epoch number: 173200 MSE_loss: 0.015739949575846207\n",
      "Epoch number: 173400 MSE_loss: 0.015719614987650714\n",
      "Epoch number: 173600 MSE_loss: 0.015699329161260153\n",
      "Epoch number: 173800 MSE_loss: 0.01567909192813158\n",
      "Epoch number: 174000 MSE_loss: 0.015658903120478924\n",
      "Epoch number: 174200 MSE_loss: 0.015638762571268592\n",
      "Epoch number: 174400 MSE_loss: 0.015618670114215437\n",
      "Epoch number: 174600 MSE_loss: 0.015598625583778612\n",
      "Epoch number: 174800 MSE_loss: 0.0155786288151575\n",
      "Epoch number: 175000 MSE_loss: 0.015558679644287657\n",
      "Epoch number: 175200 MSE_loss: 0.015538777907836763\n",
      "Epoch number: 175400 MSE_loss: 0.015518923443200633\n",
      "Epoch number: 175600 MSE_loss: 0.015499116088499165\n",
      "Epoch number: 175800 MSE_loss: 0.01547935568257254\n",
      "Epoch number: 176000 MSE_loss: 0.015459642064977097\n",
      "Epoch number: 176200 MSE_loss: 0.015439975075981535\n",
      "Epoch number: 176400 MSE_loss: 0.015420354556563131\n",
      "Epoch number: 176600 MSE_loss: 0.015400780348403665\n",
      "Epoch number: 176800 MSE_loss: 0.015381252293885699\n",
      "Epoch number: 177000 MSE_loss: 0.015361770236088855\n",
      "Epoch number: 177200 MSE_loss: 0.015342334018785889\n",
      "Epoch number: 177400 MSE_loss: 0.015322943486438994\n",
      "Epoch number: 177600 MSE_loss: 0.0153035984841961\n",
      "Epoch number: 177800 MSE_loss: 0.015284298857887119\n",
      "Epoch number: 178000 MSE_loss: 0.015265044454020294\n",
      "Epoch number: 178200 MSE_loss: 0.0152458351197785\n",
      "Epoch number: 178400 MSE_loss: 0.01522667070301561\n",
      "Epoch number: 178600 MSE_loss: 0.01520755105225291\n",
      "Epoch number: 178800 MSE_loss: 0.01518847601667551\n",
      "Epoch number: 179000 MSE_loss: 0.015169445446128785\n",
      "Epoch number: 179200 MSE_loss: 0.015150459191114693\n",
      "Epoch number: 179400 MSE_loss: 0.015131517102788485\n",
      "Epoch number: 179600 MSE_loss: 0.015112619032954978\n",
      "Epoch number: 179800 MSE_loss: 0.015093764834065259\n",
      "Epoch number: 180000 MSE_loss: 0.015074954359213072\n",
      "Epoch number: 180200 MSE_loss: 0.015056187462131493\n",
      "Epoch number: 180400 MSE_loss: 0.015037463997189485\n",
      "Epoch number: 180600 MSE_loss: 0.015018783819388487\n",
      "Epoch number: 180800 MSE_loss: 0.01500014678435907\n",
      "Epoch number: 181000 MSE_loss: 0.014981552748357524\n",
      "Epoch number: 181200 MSE_loss: 0.014963001568262635\n",
      "Epoch number: 181400 MSE_loss: 0.014944493101572304\n",
      "Epoch number: 181600 MSE_loss: 0.014926027206400244\n",
      "Epoch number: 181800 MSE_loss: 0.014907603741472798\n",
      "Epoch number: 182000 MSE_loss: 0.014889222566125649\n",
      "Epoch number: 182200 MSE_loss: 0.014870883540300542\n",
      "Epoch number: 182400 MSE_loss: 0.014852586524542152\n",
      "Epoch number: 182600 MSE_loss: 0.01483433137999489\n",
      "Epoch number: 182800 MSE_loss: 0.014816117968399739\n",
      "Epoch number: 183000 MSE_loss: 0.014797946152090966\n",
      "Epoch number: 183200 MSE_loss: 0.014779815793993298\n",
      "Epoch number: 183400 MSE_loss: 0.014761726757618518\n",
      "Epoch number: 183600 MSE_loss: 0.0147436789070625\n",
      "Epoch number: 183800 MSE_loss: 0.01472567210700225\n",
      "Epoch number: 184000 MSE_loss: 0.014707706222692657\n",
      "Epoch number: 184200 MSE_loss: 0.014689781119963566\n",
      "Epoch number: 184400 MSE_loss: 0.014671896665216786\n",
      "Epoch number: 184600 MSE_loss: 0.01465405272542305\n",
      "Epoch number: 184800 MSE_loss: 0.01463624916811907\n",
      "Epoch number: 185000 MSE_loss: 0.014618485861404554\n",
      "Epoch number: 185200 MSE_loss: 0.014600762673939343\n",
      "Epoch number: 185400 MSE_loss: 0.014583079474940355\n",
      "Epoch number: 185600 MSE_loss: 0.0145654361341788\n",
      "Epoch number: 185800 MSE_loss: 0.014547832521977314\n",
      "Epoch number: 186000 MSE_loss: 0.014530268509206965\n",
      "Epoch number: 186200 MSE_loss: 0.014512743967284504\n",
      "Epoch number: 186400 MSE_loss: 0.014495258768169558\n",
      "Epoch number: 186600 MSE_loss: 0.014477812784361756\n",
      "Epoch number: 186800 MSE_loss: 0.014460405888897914\n",
      "Epoch number: 187000 MSE_loss: 0.014443037955349328\n",
      "Epoch number: 187200 MSE_loss: 0.014425708857818914\n",
      "Epoch number: 187400 MSE_loss: 0.014408418470938591\n",
      "Epoch number: 187600 MSE_loss: 0.014391166669866534\n",
      "Epoch number: 187800 MSE_loss: 0.014373953330284364\n",
      "Epoch number: 188000 MSE_loss: 0.014356778328394432\n",
      "Epoch number: 188200 MSE_loss: 0.014339641540917422\n",
      "Epoch number: 188400 MSE_loss: 0.014322542845089237\n",
      "Epoch number: 188600 MSE_loss: 0.014305482118658815\n",
      "Epoch number: 188800 MSE_loss: 0.014288459239885148\n",
      "Epoch number: 189000 MSE_loss: 0.014271474087534852\n",
      "Epoch number: 189200 MSE_loss: 0.014254526540879567\n",
      "Epoch number: 189400 MSE_loss: 0.014237616479693224\n",
      "Epoch number: 189600 MSE_loss: 0.014220743784249678\n",
      "Epoch number: 189800 MSE_loss: 0.014203908335320023\n",
      "Epoch number: 190000 MSE_loss: 0.014187110014170118\n",
      "Epoch number: 190200 MSE_loss: 0.014170348702558023\n",
      "Epoch number: 190400 MSE_loss: 0.014153624282731578\n",
      "Epoch number: 190600 MSE_loss: 0.014136936637425773\n",
      "Epoch number: 190800 MSE_loss: 0.014120285649860398\n",
      "Epoch number: 191000 MSE_loss: 0.014103671203737572\n",
      "Epoch number: 191200 MSE_loss: 0.014087093183239164\n",
      "Epoch number: 191400 MSE_loss: 0.014070551473024537\n",
      "Epoch number: 191600 MSE_loss: 0.014054045958228054\n",
      "Epoch number: 191800 MSE_loss: 0.014037576524456677\n",
      "Epoch number: 192000 MSE_loss: 0.014021143057787564\n",
      "Epoch number: 192200 MSE_loss: 0.014004745444765732\n",
      "Epoch number: 192400 MSE_loss: 0.013988383572401683\n",
      "Epoch number: 192600 MSE_loss: 0.013972057328169052\n",
      "Epoch number: 192800 MSE_loss: 0.013955766600002242\n",
      "Epoch number: 193000 MSE_loss: 0.013939511276294198\n",
      "Epoch number: 193200 MSE_loss: 0.013923291245894032\n",
      "Epoch number: 193400 MSE_loss: 0.013907106398104807\n",
      "Epoch number: 193600 MSE_loss: 0.013890956622681125\n",
      "Epoch number: 193800 MSE_loss: 0.013874841809827029\n",
      "Epoch number: 194000 MSE_loss: 0.013858761850193612\n",
      "Epoch number: 194200 MSE_loss: 0.013842716634876849\n",
      "Epoch number: 194400 MSE_loss: 0.013826706055415403\n",
      "Epoch number: 194600 MSE_loss: 0.013810730003788324\n",
      "Epoch number: 194800 MSE_loss: 0.01379478837241302\n",
      "Epoch number: 195000 MSE_loss: 0.013778881054142841\n",
      "Epoch number: 195200 MSE_loss: 0.013763007942265093\n",
      "Epoch number: 195400 MSE_loss: 0.013747168930498849\n",
      "Epoch number: 195600 MSE_loss: 0.013731363912992704\n",
      "Epoch number: 195800 MSE_loss: 0.013715592784322753\n",
      "Epoch number: 196000 MSE_loss: 0.013699855439490436\n",
      "Epoch number: 196200 MSE_loss: 0.013684151773920428\n",
      "Epoch number: 196400 MSE_loss: 0.013668481683458557\n",
      "Epoch number: 196600 MSE_loss: 0.013652845064369673\n",
      "Epoch number: 196800 MSE_loss: 0.013637241813335602\n",
      "Epoch number: 197000 MSE_loss: 0.013621671827453172\n",
      "Epoch number: 197200 MSE_loss: 0.013606135004232045\n",
      "Epoch number: 197400 MSE_loss: 0.013590631241592703\n",
      "Epoch number: 197600 MSE_loss: 0.013575160437864464\n",
      "Epoch number: 197800 MSE_loss: 0.013559722491783532\n",
      "Epoch number: 198000 MSE_loss: 0.013544317302490857\n",
      "Epoch number: 198200 MSE_loss: 0.013528944769530213\n",
      "Epoch number: 198400 MSE_loss: 0.013513604792846278\n",
      "Epoch number: 198600 MSE_loss: 0.01349829727278259\n",
      "Epoch number: 198800 MSE_loss: 0.013483022110079604\n",
      "Epoch number: 199000 MSE_loss: 0.013467779205872827\n",
      "Epoch number: 199200 MSE_loss: 0.013452568461690753\n",
      "Epoch number: 199400 MSE_loss: 0.013437389779453057\n",
      "Epoch number: 199600 MSE_loss: 0.013422243061468655\n",
      "Epoch number: 199800 MSE_loss: 0.013407128210433846\n",
      "Epoch number: 200000 MSE_loss: 0.0133920451294303\n",
      "Epoch number: 200200 MSE_loss: 0.013376993721923284\n",
      "Epoch number: 200400 MSE_loss: 0.013361973891759837\n",
      "Epoch number: 200600 MSE_loss: 0.013346985543166818\n",
      "Epoch number: 200800 MSE_loss: 0.01333202858074907\n",
      "Epoch number: 201000 MSE_loss: 0.013317102909487705\n",
      "Epoch number: 201200 MSE_loss: 0.013302208434738094\n",
      "Epoch number: 201400 MSE_loss: 0.013287345062228242\n",
      "Epoch number: 201600 MSE_loss: 0.013272512698056878\n",
      "Epoch number: 201800 MSE_loss: 0.013257711248691675\n",
      "Epoch number: 202000 MSE_loss: 0.013242940620967494\n",
      "Epoch number: 202200 MSE_loss: 0.013228200722084596\n",
      "Epoch number: 202400 MSE_loss: 0.013213491459606931\n",
      "Epoch number: 202600 MSE_loss: 0.01319881274146032\n",
      "Epoch number: 202800 MSE_loss: 0.013184164475930715\n",
      "Epoch number: 203000 MSE_loss: 0.013169546571662527\n",
      "Epoch number: 203200 MSE_loss: 0.013154958937656863\n",
      "Epoch number: 203400 MSE_loss: 0.013140401483269821\n",
      "Epoch number: 203600 MSE_loss: 0.013125874118210817\n",
      "Epoch number: 203800 MSE_loss: 0.013111376752540834\n",
      "Epoch number: 204000 MSE_loss: 0.01309690929667083\n",
      "Epoch number: 204200 MSE_loss: 0.013082471661359946\n",
      "Epoch number: 204400 MSE_loss: 0.013068063757713957\n",
      "Epoch number: 204600 MSE_loss: 0.013053685497183568\n",
      "Epoch number: 204800 MSE_loss: 0.013039336791562717\n",
      "Epoch number: 205000 MSE_loss: 0.013025017552987092\n",
      "Epoch number: 205200 MSE_loss: 0.013010727693932337\n",
      "Epoch number: 205400 MSE_loss: 0.012996467127212535\n",
      "Epoch number: 205600 MSE_loss: 0.012982235765978546\n",
      "Epoch number: 205800 MSE_loss: 0.01296803352371651\n",
      "Epoch number: 206000 MSE_loss: 0.01295386031424608\n",
      "Epoch number: 206200 MSE_loss: 0.012939716051719086\n",
      "Epoch number: 206400 MSE_loss: 0.01292560065061763\n",
      "Epoch number: 206600 MSE_loss: 0.012911514025752852\n",
      "Epoch number: 206800 MSE_loss: 0.012897456092263268\n",
      "Epoch number: 207000 MSE_loss: 0.012883426765613076\n",
      "Epoch number: 207200 MSE_loss: 0.012869425961590861\n",
      "Epoch number: 207400 MSE_loss: 0.012855453596307844\n",
      "Epoch number: 207600 MSE_loss: 0.012841509586196571\n",
      "Epoch number: 207800 MSE_loss: 0.012827593848009211\n",
      "Epoch number: 208000 MSE_loss: 0.012813706298816119\n",
      "Epoch number: 208200 MSE_loss: 0.012799846856004493\n",
      "Epoch number: 208400 MSE_loss: 0.012786015437276648\n",
      "Epoch number: 208600 MSE_loss: 0.01277221196064865\n",
      "Epoch number: 208800 MSE_loss: 0.012758436344448848\n",
      "Epoch number: 209000 MSE_loss: 0.01274468850731647\n",
      "Epoch number: 209200 MSE_loss: 0.012730968368200037\n",
      "Epoch number: 209400 MSE_loss: 0.01271727584635602\n",
      "Epoch number: 209600 MSE_loss: 0.012703610861347364\n",
      "Epoch number: 209800 MSE_loss: 0.012689973333042068\n",
      "Epoch number: 210000 MSE_loss: 0.012676363181611723\n",
      "Epoch number: 210200 MSE_loss: 0.012662780327530224\n",
      "Epoch number: 210400 MSE_loss: 0.012649224691572225\n",
      "Epoch number: 210600 MSE_loss: 0.012635696194811795\n",
      "Epoch number: 210800 MSE_loss: 0.012622194758621025\n",
      "Epoch number: 211000 MSE_loss: 0.0126087203046686\n",
      "Epoch number: 211200 MSE_loss: 0.012595272754918577\n",
      "Epoch number: 211400 MSE_loss: 0.012581852031628806\n",
      "Epoch number: 211600 MSE_loss: 0.012568458057349746\n",
      "Epoch number: 211800 MSE_loss: 0.012555090754923006\n",
      "Epoch number: 212000 MSE_loss: 0.012541750047480045\n",
      "Epoch number: 212200 MSE_loss: 0.012528435858440837\n",
      "Epoch number: 212400 MSE_loss: 0.012515148111512484\n",
      "Epoch number: 212600 MSE_loss: 0.012501886730687988\n",
      "Epoch number: 212800 MSE_loss: 0.012488651640244847\n",
      "Epoch number: 213000 MSE_loss: 0.012475442764743785\n",
      "Epoch number: 213200 MSE_loss: 0.012462260029027507\n",
      "Epoch number: 213400 MSE_loss: 0.012449103358219235\n",
      "Epoch number: 213600 MSE_loss: 0.012435972677721634\n",
      "Epoch number: 213800 MSE_loss: 0.01242286791321536\n",
      "Epoch number: 214000 MSE_loss: 0.012409788990657922\n",
      "Epoch number: 214200 MSE_loss: 0.012396735836282266\n",
      "Epoch number: 214400 MSE_loss: 0.012383708376595642\n",
      "Epoch number: 214600 MSE_loss: 0.012370706538378322\n",
      "Epoch number: 214800 MSE_loss: 0.012357730248682322\n",
      "Epoch number: 215000 MSE_loss: 0.012344779434830123\n",
      "Epoch number: 215200 MSE_loss: 0.012331854024413625\n",
      "Epoch number: 215400 MSE_loss: 0.012318953945292624\n",
      "Epoch number: 215600 MSE_loss: 0.012306079125593889\n",
      "Epoch number: 215800 MSE_loss: 0.012293229493709765\n",
      "Epoch number: 216000 MSE_loss: 0.012280404978297015\n",
      "Epoch number: 216200 MSE_loss: 0.012267605508275645\n",
      "Epoch number: 216400 MSE_loss: 0.012254831012827663\n",
      "Epoch number: 216600 MSE_loss: 0.012242081421395955\n",
      "Epoch number: 216800 MSE_loss: 0.012229356663683022\n",
      "Epoch number: 217000 MSE_loss: 0.01221665666964987\n",
      "Epoch number: 217200 MSE_loss: 0.01220398136951486\n",
      "Epoch number: 217400 MSE_loss: 0.012191330693752436\n",
      "Epoch number: 217600 MSE_loss: 0.012178704573092143\n",
      "Epoch number: 217800 MSE_loss: 0.012166102938517274\n",
      "Epoch number: 218000 MSE_loss: 0.0121535257212639\n",
      "Epoch number: 218200 MSE_loss: 0.01214097285281962\n",
      "Epoch number: 218400 MSE_loss: 0.012128444264922484\n",
      "Epoch number: 218600 MSE_loss: 0.012115939889559902\n",
      "Epoch number: 218800 MSE_loss: 0.012103459658967424\n",
      "Epoch number: 219000 MSE_loss: 0.012091003505627675\n",
      "Epoch number: 219200 MSE_loss: 0.012078571362269282\n",
      "Epoch number: 219400 MSE_loss: 0.01206616316186578\n",
      "Epoch number: 219600 MSE_loss: 0.012053778837634426\n",
      "Epoch number: 219800 MSE_loss: 0.012041418323035182\n",
      "Epoch number: 220000 MSE_loss: 0.012029081551769608\n",
      "Epoch number: 220200 MSE_loss: 0.012016768457779806\n",
      "Epoch number: 220400 MSE_loss: 0.012004478975247299\n",
      "Epoch number: 220600 MSE_loss: 0.011992213038592056\n",
      "Epoch number: 220800 MSE_loss: 0.011979970582471328\n",
      "Epoch number: 221000 MSE_loss: 0.01196775154177862\n",
      "Epoch number: 221200 MSE_loss: 0.011955555851642707\n",
      "Epoch number: 221400 MSE_loss: 0.0119433834474265\n",
      "Epoch number: 221600 MSE_loss: 0.011931234264726068\n",
      "Epoch number: 221800 MSE_loss: 0.011919108239369564\n",
      "Epoch number: 222000 MSE_loss: 0.011907005307416183\n",
      "Epoch number: 222200 MSE_loss: 0.011894925405155276\n",
      "Epoch number: 222400 MSE_loss: 0.011882868469105107\n",
      "Epoch number: 222600 MSE_loss: 0.011870834436012044\n",
      "Epoch number: 222800 MSE_loss: 0.01185882324284944\n",
      "Epoch number: 223000 MSE_loss: 0.011846834826816676\n",
      "Epoch number: 223200 MSE_loss: 0.011834869125338097\n",
      "Epoch number: 223400 MSE_loss: 0.011822926076062145\n",
      "Epoch number: 223600 MSE_loss: 0.01181100561686029\n",
      "Epoch number: 223800 MSE_loss: 0.011799107685825988\n",
      "Epoch number: 224000 MSE_loss: 0.011787232221273862\n",
      "Epoch number: 224200 MSE_loss: 0.011775379161738611\n",
      "Epoch number: 224400 MSE_loss: 0.011763548445974098\n",
      "Epoch number: 224600 MSE_loss: 0.011751740012952363\n",
      "Epoch number: 224800 MSE_loss: 0.011739953801862671\n",
      "Epoch number: 225000 MSE_loss: 0.011728189752110547\n",
      "Epoch number: 225200 MSE_loss: 0.01171644780331693\n",
      "Epoch number: 225400 MSE_loss: 0.01170472789531705\n",
      "Epoch number: 225600 MSE_loss: 0.011693029968159687\n",
      "Epoch number: 225800 MSE_loss: 0.011681353962106103\n",
      "Epoch number: 226000 MSE_loss: 0.011669699817629152\n",
      "Epoch number: 226200 MSE_loss: 0.011658067475412384\n",
      "Epoch number: 226400 MSE_loss: 0.011646456876349095\n",
      "Epoch number: 226600 MSE_loss: 0.011634867961541465\n",
      "Epoch number: 226800 MSE_loss: 0.011623300672299573\n",
      "Epoch number: 227000 MSE_loss: 0.011611754950140598\n",
      "Epoch number: 227200 MSE_loss: 0.011600230736787785\n",
      "Epoch number: 227400 MSE_loss: 0.011588727974169688\n",
      "Epoch number: 227600 MSE_loss: 0.011577246604419222\n",
      "Epoch number: 227800 MSE_loss: 0.011565786569872769\n",
      "Epoch number: 228000 MSE_loss: 0.011554347813069252\n",
      "Epoch number: 228200 MSE_loss: 0.011542930276749389\n",
      "Epoch number: 228400 MSE_loss: 0.011531533903854738\n",
      "Epoch number: 228600 MSE_loss: 0.011520158637526812\n",
      "Epoch number: 228800 MSE_loss: 0.011508804421106253\n",
      "Epoch number: 229000 MSE_loss: 0.011497471198132\n",
      "Epoch number: 229200 MSE_loss: 0.011486158912340366\n",
      "Epoch number: 229400 MSE_loss: 0.011474867507664263\n",
      "Epoch number: 229600 MSE_loss: 0.011463596928232277\n",
      "Epoch number: 229800 MSE_loss: 0.011452347118367909\n",
      "Epoch number: 230000 MSE_loss: 0.011441118022588717\n",
      "Epoch number: 230200 MSE_loss: 0.011429909585605426\n",
      "Epoch number: 230400 MSE_loss: 0.011418721752321165\n",
      "Epoch number: 230600 MSE_loss: 0.011407554467830617\n",
      "Epoch number: 230800 MSE_loss: 0.011396407677419234\n",
      "Epoch number: 231000 MSE_loss: 0.011385281326562365\n",
      "Epoch number: 231200 MSE_loss: 0.01137417536092449\n",
      "Epoch number: 231400 MSE_loss: 0.011363089726358444\n",
      "Epoch number: 231600 MSE_loss: 0.011352024368904473\n",
      "Epoch number: 231800 MSE_loss: 0.011340979234789599\n",
      "Epoch number: 232000 MSE_loss: 0.011329954270426754\n",
      "Epoch number: 232200 MSE_loss: 0.01131894942241399\n",
      "Epoch number: 232400 MSE_loss: 0.011307964637533701\n",
      "Epoch number: 232600 MSE_loss: 0.011296999862751775\n",
      "Epoch number: 232800 MSE_loss: 0.011286055045217012\n",
      "Epoch number: 233000 MSE_loss: 0.011275130132260053\n",
      "Epoch number: 233200 MSE_loss: 0.011264225071392886\n",
      "Epoch number: 233400 MSE_loss: 0.011253339810307928\n",
      "Epoch number: 233600 MSE_loss: 0.011242474296877329\n",
      "Epoch number: 233800 MSE_loss: 0.011231628479152092\n",
      "Epoch number: 234000 MSE_loss: 0.011220802305361488\n",
      "Epoch number: 234200 MSE_loss: 0.011209995723912214\n",
      "Epoch number: 234400 MSE_loss: 0.011199208683387632\n",
      "Epoch number: 234600 MSE_loss: 0.01118844113254705\n",
      "Epoch number: 234800 MSE_loss: 0.011177693020325\n",
      "Epoch number: 235000 MSE_loss: 0.011166964295830438\n",
      "Epoch number: 235200 MSE_loss: 0.011156254908346137\n",
      "Epoch number: 235400 MSE_loss: 0.01114556480732778\n",
      "Epoch number: 235600 MSE_loss: 0.011134893942403386\n",
      "Epoch number: 235800 MSE_loss: 0.011124242263372488\n",
      "Epoch number: 236000 MSE_loss: 0.011113609720205498\n",
      "Epoch number: 236200 MSE_loss: 0.011102996263042939\n",
      "Epoch number: 236400 MSE_loss: 0.011092401842194713\n",
      "Epoch number: 236600 MSE_loss: 0.01108182640813947\n",
      "Epoch number: 236800 MSE_loss: 0.011071269911523818\n",
      "Epoch number: 237000 MSE_loss: 0.0110607323031617\n",
      "Epoch number: 237200 MSE_loss: 0.01105021353403362\n",
      "Epoch number: 237400 MSE_loss: 0.011039713555286011\n",
      "Epoch number: 237600 MSE_loss: 0.011029232318230544\n",
      "Epoch number: 237800 MSE_loss: 0.011018769774343393\n",
      "Epoch number: 238000 MSE_loss: 0.01100832587526454\n",
      "Epoch number: 238200 MSE_loss: 0.010997900572797203\n",
      "Epoch number: 238400 MSE_loss: 0.010987493818907021\n",
      "Epoch number: 238600 MSE_loss: 0.010977105565721504\n",
      "Epoch number: 238800 MSE_loss: 0.010966735765529248\n",
      "Epoch number: 239000 MSE_loss: 0.010956384370779365\n",
      "Epoch number: 239200 MSE_loss: 0.010946051334080784\n",
      "Epoch number: 239400 MSE_loss: 0.01093573660820154\n",
      "Epoch number: 239600 MSE_loss: 0.01092544014606822\n",
      "Epoch number: 239800 MSE_loss: 0.010915161900765183\n",
      "Epoch number: 240000 MSE_loss: 0.010904901825534039\n",
      "Epoch number: 240200 MSE_loss: 0.010894659873772876\n",
      "Epoch number: 240400 MSE_loss: 0.010884435999035755\n",
      "Epoch number: 240600 MSE_loss: 0.01087423015503193\n",
      "Epoch number: 240800 MSE_loss: 0.01086404229562525\n",
      "Epoch number: 241000 MSE_loss: 0.010853872374833656\n",
      "Epoch number: 241200 MSE_loss: 0.010843720346828312\n",
      "Epoch number: 241400 MSE_loss: 0.010833586165933195\n",
      "Epoch number: 241600 MSE_loss: 0.01082346978662429\n",
      "Epoch number: 241800 MSE_loss: 0.010813371163529128\n",
      "Epoch number: 242000 MSE_loss: 0.01080329025142606\n",
      "Epoch number: 242200 MSE_loss: 0.01079322700524366\n",
      "Epoch number: 242400 MSE_loss: 0.01078318138006011\n",
      "Epoch number: 242600 MSE_loss: 0.010773153331102648\n",
      "Epoch number: 242800 MSE_loss: 0.010763142813746872\n",
      "Epoch number: 243000 MSE_loss: 0.010753149783516209\n",
      "Epoch number: 243200 MSE_loss: 0.01074317419608122\n",
      "Epoch number: 243400 MSE_loss: 0.010733216007259147\n",
      "Epoch number: 243600 MSE_loss: 0.010723275173013134\n",
      "Epoch number: 243800 MSE_loss: 0.010713351649451799\n",
      "Epoch number: 244000 MSE_loss: 0.010703445392828553\n",
      "Epoch number: 244200 MSE_loss: 0.01069355635954101\n",
      "Epoch number: 244400 MSE_loss: 0.010683684506130497\n",
      "Epoch number: 244600 MSE_loss: 0.010673829789281303\n",
      "Epoch number: 244800 MSE_loss: 0.010663992165820224\n",
      "Epoch number: 245000 MSE_loss: 0.010654171592716003\n",
      "Epoch number: 245200 MSE_loss: 0.010644368027078662\n",
      "Epoch number: 245400 MSE_loss: 0.010634581426159013\n",
      "Epoch number: 245600 MSE_loss: 0.010624811747348065\n",
      "Epoch number: 245800 MSE_loss: 0.0106150589481764\n",
      "Epoch number: 246000 MSE_loss: 0.010605322986313686\n",
      "Epoch number: 246200 MSE_loss: 0.010595603819568125\n",
      "Epoch number: 246400 MSE_loss: 0.010585901405885828\n",
      "Epoch number: 246600 MSE_loss: 0.010576215703350308\n",
      "Epoch number: 246800 MSE_loss: 0.010566546670181873\n",
      "Epoch number: 247000 MSE_loss: 0.01055689426473719\n",
      "Epoch number: 247200 MSE_loss: 0.010547258445508606\n",
      "Epoch number: 247400 MSE_loss: 0.0105376391711237\n",
      "Epoch number: 247600 MSE_loss: 0.010528036400344658\n",
      "Epoch number: 247800 MSE_loss: 0.010518450092067822\n",
      "Epoch number: 248000 MSE_loss: 0.010508880205323095\n",
      "Epoch number: 248200 MSE_loss: 0.010499326699273427\n",
      "Epoch number: 248400 MSE_loss: 0.010489789533214285\n",
      "Epoch number: 248600 MSE_loss: 0.010480268666573095\n",
      "Epoch number: 248800 MSE_loss: 0.010470764058908788\n",
      "Epoch number: 249000 MSE_loss: 0.010461275669911188\n",
      "Epoch number: 249200 MSE_loss: 0.0104518034594005\n",
      "Epoch number: 249400 MSE_loss: 0.010442347387326889\n",
      "Epoch number: 249600 MSE_loss: 0.01043290741376988\n",
      "Epoch number: 249800 MSE_loss: 0.010423483498937793\n",
      "Epoch number: 250000 MSE_loss: 0.010414075603167365\n",
      "Epoch number: 250200 MSE_loss: 0.010404683686923157\n",
      "Epoch number: 250400 MSE_loss: 0.010395307710797046\n",
      "Epoch number: 250600 MSE_loss: 0.010385947635507747\n",
      "Epoch number: 250800 MSE_loss: 0.010376603421900318\n",
      "Epoch number: 251000 MSE_loss: 0.010367275030945604\n",
      "Epoch number: 251200 MSE_loss: 0.010357962423739793\n",
      "Epoch number: 251400 MSE_loss: 0.010348665561503934\n",
      "Epoch number: 251600 MSE_loss: 0.010339384405583353\n",
      "Epoch number: 251800 MSE_loss: 0.010330118917447293\n",
      "Epoch number: 252000 MSE_loss: 0.01032086905868834\n",
      "Epoch number: 252200 MSE_loss: 0.010311634791021899\n",
      "Epoch number: 252400 MSE_loss: 0.010302416076285842\n",
      "Epoch number: 252600 MSE_loss: 0.010293212876439889\n",
      "Epoch number: 252800 MSE_loss: 0.010284025153565246\n",
      "Epoch number: 253000 MSE_loss: 0.010274852869864052\n",
      "Epoch number: 253200 MSE_loss: 0.010265695987658878\n",
      "Epoch number: 253400 MSE_loss: 0.010256554469392373\n",
      "Epoch number: 253600 MSE_loss: 0.010247428277626683\n",
      "Epoch number: 253800 MSE_loss: 0.010238317375043013\n",
      "Epoch number: 254000 MSE_loss: 0.010229221724441169\n",
      "Epoch number: 254200 MSE_loss: 0.010220141288739144\n",
      "Epoch number: 254400 MSE_loss: 0.010211076030972538\n",
      "Epoch number: 254600 MSE_loss: 0.010202025914294206\n",
      "Epoch number: 254800 MSE_loss: 0.010192990901973792\n",
      "Epoch number: 255000 MSE_loss: 0.010183970957397174\n",
      "Epoch number: 255200 MSE_loss: 0.010174966044066102\n",
      "Epoch number: 255400 MSE_loss: 0.01016597612559776\n",
      "Epoch number: 255600 MSE_loss: 0.010157001165724225\n",
      "Epoch number: 255800 MSE_loss: 0.010148041128292117\n",
      "Epoch number: 256000 MSE_loss: 0.010139095977262149\n",
      "Epoch number: 256200 MSE_loss: 0.010130165676708549\n",
      "Epoch number: 256400 MSE_loss: 0.01012125019081881\n",
      "Epoch number: 256600 MSE_loss: 0.010112349483893099\n",
      "Epoch number: 256800 MSE_loss: 0.010103463520343924\n",
      "Epoch number: 257000 MSE_loss: 0.01009459226469564\n",
      "Epoch number: 257200 MSE_loss: 0.010085735681584048\n",
      "Epoch number: 257400 MSE_loss: 0.010076893735755908\n",
      "Epoch number: 257600 MSE_loss: 0.010068066392068586\n",
      "Epoch number: 257800 MSE_loss: 0.010059253615489592\n",
      "Epoch number: 258000 MSE_loss: 0.010050455371096152\n",
      "Epoch number: 258200 MSE_loss: 0.010041671624074806\n",
      "Epoch number: 258400 MSE_loss: 0.010032902339720907\n",
      "Epoch number: 258600 MSE_loss: 0.01002414748343833\n",
      "Epoch number: 258800 MSE_loss: 0.010015407020738977\n",
      "Epoch number: 259000 MSE_loss: 0.010006680917242364\n",
      "Epoch number: 259200 MSE_loss: 0.009997969138675221\n",
      "Epoch number: 259400 MSE_loss: 0.009989271650871058\n",
      "Epoch number: 259600 MSE_loss: 0.009980588419769799\n",
      "Epoch number: 259800 MSE_loss: 0.00997191941141736\n",
      "Epoch number: 260000 MSE_loss: 0.009963264591965176\n",
      "Epoch number: 260200 MSE_loss: 0.009954623927669928\n",
      "Epoch number: 260400 MSE_loss: 0.009945997384893024\n",
      "Epoch number: 260600 MSE_loss: 0.009937384930100233\n",
      "Epoch number: 260800 MSE_loss: 0.009928786529861336\n",
      "Epoch number: 261000 MSE_loss: 0.009920202150849633\n",
      "Epoch number: 261200 MSE_loss: 0.00991163175984166\n",
      "Epoch number: 261400 MSE_loss: 0.009903075323716695\n",
      "Epoch number: 261600 MSE_loss: 0.009894532809456442\n",
      "Epoch number: 261800 MSE_loss: 0.009886004184144563\n",
      "Epoch number: 262000 MSE_loss: 0.009877489414966426\n",
      "Epoch number: 262200 MSE_loss: 0.00986898846920851\n",
      "Epoch number: 262400 MSE_loss: 0.009860501314258245\n",
      "Epoch number: 262600 MSE_loss: 0.009852027917603456\n",
      "Epoch number: 262800 MSE_loss: 0.009843568246832102\n",
      "Epoch number: 263000 MSE_loss: 0.00983512226963179\n",
      "Epoch number: 263064 MSE_loss: 0.009832422443745858 Max_loss: 0.014999957205486192\n"
     ]
    }
   ],
   "source": [
    "#binary inputs!!!! \n",
    "training_patterns = np.array([\n",
    "    [0, 1, 1],\n",
    "    [1, 1, 0],\n",
    "    [0, 0, 0]])\n",
    "\n",
    "test_patterns = np.array([\n",
    "    [0, 0.5, 1],\n",
    "    [1, 0.5, 0],\n",
    "    [0, 1, 0]])\n",
    "\n",
    "input_shape = 3\n",
    "output_shape = 3\n",
    "epochs = 90000000000\n",
    "learning_rate = 0.001\n",
    "\n",
    "def logistic(x):\n",
    "    return 1/(1 + np.exp(-x))\n",
    "def logisticprime(x):\n",
    "    return logistic(x)*(1-logistic(x))\n",
    "\n",
    "\n",
    "def initialize_parameters(input_shape, output_shape):\n",
    "    wlayer1 = np.random.randn(input_shape, output_shape) * 0.1\n",
    "    wlayer1 = (wlayer1 - np.min(wlayer1)) / (np.max(wlayer1) - np.min(wlayer1))\n",
    "    blayer1 = np.zeros((1, input_shape))\n",
    "    return {'w1': wlayer1, 'b1':blayer1}\n",
    "\n",
    "def mse_loss(y_true, y_pred):\n",
    "    return np.mean((y_pred - y_true) ** 2)\n",
    "    \n",
    "def fpass(p, parameters, activation_function):\n",
    "    z1 = np.dot(p, parameters['w1']) + parameters['b1']\n",
    "    activation1 = activation_function(z1)\n",
    "    return {'z1': z1, 'a1': activation1}\n",
    "    \n",
    "def bpass(p, t, parameters, cache, activation_prime):\n",
    "    dz1 = (cache['a1'] - t) * activation_prime(cache['z1'])\n",
    "    p_reshaped = p.reshape(-1, 1)  #p doit être un vecteur colonne !!!!!!!!\n",
    "    dw1 = np.dot(p_reshaped, dz1) / len(p)\n",
    "    db1 = np.sum(dz1, axis=0, keepdims=True) / len(p)\n",
    "    return {'dw1': dw1, 'db1': db1}\n",
    "\n",
    "def delta_rule(parameters, grads, lr=learning_rate):\n",
    "    parameters['w1'] -= lr * grads['dw1']\n",
    "    parameters['b1'] -= learning_rate * grads['db1']\n",
    "    return parameters\n",
    "\n",
    "def train(activation_function, activation_prime, inputs=training_patterns, num_epochs=epochs, lr=learning_rate, input_shape=input_shape, output_shape=output_shape):\n",
    "    parameters = initialize_parameters(input_shape, output_shape)\n",
    "    for epoch in range(num_epochs):\n",
    "        total_loss = 0\n",
    "        list_loss = []\n",
    "        for p in inputs:\n",
    "            cache = fpass(p, parameters, activation_function=activation_function)\n",
    "            loss = mse_loss(p, cache['a1'])\n",
    "            total_loss += loss\n",
    "            list_loss.append(loss)\n",
    "            grads = bpass(p, p, parameters, cache=cache, activation_prime=activation_prime)\n",
    "            parameters = delta_rule(parameters, grads=grads, lr=lr)\n",
    "        average_loss = total_loss / len(inputs)\n",
    "        if epoch % 200 == 0:\n",
    "            print(\"Epoch number:\", epoch, \"MSE_loss:\", average_loss)\n",
    "        # #stopping criterion 1: \n",
    "        # if average_loss < 0.001:\n",
    "        #     print(\"Epoch number:\", epoch, \"MSE_loss:\", average_loss)\n",
    "        #     break\n",
    "        #stopping criterion 2: PREFERABLE\n",
    "        if np.max(list_loss) < 0.015:\n",
    "            print(\"Epoch number:\", epoch, \"MSE_loss:\", average_loss, \"Max_loss:\", np.max(list_loss))\n",
    "            break\n",
    "    return parameters\n",
    "\n",
    "trained_parameters = train(activation_function=logistic, activation_prime=logisticprime, inputs=training_patterns)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "55ff7109-470d-4014-b660-f8d70d1036ea",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SUPPOSED ATTRACTOR:  [[4.000e-04 1.210e-02 8.323e-01]]\n",
      "STILL NOT REINJECTED:  [[0.0025 0.017  0.5164]]\n",
      "#########################################################\n",
      "Reinjection n°: 1  Output:  [[0.0574 0.3001 0.4047]] MSError: 0.089669\n",
      "Euclidian distance: [[0.057  0.288  0.4276]] Mean distance: 0.2575\n",
      "#########################################################\n",
      "Reinjection n°: 2  Output:  [[0.1033 0.4487 0.3653]] MSError: 0.139752\n",
      "Euclidian distance: [[0.1028 0.4365 0.467 ]] Mean distance: 0.3355\n",
      "#########################################################\n",
      "Reinjection n°: 3  Output:  [[0.1416 0.5481 0.3542]] MSError: 0.178596\n",
      "Euclidian distance: [[0.1411 0.536  0.4781]] Mean distance: 0.3851\n",
      "#########################################################\n",
      "Reinjection n°: 4  Output:  [[0.1736 0.6225 0.3559]] MSError: 0.209815\n",
      "Euclidian distance: [[0.1731 0.6103 0.4764]] Mean distance: 0.42\n",
      "#########################################################\n",
      "Reinjection n°: 5  Output:  [[0.2003 0.6799 0.3637]] MSError: 0.235112\n",
      "Euclidian distance: [[0.1999 0.6677 0.4686]] Mean distance: 0.4454\n",
      "#########################################################\n",
      "Reinjection n°: 6  Output:  [[0.2224 0.7238 0.3742]] MSError: 0.255196\n",
      "Euclidian distance: [[0.222  0.7116 0.4582]] Mean distance: 0.4639\n",
      "#########################################################\n",
      "Reinjection n°: 7  Output:  [[0.2403 0.7567 0.3851]] MSError: 0.270622\n",
      "Euclidian distance: [[0.2399 0.7445 0.4472]] Mean distance: 0.4772\n",
      "#########################################################\n",
      "Reinjection n°: 8  Output:  [[0.2541 0.7809 0.3954]] MSError: 0.282057\n",
      "Euclidian distance: [[0.2537 0.7687 0.4369]] Mean distance: 0.4864\n",
      "#########################################################\n",
      "Reinjection n°: 9  Output:  [[0.2645 0.7983 0.4046]] MSError: 0.290247\n",
      "Euclidian distance: [[0.264  0.7862 0.4277]] Mean distance: 0.4926\n",
      "#########################################################\n",
      "Reinjection n°: 10  Output:  [[0.2718 0.8109 0.4126]] MSError: 0.295909\n",
      "Euclidian distance: [[0.2713 0.7987 0.4197]] Mean distance: 0.4966\n",
      "#########################################################\n"
     ]
    }
   ],
   "source": [
    "# testing on input patterns, and reinjecting on the third to denoise it\n",
    "test_patterns = np.array([\n",
    "    [-1, -1, 1],\n",
    "    [1, 1, -1],\n",
    "    [-0.7, -0.8, 0.6]])\n",
    "attractor = fpass(test_patterns[0], trained_parameters, activation_function=logistic)\n",
    "print('SUPPOSED ATTRACTOR: ',np.round(attractor['a1'],4))\n",
    "p = test_patterns[2]\n",
    "cache = fpass(p, trained_parameters, activation_function=logistic)\n",
    "print('STILL NOT REINJECTED: ',np.round(cache['a1'],4))\n",
    "print('#########################################################')\n",
    "cache = fpass(p, trained_parameters, activation_function=logistic)\n",
    "for reinjection_number in range(10):\n",
    "    reinjection_number +=1\n",
    "    cache = fpass(cache['a1'], trained_parameters, activation_function=logistic)\n",
    "    mse_error = np.mean((cache['a1'] - attractor['a1']) ** 2)\n",
    "    euclidian_distance = np.absolute(np.subtract(cache['a1'], attractor['a1']))\n",
    "    mean_distance = np.mean(euclidian_distance)\n",
    "    print('Reinjection n°:', reinjection_number, ' Output: ',np.round(cache['a1'],4),\n",
    "          'MSError:', np.round(mse_error,6))\n",
    "    print('Euclidian distance:', np.round(euclidian_distance,4), 'Mean distance:',np.round(mean_distance,4))\n",
    "    print('#########################################################')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a871647b-c804-48d1-be06-7f50a9639f4e",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0ccc8e9d-bcaf-4c60-ad1d-90ba9317d28f",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "db950dd4-b1fc-4daf-afee-41a6c89f6a32",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0713f06f-0926-4b5d-8a68-e12a75fc0519",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}