model/geração_texto.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "h8QaY6J8j6BT"
   },
   "source": [
    "# Gerador de texto de João Guimarães Rosa"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "id": "ljt2BmuUj6BZ"
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "import numpy as np\n",
    "import os"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "4c5qEnkLj6Bf"
   },
   "source": [
    "## Lendo os dados"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "F8d-NRDnj6Bg",
    "outputId": "9e19b4c3-4166-4c08-f3cc-c0c1d5c94942"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tamanho do corpus: 4726513 caracteres\n"
     ]
    }
   ],
   "source": [
    "text = open(\"obras.txt\", 'rb').read().decode(encoding='utf-8')\n",
    "print(f'Tamanho do corpus: {len(text)} caracteres')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "YCkpQ8Doj6Bh",
    "outputId": "fe480fbe-7053-4df2-e6d4-cbcb41a0499e"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "132 caracteres únicos\n"
     ]
    }
   ],
   "source": [
    "vocab = sorted(set(text))\n",
    "print(f'{len(vocab)} caracteres únicos')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5yp2mVFAj6Bi"
   },
   "source": [
    "## Processando o texto\n",
    "### Vetorizando o texto\n",
    "\n",
    "Antes do treinamento, precisamos mapear as strings para uma representação numérica. Para isso criaremos duas tabelas de pesquisa: uma mapeando caracteres para números e outra para números para caracteres."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "id": "8BQSDDkmj6Bj"
   },
   "outputs": [],
   "source": [
    "char2idx = {u:i for i, u in enumerate(vocab)}\n",
    "\n",
    "idx2char = np.array(vocab)\n",
    "\n",
    "text_as_int = np.array([char2idx[c] for c in text])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "mhQcypcij6Bj",
    "outputId": "d3ad6346-970c-4872-d077-26b5f17def37"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  '\\n':   0,\n",
      "  '\\x0c':   1,\n",
      "  ' ' :   2,\n",
      "  '!' :   3,\n",
      "  '\"' :   4,\n",
      "  '&' :   5,\n",
      "  \"'\" :   6,\n",
      "  '(' :   7,\n",
      "  ')' :   8,\n",
      "  '*' :   9,\n",
      "  ',' :  10,\n",
      "  '-' :  11,\n",
      "  '.' :  12,\n",
      "  '/' :  13,\n",
      "  '0' :  14,\n",
      "  '1' :  15,\n",
      "  '2' :  16,\n",
      "  '3' :  17,\n",
      "  '4' :  18,\n",
      "  '5' :  19,\n",
      "  ...\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "print('{')\n",
    "for char,_ in zip(char2idx, range(20)):\n",
    "    print('  {:4s}: {:3d},'.format(repr(char), char2idx[char]))\n",
    "print('  ...\\n}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "PJ-E4X4dj6CH",
    "outputId": "4defbe1a-1875-4da3-ffad-af11f85a11a0"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "'Grande Sertão' ---- caracteres mapeados para int ---- > [ 33  73  56  69  59  60   2  45  60  73  75 102  70]\n"
     ]
    }
   ],
   "source": [
    "print('{} ---- caracteres mapeados para int ---- > {}'.format(repr(text[:13]), text_as_int[:13]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5rZd1MKPj6CI"
   },
   "source": [
    "## A tarefa de previsão\n",
    "\n",
    "Dado um caractere, ou uma sequência de caractere, qual é o próximo caractere mais provável? Esta é a tarefa para a qual estamos treinando o modelo. A entrada para o modelo será uma sequência de caracteres e  treinaremos o modelo para prever a saída - o caractere a seguir em cada timestep.\n",
    "\n",
    "Vamos dividir o texto em sequências de exemplo. Cada sequência de entrada conterá caracteres seq_length do texto.\n",
    "\n",
    "Para cada sequência de entrada, os targets correspondentes contêm o mesmo comprimento de texto, exceto deslocado um caractere para a direita."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "J-r1vEA8j6CI",
    "outputId": "d0a72140-4df2-4078-bd2a-0a4e7ac7048d"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "G\n",
      "r\n",
      "a\n",
      "n\n",
      "d\n",
      "e\n"
     ]
    }
   ],
   "source": [
    "seq_length = 100\n",
    "examples_per_epoch = len(text)//(seq_length+1)\n",
    "\n",
    "# Criando exemplos de treinamento e targets \n",
    "char_dataset = tf.data.Dataset.from_tensor_slices(text_as_int)\n",
    "\n",
    "for i in char_dataset.take(6):\n",
    "    print(idx2char[i.numpy()])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ltY1pMuJj6CJ"
   },
   "source": [
    "O método batch nos permite converter facilmente esses caracteres individuais em sequências do tamanho desejado."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "35C-H-_Zj6CJ",
    "outputId": "3bdf27fb-327e-44e7-c482-c8aeceb41c26"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "'Grande Sertão: Veredas\\n\\n \\n― Nonada. Tiros que o senhor ouviu foram de briga de homem não, Deus esteja'\n",
      "'.\\nAlvejei mira em árvore, no quintal, no baixo do córrego. Por meu acerto. Todo dia isso\\nfaço,  gosto'\n",
      "';  desde  mal  em  minha  mocidade.  Daí,  vieram  me  chamar.  Causa  dum\\nbezerro! um bezerro branco'\n",
      "', erroso, os olhos de nem ser ― se viu ―; e com máscara de\\ncachorro.  Me  disseram;  eu  não  quis  a'\n",
      "'vistar.  Mesmo  que,  por  defeito  como  nasceu,\\narrebitado  de  beiços,  esse  figurava  rindo  fei'\n"
     ]
    }
   ],
   "source": [
    "sequences = char_dataset.batch(seq_length+1, drop_remainder=True)\n",
    "\n",
    "for item in sequences.take(5):\n",
    "    print(repr(''.join(idx2char[item.numpy()])))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "id": "Jxsjxw_hj6CK"
   },
   "outputs": [],
   "source": [
    "def split_input_target(chunk):\n",
    "    input_text = chunk[:-1]\n",
    "    target_text = chunk[1:]\n",
    "    return input_text, target_text\n",
    "\n",
    "dataset = sequences.map(split_input_target)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "NwMY_NYnj6CK",
    "outputId": "219e3f73-9930-4712-e519-dd9b88b68b1f"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input data:  'Grande Sertão: Veredas\\n\\n \\n― Nonada. Tiros que o senhor ouviu foram de briga de homem não, Deus estej'\n",
      "Target data: 'rande Sertão: Veredas\\n\\n \\n― Nonada. Tiros que o senhor ouviu foram de briga de homem não, Deus esteja'\n"
     ]
    }
   ],
   "source": [
    "for input_example, target_example in  dataset.take(1):\n",
    "    print('Input data: ', repr(''.join(idx2char[input_example.numpy()])))\n",
    "    print('Target data:', repr(''.join(idx2char[target_example.numpy()])))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "VY67wORpj6CK"
   },
   "source": [
    "Cada índice desses vetores é processado como uma etapa única. Para a entrada na etapa de tempo 0, o modelo recebe o índice para \"G\" e tenta prever o índice para \"r\" como o próximo caractere. No próximo timestep, ele faz a mesma coisa, mas a RNN considera o contexto da etapa anterior além do caractere de entrada atual."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "G-wU1W55j6CL",
    "outputId": "77780ad0-e7dc-4425-a5bf-6301b4bf6380"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Step    0\n",
      "  input: 33 ('G')\n",
      "  expected output: 73 ('r')\n",
      "Step    1\n",
      "  input: 73 ('r')\n",
      "  expected output: 56 ('a')\n",
      "Step    2\n",
      "  input: 56 ('a')\n",
      "  expected output: 69 ('n')\n",
      "Step    3\n",
      "  input: 69 ('n')\n",
      "  expected output: 59 ('d')\n",
      "Step    4\n",
      "  input: 59 ('d')\n",
      "  expected output: 60 ('e')\n"
     ]
    }
   ],
   "source": [
    "for i, (input_idx, target_idx) in enumerate(zip(input_example[:5], target_example[:5])):\n",
    "    print(\"Step {:4d}\".format(i))\n",
    "    print(\"  input: {} ({:s})\".format(input_idx, repr(idx2char[input_idx])))\n",
    "    print(\"  expected output: {} ({:s})\".format(target_idx, repr(idx2char[target_idx])))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "id": "qUyxtDF4j6CL"
   },
   "outputs": [],
   "source": [
    "# Batch size\n",
    "BATCH_SIZE = 64\n",
    "\n",
    "# Buffer size embaralha o nosso dataset \n",
    "BUFFER_SIZE = 10000\n",
    "\n",
    "dataset = dataset.shuffle(BUFFER_SIZE).batch(BATCH_SIZE, drop_remainder=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "O8f_JdtHj6CM"
   },
   "source": [
    "### Construindo o modelo\n",
    "\n",
    "Usamos tf.keras.Sequential para definir o modelo. Para este exemplo simples, três camadas são usadas para definir nosso modelo:\n",
    "\n",
    "- tf.keras.layers.Embedding : A camada de entrada. Uma tabela de pesquisa treinável que mapeará os números de cada caractere para um vetor com dimensões embedding_dim ;\n",
    "\n",
    "- tf.keras.layers.GRU : Um tipo de RNN com units=rnn_units tamanho units=rnn_units (também poderíamos usar uma camada LSTM aqui.)\n",
    "\n",
    "- tf.keras.layers.Dense : A camada de saída, com saídas vocab_size ."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "id": "8ZeNgaXij6CM"
   },
   "outputs": [],
   "source": [
    "# Tamanho do vocabulário em caracteres\n",
    "vocab_size = len(vocab)\n",
    "\n",
    "# Dimensão dos embeddings \n",
    "embedding_dim = 256\n",
    "\n",
    "# Número de unidades da RNN \n",
    "rnn_units = 1024"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "id": "gphEARIKj6CM"
   },
   "outputs": [],
   "source": [
    "def build_model(vocab_size, embedding_dim, rnn_units, batch_size):\n",
    "    model = tf.keras.Sequential([\n",
    "        tf.keras.layers.Embedding(vocab_size, embedding_dim,\n",
    "                                  batch_input_shape=[batch_size, None]),\n",
    "        tf.keras.layers.GRU(rnn_units,\n",
    "                            return_sequences=True,\n",
    "                            stateful=True,\n",
    "                            recurrent_initializer='glorot_uniform'),\n",
    "        tf.keras.layers.Dense(vocab_size)\n",
    "    ])\n",
    "    return model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "id": "yQH5LMYEj6CN"
   },
   "outputs": [],
   "source": [
    "model = build_model(\n",
    "    vocab_size=len(vocab),\n",
    "    embedding_dim=embedding_dim,\n",
    "    rnn_units=rnn_units,\n",
    "    batch_size=BATCH_SIZE)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "tkL0xQN9j6CO",
    "outputId": "d7d3d2cd-6b7f-4547-f29b-41e8251139ea"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(64, 100, 132) # (batch_size, sequence_length, vocab_size)\n"
     ]
    }
   ],
   "source": [
    "for input_example_batch, target_example_batch in dataset.take(1):\n",
    "    example_batch_predictions = model(input_example_batch)\n",
    "    print(example_batch_predictions.shape, \"# (batch_size, sequence_length, vocab_size)\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "PVw0POnWj6CP",
    "outputId": "5436b884-dbdd-4665-8fba-db5644051449"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "embedding (Embedding)        (64, None, 256)           33792     \n",
      "_________________________________________________________________\n",
      "gru (GRU)                    (64, None, 1024)          3938304   \n",
      "_________________________________________________________________\n",
      "dense (Dense)                (64, None, 132)           135300    \n",
      "=================================================================\n",
      "Total params: 4,107,396\n",
      "Trainable params: 4,107,396\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "fLlUrNAGj6CQ"
   },
   "source": [
    "Para obter previsões reais do modelo, precisamos de uma amostra da distribuição de saída para obter índices de caracteres reais. Esta distribuição é definida pelos logits sobre o vocabulário dos caracteres."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "id": "y3dtHarcj6CQ"
   },
   "outputs": [],
   "source": [
    "sampled_indices = tf.random.categorical(example_batch_predictions[0], num_samples=1)\n",
    "sampled_indices = tf.squeeze(sampled_indices,axis=-1).numpy()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Dr134IVTj6CQ",
    "outputId": "3e96c3b7-a0ab-4a19-a692-ff9ccc6344de"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 73,  18,  32, 123,  35,  27, 125,  11, 105, 109,  90,  29, 121,\n",
       "         9,   9, 113,  72,   0,  76, 124,  94, 114,  89,  32,  26,  29,\n",
       "        23,  11,  84,  83, 116,  39,  73, 109,  92,  64, 128, 124,  24,\n",
       "       128,  71,  20,  15,  69,  98,  46,  98,   4, 126, 126,  27,   6,\n",
       "        40, 103,  32,  74, 106,  53,  38,  25,  94,  78,  99,  94,  29,\n",
       "        18, 119, 109, 102,  16,  56,  38,  41,  23,  78,   0,  19, 115,\n",
       "         9,  23, 102,  35,  43,  79,  68, 111, 101,  88,  33,   7,  59,\n",
       "        23,  66,  21,  41, 112,  88,  54,  75,   5])"
      ]
     },
     "execution_count": 19,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sampled_indices"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ZSMfhCRqj6CR"
   },
   "source": [
    "Decodifique-os para ver o texto previsto por este modelo não treinado:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "LH6B1WJSj6CS",
    "outputId": "f933cd95-c2fd-4c44-9bf2-35d84e3148ea"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input: \n",
      " 'ntes  brenhas.\\nPor que é que iam, nem esperando eu desse minha primeira ganhada?\\n\\n― A isso, meio aco'\n",
      "\n",
      "Next Char Predictions: \n",
      " 'r4F–IA―-çìÂCü**òq\\nu—ÍóÁF?C9-´ªõMrìÉi“—:“p61nÚTÚ\"‘‘A\\'NäFsè[L;ÍwàÍC4úìã2aLO9w\\n5ô*9ãIQxmïâÀG(d9k7OñÀ]t&'\n"
     ]
    }
   ],
   "source": [
    "print(\"Input: \\n\", repr(\"\".join(idx2char[input_example_batch[1]])))\n",
    "print()\n",
    "print(\"Next Char Predictions: \\n\", repr(\"\".join(idx2char[sampled_indices ])))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "fuQYS-rxj6CS"
   },
   "source": [
    "### Treinando o modelo\n",
    "\n",
    "Neste ponto, o problema pode ser tratado como um problema de classificação padrão. Dado o estado RNN anterior e a entrada desta etapa de tempo, prevemos a classe do próximo caractere.\n",
    "\n",
    "A loss tf.keras.losses.sparse_categorical_crossentropy padrão funciona neste caso porque é aplicada na última dimensão das previsões."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "P3Tdnmauj6CS",
    "outputId": "d129687b-6898-4355-8326-075a1863e862"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Prediction shape:  (64, 100, 132)  # (batch_size, sequence_length, vocab_size)\n",
      "scalar_loss:       4.8817596\n"
     ]
    }
   ],
   "source": [
    "def loss(labels, logits):\n",
    "    return tf.keras.losses.sparse_categorical_crossentropy(labels, logits, from_logits=True)\n",
    "\n",
    "example_batch_loss = loss(target_example_batch, example_batch_predictions)\n",
    "print(\"Prediction shape: \", example_batch_predictions.shape, \" # (batch_size, sequence_length, vocab_size)\")\n",
    "print(\"scalar_loss:      \", example_batch_loss.numpy().mean())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "id": "M0yilVImj6CT"
   },
   "outputs": [],
   "source": [
    "model.compile(optimizer='adam', loss=loss)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "hFSBBjhej6CT"
   },
   "source": [
    "Usaremos um tf.keras.callbacks.ModelCheckpoint para garantir que os pontos de verificação sejam salvos durante o treinamento:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "id": "0eZqfX3Mj6CT"
   },
   "outputs": [],
   "source": [
    "# Diretório em que os checkpoints serão salvos\n",
    "checkpoint_dir = './training_checkpoints'\n",
    "\n",
    "# Nome dos arquivos de checkpoint\n",
    "checkpoint_prefix = os.path.join(checkpoint_dir, \"ckpt_{epoch}\")\n",
    "\n",
    "checkpoint_callback = tf.keras.callbacks.ModelCheckpoint(\n",
    "    filepath=checkpoint_prefix,\n",
    "    save_weights_only=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "HF-NFl_Pj6CT"
   },
   "source": [
    "#### O treinamento\n",
    "\n",
    "Para manter o tempo de treinamento razoável, usaremos 10 épocas para treinar o modelo. Com o Colab, podemos usar uma GPU para acelerar o treinamento."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "id": "MPubf68-j6CT"
   },
   "outputs": [],
   "source": [
    "EPOCHS = 10"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "s0YrFGihj6CT",
    "outputId": "05028ef8-adbf-4b20-8bf8-ee9f11d25d4c"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "731/731 [==============================] - 42s 57ms/step - loss: 1.3190\n",
      "Epoch 2/10\n",
      "731/731 [==============================] - 44s 58ms/step - loss: 1.3074\n",
      "Epoch 3/10\n",
      "731/731 [==============================] - 45s 60ms/step - loss: 1.2979\n",
      "Epoch 4/10\n",
      "731/731 [==============================] - 44s 59ms/step - loss: 1.2898\n",
      "Epoch 5/10\n",
      "731/731 [==============================] - 43s 58ms/step - loss: 1.2823\n",
      "Epoch 6/10\n",
      "731/731 [==============================] - 44s 59ms/step - loss: 1.2770\n",
      "Epoch 7/10\n",
      "731/731 [==============================] - 45s 60ms/step - loss: 1.2729\n",
      "Epoch 8/10\n",
      "731/731 [==============================] - 45s 60ms/step - loss: 1.2694\n",
      "Epoch 9/10\n",
      "731/731 [==============================] - 45s 60ms/step - loss: 1.2670\n",
      "Epoch 10/10\n",
      "731/731 [==============================] - 45s 60ms/step - loss: 1.2660\n"
     ]
    }
   ],
   "source": [
    "history = model.fit(dataset, epochs=EPOCHS, callbacks=[checkpoint_callback])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "2TaqKLlpj6CT"
   },
   "source": [
    "### Gerar texto"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "id": "SYL0aULHj6CU",
    "outputId": "b4fc3b1e-0c3b-4acb-ba9f-0d66aa0e5ebc"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.google.colaboratory.intrinsic+json": {
       "type": "string"
      },
      "text/plain": [
       "'./training_checkpoints/ckpt_10'"
      ]
     },
     "execution_count": 27,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tf.train.latest_checkpoint(checkpoint_dir)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "id": "TLpkovpKj6CU"
   },
   "outputs": [],
   "source": [
    "model = build_model(vocab_size, embedding_dim, rnn_units, batch_size=1)\n",
    "\n",
    "model.load_weights(tf.train.latest_checkpoint(checkpoint_dir))\n",
    "\n",
    "model.build(tf.TensorShape([1, None]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "RxWk6Isbj6CU",
    "outputId": "10ea8a6b-321e-4bbe-d3b7-cb3cea9f1501"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential_1\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "embedding_1 (Embedding)      (1, None, 256)            33792     \n",
      "_________________________________________________________________\n",
      "gru_1 (GRU)                  (1, None, 1024)           3938304   \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (1, None, 132)            135300    \n",
      "=================================================================\n",
      "Total params: 4,107,396\n",
      "Trainable params: 4,107,396\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model.summary()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "S6oH7DCpj6CU"
   },
   "source": [
    "## O loop de previsão"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "id": "asGdCebfj6CU"
   },
   "outputs": [],
   "source": [
    "def generate_text(model, start_string):\n",
    "    \n",
    "    # Número de caracteres a serem gerados\n",
    "    num_generate = 1000\n",
    "\n",
    "    # Vetorização\n",
    "    input_eval = [char2idx[s] for s in start_string]\n",
    "    input_eval = tf.expand_dims(input_eval, 0)\n",
    "\n",
    "    # Lista vazia para guardar os resultados.\n",
    "    text_generated = []\n",
    "\n",
    "    temperature = 1.0\n",
    "\n",
    "    model.reset_states()\n",
    "    for i in range(num_generate):\n",
    "        predictions = model(input_eval)\n",
    "        \n",
    "        # remove a dimensão do batch\n",
    "        predictions = tf.squeeze(predictions, 0)\n",
    "\n",
    "        # utiliza uma distribuição categórica para prever o caractere que o modelo retorna\n",
    "        predictions = predictions / temperature\n",
    "        predicted_id = tf.random.categorical(predictions, num_samples=1)[-1,0].numpy()\n",
    "\n",
    "        # Passa o caractere previsto como próximo input para o modelo junto \n",
    "        # com o estado anterior da célula \n",
    "        input_eval = tf.expand_dims([predicted_id], 0)\n",
    "\n",
    "        text_generated.append(idx2char[predicted_id])\n",
    "\n",
    "    return (start_string + ''.join(text_generated))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "_5K4VyL1j6CU",
    "outputId": "69b0c5fb-e14f-41fd-cfca-ec60baa37823"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Diadorim me instruiu\n",
      "se traçar o razoável, o que de mim são da burra. Dizia que se assombrar. Enfim, ainda\n",
      "ele todo a pressentir-se, quanto o final desse ar!: acho que o sorriso. Antes de contar, melhor\n",
      "nem assim, ela era de valor receio. O são-gente prestarão tantas compinações\n",
      "calma, e próximo de meio da rua — depois pensava nele, conforme o malfeito. Via de vez de viver, com medida ainda de\n",
      "um corte de presente, por meu quieto. Ninguém sabia dar\n",
      "um come perfum de trato e própria a Nha\n",
      "ele ao lembrar a chegar até com todo encanado. Mas dubluziu duro, do que para a varanda. Levantava\n",
      "na feita ou com outras era branca; e, ainda que leve companhia de Miguilim. Mechéu com uma\n",
      "palma de escapada. Pislar e cada liso. Redrata a esta, muito judiada! — de todo o tempo, de fato. Aí, toda-a-a-loirava-se a rolição, a\n",
      "barberintebre vinha em lugar a donas; rezoante, mios\n",
      "e Firrços, quentes e lentos, a casa-de-cria e sem forro nenhuna. Mas,\n",
      "cata, menino e de, no perdão.\n",
      "Outras vez pervados para a sanfona, e\n"
     ]
    }
   ],
   "source": [
    "print(generate_text(model, start_string=u\"Diadorim\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "t5p6l98pj6CU"
   },
   "source": [
    "## Salvar modelo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.save('modelo.h5')"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "name": "geração-texto.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "nlp",
   "language": "python",
   "name": "nlp"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}