How to generate states.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# How to generate states\n",
    "1.  “Layer 1 Input”: The variable x n (t) to “the DDQN is size 2K + 2 vector”. Initial K + 1\n",
    "input entries indicate the action taken at time t − 1. Specifically, if the user has not\n",
    "transmitted at time slot t − 1, the first entrance is set to 1 and the subsequent K entries are\n",
    "set to 0. The following K input entries are the capacity of each channel. The last input is 1\n",
    "if ACK signal has been received. Else, if communication has failed or no communication\n",
    "has been made, and it is set to 0.\n",
    "\n",
    "Since i have to feed the states in the DQN , first I created state for an user\n",
    "![alt text](state_input_dqn.png \"Title\")\n",
    "This is the state format for a user in one time slot in the environment with 3 users and 2 channels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# imports\n",
    "from multi_user_network_env import env_network\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "NUM_USERS = 3\n",
    "NUM_CHANNELS = 2\n",
    "ATTEMPT_PROB = 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "#initializing the environment\n",
    "env = env_network(NUM_USERS,NUM_CHANNELS,ATTEMPT_PROB)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "#It creates a one hot vector of a number as num with size as len\n",
    "def one_hot(num,len):\n",
    "    assert num >=0 and num < len ,\"error\"\n",
    "    vec = np.zeros([len],np.int32)\n",
    "    vec[num] = 1\n",
    "    return vec"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 0, 0, 0, 0, 1], dtype=int32)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "one_hot(5,6)  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One-hot vector of ```len (6)``` having ```value (5)```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# generates states from action and obs as returned from environment\n",
    "def state_generator(action,obs):\n",
    "    input_vector = []\n",
    "    if action is None:\n",
    "        print ('None')\n",
    "        sys.exit()\n",
    "    for user_i in range(action.size):\n",
    "        input_vector_i = one_hot(action[user_i],NUM_CHANNELS+1)\n",
    "        channel_alloc = obs[-1]\n",
    "        input_vector_i = np.append(input_vector_i,channel_alloc)\n",
    "        input_vector_i = np.append(input_vector_i,int(obs[user_i][0]))    #ACK\n",
    "        input_vector.append(input_vector_i)\n",
    "    return input_vector"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[(0, 0.0), (0, 0.0), (1, 1.0), array([1, 0], dtype=int32)]\n"
     ]
    }
   ],
   "source": [
    "# user 1 and 2 takes action 1 and user 3 takes action 2 \n",
    "action = np.array([1,1,2])\n",
    "obs = env.step(action)\n",
    "print obs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[array([0, 1, 0, 1, 0, 0]), array([0, 1, 0, 1, 0, 0]), array([0, 0, 1, 1, 0, 1])]\n"
     ]
    }
   ],
   "source": [
    "states = state_generator(action, obs)\n",
    "print states\n",
    "assert len(states) == NUM_USERS"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```[array([0, 1, 0, 1, 0, 0])```\n",
    "```array([0, 1, 0, 1, 0, 0])```\n",
    "```array([0, 0, 1, 1, 0, 1])]```\n",
    "\n",
    "Each array represents states for each user in th format given above\n",
    "\n",
    "Since there are 3 users , ```state_generator(action, obs)``` returned 3 arrays"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Rest of the training is in ```train.py``` file in this repository"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}