Watch_Me_Build_a_Trading_Bot (1).ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Watch Me Build a Trading Bot.ipynb",
      "version": "0.3.2",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fIe7_qVIqBZQ",
        "colab_type": "text"
      },
      "source": [
        "## Watch Me Build a Trading Bot\n",
        "\n",
        "![alt text](https://i.imgur.com/0DbW918.png)\n",
        "\n",
        "\n",
        "\n",
        "## The Stack\n",
        "- Node.js (web app)\n",
        "- Gekko\n",
        "- ConvNet.js (deep reinforcement learning)\n",
        "- Sentiment Analysis\n",
        "- Bitcoin API\n",
        "\n",
        "![alt text](https://www.woolha.com/media/2018/08/nodejs-expressjs-mongodb-vuejs-webpack.jpg)\n",
        "\n",
        "\n",
        "## Steps in this tutorial\n",
        "1. Key features of a good personal trading bot\n",
        "2. Architecture (Important parts of the codebase)\n",
        "3. Installation\n",
        "4. Analyzing Technical Indicators\n",
        "5. Implementing a custom trading strategy (Actor-Critic Deep Reinforcement Learning [New])\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2q-824LSpTzD",
        "colab_type": "text"
      },
      "source": [
        "# Trading Bots Explained\n",
        "\n",
        "## Whats a trading bot?\n",
        "\n",
        "![alt text](https://cialu.net/wp-content/uploads/2018/05/zenbot-crypto-trading-paper-mode.png)\n",
        "\n",
        "- Algorithmic trading involves creating a set of repeatable instructions to place a trade. \n",
        "- Theoretically, automating trading in this way can generate profits in a way no human could.\n",
        "- These trading algorithms (set of instructions) can depend on a variety of factors like timing, price, quantity, and human sentiment.\n",
        "\n",
        "## What's an example of a simple trading algorithm? Let's look at the naive, average, and simple exponential smoothing methods \n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "NkzS5VuvYj5K",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import pandas as pd \n",
        "import numpy as np \n",
        "import matplotlib.pyplot as plt \n",
        "\n",
        "\n",
        "#Importing data\n",
        "df = pd.read_csv('train.csv')\n",
        "\n",
        "#Subsetting the dataset\n",
        "#Index 11856 marks the end of year 2013\n",
        "df = pd.read_csv('train.csv', nrows = 11856)\n",
        "\n",
        "#Creating train and test set \n",
        "#Index 10392 marks the end of October 2013 \n",
        "train=df[0:10392] \n",
        "test=df[10392:]\n",
        "\n",
        "#Aggregating the dataset at daily level\n",
        "df.Timestamp = pd.to_datetime(df.Datetime,format='%d-%m-%Y %H:%M') \n",
        "df.index = df.Timestamp \n",
        "df = df.resample('D').mean()\n",
        "train.Timestamp = pd.to_datetime(train.Datetime,format='%d-%m-%Y %H:%M') \n",
        "train.index = train.Timestamp \n",
        "train = train.resample('D').mean() \n",
        "test.Timestamp = pd.to_datetime(test.Datetime,format='%d-%m-%Y %H:%M') \n",
        "test.index = test.Timestamp \n",
        "test = test.resample('D').mean()\n",
        "\n",
        "#Plotting data\n",
        "train.Count.plot(figsize=(15,8), title= 'Daily Ridership', fontsize=14)\n",
        "test.Count.plot(figsize=(15,8), title= 'Daily Ridership', fontsize=14)\n",
        "plt.show()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TgmfzHZPc1G8",
        "colab_type": "text"
      },
      "source": [
        "![alt text](https://s3-ap-south-1.amazonaws.com/av-blog-media/wp-content/uploads/2018/02/train_test-768x438.png)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "o_O5gDoqpTyX",
        "colab_type": "text"
      },
      "source": [
        "Naive method ![alt text](https://s3-ap-south-1.amazonaws.com/av-blog-media/wp-content/uploads/2018/01/Screen-Shot-2018-01-25-at-7.45.20-PM.png)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xziss4XadO10",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "dd= np.asarray(train.Count)\n",
        "y_hat = test.copy()\n",
        "y_hat['naive'] = dd[len(dd)-1]\n",
        "plt.figure(figsize=(12,8))\n",
        "plt.plot(train.index, train['Count'], label='Train')\n",
        "plt.plot(test.index,test['Count'], label='Test')\n",
        "plt.plot(y_hat.index,y_hat['naive'], label='Naive Forecast')\n",
        "plt.legend(loc='best')\n",
        "plt.title(\"Naive Forecast\")\n",
        "plt.show()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NISjMj84dSb8",
        "colab_type": "text"
      },
      "source": [
        "![alt text](https://s3-ap-south-1.amazonaws.com/av-blog-media/wp-content/uploads/2018/02/naive-768x519.png)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hRECNsVndYF9",
        "colab_type": "text"
      },
      "source": [
        "simple average ![alt text](https://s3-ap-south-1.amazonaws.com/av-blog-media/wp-content/uploads/2018/01/Screen-Shot-2018-01-25-at-7.45.10-PM-300x82.png) "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "odPijLimdWR8",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "y_hat_avg = test.copy()\n",
        "y_hat_avg['avg_forecast'] = train['Count'].mean()\n",
        "plt.figure(figsize=(12,8))\n",
        "plt.plot(train['Count'], label='Train')\n",
        "plt.plot(test['Count'], label='Test')\n",
        "plt.plot(y_hat_avg['avg_forecast'], label='Average Forecast')\n",
        "plt.legend(loc='best')\n",
        "plt.show()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-iWVKQyfdcU0",
        "colab_type": "text"
      },
      "source": [
        "![alt text](https://s3-ap-south-1.amazonaws.com/av-blog-media/wp-content/uploads/2018/02/avg-768x511.png)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1w-iB7NVmKn0",
        "colab_type": "text"
      },
      "source": [
        "Moving Average\n",
        "![alt text](https://s3-ap-south-1.amazonaws.com/av-blog-media/wp-content/uploads/2018/01/Screen-Shot-2018-01-25-at-7.47.33-PM.png)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TO9hDJppSlIL",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "y_hat_avg = test.copy()\n",
        "y_hat_avg['moving_avg_forecast'] = train['Count'].rolling(60).mean().iloc[-1]\n",
        "plt.figure(figsize=(16,8))\n",
        "plt.plot(train['Count'], label='Train')\n",
        "plt.plot(test['Count'], label='Test')\n",
        "plt.plot(y_hat_avg['moving_avg_forecast'], label='Moving Average Forecast')\n",
        "plt.legend(loc='best')\n",
        "plt.show()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RLeKUMJqdk4F",
        "colab_type": "text"
      },
      "source": [
        "![alt text](https://s3-ap-south-1.amazonaws.com/av-blog-media/wp-content/uploads/2018/02/moving_avg-850x428.png)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jFTcC1cvdv6k",
        "colab_type": "text"
      },
      "source": [
        "\n",
        "[![IMAGE ALT TEXT HERE](https://img.youtube.com/vi/d4Sn6ny_5LI/0.jpg)](https://www.youtube.com/watch?v=d4Sn6ny_5LI)\n",
        "\n",
        "Based on this type of trend prediction, an algorithm could conduct the following rules\n",
        "\n",
        "![alt text](https://www.business-science.io/figure/source/2018-05-31-backtesting-quantopian-zipline-tibbletime-furrr-flyingfox/unnamed-chunk-9-1.png)\n",
        "\n",
        "1. Buy 50 shares of a stock when its 50-day moving average goes above the 200-day moving average. (A moving average is an average of past data points that smooths out day-to-day price fluctuations and thereby identifies trends.)  \n",
        "2. Sell shares of the stock when its 50-day moving average goes below the 200-day moving average.\n",
        "\n",
        "- Using these two simple instructions, a computer program will automatically monitor the stock price (and the moving average indicators) and place the buy and sell orders when the defined conditions are met. \n",
        "- The trader no longer needs to monitor live prices and graphs or put in the orders manually. \n",
        "-The algorithmic trading system does this automatically by correctly identifying the trading opportunity. \n",
        "\n",
        "## Benefits of Algorithmic Trading\n",
        "Algo-trading provides the following benefits:\n",
        "\n",
        "![alt text](https://i.pinimg.com/originals/59/fa/cb/59facb180a65506270a53e1e660e3386.png)\n",
        "\n",
        "- Trades are executed at the best possible prices.\n",
        "- Trade order placement is instant and accurate (there is a high chance of execution at the desired levels).\n",
        "- Trades are timed correctly and instantly to avoid significant price changes.\n",
        "- Reduced transaction costs.\n",
        "- Simultaneous automated checks on multiple market conditions.\n",
        "- Reduced risk of manual errors when placing trades.\n",
        "- Algo-trading can be backtested using available historical and real-time data to see if it is a viable trading strategy.\n",
        "- Reduced possibility of mistakes by human traders based on emotional and psychological factors.\n",
        "\n",
        "Most algo-trading today is high-frequency trading (HFT), which attempts to capitalize on placing a large number of orders at rapid speeds across multiple markets and multiple decision parameters based on preprogrammed instructions. \n",
        "\n",
        "![alt text](https://qph.fs.quoracdn.net/main-qimg-9b40d531a710ef42980b2196d72f97a6)\n",
        "\n",
        "## Who Uses Trading bots?\n",
        "\n",
        "- Algorithmic trading is dominated by large trading firms, such as hedge funds, investment banks and proprietary trading firms. \n",
        "- Given the abundant resource availability due to their large size, such firms usually build their own proprietary trading software, including large trading systems with dedicated data centers and support staff.\n",
        "- At an individual level, experienced proprietary traders and quants use algorithmic trading. \n",
        "- Proprietary traders, who are less tech-savvy, may purchase readymade trading software for their algorithmic trading needs. \n",
        "- The software is either offered by their brokers or purchased from third-party providers.\n",
        "- Quants have a good knowledge of both trading and computer programming, and they develop trading software on their own. \n",
        "\n",
        "![alt text](https://thumbs.dreamstime.com/z/biggest-banks-world-logos-high-quality-vector-collection-eps-file-available-n-78364221.jpg)\n",
        "\n",
        "for example https://www.glassdoor.com/job-listing/algorithmic-trading-developer-barclays-JV_KO0,29_KE30,38.htm?jl=3142875124&ctt=1556750201749&srs=EI_JOBS \n",
        "\n",
        "\n",
        "## 10 Key features of a profitable trading bot (identifying a strategy)\n",
        "\n",
        "Use this as a checklist as you develop a trading strategy. If our algorithm covers all of these points, we're good. \n",
        "\n",
        "## 1 - Market selection: is this the market you want to trade?\n",
        "## 2- Market direction: what direction is the market moving, if any?\n",
        "## 3 - Setup: what are the conditions required to be present before you enter or exit?\n",
        "## 4- Entry: when should you open a position?\n",
        "## 5- Protective stop-loss: how would you know when to exit to preserve your capital?\n",
        "## 6 - Re-entry: how do you re-enter a trade if you’re stopped out of a good move?\n",
        "## 7 - Profit-booking: under what conditions do you take profits?\n",
        "## 8 - Money management: knowing that when wrong you lose X amount of capital, how big a position are you willing to take?\n",
        "## 9 - Portfolio selection: what basket of commodities, stocks or assets do you want to trade?\n",
        "## 10 - Multiple systems: in order to have a smooth performance, do you require multiple trading systems?\n",
        "\n",
        "## Then we backtest -- analyze the strategies performance on historical data and remove biases\n",
        "\n",
        "\n",
        "- How do you decide if the strategy you chose was good or bad? How do you judge your hypothesis?\n",
        "\n",
        "- This is where backtesting the strategy comes as an essential tool for the estimation of the performance of the designed hypothesis based on historical data.\n",
        "\n",
        "- A strategy can be considered to be good if the backtest results and performance statistics back the hypothesis. Hence, it is important to choose historical data with a sufficient number of data points.\n",
        "\n",
        "\n",
        "![alt text](https://www.coensio.com/images/Backtesting%20in%20mt4.png)\n",
        "\n",
        "## Lastly, we execute the strategy, link to a brokerage and minimize the transaction costs.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JI0l3kOvmCpw",
        "colab_type": "text"
      },
      "source": [
        "### The general idea has been to choose a strategy paradigm (hedging, execution based, alpha generating, etc) then create an algorithm for that paradigm. But we're going to use machine learning to learn the strategy paradigm. Our focus is not on the theory of algorithmic strategy but instead on data quality and machine learning theory.  "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1tdxQLFZyKR9",
        "colab_type": "text"
      },
      "source": [
        "## Architecture (Important Parts of the codebase)\n",
        "\n",
        "#### Vue.js (front end)\n",
        "\n",
        "![alt text](https://v1.vuejs.org/images/mvvm.png)\n",
        "\n",
        "#### Redux (state management)\n",
        "\n",
        "![alt text](https://cdn-images-1.medium.com/max/1600/1*87dJ5EB3ydD7_AbhKb4UOQ.png)\n",
        "\n",
        "#### TOML (Serialization)\n",
        "\n",
        "![alt text](http://genericgamedev.com/wp-content/uploads/2015/04/serialisation-header.jpg)\n",
        "\n",
        "- Database is MongoDB, postgres, and sqllite (options)\n",
        "- TOML format https://github.com/toml-lang/toml data , general.toml selects database type as sqlite \n",
        "- BudFox is the real time market\n",
        "- Exchange/wrappers contain HTTP request codes to get all the market data from the various exchanges via their respective JS APIs\n",
        "- It creates candles for more accurate predictions\n",
        "- Plugins store all the bulk of the code\n",
        "- It uses Vue Redux for the front end\n",
        "\n",
        "\n",
        "### Every Gekko instance has two core components:\n",
        "\n",
        "- A market\n",
        "- A GekkoStream\n",
        "\n",
        "### Communication between those two components is handled by Node.JS' Stream API. The Market implements a Readable Stream interface while the GekkoStream implements a Writeable Stream interface.\n",
        "\n",
        "- All markets in Gekko eventually output candle data. \n",
        "- Where these candles come from and how old they are does not matter to the GekkoStream they get piped into. On default Gekko looks for markets in the core/markets/ directory. \n",
        "- A GekkoStream is nothing more than a collection of plugins. Plugins are simple modules that can subscribe to events, and do something based on event data. The most basic event every GekkoStream has is the \"candle\" event, this event comes from the market.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bKIzyGk-7rul",
        "colab_type": "text"
      },
      "source": [
        "###  Installing Dependencies\n",
        "\n",
        "\n",
        "##### Step 1 - Install Gekko\n",
        "https://gekko.wizb.it/docs/installation/installing_gekko.html\n",
        "\n",
        "##### Step 2 - Install Convnet.js\n",
        "\n",
        "##### Step 3 - Install Tensorflow.js"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Z6fq2JuV9aRu",
        "colab_type": "text"
      },
      "source": [
        "### Technical Indicactors\n",
        "\n",
        "![alt text](https://www.visualcapitalist.com/wp-content/uploads/2017/05/tech-indicators-share.png)\n",
        "\n",
        "\n",
        "### Volume\n",
        "\n",
        "- Accumulation/Distribution Index (ADI)\n",
        "- On-Balance Volume (OBV)\n",
        "- On-Balance Volume mean (OBV mean)\n",
        "- Chaikin Money Flow (CMF)\n",
        "- Force Index (FI)\n",
        "- Ease of Movement (EoM, EMV)\n",
        "- Volume-price Trend (VPT)\n",
        "- Negative Volume Index (NVI)\n",
        "\n",
        "### Volatility\n",
        "\n",
        "- Average True Range (ATR)\n",
        "- Bollinger Bands (BB)\n",
        "- Keltner Channel (KC)\n",
        "- Donchian Channel (DC)\n",
        "\n",
        "### Trend\n",
        "\n",
        "- Moving Average Convergence Divergence (MACD)\n",
        "- Average Directional Movement Index (ADX)\n",
        "- Vortex Indicator (VI)\n",
        "- Trix (TRIX)\n",
        "- Mass Index (MI)\n",
        "- Commodity Channel Index (CCI)\n",
        "- Detrended Price Oscillator (DPO)\n",
        "- KST Oscillator (KST)\n",
        "- Ichimoku Kinkō Hyō (Ichimoku)\n",
        "\n",
        "### Momentum\n",
        "\n",
        "- Money Flow Index (MFI)\n",
        "- Relative Strength Index (RSI)\n",
        "- True strength index (TSI)\n",
        "- Ultimate Oscillator (UO)\n",
        "- Stochastic Oscillator (SR)\n",
        "- Williams %R (WR)\n",
        "- Awesome Oscillator (AO)\n",
        "\n",
        "### Others\n",
        "\n",
        "- Daily Return (DR)\n",
        "- Daily Log Return (DLR)\n",
        "- Cumulative Return (CR)\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DHqwQI02BIhQ",
        "colab_type": "text"
      },
      "source": [
        "### Trading Strategy (Deep Reinforcement Learning)\n",
        "\n",
        "### Partially Observable Markov Decision Process\n",
        "\n",
        "![alt text](https://slideplayer.com/slide/3007502/11/images/5/Markov+Decision+Processes+%28MDPs%29.jpg)\n",
        "![alt text](https://image.slidesharecdn.com/ml-sep-09-091009141615-phpapp01/95/regretbased-reward-elicitation-for-markov-decision-processes-39-728.jpg?cb=1255098159)\n",
        "![alt text](https://qph.fs.quoracdn.net/main-qimg-b059b262a8032e0c40346aa0f40a5678.webp)\n",
        "![alt text](https://www.researchgate.net/profile/Michael_Mccarthy10/publication/43479983/figure/fig2/AS:394256331100198@1471009442914/Partially-Observable-Markov-Decision-Process-iterative-belief-updating-procedure.png)\n",
        "![alt text](https://cdn-images-1.medium.com/max/1600/1*Hzql_1t0-wwDxiz0C97AcQ.png)\n",
        "\n",
        "## Actor Critic Network\n",
        "\n",
        "![alt text](https://sergioskar.github.io//assets/img/posts/DRL.jpg)\n",
        "![alt text](https://i.ytimg.com/vi/KHZVXao4qXs/maxresdefault.jpg)\n",
        "\n",
        "Actor Critic - We’ll using two neural networks:\n",
        "\n",
        "- a Critic that measures how good the action taken is (value-based)\n",
        "- an Actor that controls how our agent behaves (policy-based)\n",
        "\n",
        "### Mastering this architecture is essential to understanding state of the art algorithms such as Proximal Policy Optimization (aka PPO). PPO is based on Advantage Actor Critic. \n",
        "\n",
        "![alt text](https://cdn-images-1.medium.com/max/800/1*e1N-YzQmJt-5KwUkdUvAHg.png)\n",
        "\n",
        "Actor\n",
        "![alt text](https://cdn-images-1.medium.com/max/800/0*xoZipWE6lQgWyRh1.)\n",
        "Critic\n",
        "![alt text](https://cdn-images-1.medium.com/max/800/0*vQZrik2laT8hdRMb.)\n",
        "\n",
        "### Because we have two models (Actor and Critic) that must be trained, it means that we have two set of weights (𝜃 for our action and w for our Critic) that must be optimized separately:\n",
        "\n",
        "- At each time-step t, we take the current state (St) from the environment and pass it as an input through our Actor and our Critic.\n",
        "- Our Policy takes the state, outputs an action (At), and receives a new state (St+1) and a reward (Rt+1).\n",
        "-Thanks to that: the Critic computes the value of taking that action at that state, the Actor updates its policy parameters (weights) using this q value \n",
        "- Thanks to its updated parameters, the Actor produces the next action to take at At+1 given the new state St+1. The Critic then updates its value parameters:\n",
        "\n",
        "![alt text](https://cdn-images-1.medium.com/max/800/1*0gZsoyvY01liRdZZXilZpA.png)\n",
        "\n",
        "\n",
        "### in the context of our problem\n",
        "\n",
        "- Environment - Time series data\n",
        "- Observation - sentiment, technical indicators, \n",
        "- State - Account Balance, \n",
        "- Action - buy, sell, hold\n",
        "- Reward - +1/-1 based on whether or not we made a profit from the action\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gp14MinxG2DD",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#simple example in python , yay openai gym\n",
        "\n",
        "import gym\n",
        "env = gym.make('CartPole-v0')\n",
        "for i_episode in range(20):\n",
        "    observation = env.reset()\n",
        "    for t in range(100):\n",
        "        env.render()\n",
        "        print(observation)\n",
        "        action = env.action_space.sample()\n",
        "        observation, reward, done, info = env.step(action)\n",
        "        if done:\n",
        "            print(\"Episode finished after {} timesteps\".format(t+1))\n",
        "            break\n",
        "env.close()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yEs-ARkGKK9d",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#!/usr/bin/env python\n",
        "\"\"\"Simple Sairen trading example using a random agent.\"\"\"\n",
        "from sairen import MarketEnv\n",
        "\n",
        "\n",
        "def main():\n",
        "    \"\"\"Create a market environment, instantiate a random agent, and run the agent for one episode.\"\"\"\n",
        "    env = MarketEnv(\"AAPL\", episode_steps=20)   # Apple stock, 1-second bars by default\n",
        "    agent = ActorCritic(env.action_space)       # Actions are continuous from -1 = go short to +1 = go long.  0 is go flat.  Sets absolute target position.\n",
        "    observation = env.reset()       # An observation is a numpy float array, values: time, bid, bidsize, ask, asksize, last, lastsize, lasttime, open, high, low, close, vwap, volume, open_interest, position, unrealized_gain\n",
        "    done = False\n",
        "    total_reward = 0.0              # Reward is the profit realized when a trade closes\n",
        "    while not done:\n",
        "        env.render()\n",
        "        observation, reward, done, info = env.step(agent.act(observation))\n",
        "        total_reward += reward\n",
        "\n",
        "    print('\\nTotal profit: {:.2f}'.format(total_reward))        # Sairen will automatically (try to) cancel open orders and close positions on exit\n",
        "\n",
        "\n",
        "class RandomAgent:\n",
        "    \"\"\"Agent that randomly samples the action space.\"\"\"\n",
        "    def __init__(self, action_space):\n",
        "        \"\"\":param gym.Space action_space: The Space to sample from.\"\"\"\n",
        "        self.action_space = action_space\n",
        "\n",
        "    def act(self, observation):\n",
        "        \"\"\":Return: a random action from the action space.\"\"\"\n",
        "        return self.action_space.sample()       # Here the observation is ignored, but a less-random agent would want it.\n",
        "\n",
        "\n",
        "if __name__ == \"__main__\":\n",
        "    main()"
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}