Covid-19 Death Rate Prediction DDW Task 1.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## SC04 Group 08 DDW 2D Task 1\n",
    "\n",
    "### Introduction:\n",
    "\n",
    "#### The appearance of COVID-19 has caused widespread death and devastation economically and socially on a scale not seen since the Spanish flu in 1918. As of November 2021, a record 5.17M deaths have been reported worldwide, and with the flu continuously spreading, this number will only increase. As such our group has created a Machine Learning Algorithm to help governments calculate the total death toll exacted by the virus.\n",
    "\n",
    "#### Our Machine Learning Algorithm will aim to predict the total deaths due to COVID-19 of any geographical area on Earth, as long as the necessary data from our determined variables are provided.\n",
    "\n",
    "\n",
    "##### Assumptions: \n",
    "1. We will assume that data used to train the machine learning algorithm is not affected by the geographical region it originates from.\n",
    "\n",
    "2. We will assume that the data taken is accurate, and not under/over-reported.\n",
    "\n",
    "3. We will assume that data columns that have large numbers of data values missing will have little effect on the calculation of total deaths."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Import statements for various python libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "import numpy as np\n",
    "import math"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Pre-processing\n",
    "\n",
    "#### Importing the dataset from .csv format to a pandas dataframe and displaying it. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_main = pd.read_csv(r\"owid-covid-data.csv\") ## 131707 rows x 65 columns\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Listing all the countries in the dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "series_main_countries = df_main['location'].drop_duplicates()  ## Series that contains all the country names: 237 countries\n",
    "ls_main_countries = series_main_countries.tolist() # list of the countries\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Preliminary filtering: Removing unwanted rows and columns based on preliminary filtering\n",
    "\n",
    "\n",
    "1. Several rows in the countries listing do not contain any countries data, or represents a sum of data from various countries which can distort results\n",
    "\n",
    "\n",
    "\n",
    "2. We have also observed that several columns in the dataset are missing large segments of data. We will remove these columns as they do not provide enough data for us to accurately predict or assume its effect on total deaths.\n",
    "\n",
    "\n",
    "3. We will remove several other columns which do not have a numerical value that can affect the total death (Continent, iso_code etc.)\n",
    "\n",
    "\n",
    "4. Finally we will remove rows which have any data missing in them. This is to ensure that the DataFrame we are working with is complete with values from all the remaining columns. As we are predicting the deaths based on the data provided by any geographical area, the classification of where the data comes from will be assumed to have little effect.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Removal of rows with overlapping data under the countries column\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "## Removing the rows that are not even countries: 10 values\n",
    "## Asia, Europe, European Union, High income, International, Low income, \n",
    "## lower middle income, North America, Upper middle income,World\n",
    "\n",
    "non_countries = ['Asia', 'Africa', 'Europe', 'European Union', 'High income', 'International', 'Low income', 'Lower middle income', 'North America', 'Upper middle income', 'World' ]\n",
    "df_main2 = df_main[df_main['location'].isin(non_countries) == False]\n",
    "ls_countries_cleaned = df_main2['location'].drop_duplicates().tolist()\n",
    "\n",
    "#display(df_main2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Checking how many missing data sets each specific column has.\n",
    "\n",
    " - Any column which has 10000 data sets missing will be removed as they will affect the building of the training model. 10000 missing data sets is too large for us to be able to reasonably assume or insert in any values that can be used to train the model accurately."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "all_features = ['total_cases', 'new_cases','new_cases_smoothed', 'total_deaths', 'new_deaths','new_deaths_smoothed', 'total_cases_per_million',\n",
    "       'new_cases_per_million', 'new_cases_smoothed_per_million','total_deaths_per_million', 'new_deaths_per_million',\n",
    "       'new_deaths_smoothed_per_million', 'reproduction_rate', 'icu_patients','icu_patients_per_million', 'hosp_patients','hosp_patients_per_million', 'weekly_icu_admissions',\n",
    "       'weekly_icu_admissions_per_million', 'weekly_hosp_admissions',\n",
    "       'weekly_hosp_admissions_per_million', 'new_tests', 'total_tests',\n",
    "       'total_tests_per_thousand', 'new_tests_per_thousand',\n",
    "       'new_tests_smoothed', 'new_tests_smoothed_per_thousand',\n",
    "       'positive_rate', 'tests_per_case', 'total_vaccinations',\n",
    "       'people_vaccinated', 'people_fully_vaccinated', 'total_boosters',\n",
    "       'new_vaccinations', 'new_vaccinations_smoothed',\n",
    "       'total_vaccinations_per_hundred', 'people_vaccinated_per_hundred',\n",
    "       'people_fully_vaccinated_per_hundred', 'total_boosters_per_hundred',\n",
    "       'new_vaccinations_smoothed_per_million', 'stringency_index',\n",
    "       'population', 'population_density', 'median_age', 'aged_65_older',\n",
    "       'aged_70_older', 'gdp_per_capita', 'extreme_poverty',\n",
    "       'cardiovasc_death_rate', 'diabetes_prevalence', 'female_smokers',\n",
    "       'male_smokers', 'handwashing_facilities', 'hospital_beds_per_thousand',\n",
    "       'life_expectancy', 'human_development_index']\n",
    "\n",
    "\n",
    "def check_miss_col(main_country_list, all_features):\n",
    "    display_dict = {}\n",
    "\n",
    "    for i in main_country_list:\n",
    "        display_dict[i] = main_country_list[i].isnull().sum(axis=0)\n",
    "        display_dict = {k: v for k, v in sorted(display_dict.items(), key=lambda item: item[1])}\n",
    "                "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "features_list = ['total_cases', 'new_cases',\n",
    "       'new_cases_smoothed', 'total_deaths', 'new_deaths',\n",
    "       'new_deaths_smoothed', 'total_cases_per_million',\n",
    "       'new_cases_per_million', 'new_cases_smoothed_per_million',\n",
    "       'total_deaths_per_million', 'new_deaths_per_million',\n",
    "       'new_deaths_smoothed_per_million', 'reproduction_rate', 'icu_patients',\n",
    "       'icu_patients_per_million', 'hosp_patients', \n",
    "        'new_tests', 'total_tests',\n",
    "       'total_tests_per_thousand', 'new_tests_per_thousand',\n",
    "       'new_tests_smoothed', 'new_tests_smoothed_per_thousand',\n",
    "       'positive_rate', 'tests_per_case', 'total_vaccinations',\n",
    "       'people_vaccinated', 'people_fully_vaccinated', \n",
    "        'new_vaccinations_smoothed',\n",
    "       'total_vaccinations_per_hundred', 'people_vaccinated_per_hundred',\n",
    "       'new_vaccinations_smoothed_per_million', 'stringency_index',\n",
    "       'population', 'population_density', 'median_age', 'aged_65_older',\n",
    "       'aged_70_older', 'gdp_per_capita', 'extreme_poverty',\n",
    "       'cardiovasc_death_rate', 'diabetes_prevalence', 'female_smokers',\n",
    "       'male_smokers', 'hospital_beds_per_thousand',\n",
    "       'life_expectancy', 'human_development_index','new_people_vaccinated_smoothed', 'new_people_vaccinated_smoothed_per_hundred']\n",
    "\n",
    "to_remove = ['new_vaccinations', 'people_fully_vaccinated_per_hundred', 'new_vaccinations', 'hosp_patients_per_million', 'total_boosters', \n",
    "             'total_boosters_per_hundred', 'weekly_hosp_admissions', 'weekly_hosp_admissions_per_million', \n",
    "             'weekly_icu_admissions', 'weekly_icu_admissions_per_million', 'handwashing_facilities', 'iso_code', 'continent',\n",
    "            'location', 'date', 'tests_units']\n",
    "\n",
    "def remove_empty_rows(uncleaned_countries, features_list, remove_list):\n",
    "    \n",
    "    uncleaned_countries = uncleaned_countries.drop(remove_list, axis=1)\n",
    "    for features in features_list:\n",
    "        uncleaned_countries = uncleaned_countries[uncleaned_countries[features].notna()]\n",
    "    return uncleaned_countries\n",
    "\n",
    "cleaned_countries = remove_empty_rows(df_main2, features_list, to_remove)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data selection:\n",
    "\n",
    "From the cleaned dataframe above with 48 variables, we use the cor function from pandas to determine which variables have the largest correlation to the total deaths that can be calculated by our Machine Learning Model.\n",
    "\n",
    "We then filter out which variables are the best from each catergory based on their correlation score."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "total_cases                0.992716\n",
       "total_deaths               1.000000\n",
       "total_tests                0.962401\n",
       "new_tests_smoothed         0.830114\n",
       "total_vaccinations         0.912164\n",
       "people_vaccinated          0.926536\n",
       "people_fully_vaccinated    0.889990\n",
       "population                 0.975168\n",
       "Name: total_deaths, dtype: float64"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cor = cleaned_countries.corr()\n",
    "cor_target = abs(cor[\"total_deaths\"])\n",
    "relevant_features = cor_target[cor_target>0.8] \n",
    "relevant_features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data Visualization\n",
    "\n",
    "We then use Seaborn to visualize the various variables of data we have selected. This is to see if we can pre-emptively determine any relationships between the variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "required_columns = ['total_cases', 'total_tests', 'people_vaccinated', 'population', 'total_deaths']\n",
    "df_4_vars = cleaned_countries[required_columns]\n",
    "df_4_vars.to_csv('df_to_check.csv')\n",
    "#sns.pairplot(df_4_vars)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Final Data-Cleaning Procedure\n",
    "\n",
    "We then remove the rest of the unwanted columns in the dataframe that do not have a substantial effect on the total deaths."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "relevant_features = ['total_cases', 'total_tests', 'people_vaccinated', 'population', 'total_deaths']\n",
    "removing_list = []\n",
    "def remove_useless_cols(cleaned_countries, relevant_features, features_list ):\n",
    "    for features in features_list:\n",
    "        if features not in relevant_features:\n",
    "            removing_list.append(features)\n",
    "    cleaned_countries = cleaned_countries.drop(removing_list, axis=1)\n",
    "    return cleaned_countries\n",
    "cleaned_data = remove_useless_cols(cleaned_countries, relevant_features, features_list) \n",
    "#display(cleaned_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Code from Cohort class necessary for Multiple Linear Regression¶\n",
    "\n",
    "The following cell contains all the code necessary for training and testing the Multiple Linear Regression machine learning model. The code is closely referred to from the week 9 Cohort materials.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "def normalize_z(dfin):\n",
    "    dfout = (dfin-dfin.mean(axis=0))/dfin.std(axis=0)\n",
    "    return dfout\n",
    "\n",
    "def get_features_targets(df, feature_names, target_names):\n",
    "    df_feature = df.loc[:,feature_names]\n",
    "    df_target = df.loc[:,target_names]\n",
    "    return df_feature, df_target\n",
    "\n",
    "def prepare_feature(df_feature):\n",
    "    cols = len(df_feature.columns)\n",
    "    #convert df to numpy array for feature matrix\n",
    "    feature = df_feature.to_numpy().reshape(-1,cols)\n",
    "    #to get the no of rows\n",
    "    m=df_feature.shape[0]\n",
    "    # to create a vector of ones, m rows\n",
    "    array1=np.ones((m,1))\n",
    "    #add the column one with the feature matrix\n",
    "    x=np.concatenate((array1,feature),axis=1)\n",
    "    \n",
    "    return x\n",
    "\n",
    "def prepare_target(df_target):\n",
    "    cols = len(df_target.columns)\n",
    "    #convert df to numpy array for target vector\n",
    "    target = df_target.to_numpy().reshape(-1,cols)\n",
    "    return target\n",
    "\n",
    "def predict(df_feature, beta):\n",
    "    norm_feature = normalize_z(df_feature)\n",
    "    X=prepare_feature(norm_feature)\n",
    "    return predict_norm(X,beta)\n",
    "\n",
    "def predict_norm(X, beta):\n",
    "    return np.matmul(X,beta)\n",
    "\n",
    "def split_data(df_feature, df_target, random_state=None, test_size=0.5):\n",
    "    indices = df_feature.index\n",
    "    if random_state !=None:\n",
    "        np.random.seed(random_state)\n",
    "    num_rows=len(indices)\n",
    "    k=int(test_size*num_rows)\n",
    "    test_indices=np.random.choice(indices,k,replace=False)\n",
    "    \n",
    "    indices=set(indices)\n",
    "    test_indices=set(test_indices)\n",
    "    train_indices=indices-test_indices\n",
    "    \n",
    "    df_feature_train=df_feature.loc[train_indices,:]\n",
    "    df_feature_test=df_feature.loc[test_indices,:]\n",
    "    df_target_train=df_target.loc[train_indices,:]\n",
    "    df_target_test=df_target.loc[test_indices,:]\n",
    "    return df_feature_train, df_feature_test, df_target_train, df_target_test\n",
    "  \n",
    "def r2_score(y, ypred):\n",
    "    \n",
    "    mean_y=np.mean(y)\n",
    "    \n",
    "    ss_res=np.sum((y-ypred)**2)\n",
    "    ss_tot=np.sum((y-mean_y)**2)\n",
    "    r_square=1-(ss_res/ss_tot)\n",
    "    print(len(ypred))\n",
    "    return r_square\n",
    "\n",
    "def adjusted_r2(y,r2,beta):\n",
    "\n",
    "    adj = 1-((1-r2)*(len(y)-1)/(len(y)-len(beta)-1))\n",
    "    return adj\n",
    "\n",
    "def mean_squared_error(target, pred):\n",
    "    n=target.shape[0]\n",
    "    \n",
    "    return (1/n)*np.sum((target-pred)**2)\n",
    "\n",
    "\n",
    "def root_mean_squared_error(target, pred):\n",
    "    n=target.shape[0]\n",
    "    \n",
    "    return math.sqrt((1/n)*np.sum((target-pred)**2))\n",
    "\n",
    "def mean_average_square(target, pred):\n",
    "    n=target.shape[0]\n",
    "    \n",
    "    return (1/n)*np.sum(abs(target-pred))\n",
    "\n",
    "def compute_cost(X, y, beta):\n",
    "    J = 0\n",
    "    m=X.shape[0]\n",
    "    yp=np.matmul(X,beta)\n",
    "    error=yp-y\n",
    "    J=(1/(2*m)) *np.matmul(error.T,error)\n",
    "    #print(J)\n",
    "\n",
    "    J = J[0][0]\n",
    "    return J\n",
    "\n",
    "\n",
    "def gradient_descent(X, y, beta, alpha, num_iters):\n",
    "    m=len(y)\n",
    "    J_storage=np.zeros(num_iters)\n",
    "    for n in range(num_iters):\n",
    "        #calculate yp,predicted value using the old beta\n",
    "        yp=np.matmul(X,beta)\n",
    "        #calculate the error yp-y\n",
    "        error=yp-y\n",
    "        #calculate the delta for beta\n",
    "        delta=np.matmul(X.T,error)\n",
    "        #caluclate the new beta\n",
    "        beta=beta-(alpha/m)*delta\n",
    "        #calculate J for the new beta\n",
    "        J_storage[n]=compute_cost(X,y,beta)\n",
    "    return beta, J_storage\n",
    "\n",
    "\n",
    "def gradient_descent(X, y, beta, alpha, num_iters):\n",
    "    m=len(y)\n",
    "    J_storage=np.zeros(num_iters)\n",
    "    for n in range(num_iters):\n",
    "        #calculate yp,predicted value using the old beta\n",
    "        yp=np.matmul(X,beta)\n",
    "        #calculate the error yp-y\n",
    "        error=yp-y\n",
    "        #calculate the delta for beta\n",
    "        delta=np.matmul(X.T,error)\n",
    "        #caluclate the new beta\n",
    "        beta=beta-(alpha/m)*delta\n",
    "        #calculate J for the new beta\n",
    "        J_storage[n]=compute_cost(X,y,beta)\n",
    "    return beta, J_storage"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Method for Evaluation of Machine Learning Model\n",
    "\n",
    "In order to validate our Machine learning Model, we have determined to use the following methods:\n",
    "\n",
    "1. Adjusted R-Squared: We have chosen Adjusted R-Squared as one of our methods of evaluation primarily because it ensures that only variables that are useful to the Machine Learning Algorithm are used in the model. Any excess variables which do not have a meaningful impact on the model will penalize the R-Squared Value and thus prevent Over-fitting. \n",
    "\n",
    "2. Root-Mean-Squared-Error: We have also chosen Root-Mean-Squared-Error as another method of evaluation as it can help us accurately determine the amount of error in between the predicted and actual values from the Machine Learning Model.\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### RMSE Algorithm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "import statistics\n",
    "\n",
    "def rmse_score(df_feature, df_target, beta, alpha, num_iters, random_state=100, test_size=0.5):\n",
    "    index = df_feature.index\n",
    "    index = list(index)\n",
    "    \n",
    "    rmse_scores = []\n",
    "    \n",
    "    df_features_train, df_features_test, df_target_train, df_target_test = split_data(df_feature, df_target, random_state, test_size)\n",
    "    \n",
    "    X = df_features_train\n",
    "    X = prepare_feature(X)\n",
    "    y = prepare_target(df_target_train)\n",
    "    \n",
    "    beta, J = gradient_descent(X, y, beta, alpha, num_iters)\n",
    "    pred = predict(df_features_test, beta)\n",
    "    target = prepare_target(df_target_test)\n",
    "    rmse = np.sqrt(mean_squared_error(target, pred).item(0))\n",
    "    rmse_scores.append(rmse)\n",
    "    \n",
    "    return statistics.mean(rmse_scores)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data-splitting and Adjusted R-Squared Calculation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1217\n",
      "Adjusted r2 value:  0.990253423841115\n",
      "RMSE Value: 0.10293399777896213\n"
     ]
    }
   ],
   "source": [
    "model_features = ['total_cases', 'people_vaccinated', 'population', 'total_tests']\n",
    "\n",
    "df_features, df_target = get_features_targets(cleaned_data, model_features, ['total_deaths'])\n",
    "df_features = normalize_z(df_features)\n",
    "df_target = normalize_z(df_target)\n",
    "n = len(model_features)\n",
    "rmse = rmse_score(df_features, df_target, np.ones((n+1,1)), 0.01, 1500, test_size=0.3)\n",
    "df_features_train, df_features_test, df_target_train, df_target_test = split_data(df_features, df_target, 100, 0.3)\n",
    "\n",
    "\n",
    "num_cols = len(model_features)+1\n",
    "df_features_train_z = normalize_z(df_features_train)\n",
    "X = prepare_feature(df_features_train_z)\n",
    "target = prepare_target(df_target_train)\n",
    "iterations = 3000\n",
    "alpha = 0.01\n",
    "\n",
    "beta = np.zeros((num_cols,1))\n",
    "beta, J_storage = gradient_descent(X, target, beta, alpha, iterations)\n",
    "\n",
    "\n",
    "pred = predict(df_features_test, beta)\n",
    "target = prepare_target(df_target_test)\n",
    "r2 = r2_score(target, pred)\n",
    "\n",
    "adj_r2 = adjusted_r2(target,r2,beta)\n",
    "\n",
    "print(\"Adjusted r2 value: \", adj_r2)\n",
    "print('RMSE Value:', rmse)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Analysis of Data:\n",
    "\n",
    "The adjusted R-Squared value and RMSE Values all indicate that our Machine Learning Model is relatively accurate and the results fit the model extremely well. However, there are concerns that the model may be overfitted due to the extremely high correlation that is not so commonly found.\n",
    "\n",
    "\n",
    "To double-check our data and results, we have thus decided to carry out a set of Linear Regressions between the Total Deaths and the Variables we have selected."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1217\n",
      "r2 value of all features 0.9859587777228399\n",
      "1217\n",
      "r2 value of total_cases 0.9859587777228399\n",
      "1217\n",
      "r2 value of people_vaccinated 0.8811938410264417\n",
      "1217\n",
      "r2 value of population 0.9628907388696735\n",
      "1217\n",
      "r2 value of total_tests 0.9165718091414302\n"
     ]
    }
   ],
   "source": [
    "def r2_score_train(df_feature, df_target, beta, alpha, num_iters, random_state=100, test_size=0.5):\n",
    "    df_features_train, df_features_test, df_target_train, df_target_test = split_data(df_feature, df_target, random_state, test_size)\n",
    "    df_features_train_z = normalize_z(df_features_train)\n",
    "    X = prepare_feature(df_features_train_z)\n",
    "    target = prepare_target(df_target_train)\n",
    "    beta, J = gradient_descent(X,target,beta,alpha,num_iters)\n",
    "    pred = predict(df_features_test,beta)\n",
    "    target = df_target_test.to_numpy()\n",
    "    r2 = r2_score(target,pred)\n",
    "\n",
    "    return r2,df_target_test,pred\n",
    "\n",
    "df_features, df_target = get_features_targets(cleaned_countries, ['total_cases'], ['total_deaths'])\n",
    "df_features = normalize_z(df_features)\n",
    "n = 1\n",
    "r2,df_target_test,pred = r2_score_train(df_features, df_target, np.ones((n+1,1)), 0.01, 1500, test_size=0.3)\n",
    "print(\"r2 value of all features\",r2)\n",
    "\n",
    "\n",
    "for column in model_features:\n",
    "    df_features, df_target = get_features_targets(cleaned_countries, [column], ['total_deaths'])\n",
    "    df_features = normalize_z(df_features)\n",
    "    n = 1\n",
    "    r2,df_target_test,pred = r2_score_train(df_features, df_target, np.ones((n+1,1)), 0.01, 1500, test_size=0.3)\n",
    "    print(\"r2 value of\" ,str(column),r2)\n",
    "    #plt.scatter(df_features_test[column],df_target_test)\n",
    "    #plt.scatter(df_features_test[column],pred)\n",
    "    #plt.show()\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    " #plt.plot(J_storage)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Final Analysis and Conclusions:\n",
    "\n",
    "Reviewing the Results, we have determined that the individual variables all have a very large correlation with the total deaths itself. This justifies the large R-Squared Values and Low RMSE Values we have obtained and is not a by-product of Over-fitting."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Further Improvements:\n",
    "\n",
    "To further improve the Model without Over-fitting, we will now add variables to our model and use the Adjusted R-Square and RMSE as metrics to determine if the variable added produces a more accurate model. We have determined that ICU Patients are the only additional unique variable within 0.6 correlation score that can be added."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1217\n",
      "Adjusted r2 value:  0.9905207814865252\n",
      "RMSE Value: 0.10962301960392118\n"
     ]
    }
   ],
   "source": [
    "model_features = ['total_cases', 'people_vaccinated', 'population', 'total_tests', 'icu_patients']\n",
    "\n",
    "\n",
    "df_features, df_target = get_features_targets(cleaned_countries, model_features, ['total_deaths'])\n",
    "df_features = normalize_z(df_features)\n",
    "df_target = normalize_z(df_target)\n",
    "n = len(model_features)\n",
    "rmse = rmse_score(df_features, df_target, np.ones((n+1,1)), 0.01, 1500, test_size=0.3)\n",
    "df_features_train, df_features_test, df_target_train, df_target_test = split_data(df_features, df_target, 100, 0.3)\n",
    "\n",
    "\n",
    "num_cols = len(model_features)+1\n",
    "df_features_train_z = normalize_z(df_features_train)\n",
    "X = prepare_feature(df_features_train_z)\n",
    "target = prepare_target(df_target_train)\n",
    "iterations = 3000\n",
    "alpha = 0.01\n",
    "\n",
    "beta = np.zeros((num_cols,1))\n",
    "beta, J_storage = gradient_descent(X, target, beta, alpha, iterations)\n",
    "\n",
    "\n",
    "pred = predict(df_features_test, beta)\n",
    "target = prepare_target(df_target_test)\n",
    "r2 = r2_score(target, pred)\n",
    "\n",
    "adj_r2 = adjusted_r2(target,r2,beta)\n",
    "\n",
    "print(\"Adjusted r2 value: \", adj_r2)\n",
    "print('RMSE Value:', rmse)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Analysis of Result:\n",
    "\n",
    "While the Adjusted R-Squared value has increased, the RMSE value decreases and thus we cannot make a clear decision on whether the ICU_Patients has a positive effect on the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1217\n",
      "Adjusted r2 value:  0.990253423841115\n",
      "RMSE Value: 0.10293399777896213\n"
     ]
    }
   ],
   "source": [
    "model_features = ['total_cases', 'people_vaccinated', 'population', 'total_tests']\n",
    "\n",
    "\n",
    "df_features, df_target = get_features_targets(cleaned_countries, model_features, ['total_deaths'])\n",
    "df_features = normalize_z(df_features)\n",
    "df_target = normalize_z(df_target)\n",
    "n = len(model_features)\n",
    "rmse = rmse_score(df_features, df_target, np.ones((n+1,1)), 0.01, 1500, test_size=0.3)\n",
    "df_features_train, df_features_test, df_target_train, df_target_test = split_data(df_features, df_target, 100, 0.3)\n",
    "\n",
    "\n",
    "num_cols = len(model_features)+1\n",
    "df_features_train_z = normalize_z(df_features_train)\n",
    "X = prepare_feature(df_features_train_z)\n",
    "target = prepare_target(df_target_train)\n",
    "iterations = 3000\n",
    "alpha = 0.01\n",
    "\n",
    "beta = np.zeros((num_cols,1))\n",
    "beta, J_storage = gradient_descent(X, target, beta, alpha, iterations)\n",
    "\n",
    "\n",
    "pred = predict(df_features_test, beta)\n",
    "target = prepare_target(df_target_test)\n",
    "r2 = r2_score(target, pred)\n",
    "\n",
    "adj_r2 = adjusted_r2(target,r2,beta)\n",
    "\n",
    "print(\"Adjusted r2 value: \", adj_r2)\n",
    "print('RMSE Value:', rmse)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Final Analysis and Conclusion\n",
    "\n",
    "The current Iteration of the Model is quite optimal. Any additional variables we can add to the model have a lower correlation value with total deaths and will likely reduce the performance of the model. Existing variables that have a high correlation with total deaths will overlap directly with the main variables we have chosen and definitely result in over-fitting of the model. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### References\n",
    "\n",
    "https://www.statisticshowto.com/probability-and-statistics/statistics-definitions/adjusted-r2/\n",
    "\n",
    "https://www.statisticshowto.com/probability-and-statistics/regression-analysis/rmse-root-mean-square-error/\n",
    "\n",
    "https://github.com/owid/covid-19-data/tree/master/public/data\n",
    "\n",
    "\n"
   ]
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "3d199f5cb40e9670dee91e6f716085b387e29e9694906bd8ddba516105d9b31f"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}