Concrete_Strength2.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Machine Learning with Concrete Strength\n",
    "\n",
    "Concrete strength is affected by factors such as water to cement ratio, raw material quality, the ratio of coarse or fine aggregate, concrete age, concrete compaction, temperature, relative humidity, and other factors during the curing of the concrete. The data includes the following information for 1030 concrete samples.\n",
    "\n",
    "- **Input variables:**\n",
    "  - Cement: kg/m$^3$ mixture\n",
    "  - Blast Furnace Slag: kg/m$^3$ mixture\n",
    "  - Fly Ash: kg/m$^3$ mixture\n",
    "  - Water: kg/m$^3$ mixture\n",
    "  - Superplasticizer: kg/m$^3$ mixture\n",
    "  - Coarse Aggregate: kg/m$^3$ mixture\n",
    "  - Fine Aggregate: kg/m$^3$ mixture\n",
    "  - Age: Day (1~365)\n",
    "- **Output variable:**\n",
    "  - Concrete compressive strength: MPa\n",
    "\n",
    "```python\n",
    "url = 'http://apmonitor.com/pds/uploads/Main/cement_strength.txt'\n",
    "```\n",
    "\n",
    "![Concrete Strength](http://apmonitor.com/pds/uploads/Main/concrete_strength_machine_learning.png)\n",
    "\n",
    "The full problem statement is on the [Machine Learning for Engineers course website](http://apmonitor.com/pds/index.php/Main/CementStrength)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Import Packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "from matplotlib import cm\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import plot_confusion_matrix\n",
    "from sklearn.feature_selection import SelectKBest, f_classif, f_regression\n",
    "\n",
    "# Import classifier models\n",
    "from sklearn.linear_model import LogisticRegression # Logistic Regression\n",
    "from sklearn.naive_bayes import GaussianNB # Naïve Bayes\n",
    "from sklearn.linear_model import SGDClassifier # Stochastic Gradient Descent\n",
    "from sklearn.neighbors import KNeighborsClassifier # K-Nearest Neighbors\n",
    "from sklearn.tree import DecisionTreeClassifier # Decision Tree\n",
    "from sklearn.ensemble import RandomForestClassifier # Random Forest\n",
    "from sklearn.svm import SVC # Support Vector Classifier\n",
    "from sklearn.neural_network import MLPClassifier # Neural Network\n",
    "\n",
    "# Import regression models\n",
    "from sklearn.linear_model import LinearRegression\n",
    "from sklearn.neural_network import MLPRegressor\n",
    "from sklearn.neighbors import KNeighborsRegressor\n",
    "from sklearn import svm\n",
    "import statsmodels.api as sm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Import Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "url = 'http://apmonitor.com/pds/uploads/Main/cement_strength.txt'\n",
    "data = pd.read_csv(url)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Pair Plot of Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Divide Data between High and Low Strength"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 1: Data Visualization and Cleansing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Summary Statistics\n",
    "\n",
    "Generate summary information to [statistically describe the data](https://apmonitor.com/pds/index.php/Main/StatisticsMath)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Check for balanced classification dataset ('high' and 'low' strength should have about equal amounts)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-02-08T15:59:44.995391Z",
     "start_time": "2021-02-08T15:59:44.733837Z"
    },
    "scrolled": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Convert String Categories (Text) to Binary (0 or 1)\n",
    "\n",
    "One-hot encoding translates character labels into a binary representation (0 or 1) for classification. Investigate the data types with `data.dtypes`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data Cleansing\n",
    "\n",
    "There is one row that contains an outlier. Identify the outlier with boxplots."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "View outliers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Remove rows that contain outliers. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Verify that the outliers are removed with another box plot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data Correlation\n",
    "\n",
    "Generate a heat map of the data correlation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data Distributions and Pair Plot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 2: Classification\n",
    "\n",
    "Train and test a classifier to distinguish between high and low strength concrete. Test at least 8 classifiers of your choice. Recommend a best classifier among the 8 that are tested. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Divide Input Features (X) and Output Label (y)\n",
    "\n",
    "Divide the data into sets that contain the input features (X) and output label (y=`csMPa`). Save data feature columns with `X_names=list(data.columns)` and remove `strength` with `X_names.remove('strength')` and remove `csMPa` with `X_names.remove('csMPa')`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-02-08T15:20:12.429259Z",
     "start_time": "2021-02-08T15:20:09.844Z"
    },
    "scrolled": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data scaling\n",
    "\n",
    "Scale the input features with a `StandardScaler` or a `MinMaxScaler`. Why do classifiers return an error if the output label is scaled with `StandardScaler`?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-02-08T16:00:38.701234Z",
     "start_time": "2021-02-08T16:00:31.664225Z"
    }
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Answer: The output label should not be scaled because it needs to be categorical data as an integer instead of a continuous real number."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Train / Test Split\n",
    "\n",
    "Randomly select values that split the data into a train (80%) and test (20%) set by using the sklearn `train_test_split` function with `shuffle=True`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-02-08T15:20:12.439264Z",
     "start_time": "2021-02-08T15:20:09.849Z"
    }
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Evaluate the Best Features\n",
    "\n",
    "Use `SelectKBest` to evaluate the best features for the classifier."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-02-08T15:20:12.446257Z",
     "start_time": "2021-02-08T15:20:09.859Z"
    },
    "scrolled": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Train (fit) and Test Classification with Logistic Regression "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-02-08T15:20:12.441259Z",
     "start_time": "2021-02-08T15:20:09.851Z"
    },
    "scrolled": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-02-08T15:20:12.443259Z",
     "start_time": "2021-02-08T15:20:09.853Z"
    }
   },
   "source": [
    "### Train 8 Classifiers\n",
    "\n",
    "Create 8 classifier objects and train."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-02-08T15:20:12.444259Z",
     "start_time": "2021-02-08T15:20:09.854Z"
    }
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Classifier Evaluation\n",
    "\n",
    "Report the confusion matrix on the test set for each classifier. Discuss the performance of each. A confusion matrix shows correct classification (diagonals) and incorrect classification (off-diagonals) groups from the test set. Generate a confusion matrix for each classifier."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-02-08T15:20:12.445258Z",
     "start_time": "2021-02-08T15:20:09.857Z"
    }
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 3: Regression\n",
    "\n",
    "Develop a regression model to predict Tension Strength (MPa). Compare predicted concrete strength with the regression model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Scale Data\n",
    "\n",
    "Scale `data` with `StandardScaler` or `MinMaxScaler`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Select Input Features (X) and Output Label (y)\n",
    "\n",
    "Using the 8 concrete properties as the input features.\n",
    "\n",
    "- Cement: kg/m$^3$ mixture\n",
    "- Blast Furnace Slag: kg/m$^3$ mixture\n",
    "- Fly Ash: kg/m$^3$ mixture\n",
    "- Water: kg/m$^3$ mixture\n",
    "- Superplasticizer: kg/m$^3$ mixture\n",
    "- Coarse Aggregate: kg/m$^3$ mixture\n",
    "- Fine Aggregate: kg/m$^3$ mixture\n",
    "- Age: Day (1~365)\n",
    "\n",
    "The output label is the `csMPa`.\n",
    "\n",
    "- Concrete Strength (MPa)\n",
    "\n",
    "Divide the data into sets that contain the input features (X) and output label (y=`csMPa`). Save data feature columns with `X_names=list(data.columns)[0:8]`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Select Best Features for Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Split Data\n",
    "\n",
    "Randomly select values that split the data into a train (80%) and test (20%) set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Regression Fit\n",
    "\n",
    "Use 3 regression methods. Use Linear Regression, Neural Network (Deep Learning), and another regression method of your choice. Discuss the performance of each. Possible regression methods are:\n",
    "\n",
    "- Linear Regression\n",
    "- Neural Network (Deep Learning)\n",
    "- K-Nearest Neighbors\n",
    "- Support Vector Regressor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Validation\n",
    "\n",
    "Report the correlation coefficient ($R^2$) for the train and test sets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Parity Plot\n",
    "\n",
    "A parity plot is a scatter plot with predicted versus measured. A parity plot of the training and test data is a good way to see the overall fit of tension strength."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A joint plot shows two variables, with the univariate and joint distributions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "hide_input": false,
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}