LinearRegression.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Week 1 - Exercise 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Author: Khushee Kapoor\n",
    "\n",
    "Last Updated: 20/03/2022\n",
    "\n",
    "In this notebook, I have implemented the Linear Regression algorithm using the Scikit-Learn library in Python to predict the median value of a house by training the model on the Boston Housing dataset."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Meta-Data about Boston Dataset:**\n",
    "\n",
    "- crim - per capita crime rate by town\n",
    "- zn - proportion of residential land zoned for lots over 25,000 sq.ft.\n",
    "- indus - proportion of non-retail business acres per town.\n",
    "- chas - Charles River dummy variable (1 if tract bounds river; 0 otherwise)\n",
    "- nox - nitric oxides concentration (parts per 10 million)\n",
    "- rm - average number of rooms per dwelling\n",
    "- age - proportion of owner-occupied units built prior to 1940\n",
    "- dis - weighted distances to five Boston employment centres\n",
    "- rad - index of accessibility to radial highways\n",
    "- tax - full-value property-tax rate per 10,000 usd\n",
    "- ptratio - pupil-teacher ratio by town\n",
    "- black - 1000(Bk - 0.63)^2 where Bk is the proportion of blacks by town\n",
    "- lstat - % lower status of the population\n",
    "- medv - Median value of owner-occupied homes in 1000's usd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Setting Up"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To start, we have imported the following libraries:\n",
    "\n",
    "- NumPy: to implement the Linear Regression algorithm\n",
    "- Pandas: to manipulate the dataframe\n",
    "- MatPlotLib: for data visualization\n",
    "- Seaborn: for data visulization"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# importing the libraries\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "from matplotlib import pyplot as plt\n",
    "import seaborn as sns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Q1. Use the boston.csv dataset and determine the best 5 features to predict ‘MEDV’. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To solve Question 1, we read the boston housing csv file into a dataframe using the read_csv() function from the Pandas library."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# reading the dataset into a dataframe\n",
    "df = pd.read_csv('Boston.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data Preprocessing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before we begin, let us perform some basic exploratory data analysis. First, we view a glimpse of the dataframe using the head() function from the Pandas library."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>crim</th>\n",
       "      <th>zn</th>\n",
       "      <th>indus</th>\n",
       "      <th>chas</th>\n",
       "      <th>nox</th>\n",
       "      <th>rm</th>\n",
       "      <th>age</th>\n",
       "      <th>dis</th>\n",
       "      <th>rad</th>\n",
       "      <th>tax</th>\n",
       "      <th>ptratio</th>\n",
       "      <th>black</th>\n",
       "      <th>lstat</th>\n",
       "      <th>medv</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0.00632</td>\n",
       "      <td>18.0</td>\n",
       "      <td>2.31</td>\n",
       "      <td>0</td>\n",
       "      <td>0.538</td>\n",
       "      <td>6.575</td>\n",
       "      <td>65.2</td>\n",
       "      <td>4.0900</td>\n",
       "      <td>1</td>\n",
       "      <td>296</td>\n",
       "      <td>15.3</td>\n",
       "      <td>396.90</td>\n",
       "      <td>4.98</td>\n",
       "      <td>24.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>0.02731</td>\n",
       "      <td>0.0</td>\n",
       "      <td>7.07</td>\n",
       "      <td>0</td>\n",
       "      <td>0.469</td>\n",
       "      <td>6.421</td>\n",
       "      <td>78.9</td>\n",
       "      <td>4.9671</td>\n",
       "      <td>2</td>\n",
       "      <td>242</td>\n",
       "      <td>17.8</td>\n",
       "      <td>396.90</td>\n",
       "      <td>9.14</td>\n",
       "      <td>21.6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>0.02729</td>\n",
       "      <td>0.0</td>\n",
       "      <td>7.07</td>\n",
       "      <td>0</td>\n",
       "      <td>0.469</td>\n",
       "      <td>7.185</td>\n",
       "      <td>61.1</td>\n",
       "      <td>4.9671</td>\n",
       "      <td>2</td>\n",
       "      <td>242</td>\n",
       "      <td>17.8</td>\n",
       "      <td>392.83</td>\n",
       "      <td>4.03</td>\n",
       "      <td>34.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>0.03237</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.18</td>\n",
       "      <td>0</td>\n",
       "      <td>0.458</td>\n",
       "      <td>6.998</td>\n",
       "      <td>45.8</td>\n",
       "      <td>6.0622</td>\n",
       "      <td>3</td>\n",
       "      <td>222</td>\n",
       "      <td>18.7</td>\n",
       "      <td>394.63</td>\n",
       "      <td>2.94</td>\n",
       "      <td>33.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>0.06905</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.18</td>\n",
       "      <td>0</td>\n",
       "      <td>0.458</td>\n",
       "      <td>7.147</td>\n",
       "      <td>54.2</td>\n",
       "      <td>6.0622</td>\n",
       "      <td>3</td>\n",
       "      <td>222</td>\n",
       "      <td>18.7</td>\n",
       "      <td>396.90</td>\n",
       "      <td>5.33</td>\n",
       "      <td>36.2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0     crim    zn  indus  chas    nox     rm   age     dis  rad  \\\n",
       "0           1  0.00632  18.0   2.31     0  0.538  6.575  65.2  4.0900    1   \n",
       "1           2  0.02731   0.0   7.07     0  0.469  6.421  78.9  4.9671    2   \n",
       "2           3  0.02729   0.0   7.07     0  0.469  7.185  61.1  4.9671    2   \n",
       "3           4  0.03237   0.0   2.18     0  0.458  6.998  45.8  6.0622    3   \n",
       "4           5  0.06905   0.0   2.18     0  0.458  7.147  54.2  6.0622    3   \n",
       "\n",
       "   tax  ptratio   black  lstat  medv  \n",
       "0  296     15.3  396.90   4.98  24.0  \n",
       "1  242     17.8  396.90   9.14  21.6  \n",
       "2  242     17.8  392.83   4.03  34.7  \n",
       "3  222     18.7  394.63   2.94  33.4  \n",
       "4  222     18.7  396.90   5.33  36.2  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# viewing the first 5 rows of the dataframe\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we view a summary of the attributes using the describe() function from the Pandas library."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>crim</th>\n",
       "      <th>zn</th>\n",
       "      <th>indus</th>\n",
       "      <th>chas</th>\n",
       "      <th>nox</th>\n",
       "      <th>rm</th>\n",
       "      <th>age</th>\n",
       "      <th>dis</th>\n",
       "      <th>rad</th>\n",
       "      <th>tax</th>\n",
       "      <th>ptratio</th>\n",
       "      <th>black</th>\n",
       "      <th>lstat</th>\n",
       "      <th>medv</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>506.000000</td>\n",
       "      <td>506.000000</td>\n",
       "      <td>506.000000</td>\n",
       "      <td>506.000000</td>\n",
       "      <td>506.000000</td>\n",
       "      <td>506.000000</td>\n",
       "      <td>506.000000</td>\n",
       "      <td>506.000000</td>\n",
       "      <td>506.000000</td>\n",
       "      <td>506.000000</td>\n",
       "      <td>506.000000</td>\n",
       "      <td>506.000000</td>\n",
       "      <td>506.000000</td>\n",
       "      <td>506.000000</td>\n",
       "      <td>506.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>253.500000</td>\n",
       "      <td>3.613524</td>\n",
       "      <td>11.363636</td>\n",
       "      <td>11.136779</td>\n",
       "      <td>0.069170</td>\n",
       "      <td>0.554695</td>\n",
       "      <td>6.284634</td>\n",
       "      <td>68.574901</td>\n",
       "      <td>3.795043</td>\n",
       "      <td>9.549407</td>\n",
       "      <td>408.237154</td>\n",
       "      <td>18.455534</td>\n",
       "      <td>356.674032</td>\n",
       "      <td>12.653063</td>\n",
       "      <td>22.532806</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>146.213884</td>\n",
       "      <td>8.601545</td>\n",
       "      <td>23.322453</td>\n",
       "      <td>6.860353</td>\n",
       "      <td>0.253994</td>\n",
       "      <td>0.115878</td>\n",
       "      <td>0.702617</td>\n",
       "      <td>28.148861</td>\n",
       "      <td>2.105710</td>\n",
       "      <td>8.707259</td>\n",
       "      <td>168.537116</td>\n",
       "      <td>2.164946</td>\n",
       "      <td>91.294864</td>\n",
       "      <td>7.141062</td>\n",
       "      <td>9.197104</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.006320</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.460000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.385000</td>\n",
       "      <td>3.561000</td>\n",
       "      <td>2.900000</td>\n",
       "      <td>1.129600</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>187.000000</td>\n",
       "      <td>12.600000</td>\n",
       "      <td>0.320000</td>\n",
       "      <td>1.730000</td>\n",
       "      <td>5.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>127.250000</td>\n",
       "      <td>0.082045</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>5.190000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.449000</td>\n",
       "      <td>5.885500</td>\n",
       "      <td>45.025000</td>\n",
       "      <td>2.100175</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>279.000000</td>\n",
       "      <td>17.400000</td>\n",
       "      <td>375.377500</td>\n",
       "      <td>6.950000</td>\n",
       "      <td>17.025000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>253.500000</td>\n",
       "      <td>0.256510</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>9.690000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.538000</td>\n",
       "      <td>6.208500</td>\n",
       "      <td>77.500000</td>\n",
       "      <td>3.207450</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>330.000000</td>\n",
       "      <td>19.050000</td>\n",
       "      <td>391.440000</td>\n",
       "      <td>11.360000</td>\n",
       "      <td>21.200000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>379.750000</td>\n",
       "      <td>3.677082</td>\n",
       "      <td>12.500000</td>\n",
       "      <td>18.100000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.624000</td>\n",
       "      <td>6.623500</td>\n",
       "      <td>94.075000</td>\n",
       "      <td>5.188425</td>\n",
       "      <td>24.000000</td>\n",
       "      <td>666.000000</td>\n",
       "      <td>20.200000</td>\n",
       "      <td>396.225000</td>\n",
       "      <td>16.955000</td>\n",
       "      <td>25.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>506.000000</td>\n",
       "      <td>88.976200</td>\n",
       "      <td>100.000000</td>\n",
       "      <td>27.740000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.871000</td>\n",
       "      <td>8.780000</td>\n",
       "      <td>100.000000</td>\n",
       "      <td>12.126500</td>\n",
       "      <td>24.000000</td>\n",
       "      <td>711.000000</td>\n",
       "      <td>22.000000</td>\n",
       "      <td>396.900000</td>\n",
       "      <td>37.970000</td>\n",
       "      <td>50.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       Unnamed: 0        crim          zn       indus        chas         nox  \\\n",
       "count  506.000000  506.000000  506.000000  506.000000  506.000000  506.000000   \n",
       "mean   253.500000    3.613524   11.363636   11.136779    0.069170    0.554695   \n",
       "std    146.213884    8.601545   23.322453    6.860353    0.253994    0.115878   \n",
       "min      1.000000    0.006320    0.000000    0.460000    0.000000    0.385000   \n",
       "25%    127.250000    0.082045    0.000000    5.190000    0.000000    0.449000   \n",
       "50%    253.500000    0.256510    0.000000    9.690000    0.000000    0.538000   \n",
       "75%    379.750000    3.677082   12.500000   18.100000    0.000000    0.624000   \n",
       "max    506.000000   88.976200  100.000000   27.740000    1.000000    0.871000   \n",
       "\n",
       "               rm         age         dis         rad         tax     ptratio  \\\n",
       "count  506.000000  506.000000  506.000000  506.000000  506.000000  506.000000   \n",
       "mean     6.284634   68.574901    3.795043    9.549407  408.237154   18.455534   \n",
       "std      0.702617   28.148861    2.105710    8.707259  168.537116    2.164946   \n",
       "min      3.561000    2.900000    1.129600    1.000000  187.000000   12.600000   \n",
       "25%      5.885500   45.025000    2.100175    4.000000  279.000000   17.400000   \n",
       "50%      6.208500   77.500000    3.207450    5.000000  330.000000   19.050000   \n",
       "75%      6.623500   94.075000    5.188425   24.000000  666.000000   20.200000   \n",
       "max      8.780000  100.000000   12.126500   24.000000  711.000000   22.000000   \n",
       "\n",
       "            black       lstat        medv  \n",
       "count  506.000000  506.000000  506.000000  \n",
       "mean   356.674032   12.653063   22.532806  \n",
       "std     91.294864    7.141062    9.197104  \n",
       "min      0.320000    1.730000    5.000000  \n",
       "25%    375.377500    6.950000   17.025000  \n",
       "50%    391.440000   11.360000   21.200000  \n",
       "75%    396.225000   16.955000   25.000000  \n",
       "max    396.900000   37.970000   50.000000  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# viewing a summary of the attributes\n",
    "df.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To further dive into column descriptions, let's also plot a boxplot for all the columns using the boxplot() function from the Pandas library."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5, 0, 'Columns')"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlYAAAFNCAYAAADCXCHaAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAA5NUlEQVR4nO3dfXxU9Z33/9cnCQQkyI1oitBCt/W3C9KtNtZfvWlLZAuo9Wa3WARbAalUW1L7s62C9NrWva6saKu7vUCsN1BplSBgvQMBXQy6traKrXeQUmkRG8tq1QgmQiDJ5/fHORMmITcncCZzhryfj0cec+Y73znzzpyZM5/5njPnmLsjIiIiIocvL9sBRERERI4UKqxEREREYqLCSkRERCQmKqxEREREYqLCSkRERCQmKqxEREREYqLCSkR6LDNzM/t4tnOIyJFDhZWIJIKZvWZme8ys1sxqzGyNmX0427kAzGy6mT2d7RwiknwqrEQkSc5z9yJgKPAmsCDLeUREukSFlYgkjrvvBVYBowHMbICZ/dzM/mZmO8zs+2aWZ2aDzazazM4L+xWZ2TYzuzS8freZ/dTMHjez983sSTMb0dZjdvAYo4CfAqeFo2nvdcuTICI5SYWViCSOmR0FTAZ+EzYtAAYAfwd8HrgUmOHu7wKXAXea2XHAfwAvuPvP02Z3CfC/gSHAC8C97Txse49RBVwBPOPuRe4+MKZ/U0SOQAXZDiAikuZBM2sAioC3gAlmlk9QZJ3s7u8D75vZzcBXgcXu/piZrQQ2AMcAn2g1zzXu/hSAmc0DdpnZh939L6kOnT1GJv9hETmyaMRKRJLkwnBEqBCYDTwJDAd6AzvS+u0AhqVdvwMYA/zM3d9pNc/mAsrda4F3geNb9RkS4TFERDqlwkpEEsfdG939l0Aj8BlgP5C+b9RHgDegebTpduDnwJVtHD6h+ZeFZlYEDAb+2qrP2x09BuCH8/+ISM+hwkpEEscCFwCDgFeAFUC5mfUPdz6/Grgn7H5deHkZ8GPg52GxlXKOmZ1pZr0J9rX6bfpmQAgKuU4e401geDgPEZF2qbASkSR5xMxqgd1AOTDN3TcDZUAd8GfgaWAZsMTMSggKoEvD4uhGgtGlOWnzXAb8gGATYAnBzuxtafMxwtueADYD/2Nmb8fzr4rIkcjcNcItIkcmM7sbqHb372c7i4j0DBqxEhEREYmJCisRERGRmGhToIiIiEhMNGIlIiIiEhMVViIiIiIxScQpbYYMGeIjR46MZV51dXX069cvlnnFRZmiS2IuZYpGmaJLYi5likaZoktirrgyPf/882+7+7Ft3ujuWf8rKSnxuFRWVsY2r7goU3RJzKVM0ShTdEnMpUzRKFN0ScwVVyZgk7dT02hToIiIiEhMVFiJiIiIxESFlYiIiEhMVFiJiIiIxESFlYiIiEhMVFiJiIiIxESFlYiIiEhMVFiJiIiIxESFlYiIiEhMVFiJiIiIxKTTwsrM/t7MXkj7221m3zazwWb2uJm9Gl4OSrvPXDPbZmZbzWxCZv8FERHJdWZ20F9paWmb7SJJ1mlh5e5b3f0kdz8JKAE+AB4A5gAb3P0EYEN4HTMbDVwMnAhMBBaZWX5m4ouIyJGgrXOujbh2dZvtIknW1U2B44A/ufsO4AJgadi+FLgwnL4AWO7u9e6+HdgGnBpDVhEREZFE62phdTFQEU4Xu/tOgPDyuLB9GPCXtPtUh20iIiIiRzSLOqxqZr2BvwInuvubZvaeuw9Mu73G3QeZ2a3AM+5+T9i+GHjU3e9vNb9ZwCyA4uLikuXLl8fyD9XW1lJUVBTLvOKiTNElMZcyRaNM0SUxVxIzTV9Xx90T+2U7RgtJfJ6SmAmSmSuuTKWlpc+7+ylt3tjW9ut2tmlfADyWdn0rMDScHgpsDafnAnPT+q0HTuto3iUlJR6XysrK2OYVF2WKLom5lCkaZYouibmSmGnEtauzHeEgSXyekpjJPZm54soEbPJ2apqubAqcwoHNgAAPA9PC6WnAQ2ntF5tZoZl9FDgBeLYLjyMiIiKSkwqidDKzo4AvAF9Pa54PrDCzmcDrwEUA7r7ZzFYAW4AG4Jvu3hhrahEREZEEilRYufsHwDGt2t4h+JVgW/3LgfLDTiciIiKSQ3TkdREREZGYqLASERERiYkKKxEREZGYqLASERERiYkKKxEREZGYqLASERERiYkKKxEREZGYRDqOlYiISJw+ef1j7Nqzv9N+I+es6fD2AX178eIPxscVS+SwqbASEZFut2vPfl6bf26HfTZu3MjYsWM77NNZ4SXS3bQpUERERCQmKqxEREREYqLCSkRERCQmKqxEREREYqLCSkRERCQmKqxEREREYqLCSkRERCQmKqxEREREYqLCSkRERCQmKqxEREREYqLCSkRERCQmKqxEREREYqLCSkRERCQmKqxEREREYqLCSkRERCQmKqxEREREYqLCSkRERCQmkQorMxtoZqvM7A9mVmVmp5nZYDN73MxeDS8HpfWfa2bbzGyrmU3IXHwRERGR5Ig6YvUTYJ27/wPwSaAKmANscPcTgA3hdcxsNHAxcCIwEVhkZvlxBxcRERFJmk4LKzM7GvgcsBjA3fe5+3vABcDSsNtS4MJw+gJgubvXu/t2YBtwaryxRURERJInyojV3wF/A35mZr83s7vMrB9Q7O47AcLL48L+w4C/pN2/OmwTEREROaKZu3fcwewU4DfAGe7+WzP7CbAbKHP3gWn9atx9kJndCjzj7veE7YuBR939/lbznQXMAiguLi5Zvnx5LP9QbW0tRUVFscwrLsoUXRJzKVM0yhRdEnN1d6bp6+q4e2K/DvtEyRRlPnHSsosuibniylRaWvq8u5/S5o3u3uEf8CHgtbTrnwXWAFuBoWHbUGBrOD0XmJvWfz1wWkePUVJS4nGprKyMbV5xUabokphLmaJRpuiSmKu7M424dnWnfaJkijKfOGnZRZfEXHFlAjZ5OzVNp5sC3f1/gL+Y2d+HTeOALcDDwLSwbRrwUDj9MHCxmRWa2UeBE4Bno1aBIiIiIrmqIGK/MuBeM+sN/BmYQbB/1gozmwm8DlwE4O6bzWwFQfHVAHzT3RtjTy4iIiKSMJEKK3d/AWhrW+K4dvqXA+WHHktEREQk9+jI6yIiIiIxUWElIiIiEhMVViIiIiIxUWElIiIiEhMVViIiIiIxUWElIiIiEhMVViIiIiIxUWElIiIiEhMVViIiIiIxUWElIiIiEhMVViIiIiIxUWElIiIiEhMVViIiIiIxUWElIiIiEhMVViIiIiIxUWElIiIiEhMVViIiIiIxUWElIiIiEhMVViIiIiIxUWElIiIiEhMVViIiIiIxUWElIiIiEhMVViIiIiIxUWElIiIiEhMVViIiIiIxUWElIiIiEpNIhZWZvWZmL5vZC2a2KWwbbGaPm9mr4eWgtP5zzWybmW01swmZCi8iIiKSJF0ZsSp195Pc/ZTw+hxgg7ufAGwIr2Nmo4GLgROBicAiM8uPMbOIiIhIIhUcxn0vAMaG00uBjcC1Yftyd68HtpvZNuBU4JnDeKw2mVnkvu4e98OLiIiItBB1xMqBx8zseTObFbYVu/tOgPDyuLB9GPCXtPtWh22xc/eD/kZcu7rNdhEREZFMsyhFh5kd7+5/NbPjgMeBMuBhdx+Y1qfG3QeZ2a3AM+5+T9i+GHjU3e9vNc9ZwCyA4uLikuXLl8fyD01fV8fdE/vFMq+41NbWUlRUlO0YLSQxEyQzlzJFo0zRJTFXd2eKsq6Okqm71/ladtElMVdcmUpLS59P2zWqpbZGdzr6A34IfBfYCgwN24YCW8PpucDctP7rgdM6mmdJSYnHZcS1q2ObV1wqKyuzHeEgSczknsxcyhSNMkWXxFzdnSnKujpKpu5e52vZRZfEXHFlAjZ5OzVNp5sCzayfmfVPTQPjgVeAh4FpYbdpwEPh9MPAxWZWaGYfBU4Anu1SKSgiIiKSg6LsvF4MPBDuKF4ALHP3dWb2HLDCzGYCrwMXAbj7ZjNbAWwBGoBvuntjRtKLiIiIJEinhZW7/xn4ZBvt7wDj2rlPOVB+2OlEREREcoiOvC4iIiISExVWIiIiIjFRYSUiIiISExVWIiIiIjFRYSUiIiISExVWIiIiIjFRYSUiIiISExVWIiIiIjFRYSUiIiISExVWIiIiIjFRYSUiIiISExVWIiIiIjFRYSUiIiISExVWIiIiIjFRYSUiIiISExVWIiIiIjFRYSUiIiISExVWIiIiIjFRYSUiIiISExVWIiIiIjFRYSUiIiISExVWIiIiIjFRYSUiIiISExVWIiIiIjFRYSUiIiISExVWIiIiIjGJXFiZWb6Z/d7MVofXB5vZ42b2ang5KK3vXDPbZmZbzWxCJoKLiIiIJE1XRqyuAqrSrs8BNrj7CcCG8DpmNhq4GDgRmAgsMrP8eOKKiIiIJFekwsrMhgPnAnelNV8ALA2nlwIXprUvd/d6d98ObANOjSWtiIiISIKZu3feyWwVcAPQH/iuu3/RzN5z94FpfWrcfZCZLQR+4+73hO2LgbXuvqrVPGcBswCKi4tLli9fHss/NH1dHXdP7BfLvOJSW1tLUVFRtmO0kMRMkMxcyhSNMkWXxFzdnSnKujpKpu5e52vZRZfEXHFlKi0tfd7dT2nzRnfv8A/4IrAonB4LrA6n32vVrya8vBX4Slr7YuBLHT1GSUmJx2XEtatjm1dcKisrsx3hIEnM5J7MXMoUjTJFl8Rc3Z0pyro6SqbuXudr2UWXxFxxZQI2eTs1TUGEwuwM4HwzOwfoAxxtZvcAb5rZUHffaWZDgbfC/tXAh9PuPxz4a8QiUEREREKfvP4xdu3Z36Jtx41fjHTfEdeubnF9QN9evPiD8bFlk7Z1Wli5+1xgLoCZjSXYFPgVM/sRMA2YH14+FN7lYWCZmd0CHA+cADwbe3IREZEjXNPI79C/VduYu8dEvPeclvMC4OXDDyUdijJi1Z75wAozmwm8DlwE4O6bzWwFsAVoAL7p7o2HnVRERKSHeb9qPq/NP7fDPhs3bmTs2LGdzmvknDUxpZKOdKmwcveNwMZw+h1gXDv9yoHyw8wmIiIiklN05HURERGRmBzOpsBu19ZOfG3pbLhTO/CJiIhIJuRUYbVrz/5YtjVrO7OIiIhkQk4VViIiIj1NpMGAdZ33GdC3VwxppDMqrERERBKqs600EBReUfpJ99DO6yIiIiIxUWElIiIiEhMVViIiIiIxUWElIiIiEhMVViIiIiIxUWElIiIiEhMVViIiIjmorKyMPn36sOPGL9KnTx/KysqyHUnQcaxERERyTllZGbfeeit5ecH4SENDA7feeisACxYsyGa0Hk8jViIiIjnmtttuw91pbGwEoLGxEXfntttuy3Iy0YiViIhIDjGzdm9rbGxsvt3duyuSpNGIlYiISA5JFUx5eXn06hWc/69Xr17NmwXdXUVVFmnESkREJAc1NTU1F1ANDQ0qphJCI1YiIiI5KrXZr6PNg9K9VFiJiIiIxESbAkVEpNv1HzWHTyyd03nHpZ3NB+DcOCLlHDOjqakJCDYLmpk2ByaACisREel271fN57X5HRdEGzduZOzYsR32GTlnTYypckvrIkpFVTJoU6CIiIhITFRYiYiI5Kj8/PwWl5J9KqxERERy0MSJE2loaKCyspKGhgYmTpyY7UiC9rESERHJSevWreNDH/oQb775JsXFxbz55pvZjiREGLEysz5m9qyZvWhmm83s+rB9sJk9bmavhpeD0u4z18y2mdlWM5uQyX9ARESkp0oVUyqqkiPKpsB64Cx3/yRwEjDRzD4DzAE2uPsJwIbwOmY2GrgYOBGYCCwyM238FRERiUlBQdsbnNprl+7TaWHlgdrwaq/wz4ELOHCEkaXAheH0BcByd6939+3ANuDUOEOLiIj0ZA0NDQwbNqzFkdeHDRtGQ0NDlpNJpNI2HHF6Hvg4cKu7/9bMit19J4C77zSz48Luw4DfpN29OmwTERGRmOzfv58NGzbQ2NhIfn4+F198cbYjCWBdOaCYmQ0EHgDKgKfdfWDabTXuPsjMbgWecfd7wvbFwKPufn+rec0CZgEUFxeXLF++vNPHn76ujrsn9uuwT21tLUVFRYc9nzhFydTdkpgJkplLmaJRpuiSmKu7M2l9fnhKS0vJy8sjLy+PhoYGCgoKaGpqoqmpicrKymzHA5LzXKWLK1Npaenz7n5Kmze6e5f+gB8A3wW2AkPDtqHA1nB6LjA3rf964LSO5llSUuJRjLh2dad9KisrY5lPnKJk6m5JzOSezFzKFI0yRZfEXN2dSevzw0OwS06bf0mRlOcqXVyZgE3eTk0T5VeBx4YjVZhZX+CfgD8ADwPTwm7TgIfC6YeBi82s0Mw+CpwAPButBhQREZEoUvtXtXddsiPKPlZDgaXhflZ5wAp3X21mzwArzGwm8DpwEYC7bzazFcAWoAH4prs3Zia+iIhIz2Rm/PjHP2b06NFs2bKF733vezpfYAJ0Wli5+0vAyW20vwOMa+c+5UD5YacTERGRNo0dO5YlS5ZQVVXFqFGjGDt2LE888US2Y/V4OuCFiIhIDnryySe56aabmkesrrnmmmxHElRYiYiI5Jzhw4fzzjvvMGfOHPbv30+vXr3o3bs3xxxzTLaj9XgqrEREJCtGzlnTead1HfcZ0LdXTGlyy0033cTXv/71FgcELSgo4KabbspiKgEVViIikgWvzT+30z4j56yJ1K+n6tOnD8cccww7duxg2LBh1NXVZTuSEO1cgSIiIpIg5eXl3HfffWzfvp0nnniC7du3c99991Fert+NZZsKKxERkRxTVVXFypUr6dOnD6WlpfTp04eVK1dSVVWV7Wg9njYFioiI5JiBAwdy++2386Mf/ajFcawGDhyY7Wg9ngorERGRHLN7924GDhzIySefTGNjIyeffDIDBw5k9+7d2Y7W46mwEhERyTENDQ1cdNFFnH322dTX11NYWMj06dO5/fbbsx2tx1NhJSIikmMKCgpYuXIla9eupbGxkfz8fCZNmkRBgT7Ws01LQEREJMccffTR1NTUMGXKFN58802Ki4upqalh0KBB2Y7W4+VUYdV/1Bw+sXRO5x2XdjYfAB0bRUREclNNTQ39+vXj3XffBeDdd9+lX79+1NTUZDmZ5FRh9X7V/E4PFrdx40bGjh3bYZ9IR/sVERFJqN69e3P99ddz9dVXN3/u3XLLLVx33XXZjtbj5VRhJSIiIrBv3z5++MMftjhXYJ8+fdi3b1+2o/V4OkCoiIhIjhk0aBC1tbUcc8wx5OXlccwxx1BbW6t9rBJAI1YiIiI5JnUcq2XLljX/KvBLX/qSjmOVACqsREREckxbx7GaNm0ad9xxR7aj9XjaFCgiIpJjCgoKuPfeexk6dCh5eXkMHTqUe++9V8exSgAVViIiIjmmsLCQuro6zj77bB566CHOPvts6urqKCwszHa0Hk+FlYiISI6pq6vj/PPPZ8mSJZx33nksWbKE888/n7q6umxH6/FUWImIiOSg2bNns3fvXiorK9m7dy+zZ8/OdiRBhZWIiEjOGT58OJdeeimVlZU0NDRQWVnJpZdeyvDhw7MdrcfTXm4iIiI55qabbuKqq67isssuY8eOHYwYMYLGxkZuueWWbEfr8TRiJSIikmOmTJnC5MmT2blzJ+7Ozp07mTx5MlOmTMl2tB5PI1YiIiI5pqKigjVr1rB27drmA4TOnDmT008/XcVVlmnESkREJMeUl5ezePFiSktLKSgooLS0lMWLF1NeXp7taD1ep4WVmX3YzCrNrMrMNpvZVWH7YDN73MxeDS8Hpd1nrpltM7OtZjYhk/+AiIhIT1NVVcWZZ57Zou3MM8+kqqoqS4kkJcqIVQPwHXcfBXwG+KaZjQbmABvc/QRgQ3id8LaLgROBicAiM8vPRHgREZGeaNSoUTz99NMt2p5++mlGjRqVpUSS0mlh5e473f134fT7QBUwDLgAWBp2WwpcGE5fACx393p33w5sA06NObeIiEiPNW/ePGbOnNnicAszZ85k3rx52Y7W43Vp53UzGwmcDPwWKHb3nRAUX2Z2XNhtGPCbtLtVh20iIiISg9QO6mVlZVRVVTFq1CjKy8u143oCmLtH62hWBDwJlLv7L83sPXcfmHZ7jbsPMrNbgWfc/Z6wfTHwqLvf32p+s4BZAMXFxSXLly/vNMP0dXXcPbFfh31qa2spKio67PnEKUqm7pbETJDMXMoUjTJFl8RcSczU3evqKJL4PCUxEyQzV1yZSktLn3f3U9q80d07/QN6AeuBq9PatgJDw+mhwNZwei4wN63feuC0juZfUlLiUYy4dnWnfSorK2OZT5yiZOpuSczknsxcyhSNMkWXxFxJzNTd6+ookvg8JTGTezJzxZUJ2OTt1DRRfhVowGKgyt3TD+n6MDAtnJ4GPJTWfrGZFZrZR4ETgGej1YAiIiIiuSvKPlZnAF8FXjazF8K264D5wAozmwm8DlwE4O6bzWwFsIXgF4XfdPfGuIOLiIiIJE2nhZW7Pw1YOzePa+c+5YCOUiYiIpEEG0faaL/x4DaPuG/wka6iooLy8vLmndfnzZunndcTQKe0ERGRrGurWNq4cSNjx47t/jA5oKKignnz5rF48eIWp7QBVFxlmQorERGRHFNeXs7UqVNbHG5h6tSpOuRCAqiwEhERyTFbtmyhrq6OJUuWNI9YXXbZZezYsSPb0Xo8FVYiIiI5pnfv3pxxxhktRqzOOOMMdu7cme1oPZ4KKxERkRxTX1/Pfffdx4033sjo0aPZsmUL1157LQ0NDdmO1uOpsBIREckxhYWFTJo0iSVLljSPWE2ePJlVq1ZlO1qPp8JKREQkx+zbt49f/epXB+1jtW/fvmxH6/FUWImIiOSY0aNHc+GFF7bYx+qSSy7hwQcfzHa0Hk+FlYiISI6ZN29em8exKi/XsbmzTYWViIhIjkkdqyp9xErHsEoGFVYiIiI5aMqUKUyZMkVHqE+YvGwHEBERETlSqLASERERiYkKKxEREZGYqLASERERiYkKKxEREZGYqLASERHJQRUVFYwZM4Zx48YxZswYKioqsh1JUGElIiKScyoqKrjqqquoq6sDoK6ujquuukrFVQKosBIREckx11xzDQUFBSxZsoT169ezZMkSCgoKuOaaa7IdrcdTYSUiIpJjqqurWbp0KaWlpRQUFFBaWsrSpUuprq7OdrQeL+eOvD5yzprOO63ruM+Avr1iSiMiIiJyQE4VVq/NP7fTPiPnrInUT0REJFcNHz6cSy+9lGXLltHY2EhlZSWXXnopw4cPz3a0Hi+nCisRERGBm266iauuuorLLruMHTt2MGLECBobG7nllluyHa3H0z5WIiIiOWbKlClMnjyZnTt34u7s3LmTyZMnM2XKlGxH6/E0YiUiIpJjKioqWLNmDWvXrqWxsZH8/HxmzpzJ6aefruIqyzRiJSIikmPKy8tZvHhxi18FLl68mPLy8mxH6/E6LazMbImZvWVmr6S1DTazx83s1fByUNptc81sm5ltNbMJmQouIiLSU1VVVVFdXd3iyOvV1dVUVVVlO1qPF2XE6m5gYqu2OcAGdz8B2BBex8xGAxcDJ4b3WWRm+bGllcOmUyCIiOS+448/niuuuII//vGPNDU18cc//pErrriC448/PtvRerxOCyt3fwp4t1XzBcDScHopcGFa+3J3r3f37cA24NR4osrhqqioYNq0aWzevJmmpiY2b97MtGnTVFyJiOSYmpoaPvjgA4qKigAoKirigw8+oKamJsvJ5FD3sSp2950A4eVxYfsw4C9p/arDNkmAGTNmsH//fs4//3weeOABzj//fPbv38+MGTOyHU1ERLqgrq6Oo446igEDBpCXl8eAAQM46qijms8dKNlj7t55J7ORwGp3HxNef8/dB6bdXuPug8zsVuAZd78nbF8MPOru97cxz1nALIDi4uKS5cuXx/DvwPR1ddw9sV8s84pLbW1t87eKbCotLaVXr17s37+/uS11vbKyMovJDkjKc5VOmaJJQqbS0tJI/bL9ek/Cc9WaMkWTlEylpaVcfvnlTJ06tTnTsmXLuPPOO7P++k5JynOVLq5MpaWlz7v7KW3e6O6d/gEjgVfSrm8FhobTQ4Gt4fRcYG5av/XAaZ3Nv6SkxOMy4trVsc3rcC1btsxPPPFEz8vL8xNPPNGXLVuW1TyAA3766af7ypUr/fTTT29uS4rKyspsRziIMkWTxExJWh+kS+JzpUzRJCUT4AUFBc3r8PTrSZGU5ypdXJmATd5OTXOox7F6GJgGzA8vH0prX2ZmtwDHAycAzx7iY+S0iooK5s2bx+LFi1scYwTI+jFGnnvuOS666CJ69dI5EyV3ffL6x9i1Z3+n/aKcX3RA3168+IPxccQS6RYFBQU0NDRgZrg7ZkZDQwMFBTo8ZbZ1ugTMrAIYCwwxs2rgBwQF1Qozmwm8DlwE4O6bzWwFsAVoAL7p7o0Zyp5o6ccY2bhxI2PHjmXx4sWUlZVlvbBKbQpM3yQokmt27dnf6XlBU++9zkQ6ubtIgjQ0NACktg41X6baJXs6Lazcvb0qYFw7/cuBHn+EsqqqKs4888wWbWeeeaaOMSIiIrEYMGAAgwYNaj5XYE1NDbt27cp2rB5PR17PkFGjRvH000+3aHv66acZNWpUlhKJiMiR5LOf/Szbt2/niSeeYPv27Xz2s5/NdiRB5wrMmHnz5jF58mT69evH66+/zkc+8hHq6ur4yU9+ku1oIiJyBFi9ejXf+MY3OOecc/jGN77B6tWrsx1JUGHVLVLbvkVEROIwfPhw3njjDW677TZuu+02AMyMYcN06MhsU2GVIeXl5cyaNYsHH3wQM6Nfv35ccskllJeXZ33n9fRfkajok1zVf9QcPrF0Tucdl3bepf8ogI53hBdJktGjR1NdXc2gQYOoqalpvhw9enS2o/V4KqwyZMuWLdTV1bFkyZLmwy1cdtll7NixI9vRDvoViUguer9qvn4VKD3Wk08+ySWXXMILL7zArl27OP744znnnHNYtWpVtqP1eNp5PUN69+5NWVkZpaWlFBQUUFpaSllZGb179852NBERyXH19fWMG9fyx/njxo2jvr4+S4kkRSNWGbJv3z4WLlzIySefTGNjI5WVlSxcuJB9+/ZlLZOZkZ+f3+I4JwUFBTQ29shDjYmI5KyCggK++93vsmrVquatIpMmTdIBQhNASyBDRo8ezYUXXkhZWRlVVVWMGjWKqVOn8uCDD2YtU1sn6GxoaKBfv2SdW1FERDp29NFHs2vXLn7/+98zevRoXnrpJXbt2sWAAQOyHa3HU2GVIfPmzWvzlDbl5dk7dmp7Zz3X2dBFRHLLe++9x9e//nWuu+466uvrKSwsZNasWdx+++3ZjtbjqbDKkClTpvDrX/+as88+u/lFf/nll2f9F4EAV155Jeeccw6PPvpo8890RUQkd4waNYrBgwfz8Y9/nKqqKj7+8Y8zePBgHYS6HWZ2UFumfsClndczpKKigjVr1rB27Voef/xx1q5dy5o1a6ioqMhqruHDh7No0SKKiopYtGgRw4cPz2oeERHputLSUm644QbefvttmpqaePvtt7nhhhsoLS3NdrTEaauo6qj9cKmwypDy8nKmTp1KWVkZEyZMoKysjKlTp2Z1UyBAdXU1ZkZpaSlmRnV1dVbziIhI1z344IMUFhby7rvvAvDuu+9SWFiY1f14JaBNgRmyZcsW3nrrLfr164e7U1dXxx133MHbb7+d7WgiIpLjqqur+dCHPsSyZcua9+OdOnWqviwngEasMiQ/P5/du3fzxhtv4O688cYb7N69m/z8/GxHExGRI8DVV1/d4liJV199dbYjAcGuMGPGjGHcuHGMGTMm67vAdDeNWGVIQ0MDDQ0N2lFcJIMiHTF9Xed9BvTtFUMake5VXl7OokWL2LFjByNGjKCmpibbkaioqGDmzJns2bMHgM2bNzNz5kyARPx4qzuosMqgcePG8dRTT3H77bczatQoxo0bx4YNG7IdS+SI0NnpbCAovKL0E8k1gwcPpqamhr59+2Jm7N27l927dzN48OCs5rr88subi6qUPXv2JOZX8d1BmwIzaNOmTdTV1TXvY7Vp06ZsRwKCzZS33HKLNkuKiOSoo446ir59+/LOO+/Q1NTEO++8Q9++fTnqqKOymit1XMQrr7ySRx55hCuvvLJFe0+gwipD8vPz2bVrF9XV1bg71dXV7Nq1KxHFTGNjI1dffbVOZSMikqPeeOMNmpqa2L9/PwD79++nqamJN954I8vJ4Oyzz25xWJ+zzz47KznMrNNDKmTikAsqrDKkT58+AM3n5UtdptqzaeTIkfziF79g5MiR2Y4iIiKHwMyor6+nuLgYM6O4uJj6+vqMHZupK5588kkqKytpaGigsrKSJ598Mis53L3Tg4Bm4iCh2scqQ5J2+pj0N9trr73GV7/61TZvz9SRaEVEJD5NTU0AvP3227h782W21uHpnzEffPABZ511Vpt9spFv9uzZLFy4sM32TNCIVQ/h7u2ebDl1rK3ufsGnhmnT/1IHLk3/ExGRZEt9howfP77N28ePH5+1om/BggXMnj2bwsJCAAoLC5k9ezYLFizIyONpxKoHqa2tpaioqMWoWb9+/aitrc1KnrbeZPoVl0i8Pnn9Y+zas79F244bvxjpviOuXd3i+oC+vXjxB21/cEp2DBkyhLfeeoshQ4bw5ptvZjsO69evZ8KECTz++OO4O2bGF77wBdavX5/Rx23rdd5Cv4l86NsTm68+AjzSzuFaDvd1rsKqh0kVUSpgRLpfRUUF5eXlVFVVMWrUKObNm5fxn6Dv2rP/4Pf6/IO/1GzcuJGxY8d2OK9Ixw3LYa0/nKMWoNCyCO3OAnTfvn24O/v27euWx4siVUR15+dM08jv0D+ueQHw8iHfX4WVdItOv02k6WzlrW/NyZKNb6e5qKKigmnTpjX/imvz5s1MmzYN6DkHTky61h/OY+4e04V7zzkwH+BwPpijMrPmg4LW1NRkbR+mJHi/an6kIq47vkCosJJu0ea35jboW3NumTBhAo899ljzdXfnscceY8KECSquWpkxYwb79+/n/PPPZ8aMGfzsZz/j4YcfZsaMGSqsEuLlaW0XQ6n9PZuamppHYfLy8rK6szgcvDtFd2VJ6hflyJ8NnZyN4XDPxKDC6ggU9UUf5UWo0SHpSHpRFaW9J6uvr+eLX/wiDz30EBs3buShhx7ivPPOY/Xq1Z3fWbIi/cczqRFZALvx4D49aaQoSZvdUjr74l5WVsadd95JfX09hYWFXH755bm387qZTQR+AuQDd7n7/Ew9VpJE+RVbpt+IUUaHoowMQXyjQ/1HzeETS+d03hFgaWfzAsj8dvu2lmVPWnl2RXFxMfPnz2fOnDmJ2IE2SdJfR6tXr27zddWTN+EkWWqZpI9YpdadSRixypYkbXaLoqysjJ/+9KfceOON/Gf1CL49fAfXXnstQEaKq4wUVmaWD9wKfAGoBp4zs4fdfUsmHi9J0t+InfXpSXLtjZi+/CZNmsSqVaua23vi8vvE0k+02Z6+D8rN3MyxNx7LsRzb4X3a29xypEpfJ/Tt25c1a9Yw/dHd3H3O0Zx77rns2bOnR76mco27k5eXxw033MBZZ52VmGU2aNAgampqmi+7S1I2u3Wk9efwd77zneAyvL5w4UIWLlwY+7LM1IjVqcA2d/8zgJktBy4AjrjCqivbmlPaekH2hE1uufBGbM3d2bhxIytXruzRx9RqXQx15bnozg+gNkeDbjy4X6YytVdMwoEi9Fuvf4ujx8C3XoeP3faxDu8XRxGai6PFSZPaDOjuzJkzp0V7tn3/+99n9OjRbNmypblwyLSov/TL9q/P07/U1NXVcdRRRzV/ef/ggw+aj+EYN8vITM0mARPd/Wvh9a8C/6+7z07rMwuYBVBcXFyyfPnyLj9OaWlp5L6VlZVdnn8UZTvKYpvXghHxDEnGmQniy9Va1OWXC8sOkrn8MrXsUjpahplabl2ROnZbd5m+7uAzKxzqMaP69YJbx7V9UN/Dle33XlTdvfyiSEqm1DLMy8ujqamp+RKys9yS/JoaP348X/va1/jyl7/cvPxWrFjBXXfddcj7g5aWlj7v7qe0eWNqG3Gcf8BFBPtVpa5/FVjQXv+SkhKPS2VlZWzziosyRZeUXIAHb48DmdLbsi0pz1NhYWHz85L+V1hYmO1o7p6c56m1JOZSpmiSkin1XsvPz29xmZR1lHtynqvZs2d7QUGB33zzzb527Vq/+eabvaCgwGfPnn3I8wQ2eTs1TaY2BVYDH067Phz4a4YeSyRjzIxJkyZ1aXS0J9m7dy99+vShvr6+ua2wsJC9e/dmMZXIkW/8+PE89thjzZuyUpftnVKmJ0vtoH7dddc1/yrwiiuuyNivAjN1rsDngBPM7KNm1hu4GHg4Q48lErvUSgpo3nG9dbsE9u7di7tTWVmJu6uoEukG69evb3H+PQ/P06fjx7VtwYIF7N27l8rKSvbu3ZuxogoyVFi5ewMwG1gPVAEr3H1zJh5LJFNSw7qpgkFFlYgkyfr162lqaqKyspKmpiYVVQmRseNYufujwKOZmr+IiIhI0mRqU6CIiIhIj6PCSkRERCQmKqxEREREYqLCSkRERCQmKqxEREREYqLCSkRERCQmKqxEREREYpKRkzB3OYTZ34AdMc1uCPB2TPOKizJFl8RcyhSNMkWXxFzKFI0yRZfEXHFlGuHux7Z1QyIKqziZ2SZv74zTWaJM0SUxlzJFo0zRJTGXMkWjTNElMVd3ZNKmQBEREZGYqLASERERicmRWFjdke0AbVCm6JKYS5miUabokphLmaJRpuiSmCvjmY64faxEREREsuVIHLESERERyYqMFFZmNtLMXmnV9kMz+24mHi8uUTOa2Vwz22ZmW81sQid9zzezOfGlzB1m9usu9h9rZqszlSfC499tZpOy9fhyZEutX8zs38zsn7KdJ4pMvyfNbKCZfSNT8+8KM/u2mR11CPebbmbHp12/y8xGH2aWgz5Dw/aNZtblX7SFGRceTqZW86vt5PbrIs4nUr/ucKjPbVs0YtVF4RvmYuBEYCKwyMzy2+lb4O4Pu/v87syYFO5+erYzSGZZQOuRLnD3f3X3/8pmhgQtt4FAIgor4NtAm4VVe+v40HSgubBy96+5+5ZYk+WeqAVTYgqrOGXljRVWhjea2bNm9kcz+2zYPt3Mfmlm68zsVTO7Ke0+t5nZJjPbbGbXp7W/Zmb/bmbPhLd/yszWm9mfzOyKtH7fM7PnzOylVvefF448/Rfw9xHiXwAsd/d6d98OfABsNbMXzewX4ajHLWZWCdyY/k0hvO02M6s0sz+b2efNbImZVZnZ3Yf5tLbLzK4wsxfCv+3h49eaWXmY+zdmVpyBx60NL8eGy3yVmf3BzO41Mwtvmxi2PQ38S9p9W4wemtkr4be4fma2Jsz9iplNPox8l4avhxfN7Bdh8+fM7Nfh8pkU9isysw1m9jsze9nMLgjbY8vSRraR4evizvA1/5iZ9TWzk8Ll9ZKZPWBmg8xsRPh+GWJmeWb232Y2Pq4sHWRbBLwL/MmCb+mvhMv2n8zsV2GmUzOVIy3Pg2b2fPg8zQrbZobrlo3hc5h6Dx5rZveH64LnzOyMDGc7aP1iaSOjZjbfzLaEy/PHGc6Svtx+Byy2ttepbb4nM2Q+8LFw3fQf7bzPPh0+P33C99xmMxtzqA8YPg9/MLOl4XxXmdm3CIqjSgvW3YTryH8zs98Cp5nZv4avmVfM7A4LTAJOAe4N/4e+ljbyYWZTwv/lFTO7sYtRC1plbFH0WfufiZ8O12EvWvAZ27/V/c614PNySNefvZbMbKiZPRX+76+Y2WfNbD7QN2y7N+zX1nv0oH6H8PipZdnu+id8zSwJl93v015Xfc1sefj83gf0DduvtJa1x3QzW9ClYO4e+x8wEnilVdsPge+G0xuBm8Ppc4D/CqenA38GBgB9CI7G/uHwtsHhZX54/38Mr78GXBlO/wfwEtAfOBZ4K2wfT/BLACMoJlcDnwNKgJcJvqUcDWxLy3gFcEUb/9tC4Cvh9InAe8CMVEbg7nD++Wn/08Jw+m5geZjjAmA38Ikw0/PASZlYHmnZewH/DZwHOHBe2H4T8P0MPF5teDkW2AUMD//XZ4Azw2X8F+CE8DlZAaxu/XoJr78Svq6+BNyZ1j7gELOdCGwFhrRadivDjKOBbeFtBcDR4fSQ8HVicWXp4D3UkHpNhM/NV8LX9+fDtn8D/jOc/hqwCvgecHuGX0cjgSbgM2k501/HS9Je4w9mMktq2YWXfcPXyTCC9cLgtNd86j24DDgznP4IUJXBXG2uX8LX2aQw31YO/IhoYHctt1bPW/M6taP3ZAYzvRJOt/k+C6//H+DHwK3A3Bge04EzwutLwuXyGuH6IGx34MutX2fh9C84sP7cCJySdttGgmLreOB1gs+iAuAJ4MLDzNj8WO0sv94En6GfDm87Onzs6QSfXf8cvh8GHeZzmFq3fweYl5ajf/rtrZ87DrxHj2mr3yEuyw7XP8C/c+AzeyDwR6AfcDWwJGz/x3A+p4TLa1vaY6wlXGdE/SsgM9r7qWF6+y/Dy+cJnpyUDe6+C8DMtgAjCN7oXw4r3QJgKMEH30vhfR4OL18Gitz9feB9M9trZgMJCqvxwO/DfkUEK47+wAPu/kH4eKn54O4/bed/sLTpswjejLvD+7xrwUDMSndvbOf+j7i7m9nLwJvu/nL42JvD5+GFdu4Xh58AT7j7I2a2j6AAhGAZfCGDjwvwrLtXA5jZCwT/ay2w3d1fDdvvAWZ1Mp+XgR+H3/5Wu/t/H2Kes4BV7v42tFh2D7p7E7DFDoziGfDvZvY5gg+mYUBxjFnas93dXwinnwc+RvDh+2TYtpSgEMTd7zKziwi+EJwUc4627HD335jZyDBn+ut4Q9prfGQ3ZPmWmf1zOP1h4KvAk+7+bphpJfD/hLf/EzA6XNYAR5tZ/3CdEbfP0s76JbQb2AvcZWZrOPB+zKQd7v6bcLqtdWoeXX9PxqW999n/EHyJeI7g+fpWDI/1F3f/VTh9TzvzbATuT7teambXEBTKg4HNwCMdPMangY3u/jeAcFTmcwQf9nFkbGv5ObDT3Z8DcPfd4WMDlBIUDuNT7TF4DlhiZr0I1p0vtNOv9Xv0BOCdmDJ0tv4ZDpxvB7aA9CH4UvU54P8CuPtLZvZSOP03C7ZYfAZ4lWCk+Vd0QaY2Bb4DDGrVNpiW5+epDy8boUWBV5823UgwHPpRgmp9nLv/I7CG4MlpfZ+mVvdvCudtwA3uflL493F3Xxz26erxJqoJXhiE8+0P/LVVn7oO7t9Z1owws+kERWpqyHi/h+U4By+DTDhouYbT7T3/DbR8ffYBcPc/cmAk4AYz+9dDzGPtPHZ9qz4AlxB8iylx95OAN4E+MWZpT+vnbGB7HcPNBMPDq0Ux52hL+mu89es4/TWe0deVmY0lKJZOc/dPEnx52trBXfLCvql1wbAMFVUp7a5f3L0BOJXgw/tCYF0Gc6TUAXSyTs3WMXjafJ+Ftw0meF33p+W6/1C1/h/b+p/3pr4gm1kfYBEwyd0/AdwZIYd1cvshZ+xg+bW3XoNgJKs/B75kHDZ3f4qgQHkD+IWZXdq6Tzvv0TiWYUpn6x8DvpT2nv+Iu1el/oV25nkf8GWCrRIPpH1WRpKRwsrda4GdZjYOwMwGE+zo/fQhzvJoghXCrnAU4ewu3n89cJmZFYV5hpnZccBTwD+H21r7E2wi68zDwMVmVgj8gaDI2hbOd3AXc3ULMysheBN+JRyNSYo/AB81s4+F16ek3fYa8CkAM/sU8NFw+njgA3e/h2DTwKcO8bE3EHzjOyacb0fLbgDBZuX9ZlZKUKDGmSWqXUCNhfskEo7MhNM3AvcC/0qw0u8pBgA17v6Bmf0DwebJo4DPW7D/WQHByjHlMWB26oqZnZTBbB2uX8L10QB3f5Rgx+lMZmmtvXVqR+/JTHif4MMe2nmfhe4A/hfBa7yr+yq15SNmdlo4PYXgsyk9S2upQuDtcLml/3q4vfv9luB1OMSCnd+ncOD9eqgZUzpafseb2acBzKx/+B6AYNeafwF+bmYndiFHu8xsBMEyuxNYzIF14P5wFAvafo/SRr9MWQ+UmTXv13ty2P4UQTGPBfvs/WPafX5J8GVnCkGR1SWZ/DZ5KXCrmd0cXr/e3f90KDNy9xfN7PcEQ69/povDcu7+mJmNAp4Jn9tagiLjd+FOay8QvOiaN+VYuON7602C7r7ZzFYAWwhGVf4vwQ6PjRzY1Jg0swm+8VWG//+m7MYJuPvecCh7jZm9TbDiSO2Uej9wabjZ8DmC7eIQbEv/kZk1AfuBKw/xsTebWTnwZIRldy/wiJltInit/CHOLF00DfhpOEL1Z2CGmX2eYLPDGe7eaGZfMrMZ7v6zbsiTbeuAK8Jh/K3Abwi+Pf87wQfbXwneq7vC/t8iWC+9RLD+e4pg82nsOlq/hPoDD4WjIQb8f5nI0U62NtepnbwnM5HjHQt2NH6F4H3+D63fZ+EoSIO7LwsLlF+b2Vnu/sRhPHQVMM3MbifY3HMbsA9Ya2Y73b20Vc73zOxOgtHp18KsKXcTvCf3AKel3Wenmc0FKgmW76Pu/tBhZjwvnHd7y2+fBT+iWWBmfYE9BKNFqUxbzewSYKWZnXeon8lpxgLfM7P9BJ+rqRGrO4CXzOx3wGUc/B6ldT93v+Qws7TnfwP/GT6OESy/LxI8nz8Lc70APJu6g7vXWLAr0mh3f7b1DDujI6+LyBHHzIrcvTb8tv4AwU6qD2Q7l2SfBfsFrnb3jBWM0rMl4TgmIiJx+2E42vkKsJ3oOwyLiBwWjViJiIiIxEQjViIiIiIxUWElIiIiEhMVViIiIiIxUWElIolhZh+y4Pxdf7LgHHqPmlmbBzS04Dxhr3R3RhGRjqiwEpFECI8x8wDBaUA+5u6jgesITmsiIpITVFiJSFKUEpxqqfmgvOG5x542sx9ZcPb6l8MDILZgwRnoF6ZdX23BqTQws1ozu9HMnjez/7LgjPcbLTgf2Plp9/+lma0zs1ctPLu9meWb2d1pj91tB/EUkdyU6fPDiYhENYbgZNOt/QvB6V4+CQwBnjOzp7ow334Eo2DXmtkDwP8hOOn4aIITWadOjnwScDLBuca2mtkC4DhgWOpgkhac1F1EpF0asRKRpDsTqHD3Rnd/k+B8a5/uwv33ceAExy8DT7r7/nB6ZFq/De6+y933EpwGZwTB6UL+zswWmNlEYPfh/SsicqRTYSUiSbEZKGmj3SLct4GW67M+adP7085O30QwIkV4QvL0Ufv6tOlGoMDdawhGyjYC3wTuipBFRHowFVYikhRPAIVmdnmqwcw+DdQAk8P9nY4FPkfaCVNDrwEnmVmemX0YODWOQGY2BMhz9/uB/wV8Ko75isiRS/tYiUgiuLub2T8D/2lmc4C9BAXTt4Ei4EXAgWvc/X/Ck+mm/IrgnIAvE5wf8HcxxRoG/MzMUl9C58Y0XxE5QulcgSIiIiIx0aZAERERkZiosBIRERGJiQorERERkZiosBIRERGJiQorERERkZiosBIRERGJiQorERERkZiosBIRERGJyf8PBcnoA9fp50wAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plotting a boxplot\n",
    "plt.figure(figsize=(10,5))\n",
    "df.boxplot()\n",
    "plt.title('Boxplot')\n",
    "plt.xlabel('Columns')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see:\n",
    "\n",
    "- crim attribute has outliers on the higher side\n",
    "- zn attribute has outliers on the higher side\n",
    "- indus attribute has no outliers\n",
    "- chas attributenoz is discrete\n",
    "- nox attribute has no outliers\n",
    "- rm attribute has outliers on the higher side\n",
    "- age attribute has no outliers\n",
    "- dis attribute has outliers on the higher side\n",
    "- rad attribute has no outliers\n",
    "- tax attribute has no outliers\n",
    "- ptratio attribute has no outliers\n",
    "- black attribute has outliers on the lower side\n",
    "- lstat attribute has outliers on the higher side\n",
    "- medv attribute has outliers on the higher side\n",
    "- there are no missing values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To deal with the outliers without reducing our dataset, we iterate over the columns in the dataframe and replace the outliers with the median of the column using the quantile(), median(), and where() functions from the NumPy library."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# replacing the outliers with median\n",
    "for column in df.drop(df.chas).columns:\n",
    "    \n",
    "    q1 = np.quantile(df[column], 0.25)\n",
    "    q3 = np.quantile(df[column], 0.75)\n",
    "    iqr = q3 - q1\n",
    "    \n",
    "    upper_bound = q3 + 1.5*iqr\n",
    "    lower_bound = q1 - 1.5*iqr\n",
    "    \n",
    "    q2 = df[column].median()\n",
    "    \n",
    "    df[column] = np.where(df[column]>upper_bound, q2, df[column])\n",
    "    df[column] = np.where(df[column]<lower_bound, q2, df[column])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we identify the best 5 features to predict the house price using pearson's correlation. To do this, we first use the corr() function from the Pandas library to calculate the correlations between the columns, then we choose the medv column and sort the values by their absolute value in the descending order by using the sort_values() function from the Pandas library and setting the parameters key and ascending to abs and False respectively. Then we splice the resultant series to find the top 5 values and ignore the first value since that will be 1 (correlation with the column itself)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "lstat   -0.710561\n",
       "indus   -0.584221\n",
       "tax     -0.568349\n",
       "rm       0.563201\n",
       "nox     -0.554382\n",
       "Name: medv, dtype: float64"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# finding top 5 best features to predict medv\n",
    "df.corr().medv.sort_values(key=abs, ascending=False)[1:6]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see, the top 5 features to predict medv are lstat, indus, tax, rm, and nox."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Adding on to the data pre-processing process, we scale the top 5 features using min-max scaler to prevent the overpower of any one feature over the other due to difference in scale. To implement this, we use the MinMaxScaler from the Sklearn library."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# scaling the top 5 features\n",
    "from sklearn.preprocessing import MinMaxScaler\n",
    "scaler = MinMaxScaler()\n",
    "\n",
    "df[['lstat', 'indus', 'tax', 'rm', 'nox']] = scaler.fit_transform(df[['lstat', 'indus', 'tax', 'rm', 'nox']])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Q2/3. Using sklearn.linear_model, find the multiple regression model for the boston.csv dataset using the best 3 features. Find the accuracy of the model using appropriate metrics using 80, 20 split for training and test."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To solve Questions 2 and 3, we use the end result of Question 1 and choose the top 3 features - lstat, indus and tax. To begin, we separate the independent and dependent variables and store them in x and y respectively."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# separating the dependent and independent variables\n",
    "x = df[['lstat', 'indus', 'tax']]\n",
    "y = df['medv']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we split the dataset into training and testing sets. To do this, we use the train_test_split() function from the SKlearn library. We also set the train_size parameter to 0.8 to ensure that 80% of the data is put into the training set and the remaining into the testing set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [],
   "source": [
    "# splitting the dataset into training and testing sets\n",
    "from sklearn.model_selection import train_test_split\n",
    "x_train, x_test, y_train, y_test = train_test_split(x, y, train_size=0.8, random_state=150)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we build a linear regression model by using the LinearRegression module from the SKlearn library. We fit our training dataset into the model and then use it to predict the output of the testing dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "# building and training the model\n",
    "from sklearn.linear_model import LinearRegression\n",
    "linreg = LinearRegression()\n",
    "linreg.fit(x_train, y_train)\n",
    "\n",
    "# predicting the output of the testing dataset\n",
    "y_pred = linreg.predict(x_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we use the predicted values to find the root mean squared error. To do that, we use the mean_squared_error() function from the SKlearn library and the sqrt() function from the NumPy library to find it's square root. To generate deeper insights, we also find out how varied are the predicted values from the actual values by dividing the rmse by the average of the actual values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RMSE:  4.152810145494985\n",
      "Percentage Variation:  19.97202295442918 %\n"
     ]
    }
   ],
   "source": [
    "# finding the rmse\n",
    "from sklearn.metrics import mean_squared_error\n",
    "rmse = np.sqrt(mean_squared_error(y_test, y_pred))\n",
    "print('RMSE: ', rmse)\n",
    "print('Percentage Variation: ', (rmse/np.mean(y_test)*100), '%')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see, on testing the model, we get an rmse of 4.15 which causes a variation of 19%."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we find out the R-Squared score. To do this, we use the score() function of the LinearRegression() module from the SKlearn library."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "R2:  0.6017386016228203\n"
     ]
    }
   ],
   "source": [
    "# finding the r2 value\n",
    "r2 = linreg.score(x_test, y_test)\n",
    "print('R2: ', r2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see, the R2 value is 0.60, which means that 60% of the data points lie on the regression line. This is a moderate score."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}