7. Binning Numeric Columns.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 7. Binning Numeric Columns\n",
    "\n",
    "Numeric columns like year might better be represented as a nominal categorical variable which are suitable for one-hot encoding. Encoding each unique year would yield a huge number of columns with some of them having very few non-zero entries. Instead, we can use a much smaller number of bins to contain groups of years. The `KBinsDescritizer` is a new transformer introduced in scikit-learn version 0.20 that does this."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "hs = pd.read_csv('data/housing_sample.csv')\n",
    "hs.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using `KBinsDescritizer`\n",
    "\n",
    "The `KBinsDescritizer` transformer is found in the `preprocessing` module. To instantiate it you must choose the number of bins you'd like to divide your numeric data into and whether you want one-hot or ordinal encoding. The default value for `encode` is 'onehot' which returns a sparse array. We choose `onehot-dense` to return an actual numpy array. You can create the bins that have equally spaced edges, that have the same number of observations in each bin, or a more complex approach involving k-means."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = hs[['YearBuilt']].values\n",
    "y = hs.pop('SalePrice').values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's bin and encode the year built."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import KBinsDiscretizer\n",
    "kbd = KBinsDiscretizer(n_bins=6, encode='onehot-dense', strategy='uniform')\n",
    "kbd.fit_transform(hs[['YearBuilt']])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can see the edges of the bins with the `bin_edges_` attribute."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "kbd.bin_edges_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Putting it all together"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.compose import ColumnTransformer\n",
    "from sklearn.impute import SimpleImputer\n",
    "from sklearn.pipeline import Pipeline\n",
    "from sklearn.preprocessing import OneHotEncoder, StandardScaler, KBinsDiscretizer\n",
    "\n",
    "from sklearn.ensemble import RandomForestRegressor\n",
    "from sklearn.model_selection import cross_val_score, KFold, GridSearchCV\n",
    "from mymetrics import root_mean_squared_log_error\n",
    "\n",
    "# string pipeline\n",
    "string_si = SimpleImputer(strategy='constant', fill_value='MISSING')\n",
    "ohe = OneHotEncoder(sparse=False)\n",
    "steps = [('impute', string_si), ('encode', ohe)]\n",
    "string_pipe = Pipeline(steps)\n",
    "\n",
    "# numeric pipeline\n",
    "numeric_si = SimpleImputer(strategy='mean')\n",
    "ss = StandardScaler()\n",
    "steps = [('si', numeric_si), ('standardize', ss)]\n",
    "numeric_pipe = Pipeline(steps)\n",
    "\n",
    "# year transformation\n",
    "kbd = KBinsDiscretizer(n_bins=10, encode='onehot-dense', strategy='uniform')\n",
    "\n",
    "# columns\n",
    "string_cols = ['Neighborhood', 'Exterior1st']\n",
    "numeric_cols = ['LotFrontage', 'GrLivArea', 'GarageArea']\n",
    "\n",
    "transformers = [('string', string_pipe, string_cols), \n",
    "                ('numeric', numeric_pipe, numeric_cols), \n",
    "                ('year', kbd, ['YearBuilt'])]\n",
    "\n",
    "ct = ColumnTransformer(transformers)\n",
    "rfr = RandomForestRegressor()\n",
    "steps = [('transformers', ct), ('rfr', rfr)]\n",
    "final_pipe = Pipeline(steps)\n",
    "\n",
    "kf = KFold(n_splits=5, shuffle=True)\n",
    "grid = {'transformers__numeric__si__strategy': ['mean', 'median'],\n",
    "       'rfr__n_estimators': [50, 100], 'rfr__max_depth': range(2, 6)}\n",
    "gs = GridSearchCV(final_pipe, grid, cv=kf, scoring=root_mean_squared_log_error)\n",
    "gs.fit(hs, y)\n",
    "gs.best_params_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise \n",
    "\n",
    "Use `KBinsDescritizer` experimenting with the number of bins and the binning strategy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}