Feature Selection (Information gain - mutual information In Classification).ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "5b4c2221",
   "metadata": {},
   "source": [
    "MI Estimate mutual information for a discrete target variable.<br>\n",
    "\n",
    "Mutual information (MI) between two random variables is a non-negative value, which measures the dependency between the variables. It is equal to zero if and only if two random variables are independent, and higher values mean higher dependency.<br>\n",
    "\n",
    "The function relies on nonparametric methods based on entropy estimation from k-nearest neighbors distances.<br>\n",
    "\n",
    "A quantity called mutual information measures the amount of information one can obtain from one random variable given another."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "367379d5",
   "metadata": {},
   "source": [
    "The mutual information between two random variables X and Y can be stated formally as follows:\n",
    "\n",
    "<b>I(X ; Y) = H(X) – H(X | Y) Where I(X ; Y) is the mutual information for X and Y, H(X) is the entropy for X and H(X | Y) is the conditional entropy for X given Y. The result has the units of bits.\n",
    "Youtube Videos<b>\n",
    "\n",
    "Entropy: https://www.youtube.com/watch?v=1IQOtJ4NI_0\n",
    "\n",
    "Information Gain: https://www.youtube.com/watch?v=FuTRucXB9rA\n",
    "\n",
    "Gini Impurity: https://www.youtube.com/watch?v=5aIFgrrTqOw\n",
    "\n",
    "Statistical test: https://www.youtube.com/watch?v=4-rxTA_5_xA\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "54201d37",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Wine</th>\n",
       "      <th>Alcohol</th>\n",
       "      <th>Malic.acid</th>\n",
       "      <th>Ash</th>\n",
       "      <th>Acl</th>\n",
       "      <th>Mg</th>\n",
       "      <th>Phenols</th>\n",
       "      <th>Flavanoids</th>\n",
       "      <th>Nonflavanoid.phenols</th>\n",
       "      <th>Proanth</th>\n",
       "      <th>Color.int</th>\n",
       "      <th>Hue</th>\n",
       "      <th>OD</th>\n",
       "      <th>Proline</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>14.23</td>\n",
       "      <td>1.71</td>\n",
       "      <td>2.43</td>\n",
       "      <td>15.6</td>\n",
       "      <td>127</td>\n",
       "      <td>2.80</td>\n",
       "      <td>3.06</td>\n",
       "      <td>0.28</td>\n",
       "      <td>2.29</td>\n",
       "      <td>5.64</td>\n",
       "      <td>1.04</td>\n",
       "      <td>3.92</td>\n",
       "      <td>1065</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>13.20</td>\n",
       "      <td>1.78</td>\n",
       "      <td>2.14</td>\n",
       "      <td>11.2</td>\n",
       "      <td>100</td>\n",
       "      <td>2.65</td>\n",
       "      <td>2.76</td>\n",
       "      <td>0.26</td>\n",
       "      <td>1.28</td>\n",
       "      <td>4.38</td>\n",
       "      <td>1.05</td>\n",
       "      <td>3.40</td>\n",
       "      <td>1050</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>13.16</td>\n",
       "      <td>2.36</td>\n",
       "      <td>2.67</td>\n",
       "      <td>18.6</td>\n",
       "      <td>101</td>\n",
       "      <td>2.80</td>\n",
       "      <td>3.24</td>\n",
       "      <td>0.30</td>\n",
       "      <td>2.81</td>\n",
       "      <td>5.68</td>\n",
       "      <td>1.03</td>\n",
       "      <td>3.17</td>\n",
       "      <td>1185</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>14.37</td>\n",
       "      <td>1.95</td>\n",
       "      <td>2.50</td>\n",
       "      <td>16.8</td>\n",
       "      <td>113</td>\n",
       "      <td>3.85</td>\n",
       "      <td>3.49</td>\n",
       "      <td>0.24</td>\n",
       "      <td>2.18</td>\n",
       "      <td>7.80</td>\n",
       "      <td>0.86</td>\n",
       "      <td>3.45</td>\n",
       "      <td>1480</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>13.24</td>\n",
       "      <td>2.59</td>\n",
       "      <td>2.87</td>\n",
       "      <td>21.0</td>\n",
       "      <td>118</td>\n",
       "      <td>2.80</td>\n",
       "      <td>2.69</td>\n",
       "      <td>0.39</td>\n",
       "      <td>1.82</td>\n",
       "      <td>4.32</td>\n",
       "      <td>1.04</td>\n",
       "      <td>2.93</td>\n",
       "      <td>735</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Wine  Alcohol  Malic.acid   Ash   Acl   Mg  Phenols  Flavanoids  \\\n",
       "0     1    14.23        1.71  2.43  15.6  127     2.80        3.06   \n",
       "1     1    13.20        1.78  2.14  11.2  100     2.65        2.76   \n",
       "2     1    13.16        2.36  2.67  18.6  101     2.80        3.24   \n",
       "3     1    14.37        1.95  2.50  16.8  113     3.85        3.49   \n",
       "4     1    13.24        2.59  2.87  21.0  118     2.80        2.69   \n",
       "\n",
       "   Nonflavanoid.phenols  Proanth  Color.int   Hue    OD  Proline  \n",
       "0                  0.28     2.29       5.64  1.04  3.92     1065  \n",
       "1                  0.26     1.28       4.38  1.05  3.40     1050  \n",
       "2                  0.30     2.81       5.68  1.03  3.17     1185  \n",
       "3                  0.24     2.18       7.80  0.86  3.45     1480  \n",
       "4                  0.39     1.82       4.32  1.04  2.93      735  "
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "df=pd.read_csv('https://gist.githubusercontent.com/tijptjik/9408623/raw/b237fa5848349a14a14e5d4107dc7897c21951f5/wine.csv')\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "63594aef",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1, 2, 3])"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['Wine'].unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "b3e25f78",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 178 entries, 0 to 177\n",
      "Data columns (total 14 columns):\n",
      " #   Column                Non-Null Count  Dtype  \n",
      "---  ------                --------------  -----  \n",
      " 0   Wine                  178 non-null    int64  \n",
      " 1   Alcohol               178 non-null    float64\n",
      " 2   Malic.acid            178 non-null    float64\n",
      " 3   Ash                   178 non-null    float64\n",
      " 4   Acl                   178 non-null    float64\n",
      " 5   Mg                    178 non-null    int64  \n",
      " 6   Phenols               178 non-null    float64\n",
      " 7   Flavanoids            178 non-null    float64\n",
      " 8   Nonflavanoid.phenols  178 non-null    float64\n",
      " 9   Proanth               178 non-null    float64\n",
      " 10  Color.int             178 non-null    float64\n",
      " 11  Hue                   178 non-null    float64\n",
      " 12  OD                    178 non-null    float64\n",
      " 13  Proline               178 non-null    int64  \n",
      "dtypes: float64(11), int64(3)\n",
      "memory usage: 19.6 KB\n"
     ]
    }
   ],
   "source": [
    "df.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "0aebf789",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.4765449 , 0.2685969 , 0.07672314, 0.2257552 , 0.18693038,\n",
       "       0.4136781 , 0.67201153, 0.12503738, 0.29883242, 0.5449141 ,\n",
       "       0.44650475, 0.52009432, 0.55191491])"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X=df.drop(['Wine'],axis=1)\n",
    "y=df['Wine']\n",
    "\n",
    "from sklearn.feature_selection import mutual_info_classif\n",
    "mutual_info = mutual_info_classif(X,y)\n",
    "mutual_info"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "65634b81",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Flavanoids              0.672012\n",
       "Proline                 0.551915\n",
       "Color.int               0.544914\n",
       "OD                      0.520094\n",
       "Alcohol                 0.476545\n",
       "Hue                     0.446505\n",
       "Phenols                 0.413678\n",
       "Proanth                 0.298832\n",
       "Malic.acid              0.268597\n",
       "Acl                     0.225755\n",
       "Mg                      0.186930\n",
       "Nonflavanoid.phenols    0.125037\n",
       "Ash                     0.076723\n",
       "dtype: float64"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mutual_info = pd.Series(mutual_info,index=X.columns)\n",
    "mutual_info.sort_values(ascending=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "a9a122e4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<AxesSubplot:>"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABIEAAAI1CAYAAABBveqYAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAAycUlEQVR4nO3de5hlZ1km7udNmhhOEfwRQEmgAwYxYJQQEETloGHASIKAQIQRAUUdoox4ijqDgs4YQHQQ0CGiDB4gIAdtSQSVs6KQDodgiIxtjBBGISICghIC7++PvYtUiqruStdh1a7vvq+rrq619kr3w6J7197P/g7V3QEAAABgdzti6gAAAAAAbD0lEAAAAMAAlEAAAAAAA1ACAQAAAAxACQQAAAAwACUQAAAAwAD2rOeiqnpgkuckOTLJC7v73BWP/0qS+80Pb5Tklt19s4P9nre4xS1679691zcvAAAAAGu4+OKL/7m7j13tsUOWQFV1ZJLnJzktyZVJLqqqfd39vqVruvtHll3/Q0nueqjfd+/evdm/f/864gMAAACwHlX1D2s9tp7pYPdIcqC7L+/uq5Ocn+TMg1x/VpKXXr+IAAAAAGyl9ZRAt0nywWXHV87PfZGqul2SE5K8YePRAAAAANgsm70w9KOSvKK7P7fag1X1xKraX1X7r7rqqk3+owEAAABYy3pKoA8lOX7Z8XHzc6t5VA4yFay7z+vuU7v71GOPXXWNIgAAAAC2wHpKoIuSnFhVJ1TVUZkVPftWXlRVd0py8yR/ubkRAQAAANioQ5ZA3X1NkrOTvC7JZUle3t2XVtXTq+qMZZc+Ksn53d1bExUAAACAw3XILeKTpLsvTHLhinNPXXH8c5sXCwAAAIDNtNkLQwMAAACwAymBAAAAAAagBAIAAAAYgBIIAAAAYABKIAAAAIABKIEAAAAABqAEAgAAABiAEggAAABgAEogAAAAgAEogQAAAAAGoAQCAAAAGIASCAAAAGAAe6YOsNn2nnPB1BHW5YpzT586AgAAADAQI4EAAAAABqAEAgAAABiAEggAAABgAEogAAAAgAEogQAAAAAGoAQCAAAAGIASCAAAAGAASiAAAACAASiBAAAAAAagBAIAAAAYgBIIAAAAYABKIAAAAIABKIEAAAAABqAEAgAAABiAEggAAABgAEogAAAAgAEogQAAAAAGoAQCAAAAGIASCAAAAGAASiAAAACAASiBAAAAAAagBAIAAAAYgBIIAAAAYABKIAAAAIABKIEAAAAABqAEAgAAABiAEggAAABgAEogAAAAgAEogQAAAAAGoAQCAAAAGIASCAAAAGAASiAAAACAASiBAAAAAAagBAIAAAAYgBIIAAAAYABKIAAAAIABKIEAAAAABqAEAgAAABiAEggAAABgAEogAAAAgAGsqwSqqgdW1fur6kBVnbPGNY+oqvdV1aVV9ZLNjQkAAADARuw51AVVdWSS5yc5LcmVSS6qqn3d/b5l15yY5KeS3Lu7P1ZVt9yqwAAAAABcf+sZCXSPJAe6+/LuvjrJ+UnOXHHN9yV5fnd/LEm6+yObGxMAAACAjVhPCXSbJB9cdnzl/Nxyd0xyx6r6i6r6q6p64Gq/UVU9sar2V9X+q6666vASAwAAAHC9bdbC0HuSnJjkvknOSvIbVXWzlRd193ndfWp3n3rsscdu0h8NAAAAwKGspwT6UJLjlx0fNz+33JVJ9nX3Z7v775P838xKIQAAAAB2gPWUQBclObGqTqiqo5I8Ksm+Fdf8QWajgFJVt8hsetjlmxcTAAAAgI04ZAnU3dckOTvJ65JcluTl3X1pVT29qs6YX/a6JB+tqvcleWOSH+/uj25VaAAAAACun0NuEZ8k3X1hkgtXnHvqsu87yVPmXwAAAADsMJu1MDQAAAAAO5gSCAAAAGAASiAAAACAASiBAAAAAAagBAIAAAAYgBIIAAAAYABKIAAAAIABKIEAAAAABqAEAgAAABiAEggAAABgAEogAAAAgAEogQAAAAAGoAQCAAAAGIASCAAAAGAASiAAAACAASiBAAAAAAagBAIAAAAYgBIIAAAAYABKIAAAAIABKIEAAAAABqAEAgAAABiAEggAAABgAEogAAAAgAEogQAAAAAGsGfqAOx8e8+5YOoI63bFuadPHQEAAAB2JCOBAAAAAAagBAIAAAAYgBIIAAAAYADWBIKJWGsJAACA7WQkEAAAAMAAlEAAAAAAA1ACAQAAAAxACQQAAAAwACUQAAAAwACUQAAAAAADUAIBAAAADEAJBAAAADAAJRAAAADAAJRAAAAAAANQAgEAAAAMQAkEAAAAMAAlEAAAAMAAlEAAAAAAA1ACAQAAAAxgz9QBADbT3nMumDrCul1x7ulTRwAAAAZiJBAAAADAAJRAAAAAAANQAgEAAAAMQAkEAAAAMAAlEAAAAMAAlEAAAAAAA1ACAQAAAAxgXSVQVT2wqt5fVQeq6pxVHv+eqrqqqt49//rezY8KAAAAwOHac6gLqurIJM9PclqSK5NcVFX7uvt9Ky59WXefvQUZAQAAANig9YwEukeSA919eXdfneT8JGdubSwAAAAANtN6SqDbJPngsuMr5+dWelhVXVJVr6iq4zclHQAAAACbYrMWhv6jJHu7++Qkf5rkxatdVFVPrKr9VbX/qquu2qQ/GgAAAIBDWU8J9KEky0f2HDc/9wXd/dHu/sz88IVJ7rbab9Td53X3qd196rHHHns4eQEAAAA4DOspgS5KcmJVnVBVRyV5VJJ9yy+oqi9fdnhGkss2LyIAAAAAG3XI3cG6+5qqOjvJ65IcmeS3uvvSqnp6kv3dvS/JD1fVGUmuSfIvSb5nCzMDAAAAcD0dsgRKku6+MMmFK849ddn3P5XkpzY3GgAAAACbZbMWhgYAAABgB1MCAQAAAAxACQQAAAAwACUQAAAAwACUQAAAAAADUAIBAAAADEAJBAAAADAAJRAAAADAAJRAAAAAAAPYM3UAAHa+vedcMHWEdbvi3NOnjgAAADuSkUAAAAAAA1ACAQAAAAxACQQAAAAwACUQAAAAwACUQAAAAAADUAIBAAAADEAJBAAAADAAJRAAAADAAJRAAAAAAANQAgEAAAAMQAkEAAAAMAAlEAAAAMAAlEAAAAAAA1ACAQAAAAxACQQAAAAwACUQAAAAwACUQAAAAAADUAIBAAAADGDP1AEAYFR7z7lg6gjrdsW5p08dAQCADTISCAAAAGAASiAAAACAASiBAAAAAAagBAIAAAAYgBIIAAAAYABKIAAAAIABKIEAAAAABqAEAgAAABiAEggAAABgAEogAAAAgAEogQAAAAAGoAQCAAAAGIASCAAAAGAASiAAAACAASiBAAAAAAagBAIAAAAYgBIIAAAAYABKIAAAAIABKIEAAAAABrBn6gAAAJtp7zkXTB1h3a449/SpIwAAAzESCAAAAGAASiAAAACAASiBAAAAAAagBAIAAAAYwLpKoKp6YFW9v6oOVNU5B7nuYVXVVXXq5kUEAAAAYKMOWQJV1ZFJnp/kQUlOSnJWVZ20ynU3TfLkJG/f7JAAAAAAbMx6RgLdI8mB7r68u69Ocn6SM1e57ueTPCPJf2xiPgAAAAA2wXpKoNsk+eCy4yvn576gqk5Jcnx3X7CJ2QAAAADYJBteGLqqjkjyy0l+dB3XPrGq9lfV/quuumqjfzQAAAAA67SeEuhDSY5fdnzc/NySmya5S5I3VdUVSe6ZZN9qi0N393ndfWp3n3rssccefmoAAAAArpf1lEAXJTmxqk6oqqOSPCrJvqUHu/vj3X2L7t7b3XuT/FWSM7p7/5YkBgAAAOB6O2QJ1N3XJDk7yeuSXJbk5d19aVU9varO2OqAAAAAAGzcnvVc1N0XJrlwxbmnrnHtfTceCwAAAIDNtOGFoQEAAADY+ZRAAAAAAANQAgEAAAAMQAkEAAAAMAAlEAAAAMAAlEAAAAAAA1ACAQAAAAxACQQAAAAwACUQAAAAwACUQAAAAAADUAIBAAAADEAJBAAAADAAJRAAAADAAJRAAAAAAANQAgEAAAAMQAkEAAAAMAAlEAAAAMAAlEAAAAAAA1ACAQAAAAxACQQAAAAwACUQAAAAwACUQAAAAAADUAIBAAAADEAJBAAAADAAJRAAAADAAJRAAAAAAANQAgEAAAAMQAkEAAAAMAAlEAAAAMAAlEAAAAAAA1ACAQAAAAxACQQAAAAwACUQAAAAwACUQAAAAAADUAIBAAAADEAJBAAAADAAJRAAAADAAJRAAAAAAANQAgEAAAAMQAkEAAAAMAAlEAAAAMAAlEAAAAAAA1ACAQAAAAxACQQAAAAwACUQAAAAwACUQAAAAAADUAIBAAAADEAJBAAAADAAJRAAAADAAJRAAAAAAANQAgEAAAAMQAkEAAAAMAAlEAAAAMAA1lUCVdUDq+r9VXWgqs5Z5fEfqKr3VtW7q+rPq+qkzY8KAAAAwOE6ZAlUVUcmeX6SByU5KclZq5Q8L+nur+nur0vyzCS/vNlBAQAAADh86xkJdI8kB7r78u6+Osn5Sc5cfkF3f2LZ4Y2T9OZFBAAAAGCj9qzjmtsk+eCy4yuTfP3Ki6rqSUmekuSoJPdf7TeqqicmeWKS3Pa2t72+WQEAAAA4TOspgdalu5+f5PlV9V1J/luSx65yzXlJzkuSU0891WghAIAFsPecC6aOsG5XnHv61BEAYMdaz3SwDyU5ftnxcfNzazk/yUM2kAkAAACATbaeEuiiJCdW1QlVdVSSRyXZt/yCqjpx2eHpSf528yICAAAAsFGHnA7W3ddU1dlJXpfkyCS/1d2XVtXTk+zv7n1Jzq6qb03y2SQfyypTwQAAAACYzrrWBOruC5NcuOLcU5d9/+RNzgUAAADAJlrPdDAAAAAAFpwSCAAAAGAASiAAAACAASiBAAAAAAagBAIAAAAYgBIIAAAAYABKIAAAAIABKIEAAAAABqAEAgAAABiAEggAAABgAEogAAAAgAEogQAAAAAGoAQCAAAAGIASCAAAAGAAe6YOAAAAI9p7zgVTR1i3K849feoIAGwCI4EAAAAABqAEAgAAABiAEggAAABgAEogAAAAgAEogQAAAAAGoAQCAAAAGIASCAAAAGAASiAAAACAASiBAAAAAAagBAIAAAAYgBIIAAAAYABKIAAAAIABKIEAAAAABqAEAgAAABiAEggAAABgAEogAAAAgAEogQAAAAAGoAQCAAAAGIASCAAAAGAASiAAAACAASiBAAAAAAagBAIAAAAYgBIIAAAAYABKIAAAAIABKIEAAAAABrBn6gAAAACbZe85F0wdYd2uOPf0qSMAgzESCAAAAGAASiAAAACAASiBAAAAAAagBAIAAAAYgBIIAAAAYABKIAAAAIABKIEAAAAABqAEAgAAABiAEggAAABgAEogAAAAgAEogQAAAAAGsK4SqKoeWFXvr6oDVXXOKo8/pareV1WXVNXrq+p2mx8VAAAAgMN1yBKoqo5M8vwkD0pyUpKzquqkFZe9K8mp3X1yklckeeZmBwUAAADg8K1nJNA9khzo7su7++ok5yc5c/kF3f3G7v70/PCvkhy3uTEBAAAA2Ij1lEC3SfLBZcdXzs+t5QlJ/ngjoQAAAADYXHs28zerqsckOTXJfdZ4/IlJnpgkt73tbTfzjwYAAADgINYzEuhDSY5fdnzc/Nx1VNW3JvmZJGd092dW+426+7zuPrW7Tz322GMPJy8AAAAAh2E9JdBFSU6sqhOq6qgkj0qyb/kFVXXXJC/IrAD6yObHBAAAAGAjDlkCdfc1Sc5O8roklyV5eXdfWlVPr6oz5pc9K8lNkvx+Vb27qvat8dsBAAAAMIF1rQnU3RcmuXDFuacu+/5bNzkXAAAAAJtoPdPBAAAAAFhwSiAAAACAASiBAAAAAAawrjWBAAAAGNfecy6YOsK6XXHu6VNHgB3LSCAAAACAASiBAAAAAAagBAIAAAAYgBIIAAAAYABKIAAAAIABKIEAAAAABqAEAgAAABiAEggAAABgAEogAAAAgAEogQAAAAAGoAQCAAAAGIASCAAAAGAASiAAAACAASiBAAAAAAagBAIAAAAYgBIIAAAAYABKIAAAAIABKIEAAAAABqAEAgAAABiAEggAAABgAEogAAAAgAEogQAAAAAGoAQCAAAAGIASCAAAAGAASiAAAACAASiBAAAAAAagBAIAAAAYgBIIAAAAYABKIAAAAIABKIEAAAAABqAEAgAAABiAEggAAABgAEogAAAAgAEogQAAAAAGoAQCAAAAGIASCAAAAGAASiAAAACAAeyZOgAAAACMaO85F0wdYd2uOPf0qSOwCYwEAgAAABiAEggAAABgAEogAAAAgAEogQAAAAAGoAQCAAAAGIASCAAAAGAASiAAAACAASiBAAAAAAagBAIAAAAYgBIIAAAAYABKIAAAAIABrKsEqqoHVtX7q+pAVZ2zyuPfXFXvrKprqurhmx8TAAAAgI04ZAlUVUcmeX6SByU5KclZVXXSiss+kOR7krxkswMCAAAAsHF71nHNPZIc6O7Lk6Sqzk9yZpL3LV3Q3VfMH/v8FmQEAAAAYIPWMx3sNkk+uOz4yvk5AAAAABbEti4MXVVPrKr9VbX/qquu2s4/GgAAAGBo6ymBPpTk+GXHx83PXW/dfV53n9rdpx577LGH81sAAAAAcBjWUwJdlOTEqjqhqo5K8qgk+7Y2FgAAAACb6ZAlUHdfk+TsJK9LclmSl3f3pVX19Ko6I0mq6u5VdWWS70zygqq6dCtDAwAAAHD9rGd3sHT3hUkuXHHuqcu+vyizaWIAAAAA7EDbujA0AAAAANNQAgEAAAAMQAkEAAAAMAAlEAAAAMAAlEAAAAAAA1ACAQAAAAxACQQAAAAwACUQAAAAwACUQAAAAAAD2DN1AAAAAIDNsvecC6aOsG5XnHv6tv55RgIBAAAADEAJBAAAADAAJRAAAADAAJRAAAAAAANQAgEAAAAMQAkEAAAAMAAlEAAAAMAAlEAAAAAAA1ACAQAAAAxACQQAAAAwACUQAAAAwACUQAAAAAADUAIBAAAADEAJBAAAADAAJRAAAADAAJRAAAAAAANQAgEAAAAMQAkEAAAAMAAlEAAAAMAAlEAAAAAAA1ACAQAAAAxACQQAAAAwACUQAAAAwACUQAAAAAADUAIBAAAADEAJBAAAADAAJRAAAADAAJRAAAAAAANQAgEAAAAMQAkEAAAAMAAlEAAAAMAAlEAAAAAAA1ACAQAAAAxACQQAAAAwACUQAAAAwACUQAAAAAADUAIBAAAADEAJBAAAADAAJRAAAADAAJRAAAAAAANQAgEAAAAMQAkEAAAAMAAlEAAAAMAA1lUCVdUDq+r9VXWgqs5Z5fEvqaqXzR9/e1Xt3fSkAAAAABy2Q5ZAVXVkkucneVCSk5KcVVUnrbjsCUk+1t1fmeRXkjxjs4MCAAAAcPjWMxLoHkkOdPfl3X11kvOTnLnimjOTvHj+/SuSfEtV1ebFBAAAAGAjqrsPfkHVw5M8sLu/d378n5N8fXefveyav55fc+X8+O/m1/zzit/riUmeOD/8qiTv36z/IVvsFkn++ZBXcX25r1vDfd0a7uvmc0+3hvu6NdzXreG+bg33dWu4r5vPPd0a7uvWWKT7ervuPna1B/ZsZ4ruPi/Jedv5Z26Gqtrf3adOnWO3cV+3hvu6NdzXzeeebg33dWu4r1vDfd0a7uvWcF83n3u6NdzXrbFb7ut6poN9KMnxy46Pm59b9Zqq2pPkS5N8dDMCAgAAALBx6ymBLkpyYlWdUFVHJXlUkn0rrtmX5LHz7x+e5A19qHlmAAAAAGybQ04H6+5rqursJK9LcmSS3+ruS6vq6Un2d/e+JL+Z5Heq6kCSf8msKNpNFm4K24JwX7eG+7o13NfN555uDfd1a7ivW8N93Rru69ZwXzefe7o13NetsSvu6yEXhgYAAABg8a1nOhgAAAAAC04JBAAAADAAJRAAAADAAJRA61BVR1TVMVPngNVU1QnrOQcAAOx+3r9yMBaGXkNVvSTJDyT5XJKLkhyT5Dnd/axJg+0CVfWNSU7s7hdV1bFJbtLdfz91rkVVVe/s7lNWnLu4u+82VaZFV1X3S/JDSb5qfuqyJM/r7jdNFmqBVdVzk6z5w6a7f3gb4+xanls3X1XdIcmV3f2ZqrpvkpOT/HZ3/+uUuRbd/O/n9yXZm2U71Xb346fKtMiq6pSDPd7d79yuLLuFe7q1qurLVjn9ye7+7LaH2UW8f918u/Xn1SG3iB/YSd39iap6dJI/TnJOkouT+Ee0AVX1s0lOzezN9YuS3CDJ7ya595S5FlFV3SnJnZN8aVU9dNlDxyQ5eppUi6+qTk/yvCRPT/K0JJXklCS/VVVnd/eFU+ZbUPunDrDbeW7dMq9McmpVfWVm28L+YZKXJPm2SVMtvj9M8tYkf5bZmxU25tnzX4/O7HngPZn97Do5s+ffe02Ua5E9+yCPdZL7b1eQXeqdSY5P8rHM/q7eLMk/VdWHk3xfd188YbZF5v3r5tuVP6+UQGu7QVXdIMlDMhsB8NmqMmxq474jyV0ze/JPd/+/qrrptJEW1lcl+fbMfnA+eNn5T2bWWHN4fjzJQ7r7PcvOvbuq9id5bhIl0PXU3S9eflxVN5mf/7dpEu1Knlu3xue7+5qq+o4kz+3u51bVu6YOtQvcqLt/cuoQu0V33y9JqupVSU7p7vfOj++S5OcmjLawlu4pW+ZPk7yiu1+XJFX1gCQPy+xDjF9L8vUTZltk3r9uvl3588qaQGt7QZIrktw4yVuq6nZJPjFpot3h6p7NQewkqaobT5xnYXX3H3b345J8e3c/btnXD3f326bOt8BuvaIASpJ09yVJbjVBnl2jqu4yfwN9aZL3VdXFVXXnqXPtEp5bt8Znq+qsJI9N8pr5uRtMmGe3eE1VGU21+b5qqQBKku7+6yRfPWGehVdVT6qqmy07vnlV/ZcJI+0W91wqgJKku/8kyb26+6+SfMl0sRae96+bb1f+vLIm0PVQVXu6+5qpcyyyqvqxJCcmOS3JLyZ5fJKXdPdzJw22wHbrXNWpHGw9JWstbUxVvS3Jz3T3G+fH903yP7v7G6bMtRt4bt0aVXVSZusr/GV3v3S+6P4juvsZE0dbSFX1ycyKysrsTcpnknx2ftzdbRHTDaiqlyb5VGZTQZPk0ZmtDXbWdKkWW1W9u7u/bsW5d3X3XSeKtCtU1Z8keX2S8+enHpnZz68HJrlo5VqXHD7vXw/Pbv95pQRaoaqecrDHu/uXtyvLblVVpyV5QGb/iF7X3X86caSFNn9j/dbM5vx+Ya5qd79yslALrKr+NclbVnsoyTd29823N9HuUVXv6e6vPdQ5Do/nVhhbVR2d5AeTfPP81FuS/Hp3/8d0qRZbVb03ycnzkZapqiOTXNLdRrFuQFXdIsnPJvnG+am/yGwdxo8nuW13H5gq2yLy/pXrSwm0wnxxzWS23srdk+ybHz84yTu6+zGTBIM1rPYpFYevqu4z//aGmY2s6CQHkvx7knT3myeKtvCq6tWZrVnzO/NTj0lyt+7+julSwRebv/Fb7QXS0ieAJ29zpF2lql7f3d9yqHMwtap6VpLbZTbNJkm+P8kHuvvHpksF17Xs/euquvtp25Vlt6mqeyd5d3d/qqoek9lmMf+ruz8wcbQNUQKtoarekuT07v7k/PimSS7o7m8++H/Jwcx3sXpGkltm9mJ6Vwypm1JV/UKSt9m1anPMF9T7H5lNp1l6gj8+yf9J8tO2Lz18VXXzzD7pW/rk761Jfq67PzZdqt1h2bDlJDkqs3VrPuW59fDM11FYU3f/w3Zl2U3mI1VunOQNSe6b2WuAZLar5Wu7+04TRVtoVfXy7n7EWuWl0vLwVdURSZ6YZKmgvCSztQOfNF2qxVVV+w72eHefsV1ZYD2q6pIkX5vZbov/J8kLM5sWfp+D/Xc7nd3B1narJFcvO746FoXdDM9M8uDuvmzqILvIk5P8dFXtqrmqE3pmkpskOWFZCXxMkl+afz15wmwLbV72/PDUOXaj7v7CTmBVVUnOTHLP6RIttuUlT1XdKrORwclsRPBHpkm1K3x/kv+a5Csym8K8VAJ9IsnzJsq0Gyz9XPr2SVPsQt39+ap6e5I7JHlEklskMd3+8N0ryQeTvDTJ23PtcwCboKqOy2wn23vPT701yZO7+8rpUi28a7q7q+rMzHZc+82qesLUoTbKSKA1VNXPZPZk/+r5qYckeVl3/+JkoXaBqvqL7r73oa+EaVTV3ya5Y694cpyvA/A33X3iNMkWX1XdMcmP5YsXMb//VJl2M4uXblxVPSLJs5K8KbM3K9+U5Me7+xVT5lp0VfVDFi3ffPOFy/9xaQ2gqrphklt19xWTBltA859XZ82//jnJy5L8WHcfdJQgBzd/LXVaZvf15CQXJHlpd186abBdoqr+NMlLct1p94/u7tOmS7XYqurNSV6b2QyBb0rykcymhy30CEsl0EFU1SmZ/Z+dJG/p7ndNmWc3qKrnJLl1kj/IbJX1JEl3v2qqTIuqqu7U3X8z/3v6Rbr7ndudaTeoqv/b3Xe8vo9xaFX1niT/O1+8iPnFk4XaJeZTbZcckeTUJPfp7ntNFGlXmP+dPW1p9M98N8Y/s5j5xlXVN+SLC+HfnizQLlBV+5N8Q3dfPT8+KslfdPfdD/5fslJVfT6zURRPWFqkuKou7+7bT5ts96iqL8msDHpWkqd1t9GAG7TGbnbWDt2Aqrp1ku/KbCTwn1fVNyd5UXffYeJoG2I62ApVdUx3f6KqvizJFfOvpce+rLv/Zapsu8QxST6d2Q42SzqJEuj6e0pm89SfvcpjncToisPzvqr67pVvRuaLwf3NRJl2i2u6+9enDrFLPXjZ99dk9rPrzGmi7CpHrJj+9dHMSjY2oKp+J7PpNe/OtYVwJ1ECbcyepQIoSbr76nkRxPX30CSPSvLGqnptZluZm7q0Ceblz+mZFUB7k/xqrp15wcZ8dP569aXz47My+7nFYeruf6qqNyb5rqr63SR/n+R/TZtq44wEWqGqXtPd315Vf5/ZC5LlT/jtEwDY3arqNpmVkv+e2YiVZDaq4oZJvqO7PzRVtkU1L9WT2XpAH8nsxd7ykYDKdXak+c5AJ+faF9SPzGx76J+cLtXiq6rLkpy0ctotGzOfCvLc7t43Pz4zyQ/bde3wVdWNMyvUz8rsw7XfTvLq7v6TSYMtqKr67SR3SXJhkvO7+68njrSrzDc1eG5may91krdl9hyw0DtZTWG3TwlVArEtquonuvuZVfXcrL5zhcViN1FV3bq7/2nqHIusqu6f5M7zw/d19+unzLPI1ijVlyjXN2Ct59Qlnls3rqoelmWLbHa3T6w3qKp+P7M3Jv84dZbdpKrukOT3Mlt4uzJbgPe7l6YzsTHzHS6/M8kjFWuHZz7N7lPzw+U/u2xqwo6y26eEKoEOoqrOSLK0Jfybuvs1U+ZZZFX14O7+o6p67GqPd/eLtzvTblZVF3T36VPnALbWiufUpyX52eWPe25lJ5oPrf+6JO/IdUcF2h56E1TVTZKku/9t6izA9pmvW/d9+eL11h4/VaZFVVUPyWxK6L0zWxj6/CQv7O4Tpsy1WZRAa6iqczPbEvb35qfOSnJRd//0dKnguua7LPxZd99v6ixwKFV1gyQ/mGXlepIXdPdnJwu1i9gNbPPNF9x+RpJbZvZJtU+rN0FV3We189395u3OsttU1emZjWI9eulcdz99ukTAdqmqt2U2emXlBhyvnCzUgtutU0KVQGuoqkuSfF13f35+fGSSdy36dnBTqao/ysGnLPj07zBV1euTPLS7Pz51FjiYqnphkhskWRqd8p+TfK67v3e6VLtHVb2zu1fdLZDDU1UHkjy4uy+bOgscSlX97yQ3SnK/JC9M8vDMdrR5wqTBgG1hJ7CttZumhNod7OBulmRpwdIvnTDHbvBLUwfYxf4tyXvnC0IuzbO2Fgg70d1XbK39hvkW3LBTfVgBtPmq6p6ZLV761UmOSnJkkk8ZYbVh39DdJ1fVJd39tKp6dpI/njoUsG1eU1Xf1t0XTh1kN+rujyU5b/610JRAa/vFJO+az1uvzKYvnDNtpMW1fIj3fLvSO84P328qyIa9av4FO93nquoO3f13SVJVt8+y4cpcf1X1yVw7yvJGVfWJpYdi2tJm2F9VL0vyB7nu2jWeczfmeZmttfD7me2++N259nUBh+/f579+uqq+IrOtob98wjzA9npykp+uqquTXB2vBViDEmgN3f3SqnpTZusCJclP2m1p46rqvplNBbkisyem46vqsd39lgljLbTufrFijQXx40neWFWXZ/bv/3ZJHjdtpMXW3TedOsMud0ySTyd5wLJzHcX7hnX3gao6srs/l+RFVfWuJD81da4F95qqulmSZyV5Z2Z/V39j0kTAtvGagPWyJtBBVNVtMnuTsnx1dWXFBlTVxUm+q7vfPz++Y5KXdvfdpk22uFYr1pIo1tiRqupLknzV/PD93f2Zg10P7D5V9ZYk35rZujX/lOQfk3zPiumibMD8ufZo6wXCOKqqkjw6yQnd/fNVdXySL+/ud0wcjR1GCbSGqnpGkkcmuTTJ5+en2wLGGzOfp37yoc6xfoo1FkVVPSnJ73X3v86Pb57krO7+tUmDwRrmz6e/nuRW3X2Xqjo5yRnd/QsTR1toVXW7JB/ObD2gH8ls3cVf6+4Dkwbbharq1kaywxiq6tcze996/+7+6vnrrD/p7rsf4j9lMEqgNVTV+5Oc7FPqzVVVL8psDZDfnZ96dJIju/vx06VabIo1FsVqu1bY1pydrKrenNk0xhcs/T2tqr/u7rtMm2zxmca8Parqgu4+feocwNZb2iV0+WurqnqPUZasZE2gtV2e2VbGSqDN9QNJnpRkaeeqtyYxCmBj9s+33l5erO2fMA+s5ciqqp5/+lBVR2Y2EgB2qht19ztmI+y/4JqpwuwW1gfcPgogGMpn56+tll5nHZtrZ7TAFyiB1vbpJO+uqtfnujuC2Hb7MM2flN7T3XdK8stT59lFfjCKNRbDa5O8rKpeMD/+/vk52Kn+uarukGtfUD88s/Vr2JhnJ3nAymnMSUxj3oCqumeSS7v7k/PjY5J8dXe/fdpkwDb51SSvTnLLqvofSR6e5L9NG4mdyHSwNVTVY1c7390v3u4su0lV/WGSH+ruD0ydBdheVXVEZsXPt8xP/WmSF853B4Idp6pun+S8JN+Q5GNJ/j7JY7r7iilzLTrTmLfGfIe1U5aNtjwiyf7uPmXaZMB2qao7ZfY6q5K8vrsvmzgSO5ASiG013xHkrknekeRTS+ctuH39VdV7M/90ejVeTANsjqq6cZIjlkZYsDHWB9waa6y7plyDgcxnXtwq193d2ofvXIcSaA1VdWKSX0xyUpKjl8539+0nC7ULVNV9Vjvf3W/e7iyLbr67ypq6+x+2KwsczCEKy7ZgITvVfJvthyXZm+u+oH76VJl2g/l9fVKSb5yfemtmu4NZh3EDqupVSd6U2Y52SfJfktyvux8yVSZg+1TVDyX52cx2X/xcZqOBWhHMSkqgNVTVn2f2j+hXkjw4yeMy+xTwqZMGW1BVdXRmi0J/ZZL3JvnN7ra45iapqlslWdr+8R3d/ZEp88ByaxSWleT4JD/V3d+2zZFgXarqtUk+nuTizF5QJ0m6+9mThVpw80+pL52vD8gmqqpbZrYmyP0zK95fn+S/ek0AY6iqA0m+vrs/OnUWdjYl0Bqq6uLuvltVvbe7v2b5uamzLaKqelmSz2b2ad+DkvxDdz952lS7Q1U9IsmzMvv0r5J8U5If7+5XTJkLVlNVd03yXUm+M7P1VV7Z3c+bNhWsznbwW8P6gACbr6remOQ0H7RzKHYHW9tn5gvq/W1VnZ3kQ0luMnGmRXbSsjLtNzNbE4jN8TNJ7r70Sd98O8g/S6IEYkeY7/xz1vzrn5O8LLMPIe43aTA4tLdV1dd093unDrLL3DzJpVVlfcBNUFU/0d3PrKrnZpWpt3a2hd2tqp4y//byJG+qqgty3d2t7crMdSiB1vbkJDfKbNvtn09yvySr7hjGunx26Zvuvqaqpsyy2xyxYqj3R5McMVUYWMXfZDYK8Nu7+0CSVNWPTBsJ1rZsHas9SR5XVZdn9oLa+gqb479PHWCXWdr9Z/+kKYCp3HT+6wfmX0fNv2BVpoOtoapO6e53Tp1jt6iqz+XaT/sqyQ2TfDrXvqA+Zqpsi66qnpXk5CQvnZ96ZJJLuvsnp0sF16qqhyR5VJJ7J3ltkvMz2xr+hClzwVosvL81rA8IsPWq6pjM3l/Z0ZJVKYHWMJ9TeevMptS8rLv/euJIcB1V9ZVJbtXdf1FVD821u6z8a5Lf6+6/mywcrGK+zfaZmU0Lu3+S307y6u7+k0mDwQrKiq1hfcCtUVV/lLV3YDTNDgZRVacmeVGuHRn08SSP7+6Lp0vFTqQEOoiqunWSR2Q2suKYzMqgX5g2FcxU1Wsy21npvSvOf02S/9ndD54mGRxaVd08s8WhH9nd3zJ1HlhOWbE1Vmy2sSez3SxPmTjWwquq+xzs8e5+83ZlAaZTVZckeVJ3v3V+/I1Jfs0UZlZSAq3D/E31T2T2ZsX8SnaEqrqou+++xmNfeKENwPWjrNgaVfXO5fdx5TEAh6+q3tXdd11xzvMsX8TC0Guoqq/ObATQwzJbaPdlSX500lBwXTc7yGM33K4QALuQzQy2xtdW1Sfm31eSG86PrQ+4CarqxCS/mOSkJEcvne/u208WCthOb66qF2S2Tmhn9l72TVV1SpJY75YlRgKtoar+MrPFS3+/u//f1Hlgpap6aZI3dPdvrDj/vUlO6+5HTpMMYLHZzIBFVFV/nuRnk/xKkgcneVxmO4g+ddJgwLaYr2m7lu7u+29bGHY0JRAsqKq6VZJXJ7k6ydKCb6dmtiXkd3T3P02VDQDYXlV1cXffbcV0xou7+25TZwNg5zAdbA2G1LLTdfeHk3xDVd0vyV3mpy/o7jdMGAsAmMZnquqIJH9bVWcn+VCSm0ycCZhQVZ1iGhgrGQm0BkNqAQBYFFV19ySXZbZm4M8n+dIkz+zuv5oyFzCdqvqN7v6+qXOwsyiB1mBILQAAALCbmA62NkNqAQDY0apq38Ee7+4ztisLsP2Wdv9ai+lgrGQk0BoMqQUAYKerqquSfDCzbaHfntkudl/Q3W+eIhewPZbtCnZ0ZpvEvCez54GTk+zv7ntNlY2dSQkEAAALqqqOTHJakrMye9N3QZKXdvelkwYDtlVVvSrJz3b3e+fHd0nyc9398GmTsdMogVaoqj9KsuZNMaQWAICdqKq+JLMy6FlJntbdz5s4ErBNqurS7r7zoc6BNYG+2C9NHQAAANZrXv6cnlkBtDfJryZ59ZSZgG13SVW9MMnvzo8fneSSCfOwQxkJtEJV3ba7PzB1DgAAOJSq+u0kd0lyYZLzu/uvJ44ETKCqjk7yg0m+eX7qLUl+vbv/Y7pU7ERKoBWq6p3dfcr8+1d298OmzgQAAKupqs8n+dT8cPkL+0rS3X3M9qcCYKcyHeyLLd9R4faTpQAAgEPo7iOmzgBMp6pe3t2PqKr3ZpW1bbv75AlisYMpgb5Yr/E9AAAA7CRPnv/67ZOmYGGYDrZCVX0usyG1leSGST699FAMqQUAAGAHqqpbJbn7/PAd3f2RKfOwMymBAAAAYIFV1SOSPCvJmzIbwPBNSX68u18xZS52HiUQAAAALLCqek+S05ZG/1TVsUn+rLu/dtpk7DQWkgMAAIDFdsSK6V8fjff7rMLC0AAAALDYXltVr0vy0vnxI5NcOGEedijTwQAAAGDBVdXDktx7fvjW7n71lHnYmZRAAAAAAAMwRxAAAAAWWFU9tKr+tqo+XlWfqKpPVtUnps7FzmMkEAAAACywqjqQ5MHdfdnUWdjZjAQCAACAxfZhBRDrYSQQAAAALLCqek6SWyf5gySfWTrf3a+aKhM7ky3iAQAAYLEdk+TTSR6w7FwnUQJxHUYCAQAAAAzASCAAAABYYFV1dJInJLlzkqOXznf34ycLxY5kYWgAAABYbL+T2ZpA/ynJm5Mcl+STkyZiRzIdDAAAABZYVb2ru+9aVZd098lVdYMkb+3ue06djZ3FSCAAAABYbJ+d//qvVXWXJF+a5JYT5mGHsiYQAAAALLbzqurmSf57kn1JbjL/Hq7DdDAAAABYYFV1ZHd/buoc7HymgwEAAMBi+/uqOq+qvqWqauow7FxKIAAAAFhsd0ryZ0melOSKqnpeVX3jxJnYgUwHAwAAgF1ivjbQc5I8uruPnDoPO4uRQAAAALDgquo+VfVrSS5OcnSSR0wciR3ISCAAAABYYFV1RZJ3JXl5kn3d/alpE7FTKYEAAABggVXVMd39ialzsPMpgQAAAGCBVdXRSZ6Q5M6ZTQVLknT34ycLxY5kTSAAAABYbL+T5NZJ/lOSNyc5LsknJ03EjmQkEAAAACywqnpXd9+1qi7p7pOr6gZJ3trd95w6GzuLkUAAAACw2D47//Vfq+ouSb40yS0nzMMOtWfqAAAAAMCGnFdVN0/y35PsS3KTJE+dNhI7kelgAAAAAAMwEggAAAAWUFU95WCPd/cvb1cWFoMSCAAAABbTTacOwGJRAgEAAMBiulF3/2RVfWd3//7UYdj57A4GAAAAi+nbqqqS/NTUQVgMRgIBAADAYnptko8luUlVfWLZ+UrS3X3MNLHYqewOBgAAAAusqv6wu8+cOgc7nxIIAAAAYADWBAIAAIAFVlUPraq/raqPV9UnquqTK6aHQRIjgQAAAGChVdWBJA/u7sumzsLOZiQQAAAALLYPK4BYDyOBAAAAYIFV1XOS3DrJHyT5zNL57n7VVJnYmWwRDwAAAIvtmCSfTvKAZec6iRKI6zASCAAAAGAA1gQCAACABVZVx1XVq6vqI/OvV1bVcVPnYudRAgEAAMBie1GSfUm+Yv71R/NzcB2mgwEAAMACq6p3d/fXHeocGAkEAAAAi+2jVfWYqjpy/vWYJB+dOhQ7j5FAAAAAsMCq6nZJnpvkXpntCva2JD/c3R+YNBg7jhIIAAAAYAB7pg4AAAAAXH9V9dSDPNzd/fPbFoaFYCQQAAAALKCq+tFVTt84yROS/H/dfZNtjsQOpwQCAACABVdVN03y5MwKoJcneXZ3f2TaVOw0poMBAADAgqqqL0vylCSPTvLiJKd098emTcVOpQQCAACABVRVz0ry0CTnJfma7v63iSOxw5kOBgAAAAuoqj6f5DNJrslsa/gvPJTZwtDHTBKMHUsJBAAAADCAI6YOAAAAAMDWUwIBAAAADEAJBAAAADAAJRAAAADAAJRAAAAAAAP4/wEmHWKyzM+rdQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 1440x576 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#let's plot the ordered mutual_info values per feature\n",
    "mutual_info.sort_values(ascending=False).plot.bar(figsize=(20, 8))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d1c1bf46",
   "metadata": {},
   "source": [
    "\n",
    "Difference Between Information Gain And Mutual Information\n",
    "\n",
    "I(X ; Y) = H(X) – H(X | Y) and IG(S, a) = H(S) – H(S | a)\n",
    "\n",
    "As such, mutual information is sometimes used as a synonym for information gain. Technically, they calculate the same quantity if applied to the same data.\n",
    "Comparion of Univariate And Mutual Information\n",
    "\n",
    "Comparison of F-test and mutual information https://scikit-learn.org/stable/auto_examples/feature_selection/plot_f_test_vs_mi.html#sphx-glr-auto-examples-feature-selection-plot-f-test-vs-mi-py\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c79de8fb",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}