dbal_keras.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Deep Bayesian Active Learning with Image Data\n",
    "\n",
    "This is an implementation of the paper Deep Bayesian Active Learning with Image Data using keras and modAL. [modAL](https://modal-python.readthedocs.io/en/latest/) is an active learning framework for Python3, designed with modularity, flexibility and extensibility in mind. Built on top of scikit-learn, it allows you to rapidly create active learning workflows with nearly complete freedom. What is more, you can easily replace parts with your custom built solutions, allowing you to design novel algorithms with ease.\n",
    "\n",
    "## Active Learning\n",
    "\n",
    "In this notebook, we are concerned with pool-based Active Learning. In this setting, we have a large amount of unlabelled data and a small initial labelled training set and we want to choose what data should be labelled next.\n",
    "\n",
    "To do so, there are several query strategies. In this notebook, we will be using uncertainty sampling: the data chosen to be annotated is the one that maximizes an uncertainty criterion (entropy, gini index, variation ratios ...). \n",
    "\n",
    "## Dropout-Based Bayesian Deep Neural Networks\n",
    "\n",
    "In this Notebook, we will select the data from the unlabelled pool that maximizes the uncertainty of our model. But the model we will be using will be a Bayesian Deep Neural Network. \n",
    "\n",
    "Unlike Traditional Deep Learning, where we are looking for the set of weights that maximizes the likelihood of the data (MLE), in bayesian deep learning we are looking for the posterior distribution over the weights and the prediction is then obtained by marginalizing out the weights. As a result, Bayesian models are less prone to overfitting. But unfortunately for big deep models, the posterior distribution is intractable, and we need approximations. \n",
    "\n",
    "In 2015, [Gal and Ghahramani](https://arxiv.org/pdf/1506.02142.pdf) showed that deep models with dropout layers can be viewed as a lightweight bayesian approximation. The prior and posterior distributions are simply Bernoulli distributions (0 or the learned value). And the predictions can be cheaply obtained at test time by performing Monte Carlo integrations with dropout layers activated.\n",
    "\n",
    "So in a nutshell, Dropout-based Bayesian Neural Nets are simply Neural Nets with Dropout layers activated at test time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "import keras\n",
    "import numpy as np\n",
    "from keras import backend as K\n",
    "from keras.datasets import mnist\n",
    "from keras.models import Sequential\n",
    "from keras.layers import Dense, Dropout, Activation, Flatten, Conv2D, MaxPooling2D\n",
    "from keras.regularizers import l2\n",
    "from keras.wrappers.scikit_learn import KerasClassifier\n",
    "from modAL.models import ActiveLearner\n",
    "import tensorflow as tf\n",
    "\n",
    "tf.logging.set_verbosity(tf.logging.ERROR)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "def create_keras_model():\n",
    "    model = Sequential()\n",
    "    model.add(Conv2D(32, (4, 4), activation='relu'))\n",
    "    model.add(Conv2D(32, (4, 4), activation='relu'))\n",
    "    model.add(MaxPooling2D(pool_size=(2, 2)))\n",
    "    model.add(Dropout(0.25))\n",
    "    model.add(Flatten())\n",
    "    model.add(Dense(128, activation='relu'))\n",
    "    model.add(Dropout(0.5))\n",
    "    model.add(Dense(10, activation='softmax'))\n",
    "    model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=[\"accuracy\"])\n",
    "    return model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### create the classifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "classifier = KerasClassifier(create_keras_model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### read training data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "(X_train, y_train), (X_test, y_test) = mnist.load_data()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### preprocessing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train = X_train.reshape(60000, 28, 28, 1).astype('float32') / 255.\n",
    "X_test = X_test.reshape(10000, 28, 28, 1).astype('float32') / 255.\n",
    "y_train = keras.utils.to_categorical(y_train, 10)\n",
    "y_test = keras.utils.to_categorical(y_test, 10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### initial labelled data\n",
    "We initialize the labelled set with 20 balanced randomly sampled examples"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "initial_idx = np.array([],dtype=np.int)\n",
    "for i in range(10):\n",
    "    idx = np.random.choice(np.where(y_train[:,i]==1)[0], size=2, replace=False)\n",
    "    initial_idx = np.concatenate((initial_idx, idx))\n",
    "\n",
    "X_initial = X_train[initial_idx]\n",
    "y_initial = y_train[initial_idx]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### initial unlabelled pool"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_pool = np.delete(X_train, initial_idx, axis=0)\n",
    "y_pool = np.delete(y_train, initial_idx, axis=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Query Strategies"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Uniform\n",
    "All the acquisition function we will use will be compared to the uniform acquisition function $\\mathbb{U}_{[0,1]}$ which will be our baseline that we would like to beat."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "def uniform(learner, X, n_instances=1):\n",
    "    query_idx = np.random.choice(range(len(X)), size=n_instances, replace=False)\n",
    "    return query_idx, X[query_idx]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Entropy\n",
    "Our first acquisition function is the entropy:\n",
    "$$ \\mathbb{H} = - \\sum_{c} p_c \\log(p_c)$$\n",
    "where $p_c$ is the probability predicted for class c. This is approximated by:\n",
    "\\begin{align}\n",
    "p_c &= \\frac{1}{T} \\sum_t p_{c}^{(t)} \n",
    "\\end{align}\n",
    "where $p_{c}^{t}$ is the probability predicted for class c at the t th feedforward pass."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "def max_entropy(learner, X, n_instances=1, T=100):\n",
    "    random_subset = np.random.choice(X.shape[0], 2000, replace=False)\n",
    "    MC_output = K.function([learner.estimator.model.layers[0].input, K.learning_phase()],\n",
    "                           [learner.estimator.model.layers[-1].output])\n",
    "    learning_phase = True\n",
    "    MC_samples = [MC_output([X[random_subset], learning_phase])[0] for _ in range(T)]\n",
    "    MC_samples = np.array(MC_samples)  # [#samples x batch size x #classes]\n",
    "    expected_p = np.mean(MC_samples, axis=0)\n",
    "    acquisition = - np.sum(expected_p * np.log(expected_p + 1e-10), axis=-1)  # [batch size]\n",
    "    idx = (-acquisition).argsort()[:n_instances]\n",
    "    query_idx = random_subset[idx]\n",
    "    return query_idx, X[query_idx]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Variation Ratio\n",
    "the Variation ratio is computed according to:\n",
    "\\begin{align}\n",
    "\n",
    "\\end{align}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "def var_ratio(learner, X, n_instances=1, T=100):\n",
    "    random_subset = np.random.choice(X.shape[0], 2000, replace=False)\n",
    "    MC_output = K.function([learner.estimator.model.layers[0].input, K.learning_phase()],\n",
    "                           [learner.estimator.model.layers[-1].output])\n",
    "    learning_phase = True\n",
    "    MC_samples = [MC_output([X[random_subset], learning_phase])[0] for _ in range(T)]\n",
    "    MC_samples = np.array(MC_samples)  # [#samples x batch size x #classes]\n",
    "    preds = np.argmax(a, axis=2)\n",
    "    mode, count = stats.mode(preds, axis=0)\n",
    "    acquisition = (1 - count / preds.shape[1]).reshape((-1,))\n",
    "    idx = (-acquisition).argsort()[:n_instances]\n",
    "    query_idx = random_subset[idx]\n",
    "    return query_idx, X[query_idx]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "def bald(learner, X, n_instances, T=100):\n",
    "    random_subset = np.random.choice(X.shape[0], 2000, replace=False)\n",
    "    MC_output = K.function([learner.estimator.model.layers[0].input, K.learning_phase()],\n",
    "                           [learner.estimator.model.layers[-1].output])\n",
    "    learning_phase = True\n",
    "    MC_samples = [MC_output([X[random_subset], learning_phase])[0] for _ in range(T)]\n",
    "    MC_samples = np.array(MC_samples)  # [#samples x batch size x #classes]\n",
    "    expected_entropy = - np.mean(np.sum(MC_samples * np.log(MC_samples + 1e-10), axis=-1), axis=0)  # [batch size]\n",
    "    expected_p = np.mean(MC_samples, axis=0)\n",
    "    entropy_expected_p = - np.sum(expected_p * np.log(expected_p + 1e-10), axis=-1)  # [batch size]\n",
    "    acquisition = entropy_expected_p - expected_entropy\n",
    "    idx = (-acquisition).argsort()[:n_instances]\n",
    "    query_idx = random_subset[idx]\n",
    "    return query_idx, X[query_idx]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Active Learning Procedure"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "def active_learning_procedure(query_strategy,\n",
    "                              X_test,\n",
    "                              y_test,\n",
    "                              X_pool,\n",
    "                              y_pool,\n",
    "                              X_initial,\n",
    "                              y_initial,\n",
    "                              estimator,\n",
    "                              epochs=50,\n",
    "                              batch_size=128,\n",
    "                              n_queries=100,\n",
    "                              n_instances=10,\n",
    "                              verbose=0):\n",
    "    learner = ActiveLearner(estimator=estimator,\n",
    "                            X_training=X_initial,\n",
    "                            y_training=y_initial,\n",
    "                            query_strategy=query_strategy,\n",
    "                            verbose=verbose\n",
    "                           )\n",
    "    perf_hist = [learner.score(X_test, y_test, verbose=verbose)]\n",
    "    for index in range(n_queries):\n",
    "        query_idx, query_instance = learner.query(X_pool, n_instances)\n",
    "        learner.teach(X_pool[query_idx], y_pool[query_idx], epochs=epochs, batch_size=batch_size, verbose=verbose)\n",
    "        X_pool = np.delete(X_pool, query_idx, axis=0)\n",
    "        y_pool = np.delete(y_pool, query_idx, axis=0)\n",
    "        model_accuracy = learner.score(X_test, y_test, verbose=0)\n",
    "        print('Accuracy after query {n}: {acc:0.4f}'.format(n=index + 1, acc=model_accuracy))\n",
    "        perf_hist.append(model_accuracy)\n",
    "    return perf_hist"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy after query 1: 0.4677\n",
      "Accuracy after query 2: 0.5179\n",
      "Accuracy after query 3: 0.5606\n",
      "Accuracy after query 4: 0.6541\n",
      "Accuracy after query 5: 0.7245\n",
      "Accuracy after query 6: 0.6952\n",
      "Accuracy after query 7: 0.7183\n",
      "Accuracy after query 8: 0.7759\n",
      "Accuracy after query 9: 0.8066\n",
      "Accuracy after query 10: 0.7511\n",
      "Accuracy after query 11: 0.7592\n",
      "Accuracy after query 12: 0.8494\n",
      "Accuracy after query 13: 0.8656\n",
      "Accuracy after query 14: 0.8788\n",
      "Accuracy after query 15: 0.8816\n",
      "Accuracy after query 16: 0.8990\n",
      "Accuracy after query 17: 0.8869\n",
      "Accuracy after query 18: 0.9089\n",
      "Accuracy after query 19: 0.9076\n",
      "Accuracy after query 20: 0.9043\n",
      "Accuracy after query 21: 0.9105\n",
      "Accuracy after query 22: 0.9178\n",
      "Accuracy after query 23: 0.9241\n",
      "Accuracy after query 24: 0.9256\n",
      "Accuracy after query 25: 0.9323\n",
      "Accuracy after query 26: 0.9269\n",
      "Accuracy after query 27: 0.9366\n",
      "Accuracy after query 28: 0.9346\n",
      "Accuracy after query 29: 0.9413\n",
      "Accuracy after query 30: 0.9458\n",
      "Accuracy after query 31: 0.9458\n",
      "Accuracy after query 32: 0.9481\n",
      "Accuracy after query 33: 0.9520\n",
      "Accuracy after query 34: 0.9434\n",
      "Accuracy after query 35: 0.9515\n",
      "Accuracy after query 36: 0.9496\n",
      "Accuracy after query 37: 0.9412\n",
      "Accuracy after query 38: 0.9553\n",
      "Accuracy after query 39: 0.9579\n",
      "Accuracy after query 40: 0.9562\n",
      "Accuracy after query 41: 0.9576\n",
      "Accuracy after query 42: 0.9570\n",
      "Accuracy after query 43: 0.9638\n",
      "Accuracy after query 44: 0.9646\n",
      "Accuracy after query 45: 0.9627\n",
      "Accuracy after query 46: 0.9644\n",
      "Accuracy after query 47: 0.9663\n",
      "Accuracy after query 48: 0.9690\n",
      "Accuracy after query 49: 0.9663\n",
      "Accuracy after query 50: 0.9626\n",
      "Accuracy after query 51: 0.9632\n",
      "Accuracy after query 52: 0.9662\n",
      "Accuracy after query 53: 0.9708\n",
      "Accuracy after query 54: 0.9725\n",
      "Accuracy after query 55: 0.9656\n",
      "Accuracy after query 56: 0.9701\n",
      "Accuracy after query 57: 0.9681\n",
      "Accuracy after query 58: 0.9698\n",
      "Accuracy after query 59: 0.9665\n",
      "Accuracy after query 60: 0.9739\n",
      "Accuracy after query 61: 0.9739\n",
      "Accuracy after query 62: 0.9741\n",
      "Accuracy after query 63: 0.9695\n",
      "Accuracy after query 64: 0.9725\n",
      "Accuracy after query 65: 0.9763\n",
      "Accuracy after query 66: 0.9784\n",
      "Accuracy after query 67: 0.9737\n",
      "Accuracy after query 68: 0.9741\n",
      "Accuracy after query 69: 0.9731\n",
      "Accuracy after query 70: 0.9740\n",
      "Accuracy after query 71: 0.9749\n",
      "Accuracy after query 72: 0.9761\n",
      "Accuracy after query 73: 0.9777\n",
      "Accuracy after query 74: 0.9778\n",
      "Accuracy after query 75: 0.9755\n",
      "Accuracy after query 76: 0.9780\n",
      "Accuracy after query 77: 0.9769\n",
      "Accuracy after query 78: 0.9777\n",
      "Accuracy after query 79: 0.9760\n",
      "Accuracy after query 80: 0.9770\n",
      "Accuracy after query 81: 0.9785\n",
      "Accuracy after query 82: 0.9786\n",
      "Accuracy after query 83: 0.9809\n",
      "Accuracy after query 84: 0.9808\n",
      "Accuracy after query 85: 0.9810\n",
      "Accuracy after query 86: 0.9797\n",
      "Accuracy after query 87: 0.9806\n",
      "Accuracy after query 88: 0.9790\n",
      "Accuracy after query 89: 0.9805\n",
      "Accuracy after query 90: 0.9813\n",
      "Accuracy after query 91: 0.9814\n",
      "Accuracy after query 92: 0.9815\n",
      "Accuracy after query 93: 0.9825\n",
      "Accuracy after query 94: 0.9828\n",
      "Accuracy after query 95: 0.9822\n",
      "Accuracy after query 96: 0.9820\n",
      "Accuracy after query 97: 0.9806\n",
      "Accuracy after query 98: 0.9810\n",
      "Accuracy after query 99: 0.9830\n",
      "Accuracy after query 100: 0.9830\n"
     ]
    }
   ],
   "source": [
    "estimator = KerasClassifier(create_keras_model)\n",
    "entropy_perf_hist = active_learning_procedure(max_entropy,\n",
    "                                              X_test,\n",
    "                                              y_test,\n",
    "                                              X_pool,\n",
    "                                              y_pool,\n",
    "                                              X_initial,\n",
    "                                              y_initial,\n",
    "                                              estimator,)\n",
    "np.save(\"keras_max_entropy.npy\", entropy_perf_hist)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy after query 1: 0.5112\n",
      "Accuracy after query 2: 0.5954\n",
      "Accuracy after query 3: 0.6834\n",
      "Accuracy after query 4: 0.7142\n",
      "Accuracy after query 5: 0.7049\n",
      "Accuracy after query 6: 0.7340\n",
      "Accuracy after query 7: 0.7587\n",
      "Accuracy after query 8: 0.7455\n",
      "Accuracy after query 9: 0.7708\n",
      "Accuracy after query 10: 0.7817\n",
      "Accuracy after query 11: 0.7489\n",
      "Accuracy after query 12: 0.7904\n",
      "Accuracy after query 13: 0.8282\n",
      "Accuracy after query 14: 0.8180\n",
      "Accuracy after query 15: 0.8261\n",
      "Accuracy after query 16: 0.8081\n",
      "Accuracy after query 17: 0.8337\n",
      "Accuracy after query 18: 0.8178\n",
      "Accuracy after query 19: 0.8248\n",
      "Accuracy after query 20: 0.8833\n",
      "Accuracy after query 21: 0.8728\n",
      "Accuracy after query 22: 0.8513\n",
      "Accuracy after query 23: 0.8529\n",
      "Accuracy after query 24: 0.8573\n",
      "Accuracy after query 25: 0.8674\n",
      "Accuracy after query 26: 0.8378\n",
      "Accuracy after query 27: 0.8805\n",
      "Accuracy after query 28: 0.8695\n",
      "Accuracy after query 29: 0.8752\n",
      "Accuracy after query 30: 0.9010\n",
      "Accuracy after query 31: 0.8991\n",
      "Accuracy after query 32: 0.9052\n",
      "Accuracy after query 33: 0.9430\n",
      "Accuracy after query 34: 0.9481\n",
      "Accuracy after query 35: 0.9513\n",
      "Accuracy after query 36: 0.9505\n",
      "Accuracy after query 37: 0.9453\n",
      "Accuracy after query 38: 0.9534\n",
      "Accuracy after query 39: 0.9602\n",
      "Accuracy after query 40: 0.9577\n",
      "Accuracy after query 41: 0.9621\n",
      "Accuracy after query 42: 0.9618\n",
      "Accuracy after query 43: 0.9587\n",
      "Accuracy after query 44: 0.9598\n",
      "Accuracy after query 45: 0.9641\n",
      "Accuracy after query 46: 0.9661\n",
      "Accuracy after query 47: 0.9632\n",
      "Accuracy after query 48: 0.9589\n",
      "Accuracy after query 49: 0.9677\n",
      "Accuracy after query 50: 0.9611\n",
      "Accuracy after query 51: 0.9640\n",
      "Accuracy after query 52: 0.9673\n",
      "Accuracy after query 53: 0.9672\n",
      "Accuracy after query 54: 0.9699\n",
      "Accuracy after query 55: 0.9730\n",
      "Accuracy after query 56: 0.9713\n",
      "Accuracy after query 57: 0.9681\n",
      "Accuracy after query 58: 0.9708\n",
      "Accuracy after query 59: 0.9720\n",
      "Accuracy after query 60: 0.9737\n",
      "Accuracy after query 61: 0.9708\n",
      "Accuracy after query 62: 0.9732\n",
      "Accuracy after query 63: 0.9731\n",
      "Accuracy after query 64: 0.9717\n",
      "Accuracy after query 65: 0.9711\n",
      "Accuracy after query 66: 0.9726\n",
      "Accuracy after query 67: 0.9775\n",
      "Accuracy after query 68: 0.9752\n",
      "Accuracy after query 69: 0.9741\n",
      "Accuracy after query 70: 0.9776\n",
      "Accuracy after query 71: 0.9738\n",
      "Accuracy after query 72: 0.9761\n",
      "Accuracy after query 73: 0.9750\n",
      "Accuracy after query 74: 0.9755\n",
      "Accuracy after query 75: 0.9755\n",
      "Accuracy after query 76: 0.9730\n",
      "Accuracy after query 77: 0.9774\n",
      "Accuracy after query 78: 0.9771\n",
      "Accuracy after query 79: 0.9777\n",
      "Accuracy after query 80: 0.9774\n",
      "Accuracy after query 81: 0.9773\n",
      "Accuracy after query 82: 0.9782\n",
      "Accuracy after query 83: 0.9767\n",
      "Accuracy after query 84: 0.9754\n",
      "Accuracy after query 85: 0.9747\n",
      "Accuracy after query 86: 0.9778\n",
      "Accuracy after query 87: 0.9765\n",
      "Accuracy after query 88: 0.9770\n",
      "Accuracy after query 89: 0.9791\n",
      "Accuracy after query 90: 0.9770\n",
      "Accuracy after query 91: 0.9777\n",
      "Accuracy after query 92: 0.9787\n",
      "Accuracy after query 93: 0.9791\n",
      "Accuracy after query 94: 0.9809\n",
      "Accuracy after query 95: 0.9806\n",
      "Accuracy after query 96: 0.9803\n",
      "Accuracy after query 97: 0.9822\n",
      "Accuracy after query 98: 0.9791\n",
      "Accuracy after query 99: 0.9809\n",
      "Accuracy after query 100: 0.9817\n"
     ]
    }
   ],
   "source": [
    "estimator = KerasClassifier(create_keras_model)\n",
    "bald_perf_hist = active_learning_procedure(bald,\n",
    "                                           X_test,\n",
    "                                           y_test,\n",
    "                                           X_pool,\n",
    "                                           y_pool,\n",
    "                                           X_initial,\n",
    "                                           y_initial,\n",
    "                                           estimator,)\n",
    "np.save(\"keras_bald.npy\", bald_perf_hist)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "estimator = KerasClassifier(create_keras_model)\n",
    "var_ratio_perf_hist = active_learning_procedure(var_ratio,\n",
    "                                               X_test,\n",
    "                                               y_test,\n",
    "                                               X_pool,\n",
    "                                               y_pool,\n",
    "                                               X_initial,\n",
    "                                               y_initial,\n",
    "                                               estimator,)\n",
    "np.save(\"keras_bald.npy\", bald_perf_hist)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy after query 1: 0.6060\n",
      "Accuracy after query 2: 0.6537\n",
      "Accuracy after query 3: 0.7004\n",
      "Accuracy after query 4: 0.7308\n",
      "Accuracy after query 5: 0.7513\n",
      "Accuracy after query 6: 0.7800\n",
      "Accuracy after query 7: 0.7850\n",
      "Accuracy after query 8: 0.7974\n",
      "Accuracy after query 9: 0.8412\n",
      "Accuracy after query 10: 0.8440\n",
      "Accuracy after query 11: 0.8126\n",
      "Accuracy after query 12: 0.8447\n",
      "Accuracy after query 13: 0.8616\n",
      "Accuracy after query 14: 0.8714\n",
      "Accuracy after query 15: 0.8841\n",
      "Accuracy after query 16: 0.8793\n",
      "Accuracy after query 17: 0.8703\n",
      "Accuracy after query 18: 0.8725\n",
      "Accuracy after query 19: 0.8834\n",
      "Accuracy after query 20: 0.8881\n",
      "Accuracy after query 21: 0.8805\n",
      "Accuracy after query 22: 0.8885\n",
      "Accuracy after query 23: 0.9013\n",
      "Accuracy after query 24: 0.9102\n",
      "Accuracy after query 25: 0.9053\n",
      "Accuracy after query 26: 0.8989\n",
      "Accuracy after query 27: 0.9087\n",
      "Accuracy after query 28: 0.9083\n",
      "Accuracy after query 29: 0.9118\n",
      "Accuracy after query 30: 0.9123\n",
      "Accuracy after query 31: 0.9150\n",
      "Accuracy after query 32: 0.9092\n",
      "Accuracy after query 33: 0.9107\n",
      "Accuracy after query 34: 0.9214\n",
      "Accuracy after query 35: 0.9195\n",
      "Accuracy after query 36: 0.9286\n",
      "Accuracy after query 37: 0.9160\n",
      "Accuracy after query 38: 0.9236\n",
      "Accuracy after query 39: 0.9253\n",
      "Accuracy after query 40: 0.9190\n",
      "Accuracy after query 41: 0.9275\n",
      "Accuracy after query 42: 0.9273\n",
      "Accuracy after query 43: 0.9286\n",
      "Accuracy after query 44: 0.9345\n",
      "Accuracy after query 45: 0.9339\n",
      "Accuracy after query 46: 0.9329\n",
      "Accuracy after query 47: 0.9305\n",
      "Accuracy after query 48: 0.9408\n",
      "Accuracy after query 49: 0.9365\n",
      "Accuracy after query 50: 0.9385\n",
      "Accuracy after query 51: 0.9248\n",
      "Accuracy after query 52: 0.9438\n",
      "Accuracy after query 53: 0.9446\n",
      "Accuracy after query 54: 0.9468\n",
      "Accuracy after query 55: 0.9420\n",
      "Accuracy after query 56: 0.9456\n",
      "Accuracy after query 57: 0.9420\n",
      "Accuracy after query 58: 0.9413\n",
      "Accuracy after query 59: 0.9416\n",
      "Accuracy after query 60: 0.9406\n",
      "Accuracy after query 61: 0.9454\n",
      "Accuracy after query 62: 0.9414\n",
      "Accuracy after query 63: 0.9485\n",
      "Accuracy after query 64: 0.9516\n",
      "Accuracy after query 65: 0.9482\n",
      "Accuracy after query 66: 0.9413\n",
      "Accuracy after query 67: 0.9450\n",
      "Accuracy after query 68: 0.9475\n",
      "Accuracy after query 69: 0.9488\n",
      "Accuracy after query 70: 0.9434\n",
      "Accuracy after query 71: 0.9528\n",
      "Accuracy after query 72: 0.9516\n",
      "Accuracy after query 73: 0.9496\n",
      "Accuracy after query 74: 0.9501\n",
      "Accuracy after query 75: 0.9502\n",
      "Accuracy after query 76: 0.9493\n",
      "Accuracy after query 77: 0.9522\n",
      "Accuracy after query 78: 0.9521\n",
      "Accuracy after query 79: 0.9513\n",
      "Accuracy after query 80: 0.9552\n",
      "Accuracy after query 81: 0.9561\n",
      "Accuracy after query 82: 0.9540\n",
      "Accuracy after query 83: 0.9545\n",
      "Accuracy after query 84: 0.9548\n",
      "Accuracy after query 85: 0.9496\n",
      "Accuracy after query 86: 0.9565\n",
      "Accuracy after query 87: 0.9544\n",
      "Accuracy after query 88: 0.9556\n",
      "Accuracy after query 89: 0.9572\n",
      "Accuracy after query 90: 0.9510\n",
      "Accuracy after query 91: 0.9585\n",
      "Accuracy after query 92: 0.9565\n",
      "Accuracy after query 93: 0.9549\n",
      "Accuracy after query 94: 0.9527\n",
      "Accuracy after query 95: 0.9579\n",
      "Accuracy after query 96: 0.9630\n",
      "Accuracy after query 97: 0.9637\n",
      "Accuracy after query 98: 0.9605\n",
      "Accuracy after query 99: 0.9571\n",
      "Accuracy after query 100: 0.9584\n"
     ]
    }
   ],
   "source": [
    "estimator = KerasClassifier(create_keras_model)\n",
    "uniform_perf_hist = active_learning_procedure(uniform,\n",
    "                                              X_test,\n",
    "                                              y_test,\n",
    "                                              X_pool,\n",
    "                                              y_pool,\n",
    "                                              X_initial,\n",
    "                                              y_initial,\n",
    "                                              estimator,)\n",
    "np.save(\"keras_uniform.npy\", uniform_perf_hist)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x7f3ff4218e80>"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEBCAYAAACXArmGAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzs3Xl8VPW9+P/XOWdmMjPZ90x2EiAJISwRkE1ABERAcV+w1WqL12prf71tv9reXrW3y+/axX5vbb22dtNWq9K6sqsIIrvsECAEsu/LTJbJrOec7x8DE2ICCSFAIJ/n49HHI5zlM+czY+c9n+39kXRd1xEEQRAEQL7cDyAIgiAMHSIoCIIgCEEiKAiCIAhBIigIgiAIQSIoCIIgCEEiKAiCIAhBIigIgiAIQSIoCIIgCEF9BoXnnnuOuXPnkpOTQ3Fxca/XqKrKj370I+bNm8f8+fNZsWJFv84JgiAIQ4uhrwtuuOEGHnjgAe6///6zXvPBBx9QUVHB+vXrcTgc3HrrrUybNo3U1NRznhMEQRCGlj5bCpMmTcJms53zmtWrV3PXXXchyzIxMTHMmzePtWvX9nlOEARBGFoGZUyhtraW5OTk4L9tNht1dXV9nhMEQRCGFjHQLAiCIAT1OabQHzabjZqaGsaNGwd0bx2c69z5sNudaNr5J3SNjQ2jubnjvO+7kok6Dw+izsPDQOssyxLR0aHnfd+gBIWFCxeyYsUKFixYgMPh4KOPPuK1117r89z50DR9QEHh9L3Djajz8CDqPDxcyjr3GRR+8pOfsH79epqamnjooYeIiopi1apVLF++nCeeeIKCggKWLl3K/v37WbBgAQCPP/44aWlpAOc8JwiCIAwt0pWyyU5zc8eAomV8fDiNje0X4YmGLlHn4UHUeXgYaJ1lWSI2Nuz87zvvOwRBEISrlggKgiAIQpAICoIgCEKQCAqCIAhCkAgKgiAIQpAICoIgCELQoCxeEwRBGM7cXj/VTU46On043T68fo24SDMJ0VbiIszIstSvchrsnWw+UIvbqwJgNMgsuynvYj56DyIoCIJw1dB0Hb9fw6DIyLKEput4vCoen4o1xIDJqJxXeR6fSlVjB00ON35VO/U/HVXV8Gs6rR1ejlc5qKjvQDtjyZeMhoYESEgSmE0KIUaFUIuRa1M0pnAAa/MxTGPnwZgF1Da7WLezgh1H6jFLfjLNrSTLzUQoHTSPjyIyJm6Q36mzE0FBEIQrnqbpbDlUy7ubS7G3ewCQJanbF7XRIJOfGcPEUXFEhJo4WdPGydo2ZEli2thECkfFYzTIlNW1s+1wHYdLW6hr6eRcy3sNikyWLZybpqaTZQsjzllCePlmlIaj6LIBv2LBI5nxyGbcmNH9HlIrKvDqCpV6NKk73qJo8xb+7pyBLaST76WXkew8gqQFWgqSJZLkKIm2i/rufaFOl/C1BEEYYlwePy6Pn5gI8+V+lH7TdJ3Smjbs7R78mobH5ebjvbVUNbnISo5gbmEKqqbjVzVkScJsMhBilKlp7mTf8Ub2lTQBIEmQGh9Gp9vPH94vwhpiIMxqpN3RyhzLUW6L8OMonEFMxkgSY6yYDDIGRUaRJRTVg9J8AuyV4KpC73Sg7i5Hb29ECo3BOH4RACZPBxa3E93jRPd0oKt+yLyZ48ZxHKzykufdT4H9Y35ifgdZ84HbjDFvDobUAuS4DCRrFCEJEXAJV3GLoCAIw1Cn289HuytZv7MSv6bxf+4rJMPUgr/qMKa8OUghvWfXVBtO4juxg5DCW3q9xun2sbOonv0nmnF7VVRVQ5IlZo9PZlp+UqBLR9PZtK+adTsriY00k5cewQSO4unspNEl0+iEtHCVrAgPRlcLSDKEhNKhmjjUoVFS3U67RyNBbiPD0IRNcTAaC96CQlIm34ASn4Ek9d6Hf9+cTJo3vwXtDYRmF2LOzgVzOMUn69l7oBhb237Gx+3DoLpBNUHpIQxMwmCcht7eiGavRm2qQGup6GqFmCzI1ijkKBvGa+/CkFmIJJ/7q3UqMLUQIB+1aQbe/WtQErMxjp6JZLKcxyc5+ETuo6uQqPPwMNA6f7q/hrc2lNDp8TNhZBxVjR1YfXb+PWI1ktcJIaGEFN6CccxcJMUYvM9Xsh33pj+C6keOScOy6DvI1iicbh8lhw7TcWwHuxvMHPMmEBkdTVSYCaOk4ne20tDiJCnGyvRxqaw73E5FfQfZKRF4vSqzOtcxJeRkj+f06AY6DVGg6yj+TiySB6OkBc/rRgtqdCbEZhDS2YBaeQA0P1JEAoaMiRgyC1ESRyLJgXEEzVGL6+MX0ZorkaxR6J0OQEIyh6G7u95HQ2YhpsKlyGGxeA+tx3twPfjcAEiWCOSYNJTEkSi2HJSELCTjxW1lXercRyIoXIVEnYeHM+t85oBquNWIIvecba7rOu99Vsr7W8rITY/inrmjyEgKp76uEee7PyFUcmOe/VWMJzahVh8GSySGtAIMKWPQ7DV4961ESRqNL3sWbHsVt2xljTKX1LYDTDaVcHqCjY6EHGUDjxPd1drjOY5p6ShT7mX8hFy8W/+Or2gDLZkL0HNvwBYGBs1Ng8vIxqMdbCuqJ8xiZHRaJKNSo5g1KRV/pxtUPxhDkKSueuoeJ76Tu/CX7UatPgKaH2QDcpQNOSoJf8UBJMWI+frlKGnj0Foq8ZftQe90IIXFIofFIsdloESndH/fPE40ew1SVBKyOXwQP8H+EUHhLERQ6D9R5yuD2lSOFBqNbIno9z265sdXtpfiTWsp7TCzzj0ej25E1XRAJ9dYQ15EJ3OuSSfEHIIcHo+SnIcmyfx9fTGb9tUws8DGgzfloMgyuurHtfoX+OtLeLFjAZW6DVXTGKFXMcNczGhTPRYCA7fHzWN5o2MKTW1+0pQm/i18A+GyG01ScGXOIm7aLdDegFpdhNpUjmwJRwqLRQqNRpINqKpGU00VkeUbQVdRknJQqw9jGr8I05S7ztrlc6b+fs6614W/6iBqQymavRrNUYMcZcM862Hk0Oh+v99DgQgKZyGCQv+JOg89tc1ONu2rYfexBu6bk0Fewxr8xVtAVjCMmIQxbw6KLbfXL0ZdU1HrS1Ar9uM7vhW900GbZiZcdtNpjOFg0q1gMJHT+CHRHSd63m+yUqxnsrUljpyxOcy/vhDcHfhKtuE/vhWttQ7z3H/jhDGXzw7WEmYxEhlmQtN0TlTa6agtRfK5sIdmkpUcSZYtgqzkSNIsTji5DWPuLOTw+H6/F5rTjmfHW/hLtmEccwMhM77Ur4AAQ/9zvhhEUDgLERT6T9R56GhzevnDB4cpKrOjyBLZllbuNn5CvNyGacIidL8XX/EW8HYip43HOveR4ACu1tmKZ+c/8ZftBm8nSDJSSj5/L02i0TqS//+uRBrefwHdaQd0MFoIueZWDkk5/GVVEWMzI5iT7qd+32Zy5XIskq/H8ym2HIz5N2DMmnLWOmi6jtujYjUP7rwUrdOBZInsd0CAofs5X0yXOiiI2UeC0If2Ti9Hyu0Ujo7HoPQ/M4zL4+fXK/ZT2+TkzlkZzFAOIB1cSZvfxNakZdw4ZT4Ax+OuZ/+at1lcsYfO1/+D0AWP42qoQNq9Akn1Um7NZ9ScmVgzC3hvRx0728v4/tIcQkekEXrnj/Hs/CfICiGFS5HMYRQCbT4Dr647xo5SSI2fT/bi0cQa2tDa6tFa65FkA4asycjhfS+KkiVp0AMCgGyNGvQyhQsngoIgnIWu6+woquf1j47T4fKREh/KVxZkke4txTDimuCslt54Wup4bfUhXI1Ovj0nidTyv6K1VGLImswW73TWH7STO6Udj0/lt+8XY4uZwgembGa3r8S86qeYgBO+BHZGLmBntULiJp1lJjdrd1QwOTeBUamBL1TJZMU884Eerz9nYgq6rmPv8HLz9AyMBgWIRonLuEjvlnC1EEFBEE7R3R1ozhaU2HSqm5y8taGEgyebyUqO4M452bz3WSl7336VBMtBDJPuxFK4pGcZmh/XtjdQD3/EPQCRwF7QrVGYFzyBMbOQJW4fW0u288dVRbS0uYmNMPPv904gwmqirHwCJze/gTcihezZS5gQaWVWuZ3/fe8Qv3pjHwZF5q452f2qz/WFqYP6/gjDgwgKwrCi6/qplbA6CVGBRUKa04734Dp8RZ+A38NJ42hebpiIarBy3w2juOGaVGRZYlJWOO43XkXVJPyfv4OWOpHQhK7pi1qng7a1v0VpKuFTdw4JOeOZkBX4RW/ImIBksgIQajZy9/Uj+dOqI8RGhPDdUwEBIDPDBhnf7vbMuRnRPPOVybyy9hj5I2KIi7q8i5uEq5sICsKwcKzCzvtbyqhs6KDDFRhwzR8Rwx22CmKL30fXVIoYSZXLzDz9EM/G1xIy62HCR6YFy1BObMKoeygf8wAJRf+g6t0XSb77h8REmmk/uR/fp38CbycrvLOZuGAxk3ITzvo808cmoWo6eRnR/UoxERNh5tt3j7/wN0K45Nq87dR01FHrrMfpc3JdyjQiQ/o/DflSE0FBuKpojjr8dceQTFYkcxhyVDJu2cpL7x1GliUmjoojPTEcl8dP2771xLZu5bAvhX85pxCekMyca1OwJLrxb/4j2obf4NO+inH0THS/B9+BdSipYxl73Vwq6CTzyD9552+vkiC3MsNYhF2NYGPYPdxx36w+v+glSWLW+ORL9K4Il8vWml28fvSf6HTNnPysZgcP59/P6Oj+dQNeaiIoCFckXdd7TGVUm8rpXPlcYPrmaQYTe6Lm09YZy38+OInMpMAvNO/RTXgMW2mNyqUi9lYeG5PMCFvXrzfTbc/iWvc/uDf+CWQF3e1Ed7djmngzAGkzF+Fo2MttzdsAqI2fim/sUh7OTup37nzh6lbiKOWNY28zOjqbBRnXYwtNwulz8sdDf+M3e//AkqwbWZAxB1kaWnudiXUKV6Grvc66ruFa8zx6Rwsh05dhSB1LpNRK9Ss/BMWI5cZvgSShu9pp3fEOpuYSKsPGknvzg6gNpahVh/Ad24ySNhbLgie65ffp9jp+D641v0atKwaTBSUqGcstPwgGI81Rh3vb65gKbsSQmn8p3wLg6v+ce3Ol1LnZ1cLPP38Bq9HC9675JlZj1ziQ2+/m9aP/YnfDfsbH5fPAmHswG87eshSL185CBIX+u9rr7Du2GfemP6GZQpG9TgyZhehNZWiqH8NNT+EKiSUyzISu6/zkrzuZ4N7JXONegonxjWYMI67BPPNBJIPpnK+l+9y41jyPWleMZeH/hyF9wiWoYf9c7Z9zby5Hnfc1HmJbzU5mpU5nTExOn4vt3H4Pz+95kRa3ne9d8w0SQ3uOLem6zsaqLbxdspJEazyPFDyIIskcaCqivK2Spdk3EW0OTFIQi9cE4RycjhZcm1+n2hfPiy3zuSW2hBnle8FgYlXonXz612J8fg2TQSYi1ERTq5tFt96LNeom1PoSlKRRyPEjzrnG4EyS0Yzlpn9HbTiJknxpt0UUulM1lYH+hm3obMKn+UgJs/Va7uHmo2yv241BUvhK/n3BLh23380bR9+mw+fkUPNR0sKSmZ8xh/HxYzH0kh5b13X+duQtajrqeGz8w70GBAiMKV2fNhNbaCJ/PvQaP93xK/x6YGOd5NCkbpsDXWoiKAhXjAMnmmhZ/wcmyG4qRtzP/SlZbNoXzfq6DCR0JKuRWeMSSIq10uhw0WB3MWFkHJNy4pGkBJSkUQN6XcloxpAyZpBrI/RF13X2NR7iSMsxytoqqemoIzM6lQWpcymIG9PjF7tX9dHp7yTCFN6tn97td/N/97xEq7eNsbF5LBoxD1toIsX2ExS1HGNPwwHavR1YDRY6/S7SwlOYnzEHgA/LN9Lu6+DfCx+jobOR9RWf8OfDrxNqtDIlsZBpyZO7BZp15RvY13iQ20YuZkxsTp91zI0Zxf+Z/ATryzeQYI1nXFw+CdZLt/Vmb/oVFEpLS3nqqadwOBxERUXx3HPPkZmZ2e2axsZGnn76aaqqqvD7/Tz66KMsXboUgBdeeIHXX3+dhIRA1CwsLOSZZ54Z3JoIVzVN19mwZiNfNRXjGjmPm+fOBGDW+GQq6tuxhIYQazWKQd6rhK7rvHNiFR9XfIrVYCEjIo3c9FEcai7i9wdfIS08hSRrIi1uO3aPg3ZvBz4tMNV4VFQW35jwteAv+VWlH9Lmbef61JnsrNvDzz9/AYOk4NdVjLKRMTGjmWqbRH5sLn86/BorT64jPzYXi8HMx5WfMilxAtlRmWRHZXKt7RqOtBxne+0uNldv45Oqz5gQP5YlWTfS5Gpm5cn1TE4s5Ia0Wf2ua5wlhmW5d16U93Eg+jWm8MADD3DHHXewdOlS3nvvPf71r3/x6quvdrvmO9/5DllZWTz++OO0tLRw++23849//AObzcYLL7xAZ2cnTz755IAfVIwp9N9QrrNf1c4rf9BppTUOPO/+iPhQnZhlzyEZQ7qdH8p1vlguR53rnQ18WLGJrMgMxsTmEBUSOeCyqjtq8ageFEnBKBuJs8RgUkyomso/jr3NttpdzE6dzp2jbgn+8o+OtbL60Kd8VLEJr+olOiSKGHMUESHhhBqsuFUP68s/YWbKVO7LuZ3qjlr+e9f/MN02mfty78Dld/NZ9XbavO3kxYxmVFQWxjMmGrR7O/jJjl8RY44iwRrPvsZDPH3t94i19Ey33eFzsqlqKxsqPsWjejHICrbQRL5d+Bims0xeGIghN6bQ3NxMUVERf/nLXwBYsmQJP/7xj2lpaSEmJiZ43dGjR3nwwQcBiImJITc3lzVr1vDwww+f90MJV6eishZ+984hbp6eycJr04PHNdXH5++9hUnyk5U/lrDU7B7ZM+v2fMpYgx0mfa1HQBB68qo+3j+xBq/m485RN2NSzj2g3l8fnFzH3saDbKvdBUBaeArXp85kUuIElH6O03T6Onmz+F0+r9/X7biERKI1HpNioqK9ipsy57F4xPxu/x0YZIVptklMs006a/m6rvNhxUZSw5LZWbcbq8HCLdk3AWAxmINdQ70JN4VxT85t/OnQ36lor2ZBxvW9BgSAMGMoi0fMZ3bKdNZXfEKJo5TlY788qAHhcugzKNTW1pKYmIiiBD5wRVFISEigtra2W1DIz89n9erVFBQUUFVVxd69e0lN7cq9smrVKj777DPi4+P55je/ycSJEy9CdYShavexRn7//iFUVWfVtjJmT0jGEmJA62ihedVvyGstC1y48ROcgJY1g4gbvoYkSeiaSkrNxzRJsWSOmX4Za3FlqHXW8+dDr1HjrENCorytkkcKHjzrl1t/Nbla2Nd4iPnpc5icNJHDzUfZVbeXV4+8yarS9cxMnopP92N3O3B4WvFrfvyaCujYQpPIjEzDrITwdskq2rzt3JQ5j6zIDFRdxaN6qXM2UNVRTUNnM3eNXsqc1BkDes5bshdS3VHLG8feBuBLeXcTarT2+/7ChHEcTprEMXsJCzKu7/P6MFMot4/smQfrStVn99GhQ4d48sknWbVqVfDYokWL+MUvfkF+ftfc7JaWFn72s59RXFxMcnIyISEh2Gw2nnrqKRobG4mKisJoNLJlyxa++93vsnr1aqKjr6wdkITzp+s6H+6s4Hcr9jEqPZr7b8zl6T9s44FFeSwe4aLh3f+L2+ViFbO5/Uu3sW3TTrQT25luOkbMDQ8SNfUWaretxbXhZUpGf4kFd912uas0pG2t2M2LO1/BbAjhG9c+hKZr/Gb7n1Fkhe9Mf4QxCecebPerfg43FrOrej9Obydfn/xlTKem7f51z1usK9nE75b8lJhTaa91XWdP7SHeLlrD8eZSAKLNkcRYojAZTBhlA6quUuaownlqUWFKeBLfmPoVsmMuXsbWDq+T//zol0RbIvnhnCfOe4GYruv4NP8V/6t/IPpsKdhsNurr61FVFUVRUFWVhoYGbLbuU7tiYmL45S9/Gfz38uXLyc4OLOOOj+/alWnGjBnYbDaOHz/OlCln39jji8SYQv8NlTqX17WzYmMJRWV28kfE8I3bCggxKRRkxbJn02au2bkOjzmOXzquZ8nCqYRZQ5l/0/V8/Hk2+7f/kXEfv4rLGI3jsxXU+WOJH3PtWes1VOr8RaqmcrK1jMzIDIy9TGG8EF+ss93t4MUdr5ASlszygi8TaQis0P5u4eP8/uArPLf5RZ6e+j0iTD33GXb73awp+5jPqnfgVt2YZCM+zY/qhS/l3YXL7+bjk1u4JnECqlOh0dn1uunGTL417lHavO1YjdZe66nrOo2uJho6mxgdPRKTahzQ53U+n/P/ueYJJCSam5zn/Tpd3Bdw7+AYcmMKsbGx5OXlsXLlSpYuXcrKlSvJy8vr1nUEYLfbCQ8Px2AwsG3bNoqLi/nNb34DQH19PYmJiQAcOXKE6upqRowYcd4PKwxdbq+flVvL6XB5AWhz+thf0oTVbODeG0YxtzAlOMC8dEI4lg0f02mM5mXXYuQoE9PGJgbLmjUhhf/cfj2p0kqkdf9DiK6zWVrEo/Hn/x/45aLpGvsaD7Hy5HrqOxuYGF/Aw2PvD/5i1XSNPfX7GRmdNaDB2obOJoxuHejqb3+nZBU6Ol/Jv69bwrXE0AQeKXiQn+38Ne+WrOaBMfcEz+m6zu76fbxdsopWbxuTEicwKXECOdGjWF/+CWvKPmJEZDpu1YNH9TI37bpen0eSpHMmeZMkiQRrPAnW/m/beaF6W0cg9K1f79qzzz7LU089xYsvvkhERATPPfccEGgNPPHEExQUFHDgwAF++tOfIssy0dHRvPTSS1gsgaXdzz//PIcPH0aWZYxGIz//+c+7tR6EK5vPr/HCvw5ytMJOZGigq0GRJRZOTWfx1Ays5q4muO51kXjgz7hkeL5xJnV+P48uDWwif5rRoDB/2ih+9/Es/iNuLaXuSCKyx5/Xto2XQ31nI2WtFVR11HDMXkJ1Ry1J1gRmJl/LZzU7eLdkNbePWoJX9fJK0ZvsazxIpCmcR8c/RHp4//c+qGiv4td7XsJqNPP4uK+RHJZEsf0Euxv2syhzHnGWmB73JIUmMC99NuvKNzDNNolR0dl4VR+vFP2DfY2HSA9PYXnBA4yI7JoAsGjEPMraKlhR/B5mg5nRUdmkhaf0KFu4uog0F1ehS1lnVdP433cPs6e4ka8uzmNGQc8Vo2dyffhb/GW7aSp8hB9/6CItIYxnHpqM/IUvfJ9f5cmXtmFSO3G4dL5x9yTGZsWetdzL/TmXt1Xy889fAMAoG0gJS2ZWyjQmJ01EQmLF8ffYVLWVJSMWcLD5CBVtVczPmMOuur04fU4eyl/GuPi+8yc1u+z8cvdvUSQFJB2v6uOx8Q/z2pF/4lY9/Oe13z1rP7hX9fLjHb/CpJh4YsJyXj74N8raKliafRM3pM/qtd+9w+fkuV2/ocVt59FxX6Eg7vIu4rvcn/PlMOS6jwThbHRd569rjrKnuJH75o3qMyDo3k78pZ9jGr+IzMKp3OWrIDcjukdAgEBrYfG0TF77sJgQo0JO+tDez/e44yQA373mG6SHp/SYnnnnqFuwu1tZWboek2xkecEDjI/PZ07qTF468Bf+cPBVRkVnYwtNJMmagCxJdPpcOP2dRIdEkR01gmhzJC/u/xM+zccT1zxCQlwkz378a361+0U0XetzOqRJMXHXqFv4/cFX+NH2n6PqGl8d+yUmJhSc9Z4wYyiPjX+YQ01HyI/NHZw3SxjSRFAQBmzzgVq2HKzjlhmZzJ+U1uf1WkcLAHJcBpIkcdPUc88+mTXextod5WSnRJ7aY/jSOdR0hH+VfMBD+cv61bVT2lpOnDmmW/fLmWRJ5qH8+1hT9jETEwqCZUaGhPPtwkf54OQ6TrSWsb12Fx7VG7zv9MpbCMzjlyWZb0z4GrbQROLDwvn3wq/z2/1/It4Sy/j4sX0+57j4fCbEj+W4/SSPj/8K2VGZfd5jC03EFprY53XC1UEEBWFAWp1e3tpQwui0KJbO7N+kAd15KiiE9uzz7o3RoPD0VyZjNFz6fPPryjfQ0NnEb/f+kScmPkJqeGBDHK/qpdZZT0ZEVxDUdZ3S1nJG9bFpikkxsfTUIqovHr9j1M3BshyeViRJwmqwYlKMNLvsnGgt5WRrOXkxo7ttzhJtjuI/pgS27+zvmMvD+ffj11VCBmlBm3B1EUFBGJA3NxzH41N5cGHfqYRPO91SkML6FxQAwq2X/ouruqOWk63lzEmdwb7GQ7yw72UeKXiQ444TfFL5GR0+J9+55nGyIgMtHbvHQau3nRGRFz7vXpKkYMrk02It0cRaopmSVNjrPec7B1+RFRQubctLuHIMrS1/hCGpor6d7/9hOys2ltDq9HK4tIXth+tZPC0DW2xov8vRnXaQJCTrwPPlXAqfVW/HIBu4acQ8vjXxERRJ5vk9L/LByXWkR6SiSAr7Gw8Fry9tLQdgRETvXUeCcCURLQWhT2t2VNDc6mLt9go++ryKEKNCYoyVxdPO75ex1tESyGl0GeaPN7taKHGUEmuJId4SR4QprNcWjtvvYWfdHq5JGE+YMZQwYyjfmvhvbK7ZzrVJk0gLT+a3+/7IgabD3DZyMQClbRUYZSOpYWLPZeHKJ4KCcE72dg+fH21gbmEq1xemsGZ7OXuKG/nKwpzzHvzVnS3n1XU0WLyqlxf3/5m6zobgMavBQlZkJiOjRpAbM5q0U2MGn9fvxa16mJkyNXhtYmgCd466JfjvgrgxvFX8LvXOBhJDEyhtrSA9PLXfCeEEYSgTQUEIKq1tY+XWMr58Yw5RYYFMpJ/srULTdG6YlEpClIWHFuXx0KKB7UCmO+3I0Zf+1/TbJauo62zgofxlWAwWGjubqO6ooaS1lEPNR+DEagoTxnFr9iI+q95OSpjtnF1B404FhQNNRcyxxFDVXs2ctJmXsEaCcPGIoCAAgVXJf1xZRG1zJx0uH9+7b2JgH9m9NYwfGUdClKXvQs5B13U0ZwvKJd7g/mBTEZurt3FD2iwmJZ7aX/mMHbHavO1srt7Oh+Ub2d94GFVXuWf0beccPI82R5EWlsyBpiKyo0bg19VBGWQWhKFADDQLAKzZUU5tcydzJqZwvKqVf248wfaiejpcPuZN6n+ZRIU7AAAgAElEQVQKhrPyucDnRj5L95FH9aJq6oW/zhlaPW38/cgKUsJs3Jy9sNdrIkzhLB4xn2emfo/ChPEkhSYyOanvtO4F8fmUtpZzsKkIEIPMwtVDtBQE6lo6Wbm1nCl5CTxwYw6KLLF+VyXhViMp8aHkZVx4ivPgdNRe1ih4VS//tf0XTE6cyK0jF5132c2uFk5WleDp0AgxmGjsbOZA02GKWorRdY2H8pf1maE02hzFV/Lv7fdrjosbw+rSD9lYtYUYc/Q5k8EJwpVEBIVhTtd1Xl17FKNB5r4bArn275k7krK6Nk5Ut3H7rKxBSUR3roVr22o/x+FpZWfdbm7JXnhe8+41XesxiAwQaYpgSlIh022TL8pq3NSwZKJDorB7HIyIHdgYiyAMRSIoDFMer8qe441sPVjL0QoHD9yYQ+SpwWWDIvON2wrYXlTP9LHnzmfUX2dbuKZqKh9XfIpJMdHqbedkazkjo7qvkNZ1ndK2CrbV7CI9IoXrUqYFz+1tOBAYRJ54NzFyHB6/l1CjldTw5PNe1HU+JEmiIG4Mn1ZvFeMJwlVFBIVhaPexBv648ggen0psRAi3XjeCWRO6zwqKDAvhximD109+toVrexsP0uxu4cEx9/L60X+yp2F/t6Cwr+Egq8s+orqjFgmJ7XWfkx6eSkZEGpqusabsY5JCE7lx1OwL3Ezl/E1KnMCWmh3kRI+8pK8rCBeTGGgeZhwdHv665ihJsVaeur+Q574+nVtmjOg1U+lg6m3hmq7rfFS+kURrPJMSJ5Afm8u+hoNougYE9gT+y+HXUTWV+3Ju58fTv0+EKZy/HXkLn+ZnX+Mhap313JQx96K2Cs4mOyqTX836L5LDki75awvCxSJaCsNIYPzgGF6/xr/dkk9STP83M7/g13a2oIZF8/zuF4k2R3FD2iw6/S4qO2q4P/dOZElmYsI49jUeCnYhvX9iDdKprKCn8wHdl3M7/3vgL6wu/ZBDTUdItMZTmDj+ktXji4zDcA9f4eomgsIwsv1wPftKmrhn7shLGhAg0H1UFBPJidYyDO0GPq/fh1kxE2EKZ/KpRG9jY/Mwygb2NOzHICvsbtjPwswbuiWIGxuXx9SkSawv/wSAB8fce1laCYJwtRJBYZhwdHh4/aNiRqZE9mvvg8F0euHajgQzMaZovj/5W2yt3cWWmh3ckDYrOF3UbAgJdiFVtdcSbgxjfvrsHuXdMepmjtqPY5KNXJNw+VoJgnA1EkFhmHhzQwlev8ZDi3KR5Uu817HPRbPu5bjWwWLbfKxGK/PSZzOvly/8011Ird527s25DbPB3OMaq9HCk5OfABD5hgRhkIl29zBwvMrBjqJ6Fk5JP69U1+dD13Uq22uCg8Rn0jpa+DzCggRMs00+Zzmnu5ASrQlMt00563URpnAiTOEX+tiCIHyBaClc5TRd5/WPjhMdHsKiPra/HKhmVwv/OPY2R1qKmZt2XXAXsdP8HU3sjjCTF5bWYwOZLzIbQlhe8CDRIZGiFSAIl4EICle5rQfrKK9rZ/mSMYSYBudLVtd12n0d2N0Oiu0nWF32EQCjorLYULmZ3JjR5J+RdO5w8zHaDArTbZP6Vf6Z9wqCcGmJoHAVc3n8/GvTCbKSI7g2f3BSPeyq28ubxe/g8ruDx8bE5nBfzu2EGcP4xecv8LeiN/nBtd8Odu9sbz9JmF+jwNb7dpKCIAwdIihcxd7fUkqr08s37ijotjjthKOMTVVbuDfnNqzG/k1N1XSNVSfXs7Z8A1mRmVyTOJ6YkCjiLLHYQhOD+ZEeyl/Gzz//Da8WvUl+bC676vZS7rdzvUvHYAi5KPUUBGHwiKBwhfp4dxWfH21gXHYsE0bF9RhAPl7lYP3OSuZMSCY7uSu1hK7r/PP4e1S0V9Puc/L4+Icx9JJBtNnVwtslqzAbQogwhVPTUceh5iNMt03hnpxbe70HIDksidtH3sybxe9wpKWYtLBklvhCmaYPbv0FQbg4RFC4Qm0+UENNUyfHKh2s2HiCkSmRPHLLGOIiLbg9fv608gixkWbuur57Xp6j9uNUtFdTEDeGg01FvHHsHe7PvbNHJtT1FRs50HSYCFM4bd52ILA+4PrUmX1mTb0uZSoRpjASQxOwhSbifOsHl2XHNUEQzp8IClcgv6pR3ehkwZQ0rp+Ywp7iJt777CT/9dfP+bel+RyrbKXB4eLJZROxhHT/iNeXfUKkKYKvjv0S68o+Zk3Zx8RbYrkxc27wmk5fJztrd3Nt0jV8Ke8uNF1D1dR+p3SQJIkJCQXA5dtxTRCEgRFB4QpU0+RE1XTSE8KJi7SwYHIa47Nj+e07B3n+zX3oOsyflEZOevfNcUpbKyh2nOC2kYsxygYWj1hAo6uZ90+uJSsyg1HR2UBgfwOv5mN26gwAZElGVga2pEVrKgvsuBYed0F1FgTh0ujX/9NLS0u55557uPHGG7nnnnsoKyvrcU1jYyNf//rXufnmm7npppt47733gudUVeVHP/oR8+bNY/78+axYsWLQKjAcVdR3AJCeGBY8lhhj5YdfnsTUMUnkZkRz++ysHvd9WP4JVoOFmcnXAoFf9Mty7yTOHMPfj/4Tr+pF0zU2VW0lO3IEaeEX1uWj+724P3kZyRqFcdSMCypLEIRLo19B4ZlnnmHZsmWsW7eOZcuW8fTTT/e45r//+78ZO3YsH3zwAa+99hq//vWvqa2tBeCDDz6goqKC9evX8+abb/LCCy9QVVU1uDUZRirq2wkxKiR+IaldiElh+c1j+MUTswgxdl+TUOusZ3/TYWanTu+WOiJEMXF/3p00uZr54OQ6DjUdodndwpy08/8S130edL1rRNmz619ojhrMs7+KZA47x52CIAwVfQaF5uZmioqKWLJkCQBLliyhqKiIlpaWbtcdPXqU6667DoCYmBhyc3NZs2YNAKtXr+auu+5ClmViYmKYN28ea9euHey6DBsV9e2kJYSd1x4Im6u3YZQNzEmd2ePc6OiRzEyZyieVn/HuidVEhUQyPu78xgA0p52OVx+n851n8ZVsw191GN/B9RjHzMWQVnBeZQmCcPn0OaZQW1tLYmIiihL45akoCgkJCdTW1hIT07W1Yn5+PqtXr6agoICqqir27t1LampqsIzk5K6uCJvNRl1d3Xk9aGzswH9pxsdfuTlyNu6uBEliTmHgvdQ0ncpGJ3MnpZ2zXmee03SNA1sPM9E2lhEpvW8I87Wouzmy9hj1nY3cV7CUpMRzp6P4Iqf9KE7Vj+Ruw73h9wAYopNIWfxVZFPPpHYXw5X8OQ+UqPPwcCnrPGgDzU899RQ/+9nPWLp0KcnJyUydOhWDYfDGsZubO9C085/sHh8fTmNj+6A9x6Wk6zp//uAwPr9GTnI4BkWm3t6Jy+MnPiLkrPX6Yp1POMqwu1sZE5l3zvfiyzl3s7ZsAxOiJpz3e+YpPwGA5Y6foNYdx3d8K6bxi2hu9QG+8yprIK7kz3mgRJ2Hh4HWWZalAf2Y7vNb22azUV9fj6qqKIqCqqo0NDRgs3Xf0D0mJoZf/vKXwX8vX76c7OzsYBk1NTWMGzcO6NlyEHrX1OrG3u4B4Ei5nYKs2F4Hmfuyt/EABklhbFzeOa8bFZ0dnIF0vjR7DVJoNFJIKIaMCRgyJgyoHEEQLq8+xxRiY2PJy8tj5cqVAKxcuZK8vLxuXUcAdrsdv98PwLZt2yguLg6OQyxcuJAVK1agaRotLS189NFH3HjjjYNdl6tOcaUDAEWW2FFUDwTGExRZIiWuf0FB0zX2NhwkL3Y0ll72JhgsmqMGOUoEekG40vWrf+fZZ5/lqaee4sUXXyQiIoLnnnsOCLQGnnjiCQoKCjhw4AA//elPkWWZ6OhoXnrpJSwWCwBLly5l//79LFiwAIDHH3+ctLRLu/vXlai40kGo2cCEUXHsPtaI16dSXt+OLTYUo6F/6wbK26pweFq5JWvhRXtOXdfR7DUYc2ddtNcQBOHS6FdQyM7O7nVtwcsvvxz8e/bs2cye3XMnLQgMTv/oRz8a4CMOX8WVDkalRjE1P4ktB+s4cKKZyvoOxo6I6fvmU/Y1HkSRFAr66Dq6ELqzBfwe0VIQhKuA2HltiHJ0eKi3uxidFkVeejQRoSY+/LySVqeXtMT+zUTQdZ29DQfJiRnZ72yoA6HZqwFEfiNBuAqIoDBEnR5PyEmPQpYlJucmcLyqFYCMfg4yV3ZU0+xuYWL8uIv2nACaPbBIUQQFQbjyiaAwRB2vbCXEqARnGV2b17VJTlpC95bC5urtvFuyukcZO2v3IEsy4+LHXNRn1RzVSOZwZPPwmz8uCFcbkRBviDpW6WBkSgSKHIjb2SkRxEaYkWWwmrs+tu21n/PGsbcByIkeSV7saADsbgeba7YzOXEiYcbQni8wiFR7jWglCMJVQgSFIajD5aO6sYPJuSOCxyRJ4qFFufhVLXjscPMxXjv6T3KiR9LkauGdE6vIiQnsn7Cm7GN0XWfxiPkX9Vl1XUdz1GLMmnJRX0cQhEtDBIUh4s1j7xJqtLB4xAJKqlrRgdFp3VNNjMnsmnVU0VbFHw/9jeTQJJYXPEBR81H+fPh1tldto9CbxrbaXVyXMpVYS/9nKg2E7moFj1O0FAThKiGCwhDg8LTyafVWAMwGMy1VKRgUiazkiLPe81bxu1gNFh4b/zAWg5nChPFsqPyMD46vZt9+F4YwCwszb7joz67ZawDEdFRBuEqIgeYhYG/DQQBGRWXxTskqPqvYTZYtAqNB6fX6Wmc9pW0VXJ82k8iQQOCQJInbRi6mDR+HrQZmWdKJMA3OwK+u66inpp1+keY4FRRES0EQrgoiKAwBexsOYLMmEdkwE7U9GjV1L9dNP3tKim01u5AlmWuTrul2fGTUCMb5TYSqGjNragbt+dTaY3Su+A/8VYd6nNPsNWCyIFnPL6uqIAhDkwgKl5nD08rJ1nI66uL4bH8D14XfTJwlmo8bVqPpWo/r/ZqfHXW7GRc3hnBTz/UK97VLfKfCjrmxHLWxbFCeUWsL5F3yHd3U85w9kPNIOo+9HQRBGLpEULjM9jUeQkensSyKO2dnc//csdycfSP1nY0caCrqcf2h5qN0+JxMs03utTzZ00lcdiEoJnxHNw7KM+qdgUVz/rI9aO7uKXw1Rw2K6DoShKuGCAqX2b6Gg4RLMejuMCbnJgAwMb6AWHMM68s/6ba9JcC2mp1EmiLIixnda3m6uwNjVCKG7Cn4Sraje10X/Ix6pwMkGTQVf8n2bq+lu9rEILMgXEVEULiMWj3tlDhK0VuTyEgMJy4qkFVWkRXmpc+mvK2S446TwesdnlYONx9jqm0SitxzEFpXfYHEdJZwTHlzwOfGd2LHBT+n3tmKHGVDjsvEd2xz8Lj34DoA5ISsC34NQRCGBhEULqP9p7qOmsqjKRwd1+3cVNskwo1hfFi+EQjMANpUtRUdnam2Sb2Wp3ucACiWMOSEbOSY1F7HAc6X1ulAskZhzLkOrbkCtakcf3UR3r0rMYy+DoMt54JfQxCEoUEEhctob8MBwuVodFcYhTkJ3c6ZFCNz0mZS1HKMzdXb+NXuF1lf/gljY/NIsMb1Wp7uDgQF2RKOJEkYsq9FaywNBouB0jsdSNZIjCOngmLAu28V7g2/R45KwjzjSxdUtiAIQ4sICpdJfWcjxY4TKG2pJMaEkhzbM7X1rJRpmJUQ3jj2DnaPg/tybueRggfOWqbuCWzVKVsCs5KU+ECaDLW5YsDPqet6oPvIGhXYajPzGvwnd6J7nZhveAzJGDLgsgVBGHrEiubLZFPVVhRJoaEknhsL43ud0mk1WvjymHtweFqZbpuCSTGes8xg99GpbKVybDoAWlMZJA9wkx2PEzQ/kjUSAGPe9fhP7iJk2v0osWL3PEG42oigcBm4/G521H5OeshoirwmCkfHn/XaCfFj+1+w+1RLwRoGXpAtEUihMahN5f26XW2pxrvnPcxzvoZkMAGB8QQAyRoNgCE5l7AHXkAKubiZVwVBuDxE99FlsKN2N27Vg9aQQXR4CJm2QUpH8YWWAoASl4HWz6Dg3fs+/pM70Voqu8oMBoXI4DEREATh6iWCwiWm6RqbqraQGZ5O6QmFCSPjkAdpNbDu6QBJQTJ1pciQ4zLRHHXoPve5n6vTgb/088DfrfVdZZ4KCrJIYyEIw4IICpfYkZZiGlxN5IZOxONTu6XDvlC624lkDu02PqHEZQA6anPl2W8EfEc/BU0FpG5Boav7SAQFQRgORFC4xDZWbSHSFI7anIhEYA/mwaJ7OpBCuudDkuMygFODzWe7T1PxHdmIkpKPFBaD1tbQda6zFYxmMctIEIYJERQuIZffzZHmYq61TeJoRRvpSeGEWc49o+h86B5nj/5+yRqFZIk452Czv2IfurMFY/5c5MikYAI8OL1GQbQSBGG4EEHhEjrhKEVHJzsimxPVreRlRA9q+bqnA8ncvaUgSVJgXOEcQcF3eANSaAyG9AnIEQlfGFNoFeMJgjCMiKBwCZU4SlEkBV9bBKqmD35QcDuhl5lBSlwGmr0a3e/tcU511KBWH8aYNwdJVpAjE8DjRD81vVUTLQVBGFZEULiEih0nyIxIo6SyA0WWGJ06uF+2vbUU4NS4gq6htVR1O66523F/+DswhGDMnQ2AFJEYONfWEFjN7HR0m44qCMLVTQSFS8Ttd1PZXs2oqCyOltvJSo4gxNT7dpsDofu94Pf2uoZAOTXYfOa4gu7txLX6l2ht9Vhu/BbyqS9+OfJ0UKgHnwtUr+g+EoRhpF8rmktLS3nqqadwOBxERUXx3HPPkZmZ2e2a5uZmvv/971NbW4vP52Pq1Kn88Ic/xGAw8MILL/D666+TkBBI+lZYWMgzzzwz6JUZyk60lqPpGmmhGbxbV8vN0zMHtfzTC9d6CwpSWByEhAbHFXSPk861v0ZrqcKy4AkMKWOC18rh8QSmpTacMR1VtBQEYbjoV1B45plnWLZsGUuXLuW9997j6aef5tVXX+12zUsvvUR2djZ/+MMf8Pl8LFu2jPXr17No0SIAbr31Vp588snBr8EV4rj9BIqk4HVEoOu1F2GQ+VRQ6KX7SJIklNh0/DVFuDb8PrBITfNjvuExDOnju19rMCGFRqO11Qd3XBNjCoIwfPTZfdTc3ExRURFLliwBYMmSJRQVFdHS0tLtOkmScDqdaJqG1+vF5/ORmJh4cZ76ClTiOElGRCrHK52YDDJZyYP767urpdAzKEAgY6re1oC/Yj/GnOuw3vYsxqyzbOkZmYjWWt+V4iJUBAVBGC76bCnU1taSmJiIogT6vxVFISEhgdraWmJiulbjPvbYY3zzm99k5syZuFwu7r//fq655prg+VWrVvHZZ58RHx/PN7/5TSZOnHgRqjM0uf0eyturmJ8+h1177YxKi8JoGNzhnNOzhc6Wl8g0YTGKLQclOS+Y7O5s5IhE/GW70Z0ixYUgDDeDliV17dq15OTk8Morr+B0Olm+fDlr165l4cKF3HvvvTz66KMYjUa2bNnCY489xurVq4mO7n8XSmxs77+A+yM+fnASzg3U/rpKNF1jcuZYVreeYHJ+0qA/U1u1HzcQlxxonfUsPxxSk/pVliM5jZajGzG5m/AaTMQnJ/Sa2nuoudyf8+Ug6jw8XMo69xkUbDYb9fX1qKqKoiioqkpDQwM2m63bdX//+9/52c9+hizLhIeHM3fuXHbs2MHChQuJj+9KDT1jxgxsNhvHjx9nypQp/X7Q5uYONE3v+8IviI8Pp7Gx/bzvG0y7yg4hSzJRWjxeXzG6qg36M3kamwFocUJCJBdUvk8JtAw6yg6DJZKmpo5BecaLaSh8zpeaqPPwMNA6y7I0oB/TffZhxMbGkpeXx8qVKwFYuXIleXl53bqOAFJTU/n0008B8Hq9bNu2jVGjRgFQX9+1QvbIkSNUV1czYsSI837YK9Vx+0kywtNAC3TBhRgHbypqkKcDZAMYLjxH0elpqXprneg6EoRhpl/dR88++yxPPfUUL774IhERETz33HMALF++nCeeeIKCggJ+8IMf8Mwzz3DzzTejqirXXnstd999NwDPP/88hw8fRpZljEYjP//5z7u1Hq4G5W2VrCn7iJsy55ER0bUjmaqpVLRXcX3aTDw+FQDzIK5POO103qPB6OaRI7o+GzEdVRCGl34FhezsbFasWNHj+Msvvxz8Oz09nb/85S+93n86iFyNfJqfNaUf8WHFRjRdIznU1i0oODytqLpKojUejzcQFAZz0dppursDyTw4m99IhhCk0Gh0px0pdHCnzgqCMLSJ7TgvgF/z86vdv6OyvZqpSZM4aj9Ok6u52zVNrsDU3VhzDO7TQeEidB8FWgoDH4z/IjkiEdVpFy0FQRhmRJqLC9DQ2URlezW3jVzMl8fcTZI1gSZ39/Ubzaf+HWeJCXYfXZSWQi9psy+EHBlYfS7GFARheBFB4QK0eQMzAjLCUwGItcTQ7OoeFJpcLYGZRyGRwZaC+aK0FDpgEFsKpxPjidXMgjC8iKBwAU4HhQhTYA5xvCWWDp8Tl79rP+RmdwsxIVEosnJxWwqntuIcLEr8CJCU4EwkQRCGBxEULkAwKIREAIGWAnSNI5z++/Rxt9cPDH5LQfd7QfUO6piCIWUMYQ/85lSCPEEQhgsRFC5Aq6cNo2zErATWBsSd+vJvPmOwudnVEjx+obOPtI7mbvsnn3auDKkXYrDLEwRh6BNB4QK0eduJNIUH1wbEmWMBgoPNbr+Hdl8HseZTQeEC1ym4N/0Z17r/6XFc95zKe9RLhlRBEITzIYLCBWjztBMR0pWTxGq0EGqw0niqpdDitgNdLQi3V0WWJAzKwN52raUSzV6N5rR3O95XMjxBEIT+EkHhArR624kwRXQ7duYMpNNrFmLP6D4KMSkDWnWsudvRXW0AqNVF3c6day8FQRCE8yGCwgVo87YTGdI9e2GcJSYYDJpPtxROdSu5fWq3riO1uRLt1EY2fdHsNcG//dWHu527WGMKgiAMPyIoDJBX9eHyu4LTUU+Ls8TS7Laj6RpNrmZCFBOhRmvgHp/abTWza+3zuDe+TH+cDgpyQhZqdRG63pUxtqv7SLQUBEG4MCIoDFB7cI1C9+6jOHMMmq5hdztodrcQZ4kNdhe5T3UfAeiqD91pR606hNZa360M97Z/4D2wrtsxzVEDhhCMo69D73SgOWq7TnqcoBigj81zBEEQ+iKCwgC1BoNC91/ncZZTM5BcLYE1CuauFOMeb1dLQT9jsNh75JPg32rDCXwH1+Et+rhbuZq9Bjk6GUNqfuC6M8YVtI4WpJCwK2IjHEEQhjYRFAbo9MK1yJAvtBSCC9iau61RgO5jClpHYDBaskTiO7Y5sAAN8Ox6GwC9rQHt1B7JAJq9GjkqGTkiASk8HvXUuILaUo3/5E4M6RMuRjUFQRhmRFAYoDZPYCbQF8cUokIikSWZ0rYKvJrvHC2FQFAwFd4CHif+k7vw1xxBrT6MIWsyAGrd8cC1Hid6pwM5OhkIrDb21xxF11Q8W/4GJgumKXdc3AoLgjAsiKAwQG3ediQkwr/QfaTICjHmaI62BL7Qz2wpeHxdYwraqaBgHD0DOTIJb9HHeHe9jWSNwjzrYVCMwaBwevxAORUUlJQx4HPh2f4Gau1RQibfiWwefvvWCoIw+ERQGKBWTzthplBkqedbGG+Jxe4JdP3Entl95FWDeY/0DjuYrEhGM8Yxc9EaTqLWH8dUeAuSyYKSkIVaHwgKqr0aADk6BQAlOQ8A36EPkeMyMebOvngVFQRhWBFBYYACKS4iej13ZiCINQd2LtN1Pbh4DQLdR3JY4Drj6BmgmJDC4zDmzAJASRyF1lSO7vMEpqMqRqSwOABkSwRybBogYZ75AJIsPkZBEAaH2HltgNq8bd1SXJwp7tQ4QoQpHJMSmCbqVzU0Xe8aaHa2IIUGrpNCQrHMfxzJEoGkBD4SJWk07FuJ2nACzVGDHGXr9uUfMukOtI5mlISsi1ZHQRCGHxEUBqjV005ymK3Xc6enpcZ9oesI6DYlVYnLDJ43pI/vVoaSmA1IqPXH0ew1KEmjup03ZIjZRoIgDD7R7zAAmq7R7us4a/fR6WDwxZlHEEibras+dFcbUlhMr/dDoPUgx6TgrzyI3tGMHJU8iDUQBEHonQgKA+D0daLpWo/pqKfFWWKRkIg/1WKAwBoFALPJEFy4JoeePShAoAtJqy8JXHtqkFkQBOFiEkFhALp2XOs9KFgMZr4x4WvMSZsZPBbcitOodC1c6zModHUZKSIoCIJwCYgxhQFoPcvCtTPlxnQfAwh2Hxnl4MI1KSz6nK+jJI0O/CEbkCLEtpiCIFx8IigMQDDFxVnGFHpzOiiYTQa0lv51H8lhsUihMUghViR5cPd1FgRB6I3oPhqANk/P7iNN13lzw3Fqm5293nN6TCHEpKB3tAQXrvUlZMqdmCYsGYSnFgRB6JtoKQxAm7cdsxJCiNKVqrrR4WLdzkrCLEYWT+u52Y3njCmpurOlz1bCacZR0wfnoQVBEPqhXy2F0tJS7rnnHm688Ubuueceyv5fe3ceHmV5Ln78O1uSyb7DhBACCJiySjhaC1ZFBLSJcESx5pRiFdxotZ7jgrZlKbY/wVaLgAY5rkWtRQtKRJpS9WelSlksAsEosiSBJJNM1pnMZJb3PX9MMjBksuEkhMn9uS6vKzPv806eJw5zz7Pdz/HjbcpYLBbuvPNOcnNzmTlzJsuWLcPtdgPg8XhYvnw506ZN49prr2XTpk1BbURvqw+wca2q1g6A1e4KeI/DN3yk825c62A5qhBCnC9dCgpLly4lLy+Pv/71r+Tl5bFkyZI2ZfLz8xk+fDhbt25l69atHDp0iMLCQgC2bt1KSUkJhYWFvPnmm6xZs4aysrLgtqQXNTgb20wym+tagkJT4KBw5uoj1VaLNqrjSU5YCC0AAB8YSURBVGYhhDgfOg0KFouFoqIicnK849o5OTkUFRVRU1PjV06j0WCz2VAUBafTicvlYsCAAQBs27aNm2++Ga1WS2JiItOmTWP79u090Jze0dDcNu+RuZOeQrPTQ5hei0Z1d7pxTQghzpdOg0J5eTkDBgxAp/OuftHpdKSmplJeXu5X7t577+XYsWNMmTLF9192drbvNdLSTu/INZlMVFRUBLMdvare2dCmp1DV2lNwtDN81JI2u6sb14QQ4nwI2kTz9u3bGTVqFK+88go2m42FCxeyfft2Zs6cGZTXT0o690PpU1KCd9aAw+Wg2ePElJjs97o1jc0A2Js9gX+fVkNkhIFYgwMbED9oEJFBrNfZgtnmC4W0uX+QNvesToOCyWSisrISj8eDTqfD4/FgNpsxmfyTwW3cuJHf/va3aLVaYmJimDp1Krt27WLmzJmYTCZOnTrFuHHjgLY9h66wWKwoitqte8D7x6yqauz2fe0pt1UCoHeF+15XVVXfUtR6a3PA31ff4MCg01DbMpfS4I7AFsR6nSnYbb4QSJv7B2lz12m1mnP6Mt3p8FFSUhJZWVkUFBQAUFBQQFZWFomJ/sMf6enpfPzxxwA4nU4+/fRTRozw7uqdOXMmmzZtQlEUampq2LFjBzNmzOh2ZfuCA9VFAFwUP9T3XL3NidOlEG000ORwBwxeraeuKTJ8JITow7q0+mjZsmVs3LiRGTNmsHHjRpYvXw7AwoULOXDgAACPPfYYe/fuJTc3l9mzZ5OZmcncuXMBmDVrFunp6UyfPp25c+eyaNEiBg8e3ENN6ln7KveTGZvhd5BO6yTzUFMsKmALMK/Q7PKeutadjWtCCNHbujSnMHz48IB7CzZs2OD7OSMjg5deeing/TqdzhdILmTmpipKraeYc5H/DuPWSeahphgOHLVgtbuIiQzzK9Ps9BAbGdatjWtCCNHbJM1FN+wzfwHAJanj/J4319rRaCBzoHeZaqBlqQ6nRzauCSH6PAkK3bC3cj/D4jJJiIj3e76qzk5iTATxMd7eQaCg4J1T0MvGNSFEnyZBoYsqbJWcslWQnTq+zTVznZ3UBCPREQagnaDg9GDU4924JkFBCNFHSVDoor3mL9CgYULqmDbXzLV2UuKNRBkDBwVFUXG6FWK03rkHTWR8m9cQQoi+QIJCF6iqyr7K/VwUP5T48Di/a/ZmN1a7i9QEIxFhOvQ6TZug0Jr3KFr17mXQRklQEEL0TRIUuuCQ5UsqmsxMPGuCGU4vR02NN6LRaIgyGrCdFRRaM6RG4w0KmkgZPhJC9E0SFDpxylrBS4feYFC0ictMk9pcb12OmhJvBCDaaKCxKXBPwai0BgXpKQgh+iYJCh1odFrJ/+JlwnQG7hn3E79DdVq1psxOTfAGhRijAVuTk+Y9m/HUeFNa+I7idDeCRovG2P9ytwghLgwSFNrhUTxsOPAqDc4G7ho3v80y1FbmWjvRRgPGcO8+wCijgXhHKc597+D6aicADqf3sKFwdyOayHg0GvmzCyH6Jvl0asdXdd/wTf1x5o78TzJjM9otV9WyHLVVtNFAlsubH0ltrAKg2aUAoHc1oomMa/siQgjRR0hQaEdlk/cDfXTSxR2WM9f6B4W4MIXRumMAKFYLcHpOQd/cgFbmE4QQfZgEhXaYm6oJ14URG9Z+6lm3R6Gm0UFq/OmgMKT5S8I1bkjMQG3wBpbW4SOto142rgkh+rSgHbITaqqaqkk1JqPRaNotU9vYjKpCUuzpjKem2s8pd8eRPGgihgNbUJ12mp0edHjQOG2y8kiIM3g8bmprq3C7nV0qbzZrURSlh2vVt3TWZr0+jISEFHS64HycS1Boh9lezZCY9A7LtG5Si4707mT21JwkylpKoXMS1xjiMQBKYzXNLg9xLbuZZfhIiNNqa6uIiIgkKmpgh1/AWun1Wtzu/hUUOmqzqqrYbA3U1laRnGwKWKa7ZPgoALfixmKvISUyucNyrecmRLekt3AVf4yq0bKneSiNOm/GVKWxCofTQ3xrigvZzSyEj9vtJCoqtksBQbSl0WiIiortck+rKyQoBGCx16CikmrsOCj4egpGA6riwf31P/GkjcOqGqnHuxdBbaym2ekhKcwByG5mIc4mAeHbCfbfT4JCAGZ7NQCpnfUU7N4J5KgIA0p9BaqjEUPmJQDUu8JBH+7tKbg8JBlagoL0FIQQfZgEhQDMTa1BIaXDcq05jqKMepTaUwCEp2ag0YDV4UIbk+LrKcTpHKDVoQnv/kHaQoi+qbGxkddee+V8VyOoJCgEYG6qIkofSZQhssNyVrsLY7genVaLUnsS0KBPSCMqwoDV7kYTk4zSWNUy0dzUsptZuspChAqrtZHXX3+13etut7sXaxMcsvooALPd0unQEXh7A1ER3j+hUnsKTUwyGn04MZEGrHYX2pRkXOVf4jC4iaVJlqMK0ccdOnSQ/Pw12Gze5JULFtzN0KHDWLBgHjfccCOffbYTh8PB4sVLGD9+Ak89tRKr1cptt+URERFBfv6L/PSndzJ27HiKig4SFhbGk0+u5v33C3jjjT+i0WhIS0vn4YcfIyEhkW3btlJY+D5RUVGUlZURFxfHr371a1JSUpk3by6PPbaUsWPHAvCnP23kxIkTPPLIL3r0byBBIQBzUxUjE4Z3Ws5md/tWHim1J9EmDAK8+Y+sTU60MSngcqBRm4jGhjYytUfrLcSFbOeBcj75orzDMhoNqGr3X3vKOBOTx3a8ZLOxsZHf/e63PPnkMyQnJ1NdXc3ChT9m1aqnqa+vZ8yYcdx11yIKC98nP/8ZnnvuRf77vx9hwYJ5vPzy636vdfToEX7/+zXo9XqOHj1Cfv5aXnhhI8nJyWzY8BxPP/0kv/71/wPgiy/28/LLr5GRkcmLLz7P6tW/4/HHVzFnzlw2b36LsWPHoqoqW7a8zYoVK7vf+G6S4aOzOD1O6prrSTV2PJ8A3uGjKKMBVXGj1FegS0gDvJlSrXY3mlhvb8PoriVStckksxB92MGD+ykvP8WDD97Hbbfl8eCD96HRaPB4PBiNkUyefAUAo0eP5eTJkx2+1rXXzkSv937n3rdvD5dfPpnkZO/nwaxZN7Jnz798ZceNG09GRiYAubmz2bt3DwAzZ+awa9en1NfXs2vXpyQkJDJixMhgN7sN6SmcpcruzVeUGpnUaVlby4lrSr0ZFI9fT+FYeQPaGG8ivXi3hXB9syxHFaIDk8d2/m2+JzevqSoMHz6Cdes2+D1fXn6KsDCD77FWq8Xj6XiuwGg8PR+pqm2XjbY3taiqqq9sREQE1147k4KCd9m7dw833nhzd5pzzqSncJaurjwC7+a16AhDyyQzvqAQ3dpTiPYGljS1wntdegpC9FljxoyjrKyEffv2+J47fPgQagfjVVFRUTgcjg4nlLOz/4NPP92JxeL9bNm6dQuTJl3qu37gwH5KS0sA2LZtKxMnZvuu3Xjjzbz55usUFx/mqquuOee2dYf0FM5S1RIUUoyBewqKvYGmzcsJv+pOmhxu73LUulOABm2C91tOjNGA26Pg0kSgGiLJcFcCcuKaEH1ZbGwsTzzxFOvWrWb16t/jdrtISxvEz3/+UAf3xDF9+nXMn/9DYmJiyc9/sU2ZYcOGc9ddi3jggUUtE82DeOihx3zXJ0zI5oUX1nPs2FHfRHOrtLRBDBmSSVbWaAwGQ5vX7gkSFM5Saa8iNiyGCH1EwOtK1VFUqwVH0f9H5SKijAaUmpO+lUfgHT4CaLQ7saoxDNC1BgUZPhKiL8vKGs3atc+3ef699/7u+9lkSvN7/Mgjv/QrG+j+667L4brrcgL+TqMxgmXLfhPwms1m5dixb3j00SVdqn8wyPDRWaqaqjtcjuqp8W5SU8v2o0Uh2mhAqTuFtmWSGU7nQiouqaO0KRxty/ihDB8JIbpqy5a3+NGP5nLrrfNITR3Qa7+3Sz2FY8eOsXjxYurq6oiPj2flypVkZmb6lXn44YcpLi72PS4uLmbdunVcc801rFmzhtdff53UVO+SzIkTJ7J06dLgtSKIzE3VjE3Oave6Uus9d1njtDFcX0lU2DiUugrCBo/zlWkNCm999A3TtN7EeOgMENbxZjghRP9y/fW5XH99bsBrs2ffxOzZN/V6ZtguBYWlS5eSl5fHrFmzeOedd1iyZAmvvuq/i2/VqlW+n7/88kvmz5/PFVdc4Xtu9uzZPPLII0Gqds+wu+00uqwdTjIrtSfRDRyJy3yMcWElxKn1oLh9k8xwOijU25wM/M5gqDgou5mFEBeEToePLBYLRUVF5OR4x8NycnIoKiqipqam3XveeustcnNzCQsLC15Ne0HryqP2UmarioJSewptylAa40YwLqyUqGYzQMDho2ijgYtHj/Bel0lmIcQFoNOgUF5ezoABA9DpdADodDpSU1MpLw+889DpdLJ161bmzJnj9/x7771Hbm4ut99+O59//nkQqh58p6zepaMD29l5rDZWgceFLmEQ5piLidc2EV66CwBtvH9QGJgYyY1XDiMicSAg2VGFEBeGoK8+2rFjB2lpaWRlnR6X/+EPf8jdd9+NwWBg586d3HvvvWzbto2EhK6vxklKOvfsoikpMV0qZy6pxKiPYPSQoWg1beOlraYIG5A4bCQ1dY14TmjQlX2BPi6V1DT/3sWGX1wLgOJq5jgQmZRKchfrEQxdbXMokTZfeMxmLXp999a7dLd8KOiszVqtNmjvhU6DgslkorKyEo/Hg06nw+PxYDabMZkC7zx8++232/QSUlJOj9FPnjwZk8nE119/zaWXXnr27e2yWKwoSveTnqSkxFBV1dilssWVx0iPTsNSbQt4vfn4EQAaNPGU19dzTDFxke4Uapypw98RfsVtuE0ju1yPb6s7bQ4V0uYLk6Io3ZpE7elJ1ylTJlFY+DGRkV1fFLJv3x7WrVvNCy/8sc218vJTLFgwz28Ja3d1pc2KorR5L2i1mnP6Mt1pyE1KSiIrK4uCggIACgoKyMrKIjExsU3ZiooK9u7d65t/aFVZWen7+fDhw5w8eZKhQ4d2u7I9ya24OWk9xZDYwe2WUWpb9iMYIrDaXRzReZPmnTl0FEhY1lXoOikjhBB9QZeGj5YtW8bixYt59tlniY2NZeVKb6a+hQsXct999/lSu27evJmrr76a+Hj/8fOnnnqKQ4cOodVqMRgMrFq1yq/30BecslbgVj1kxKS3W0apOZ0J1eZw4Yi4CJS96NNG9VY1hRA97I03/sju3buor6/jrrsW+dJLLF/+S0pKTuByORk0aDCPPrqE2NjYNve//faf+fOfXycpKZlLLsluc72v61JQGD58OJs2bWrz/IYN/omj7rnnnoD3twaRvuxEo3f/QXs9BW8m1HLCMrz7Eax2FwnR8UTftE6WmgoRBK6vduIq/rjDMhqNpsNcRO0xjPo+hpGTu1RWq9WSn/8iJSXHufvuOxg//hISEhK5//4HfV94n3/+WV577RXuuednfvceOfI1r776Ii+99BqJiUn87ndPdLuu55ukuWhR0lBKlCGSpIjAk99KfaVfJlSb3UV6SrQEBCFCTE7OLAAyMjIZOXIUhw4dYMqUK9m+vYDCwu243S7sdgeDB2e0uffzz/fyve9NITHRmztt1qz/5MMP/9ar9f+2JCi0ONFYRkZMersf8r5MqIne4SWr4/QBO0KIb88wcnKn3+Z7e3evt1OiYf/+z9my5W2ee+5FEhISKCzczrvv/iVA+XM4AaiP6X9ruwJwepyU2yoZ0sl8AhoN2ngTbo9Cs9PjO4pTCBE63nvvXQBKS0s4cqSY0aPH0NjYSFRUNHFxcTidTl+Zs02cOIlPP91Jba13c29BwTu9Vu9gkU81oMxajqIqZHS28ig2FY0+DKu1GUB6CkKEoLCwMO6553bq6up46CHvWcrf/e73KCx8n7y8m0hNTeXii7MoKjrU5t6LLhrBvHk/4Z577iAxMYnLL59yHlrw7UhQAE40lAIwJLajnkIZujPmE+B0imwhRGj45BPvATt5eT/2e16v1/vOVD7bxImT/PYozJkzlzlz5voez5t3W/Ar2oNk+AgoaSwjNiyGuLC2y8sAVLcTpcHsm2S2SlAQQoQoCQrAiYYyhsR2MMlcXwmqckZQ8B69Fx0hQUEIEVr6fVCwux2Ym6o63rRW33LGcrw3tYfN4e0pyJyCECLU9PugUNZ4EhW14/QW9d40HdpYb/bU03MKMiUjhAgt/T4olNu85yGkRQ1st4xSX4nGGIcmzAiA1eFCr9MQbtD1Sh2FEKK39PugUG23YNDqiQsPPMkMoDZUoo07fUaqze4iymiQ3cxCiJDT74NCld1CkjEp4PkJrZT6Cr+gYLW7ZZJZCBGS+n1QqLZbSDG2TQMOUG9tRnXaUe0NaAL0FIQQorq6ip/97C7f448//oj/+q+b+MlP8igpOX7+KnaO+nVQUFWVKruFFGPbM5nLzFYeWLuT0m+OAqCNPaOn4HDJyiMhBADJySmsWbPe9/idd/7CHXfczUsvvU5GRmaXX8fj8fRA7bqvXy+fqXc24FJcJBuT2lyrrncAUHOyhARAG3d6ItpqdzHM1K//dEKEpLNPSmt9/L//+0cWLJjHDTfcyGef7cThcLB48RLGj5/gd88zz/yeL774nJKSE2zevIk1a9bz2Wf/ZP36tSiKQnx8Ag899Bjp6YPZt28PzzzzFOPHT+Dw4SLmz7+Djz76OwaDgbKyUk6eLOPKK6/m+9+/kuefz8dsrmTu3Dzmzr21R/8G/fqTrdruTVoVKCjYm70b1Nx1LXsUWpajqqqKzS4ZUoUItl3le/m0fHeHZTSa1syl3XO56T+4zPTtDrypr69nzJhx3HXXIgoL3yc//xmee+5FvzL33fc/fPVVMbfeOo/Jk6+gtraGxx9fwpo1zzN06DAKCrawfPkv2bDhFQCOHj3Cgw8u5oEHHgbgo4/+zrFjR1m9+jkUReGmm3JparKxdu3zWCzV5OXNISdnVreOC+2ufj18VGW3AJASICg0tQQFnbUKTWQ8GkM4AE6XgtujSFAQop8xGiOZPPkKAEaPHsvJkyc7vefQoYMMHz6SoUOHAXD99Tdw5MhXNDV5z4FPTx/MmDHj/O654oqrCAsLIyIigoyMIXzve1PQarWkpKQSExNLVZU5yC3z1797Ck3VaDXagAfrtAaFCKcFbdKZK48k75EQPeEyU3an3+Z7+jwFnU6HopzuijidTt/PYWGn/81rtVo8HncXXlGlo5XrRmPbb/zh4WF+vycszP9x137vuev3PYWE8Hh02rab0FqHj+KVOjRnTDJX1dkBSIwJ751KCiF6TWJiEm63m7Iyb+bkv/1t+7d6vdGjx3HkyFecOHEcgPffL2DEiFFERkZ926r2mP7dU7DXBBw6Am9QMGqcRGubcYQnYWx5vrTKCsDg1OheqqUQorfo9Xruv/9/eOCBRQwYMJCJEyd9q9dLSEjgl7/8NcuX/wKPx0N8fAJLlqwIUm17hka9QM6Ps1isft26rkpJiaGqqjHgtYc+XsrEAeO5ddSNAChN9TTvepPwy29l/fYTVHxdxINx26gafxvDLrsKgBe3HeaLI9X84b4rzrktPa2jNocqafOFqaLiBAMHDuly+d4+jrMv6EqbA/0dtVoNSUnd//Lab3sKTa4mmtx2v56Cp+wg7q//iTY6CXvzcIZEeYeKKt0xDGspU2a2ki69BCFEiOq3cwqBVh4pDd5ZfWfRB7gcdjIjvUGhtMk7eORRFE5W22ToSAgRsvptT6G6JSicuUdBaawCrR6abQznEEnh9TQQTUW9d9LZXGvH5VZIT5GgIIQITf02KFQFCgoNZnQDhqN6XEyq/AKNEoXNkEhlbRMApWaZZBYi2FRVlYzD30Kwp4X79fBRXFgM4brTa4DVBjPa2FTCxl1HkqaBRFc5LmMyVXV2FEWl1GxFp9VgSuq7y8mEuJDo9WHYbA1B/2DrL1RVxWZrQK8P67xwF/XbnkK13eLXS1BdDm821NhUGHwJ1Z5oknVWtHEDcB9XqWlwUGq2YkqKxKDvt7FUiKBKSEihtrYKq7WuS+W1Wi2K0r9WH3XWZr0+jISElKD9vn4bFKqaLFycOML3WGmoArw5jprdKh85srgpajfG5DTARWWtnVKzlVEZ8eepxkKEHp1OT3KyqcvlQ2EZbnf1dpu7FBSOHTvG4sWLqaurIz4+npUrV5KZmelX5uGHH6a4uNj3uLi4mHXr1nHNNdfg8Xh4/PHH+cc//oFGo+HOO+/k5ptvDmpDusPpcVHvbPBLmd268kgbm0qTw8U/m0fy3bGDGTgyG/7+GUfLG6htbJb5BCFESOtSUFi6dCl5eXnMmjWLd955hyVLlvDqq6/6lVm1apXv5y+//JL58+dzxRXeDV5bt26lpKSEwsJC6urqmD17Npdffjnp6elBbErXVfuWo54+XEc9IyjYaz140GHPuJy4WCNhei17i73XB8vKIyFECOs0KFgsFoqKinjppZcAyMnJYcWKFdTU1JCYGPjEsrfeeovc3FxfIqdt27Zx8803o9VqSUxMZNq0aWzfvp0FCxZ0uaJa7bmvTjj73ka3lZTIRNJiB/quaTwO9ClD0Bmj8dTUk5pgJD4mHL1Oy8WZCVRYmkhNMDI0LfZb1aW3XAh1DDZpc/8gbe65e6ALQaG8vJwBAwag03mTxul0OlJTUykvLw8YFJxOJ1u3buXll1/2e420tDTfY5PJREVFRbcqmpBw7it+zt7qPSXpEqaMuMS/0IwfAz/2lb9s3CDfpd/cM+Wcf/f5ci7b2y900ub+Qdrcs4K+jGbHjh2kpaWRlZUV7JcWQgjRwzoNCiaTicrKSt/5oR6PB7PZjMkUeMXA22+/zZw5c9q8xqlTp3yPy8vLGThw4Nm3CiGEOM86DQpJSUlkZWVRUFAAQEFBAVlZWQGHjioqKti7dy85OTl+z8+cOZNNmzahKAo1NTXs2LGDGTNmBKkJQgghgqVLw0fLli1j48aNzJgxg40bN7J8+XIAFi5cyIEDB3zlNm/ezNVXX018vP9a/lmzZpGens706dOZO3cuixYtYvDgwUFshhBCiGC4YM5TEEII0fMkX4MQQggfCQpCCCF8JCgIIYTwkaAghBDCJ6SDwrFjx7jllluYMWMGt9xyC8ePHz/fVQqq2tpaFi5cyIwZM8jNzeWnP/0pNTU1APz73//mhhtuYMaMGdx+++1YLJbzXNvgW7t2LaNGjeKrr74CQrvNzc3NLF26lOnTp5Obm8uvfvUrILTf4x9++CGzZ89m1qxZ5ObmUlhYCIROm1euXMnUqVP93sPQcft6pe1qCJs3b566ZcsWVVVVdcuWLeq8efPOc42Cq7a2Vv3ss898j5944gn10UcfVRVFUadNm6bu3r1bVVVVXbdunbp48eLzVc0ecfDgQfWOO+5Qr7rqKrW4uDjk27xixQr1N7/5jaooiqqqqlpVVaWqaui+xxVFUSdNmqQWFxerqqqqhw8fVidMmKB6PJ6QafPu3bvVU6dOqVdffbWvnara8f/T3mh7yAaF6upqNTs7W3W73aqqqqrb7Vazs7NVi8VynmvWc7Zv367Onz9f3b9/v/qDH/zA97zFYlEnTJhwHmsWXM3NzercuXPVkpIS3z+oUG6z1WpVs7OzVavV6vd8KL/HFUVRL730UnXPnj2qqqrqv/71L3X69Okh2eYzg0JH7euttofsITvdTeR3oVMUhTfeeIOpU6e2SUCYmJiIoii+8zAudKtXr+aGG27w2wAZym0uLS0lPj6etWvXsmvXLqKiorj//vuJiIgI2fe4RqPhD3/4A/feey+RkZHYbDbWr18f8v+uO2qfqqq90vaQnlPoT1asWEFkZCQ/+tGPzndVetTnn3/OgQMHyMvLO99V6TVut5vS0lK+853v8Je//IUHH3yQn/3sZzQ1NZ3vqvUYt9vN+vXrefbZZ/nwww957rnneOCBB0K6zX1FyPYUzkzkp9PpOk3kdyFbuXIlJ06cID8/H61W2yYBYU1NDRqN5oL/xgywe/dujh49yjXXXAN4823dcccdzJs3L2TbnJaWhl6v9+UUGz9+PAkJCURERITse/zw4cOYzWays7MByM7Oxmg0Eh4eHrJtho4/t1RV7ZW2h2xPoTuJ/C5kTz/9NAcPHmTdunW+Q43GjBmDw+Fgz549APzpT3/iuuuuO5/VDJo777yTTz75hA8++IAPPviAgQMH8sILL7BgwYKQbXNiYiKXXXYZO3fuBLwrUCwWC5mZmSH7Hh84cCAVFRUcPXoUgG+++Ybq6mqGDBkSsm2Gjj+3euszLaRzH33zzTcsXryYhoYGYmNjWblyJcOGDTvf1Qqar7/+mpycHDIzM4mIiAAgPT2ddevWsW/fPpYuXUpzczODBg3iySefJDk5uZNXvPBMnTqV/Px8Ro4cGdJtLi0t5bHHHqOurg69Xs/Pf/5zrrzyypB+j7/77rts2LABjcZ7gth9993HtGnTQqbNjz/+OIWFhVRXV5OQkEB8fDzvvfdeh+3rjbaHdFAQQgjRPSE7fCSEEKL7JCgIIYTwkaAghBDCR4KCEEIIHwkKQgghfCQoCCGE8JGgIIQQwkeCghBCCJ//AyUxzbKHsfw0AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "sns.set()\n",
    "plt.plot(entropy_perf_hist, label=\"entropy\")\n",
    "plt.plot(bald_perf_hist, label=\"bald\")\n",
    "plt.plot(uniform_perf_hist, label=\"uniform\")\n",
    "plt.ylim([0.7,1])\n",
    "plt.legend()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}