frameworks/tensorflow/tf1_manipulating_tfrecords.ipynb

{"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"name":"tf1_manipulating_tfrecords.ipynb","provenance":[],"collapsed_sections":[],"toc_visible":true},"kernelspec":{"name":"python3","display_name":"Python 3"}},"cells":[{"cell_type":"code","metadata":{"id":"Dr_NTi9CeMuA","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":34},"outputId":"5588971f-f146-4e67-add1-8c17186021ad","executionInfo":{"status":"ok","timestamp":1577958236192,"user_tz":-480,"elapsed":882,"user":{"displayName":"王DevOps","photoUrl":"https://lh3.googleusercontent.com/a-/AAuE7mAzLB0C3whTHAdHpq24UrEWqGtbhJElQxTU5_b_4g=s64","userId":"04300517850278510646"}}},"source":["import tensorflow as tf\n","import base64\n","import matplotlib.pyplot as plt\n","\n","print(\"TF Version: {}\".format(tf.__version__))"],"execution_count":26,"outputs":[{"output_type":"stream","text":["TF Version: 1.15.0\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"zmBTN_nWen4c","colab_type":"text"},"source":["# Data Preparation"]},{"cell_type":"markdown","metadata":{"id":"RnWENmyvfN5r","colab_type":"text"},"source":["Here you are going to use the MNIST dataset as an example to generate tfrecord files."]},{"cell_type":"code","metadata":{"id":"TRL0-MCiebGl","colab_type":"code","colab":{}},"source":["from tensorflow.examples.tutorials.mnist import input_data"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"oF0kVcQVes1s","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":85},"outputId":"7404f3a8-dbb4-49c0-a121-12a26c148b53","executionInfo":{"status":"ok","timestamp":1577958238866,"user_tz":-480,"elapsed":1394,"user":{"displayName":"王DevOps","photoUrl":"https://lh3.googleusercontent.com/a-/AAuE7mAzLB0C3whTHAdHpq24UrEWqGtbhJElQxTU5_b_4g=s64","userId":"04300517850278510646"}}},"source":["mnist = input_data.read_data_sets(\"/tmp/MNIST_data/\", one_hot=True)"],"execution_count":28,"outputs":[{"output_type":"stream","text":["Extracting /tmp/MNIST_data/train-images-idx3-ubyte.gz\n","Extracting /tmp/MNIST_data/train-labels-idx1-ubyte.gz\n","Extracting /tmp/MNIST_data/t10k-images-idx3-ubyte.gz\n","Extracting /tmp/MNIST_data/t10k-labels-idx1-ubyte.gz\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"id":"2SbnonOzfb5I","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":34},"outputId":"54ecea24-4421-4cd2-e4f2-cf4003c0478c","executionInfo":{"status":"ok","timestamp":1577958238867,"user_tz":-480,"elapsed":1131,"user":{"displayName":"王DevOps","photoUrl":"https://lh3.googleusercontent.com/a-/AAuE7mAzLB0C3whTHAdHpq24UrEWqGtbhJElQxTU5_b_4g=s64","userId":"04300517850278510646"}}},"source":["mnist.train.images.shape, mnist.train.labels.shape"],"execution_count":29,"outputs":[{"output_type":"execute_result","data":{"text/plain":["((55000, 784), (55000, 10))"]},"metadata":{"tags":[]},"execution_count":29}]},{"cell_type":"markdown","metadata":{"id":"fN1qdAace8xr","colab_type":"text"},"source":["# Write out tfrecord files"]},{"cell_type":"markdown","metadata":{"id":"_nzlWuv-fq1-","colab_type":"text"},"source":["The `tfrecord` file is a highly compressed format that is mainly composed of `tf.train.Feature()`. You first need to define the transformation function from different data types to `tf.train.Feature`. \n","\n","The following are three different examples of three datatypes.\n"]},{"cell_type":"code","metadata":{"id":"MoV8K7w1e8gF","colab_type":"code","colab":{}},"source":["def _bytes_feature(value):\n","  \"\"\"Returns a bytes_list from a string / byte.\"\"\"\n","  if isinstance(value, type(tf.constant(0))):\n","    value = value.numpy() # BytesList won't unpack a string from an EagerTensor.\n","  return tf.train.Feature(bytes_list=tf.train.BytesList(value=[value]))\n","\n","def _float_feature(value):\n","  \"\"\"Returns a float_list from a float / double.\"\"\"\n","  return tf.train.Feature(float_list=tf.train.FloatList(value=[value]))\n","\n","def _int64_feature(value):\n","  \"\"\"Returns an int64_list from a bool / enum / int / uint.\"\"\"\n","  return tf.train.Feature(int64_list=tf.train.Int64List(value=[value]))"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"9sWGzWPYghRk","colab_type":"text"},"source":["After you define the way to transform each data type, you next need to define a data schema describing the compose of a single piece of data (like a single data row). The following is an example of a data chunk composed of two attributes, that are images, and labels."]},{"cell_type":"code","metadata":{"id":"lcx2dxQ2ez81","colab_type":"code","colab":{}},"source":["def create_tf_example(imgInByte, labelInByte):\n","  tf_example = tf.train.Example(features=tf.train.Features(feature={\\\n","    'image': _bytes_feature(imgInByte) \\\n","  , 'label': _bytes_feature(labelInByte) \\\n","  }))\n","  return tf_example"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"zKoJGHybhPbu","colab_type":"text"},"source":["The following is an example to manipulate the image data. First, transform the data into the binary format and then encode the data into base64 format. These transformations are common while transmitting images over the network.\n","\n"," A function `Base64Converter` helps you to encode the binary data in base64 format and also introduce a safe way to do the encoding, called safe base64. Unlike the origin base64 encoding, the safe way tries to replace the '+' with '-' and replace the '/' with '_'."]},{"cell_type":"code","metadata":{"id":"7G9XZAxIfHJf","colab_type":"code","colab":{}},"source":["def Base64Converter(byte_string):\n","  _byteStr = base64.b64encode(byte_string)\n","  return _byteStr[:-2].replace(b\"+\", b\"-\").replace(b\"/\", b\"_\")"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"HbCRqo3CjCrM","colab_type":"text"},"source":["Now you can generate a `tfrecord` file by sequential compressing each piece of data."]},{"cell_type":"code","metadata":{"id":"aiTMNwWZfIge","colab_type":"code","colab":{}},"source":["with tf.python_io.TFRecordWriter(\"./mnist_test.tfrecord\") as writer:\n","  for i in range(mnist.test.num_examples):\n","    writer.write(\n","      create_tf_example(\n","        Base64Converter(mnist.test.images[i].astype(\"float32\").tobytes()),  # [None, 784]\n","        Base64Converter(mnist.test.labels[i].astype(\"float32\").tobytes())   # [None, 10]\n","      ).SerializeToString()\n","    )"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"27YXxtDHjaZI","colab_type":"text"},"source":["# Read the tfrecord files"]},{"cell_type":"markdown","metadata":{"id":"-YDd2g-6kuL1","colab_type":"text"},"source":["The following function is an example showing how to decode the each piece of data and to transform it into the correct datatype."]},{"cell_type":"code","metadata":{"id":"MfFnEXNRjYZk","colab_type":"code","colab":{}},"source":["def read_and_decode(filename_queue):\n","  # create a tfrecord object\n","  reader = tf.TFRecordReader()\n","  _, serialized_example = reader.read(filename_queue)\n","  \n","  # decode the example\n","  features = tf.parse_single_example(\n","      serialized_example,\n","      features={'image':tf.FixedLenFeature([], tf.string), \n","                'label':tf.FixedLenFeature([], tf.string)})\n","  \n","  label = tf.io.decode_base64(features[\"label\"])\n","  label = tf.io.decode_raw(label, tf.float32)\n","  \n","  # it must decode byteslist from string type to uint8 type\n","  image = tf.io.decode_base64(features[\"image\"])\n","  image = tf.io.decode_raw(image, tf.float32)\n","  return image, label"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"EiWDNUepkIo_","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":866},"outputId":"8d1317e5-7305-4a9d-d3c5-c7f595422b00","executionInfo":{"status":"ok","timestamp":1577958260813,"user_tz":-480,"elapsed":1586,"user":{"displayName":"王DevOps","photoUrl":"https://lh3.googleusercontent.com/a-/AAuE7mAzLB0C3whTHAdHpq24UrEWqGtbhJElQxTU5_b_4g=s64","userId":"04300517850278510646"}}},"source":["tf.reset_default_graph()\n","\n","with tf.name_scope('input'):\n","  # return a QueueRunner object and FIFOQueue object inside in\n","  filename_queue = tf.train.string_input_producer(\n","      [\"./mnist_test.tfrecord\"], num_epochs=None)\n","  \n","  image, label = read_and_decode(filename_queue)\n","  \n","  with tf.Session() as sess:\n","\n","    # because one epoch variable is built inside string_input_produer (image_raw)\n","    # and the variable is belonging to tf.GraphKeys.LOCAL_VARIABLES\n","    # tf.local_variables_initializer() is necessary\n","    init_op = tf.group(tf.global_variables_initializer(), tf.local_variables_initializer())\n","    sess.run(init_op)\n","\n","    coord=tf.train.Coordinator()\n","    threads= tf.train.start_queue_runners(coord=coord)\n","    \n","    for i in range(0, 3, 1):\n","      img, lbl = sess.run([image, label])\n","      print(lbl)\n","      img = img.reshape(28, 28)\n","      plt.imshow(img, cmap=plt.cm.gray)\n","      plt.show()"],"execution_count":35,"outputs":[{"output_type":"stream","text":["[0. 0. 0. 0. 0. 0. 0. 1. 0. 0.]\n"],"name":"stdout"},{"output_type":"display_data","data":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAM3ElEQVR4nO3dXahc9bnH8d/vpCmI6UXiS9ik0bTB\nC8tBEo1BSCxbQktOvIjFIM1FyYHi7kWUFkuo2It4WaQv1JvALkrTkmMJpGoQscmJxVDU4o5Es2NI\njCGaxLxYIjQRJMY+vdjLso0za8ZZa2ZN8nw/sJmZ9cya9bDMz7VmvczfESEAV77/aroBAINB2IEk\nCDuQBGEHkiDsQBJfGeTCbHPoH+iziHCr6ZW27LZX2j5o+7Dth6t8FoD+cq/n2W3PkHRI0nckHZf0\nmqS1EfFWyTxs2YE+68eWfamkwxFxJCIuSPqTpNUVPg9AH1UJ+zxJx6a9Pl5M+xzbY7YnbE9UWBaA\nivp+gC4ixiWNS+zGA02qsmU/IWn+tNdfL6YBGEJVwv6apJtsf8P2VyV9X9L2etoCULeed+Mj4qLt\nByT9RdIMSU9GxP7aOgNQq55PvfW0ML6zA33Xl4tqAFw+CDuQBGEHkiDsQBKEHUiCsANJEHYgCcIO\nJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnC\nDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJ9Dw+uyTZPirpnKRPJV2MiCV1NAWgfpXCXrgrIv5Rw+cA\n6CN244EkqoY9JO2wvcf2WKs32B6zPWF7ouKyAFTgiOh9ZnteRJywfb2knZIejIjdJe/vfWEAuhIR\nbjW90pY9Ik4Uj2ckPS1paZXPA9A/PYfd9tW2v/bZc0nflTRZV2MA6lXlaPxcSU/b/uxz/i8iXqil\nKwC1q/Sd/UsvjO/sQN/15Ts7gMsHYQeSIOxAEoQdSIKwA0nUcSNMCmvWrGlbu//++0vnff/990vr\nH3/8cWl9y5YtpfVTp061rR0+fLh0XuTBlh1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkuCuty4dOXKk\nbW3BggWDa6SFc+fOta3t379/gJ0Ml+PHj7etPfbYY6XzTkxcvr+ixl1vQHKEHUiCsANJEHYgCcIO\nJEHYgSQIO5AE97N3qeye9VtuuaV03gMHDpTWb7755tL6rbfeWlofHR1tW7vjjjtK5z127Fhpff78\n+aX1Ki5evFha/+CDD0rrIyMjPS/7vffeK61fzufZ22HLDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJ\ncD/7FWD27Nlta4sWLSqdd8+ePaX122+/vaeeutHp9/IPHTpUWu90/cKcOXPa1tavX18676ZNm0rr\nw6zn+9ltP2n7jO3JadPm2N5p++3isf2/NgBDoZvd+N9LWnnJtIcl7YqImyTtKl4DGGIdwx4RuyWd\nvWTyakmbi+ebJd1Tc18AatbrtfFzI+Jk8fyUpLnt3mh7TNJYj8sBUJPKN8JERJQdeIuIcUnjEgfo\ngCb1eurttO0RSSoez9TXEoB+6DXs2yWtK56vk/RsPe0A6JeO59ltPyVpVNK1kk5L2ijpGUlbJd0g\n6V1J90XEpQfxWn0Wu/Ho2r333lta37p1a2l9cnKybe2uu+4qnffs2Y7/nIdWu/PsHb+zR8TaNqUV\nlToCMFBcLgskQdiBJAg7kARhB5Ig7EAS3OKKxlx//fWl9X379lWaf82aNW1r27ZtK533csaQzUBy\nhB1IgrADSRB2IAnCDiRB2IEkCDuQBEM2ozGdfs75uuuuK61/+OGHpfWDBw9+6Z6uZGzZgSQIO5AE\nYQeSIOxAEoQdSIKwA0kQdiAJ7mdHXy1btqxt7cUXXyydd+bMmaX10dHR0vru3btL61cq7mcHkiPs\nQBKEHUiCsANJEHYgCcIOJEHYgSS4nx19tWrVqra1TufRd+3aVVp/5ZVXeuopq45bdttP2j5je3La\ntEdtn7C9t/hr/18UwFDoZjf+95JWtpj+m4hYVPw9X29bAOrWMewRsVvS2QH0AqCPqhyge8D2m8Vu\n/ux2b7I9ZnvC9kSFZQGoqNewb5K0UNIiSScl/ardGyNiPCKWRMSSHpcFoAY9hT0iTkfEpxHxL0m/\nk7S03rYA1K2nsNsemfbye5Im270XwHDoeJ7d9lOSRiVda/u4pI2SRm0vkhSSjkr6UR97xBC76qqr\nSusrV7Y6kTPlwoULpfNu3LixtP7JJ5+U1vF5HcMeEWtbTH6iD70A6CMulwWSIOxAEoQdSIKwA0kQ\ndiAJbnFFJRs2bCitL168uG3thRdeKJ335Zdf7qkntMaWHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeS\nYMhmlLr77rtL688880xp/aOPPmpbK7v9VZJeffXV0jpaY8hmIDnCDiRB2IEkCDuQBGEHkiDsQBKE\nHUiC+9mTu+aaa0rrjz/+eGl9xowZpfXnn28/5ifn0QeLLTuQBGEHkiDsQBKEHUiCsANJEHYgCcIO\nJMH97Fe4TufBO53rvu2220rr77zzTmm97J71TvOiNz3fz257vu2/2n7L9n7bPy6mz7G90/bbxePs\nupsGUJ9uduMvSvppRHxL0h2S1tv+lqSHJe2KiJsk7SpeAxhSHcMeEScj4vXi+TlJByTNk7Ra0ubi\nbZsl3dOvJgFU96Wujbe9QNJiSX+XNDciThalU5LmtplnTNJY7y0CqEPXR+Ntz5K0TdJPIuKf02sx\ndZSv5cG3iBiPiCURsaRSpwAq6SrstmdqKuhbIuLPxeTTtkeK+oikM/1pEUAdOu7G27akJyQdiIhf\nTyttl7RO0i+Kx2f70iEqWbhwYWm906m1Th566KHSOqfXhkc339mXSfqBpH229xbTHtFUyLfa/qGk\ndyXd158WAdShY9gj4m+SWp6kl7Si3nYA9AuXywJJEHYgCcIOJEHYgSQIO5AEPyV9Bbjxxhvb1nbs\n2FHpszds2FBaf+655yp9PgaHLTuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJMF59ivA2Fj7X/264YYb\nKn32Sy+9VFof5E+Roxq27EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBOfZLwPLly8vrT/44IMD6gSX\nM7bsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5BEN+Ozz5f0B0lzJYWk8Yj4re1HJd0v6YPirY9ExPP9\najSzO++8s7Q+a9asnj+70/jp58+f7/mzMVy6uajmoqSfRsTrtr8maY/tnUXtNxHxy/61B6Au3YzP\nflLSyeL5OdsHJM3rd2MA6vWlvrPbXiBpsaS/F5MesP2m7Sdtz24zz5jtCdsTlToFUEnXYbc9S9I2\nST+JiH9K2iRpoaRFmtry/6rVfBExHhFLImJJDf0C6FFXYbc9U1NB3xIRf5akiDgdEZ9GxL8k/U7S\n0v61CaCqjmG3bUlPSDoQEb+eNn1k2tu+J2my/vYA1KWbo/HLJP1A0j7be4tpj0haa3uRpk7HHZX0\no750iEreeOON0vqKFStK62fPnq2zHTSom6Pxf5PkFiXOqQOXEa6gA5Ig7EAShB1IgrADSRB2IAnC\nDiThQQ65a5vxfYE+i4hWp8rZsgNZEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoMesvkfkt6d9vraYtow\nGtbehrUvid56VWdvN7YrDPSimi8s3J4Y1t+mG9behrUvid56Naje2I0HkiDsQBJNh3284eWXGdbe\nhrUvid56NZDeGv3ODmBwmt6yAxgQwg4k0UjYba+0fdD2YdsPN9FDO7aP2t5ne2/T49MVY+idsT05\nbdoc2zttv108thxjr6HeHrV9olh3e22vaqi3+bb/avst2/tt/7iY3ui6K+lrIOtt4N/Zbc+QdEjS\ndyQdl/SapLUR8dZAG2nD9lFJSyKi8QswbH9b0nlJf4iI/y6mPSbpbET8ovgf5eyI+NmQ9PaopPNN\nD+NdjFY0Mn2YcUn3SPpfNbjuSvq6TwNYb01s2ZdKOhwRRyLigqQ/SVrdQB9DLyJ2S7p0SJbVkjYX\nzzdr6h/LwLXpbShExMmIeL14fk7SZ8OMN7ruSvoaiCbCPk/SsWmvj2u4xnsPSTts77E91nQzLcyN\niJPF81OS5jbZTAsdh/EepEuGGR+addfL8OdVcYDui5ZHxK2S/kfS+mJ3dSjF1HewYTp32tUw3oPS\nYpjx/2hy3fU6/HlVTYT9hKT5015/vZg2FCLiRPF4RtLTGr6hqE9/NoJu8Xim4X7+Y5iG8W41zLiG\nYN01Ofx5E2F/TdJNtr9h+6uSvi9pewN9fIHtq4sDJ7J9taTvaviGot4uaV3xfJ2kZxvs5XOGZRjv\ndsOMq+F11/jw5xEx8D9JqzR1RP4dST9vooc2fX1T0hvF3/6me5P0lKZ26z7R1LGNH0q6RtIuSW9L\n+n9Jc4aotz9K2ifpTU0Fa6Sh3pZrahf9TUl7i79VTa+7kr4Gst64XBZIggN0QBKEHUiCsANJEHYg\nCcIOJEHYgSQIO5DEvwEvYRv57rmVLgAAAABJRU5ErkJggg==\n","text/plain":["<Figure size 432x288 with 1 Axes>"]},"metadata":{"tags":[]}},{"output_type":"stream","text":["[0. 0. 1. 0. 0. 0. 0. 0. 0. 0.]\n"],"name":"stdout"},{"output_type":"display_data","data":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAANYElEQVR4nO3df4hd9ZnH8c9n3QTEFk0iOwxG1hr1\nj7iolVEWVxaX2uiKJgakJshiqTD9o0LF+CNkhQiLKLvb3T8DUxoatWvTkJjGumzqhvpjwQRHiTHR\ntBpJbMIkQzZgE0Rqkmf/mDPLVOeeOznn3ntu8rxfMNx7z3PvOQ9XPzm/7jlfR4QAnPv+rOkGAPQG\nYQeSIOxAEoQdSIKwA0n8eS8XZptD/0CXRYSnm15rzW77dtu/tf2R7ZV15gWgu1z1PLvt8yT9TtK3\nJR2U9Jak5RHxfslnWLMDXdaNNfuNkj6KiI8j4o+Sfi5pSY35AeiiOmG/RNLvp7w+WEz7E7aHbY/a\nHq2xLAA1df0AXUSMSBqR2IwHmlRnzX5I0qVTXs8vpgHoQ3XC/pakK21/w/ZsScskbelMWwA6rfJm\nfESctP2gpK2SzpO0NiL2dKwzAB1V+dRbpYWxzw50XVd+VAPg7EHYgSQIO5AEYQeSIOxAEoQdSIKw\nA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EASPb2VNKp55JFHSuvnn39+y9o111xT+tl77rmnUk+T1qxZ\nU1p/8803W9aee+65WsvGmWHNDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJcHfZPrB+/frSet1z4U3a\nt29fy9qtt95a+tlPPvmk0+2kwN1lgeQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJrmfvgSbPo+/du7e0\nvnXr1tL65ZdfXlq/6667SusLFixoWbvvvvtKP/v000+X1nFmaoXd9n5JxyWdknQyIoY60RSAzuvE\nmv3vIuJoB+YDoIvYZweSqBv2kPRr22/bHp7uDbaHbY/aHq25LAA11N2MvzkiDtn+C0mv2N4bEa9P\nfUNEjEgakbgQBmhSrTV7RBwqHsclvSjpxk40BaDzKofd9gW2vz75XNIiSbs71RiAzqqzGT8g6UXb\nk/P5j4j4r450dZYZGio/47h06dJa89+zZ09pffHixS1rR4+Wnyg5ceJEaX327Nml9e3bt5fWr732\n2pa1efPmlX4WnVU57BHxsaTW/yUB9BVOvQFJEHYgCcIOJEHYgSQIO5AEl7h2wODgYGm9OD3ZUrtT\na7fddltpfWxsrLRex4oVK0rrCxcurDzvl19+ufJnceZYswNJEHYgCcIOJEHYgSQIO5AEYQeSIOxA\nEpxn74CXXnqptH7FFVeU1o8fP15aP3bs2Bn31CnLli0rrc+aNatHnaAu1uxAEoQdSIKwA0kQdiAJ\nwg4kQdiBJAg7kATn2XvgwIEDTbfQ0qOPPlpav+qqq2rNf8eOHZVq6DzW7EAShB1IgrADSRB2IAnC\nDiRB2IEkCDuQhCOidwuze7cwSJLuvPPO0vqGDRtK6+2GbB4fHy+tl10P/9prr5V+FtVExLQDFbRd\ns9tea3vc9u4p0+bafsX2h8XjnE42C6DzZrIZ/1NJt39p2kpJ2yLiSknbitcA+ljbsEfE65K+fF+k\nJZLWFc/XSbq7w30B6LCqv40fiIjJAcYOSxpo9Ubbw5KGKy4HQIfUvhAmIqLswFtEjEgakThABzSp\n6qm3I7YHJal4LD8kC6BxVcO+RdL9xfP7Jf2yM+0A6Ja2m/G2X5B0i6SLbR+UtFrSM5J+YfsBSQck\nfaebTaK6oaGh0nq78+jtrF+/vrTOufT+0TbsEbG8RelbHe4FQBfxc1kgCcIOJEHYgSQIO5AEYQeS\n4FbS54DNmze3rC1atKjWvJ999tnS+hNPPFFr/ugd1uxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kAS3\nkj4LDA4OltbffffdlrV58+aVfvbo0aOl9Ztuuqm0vm/fvtI6eq/yraQBnBsIO5AEYQeSIOxAEoQd\nSIKwA0kQdiAJrmc/C2zcuLG03u5cepnnn3++tM559HMHa3YgCcIOJEHYgSQIO5AEYQeSIOxAEoQd\nSILz7H1g8eLFpfXrr7++8rxfffXV0vrq1asrzxtnl7ZrdttrbY/b3j1l2pO2D9neWfzd0d02AdQ1\nk834n0q6fZrp/x4R1xV//9nZtgB0WtuwR8Trko71oBcAXVTnAN2DtncVm/lzWr3J9rDtUdujNZYF\noKaqYV8jaYGk6ySNSfpRqzdGxEhEDEXEUMVlAeiASmGPiCMRcSoiTkv6saQbO9sWgE6rFHbbU+9t\nvFTS7lbvBdAf2p5nt/2CpFskXWz7oKTVkm6xfZ2kkLRf0ve72ONZr9315qtWrSqtz5o1q/Kyd+7c\nWVo/ceJE5Xnj7NI27BGxfJrJP+lCLwC6iJ/LAkkQdiAJwg4kQdiBJAg7kASXuPbAihUrSus33HBD\nrflv3ry5ZY1LWDGJNTuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJOGI6N3C7N4trI98/vnnpfU6l7BK\n0vz581vWxsbGas0bZ5+I8HTTWbMDSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBJcz34OmDt3bsvaF198\n0cNOvurTTz9tWWvXW7vfH1x44YWVepKkiy66qLT+8MMPV573TJw6dapl7fHHHy/97GeffVZpmazZ\ngSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJzrOfA3bt2tV0Cy1t2LChZa3dtfYDAwOl9XvvvbdST/3u\n8OHDpfWnnnqq0nzbrtltX2r7N7bft73H9g+L6XNtv2L7w+JxTqUOAPTETDbjT0paERELJf21pB/Y\nXihppaRtEXGlpG3FawB9qm3YI2IsIt4pnh+X9IGkSyQtkbSueNs6SXd3q0kA9Z3RPrvtyyR9U9IO\nSQMRMbnTdVjStDtYtoclDVdvEUAnzPhovO2vSdoo6aGI+MPUWkzctXLam0lGxEhEDEXEUK1OAdQy\no7DbnqWJoP8sIjYVk4/YHizqg5LGu9MigE5oeytp29bEPvmxiHhoyvR/kfS/EfGM7ZWS5kbEY23m\nlfJW0ps2bSqtL1mypEed5HLy5MmWtdOnT9ea95YtW0rro6Ojlef9xhtvlNa3b99eWm91K+mZ7LP/\njaR/kPSe7Z3FtFWSnpH0C9sPSDog6TszmBeAhrQNe0T8j6Rp/6WQ9K3OtgOgW/i5LJAEYQeSIOxA\nEoQdSIKwA0kwZHMfeOyx0p8n1B7SuczVV19dWu/mZaRr164tre/fv7/W/Ddu3Niytnfv3lrz7mcM\n2QwkR9iBJAg7kARhB5Ig7EAShB1IgrADSXCeHTjHcJ4dSI6wA0kQdiAJwg4kQdiBJAg7kARhB5Ig\n7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkmgbdtuX2v6N7fdt77H9w2L6k7YP2d5Z/N3R/XYB\nVNX25hW2ByUNRsQ7tr8u6W1Jd2tiPPYTEfGvM14YN68Auq7VzStmMj77mKSx4vlx2x9IuqSz7QHo\ntjPaZ7d9maRvStpRTHrQ9i7ba23PafGZYdujtkdrdQqglhnfg8721yS9JumpiNhke0DSUUkh6Z80\nsan/vTbzYDMe6LJWm/EzCrvtWZJ+JWlrRPzbNPXLJP0qIv6qzXwIO9BllW84aduSfiLpg6lBLw7c\nTVoqaXfdJgF0z0yOxt8s6Q1J70k6XUxeJWm5pOs0sRm/X9L3i4N5ZfNizQ50Wa3N+E4h7ED3cd94\nIDnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEm1vONlhRyUd\nmPL64mJaP+rX3vq1L4nequpkb3/ZqtDT69m/snB7NCKGGmugRL/21q99SfRWVa96YzMeSIKwA0k0\nHfaRhpdfpl9769e+JHqrqie9NbrPDqB3ml6zA+gRwg4k0UjYbd9u+7e2P7K9sokeWrG93/Z7xTDU\njY5PV4yhN25795Rpc22/YvvD4nHaMfYa6q0vhvEuGWa80e+u6eHPe77Pbvs8Sb+T9G1JByW9JWl5\nRLzf00ZasL1f0lBENP4DDNt/K+mEpGcnh9ay/c+SjkXEM8U/lHMi4vE+6e1JneEw3l3qrdUw499V\ng99dJ4c/r6KJNfuNkj6KiI8j4o+Sfi5pSQN99L2IeF3SsS9NXiJpXfF8nSb+Z+m5Fr31hYgYi4h3\niufHJU0OM97od1fSV080EfZLJP1+yuuD6q/x3kPSr22/bXu46WamMTBlmK3DkgaabGYabYfx7qUv\nDTPeN99dleHP6+IA3VfdHBHXS/p7ST8oNlf7Ukzsg/XTudM1khZoYgzAMUk/arKZYpjxjZIeiog/\nTK01+d1N01dPvrcmwn5I0qVTXs8vpvWFiDhUPI5LelETux395MjkCLrF43jD/fy/iDgSEaci4rSk\nH6vB764YZnyjpJ9FxKZicuPf3XR99ep7ayLsb0m60vY3bM+WtEzSlgb6+ArbFxQHTmT7AkmL1H9D\nUW+RdH/x/H5Jv2ywlz/RL8N4txpmXA1/d40Pfx4RPf+TdIcmjsjvk/SPTfTQoq/LJb1b/O1pujdJ\nL2his+4LTRzbeEDSPEnbJH0o6b8lze2j3p7TxNDeuzQRrMGGertZE5vouyTtLP7uaPq7K+mrJ98b\nP5cFkuAAHZAEYQeSIOxAEoQdSIKwA0kQdiAJwg4k8X98jzceoKWtgAAAAABJRU5ErkJggg==\n","text/plain":["<Figure size 432x288 with 1 Axes>"]},"metadata":{"tags":[]}},{"output_type":"stream","text":["[0. 1. 0. 0. 0. 0. 0. 0. 0. 0.]\n"],"name":"stdout"},{"output_type":"display_data","data":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAMEElEQVR4nO3dXYhc5R3H8d+vabwwepFUE4OKsRJR\nUUzKIoKhWnzBBiHmRoxQEiqsFwYi9KJiLxRKQaTaCy+EFcU0WF+IBqPWaBrEtDeaVVNNfIlWIias\nWSWCb4g1+fdiT8oad85s5pwzZ9z/9wPLzDzPnDl/DvnlOXNe5nFECMDM95O2CwDQH4QdSIKwA0kQ\ndiAJwg4k8dN+rsw2h/6BhkWEp2qvNLLbvtr2u7bft31rlc8C0Cz3ep7d9ixJeyRdKWmfpB2SVkXE\nWyXLMLIDDWtiZL9I0vsR8UFEfCvpUUkrKnwegAZVCfupkj6a9Hpf0fY9todtj9oerbAuABU1foAu\nIkYkjUjsxgNtqjKy75d0+qTXpxVtAAZQlbDvkLTY9pm2j5N0vaTN9ZQFoG4978ZHxHe210p6XtIs\nSQ9GxO7aKgNQq55PvfW0Mr6zA41r5KIaAD8ehB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKE\nHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4k0dcpm5HP2Wef3bHvnXfe\nKV123bp1pf333ntvTzVlxcgOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0lwnh2NWrp0ace+w4cPly67\nb9++ustJrVLYbe+V9IWkQ5K+i4ihOooCUL86RvZfRcSnNXwOgAbxnR1IomrYQ9ILtl+1PTzVG2wP\n2x61PVpxXQAqqLobvywi9tueL2mr7XciYvvkN0TEiKQRSbIdFdcHoEeVRvaI2F88jkvaJOmiOooC\nUL+ew257ju0TjzyXdJWkXXUVBqBeVXbjF0jaZPvI5/wtIrbUUhVmjCVLlnTs++qrr0qX3bRpU93l\npNZz2CPiA0kX1lgLgAZx6g1IgrADSRB2IAnCDiRB2IEkuMUVlZx//vml/WvXru3Yt2HDhrrLQQlG\ndiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgvPsqOScc84p7Z8zZ07Hvscee6zuclCCkR1IgrADSRB2\nIAnCDiRB2IEkCDuQBGEHknBE/yZpYUaYmeeVV14p7T/55JM79nW7F77bT01jahHhqdoZ2YEkCDuQ\nBGEHkiDsQBKEHUiCsANJEHYgCe5nR6lFixaV9g8NDZX279mzp2Mf59H7q+vIbvtB2+O2d01qm2d7\nq+33ise5zZYJoKrp7MY/JOnqo9pulbQtIhZL2la8BjDAuoY9IrZLOnhU8wpJ64vn6yVdW3NdAGrW\n63f2BRExVjz/WNKCTm+0PSxpuMf1AKhJ5QN0ERFlN7hExIikEYkbYYA29Xrq7YDthZJUPI7XVxKA\nJvQa9s2SVhfPV0t6qp5yADSl62687UckXSbpJNv7JN0u6U5Jj9u+UdKHkq5rski059JLL620/Cef\nfFJTJaiqa9gjYlWHrstrrgVAg7hcFkiCsANJEHYgCcIOJEHYgSS4xRWlLrjggkrL33XXXTVVgqoY\n2YEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCaZsTu7iiy8u7X/22WdL+/fu3Vvaf8kll3Ts++abb0qX\nRW+YshlIjrADSRB2IAnCDiRB2IEkCDuQBGEHkuB+9uSuuOKK0v558+aV9m/ZsqW0n3Ppg4ORHUiC\nsANJEHYgCcIOJEHYgSQIO5AEYQeS4Dx7chdeeGFpf7ffO9i4cWOd5aBBXUd22w/aHre9a1LbHbb3\n295Z/C1vtkwAVU1nN/4hSVdP0f6XiFhS/P293rIA1K1r2CNiu6SDfagFQIOqHKBba/uNYjd/bqc3\n2R62PWp7tMK6AFTUa9jvk3SWpCWSxiTd3emNETESEUMRMdTjugDUoKewR8SBiDgUEYcl3S/ponrL\nAlC3nsJue+Gklysl7er0XgCDoevvxtt+RNJlkk6SdEDS7cXrJZJC0l5JN0XEWNeV8bvxfXfKKaeU\n9u/cubO0/7PPPivtP/fcc4+5JjSr0+/Gd72oJiJWTdH8QOWKAPQVl8sCSRB2IAnCDiRB2IEkCDuQ\nBLe4znBr1qwp7Z8/f35p/3PPPVdjNWgTIzuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJMF59hnujDPO\nqLR8t1tc8ePByA4kQdiBJAg7kARhB5Ig7EAShB1IgrADSXCefYa75pprKi3/9NNP11QJ2sbIDiRB\n2IEkCDuQBGEHkiDsQBKEHUiCsANJcJ59Bli2bFnHvm5TNiOPriO77dNtv2j7Ldu7ba8r2ufZ3mr7\nveJxbvPlAujVdHbjv5P0u4g4T9LFkm62fZ6kWyVti4jFkrYVrwEMqK5hj4ixiHiteP6FpLclnSpp\nhaT1xdvWS7q2qSIBVHdM39ltL5K0VNLLkhZExFjR9bGkBR2WGZY03HuJAOow7aPxtk+Q9ISkWyLi\n88l9ERGSYqrlImIkIoYiYqhSpQAqmVbYbc/WRNAfjogni+YDthcW/QsljTdTIoA6dN2Nt21JD0h6\nOyLumdS1WdJqSXcWj081UiG6WrlyZce+WbNmlS77+uuvl/Zv3769p5oweKbznf0SSb+R9KbtnUXb\nbZoI+eO2b5T0oaTrmikRQB26hj0i/iXJHbovr7ccAE3hclkgCcIOJEHYgSQIO5AEYQeS4BbXH4Hj\njz++tH/58uU9f/bGjRtL+w8dOtTzZ2OwMLIDSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKe+JGZPq3M\n7t/KZpDZs2eX9r/00ksd+8bHy39T5IYbbijt//rrr0v7MXgiYsq7VBnZgSQIO5AEYQeSIOxAEoQd\nSIKwA0kQdiAJzrMDMwzn2YHkCDuQBGEHkiDsQBKEHUiCsANJEHYgia5ht3267Rdtv2V7t+11Rfsd\ntvfb3ln89f7j5QAa1/WiGtsLJS2MiNdsnyjpVUnXamI+9i8j4s/TXhkX1QCN63RRzXTmZx+TNFY8\n/8L225JOrbc8AE07pu/sthdJWirp5aJpre03bD9oe26HZYZtj9oerVQpgEqmfW287RMkvSTpTxHx\npO0Fkj6VFJL+qIld/d92+Qx244GGddqNn1bYbc+W9Iyk5yPinin6F0l6JiLO7/I5hB1oWM83wti2\npAckvT056MWBuyNWStpVtUgAzZnO0fhlkv4p6U1Jh4vm2yStkrREE7vxeyXdVBzMK/ssRnagYZV2\n4+tC2IHmcT87kBxhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJEHYg\nia4/OFmzTyV9OOn1SUXbIBrU2ga1LonaelVnbWd06ujr/ew/WLk9GhFDrRVQYlBrG9S6JGrrVb9q\nYzceSIKwA0m0HfaRltdfZlBrG9S6JGrrVV9qa/U7O4D+aXtkB9AnhB1IopWw277a9ru237d9axs1\ndGJ7r+03i2moW52frphDb9z2rklt82xvtf1e8TjlHHst1TYQ03iXTDPe6rZre/rzvn9ntz1L0h5J\nV0raJ2mHpFUR8VZfC+nA9l5JQxHR+gUYtn8p6UtJfz0ytZbtuyQdjIg7i/8o50bE7wektjt0jNN4\nN1Rbp2nG16jFbVfn9Oe9aGNkv0jS+xHxQUR8K+lRSStaqGPgRcR2SQePal4haX3xfL0m/rH0XYfa\nBkJEjEXEa8XzLyQdmWa81W1XUldftBH2UyV9NOn1Pg3WfO8h6QXbr9oebruYKSyYNM3Wx5IWtFnM\nFLpO491PR00zPjDbrpfpz6viAN0PLYuIX0j6taSbi93VgRQT38EG6dzpfZLO0sQcgGOS7m6zmGKa\n8Sck3RIRn0/ua3PbTVFXX7ZbG2HfL+n0Sa9PK9oGQkTsLx7HJW3SxNeOQXLgyAy6xeN4y/X8X0Qc\niIhDEXFY0v1qcdsV04w/IenhiHiyaG59201VV7+2Wxth3yFpse0zbR8n6XpJm1uo4wdszykOnMj2\nHElXafCmot4saXXxfLWkp1qs5XsGZRrvTtOMq+Vt1/r05xHR9z9JyzVxRP4/kv7QRg0d6vq5pH8X\nf7vbrk3SI5rYrfuvJo5t3CjpZ5K2SXpP0j8kzRug2jZoYmrvNzQRrIUt1bZME7vob0jaWfwtb3vb\nldTVl+3G5bJAEhygA5Ig7EAShB1IgrADSRB2IAnCDiRB2IEk/gciQMnFg+KOfAAAAABJRU5ErkJg\ngg==\n","text/plain":["<Figure size 432x288 with 1 Axes>"]},"metadata":{"tags":[]}},{"output_type":"stream","text":["INFO:tensorflow:Error reported to Coordinator: <class 'tensorflow.python.framework.errors_impl.CancelledError'>, Enqueue operation was cancelled\n","\t [[{{node input/input_producer/input_producer_EnqueueMany}}]]\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"id":"npPWE9WkpDlk","colab_type":"code","colab":{}},"source":[""],"execution_count":0,"outputs":[]}]}