Tensors.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "id": "vXIIduk-mzK0"
   },
   "outputs": [],
   "source": [
    "import torch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "jQKqEFiUnDxr"
   },
   "source": [
    "## Tensors\n",
    "At its core, PyTorch is a library for processing tensors. A tensor is a number, vector, matrix, or any n-dimensional array. Let's create a tensor with a single number.\n",
    " > **Note** that it's not possible to create tensors with an improper shape.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "WLo_W_6M5PAx"
   },
   "source": [
    "**CLASS torch.Tensor**\n",
    "\n",
    "There are a few main ways to create a tensor, depending on your use case.\n",
    "\n",
    "  - To create a tensor with pre-existing data, use `torch.tensor()`.\n",
    "\n",
    "  - To create a tensor with specific size, use `torch.*` tensor creation ops (see Creation Ops).\n",
    "\n",
    "  - To create a tensor with the same size (and similar types) as another tensor, use `torch.*_like tensor` creation ops (see Creation Ops).\n",
    "\n",
    "  - To create a tensor with similar type but different size as another tensor, use `tensor.new_*` creation ops."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 214
    },
    "id": "ff0K0Lp-nG1K",
    "outputId": "d3b8f990-f06a-457a-d332-fd102f7064d8"
   },
   "outputs": [
    {
     "ename": "ValueError",
     "evalue": "ignored",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-2-a7e8901e46f9>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      3\u001b[0m t5 = torch.tensor([[5., 6, 11], \n\u001b[1;32m      4\u001b[0m                    \u001b[0;34m[\u001b[0m\u001b[0;36m7\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m8\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 5\u001b[0;31m                    [9, 10]])\n\u001b[0m\u001b[1;32m      6\u001b[0m \u001b[0mts\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mValueError\u001b[0m: expected sequence of length 3 at dim 1 (got 2)"
     ]
    }
   ],
   "source": [
    "# Number\n",
    "t1 = torch.tensor(4.)\n",
    "t5 = torch.tensor([[5., 6, 11], \n",
    "                   [7, 8], \n",
    "                   [9, 10]])\n",
    "ts"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "uOk6Sbwdo8JD"
   },
   "source": [
    "*We've created three tensors: `x`, `w`, and `b`, all numbers. w and b have an additional parameter requires_grad set to True. We'll see what it does in just a moment.*\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "agba6UyWn6ST",
    "outputId": "c3cb5def-4f30-49ed-9940-723ad6d918e4"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor(17., grad_fn=<AddBackward0>)"
      ]
     },
     "execution_count": 3,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create tensors.\n",
    "x = torch.tensor(3.)\n",
    "w = torch.tensor(4., requires_grad=True)\n",
    "b = torch.tensor(5., requires_grad=True)\n",
    "x * w + b"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> **`requires_grad` tells torch that x value will be used to evaluate functions**\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "1L7AWVAyn_HQ",
    "outputId": "6c9c4764-5c53-428c-a2d9-f22fcf626e0b"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor(17., grad_fn=<AddBackward0>)"
      ]
     },
     "execution_count": 4,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y = w * x + b\n",
    "y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Ik1-IvT9phJZ"
   },
   "source": [
    "  As expected, `y` is a tensor with the value 3 * 4 + 5 = 17. What makes PyTorch unique is that we can automatically **compute the derivative of y** w.r.t. the tensors that have requires_grad set to True i.e. w and b. This feature of PyTorch is called **autograd** (automatic gradients).\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "_Vy3zNNapruf",
    "outputId": "4d20d0b4-4f51-45cc-b91a-1c3b1cb190d3"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor(17., grad_fn=<AddBackward0>)"
      ]
     },
     "execution_count": 5,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# To compute the derivatives, we can invoke the .backward method on our result y\n",
    "# Backward function Calculates the derivative of y wrt. to x\n",
    "y.backward()\n",
    "y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "MxgIH61Xw4lc"
   },
   "source": [
    "*The derivatives of y with respect to the input tensors are stored in the `.grad` property of the respective tensors.*\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "sHUS50v0rYnq",
    "outputId": "1463b529-ce5f-42eb-ea62-4b8c0bca6a57"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "dy/dx: None\n",
      "dy/dw: tensor(3.)\n",
      "dy/db: tensor(1.)\n"
     ]
    }
   ],
   "source": [
    "# Display gradients\n",
    "# Backward function Calculates the derivative of y wrt. to x\n",
    "print('dy/dx:', x.grad)\n",
    "print('dy/dw:', w.grad)\n",
    "print('dy/db:', b.grad)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(6., dtype=torch.float64)\n"
     ]
    }
   ],
   "source": [
    "x = torch.tensor(2, requires_grad=True, dtype=float)\n",
    "z = x**2 + 2*x + 1 # ----->  2x + 2\n",
    "z.backward()\n",
    "print(x.grad)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(4., dtype=torch.float64)\n",
      "tensor(1., dtype=torch.float64)\n"
     ]
    }
   ],
   "source": [
    "\n",
    "#Partial Derivatives\n",
    "u = torch.tensor(1, requires_grad=True, dtype=float)\n",
    "v = torch.tensor(2, requires_grad=True, dtype=float)\n",
    "f = u*v + u**2\n",
    "f.backward()\n",
    "print(u.grad)\n",
    "print(v.grad)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "JdvvOGeAwz3C"
   },
   "source": [
    "Instead of reinventing the wheel, PyTorch interoperates well with Numpy to leverage its existing ecosystem of tools and libraries.\n",
    "\n",
    "We can convert a Numpy array to a PyTorch tensor using `torch.from_numpy`.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "60G7h_JTxCep",
    "outputId": "e9c89cd8-0025-4dd6-d204-5665144b45e7"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([[1., 2.],\n",
       "        [3., 4.]]), tensor([[1., 2.],\n",
       "         [3., 4.]], dtype=torch.float64))"
      ]
     },
     "execution_count": 7,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "x = np.array([[1, 2], [3, 4.]])\n",
    "y = torch.from_numpy(x)\n",
    "x, y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "THvE3m9_xRLn"
   },
   "source": [
    "We can convert a PyTorch tensor to a Numpy array using the `.numpy` method of a tensor.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "v6kdWCWbxLi0",
    "outputId": "26736337-d593-4b2c-e172-1340c939e145"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1., 2.],\n",
       "       [3., 4.]])"
      ]
     },
     "execution_count": 8,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Convert a torch tensor to a numpy array\n",
    "z = y.numpy()\n",
    "z"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "9x34EmPcyB2w"
   },
   "source": [
    "Use `torch.Tensor.item()` to get a Python number from a tensor containing a single value:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "2PyPQCvPyOuL",
    "outputId": "ccc4e2b5-7f08-49cc-e689-e52e0b4d7c1c"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1"
      ]
     },
     "execution_count": 12,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = torch.tensor([[1]])\n",
    "x.item()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "upNF0cytySo5"
   },
   "source": [
    "> **Note** : `.item()` only one element tensors can be converted to Python its scalars"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 163
    },
    "id": "Rs0s89kXxXnJ",
    "outputId": "2340c1d3-95db-429d-c64e-feac7ff8ae0d"
   },
   "outputs": [
    {
     "ename": "ValueError",
     "evalue": "ignored",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-11-90962570c418>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0my\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mitem\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mValueError\u001b[0m: only one element tensors can be converted to Python scalars"
     ]
    }
   ],
   "source": [
    "y.item()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "jf6VVgk24yYp"
   },
   "source": [
    "A tensor can be created with `requires_grad=True` so that torch.autograd records operations on them for automatic differentiation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 357
    },
    "id": "T0sZHmjXyRPc",
    "outputId": "84a73326-f448-433b-880b-0cf816fb6d93"
   },
   "outputs": [
    {
     "ename": "RuntimeError",
     "evalue": "ignored",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mRuntimeError\u001b[0m                              Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-14-9dd06b250e00>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0mx\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtorch\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtensor\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1.\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m-\u001b[0m\u001b[0;36m1.\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;36m1.\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m1.\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;31m#, requires_grad=True)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      2\u001b[0m \u001b[0mout\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpow\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m2\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msum\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 3\u001b[0;31m \u001b[0mout\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mbackward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      4\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgrad\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/torch/tensor.py\u001b[0m in \u001b[0;36mbackward\u001b[0;34m(self, gradient, retain_graph, create_graph)\u001b[0m\n\u001b[1;32m    219\u001b[0m                 \u001b[0mretain_graph\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mretain_graph\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    220\u001b[0m                 create_graph=create_graph)\n\u001b[0;32m--> 221\u001b[0;31m         \u001b[0mtorch\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mautograd\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mbackward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mgradient\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mretain_graph\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcreate_graph\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    222\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    223\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0mregister_hook\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mhook\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/torch/autograd/__init__.py\u001b[0m in \u001b[0;36mbackward\u001b[0;34m(tensors, grad_tensors, retain_graph, create_graph, grad_variables)\u001b[0m\n\u001b[1;32m    130\u001b[0m     Variable._execution_engine.run_backward(\n\u001b[1;32m    131\u001b[0m         \u001b[0mtensors\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mgrad_tensors_\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mretain_graph\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcreate_graph\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 132\u001b[0;31m         allow_unreachable=True)  # allow_unreachable flag\n\u001b[0m\u001b[1;32m    133\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    134\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mRuntimeError\u001b[0m: element 0 of tensors does not require grad and does not have a grad_fn"
     ]
    }
   ],
   "source": [
    "x = torch.tensor([[1., -1.], [1., 1.]]) , #requires_grad=True)\n",
    "out = x.pow(2).sum()\n",
    "out.backward()\n",
    "x.grad"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "wKHD4-VQ5re6"
   },
   "source": [
    "> `new_zeros(size, dtype=None, device=None, requires_grad=False) → Tensor`\n",
    "\n",
    "   Returns a Tensor of size size filled with 0. By default, the returned Tensor has the same torch.dtype and torch.device as this tensor\n",
    "\n",
    "**Parameters**\n",
    "  \n",
    "- size (int...) – a list, tuple, or torch.Size of integers defining the shape of the output tensor.\n",
    "\n",
    "- dtype (torch.dtype, optional) – the desired type of returned tensor. Default: if None, same torch.dtype as this tensor.\n",
    "\n",
    "- device (torch.device, optional) – the desired device of returned tensor. Default: if None, same torch.device as this tensor.\n",
    "\n",
    "- requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "e-ToVXF343Es",
    "outputId": "e4070635-0a3c-4fe8-aa4d-32f0cc1eb0db"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0., 0., 0.],\n",
       "        [0., 0., 0.]], dtype=torch.float64)"
      ]
     },
     "execution_count": 15,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tensor = torch.tensor((), dtype=torch.float64)\n",
    "tensor.new_zeros((2, 3))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "lnmcboDn6hka",
    "outputId": "6f843332-7157-4641-b216-0618e0d8af3f"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([ 0.0282-0.5626j, -0.3077+0.1036j,  0.3809-1.4568j, -0.2048-1.1163j])"
      ]
     },
     "execution_count": 16,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X = torch.randn(4, dtype=torch.cfloat)\n",
    "X"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "7wwuglKM6bgk"
   },
   "source": [
    "real() is only supported for tensors with complex dtypes.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "c6_H5fMg7Agg"
   },
   "source": [
    "**Referances**\n",
    "\n",
    "  - [PyTorch docs about Tensor](https://pytorch.org/docs/stable/tensors.html)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1_D Tensors exercise"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The dtype of tensor object after converting it to tensor:  torch.int64\n",
      "The type of tensor object after converting it to tensor:  torch.LongTensor\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\Dana\\Anaconda3\\lib\\site-packages\\ipykernel_launcher.py:57: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The type of the new_float_tensor: torch.FloatTensor\n",
      "The type of the new_float_tensor: torch.FloatTensor\n",
      "The size of the new_float_tensor:  torch.Size([5])\n",
      "The dimension of the new_float_tensor:  1\n",
      "Original Size:  tensor([0., 1., 2., 3., 4.])\n",
      "Size after view method tensor([[0.],\n",
      "        [1.],\n",
      "        [2.],\n",
      "        [3.],\n",
      "        [4.]])\n",
      "Size after view method torch.Size([5, 1])\n",
      "Size after view method 2\n",
      "Original Size:  tensor([0., 1., 2., 3., 4.])\n",
      "Size after view method tensor([[0.],\n",
      "        [1.],\n",
      "        [2.],\n",
      "        [3.],\n",
      "        [4.]])\n",
      "The dtype of new tensor:  torch.float64\n",
      "The type of new tensor:  torch.DoubleTensor\n",
      "The numpy array from tensor:  [0. 1. 2. 3. 4.]\n",
      "The dtype of numpy array:  float64\n",
      "The new tensor points to numpy_array :  tensor([0., 0., 0., 0., 0.], dtype=torch.float64)\n",
      "and back to numpy array points to the tensor:  [0. 0. 0. 0. 0.]\n",
      "Original Size:  tensor([1, 2, 3, 4, 5])\n",
      "Size after view method tensor([[1, 2, 3, 4, 5]])\n",
      "Max value from tensor x =  tensor(1.5708)\n",
      "Min value from tensor x =  tensor(0.)\n",
      "Dot product of tensors u and v =  0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\Dana\\Anaconda3\\lib\\site-packages\\ipykernel_launcher.py:30: MatplotlibDeprecationWarning: Adding an axes using the same arguments as a previous axes currently reuses the earlier instance.  In a future version, a new instance will always be created and returned.  Meanwhile, this warning can be suppressed, and the future behavior ensured, by passing a unique label to each axes instance.\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAD8CAYAAAB3u9PLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAfrUlEQVR4nO3deXgV5fn/8fedBcK+hTWAgFIREUEjixaldfkJKmCFita68qVabbVaK9bdXv7UVu3mitSqrV/QWlBUFEFFahUkILLLVpAQIAHZ9yT3948ZNKYnJHAm5yTh87quc2XmnCfnuRmSfM7MPPOMuTsiIiIpyS5ARESqBgWCiIgACgQREQkpEEREBFAgiIhISIEgIiJABIFgZu3M7AMzW2xmC83sxhhtzMz+ZGbLzWyemZ0Ub78iIhKttAjeoxC4xd3nmFkDYLaZTXH3RSXaDAA6h4/ewFPhVxERqSLi3kNw93XuPidc3g4sBrJKNRsMvOiBGUBjM2sdb98iIhKdKPYQvmZmHYCewMxSL2UBa0qs54bPrYvxHiOBkQD16tU7uUuXLlGWKCJSo82ePXujuzc/nO+NLBDMrD7wT+Amd99W+uUY3xJzzgx3Hw2MBsjOzvacnJyoShQRqfHMbPXhfm8ko4zMLJ0gDF5y9/ExmuQC7UqstwXyouhbRESiEcUoIwP+Aix298fKaDYRuDwcbdQH2Oru/3W4SEREkieKQ0anAT8G5pvZ3PC5XwPtAdz9aWASMBBYDuwCroqgXxERiVDcgeDuHxH7HEHJNg5cH29fIiJSeXSlsoiIAAoEEREJKRBERARQIIiISEiBICIigAJBRERCCgQREQEUCCIiElIgiIgIoEAQEZGQAkFERAAFgoiIhBQIIiICKBBERCSkQBAREUCBICIiIQWCiIgACgQREQlFEghm9pyZ5ZvZgjJe729mW81sbvi4O4p+RUQkOnHfUzn0PPA48OJB2vzL3c+PqD8REYlYJHsI7j4d+CqK9xIRkeRI5DmEvmb2uZm9bWbHJ7BfERGpgKgOGZVnDnCUu+8ws4HAa0DnWA3NbCQwEqB9+/YJKk9ERBKyh+Du29x9R7g8CUg3s8wy2o5292x3z27evHkiyhMRERIUCGbWyswsXO4V9rspEX2LiEjFRHLIyMzGAv2BTDPLBe4B0gHc/WlgKHCdmRUCu4Hh7u5R9C0iItGIJBDc/ZJyXn+cYFiqiIhUUbpSWUREAAWCiIiEFAgiIgIoEEREJKRAEBERQIEgIiIhBYKIiAAKBBERCSkQREQEUCCIiEhIgSAiIoACQUpZtWoV3bp1+3r9kUce4d57701eQSLAbbfdxpNPPvn1+r333sujjz6axIpqJgWCiFR5w4cP5+WXX/56/ZVXXmHYsGFJrKhmStQd00REDlvPnj3Jz88nLy+PgoICmjRpojsqVgIFgnxLWloaxcXFX6/v2bMnidWIfGPo0KG8+uqrrF+/nuHDhye7nBpJh4zkW1q2bEl+fj6bNm1i7969vPnmm8kuSQQIDhuNGzeOV199laFDhya7nBpJewjyLenp6dx999307t2bjh070qVLl2SXJALA8ccfz/bt28nKyqJ169bJLqdGsqp8J8vs7GzPyclJdhnVy9/+Bp07Q+/eENzGWqRacYePPoK1a0FHhg6dmc129+zD+V4dMqpJ3GHMGOjbFzp0gEcfhY0bk12VSIVs2AAPPQRt28Lpp8NLLyW7oiNPJIFgZs+ZWb6ZLSjjdTOzP5nZcjObZ2YnRdGvlGIGb70F3bpBbi7cdVfw23X++TBlCpQ4WSxSFRQVwaRJcM45cNRRcN99QTD06gX/+EeyqzvyRHUO4XngceDFMl4fAHQOH72Bp8KvErX69WHaNDj55CAUioqCkPjgA2jSBNas0aEkqRL274c2bWD3bti5M3guPT044jllCmRkJLe+8nhxMUVFhRTu30d6rQxS06r/KdlI/gXuPt3MOhykyWDgRQ9OWMwws8Zm1trd10XRv5TSrBn8+99w0klQUBAcStq1KwgJkSrCLAiEefOC9ZQUaNkSpk+Hhg0TV8fO7VvYtG4V2/K/ZPemXAq3rMV2bCB131bS92+n9v5tZBTtoG7xDjLYQ5oXkU4haRSRZk4asPzCtzjmxO8mruhKkqhIywLWlFjPDZ/7r0Aws5HASEAXnsQjKys4M3fKKcFv2Zdfwr/+BUcfDUuWQK1aya5QjmC7dwdhsGULtGgBderAjh3B55jmzaPvb9/ePeQu+5zNXy5if/4yUjevoMHO1bQszKUJ26lXqv0Or8O2lAbsTqnPntT6bKnTjoL0hnhaBp5aC0+pBalpkFoLUtLo3Lxd9EUnQaICIdYxipjDm9x9NDAaglFGlVlUjde5M6xbF+yHp6XBG2/AoEFQu3ZwgFZjuSUJnn8erroqWH7/ffje94LDR0VF0Rwm2rFtM18unMG2/8wmNX8BTbd/QbvC1XSyoq/b5NOUglrtWNr0exQ3ak96k7ZkNGtLoxbtadrqKOo3aEz9+EupdhIVCLlAyQhtC+QlqO8jW5063yxfcEHwm9ejBwwbFgTDpk1Qr/TnI5HobdkSnMYC6NcvONWVEg5rSU8PHofKi4tZ9+Uy8uZ9QNGXM8nc/BkdClfR1YLPkptoxNqMzsxu8V3Ss06gSftutOrYlRYNGtMimn9WjZKoQJgI3GBm4whOJm/V+YMkSUuDBQuCw0n9+gUnoZ96Cq69NtmVSQ3229/CbbcFy3PmQM+eh/9eBXmrWDVrEqz8gKO25tCGr2gD7PQMVmYcx6dtrqFux95kdelNZpujaBbJv+DIEMmFaWY2FugPZAIbgHuAdAB3f9rMjGAU0rnALuAqdy/3ijNdmFbJiothwAB4991gfePG4IS0SETWrQvOFQBcfDGMHXvog9z27tnF0pnvsHPhO7Ta+Akdir8EYDMNWVn/JArb9SXzuNPp0LVXjRjpE694LkzTlcoC8+dD9+7B8n33wd13J7ceqRFuvhl+//tgedkyOOaYin/v1k0bWPbv8aQsfZvvbP+U+rabPZ7OsowT2Nm2H5nd/x+duvUhJTW1coqvxhQIEj/34EzfCy8E62vWBBe1iRyi5cuD8QwAN930TSiU56v8tSyb9r/UX/4Gx+6dT5oVs5HGrGh6OhndzuPYPueTUfdIPNV7aBQIEp3//Ac6dQqWr7sOStylSuRg3OHSS2HcuGA9Lw/Km4Nu6+aNfDHtf8n44jW67v6MNCtmdUpb8lqfTbOTh3DMif20F3CI4gkEHXCTb+vYMfjNvv32YGKZp56CxYtBs57KQXz2WXAdJAQ/NgdOIMeyf99eFk4fj3/2d47fMYNeVshaa8mstpfTsu+ldOx6CkelaJq1ZNAegpQtPz+4qA1gyBAYP17TXsi3FBcH1xFMnx6sb94MjRvHbrtqcQ7rp43hmA1vk8kWNtGIZS0H0LT3pXTu0Q9TCERCewhSOVq0CPYW/vAH+MUvgkHjM2cGM4/JEW/atCAMAP7yF7j66v9us3vndhZMfo5Gi/7OdwqXkuWpLKjXh9yel3H8GRfRp1bthNYsB6c9BKmYbdugUaNguVcv+Phj0LHdI9L+/dC1a3DyuFGjYGhpyesfAb5cOpe8qU/SNf9NGrKT1SntWHf0D+l81tU0a6nBCpVJewhS+Ro2DPYWXnoJLrssuMBt8uRg3mI5YowfDxddFCy//nowE8oBRYWFzHt/HOmzn6Xb3rm09lTmNTydjL4j6drnXJ0XqAa0hyCHbs+e4AY8GzYEQ1OXLw+mwZAaa9euYNK5XbuC8QXz5wefCSCYLXTBW0+R9cULtPV1rCeT/3QYRudzf0pmK01QmWi6Y5okVkYGrF8f3NkkNzdYPzDWUGqcZ58NprvatQs+/DAYdJaWBhtyV/DJMz+j6NGu9F7yEDtTGzK71x/IvGMxfa98SGFQDemQkRy+AQOgsDCYYvuSS4JB6Fu3QoMGya5MIvDVV9/MZHLmmcEMJykpwfmB9ZMepufmyWRSzOcN+lH3jBvpcspZyS1Y4qY9BIlPamowW9knnwTnGBo2hCeeSHZVEqcHHvgmDD7/HKZOhZXzP2LOIxfQ9qX+dN88hTnNh7Dhyhmc9Ms3FAY1hM4hSHTcgym233orWC8ogMzM5NYkh2Tt2m9mLPnxj4OZTBbPfIfCab+l+57ZbKMuC7N+yHcG3arRQlWUziFI1WAGb74JCxcG682bw113JbcmqbCf/eybMFixAm6/fjILHzqDru9cTNae5XzS6edw0wL6/s8fFQY1lPYQpHK4w8iRMGZMsL56NeiWqFXS0qVw7LHB8q23wtVDp7D/vQc4Ye9nwdXEna/hxCE3U6eezg1VB9pDkKrHLBiesnp1sH7UUTBiRBAUUiW4B3dRPRAGsz74kMuafp8uk4aStXcFM475BXVvXUifH92jMDhCKBCkcrVvH/zlufPOYH6DlBRYtCjZVR3xcnKC/4p//hN+///nMPu3F5A9bRDt9ixlxtE3kvHLBfS57F4FwRFGgSCJ8ZvfBCeZAY4/Hs47T3sLSVBUBH37BiOFO7RZxoxHL+Fne77PsTs+5ZN2/0PaL+bR58f3U7d+o2SXKkmgQJDEycwMQuDxx4OL2lJSguGqkhBTpwYXlC1bsoE3f3MtX4zoQ4+tU5jV6mL2Xj+Hvtc8QoNGTZNdpiRRJIFgZuea2RdmttzMRsV4/UozKzCzueFjRBT9SjV1/fWwfXsQCKeeCj16BB9dpVLs2xfMNHLeebu457J7WXljTwbsH8fcJuewecQM+lz3DE1bZCW7TKkC4g4EM0sFngAGAF2BS8ysa4ymL7t7j/AxJt5+pZqrXz8IgXHjgiuf0tLg7beTXVWN849/QEbtIvofO5qVo07k3qN/z6qMrqz+4WR63TSWVu0O4UbHUuNFMXVFL2C5u68EMLNxwGBAZw6lfBdfHNx855hjYODA4B4Mq1cH8yPJYduxA5o2hVO7TWHO7b+mR62lrEjtyPwznqf76RcmuzypoqI4ZJQFrCmxnhs+V9pFZjbPzF41s3ZlvZmZjTSzHDPLKThwElJqttq1Yc2aYDrt/Pxgcv2//z3ZVVVbTz4JJxy7jLHX/oBpg4bSptZGZvV4gA6353CCwkAOIopAiHVPxdLDR94AOrh7d2Aq8EJZb+buo909292zmzdvHkF5Um2cc04wWV7v3sG8CWbBjXmkQjZtgvr1trJ9xi0sGnEq5zWZzoy211D/l3M5ZcgNpKZpLks5uCgCIRco+Ym/LZBXsoG7b3L3veHqs8DJEfQrNVFqKsyYAZ9+Gqw3ahTcwlMO6p57ivnlj55h2c09uK3TGBY1/C5brvmEPiMe0xBSqbAoAmEW0NnMOppZLWA4MLFkAzNrXWJ1ELA4gn6lJjvllOAO7kOGBPdzNgsOJ8m3rFkD2cfN4Nw9/fhrn1+xo1ZTlgx8lZNvmUCr9p2TXZ5UM3HvQ7p7oZndAEwGUoHn3H2hmd0P5Lj7RODnZjYIKAS+Aq6Mt185ApjBhAmwZAkcdxy0bAm33QYPPZTsypLOHX5+/Qa6F41i5g8nsNUa8OkJ95I95Oek6F7Xcpg0uZ1UHz/9KTz1VLC8ciV07JjcepJk4cIinv71H7nvxMdoZDvIaf4Dulz6MI2a6pybaHI7OVI8+WRwjASgUye44oojavoLd7j28o/Y87e+/Lnnfayv1Z5VQ9+h9w3PKQwkEgoEqV7atg3+Mt5/P7z4YnC18/z5ya6q0k1/fyPP/uRqnuh4PkfVXs+sng9y3K8/4ugT+iS7NKlBFAhSPd11VzDOEqB7dzj77OAkdA1TuL+Y2y5/mi7TTmZE6/HMbHYh6TfN4ZTBP8VS9Osr0dLAZKm+mjYN9haeeQauvTYYsvqvf8F3v5vsyiIxYexcMufcyMOd5rKI77B1yN85tUe/ZJclNZg+Ykj195OfBHM11K4N/foF02sXFia7qsO2bctOHhvxSwYuOZPudZfySZc7OfbOT+isMJBKpkCQmqFePdizJ5jNbdEiSE+HN95IdlWH7G9/nsjGR7K5ue2zfJren/3XzaTv8Ft1lbEkhH7KpGYZOhT27g2uWxg0CJo0gbVrg/mRqrA1/1nLx4/exI8z32WVtWZe/+fp11/zDkliaQ9Bap5atWDFCnjvPdi8GerWheefT3ZVMXlxMX994HHq/bU3FzZ7j3cbXU2r23PorjCQJNCFaVKzFRfDGWfARx8F61u2BPMjVQELZy/gq3HX06/eXGbv70qzHz1Oh66a5kviowvTRMqSkhKMPDrwwaJxY/jd75Ja0v59e3nxzjs4emJ/Tqz7BVPa3knP+z9SGEjSKRDkyHDyycHewrBh8KtfBfMkrV+f8DI+nvIhS+/txeVpj/NxYV/2jJzB2SNu1fxDUiUoEOTIYQavvAJLlwbrrVvDLbckpOud27cy7rZr6f3RYJqlbuPDbk/w/QfeoEVWh4T0L1IRCgQ58nTuHE4X+nN47LEgKFasqLTuprwynk0PZzO8zlgmFQ+m7i05nDH0skrrT+RwKRDkyPXHPwZDUiG4p/Oll0Y6Wd7mgvVM/NUPOXvRVezzdHL6vcwFv3mBho2bRdaHSJQUCHJka9MmCIEHH4SxY4OT0HPnxvWWXlzMG8+OoejPpzCgzlTGp15N2ztnkX3muREVLVI5FAgiAKNGwVdfBcs9e0L//oc1WV7e6hW8P2ogF6y9hTX7W7Hsgrf5wV2/J6NOvWjrFakECgSRA5o0CfYWxoyBDz8MJsv78MMKfWtxURHj//AI9Z87jVPrzGZ83Zvp/ptP6Jrdu5KLFomOLkwTiWXXLmjVCrZvD05CL1wYzI8Uw4pF88l/4Tr61pnPv3f3IOvqJ+lw7PEJLlgkkPQL08zsXDP7wsyWm9moGK/XNrOXw9dnmlmHKPoVqTR168K2bcE9nZctC6bDeO21bzUp3L+Pfz50N1kvf4+uGSt4PfM+Tn3wA4WBVFtxT25nZqnAE8DZQC4wy8wmuvuiEs2uATa7+zFmNhx4GLg43r5FKt2QIbBvH5xwAlx4YTCran4+8+fPo/i167mo9nKm7DmNE254ksHtOiS7WpG4RLGH0AtY7u4r3X0fMA4YXKrNYOCFcPlV4Ewzswj6Fql86emwZAlMm0bxzl1MvW4AXSYNpE16AZM6/IGzHnyTVgoDqQGiCIQsYE2J9dzwuZht3L0Q2ArEHIxtZiPNLMfMcgoKCiIoTyQiZ5xBSlEh9dqk8d7+75P2s08ZeOVVupWl1BhR3A8h1if90meqK9ImeNJ9NDAagpPK8ZUmErGUFE65/23S0msluxKRyEXx0SYXaFdivS2QV1YbM0sDGgFfRdC3SMIpDKSmiiIQZgGdzayjmdUChgMTS7WZCFwRLg8F3veqPN5VROQIFPchI3cvNLMbgMlAKvCcuy80s/uBHHefCPwF+JuZLSfYMxgeb78iIhKtSO6p7O6TgEmlnru7xPIeYFgUfYmISOXQ8AgREQEUCCIiElIgiIgIoEAQEZGQAkFERAAFgoiIhBQIIiICKBBERCSkQBAREUCBICIiIQWCiIgACgQREQkpEEREBFAgiIhISIEgIiKAAkFEREIKBBERARQIIiISiisQzKypmU0xs2Xh1yZltCsys7nhY2I8fYqISOWIdw9hFPCeu3cG3gvXY9nt7j3Cx6A4+xQRkUoQbyAMBl4Il18AhsT5fiIikiTxBkJLd18HEH5tUUa7DDPLMbMZZnbQ0DCzkWHbnIKCgjjLExGRikorr4GZTQVaxXjpjkPop72755lZJ+B9M5vv7itiNXT30cBogOzsbD+EPkREJA7lBoK7n1XWa2a2wcxau/s6M2sN5JfxHnnh15VmNg3oCcQMBBERSY54DxlNBK4Il68AXi/dwMyamFntcDkTOA1YFGe/IiISsXgD4SHgbDNbBpwdrmNm2WY2JmxzHJBjZp8DHwAPubsCQUSkiin3kNHBuPsm4MwYz+cAI8Llj4ET4ulHREQqn65UFhERQIEgIiIhBYKIiAAKBBERCSkQREQEUCCIiEhIgSAiIoACQUREQgoEEREBFAgiIhJSIIiICKBAEBGRkAJBREQABYKIiIQUCCIiAigQREQkpEAQERFAgSAiIqG4AsHMhpnZQjMrNrPsg7Q718y+MLPlZjYqnj5FRKRyxLuHsAD4ATC9rAZmlgo8AQwAugKXmFnXOPsVEZGIpcXzze6+GMDMDtasF7Dc3VeGbccBg4FF8fQtIiLRSsQ5hCxgTYn13PC5mMxspJnlmFlOQUFBpRcnIiKBcvcQzGwq0CrGS3e4++sV6CPW7oOX1djdRwOjAbKzs8tsJyIi0So3ENz9rDj7yAXalVhvC+TF+Z4iIhKxRBwymgV0NrOOZlYLGA5MTEC/IiJyCOIddnqhmeUCfYG3zGxy+HwbM5sE4O6FwA3AZGAx8Iq7L4yvbBERiVq8o4wmABNiPJ8HDCyxPgmYFE9fIiJSuXSlsoiIAAoEEREJKRBERARQIIiISEiBICIigAJBRERCCgQREQEUCCIiElIgiIgIoEAQEZGQAkFERAAFgoiIhBQIIiICKBBERCSkQBAREUCBICIiIQWCiIgACgQREQnFe0/lYWa20MyKzSz7IO1Wmdl8M5trZjnx9CkiIpUjrnsqAwuAHwDPVKDt99x9Y5z9iYhIJYkrENx9MYCZRVONiIgkTaLOITjwrpnNNrORCepTREQOQbl7CGY2FWgV46U73P31CvZzmrvnmVkLYIqZLXH36WX0NxIYCdC+ffsKvr2IiMSr3EBw97Pi7cTd88Kv+WY2AegFxAwEdx8NjAbIzs72ePsWEZGKqfRDRmZWz8waHFgGziE4GS0iIlVIvMNOLzSzXKAv8JaZTQ6fb2Nmk8JmLYGPzOxz4FPgLXd/J55+RUQkevGOMpoATIjxfB4wMFxeCZwYTz8iIlL5dKWyiIgACgQREQkpEEREBFAgiIhISIEgIiKAAkFEREIKBBERARQIIiISUiCIiAigQBARkZACQUREAAWCiIiEFAgiIgIoEEREJKRAEBERQIEgIiIhBYKIiAAKBBERCSkQREQEiDMQzOx3ZrbEzOaZ2QQza1xGu3PN7AszW25mo+LpU0REKke8ewhTgG7u3h1YCtxeuoGZpQJPAAOArsAlZtY1zn5FRCRicQWCu7/r7oXh6gygbYxmvYDl7r7S3fcB44DB8fQrIiLRS4vwva4GXo7xfBawpsR6LtC7rDcxs5HAyHB1r5ktiKzCypEJbEx2ERWgOqOlOqOlOqNz7OF+Y7mBYGZTgVYxXrrD3V8P29wBFAIvxXqLGM95Wf25+2hgdPi+Oe6eXV6NyVQdagTVGTXVGS3VGR0zyznc7y03ENz9rHI6vwI4HzjT3WP9oc8F2pVYbwvkHUqRIiJS+eIdZXQucBswyN13ldFsFtDZzDqaWS1gODAxnn5FRCR68Y4yehxoAEwxs7lm9jSAmbUxs0kA4UnnG4DJwGLgFXdfWMH3Hx1nfYlQHWoE1Rk11Rkt1Rmdw67RYh/lERGRI42uVBYREUCBICIioSoVCNVhKgwzG2ZmC82s2MzKHH5mZqvMbH54buWwh4EdrkOoM6nTiphZUzObYmbLwq9NymhXFG7LuWaWsEEJ5W0fM6ttZi+Hr880sw6Jqq1UHeXVeaWZFZTYhiOSUONzZpZf1rVFFvhT+G+YZ2YnJbrGsI7y6uxvZltLbMu7k1BjOzP7wMwWh7/nN8Zoc+jb092rzAM4B0gLlx8GHo7RJhVYAXQCagGfA10TWONxBBd+TAOyD9JuFZCZxG1Zbp3J3pZhDb8FRoXLo2L9n4ev7UjCNix3+wA/BZ4Ol4cDL1fROq8EHk90baVqOB04CVhQxusDgbcJrl3qA8ysonX2B95M8rZsDZwULjcgmDqo9P/5IW/PKrWH4NVgKgx3X+zuXySqv8NVwTqrwrQig4EXwuUXgCEJ7v9gKrJ9Stb/KnCmmcW6GLMyVYX/x3K5+3Tgq4M0GQy86IEZQGMza52Y6r5RgTqTzt3XufuccHk7wQjOrFLNDnl7VqlAKOVqgnQrLdZUGKU3RFXgwLtmNjucjqMqqgrbsqW7r4PghxxoUUa7DDPLMbMZZpao0KjI9vm6TfhhZivQLCHVxaghVNb/40XhoYNXzaxdjNeTrSr8PFZUXzP73MzeNrPjk1lIeJiyJzCz1EuHvD2jnMuoQhI9FcbhqEiNFXCau+eZWQuC6zSWhJ88IhNBnZW+LeHgdR7C27QPt2cn4H0zm+/uK6KpsEwV2T4J2YblqEgNbwBj3X2vmV1LsFfz/Uqv7NBUhW1ZEXOAo9x9h5kNBF4DOiejEDOrD/wTuMndt5V+Oca3HHR7JjwQvBpMhVFejRV8j7zwa76ZTSDYrY80ECKoMyHTihysTjPbYGat3X1duDubX8Z7HNieK81sGsEnosoOhIpsnwNtcs0sDWhE4g83lFunu28qsfoswTm6qqZaTHNT8g+vu08ysyfNLNPdEzrpnZmlE4TBS+4+PkaTQ96eVeqQkdWQqTDMrJ6ZNTiwTHCyvCrO2loVtuVE4Ipw+Qrgv/ZszKyJmdUOlzOB04BFCaitItunZP1DgffL+CBTmcqts9Sx40EEx5yrmonA5eHomD7A1gOHE6sSM2t14DyRmfUi+Du66eDfFXkNBvwFWOzuj5XR7NC3ZzLPlMc4c76c4JjX3PBxYPRGG2BSqbPnSwk+Id6R4BovJEjevcAGYHLpGglGe3wePhYmusaK1pnsbRn23wx4D1gWfm0aPp8NjAmXTwXmh9tzPnBNAuv7r+0D3E/woQUgA/hH+LP7KdAp0duwgnU+GP4sfg58AHRJQo1jgXXA/vBn8xrgWuDa8HUjuJnWivD/ucxRfEmu84YS23IGcGoSavwuweGfeSX+Xg6Md3tq6goREQGq2CEjERFJHgWCiIgACgQREQkpEEREBFAgiIhISIEgIiKAAkFEREL/B3sBOWF7jW7lAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "\n",
    "\"\"\"\n",
    "\n",
    "1-D Tensors\n",
    "Exercise\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "32-bit float -                          Float Tensor\n",
    "64-bit float -                          Double Tensor\n",
    "16-bit float -                          Half Tensor\n",
    "8-bit int (unsigned)(8-bit images)-     Byte Tensor     \n",
    "8-bit int (signed)-                     Char Tensor\n",
    "16-bit int (singed)-                    Short Tensor\n",
    "32-bit int (singed)-                    Int Tensor\n",
    "64-bit int (signed)-                    Long Tensor\n",
    "\n",
    "\n",
    "\"\"\"\n",
    "\n",
    "import torch \n",
    "import numpy as np \n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "\n",
    "#Lets define the function to plot vectors in coorindate system\n",
    "def plotVec(vectors):\n",
    "    ax = plt.axes()\n",
    "\n",
    "# Vectors = [{\"vector\": vector variable, \"name\": name of vector, \"color\": color of the vector on diagram}]    \n",
    "    #For loop for drawing vectors\n",
    "    for vec in vectors:\n",
    "        ax.arrow(0, 0, *vec[\"vector\"], head_width = 0.05, color = vec[\"color\"], head_length = 0.1)\n",
    "        plt.text(*(vec[\"vector\"] + 0.1), vec[\"name\"])\n",
    "    \n",
    "    plt.ylim(-2, 2) #Setting limits for y-axis\n",
    "    plt.xlim(-2, 2) #Setting limits for x-axis\n",
    "\n",
    "\n",
    "\n",
    "# Convert a integer list with length 5 to a tensor\n",
    "\n",
    "ints_to_tensor = torch.tensor([0, 1, 2, 3, 4])\n",
    "print(\"The dtype of tensor object after converting it to tensor: \", ints_to_tensor.dtype)\n",
    "print(\"The type of tensor object after converting it to tensor: \", ints_to_tensor.type())\n",
    "type(ints_to_tensor)\n",
    "\n",
    "\n",
    "#Convert float list to a float tensor 32bit\n",
    "list_floats = [0.0, 1.0, 2.0, 3.0, 4.0]\n",
    "list_floats = torch.tensor(list_floats)\n",
    "list_floats.type()\n",
    "\n",
    "#convert the float tensor to int64 Long Tensor\n",
    "floats_int_tensor=torch.tensor(list_floats,dtype=torch.int64)\n",
    "floats_int_tensor.type()\n",
    "\n",
    "\n",
    "# Convert a integer list with length 5 to float tensor\n",
    "new_float_tensor = torch.FloatTensor([0, 1, 2, 3, 4])\n",
    "new_float_tensor.type()\n",
    "print(\"The type of the new_float_tensor:\", new_float_tensor.type())\n",
    "new_float_tensor = torch.FloatTensor([0, 1, 2, 3, 4])\n",
    "\n",
    "# Another method to convert the integer list to float tensor\n",
    "old_int_tensor = torch.tensor([0, 1, 2, 3, 4])\n",
    "new_float_tensor = old_int_tensor.type(torch.FloatTensor)\n",
    "print(\"The type of the new_float_tensor:\", new_float_tensor.type())\n",
    "\n",
    "\n",
    "# Introduce the tensor_obj.size() & tensor_ndimension.size() methods\n",
    "print(\"The size of the new_float_tensor: \", new_float_tensor.size())\n",
    "print(\"The dimension of the new_float_tensor: \",new_float_tensor.ndimension())\n",
    "\n",
    "\n",
    "'''\n",
    "The tensor_obj.view(row, column) is used for reshaping a tensor object.\n",
    "\n",
    "What if you have a tensor object with torch.Size([5]) as a new_float_tensor as shown in the previous example?\n",
    "After you execute new_float_tensor.view(5, 1), the size of new_float_tensor will be torch.Size([5, 1]).\n",
    "This means that the tensor object new_float_tensor has been reshaped from \n",
    "a one-dimensional tensor object with 5 elements to a two-dimensional tensor object with 5 rows and 1 column.\n",
    "\n",
    "'''\n",
    "\n",
    "# Introduce the tensor_obj.view(row, column) method\n",
    "twoD_float_tensor = new_float_tensor.view(5, 1)\n",
    "print(\"Original Size: \", new_float_tensor)\n",
    "print(\"Size after view method\", twoD_float_tensor)\n",
    "print(\"Size after view method\", twoD_float_tensor.size()) #Tensor of size (5,1)\n",
    "print(\"Size after view method\", twoD_float_tensor.ndimension()) #Tensor dimension is now 2\n",
    "\n",
    "#What if you have a tensor with dynamic size but you want to reshape it? You can use -1 to do just that.\n",
    "# Introduce the use of -1 in tensor_obj.view(row, column) method\n",
    "twoD_float_tensor = new_float_tensor.view(-1, 1)\n",
    "print(\"Original Size: \", new_float_tensor)\n",
    "print(\"Size after view method\", twoD_float_tensor)\n",
    "\n",
    "# Convert a numpy array to a tensor\n",
    "numpy_array = np.array([0.0, 1.0, 2.0, 3.0, 4.0])\n",
    "new_tensor = torch.from_numpy(numpy_array)\n",
    "print(\"The dtype of new tensor: \", new_tensor.dtype)\n",
    "print(\"The type of new tensor: \", new_tensor.type())\n",
    "\n",
    "# Convert a tensor to a numpy array\n",
    "back_to_numpy = new_tensor.numpy()\n",
    "print(\"The numpy array from tensor: \", back_to_numpy)\n",
    "print(\"The dtype of numpy array: \", back_to_numpy.dtype)\n",
    "\n",
    "# Set all elements in numpy array to zero \n",
    "numpy_array[:] = 0\n",
    "print(\"The new tensor points to numpy_array : \", new_tensor)\n",
    "print(\"and back to numpy array points to the tensor: \", back_to_numpy)\n",
    "\n",
    "\n",
    "#Converting a tensor to (1,5) tensor\n",
    "your_tensor = torch.tensor([1, 2, 3, 4, 5])\n",
    "your_tensor.size()\n",
    "your_new_tensor = your_tensor.view(1, 5)\n",
    "your_new_tensor.ndimension()\n",
    "print(\"Original Size: \", your_tensor)\n",
    "print(\"Size after view method\", your_new_tensor)\n",
    "\n",
    "\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "#Constructing tensor with 25 steps in the range of 0 and pi/2.\n",
    "x = torch.linspace(0, np.pi/2, steps=25)\n",
    "y = torch.sin(x)\n",
    "\n",
    "\n",
    "plt.plot(x.numpy(), y.numpy()) #We need to convert tensors to numpy arrays\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "plt.plot(x.numpy(), y.numpy())\n",
    "\n",
    "print(\"Max value from tensor x = \", x.max())\n",
    "print(\"Min value from tensor x = \", x.min())\n",
    "\n",
    "#Convert the list [-1, 1] and [1, 1] to tensors u and v. \n",
    "#Plot the tensor u and v as a vector by using the function plotVec and find the dot product\n",
    "\n",
    "u = torch.tensor([-1, 1])\n",
    "v = torch.tensor([1, 1])\n",
    "z = torch.dot(u,v)\n",
    "print(\"Dot product of tensors u and v = \", np.dot(u,v))\n",
    "\n",
    "\n",
    "plotVec([\n",
    "    {\"vector\": u.numpy(), \"name\": 'u', \"color\": 'r'},\n",
    "    {\"vector\": v.numpy(), \"name\": 'v', \"color\": 'b'}\n",
    "])\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2_D Tensors exercise"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The New 2D Tensor:  tensor([[11, 12, 13],\n",
      "        [21, 22, 23],\n",
      "        [31, 32, 33]])\n",
      "Tensor -> Numpy Array:\n",
      "The numpy array after converting:  [[11 12 13]\n",
      " [21 22 23]\n",
      " [31 32 33]]\n",
      "Type after converting:  int64\n",
      "Numpy Array -> Tensor:\n",
      "The tensor after converting: tensor([[11, 12, 13],\n",
      "        [21, 22, 23],\n",
      "        [31, 32, 33]])\n",
      "Type after converting:  torch.int64\n",
      "Pandas Dataframe to numpy:  [[ 11  12]\n",
      " [ 21  22]\n",
      " [ 31 312]]\n",
      "Type BEFORE converting:  int64\n",
      "Tensor AFTER converting:  tensor([[ 11,  12],\n",
      "        [ 21,  22],\n",
      "        [ 31, 312]])\n",
      "Type AFTER converting:  torch.int64\n",
      "Result after tensor_example[1:3]:  tensor([[21, 22, 23],\n",
      "        [31, 32, 33]])\n",
      "Dimension after tensor_example[1:3]:  2\n",
      "Result after sliced_tensor_example[1]:  tensor([31, 32, 33])\n",
      "Dimension after sliced_tensor_example[1]:  1\n",
      "Result:  tensor([31, 32, 33])\n",
      "Dimension:  1\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "tensor([[11, 12, 13],\n",
       "        [21,  0, 23],\n",
       "        [31,  0, 33]])"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\n",
    "\"\"\"\n",
    "\n",
    "2-D Tensors\n",
    "Exercise\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "32-bit float -                          Float Tensor\n",
    "64-bit float -                          Double Tensor\n",
    "16-bit float -                          Half Tensor\n",
    "8-bit int (unsigned)(8-bit images)-     Byte Tensor     \n",
    "8-bit int (signed)-                     Char Tensor\n",
    "16-bit int (singed)-                    Short Tensor\n",
    "32-bit int (singed)-                    Int Tensor\n",
    "64-bit int (signed)-                    Long Tensor\n",
    "\n",
    "\n",
    "\"\"\"\n",
    "\n",
    "import numpy as np \n",
    "import matplotlib.pyplot as plt\n",
    "import torch\n",
    "import pandas as pd\n",
    "\n",
    "#Creating 2D tensor from a list\n",
    "twoD_list = [[11, 12, 13], [21, 22, 23], [31, 32, 33]]\n",
    "twoD_tensor = torch.tensor(twoD_list)\n",
    "print(\"The New 2D Tensor: \", twoD_tensor)\n",
    "\n",
    "#Converting tensor to numpy array and back to tensor\n",
    "twoD_numpy = twoD_tensor.numpy()\n",
    "print(\"Tensor -> Numpy Array:\")\n",
    "print(\"The numpy array after converting: \", twoD_numpy)\n",
    "print(\"Type after converting: \", twoD_numpy.dtype)\n",
    "\n",
    "new_twoD_tensor = torch.from_numpy(twoD_numpy)\n",
    "print(\"Numpy Array -> Tensor:\")\n",
    "print(\"The tensor after converting:\", new_twoD_tensor)\n",
    "print(\"Type after converting: \", new_twoD_tensor.dtype)\n",
    "\n",
    "#Lets convert panadas dataframe to a tenosr\n",
    "df = pd.DataFrame({'a':[11,21,31],'b':[12,22,312]})\n",
    "\n",
    "print(\"Pandas Dataframe to numpy: \", df.values)\n",
    "print(\"Type BEFORE converting: \", df.values.dtype)\n",
    "\n",
    "\n",
    "new_tensor = torch.from_numpy(df.values)\n",
    "print(\"Tensor AFTER converting: \", new_tensor)\n",
    "print(\"Type AFTER converting: \", new_tensor.dtype)\n",
    "\n",
    "#Lets convert a Pandas Series to a tensor\n",
    "df = pd.DataFrame({'A':[11, 33, 22],'B':[3, 3, 2]})\n",
    "pandas_to_numpy = df.values\n",
    "numpy_to_tensor = torch.tensor(pandas_to_numpy)\n",
    "numpy_to_tensor\n",
    "\n",
    "\n",
    "\n",
    "#Slicing rows\n",
    "tensor_example = torch.tensor([[11, 12, 13], [21, 22, 23], [31, 32, 33]])\n",
    "sliced_tensor_example = tensor_example[1:3] #Slicing 2nd and 3rd row\n",
    "print(\"Result after tensor_example[1:3]: \", sliced_tensor_example)\n",
    "print(\"Dimension after tensor_example[1:3]: \", sliced_tensor_example.ndimension())\n",
    "\n",
    "#Dimension of 2nd row of sliced tensor\n",
    "print(\"Result after sliced_tensor_example[1]: \", sliced_tensor_example[1])\n",
    "print(\"Dimension after sliced_tensor_example[1]: \", sliced_tensor_example[1].ndimension())\n",
    "\n",
    "#Lets try to get the values in row 2 and 3 in the second column. Note that the code below will not work.\n",
    "print(\"Result: \", tensor_example[1:3][1])\n",
    "tensor_example[1:3][1]\n",
    "print(\"Dimension: \", tensor_example[1:3][1].ndimension()) #This gives dimension of 1\n",
    "\n",
    "#In order to get the values in row 2 and 3 in the second column. we have to separate with a comma.\n",
    "tensor_example[1:3, 1]\n",
    "\n",
    "\n",
    "#Lets modify values in a 2D tensor.\n",
    "tensor_ques = torch.tensor([[11, 12, 13], [21, 22, 23], [31, 32, 33]])\n",
    "#We will modify the values in the second column of the second and 3rd row. \n",
    "tensor_ques[1:3, 1] = 0\n",
    "tensor_ques"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> [Torchvision.Transforms docs](https://pytorch.org/docs/stable/torchvision/transforms.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "C:\\Users\\Dana\\Desktop\\PyTorch\n",
      "File name: img/fashion0.png\n",
      "class or y: Ankle boot\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAEICAYAAACZA4KlAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAWuUlEQVR4nO3de5CcVZnH8e+PAGouBJIQcyES5LLguhjWEBHQQgUW0RW8BdFSEDSWJbVaK+Uie5EVt2R11dXdrLVRUBTFSwkLLshFVhc1KBkxJoGoEAwSEiZAQsgNcuHZP/qN2xnmPWfo7pluOL9P1dT09NPn7dPvzDPv2/285xxFBGb27LdHtztgZiPDyW5WCCe7WSGc7GaFcLKbFcLJblYIJ3uHSbpT0gkj8DxvlHS/pE2SjmpjOyHpkJrYOyTd1HovrZfIdfZnDklvB14fEW+XtAL464i4ps1tBnBoRNzTkU7WP88s4FLgCGA5cG5ELB7O57Td+cj+zHIqcH11+0Dgzi72Zcgk7Q1cA1wB7AdcDlxT3W8jxMneYZJWSjpR0kWSvivpCkkbJS2VdJikj0paW52Cn9zU7iBJt1aP/aGk+ZKuaIrvAZwE3CxpEzAK+HV1hEfSBZJWVO3vkvTGpraHSPpfSRskPSzp2wO6faKkuyWtr55XVbuzJf20aTvHSlpUbWeRpGObYj+WdLGkn1V9uEnSpCp8ArAn8K8R8UREfAEQ8OrEPvxo9TrWS/qKpOdWsRMkrZL04Wo/rpH07qa2EyV9X9JjVR8/0fwaSuZkH15/CXydxtHsV8CNNPb5dODjwH82PfabwO3AROAi4J0DtjUHuDci+iNibHXfSyLi4Or2CuAVwHjgH4ErJE2tYhcDN1X9OAD4twHbfj1wNPASYC7wFwNfiKQJwHXAF6o+fha4TtLEpoe9HXg3MBnYGzi/uv9PgSWx+3vGJdX9dd5R9eNg4DDg75piU6rXOR04F5gvab8qNh/YXD3mrOrLcLIPt59ExI0RsQP4LrA/cElEbAe+BcyUtK+kF9BItn+IiG0R8VPg2gHbeh3/fwr/FBHx3YhYHRFPRsS3gbtp/IMA2E7jtH9aRDxebb/ZJRHxaET8AfgRMGuQp3gdcHdEfD0idkTElcBvaPxD2+UrEfG7iNgKfKdpO2OBDQO2twEYV/d6gH+PiPsjYh3wT8CZTbHtwMcjYntEXA9sAv5E0ijgzcDHImJLRNxF4y2D4WQfbv1Nt7cCD0fEzqafoZEI04B1EbGl6fH3D9hW8/v1p5D0LkmLJT0q6VHgxcCu0+iP0Dhtvr2qFpwzoPmDTbe3VH0aaBpw34D77qNxdM1tZxOwz4C2+wAb614Pu7/++6rn3+WR6h/owOfan8bbhea2A/djsZzsvWENMEHS6Kb7Zuy6IWkKMBW4Y7DGkg4EvgScB0yMiH2BZTQSnIh4MCLeGxHTgPcB/1FXbktYTePsoNkLgAeG0PZO4MhdnwVUjiT9AeOMptsvqJ4/5yFgB423KoNtp2hO9h4QEfcBfcBFkvaW9HJ2Pz0+FbhhwHveZmOAoPHHTvWB1Yt3BSW9VdKuBFhfPXbnwI1kXA8cJuntkvaUdAbwIuC/h9D2x9Xz/ZWk50g6r7r/fxJtPiDpgOqzgguBgR8qPkV11nQVjf04WtLhwLuG0L8iONl7xzuAlwOPAJ+g8cf9RBVLnsJX700/A9xG463DnwE/a3rI0cAvqk/xrwU+GBG/fzqdi4hHaHyQ9+Gqjx+hUfN/eAhttwGn00i8R4FzgNOr+5F0oaQfDGj2TRofKt5bfX1iiF09j8aHdw/S+HD0Sv5/PxbNF9X0qKo89hsan6Q/CBwcEQM/5HpWkrQSeE9E/LAD2/pnYEpEFP+pvI/sPULS0ZIOlrSHpFOA04D/AiYAf19KordL0uGSjlTDHBqluau73a9esGe3O2B/NIXG+82JwCrg/RHxqyr2xa716plnHI1T92nAWhpvb9q6pPjZwqfxZoXwabxZIUb0NL4aYWVmwygiNNj9bR3ZJZ0i6beS7pF0QTvbMrPh1fJ79uo65N/RGIm1ClgEnFnVfOva+MhuNsyG48g+B7gnIu6tLo74Fo1ykZn1oHaSfTq7DzJYxe6DIgCQNE9Sn6S+Np7LzNrUzgd0g50qPOU0PSIWAAvAp/Fm3dTOkX0Vu48oOoChjUwysy5oJ9kXAYdW0yntDbyNp064YGY9ouXT+IjYUQ1VvJHGfGiXRcQzYgJEsxKN6OWyfs9uNvyG5aIaM3vmcLKbFcLJblYIJ7tZIZzsZoVwspsVwsluVggnu1khnOxmhXCymxXCyW5WCCe7WSGc7GaF8Iowz3K7r5L8VO2Oehw3blwyfvzxx9fGfvCDgWs5Pj251zZq1Kja2I4dO2pjIyHX95RWf2c+spsVwsluVggnu1khnOxmhXCymxXCyW5WCCe7WSFcZ3+W22OP9P/znTt3JuOHHHJIMv6e97wnGd+6dWttbPPmzcm2jz/+eDJ+++23J+Pt1NJzdfDcfs21b6dvqesHUr9PH9nNCuFkNyuEk92sEE52s0I42c0K4WQ3K4ST3awQrrM/y6VqspCvs7/61a9Oxk888cRkfNWqVbWx5zznOcm2o0ePTsZPOumkZPzLX/5ybay/vz/ZNjdmPLffcsaOHVsbe/LJJ5Ntt2zZ0tJztpXsklYCG4GdwI6ImN3O9sxs+HTiyP6qiHi4A9sxs2Hk9+xmhWg32QO4SdIvJc0b7AGS5knqk9TX5nOZWRvaPY0/LiJWS5oM3CzpNxFxa/MDImIBsABAUnuzG5pZy9o6skfE6ur7WuBqYE4nOmVmnddysksaI2ncrtvAycCyTnXMzDqrndP45wNXV+N29wS+GRE3dKRX1jHbtm1rq/3RRx+djM+cOTMZT9X5c2PCb7zxxmT8qKOOSsY/9alP1cb6+tIfIS1dujQZX758eTI+Z076JDe1XxcuXJhse9ttt9XGNm3aVBtrOdkj4l7gJa22N7OR5dKbWSGc7GaFcLKbFcLJblYIJ7tZIdTukr1P68l8Bd2wSE1bnPv95oaJpspXAPvuu28yvn379tpYbihnzqJFi5Lxe+65pzbWbkly6tSpyXjqdUO67295y1uSbefPn18b6+vr47HHHhv0D8JHdrNCONnNCuFkNyuEk92sEE52s0I42c0K4WQ3K4Tr7D0gt7xvO3K/35///OfJeG4Ia07qteWWLW63Fp5a8jlX47/jjjuS8VQNH/Kv7ZRTTqmNvfCFL0y2nT59ejIeEa6zm5XMyW5WCCe7WSGc7GaFcLKbFcLJblYIJ7tZIbxkcw8YyWsdBlq/fn0ynhu3vXXr1mQ8tSzznnum//xSyxpDuo4O8LznPa82lquzv+IVr0jGjz322GQ8N0325MmTa2M33DA8M7L7yG5WCCe7WSGc7GaFcLKbFcLJblYIJ7tZIZzsZoVwnb1wo0ePTsZz9eJcfMuWLbWxDRs2JNs+8sgjyXhurH3q+oXcHAK515Xbbzt37kzGU3X+GTNmJNu2Kntkl3SZpLWSljXdN0HSzZLurr7vNyy9M7OOGcpp/FeBgdNqXADcEhGHArdUP5tZD8sme0TcCqwbcPdpwOXV7cuB0zvcLzPrsFbfsz8/ItYARMQaSbUX+kqaB8xr8XnMrEOG/QO6iFgALABPOGnWTa2W3volTQWovq/tXJfMbDi0muzXAmdVt88CrulMd8xsuGRP4yVdCZwATJK0CvgYcAnwHUnnAn8A3jqcnXy2a7fmm6rp5saET5s2LRl/4okn2oqnxrPn5oVP1eghvzZ8qk6fq5PvvffeyfjGjRuT8fHjxyfjS5YsqY3lfmezZ8+ujd111121sWyyR8SZNaHX5NqaWe/w5bJmhXCymxXCyW5WCCe7WSGc7GaF8BDXHpCbSnrUqFHJeKr0dsYZZyTbTpkyJRl/6KGHkvHUdM2QHso5ZsyYZNvcUM9c6S5V9tu+fXuybW6a69zrnjhxYjI+f/782tisWbOSbVN9S5VxfWQ3K4ST3awQTnazQjjZzQrhZDcrhJPdrBBOdrNCaCSXC/ZMNYPL1XR37NjR8rZf9rKXJePXXXddMp5bkrmdawDGjRuXbJtbkjk31fRee+3VUgzy1wDklrrOSb22T3/608m2V1xxRTIeEYMW231kNyuEk92sEE52s0I42c0K4WQ3K4ST3awQTnazQjyjxrOnxurm6r256Zhz0zmnxj+nxmwPRTt19Jzrr78+Gd+8eXMynquz56ZcTl3HkRsrn/udPve5z03Gc2PW22mb+53n+n7kkUfWxnJLWbfKR3azQjjZzQrhZDcrhJPdrBBOdrNCONnNCuFkNytET9XZ2xkbPZy16uH2yle+Mhl/85vfnIwfd9xxtbHcsse5MeG5OnpuLH7qd5brW+7vITUvPKTr8Ll5HHJ9y8ntt02bNtXG3vSmNyXbfv/732+pT9kju6TLJK2VtKzpvoskPSBpcfV1akvPbmYjZiin8V8FThnk/s9FxKzqK32Zlpl1XTbZI+JWYN0I9MXMhlE7H9CdJ2lJdZq/X92DJM2T1Cepr43nMrM2tZrsXwQOBmYBa4DP1D0wIhZExOyImN3ic5lZB7SU7BHRHxE7I+JJ4EvAnM52y8w6raVklzS16cc3AsvqHmtmvSE7b7ykK4ETgElAP/Cx6udZQAArgfdFxJrsk3Vx3vgJEyYk49OmTUvGDz300Jbb5uqmhx12WDL+xBNPJOOpsfq5cdm5dcZXr16djOfmX0/Vm3NrmOfWXx89enQyvnDhwtrY2LFjk21z1z7kxrPnxqSn9lt/f3+y7RFHHJGM180bn72oJiLOHOTuS3PtzKy3+HJZs0I42c0K4WQ3K4ST3awQTnazQvTUks3HHHNMsv3FF19cG9t///2Tbffdd99kPDUUE9LDLR999NFk29zw21wJKVeCSk2DnZsKevny5cn43Llzk/G+vvRV0Kllmffbr/YqawBmzpyZjOfce++9tbHcctEbN25MxnNDYHMlzVTpb5999km2zf29eMlms8I52c0K4WQ3K4ST3awQTnazQjjZzQrhZDcrxIjX2VP16ttuuy3ZfurUqbWxXJ08F29n6uDclMe5Wne7xo8fXxubNGlSsu3ZZ5+djJ988snJ+Pvf//5kPDVE9vHHH0+2/f3vf5+Mp+rokB6W3O7w2tzQ3lwdP9U+N3z2wAMPTMZdZzcrnJPdrBBOdrNCONnNCuFkNyuEk92sEE52s0KMaJ190qRJ8YY3vKE2fskllyTbr1ixojaWmxo4F88t/5uSq7mm6uAA999/fzKem845NZY/Nc00wJQpU5Lx008/PRlPLYsM6THpud/JS1/60rbiqdeeq6Pn9ltuSeac1BwEub+n1LwPDz74INu2bXOd3axkTnazQjjZzQrhZDcrhJPdrBBOdrNCONnNCpFdxVXSDOBrwBTgSWBBRHxe0gTg28BMGss2z42I9alt7dixg7Vr19bGc/Xm1Bjh3LLGuW3nar6pumpunu9169Yl4/fdd18ynutbarx8bsx4bk77q6++OhlfunRpMp6qs+eW0c7VwnPz9aeWq8697tyY8lwtPNc+VWfP1fBTS3yn9slQjuw7gA9HxBHAMcAHJL0IuAC4JSIOBW6pfjazHpVN9ohYExF3VLc3AsuB6cBpwOXVwy4H0pdamVlXPa337JJmAkcBvwCeHxFroPEPAZjc6c6ZWecMOdkljQW+B3woIh57Gu3mSeqT1Jd7D2Zmw2dIyS5pLxqJ/o2IuKq6u1/S1Co+FRj0k7eIWBARsyNidruDB8ysddlkV+Njw0uB5RHx2abQtcBZ1e2zgGs63z0z65Rs6Q04DngnsFTS4uq+C4FLgO9IOhf4A/DW3Ia2bdvGAw88UBvPDbddtWpVbWzMmDHJtrkplXNlnIcffrg29tBDDyXb7rlnejfnhtfmyjypYaa5KY1zQzlTrxvgiCOOSMY3b95cG8uVQ9evT1Zys/st1fdUWQ7ypblc+9ySzamhxRs2bEi2nTVrVm1s2bJltbFsskfET4G6ouBrcu3NrDf4CjqzQjjZzQrhZDcrhJPdrBBOdrNCONnNCjGUOnvHbN26lcWLF9fGr7rqqtoYwDnnnFMby023nFveNzcUNDXMNFcHz9Vcc1cW5paETg3vzS1Vnbu2IbeU9Zo1a1refq5vuesT2vmdtTt8tp3htZCu4x900EHJtv39/S09r4/sZoVwspsVwsluVggnu1khnOxmhXCymxXCyW5WiBFdsllSW0/22te+tjZ2/vnnJ9tOnpyeIi83bjtVV83Vi3N18lydPVdvTm0/NWUx5OvsuWsIcvHUa8u1zfU9J9U+VaseitzvLDeVdGo8+5IlS5Jt586dm4xHhJdsNiuZk92sEE52s0I42c0K4WQ3K4ST3awQTnazQox4nT01T3muNtmOV73qVcn4Jz/5yWQ8VacfP358sm1ubvZcHT5XZ8/V+VNSS2hDvg6fWgcA0r/TTZs2Jdvm9ktOqu+58ea5cfy53+nNN9+cjC9fvrw2tnDhwmTbHNfZzQrnZDcrhJPdrBBOdrNCONnNCuFkNyuEk92sENk6u6QZwNeAKcCTwIKI+Lyki4D3ArsWJ78wIq7PbGvkivoj6PDDD0/G210b/oADDkjGV65cWRvL1ZNXrFiRjNszT12dfSiLROwAPhwRd0gaB/xS0q4rBj4XEf/SqU6a2fDJJntErAHWVLc3SloOTB/ujplZZz2t9+ySZgJHAb+o7jpP0hJJl0nar6bNPEl9kvra6qmZtWXIyS5pLPA94EMR8RjwReBgYBaNI/9nBmsXEQsiYnZEzO5Af82sRUNKdkl70Uj0b0TEVQAR0R8ROyPiSeBLwJzh66aZtSub7GpM0XkpsDwiPtt0/9Smh70RWNb57plZpwyl9HY88BNgKY3SG8CFwJk0TuEDWAm8r/owL7WtZ2XpzayX1JXenlHzxptZnsezmxXOyW5WCCe7WSGc7GaFcLKbFcLJblYIJ7tZIZzsZoVwspsVwsluVggnu1khnOxmhXCymxXCyW5WiKHMLttJDwP3Nf08qbqvF/Vq33q1X+C+taqTfTuwLjCi49mf8uRSX6/OTderfevVfoH71qqR6ptP480K4WQ3K0S3k31Bl58/pVf71qv9AvetVSPSt66+ZzezkdPtI7uZjRAnu1khupLskk6R9FtJ90i6oBt9qCNppaSlkhZ3e326ag29tZKWNd03QdLNku6uvg+6xl6X+naRpAeqfbdY0qld6tsMST+StFzSnZI+WN3f1X2X6NeI7LcRf88uaRTwO+AkYBWwCDgzIu4a0Y7UkLQSmB0RXb8AQ9IrgU3A1yLixdV9nwLWRcQl1T/K/SLib3qkbxcBm7q9jHe1WtHU5mXGgdOBs+nivkv0ay4jsN+6cWSfA9wTEfdGxDbgW8BpXehHz4uIW4F1A+4+Dbi8un05jT+WEVfTt54QEWsi4o7q9kZg1zLjXd13iX6NiG4k+3Tg/qafV9Fb670HcJOkX0qa1+3ODOL5u5bZqr5P7nJ/Bsou4z2SBiwz3jP7rpXlz9vVjWQfbGmaXqr/HRcRfw68FvhAdbpqQzOkZbxHyiDLjPeEVpc/b1c3kn0VMKPp5wOA1V3ox6AiYnX1fS1wNb23FHX/rhV0q+9ru9yfP+qlZbwHW2acHth33Vz+vBvJvgg4VNJBkvYG3gZc24V+PIWkMdUHJ0gaA5xM7y1FfS1wVnX7LOCaLvZlN72yjHfdMuN0ed91ffnziBjxL+BUGp/IrwD+tht9qOnXC4FfV193drtvwJU0Tuu20zgjOheYCNwC3F19n9BDffs6jaW9l9BIrKld6tvxNN4aLgEWV1+ndnvfJfo1IvvNl8uaFcJX0JkVwsluVggnu1khnOxmhXCymxXCyW5WCCe7WSH+DyQX2h4/WEnnAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAEICAYAAACZA4KlAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAWFklEQVR4nO3debBcZZnH8e8PSAgJShYEQgSiEFAqIxEjBSLiuACiFFKWrCKuoRgVrZES1FJRhxlqRB1nXIqoQABFQGWpkWItHUBHQ4RAwjIJQZaYmLAkkJBAEvLMH30izc3t9725p/t2c9/fp+rW7XuePue8fZKnz+l+zvu+igjMbPjbqtsNMLOh4WQ3K4ST3awQTnazQjjZzQrhZDcrhJO9zSTdK+ntQ7CfYyQ9Jmm1pDfW2E5I2qtF7CRJNw6+ldZL5Dr7y4ekE4H3RcSJkhYB/xwR19TcZgBTIuLBtjSy9X5mAocCU4CPRcRFndyfbc5n9peXI4Hrqsd7APd2sS1b6m7gn4A7u92QUjnZ20zSw5LeJelsSVdKulTSKknzJO0t6YuSlleX4Ic1rfcaSbdWz71Z0g8kXdoU3wp4N3CTpNXA1sDd1RkeSWdJWlStf5+kY5rW3UvS/0h6WtITki7v0+x3SVooaUW1X1XrfUTS7U3beYukO6rt3CHpLU2x30n6pqTfV224UdKOm+IR8YOIuAV4bgDH8HeS/k3S7Gpf10gaX8UmVx89TpH0aPV6vty07naSZlWv5X5JX5C0OP8vN/w52TvrKOASYBxwF3ADjWM+CfgGcH7Tc38OzAYmAGcDJ/fZ1gHAQxGxLCK2r5btFxF7Vo8XAYcAOwBfBy6VNLGKfRO4sWrHq4H/6rPt9wFvBvYDjgUO7/tCqmT7DfCfVRu/A/xG0oSmp50IfBTYCRgJnNH/YRmQDwMfA3YFNlT7bfZWYB/gncBXJb2+Wv41YDLwWhpvjh+q0YZhxcneWbdFxA0RsQG4EngVcG5ErAd+AUyWNFbS7jSS7asRsS4ibgeu7bOt9/LiJfxmIuLKiFgSERsj4nJgIY03CID1NC77d42I56rtNzs3IlZGxKPAb4Fp/ezivcDCiLgkIjZExGXAAzTe0Da5MCIWRMRa4IoW2xmoSyJifkQ8C3wFOFbS1k3xr0fE2oi4m8ZHhP2q5ccC/xoRKyJiMZu/SRTLyd5Zy5oerwWeiIgXmv4G2J7G2eupiFjT9PzH+myr+fP6ZiR9WNJcSSslrQSmApsuo78ACJhdVQs+1mf1vzU9XlO1qa9dgUf6LHuExlXKlmxnoJpf/yPACF58Pal97dpn3b7HsVhO9t6wFBgvaXTTst02PZC0CzCRFl9uSdoD+DHwaWBCRIwF5tNIcCLibxHxyYjYFTgV+GGrclvCEhpXB812B/66hdsZqN2aHu9O4+rkiQGst5TGR5X+tlM0J3sPiIhHgDnA2ZJGSjqIl14eHwlcH63rpGOAAB4HkPRRGmd2qr8/KGlTAqyonvtC341kXAfsLelESdtIOg7YF/jvgaxcva5RNN6ARkgaVX3p2MqHJO1bvQF+A/hl01VRyhXAFyWNkzSJxhug4WTvJScBBwFPAv8CXA48X8WSl/ARcR/wbeB/aXx0+Afg901PeTPwp+pb/GuBz0bEX7akcRHxJI0v8j5ftfELNGr+AznbQuMLwrXAW4CZ1eO3wd9v3ulbRrwEuIjG5foo4PQB7ucbwGLgL8DNwC958TgWzTfV9KiqPPYAjW/S/wbsGRFPd7dVQ0PS74BLI+InbdjWacDxEXFo7Ya9zPnM3iMkvVnSnpK2knQEcDRwNTAe+EopiV6XpImSDq6O4z40rkSu6na7esE23W6A/d0uwK9p1LAXA6dFxF1V7Edda9XLz0ga9y+8BlhJo8T5w662qEf4Mt6sEL6MNyvEkF7GVz2sbAuNGjUqGd99991bxp566qnkumvWrEnGc1d+ufh2223XMjZu3Ljkus89l76NftmyZcn4Cy9saXVxeIgI9be8VrJXXyR9j0anjJ9ExLl1ttdNVd+Plrr5cWfy5MnJ+Pe///2WsSuvvDK57l133ZWMr1u3Lhlfv359Mj516tSWsWOOOaZlDGDRokXJ+Le+9a1kfOXKlcl4aQZ9GV/dp/wD4D00bq44QdK+7WqYmbVXnc/sBwAPRsRDEbGOxreeR7enWWbWbnWSfRIv7WSwmJd2igBA0gxJcyTNqbEvM6upzmf2/j7kbvbBNiJm0rg90l/QmXVRnTP7Yl7ao+jVNHpGmVkPqpPsdwBTquGURgLHs/mAC2bWI2rdQSfpSOA/aJTeLoiIczLP79hlfDdLZ9OmpQdkOf7445PxD3zgA8l4rl48ZsyYlrFUnRtgwoQJyXgnLViwIBnfuHFjMr7PPvsk46k6/A033JBc97zzzkvG58+fn4x3U0fq7BFxHYmul2bWO3y7rFkhnOxmhXCymxXCyW5WCCe7WSGc7GaFGNKRanr5dtlXvvKVyfjFF1/cMvaGN7whue5WW6XfU1etWpWM5/p1p7qZ5mr0I0aMSMZ32GGHZPzZZ59NxlO18k7/30uNA5C7/2DkyJHJ+G233ZaMn3xy39m7hk6rOrvP7GaFcLKbFcLJblYIJ7tZIZzsZoVwspsVwqW3ys0335yM77FH39mKX/Tkk08m18111dxmm3Tnww0bNiTjue69KbmyYG502a233rpj++6kul2iJ06cmIwffvjhyfgDDzyQjNfh0ptZ4ZzsZoVwspsVwsluVggnu1khnOxmhXCymxViSKds7qY3velNyXiqjg7wxBNPtIzl6uS5WnRuSuZJkzabVeslRo8e3TKWq2XnZmHNvbZcF9pUPTvXvTZ3f0Gua/DixYsHve2c3Ov+xCc+kYyfccYZtfY/GD6zmxXCyW5WCCe7WSGc7GaFcLKbFcLJblYIJ7tZIYrpz56ra55++unJeKrOnuuvnquz52q2559/fjK+ZMmSlrFUrRlg1113TcaXLl2ajNfpD7/tttsm191+++2T8f333z8Z/8xnPtMylvr3hPz9Bbmhx3PrT548ORmvoyNTNkt6GFgFvABsiIjpdbZnZp3Tjjvo/jEi0m+TZtZ1/sxuVoi6yR7AjZL+LGlGf0+QNEPSHElzau7LzGqoexl/cEQskbQTcJOkByLi1uYnRMRMYCb09oCTZsNdrTN7RCypfi8HrgIOaEejzKz9Bp3sksZIesWmx8BhwPx2NczM2mvQdXZJr6VxNofGx4GfR8Q5mXW6dhn/xz/+MRnfaaedkvFU3+nc2Oq5evHTTz+djB944IHJ+GGHHdYylusLf+GFFybjp556ajI+f376/T01NXLu/oNly5Yl43Pnzk3GFy5c2DKW6wufG2Mg1x/+da97XTI+derUlrEFCxYk181pe509Ih4C9ht0i8xsSLn0ZlYIJ7tZIZzsZoVwspsVwsluVohihpLeb7904eCxxx5LxlNdOXNdNXNy3SVzrr/++paxZ599Nrnuvvvum4znugZfddVVyfhRRx3VMpbrBnrnnXcm47nhwVPlsTFjxiTXzXU7znVrfvTRR5Pxgw46qGWsbumtFZ/ZzQrhZDcrhJPdrBBOdrNCONnNCuFkNyuEk92sEMOmzp7qMgjw+OOPJ+O5Loup7pipaYkh3c0T4Mknn0zGc1Kv/fnnn0+uO3HixGT8nHOSvZazrz01JXRu3VQteiBSQ2znuv7WrbOvXbs2GT/kkENaxmbNmpVcd7B8ZjcrhJPdrBBOdrNCONnNCuFkNyuEk92sEE52s0IMmzr7mWeemYznat2rV69OxlN119y2n3vuuWQ8V+OfPj09Oe6ECRNaxsaPH59cd8SIEcn4zjvvnIyn6uiQfu0jR45Mrjt27Nhk/LjjjkvGx40b1zKWq4PvsMMOyXhu/dxry/2bdoLP7GaFcLKbFcLJblYIJ7tZIZzsZoVwspsVwsluVohhU2f/wx/+kIzvsssuyfhee+2VjKfGds+NQZ6aOhjyfadz002n+lbn+l3n9p2bVjk39nuqz3pu36mx+iE/7XJq/PXRo0cn18297lzbUn3pAa6++upkvBOyZ3ZJF0haLml+07Lxkm6StLD63fruBTPrCQO5jL8IOKLPsrOAWyJiCnBL9beZ9bBsskfErcBTfRYfDWwaO2cW8P42t8vM2mywn9l3joilABGxVNJOrZ4oaQYwY5D7MbM26fgXdBExE5gJICk6vT8z699gS2/LJE0EqH4vb1+TzKwTBpvs1wKnVI9PAa5pT3PMrFMUkb6ylnQZ8HZgR2AZ8DXgauAKYHfgUeCDEdH3S7z+ttWzl/Gpvs8AU6ZMaRk77bTTkuseeuihyXhubvhc3+qVK1e2jOX6q+fqyZ2UGzc+V8vOjROQOm7z5s1LrnvSSScl470sIvo9sNnP7BFxQovQO2u1yMyGlG+XNSuEk92sEE52s0I42c0K4WQ3K8Sw6eJa14oVK5Lx2bNnt4zlpkV+xzvekYznyp+5YYlTXWxzpbVcF9icXPksFc/te9ttt03G161bl4yPGjWqZSzXJXo48pndrBBOdrNCONnNCuFkNyuEk92sEE52s0I42c0KUUydPVcPznUFTdV0c3XyZ555JhnP1cJzQy7n9p+SOy51tt1pdbrnproFt2PfuXsIunFcfWY3K4ST3awQTnazQjjZzQrhZDcrhJPdrBBOdrNCFFNnz9U1169fP+htL1q0KBnP1dlz0x7n+m2nDGCo8Frr5+S2n5J73bl7I1Jy/yY5uWGuc/dGdIPP7GaFcLKbFcLJblYIJ7tZIZzsZoVwspsVwsluVohi6uw5deqma9euTa6bqxfnxkffsGFDMp6q09eto9cZFx7SxzW379x4/KNHj07GU23LHdPhKHtml3SBpOWS5jctO1vSXyXNrX6O7GwzzayugVzGXwQc0c/y70bEtOrnuvY2y8zaLZvsEXEr8NQQtMXMOqjOF3SflnRPdZk/rtWTJM2QNEfSnBr7MrOaBpvsPwL2BKYBS4Fvt3piRMyMiOkRMX2Q+zKzNhhUskfEsoh4ISI2Aj8GDmhvs8ys3QaV7JImNv15DDC/1XPNrDdk6+ySLgPeDuwoaTHwNeDtkqYBATwMnNrBNg6JOv22c2OE1x33PRfP3SOQkmt7nbHZIV3rzrU797pzba9T48/p5fH0W8kme0Sc0M/in3agLWbWQb5d1qwQTnazQjjZzQrhZDcrhJPdrBDu4joEJk2alIyvWLEiGc+Vv1JloFx5q85Qz52Wa3tu+O/Ua6tbUnw58pndrBBOdrNCONnNCuFkNyuEk92sEE52s0I42c0K4Tp7pZNdFusOWzxy5MhkPNWFtu5Q0J0cijrXRTU3JXNuqOlU2+pM95zbdq/ymd2sEE52s0I42c0K4WQ3K4ST3awQTnazQjjZzQrhOvsQyNWDc32rc3X61Pq5WnauXpxrW2466tT2U1NN59YFWLNmTTKeMnbs2EGv+3LlM7tZIZzsZoVwspsVwsluVggnu1khnOxmhXCymxViIFM27wZcDOwCbARmRsT3JI0HLgcm05i2+diISA+AXqhcrbuuVJ/xuv2uOznufJ2+8ANZP3V/wnbbbZdcN2e49mffAHw+Il4PHAh8StK+wFnALRExBbil+tvMelQ22SNiaUTcWT1eBdwPTAKOBmZVT5sFvL9TjTSz+rboM7ukycAbgT8BO0fEUmi8IQA7tbtxZtY+A743XtL2wK+Az0XEMwP9rCZpBjBjcM0zs3YZ0Jld0ggaif6ziPh1tXiZpIlVfCKwvL91I2JmREyPiOntaLCZDU422dU4hf8UuD8ivtMUuhY4pXp8CnBN+5tnZu0ykMv4g4GTgXmS5lbLvgScC1wh6ePAo8AHO9PEl79c+aquTpaBull6y+27Tult9OjRyXWHo2yyR8TtQKt/0Xe2tzlm1im+g86sEE52s0I42c0K4WQ3K4ST3awQTnazQngo6Uo3uyzmhmuuo2430pw6be9099vUVNadPOa9ymd2s0I42c0K4WQ3K4ST3awQTnazQjjZzQrhZDcrhOvslbrDFqfkpjXuZN/q3DDWdaeL7uRxq6uTdfbhOpS0mQ0DTnazQjjZzQrhZDcrhJPdrBBOdrNCONnNCuE6ew+o0y8b0rXu3LbrxnN1/G6OK5/i/uxmNmw52c0K4WQ3K4ST3awQTnazQjjZzQrhZDcrRLbOLmk34GJgF2AjMDMivifpbOCTwOPVU78UEdd1qqGd1sn+yUuWLEnG995772Q816c8VevO1cFHjBgx6G0PJJ46rrn7B7bZpt5tIKl9l9iffSBHcwPw+Yi4U9IrgD9LuqmKfTcizutc88ysXbLJHhFLgaXV41WS7gcmdbphZtZeW/SZXdJk4I3An6pFn5Z0j6QLJI1rsc4MSXMkzanVUjOrZcDJLml74FfA5yLiGeBHwJ7ANBpn/m/3t15EzIyI6RExvQ3tNbNBGlCySxpBI9F/FhG/BoiIZRHxQkRsBH4MHNC5ZppZXdlkV6Pb0k+B+yPiO03LJzY97RhgfvubZ2btMpBv4w8GTgbmSZpbLfsScIKkaUAADwOndqSFw8DYsWOT8TFjxiTjuRLUjjvu2DJWtwtrrjRXR670liuPPfbYY8l4aojuPffcM7luTt2uv90wkG/jbwf665T8sq2pm5XId9CZFcLJblYIJ7tZIZzsZoVwspsVwsluVggPJV3p5NTDd911VzJ+3333JeMrV65MxuvUwnP14tWrVyfjueOSOq51uu5CfirsceP67a4BwOzZs5Pr5vRiHT3HZ3azQjjZzQrhZDcrhJPdrBBOdrNCONnNCuFkNyuEhnJIXEmPA480LdoReGLIGrBlerVtvdoucNsGq51t2yMiXtVfYEiTfbOdS3N6dWy6Xm1br7YL3LbBGqq2+TLerBBOdrNCdDvZZ3Z5/ym92rZebRe4bYM1JG3r6md2Mxs63T6zm9kQcbKbFaIryS7pCEn/J+lBSWd1ow2tSHpY0jxJc7s9P101h95ySfOblo2XdJOkhdXv1p22h75tZ0v6a3Xs5ko6sktt203SbyXdL+leSZ+tlnf12CXaNSTHbcg/s0vaGlgAvBtYDNwBnBAR6REchoikh4HpEdH1GzAkvQ1YDVwcEVOrZf8OPBUR51ZvlOMi4sweadvZwOpuT+NdzVY0sXmaceD9wEfo4rFLtOtYhuC4dePMfgDwYEQ8FBHrgF8AR3ehHT0vIm4Fnuqz+GhgVvV4Fo3/LEOuRdt6QkQsjYg7q8ergE3TjHf12CXaNSS6keyTgOZ5exbTW/O9B3CjpD9LmtHtxvRj54hYCo3/PMBOXW5PX9lpvIdSn2nGe+bYDWb687q6kez9DUrWS/W/gyNif+A9wKeqy1UbmAFN4z1U+plmvCcMdvrzurqR7IuB3Zr+fjWwpAvt6FdELKl+Lweuovemol62aQbd6vfyLrfn73ppGu/+phmnB45dN6c/70ay3wFMkfQaSSOB44Fru9COzUgaU31xgqQxwGH03lTU1wKnVI9PAa7pYlteolem8W41zThdPnZdn/48Iob8BziSxjfyi4Avd6MNLdr1WuDu6ufebrcNuIzGZd16GldEHwcmALcAC6vf43uobZcA84B7aCTWxC617a00PhreA8ytfo7s9rFLtGtIjptvlzUrhO+gMyuEk92sEE52s0I42c0K4WQ3K4ST3awQTnazQvw/fzKKFYLPjXoAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAEICAYAAACZA4KlAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAATqklEQVR4nO3de5BcZZ3G8e9DkklIAiSTkCsgkZtSbolbMbWK60J5WURX5A9BUIyXMq4lhRZuuSClou6F2l213PVSGxUMqAiILNSaUrIULmJtCQEjBFHugUAuBEgghkAm+e0f50Q6k+n3TPr0bfI+n6qp6elfnz6/7uTpc7rfPudVRGBm+78Det2AmXWHw26WCYfdLBMOu1kmHHazTDjsZplw2NtM0j2STurCek6X9JikrZJeU+N+QtLRTWrvlXRj611aP5HH2ccOSWcD74iIsyU9CJwfEdfXvM8AjomIB9rS5MjrOBb4V+D1wDjgduC8iPhDp9Zpe/OWfWw5FVheXn4ZcE8Pe9kX04AbgOOA2cBtQK0XKWtBRPinjT/AI8CbgYuBa4DvA88BdwPHAhcCG4HHgLc2LLcAuKW87f8A3wC+31A/ANhAEZatQAB/BB4s6xcAD5bL/w44vWHZo4H/BbYAm4CrGmoB/C1wP/BMud7de3wfAG5tuO3rKbbKW8rfr2+o/QL4EvCrsocbgZlNnqPBcr0zmtR/AfwzxYvCFooXhsGydmS57GLg0fLxXNSw7IHAsvKx3At8Gljb6/8X/fDjLXtn/Q1wBTAd+A3wc4rQzge+CPxnw21/SPGfewbFC8U5w+5rEfBQRGyIiKnlda+OiKPKyw8CfwkcAnwB+L6kuWXtSxThmw4cBvzHsPt+B/Ba4NXAGcBfD38gkgaBnwL/Xvb4FeCnkmY03Oxs4IPALGAA+LuRnxbeCKyPiKea1AHeD3wImAcMlett9AaKPYU3AZ+T9Mry+s9TvCC8HHgL8L7EOrLisHfWLyPi5xExRLGVPxS4JCJ2AD8CjpQ0TdIRFGH7XES8GBG3Uuz2Nno7L+3C7yUiromIJyJiV0RcRbGlXlSWd1Ds9s+LiO3l/Te6JCI2R8SjwM3ACSOs4u3A/RFxRUQMRcSVwO8pXtB2uywi7ouI54GrR7ofSYdR7D2c3+yxlK6IiNUR8Ufgs8AZksY11L8QEc9HxG+B31K8UEHxYvVPEfFMRKxl7xeJbDnsnbWh4fLzwKaI2NnwN8BUiq3X0xGxreH2jw27r8b363uR9H5JqyRtlrQZeBUwsyx/GhBwWzla8KFhi69vuLyt7Gm4ecCaYdetodhLGdX9SDqUYg/jm+WLRUrj418DTOClx5Na17xhyw5/HrPlsPeHdcCgpMkN1x2++4KkOcBc4M6RFpb0MuDbwLkU74OnAaspAk5ErI+Ij0TEPOCjwDebDbclPEGxd9DoCODx0SwsaTpF0G+IiH8cxSKHN1w+gmLvZNMolltH8VZlpPvJmsPeByJiDbASuFjSgKTXsefu8anAz6L8BGoEUyg+tHoSQNIHKbbslH+/u9x9huKDqwB2Dr+TCsuBYyWdLWm8pDOB44H/rlpQ0sEUn1f8KiIuGOX63ifp+PIF8IvAjxv2ilKuBi6UNF3SfIoXQMNh7yfvBV4HPAX8A3AV8EJZS+7CR8TvgC8D/0fx1uHPKD4V3+21wK8lbaX4LOATEfHwvjRXfpj2DuBTZY+fphjzH83W9vSyhw+WXwLa/XME/OnLO8OHEa8Avkexuz4JOG+UrX4RWAs8TDGq8WNeeh6z5i/V9ClJV1F8APYliv/wR0XElt521R2SfkEx7PidNtzXx4D3RMRf1W5sjPOWvU9Ieq2koyQdIOkU4DTgvyjGpD+bS9DrkjRX0onl83gcxZ7Idb3uqx+M73UD9idzgJ9QjGGvBT4WEb8pa9/qWVdjzwDF9xcWAJsphji/2dOO+oR3480y4d14s0x0dTe+PMLKhhk/Pv3PMGPGjGT9qaeaf+t0aGiopZ664cADD0zWJ02alKxv3rw5Wc91rzUiNNL1tcJefpD0NYrDFr8TEZfUub9cDQ4OJuuLFy9O1i+//PKmtfXr1zet9dpxxx2XrL/iFa9I1q+99tpkfceOHfvc0/6s5d348nvK3wDeRvHlirMkHd+uxsysveq8Z18EPBARD0XEixSfep7WnrbMrN3qhH0+ex5ksJY9D4oAQNISSSslrayxLjOrqc579pE+BNjrE5GIWAosBX9AZ9ZLdbbsa9nziKLDKI6MMrM+VCfstwPHSFogaQB4D3ufcMHM+kTLu/ERMSTpXIpDF8cBl0bEWDkBYldNnTrSuSBe8s53vjNZP+ec4Weo2tOZZ57ZtLZpU/qgtBdffLFW/aCDDkrWJ06c2LR22GGHNa0BXH99+pyUO3emj3i95pprkvXc1Bpnj4jlJA69NLP+4a/LmmXCYTfLhMNulgmH3SwTDrtZJhx2s0z4tFRdsHXr1mR9y5b06eUuvPDCZP2iiy5qWqs6THT27NnJemqcHOCZZ55J1lOPfcWKFcllly9Pj+pWfX/B9uQtu1kmHHazTDjsZplw2M0y4bCbZcJhN8uEh976wMDAQLJedcrkr3/9601r552Xng/xhRfScx5WDb1V9XbHHXc0rV122WXJZRcsWJCsP/nkk8m67clbdrNMOOxmmXDYzTLhsJtlwmE3y4TDbpYJh90sEx5n7wNVh8DOnDkzWV+zZk3T2vnnn59ctup0zoceemiy/vDDDyfrqemkqx5X1VTW0ogzE1sT3rKbZcJhN8uEw26WCYfdLBMOu1kmHHazTDjsZpnwOHsfGBoaqrV81Xh1StWUzuvXr0/WJ0+enKzPnz+/aa1qyuWIqFW3PdUKu6RHgOeAncBQRCxsR1Nm1n7t2LKfHBHpzYOZ9Zzfs5tlom7YA7hR0h2Slox0A0lLJK2UtLLmusyshrq78SdGxBOSZgErJP0+Im5pvEFELAWWAkjyJypmPVJryx4RT5S/NwLXAYva0ZSZtV/LYZc0RdJBuy8DbwVWt6sxM2uvOrvxs4HrymOKxwM/jIiftaWrzBxwQPo1t2o8OTVePW7cuOSy06ZNS9Y7qep49KrHXXW8u+2p5WcrIh4CXt3GXsysgzz0ZpYJh90sEw67WSYcdrNMOOxmmfDYRR+YOnVqsl41bfL27dub1qqG3nbt2pWsVy1f53TOVUOOVfVJkya1vO4cectulgmH3SwTDrtZJhx2s0w47GaZcNjNMuGwm2XC4+x9oO7UxKl61Vh1nfuue/9Vp9Cuuu+q7wDYnrxlN8uEw26WCYfdLBMOu1kmHHazTDjsZplw2M0y4XH2PlA1nrxt27ZkPTXeXHecvWpa5Sp1plV+4YUXaq3b9uQtu1kmHHazTDjsZplw2M0y4bCbZcJhN8uEw26WCY+z94GqsfAqqXH2uueFr9tbStVx/FXj7LNmzWpnO/u9yn9JSZdK2ihpdcN1g5JWSLq//D29s22aWV2jedn+HnDKsOsuAG6KiGOAm8q/zayPVYY9Im4Bnh529WnAsvLyMuBdbe7LzNqs1ffssyNiHUBErJPU9M2TpCXAkhbXY2Zt0vEP6CJiKbAUQFLrR0WYWS2tftS6QdJcgPL3xva1ZGad0GrYbwAWl5cXA9e3px0z65TK3XhJVwInATMlrQU+D1wCXC3pw8CjwLs72eRYN316emSy7hzoqWPGOzlOPhqpcf6qcfbUvPMAU6ZMSdZT87dX3ff+qDLsEXFWk9Kb2tyLmXWQvy5rlgmH3SwTDrtZJhx2s0w47GaZ8CGuXVB1qGZVvc7pmKvUve+6UzqnVA1JbtmyJVnPcXgtxVt2s0w47GaZcNjNMuGwm2XCYTfLhMNulgmH3SwTHmfvgqqx7Krx5P1V1fMyceLELnWSB2/ZzTLhsJtlwmE3y4TDbpYJh90sEw67WSYcdrNMeJy9C+qOo1dNu9zJ00X3ct1V971z586Wl696XPsjb9nNMuGwm2XCYTfLhMNulgmH3SwTDrtZJhx2s0x4nL0LUlMHQ/Vx3VX11Lnb64xFQ2ePta8zFfVo6gMDA01rOZ5TvnLLLulSSRslrW647mJJj0taVf6c2tk2zayu0ezGfw84ZYTrvxoRJ5Q/y9vblpm1W2XYI+IW4Oku9GJmHVTnA7pzJd1V7uZPb3YjSUskrZS0ssa6zKymVsP+LeAo4ARgHfDlZjeMiKURsTAiFra4LjNrg5bCHhEbImJnROwCvg0sam9bZtZuLYVd0tyGP08HVje7rZn1h8pxdklXAicBMyWtBT4PnCTpBCCAR4CPdrDHMa9qPLluvc4c61X33Ut1e+vksfZjUWXYI+KsEa7+bgd6MbMO8kufWSYcdrNMOOxmmXDYzTLhsJtlwoe4dkE/T8lc5/DZ0UgtX3cq66r6+PH+793IW3azTDjsZplw2M0y4bCbZcJhN8uEw26WCYfdLBMeiOyCqrHqqtM91xkLr3uYZ53DZ6uWr9tb1fN6yCGHNK09++yztdY9FnnLbpYJh90sEw67WSYcdrNMOOxmmXDYzTLhsJtlwuPsXTBhwoRkvWq8uc4x5Z08DXWn1f3+wcSJE9vZzpjnLbtZJhx2s0w47GaZcNjNMuGwm2XCYTfLhMNulonRTNl8OHA5MAfYBSyNiK9JGgSuAo6kmLb5jIh4pnOtjl1V5y+vGguvOj96P4+VpwwNDdVafseOHcm6p2ze02iejSHgUxHxSuAvgI9LOh64ALgpIo4Bbir/NrM+VRn2iFgXEXeWl58D7gXmA6cBy8qbLQPe1akmzay+fdrPkXQk8Brg18DsiFgHxQsCMKvdzZlZ+4z6u/GSpgLXAp+MiGdH+31tSUuAJa21Z2btMqotu6QJFEH/QUT8pLx6g6S5ZX0usHGkZSNiaUQsjIiF7WjYzFpTGXYVm/DvAvdGxFcaSjcAi8vLi4Hr29+embXLaHbjTwTOAe6WtKq87jPAJcDVkj4MPAq8uzMtjn0DAwO1lq8aWtu1a1fT2lgefqp63FVDb5MnT25nO2NeZdgj4lag2Rv0N7W3HTPrlLH7sm9m+8RhN8uEw26WCYfdLBMOu1kmHHazTPhU0l1QNc5eNZ5cdShonVNN91LVdwCqTiVdNc5+9NFHN62tWrWqaW1/5S27WSYcdrNMOOxmmXDYzTLhsJtlwmE3y4TDbpYJj7N3wbx582otXzUenRqnTx3rDp0/TXWq96reqr4/UPX9g02bNiXrufGW3SwTDrtZJhx2s0w47GaZcNjNMuGwm2XCYTfLhMfZu2D79u3J+oQJE5L1qrHu1Fh51Vh11THjVePwVVLHnFfdd9U4/NSpU5P1NWvWJOu58ZbdLBMOu1kmHHazTDjsZplw2M0y4bCbZcJhN8tE5Ti7pMOBy4E5wC5gaUR8TdLFwEeAJ8ubfiYilneq0bHstttuS9aPPfbYZH3atGnJ+vPPP7/PPe1W95jxuse7p8ydOzdZr/qOwH333dfOdsa80XypZgj4VETcKekg4A5JK8raVyPi3zrXnpm1S2XYI2IdsK68/Jyke4H5nW7MzNprn96zSzoSeA3w6/KqcyXdJelSSdObLLNE0kpJK2t1ama1jDrskqYC1wKfjIhngW8BRwEnUGz5vzzSchGxNCIWRsTCNvRrZi0aVdglTaAI+g8i4icAEbEhInZGxC7g28CizrVpZnVVhl3Fx7XfBe6NiK80XN/4UenpwOr2t2dm7aKqoRNJbwB+CdxNMfQG8BngLIpd+AAeAT5afpiXuq/OjdOMYZMmTUrWTz755GR95syZTWtTpkxJLlt1mGnV0FuV1Kmkq4bOHn/88WT95ptvTta3bduWrO+vImLE8dTRfBp/KzDSwh5TNxtD/A06s0w47GaZcNjNMuGwm2XCYTfLhMNulonKcfa2rizTcfaqw0g7+W8wODiYrM+ZMydZP/jgg2utf/369S3VoPoU3FVSz3s3/993W7Nxdm/ZzTLhsJtlwmE3y4TDbpYJh90sEw67WSYcdrNMdHuc/UmgcR7dmcCmrjWwb/q1t37tC9xbq9rZ28si4tCRCl0N+14rl1b267np+rW3fu0L3FurutWbd+PNMuGwm2Wi12Ff2uP1p/Rrb/3aF7i3VnWlt56+Zzez7un1lt3MusRhN8tET8Iu6RRJf5D0gKQLetFDM5IekXS3pFW9np+unENvo6TVDdcNSloh6f7y94hz7PWot4slPV4+d6skndqj3g6XdLOkeyXdI+kT5fU9fe4SfXXleev6e3ZJ44D7gLcAa4HbgbMi4nddbaQJSY8ACyOi51/AkPRGYCtweUS8qrzuX4CnI+KS8oVyekT8fZ/0djGwtdfTeJezFc1tnGYceBfwAXr43CX6OoMuPG+92LIvAh6IiIci4kXgR8BpPeij70XELcDTw64+DVhWXl5G8Z+l65r01hciYl1E3Flefg7YPc14T5+7RF9d0Yuwzwcea/h7Lf0133sAN0q6Q9KSXjczgtm7p9kqf8/qcT/DVU7j3U3Dphnvm+eulenP6+pF2Ec6P1Y/jf+dGBF/DrwN+Hi5u2qjM6ppvLtlhGnG+0Kr05/X1YuwrwUOb/j7MOCJHvQxooh4ovy9EbiO/puKesPuGXTL3xt73M+f9NM03iNNM04fPHe9nP68F2G/HThG0gJJA8B7gBt60MdeJE0pPzhB0hTgrfTfVNQ3AIvLy4uB63vYyx76ZRrvZtOM0+PnrufTn0dE13+AUyk+kX8QuKgXPTTp6+XAb8ufe3rdG3AlxW7dDoo9og8DM4CbgPvL34N91NsVFFN730URrLk96u0NFG8N7wJWlT+n9vq5S/TVlefNX5c1y4S/QWeWCYfdLBMOu1kmHHazTDjsZplw2M0y4bCbZeL/AUt4U8592D7CAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAEICAYAAACZA4KlAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAWeElEQVR4nO3de7DcZX3H8fcnIXdC7oQkBBJDLDIUsRMzKkjpoHLRiowDCtZidYzj6HipDKKONdXaMp2q2BlxGpXKxSJ4K1CYFsxwKcoAkUIIIHJJAicccw9JSEJI8u0f+4tuDmef5+TsnrN78nxeM2fOnv2eZ3/f3eR7fr/d7+/5PYoIzOzQN6zdCZjZ4HCxmxXCxW5WCBe7WSFc7GaFcLGbFcLF3mKSHpN0+iBs5zxJz0vaLukNTTxOSDquQewDkm7vf5bWSeQ++9Ah6SLgXRFxkaRngL+NiJuafMwA5kfE0y1JsvdtTAVuAo4HhgNPAJdExK8Gapv2aoe1OwE7KOcAt1W3jwUea2MuB2M78GHgKSCAc4FbJB0ZEXvamllBfBjfYpJWSXqbpMWSfiLpOknbJD0q6bWSviBpXXUI/o66cXMl3VP97i8lfUfSdXXxYcDbgTskbae2h3yk2sMj6TJJz1TjH5d0Xt3Y4yTdLelFSRsk3dAj7bdJekrS5mq7qsZ9SNK9dY/zFkkPVo/zoKS31MXukvQ1Sb+qcri92qMTEbsi4smI2AcI2AtMAiY3eA3vkvRPkh6otnWTpMlVbE711uNiSc9Vz+dLdWPHSLq6ei5PSLpUUtdB/jMemiLCXy38AlYBbwMWA7uAM6kdQV0DrAS+BIwAPgqsrBt3H/AvwEjgVGArcF1d/E3AfXU/B3Bc3c/nAzOp/QF/H/ASMKOKXV9tdxgwGji1x+P8FzAROAZYD5xVxT4E3FvdngxsBj5YPZ8Lq5+nVPG7gGeA1wJjqp8v7/HaLAd2V9v8XuI1vAtYA5wIjAN+tv+1AObsH19t5/XAy8DrqvjlwN3U/pgcXW2zq93/Lzrhy3v2gfW/EfE/UTtU/QkwjVoBvAL8GJgjaaKkY4A3An8XEbsj4l7g5h6P9U7+eAj/KhHxk4h4ISL2RcQN1A6ZF1bhV6gd9s+M2l723h7DL4+ILRHxHHAncHIvm3gn8FREXBsReyLieuC3wF/W/c6/R8TvImIncGPPx4mIk4AjgIuAnjn0dG1ErIiIl4AvAxdIGl4X//uI2BkRjwCPUCt6gAuAf4yIzRHRBfxrZjvFcLEPrLV1t3cCGyJib93PAIdT2yNviogddb//fI/Hqn+//iqS/lrSw5K2SNpCba84tQpfSu3w+YGqW/DhHsN/X3d7R5VTTzOB1T3uWw3MOpjHqf7YXA9cJun1PeN16p//ampHQ1Pr7mu0rZk9xvZ8HYvlYu8M3cBkSWPr7pu9/4ako4AZwEO9DZZ0LLXD2k9SO6yeCKygVuBExO8j4qMRMRP4GHBlo3ZbwgvUjg7qHUPtcLs/RgCvScRn190+htrRyYY+PG43tcP33h6naC72DhARq4FlwGJJIyW9mQMPj88B/juqN6W9GEftfex6AEl/Q23PTvXz+ZL2F8Dm6nf39nyQjNuA10q6SNJhkt4HnEDt/X6SpDdJOrV6bmMkfR6YDtyfGPZXkk6o/gB+Ffhp3VFRyo3AFyRNkjSL2h9Aw8XeST4AvBnYCPwDcAO1D54gcwgfEY8D36D2Id9a4E+B+h72G4H7q0/xbwY+HRErDya5iNgIvAv4XJXjpdR6/n3Z244CvlONW1M9n3dGxAvwh5N3erYRrwV+SO1wfTTwqT6m+lWgi9qHob8EfsofX8ei+aSaDlW1x34LfI3af/h5EfFie7MaHJLuovbp+/db8FgfB94fEX/edGJDnPfsHULSGyXNkzRM0lnUTjz5T2otry+XUujNkjRD0inV6/gn1I5EftHuvDqBz6DrHEcBPwemUDsM/XhE/F8V+27bshp6RgL/BswFtlBrcV7Z1ow6hA/jzQrhw3izQgzqYXw1w6o4I0eOTMbHjx+fjE+cODEZ37On8VySjRs3Jsfu2LEjGR89enQyPmnSpGT8iCOOaBjbt29fcmwu9w0b+tIIKE9EqLf7myr26oOkb1OblPH9iLi8mcc7VM2cOTMZP/3005Pxc889NxlPFcV1113XMAbw0EO9nqfzB8cff3wy/t73vjcZP+OMMxrGcn9ocrkvWbIkGbcD9fswvjpP+TvA2dROrrhQ0gmtSszMWquZ9+wLgacj4tmI2E3tU8/0LsjM2qaZYp/FgZMMujhwUgQAkhZJWiZpWRPbMrMmNfOevbcPAV71AVxELAGWQLkf0Jl1gmb27F0cOKPoaGozo8ysAzVT7A8C86vLKY0E3s+rL7hgZh2iqTPoJJ0DXEGt9XZVRHw98/tD9jD+7LPPbhj77Gc/mxy7c+fOZDzXh9+1a1cynurTn3jiiQ1jANOnT0/GV61alYynevwA3d3dDWMvvpg+3X/UqFHJ+KxZr/qI6ABLly5tGPvUp/o6iW7oGZA+e0TcRmLqpZl1Dp8ua1YIF7tZIVzsZoVwsZsVwsVuVggXu1khBvVKNZ3cZ583b14yvnjx4oaxtWvXNowBjB07NhkfNiz9Nzc37zvV6549u7nLpue2nYuneum5Hv0rr7ySjG/atCkZT/Xht2zZkhx7ySWXJOOdrFGf3Xt2s0K42M0K4WI3K4SL3awQLnazQrjYzQrh1lvlyivTi4akppnm2k+HH97bcud/lLtcc65FlbpKa25sbpppLrfcc89NU03Zuze9aGvuuaX+zXJTf6+55ppk/NZbb03G28mtN7PCudjNCuFiNyuEi92sEC52s0K42M0K4WI3K4T77JWFCxcm46nLRa9fvz45dvPmzcl4bsnm3FTPlN27dyfjueWgc7Zu3ZqM5/rwzcg9twkTJvT7sT3F1cyGLBe7WSFc7GaFcLGbFcLFblYIF7tZIVzsZoVoahXXQ8kDDzyQjN93330NY+9+97uTY++///5k/LDD0v8MuUtRb9y4sWEs14vesGFDMp5bLjqXW+q55Xr006ZNS8ZzUrlddtllTT32UNRUsUtaBWwD9gJ7ImJBK5Iys9ZrxZ79LyIivXsws7bze3azQjRb7AHcLuk3khb19guSFklaJmlZk9sysyY0exh/SkS8IOlI4A5Jv42Ie+p/ISKWAEugsyfCmB3qmtqzR8QL1fd1wC+A9NQxM2ubfhe7pHGSxu+/DbwDWNGqxMystfo9n13Sa6jtzaH2duA/IuLrmTGH5GH8M888k4zffffdyXhuPnxuTvj27dsbxrZt25YcmzN8+PBkPDfXPtVnHzFiRHJsroefm69+5513NozdcsstybFDWaP57P1+zx4RzwKv73dGZjao3HozK4SL3awQLnazQrjYzQrhYjcrhKe4VnLTTFPLA5966qnJsV//erIjmZVakhnSuY0ZMyY5dufOncl47nXJxV9++eWGsWHDmtvX5MYfyu21/vCe3awQLnazQrjYzQrhYjcrhIvdrBAudrNCuNjNCuE+eyXVq87p7u5OxnNTYOfOnZuM5y7nnJrGmpsem3vsXC87Nb0W0peDzr3muW2vXr06GbcDec9uVggXu1khXOxmhXCxmxXCxW5WCBe7WSFc7GaFcJ99EOT6xePHj0/Gc73yUaNGNYzllkUeOXJkMp7rw+eWhE5p5twGgHXr1jU1vjTes5sVwsVuVggXu1khXOxmhXCxmxXCxW5WCBe7WSHcZ++jVK881wfv6upKxk866aR+bxvS12bPLcmdWzZ57969yfjo0aOT8dR16XM9/KlTpybja9asScZTmlknYKjK7tklXSVpnaQVdfdNlnSHpKeq75MGNk0za1ZfDuN/CJzV477LgKURMR9YWv1sZh0sW+wRcQ+wqcfd5wJXV7evBt7T4rzMrMX6+559ekR0A0REt6QjG/2ipEXAon5ux8xaZMA/oIuIJcASAEnpT4vMbMD0t/W2VtIMgOq7px+Zdbj+FvvNwMXV7YuBm1qTjpkNlOxhvKTrgdOBqZK6gK8AlwM3SvoI8Bxw/kAmOdStWrUqGc/10XNzzidNatz5zG0710+eMmVKMr558+Z+P37q/ADIvy6HYi98IGWLPSIubBA6o8W5mNkA8umyZoVwsZsVwsVuVggXu1khXOxmhfAU10GQmuYJ+SmyOanxw4cPT47NTVHN5ZZrvaWmqeYuoZ2Tm55rB/Ke3awQLnazQrjYzQrhYjcrhIvdrBAudrNCuNjNCuE+ex810wvPTcVcv359Mp5bFjnX625mbG7bY8aMScZTyypPmzYtOXb79u3JuB0c79nNCuFiNyuEi92sEC52s0K42M0K4WI3K4SL3awQ7rP3UTNLNufmbacuBQ2wY8eOZHzy5MnJeMqGDRuS8bFjxybjEyZMSMZzffoUScn4scce2+/HLvEy1N6zmxXCxW5WCBe7WSFc7GaFcLGbFcLFblYIF7tZIdxn76Nm5rPn5quvWLEiGX/++eeT8VQvfNeuXcmx06dPT8ZzffLcktCp7ed69N3d3cn4zJkzk3E7UHbPLukqSeskrai7b7GkNZIerr7OGdg0zaxZfTmM/yFwVi/3fysiTq6+bmttWmbWatlij4h7gE2DkIuZDaBmPqD7pKTl1WF+w5O7JS2StEzSsia2ZWZN6m+xfxeYB5wMdAPfaPSLEbEkIhZExIJ+bsvMWqBfxR4RayNib0TsA74HLGxtWmbWav0qdkkz6n48D0j3jsys7bJ9dknXA6cDUyV1AV8BTpd0MhDAKuBjA5jjkPfWt741GX/22WeT8dWrVyfjqV721q1bk2OPOOKIZDzXC8+tPZ/q08+YMaNhrC+OOuqoZPzII49sGEtdzx7S1y+A5s67aJdssUfEhb3c/YMByMXMBpBPlzUrhIvdrBAudrNCuNjNCuFiNyuEImLwNiYN3sYOUjOtltmzZyfHXnrppcl4rvWWm6Y6derUhrGnn346OXbcuHHJ+Ny5c5PxLVu2JOO51l4zctNvt23b1jB2xRVXtDqdjhERvV6D23t2s0K42M0K4WI3K4SL3awQLnazQrjYzQrhYjcrhC8lXWlmyuKZZ56ZjD/++OPJ+OjRo5Px3DTVOXPmNIytWbMmOfb4449PxnOvS1dXVzJ+0kknNYytXbs2OXbKlCnJ+ObNm5PxWbNmNYwdd9xxybG58xOGIu/ZzQrhYjcrhIvdrBAudrNCuNjNCuFiNyuEi92sEO6zt0CqlwywfPnyZHz48OHJ+MiRI5PxUaNGJePNbDsn14dPxXPz9HPXCcidf5CKp85NAPfZzWwIc7GbFcLFblYIF7tZIVzsZoVwsZsVwsVuVoi+LNk8G7gGOArYByyJiG9LmgzcAMyhtmzzBRGRnmA8hKX6st3d3cmxufnq27dvT8YPOyz9z7Rnz56GsTFjxiTH5qQeG/J99mbOAdixY0cyPn369GQ8NZd/2rRp/cppKOvLnn0P8LmIeB3wJuATkk4ALgOWRsR8YGn1s5l1qGyxR0R3RDxU3d4GPAHMAs4Frq5+7WrgPQOVpJk176Des0uaA7wBuB+YHhHdUPuDABzZ6uTMrHX6fG68pMOBnwGfiYitUq/LSfU2bhGwqH/pmVmr9GnPLmkEtUL/UUT8vLp7raQZVXwGsK63sRGxJCIWRMSCViRsZv2TLXbVduE/AJ6IiG/WhW4GLq5uXwzc1Pr0zKxV+nIYfwrwQeBRSQ9X930RuBy4UdJHgOeA8wcmxc5wzDHHNIzl2k+51lluCmuudbd3795+bztn0qRJyXiuNZfafi63lStXJuPz589PxlOXqp4wYUJy7OTJk5PxTZs2JeOdKPs/ISLuBRq9QT+jtemY2UDxGXRmhXCxmxXCxW5WCBe7WSFc7GaFcLGbFcKXku6j1CWXhw1L/83MTdUcO3ZsMj5ixIhkfPfu3Q1juXMAIiIZP/zww5PxXJ/95ZdfbhhLLakMsGzZsmT8tNNOS8ZTU49zPf7c+QVDsc/uPbtZIVzsZoVwsZsVwsVuVggXu1khXOxmhXCxmxXCffY+mjp1asNYbj76+vXrk/ETTzwxGc/NZ08tTZzLLdcnHz9+fDKee/zUssy5pa5vvfXWZHzLli3JeCq3XB+92esAdCLv2c0K4WI3K4SL3awQLnazQrjYzQrhYjcrhIvdrBCHXjNxgKT67Ln57Bs3bkzGc9cwz/V8U/O2c33wzZvTq2y/9NJLyXjuuTcjt5R1LvfUXP7c85oxY0Yy/uSTTybjnch7drNCuNjNCuFiNyuEi92sEC52s0K42M0K4WI3K0S2zy5pNnANcBSwD1gSEd+WtBj4KLB/svYXI+K2gUq03VLXT89dFz43dzonN589dd34XI9+2rRpyXhuLv64ceP6/fipcxcA5s2bl4znromfOgcgNzY3j38o6stJNXuAz0XEQ5LGA7+RdEcV+1ZE/MvApWdmrZIt9ojoBrqr29skPQGkl/Iws45zUO/ZJc0B3gDcX931SUnLJV0lqddjVUmLJC2TlF7Lx8wGVJ+LXdLhwM+Az0TEVuC7wDzgZGp7/m/0Ni4ilkTEgohY0IJ8zayf+lTskkZQK/QfRcTPASJibUTsjYh9wPeAhQOXppk1K1vskgT8AHgiIr5Zd3/9tKDzgBWtT8/MWqUvn8afAnwQeFTSw9V9XwQulHQyEMAq4GMDkmGHmD9/fsPYypUrk2NzrbOc3DTS1JLPqUs5A/z6179Oxi+66KJkPNfaW7p0acNY7nnl4hMnTkzGU9NYc/9md955ZzI+FPXl0/h7AfUSOmR76maHIp9BZ1YIF7tZIVzsZoVwsZsVwsVuVggXu1khFBGDtzFp8DbWYql+cm7Z41y/ODfdMjfVc/Xq1Q1jRx99dHLsqlWrknEbeiKit1a59+xmpXCxmxXCxW5WCBe7WSFc7GaFcLGbFcLFblaIwe6zrwfqm8JTgQ2DlsDB6dTcOjUvcG791crcjo2IXq/fPajF/qqNS8s69dp0nZpbp+YFzq2/Bis3H8abFcLFblaIdhf7kjZvP6VTc+vUvMC59deg5NbW9+xmNnjavWc3s0HiYjcrRFuKXdJZkp6U9LSky9qRQyOSVkl6VNLD7V6frlpDb52kFXX3TZZ0h6Snqu/NrQfd2twWS1pTvXYPSzqnTbnNlnSnpCckPSbp09X9bX3tEnkNyus26O/ZJQ0Hfge8HegCHgQujIjHBzWRBiStAhZERNtPwJB0GrAduCYiTqzu+2dgU0RcXv2hnBQRn++Q3BYD29u9jHe1WtGM+mXGgfcAH6KNr10irwsYhNetHXv2hcDTEfFsROwGfgyc24Y8Ol5E3ANs6nH3ucDV1e2rqf1nGXQNcusIEdEdEQ9Vt7cB+5cZb+trl8hrULSj2GcBz9f93EVnrfcewO2SfiNpUbuT6cX0iOiG2n8e4Mg259NTdhnvwdRjmfGOee36s/x5s9pR7L1dH6uT+n+nRMSfAWcDn6gOV61v+rSM92DpZZnxjtDf5c+b1Y5i7wJm1/18NPBCG/LoVUS8UH1fB/yCzluKeu3+FXSr7+vanM8fdNIy3r0tM04HvHbtXP68HcX+IDBf0lxJI4H3Aze3IY9XkTSu+uAESeOAd9B5S1HfDFxc3b4YuKmNuRygU5bxbrTMOG1+7dq+/HlEDPoXcA61T+SfAb7Ujhwa5PUa4JHq67F25wZcT+2w7hVqR0QfAaYAS4Gnqu+TOyi3a4FHgeXUCmtGm3I7ldpbw+XAw9XXOe1+7RJ5Dcrr5tNlzQrhM+jMCuFiNyuEi92sEC52s0K42M0K4WI3K4SL3awQ/w+GFZcWT93OqgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAEICAYAAACZA4KlAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAUZUlEQVR4nO3dfbBcdX3H8feHPAAhSBNCHiDopTFgQCDUmOGxpKKAKINMBys+RYvEocLoDB2L2Fa0dko7PoxtU9tYUXwA1EEKRdBgBpuCjhJjJOGhAiGRkCdDAkkwTzf59o890c3l7u/cu2fv7nJ/n9fMnbt3v/s753t37uees/vbc44iAjMb/g7qdANm1h4Ou1kmHHazTDjsZplw2M0y4bCbZcJhbzFJj0ia04b1XCrpGUnbJZ1WYTkh6dUNau+StLD5Lq2bOOwtFhEnRcSPhmLZkt4p6Zbix88AV0fE2Ij4xVCsLyK+GRHnt3KZkuYW/2A+0MrlWjmH/eXlIuCe4vargEc62MugSRoHfIyXWd/DhcPeYpJWSXqjpBskfUfSNyRtk7Rc0vGSPiZpY7ELfn7duOMkLS4e+0NJ8yV9o65+EPAm4D5J24ERwC8lPVXUr5P0VDH+UUmX1o19taT/kfSCpE2SvtWn7TdKekLSlmK9Ksa9T9IDdcs5U9JDxXIeknRmXe1Hkv5O0oNFDwslTeiznn8A/hnYNIDn8GPF77FF0lckHVLU5khaI+na4nlcJ+n9dWOPlPTfkrYWPX66/nfImcM+tC4Gvg6MA34B/IDac34M8CngP+oeewvwM+BI4AbgPX2WNRtYGREbImJscd+pETGtuP0UcA5wBPBJ4BuSphS1vwMWFn1MBf6lz7LfCrweOBV4O3BB319E0njge9TCeiTwOeB7ko6se9g7gfcDE4HRwF/WjZ8NzAL+ve+yG3hX0cc04Hjgr+tqk4vf8xjgCmB+sdcAMB94sXjM3OLLcNiH2v9GxA8iohf4DnAUcGNE7AFuA3ok/YGkV1IL299GxO6IeAC4q8+y3sLvd+FfIiK+ExFrI2JfRHwLeILaPwiAPdR2+4+OiJ3F8uvdGBHPR8SvgfuBmf2s4i3AExHx9YjojYhbgcep/UPb7ysR8auI2AF8e/9yJI0A/g24JiL2JZ6vev8aEc9ExGbg74HL62p7gE9FxJ6IuAfYDpxQrOdPgU9ExG8j4lHg5gGub9hz2IfWhrrbO4BNEbG37meAscDRwOaI+G3d45/ps6z61+svIem9kpZJel7S88Brgf270R8FBPysmC348z7D19fd/m3RU19HA6v73Lea2ta1bDl/ATwcET9p1H8/6n//1cX693uu+Afad11HASP7jO37PGZrZKcbMADWAeMljakL/LH7i5ImA1OApf0NlvQq4EvAecBPImKvpGXUAk5ErAeuLB57NvBDSYsj4slB9LiW2t5BvVcC3x/A2POAcyVdVPw8HjhN0syIuLrBmGPrbr+yWH+Z3wC91F6q/Kqf5WTNW/YuEBGrgSXADZJGSzqDA3ePLwK+H42PRz4MCGp/7BRvWL12f1HSZZKmFj9uKR67t+9CStwDHF9M/42U9GfAicDdAxj7PmAGtd36mdR+108CH0+M+ZCkqcV7BdcDfd9UfIlir+m71J7HMZJeA7x3AP1lwWHvHu8CzgCeAz5N7Y97V1FL7sIXr00/C/yE2kuHk4EH6x7yeuCnxbv4dwEfjoinB9NcRDxH7Y28a4sePwq8NSKS76wXY5+PiPX7v4DdwNaIeAFA0vWS7u0z7BZqbyquLL4+PcBWr6b25t16am+O3srvn8esySev6E7F9Njj1N5JXw9M2x+O4U7SKuADEfHDFizrH4HJEZH9u/LesncJSa+XNE3SQZIuBC4B/ova69u/ySXoVUl6jaRTVDOb2tTcHZ3uqxv4DbruMZna680jgTXAVXUfg/1ix7p6+Tmc2q770cBGai9v7uxoR13Cu/FmmfBuvFkm2robL8m7EfY7o0aNStb37NnTpk6Gl4hQf/dXCnvxRtIXqB2U8Z8RcWOV5VlejjrqqGR97dqBfI7GBqrp3fjic8jzgTdT+3DF5ZJObFVjZtZaVV6zzwaejIiVEbGb2oEdl7SmLTNrtSphP4YDDzJYw4EHRQAgaZ6kJZKWVFiXmVVU5TV7f28CvOQNuIhYACwAv0Fn1klVtuxrOPCIoqkM7MgkM+uAKmF/CJhenE5pNPAOXnrCBTPrEk3vxkdEr6SrqZ1qaQRwU0T4RIL9WLRoUbI+bty4ZP25555L1q+88sqGtVWrViXHVnX00Ucn6/fff3/D2qGHHpocu3p133NlHOjCCy9M1l988cVkPTeV5tmLUwI1PPTSzLqHPy5rlgmH3SwTDrtZJhx2s0w47GaZcNjNMuHTUrXBiBEjkvWyQz2nTp2arC9fvrxhbdu2bcmxt99+e7L+7ne/O1kv+9127tzZsPb8888nx77iFa9I1j2PPjjesptlwmE3y4TDbpYJh90sEw67WSYcdrNMeOqtDcoOUT3uuOMqjR8/fnzD2uTJk5Njr7nmmmT91FNPTdZPOeWUZH3Lli0NayNHpv/8yn5vGxxv2c0y4bCbZcJhN8uEw26WCYfdLBMOu1kmHHazTHievQ1WrlyZrJ9++unJem9vb7K+a9euhjWp36v3DljZqajPOeecZP3ZZ59tWCs7lfSYMWOSdRscb9nNMuGwm2XCYTfLhMNulgmH3SwTDrtZJhx2s0x4nr0NHn300WS97HTMZVKnVN69e3dybNnx6GV27NiRrKfm+cuOZ9+6dWtTPVn/KoVd0ipgG7AX6I2IWa1oysxarxVb9j+JiE0tWI6ZDSG/ZjfLRNWwB7BQ0s8lzevvAZLmSVoiaUnFdZlZBVV348+KiLWSJgL3SXo8IhbXPyAiFgALACRFxfWZWZMqbdkjYm3xfSNwBzC7FU2ZWes1HXZJh0k6fP9t4HxgRasaM7PWqrIbPwm4o5hHHQncEhHfb0lXw0zqmG6APXv2JOsHHZT+nzxq1KiGtXXr1iXHLl26NFkvu+Rz2e+W+gxB2bH2L7zwQrJug9N02CNiJZC+goCZdQ1PvZllwmE3y4TDbpYJh90sEw67WSZ8iGsbrF27Nlkvm3orm6Lat29fw9rOnTuTY8sOv01N60H5tGBq+uzggw9Ojq16Gmw7kLfsZplw2M0y4bCbZcJhN8uEw26WCYfdLBMOu1kmPM/eBps2pc/H2dPTk6w//vjjyXpqLr1srrrsdM5lyk5VnVr/3r17k2PLPn9gg+Mtu1kmHHazTDjsZplw2M0y4bCbZcJhN8uEw26WCc+zt8H69esrja9yKumysWUi0hfxKTvePTVXXjbHv2XLlmTdBsdbdrNMOOxmmXDYzTLhsJtlwmE3y4TDbpYJh90sE55n7wK7du2qNL5sLrzK2NQ56aH8mPRUvexY+61btybrNjilW3ZJN0naKGlF3X3jJd0n6Yni+7ihbdPMqhrIbvxXgQv73HcdsCgipgOLip/NrIuVhj0iFgOb+9x9CXBzcftm4G0t7svMWqzZ1+yTImIdQESskzSx0QMlzQPmNbkeM2uRIX+DLiIWAAsAJDX/TpKZVdLs1NsGSVMAiu8bW9eSmQ2FZsN+FzC3uD0XuLM17ZjZUCndjZd0KzAHmCBpDfAJ4Ebg25KuAH4NXDaUTQ53ZXPZVZTNo5fNdVe9RnpqfFlvL774YqV124FKwx4RlzcondfiXsxsCPnjsmaZcNjNMuGwm2XCYTfLhMNulgkf4toFqp7uOaVs6mzEiBGVll/We2p6rezw2IkTG34K25rgLbtZJhx2s0w47GaZcNjNMuGwm2XCYTfLhMNulgnPs3eBoTyMtGzZZfPkvb29Ta8b0pdlLlt2T09Psm6D4y27WSYcdrNMOOxmmXDYzTLhsJtlwmE3y4TDbpYJz7N3garz7Km58qGcwx+I1PHyZceze569tbxlN8uEw26WCYfdLBMOu1kmHHazTDjsZplw2M0y4Xn2Njj++OOT9dGjRyfrZZd0Th0zXqbsePaql3RO1cuOZ58wYUKyboNTumWXdJOkjZJW1N13g6RnJS0rvi4a2jbNrKqB7MZ/Fbiwn/s/HxEzi697WtuWmbVaadgjYjGwuQ29mNkQqvIG3dWSHi5288c1epCkeZKWSFpSYV1mVlGzYf8iMA2YCawDPtvogRGxICJmRcSsJtdlZi3QVNgjYkNE7I2IfcCXgNmtbcvMWq2psEuaUvfjpcCKRo81s+5QOkEr6VZgDjBB0hrgE8AcSTOBAFYBHxzCHl/2ZsyYkayvWbMmWd+zZ0+yPmrUqEH3tF/Z9dmH8lj7Xbt2JcdOmjQpWT/zzDOT9R//+MfJem5Kwx4Rl/dz95eHoBczG0L+uKxZJhx2s0w47GaZcNjNMuGwm2XCh7i2wXnnnZesR0SyXuUw1LJll6k6PjW1V7bsp556Klm/6qqrknVPvR3IW3azTDjsZplw2M0y4bCbZcJhN8uEw26WCYfdLBOeZ2+D008/PVkvO4S1ymGoZXPZVU5DPRCpzwgccsghybE7d+5M1s8444ymesqVt+xmmXDYzTLhsJtlwmE3y4TDbpYJh90sEw67WSY8z94GPT09yfqWLVuS9bLj2ascc142h1/1ePYq6x4zZkyyPnny5GT94IMPblgrO431cOQtu1kmHHazTDjsZplw2M0y4bCbZcJhN8uEw26WiYFcsvlY4GvAZGAfsCAiviBpPPAtoIfaZZvfHhHpCeNhaty4ccn6hAkTkvUNGzYk62XHfafmwssuuVw2j753795kvco57UePHp0cu3DhwmT9sssuS9Zf97rXNazleE75gWzZe4FrI2IGcDrwIUknAtcBiyJiOrCo+NnMulRp2CNiXUQsLW5vAx4DjgEuAW4uHnYz8LahatLMqhvUa3ZJPcBpwE+BSRGxDmr/EICJrW7OzFpnwJ+NlzQWuB34SERsLXstWDduHjCvufbMrFUGtGWXNIpa0L8ZEd8t7t4gaUpRnwJs7G9sRCyIiFkRMasVDZtZc0rDrtom/MvAYxHxubrSXcDc4vZc4M7Wt2dmrTKQ3fizgPcAyyUtK+67HrgR+LakK4BfA+l5kGFs5syZyXrZS56y6a0q02dlU2Nl03pl02P79u1L1lO99fb2JseecMIJyXrZabBnzJjRsJbj1Ftp2CPiAaDRX1v6wuNm1jX8CTqzTDjsZplw2M0y4bCbZcJhN8uEw26WCZ9KugUuvvjiZH3Tpk3Jetklm8vmslP1sWPHJseWzeGPGjUqWS+bp9+6dWvDWtnvXXaq6LJ5+pNPPjlZz4237GaZcNjNMuGwm2XCYTfLhMNulgmH3SwTDrtZJjzP3gLTpk1L1g8//PBkvWw+ueyY9M2bNze97LLPCNx9993J+o4dO5L11GWXt23blhxb5rDDDkvWTzrppErLH268ZTfLhMNulgmH3SwTDrtZJhx2s0w47GaZcNjNMuF59hYom4ueM2dOpeWXHc9+6KGHNr3s7du3Nz0Wyo8p3717d9PLLjuf/s6dO5P15cuXN73u4chbdrNMOOxmmXDYzTLhsJtlwmE3y4TDbpYJh90sE0pdPxtA0rHA14DJwD5gQUR8QdINwJXAb4qHXh8R95QsK72yYarsOS47rrvsvPOpue7p06cnx5577rnJ+uLFi5P1p59+Olk/4ogjGtbKfu+y8wCMGzcuWe/p6WlYW716dXLsy1lE9HsxgIF8qKYXuDYilko6HPi5pPuK2ucj4jOtatLMhk5p2CNiHbCuuL1N0mPAMUPdmJm11qBes0vqAU4DflrcdbWkhyXdJKnffSpJ8yQtkbSkUqdmVsmAwy5pLHA78JGI2Ap8EZgGzKS25f9sf+MiYkFEzIqIWS3o18yaNKCwSxpFLejfjIjvAkTEhojYGxH7gC8Bs4euTTOrqjTsql3m88vAYxHxubr7p9Q97FJgRevbM7NWGci78WcB7wGWS1pW3Hc9cLmkmUAAq4APDkmHw0DZpYOrHoq5a9eupsdOnDix0ronTZqUrKcOvx05Mv3nVzb1dsEFFyTrw3l6rRkDeTf+AaC/ebvknLqZdRd/gs4sEw67WSYcdrNMOOxmmXDYzTLhsJtlwqeSboMVK9KfN6p9bqmxs88+O1k/8cQTG9be8IY3JMc++OCDyXqZ+fPnJ+upefzbbrstOfbee+9tqifrn7fsZplw2M0y4bCbZcJhN8uEw26WCYfdLBMOu1kmSk8l3dKVSb8B6g8yngCkz5PcOd3aW7f2Be6tWa3s7VURcVR/hbaG/SUrl5Z067npurW3bu0L3Fuz2tWbd+PNMuGwm2Wi02Ff0OH1p3Rrb93aF7i3ZrWlt46+Zjez9un0lt3M2sRhN8tER8Iu6UJJ/yfpSUnXdaKHRiStkrRc0rJOX5+uuIbeRkkr6u4bL+k+SU8U39PXLW5vbzdIerZ47pZJuqhDvR0r6X5Jj0l6RNKHi/s7+twl+mrL89b21+ySRgC/At4ErAEeAi6PiEfb2kgDklYBsyKi4x/AkPTHwHbgaxHx2uK+fwI2R8SNxT/KcRHxV13S2w3A9k5fxru4WtGU+suMA28D3kcHn7tEX2+nDc9bJ7bss4EnI2JlROwGbgMu6UAfXS8iFgOb+9x9CXBzcftman8sbdegt64QEesiYmlxexuw/zLjHX3uEn21RSfCfgzwTN3Pa+iu670HsFDSzyXN63Qz/ZgUEeug9scDVLt+U+uVXsa7nfpcZrxrnrtmLn9eVSfC3t8J17pp/u+siPgj4M3Ah4rdVRuYAV3Gu136ucx4V2j28udVdSLsa4Bj636eCqztQB/9ioi1xfeNwB1036WoN+y/gm7xfWOH+/mdbrqMd3+XGacLnrtOXv68E2F/CJgu6ThJo4F3AHd1oI+XkHRY8cYJkg4Dzqf7LkV9FzC3uD0XuLODvRygWy7j3egy43T4uev45c8jou1fwEXU3pF/Cvh4J3po0NcfAr8svh7pdG/ArdR26/ZQ2yO6AjgSWAQ8UXwf30W9fR1YDjxMLVhTOtTb2dReGj4MLCu+Lur0c5foqy3Pmz8ua5YJf4LOLBMOu1kmHHazTDjsZplw2M0y4bCbZcJhN8vE/wO4IXYLVgpyYwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz to ./data\\MNIST\\raw\\train-images-idx3-ubyte.gz\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "9920512it [00:06, 1502767.47it/s]                             \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracting ./data\\MNIST\\raw\\train-images-idx3-ubyte.gz to ./data\\MNIST\\raw\n",
      "Downloading http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz to ./data\\MNIST\\raw\\train-labels-idx1-ubyte.gz\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "32768it [00:00, 76339.41it/s]            \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracting ./data\\MNIST\\raw\\train-labels-idx1-ubyte.gz to ./data\\MNIST\\raw\n",
      "Downloading http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz to ./data\\MNIST\\raw\\t10k-images-idx3-ubyte.gz\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "1654784it [00:01, 914677.27it/s]                              \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracting ./data\\MNIST\\raw\\t10k-images-idx3-ubyte.gz to ./data\\MNIST\\raw\n",
      "Downloading http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz to ./data\\MNIST\\raw\\t10k-labels-idx1-ubyte.gz\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "8192it [00:00, 14787.73it/s]            \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracting ./data\\MNIST\\raw\\t10k-labels-idx1-ubyte.gz to ./data\\MNIST\\raw\n",
      "Processing...\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\Dana\\Anaconda3\\lib\\site-packages\\torchvision\\datasets\\mnist.py:480: UserWarning: The given NumPy array is not writeable, and PyTorch does not support non-writeable tensors. This means you can write to the underlying (supposedly non-writeable) NumPy array using the tensor. You may want to copy the array to protect its data or make it writeable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at  ..\\torch\\csrc\\utils\\tensor_numpy.cpp:141.)\n",
      "  return torch.from_numpy(parsed.astype(m[2], copy=False)).view(*s)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Done!\n"
     ]
    }
   ],
   "source": [
    "\"\"\"\n",
    "\n",
    "Dataset for Images\n",
    "\n",
    "\"\"\"\n",
    "\n",
    "import torch\n",
    "from PIL import Image\n",
    "import pandas as pd\n",
    "import os\n",
    "from matplotlib.pyplot import imshow\n",
    "import matplotlib.pyplot as plt\n",
    "from torch.utils.data import Dataset, DataLoader\n",
    "import torchvision.transforms as transforms\n",
    "\n",
    "#We will use MNIST dataset, 28x28x1 images\n",
    "#The dataset has 10 labels or classes\n",
    "\n",
    "\n",
    "dir_path = os.path.dirname(os.path.realpath('__file__'))\n",
    "print(dir_path)\n",
    "\n",
    "csv_file = 'index.csv'\n",
    "csv_path = os.path.join(dir_path,csv_file)\n",
    "\n",
    "data_name = pd.read_csv((csv_path))\n",
    "data_name.head() #Lets view the dataframe\n",
    "\n",
    "print('File name:', data_name.iloc[0,1])\n",
    "print('class or y:', data_name.iloc[0,0])\n",
    "\n",
    "for i in range(5): #Lets plot all the images in the imageset\n",
    "    image_name = data_name.iloc[i,1] #we iterate through every image in the i'th row, 2nd column.\n",
    "    image_path = os.path.join(dir_path,image_name) #We set the image path \n",
    "    image = Image.open(image_path) #We open the image from its path\n",
    "    #Lets see the image\n",
    "    plt.imshow(image, cmap = 'gray', vmin = 0, vmax = 255)\n",
    "    plt.title(data_name.iloc[i,1])\n",
    "    plt.show()\n",
    "    \n",
    "\n",
    "#Lets create a dataset class\n",
    "\n",
    "class Dataset(Dataset):\n",
    "    def __init__(self, csv_file, data_dir, transform = None):\n",
    "        \n",
    "        self.transform = transform\n",
    "        self.data_dir = data_dir\n",
    "        data_dircsv_file = os.path.join(self.data_dir,csv_file)\n",
    "        self.data_name = pd.read_csv(data_dircsv_file)\n",
    "        self.len = self.data_name.shape[0]\n",
    "        \n",
    "    def __lef__(self):\n",
    "        return self.len\n",
    "    \n",
    "    def __getitem__(self, idx):\n",
    "        \n",
    "        img_name = os.path.join(self.data_dir, self.data_name.iloc[idx, 1])\n",
    "        image = Image.open(img_name)\n",
    "        \n",
    "        y = self.data_name.iloc[idx, 0]\n",
    "        \n",
    "        if self.transform:\n",
    "            image = self.transform(image)\n",
    "            return image, y\n",
    "\n",
    "\n",
    "#Lets use torchvision pre-built transforms for images\n",
    "transforms.CenterCrop(20) #Lets crop the image to 20x20\n",
    "transforms.ToTensor() #Converting the image to a tensor\n",
    "\n",
    "#Or we can combine the transforms\n",
    "#Lets compose the transforms\n",
    "croptensor_data_transform = transforms.Compose([transforms.CenterCrop(20), transforms.ToTensor()])\n",
    "\n",
    "#Now we apply the composed transforms to the dataset constructor\n",
    "dataset = Dataset(csv_file = csv_file, data_dir = dir_path, transform = croptensor_data_transform)\n",
    "dataset[0][0].shape #The dataset is 20x20x1\n",
    "\n",
    "\n",
    "\n",
    "#Lets try a prebuilt dataset from torchvision\n",
    "import torchvision.datasets as dsets\n",
    "\n",
    "#We creat the dataset object\n",
    "#Root is root directory of dataset, train parameter indicates if we want to use training or testing sets. \n",
    "#Download = True: downlaods the dataset into the directory. We set transform parameter to convert image to tensor.\n",
    "dataset = dsets.MNIST(root = './data', train = False, download = True, transform = transforms.ToTensor)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Differentiation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`torch.autograd` provides classes and functions implementing automatic differentiation of arbitrary scalar valued functions with ensor s for which gradients should be computed with the requires_grad=True\n",
    "\n",
    "[PyTorch docs about torch.autograd](https://pytorch.org/docs/stable/autograd.html)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## PyTorch Autograd\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "On setting .requires_grad = True they start forming a backward graph that tracks every operation applied on them to calculate the gradients using something called a dynamic computation graph (DCG) \n",
    "\n",
    "[PyTorch Autograd](https://towardsdatascience.com/pytorch-autograd-understanding-the-heart-of-pytorchs-magic-2686cd94ec95)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## torch.autograd.backward\n",
    "-------\n",
    "\n",
    "- **Sytanx**: torch.autograd.backward(tensors: Union[torch.Tensor, Sequence[torch.Tensor]], grad_tensors: Union[torch.Tensor, Sequence[torch.Tensor], None] = None, retain_graph: Optional[bool] = None, create_graph: bool = False, grad_variables: Union[torch.Tensor, Sequence[torch.Tensor], None] = None) → None\n",
    "\n",
    "\n",
    "- **Parameters** :\n",
    "    - tensors (sequence of Tensor) – Tensors of which the derivative will be computed.\n",
    "\n",
    "    - grad_tensors (sequence of (Tensor or None)) – The “vector” in the Jacobian-vector product, usually gradients w.r.t. each element of corresponding tensors. None values can be specified for scalar Tensors or ones that don’t require grad. If a None value would be acceptable for all grad_tensors, then this argument is optional.\n",
    "\n",
    "    - retain_graph (bool, optional) – If False, the graph used to compute the grad will be freed. Note that in nearly all cases setting this option to True is not needed and often can be worked around in a much more efficient way. Defaults to the value of create_graph.\n",
    "\n",
    "    - create_graph (bool, optional) – If True, graph of the derivative will be constructed, allowing to compute higher order derivative products. Defaults to False."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([1., 2., 3.], dtype=torch.float64, requires_grad=True)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import torch\n",
    "import numpy as np\n",
    "\n",
    "x = torch.randn(2, 2, requires_grad = True)\n",
    "# From numpy\n",
    "x = np.array([1., 2., 3.]) #Only Tensors of floating point dtype can require gradients\n",
    "x = torch.from_numpy(x)\n",
    "# Now enable gradient \n",
    "x.requires_grad_(True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> **above makes the change in-place (its a common pytorch thing)**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> *Note: By PyTorch’s design, gradients can only be calculated for floating point tensors which is why I’ve created a float type numpy array before making it a gradient enabled PyTorch tensor*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`ctx` is a context object that can be used to stash information \n",
    "for backward computation. You can cache arbitrary objects for \n",
    "use in the backward pass using the `ctx.save_for_backward` method"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(4., dtype=torch.float64)\n",
      "tensor(6., dtype=torch.float64)\n",
      "tensor(4., dtype=torch.float64)\n",
      "tensor(1., dtype=torch.float64)\n",
      "The tensor x:  tensor(2., requires_grad=True)\n",
      "The result of y = x^2:  tensor(4., grad_fn=<PowBackward0>)\n",
      "The dervative at x = 2:  tensor(4.)\n",
      "data: tensor(2.)\n",
      "grad_fn: None\n",
      "grad: tensor(4.)\n",
      "is_leaf: True\n",
      "requires_grad: True\n",
      "data: tensor(4.)\n",
      "grad_fn: <PowBackward0 object at 0x0000023A48F533C8>\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\Dana\\Anaconda3\\lib\\site-packages\\ipykernel_launcher.py:59: UserWarning: The .grad attribute of a Tensor that is not a leaf Tensor is being accessed. Its .grad attribute won't be populated during autograd.backward(). If you indeed want the gradient for a non-leaf Tensor, use .retain_grad() on the non-leaf Tensor. If you access the non-leaf Tensor by mistake, make sure you access the leaf Tensor instead. See github.com/pytorch/pytorch/pull/30531 for more informations.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "grad: None\n",
      "is_leaf: False\n",
      "requires_grad: True\n",
      "<torch.autograd.function.SQBackward object at 0x0000023A60E134A8>\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<SumBackward0 object at 0x0000023A60EEFE80>\n"
     ]
    }
   ],
   "source": [
    "\"\"\"\n",
    "\n",
    "Differentiation in Pytorch\n",
    "\n",
    "\"\"\"\n",
    "\n",
    "import torch\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pylab as plt\n",
    "\n",
    "#Evaluation of derivatives in torch\n",
    "x = torch.tensor(2, requires_grad=True, dtype=float) #requires_grad tells torch that x value will be used to evaluate functions\n",
    "y = x**2\n",
    "#To calculate derivative of y\n",
    "y.backward() #Backward function Calculates the derivative of y wrt. to x\n",
    "print(x.grad) \n",
    "\n",
    "\n",
    "#Lets create a new tensor z and evaluate its derivative wrt x\n",
    "x = torch.tensor(2, requires_grad=True, dtype=float)\n",
    "z = x**2 + 2*x + 1\n",
    "z.backward()\n",
    "print(x.grad)\n",
    "\n",
    "\n",
    "#Partial Derivatives\n",
    "u = torch.tensor(1, requires_grad=True, dtype=float)\n",
    "v = torch.tensor(2, requires_grad=True, dtype=float)\n",
    "f = u*v + u**2\n",
    "f.backward()\n",
    "print(u.grad)\n",
    "print(v.grad)\n",
    "\n",
    "\n",
    "##############################################################################\n",
    "\n",
    "# Create a tensor x\n",
    "x = torch.tensor(2.0, requires_grad = True)\n",
    "print(\"The tensor x: \", x)\n",
    "\n",
    "# Create a tensor y according to y = x^2\n",
    "y = x ** 2\n",
    "print(\"The result of y = x^2: \", y)\n",
    "\n",
    "# Take the derivative. Try to print out the derivative at the value x = 2\n",
    "y.backward()\n",
    "print(\"The dervative at x = 2: \", x.grad)\n",
    "\n",
    "#Below are the attributes of x and y that torch creates. \n",
    "print('data:',x.data)\n",
    "print('grad_fn:',x.grad_fn)\n",
    "print('grad:',x.grad)\n",
    "print(\"is_leaf:\",x.is_leaf)\n",
    "print(\"requires_grad:\",x.requires_grad)\n",
    "\n",
    "print('data:',y.data)\n",
    "print('grad_fn:',y.grad_fn)\n",
    "print('grad:',y.grad)\n",
    "print(\"is_leaf:\",y.is_leaf)\n",
    "print(\"requires_grad:\",y.requires_grad)\n",
    "\n",
    "\n",
    "#We can implement our own custom autograd Functions by subclassing torch.autograd.Function \n",
    "#And implementing the forward and backward passes which operate on Tensors\n",
    "\n",
    "class SQ(torch.autograd.Function):\n",
    "\n",
    "\n",
    "    @staticmethod\n",
    "    def forward(ctx,i):\n",
    "        \"\"\"\n",
    "        In the forward pass we receive a Tensor containing the input and return\n",
    "        a Tensor containing the output. ctx is a context object that can be used\n",
    "        to stash information for backward computation. You can cache arbitrary\n",
    "        objects for use in the backward pass using the ctx.save_for_backward method.\n",
    "        \"\"\"\n",
    "        result=i**2\n",
    "        ctx.save_for_backward(i)\n",
    "        return result\n",
    "\n",
    "    @staticmethod\n",
    "    def backward(ctx, grad_output):\n",
    "        \"\"\"\n",
    "        In the backward pass we receive a Tensor containing the gradient of the loss\n",
    "        with respect to the output, and we need to compute the gradient of the loss\n",
    "        with respect to the input.\n",
    "        \"\"\"\n",
    "        i, = ctx.saved_tensors\n",
    "        grad_output = 2*i\n",
    "        return grad_output\n",
    "\n",
    "\n",
    "#Lets apply the function\n",
    "x=torch.tensor(2.0,requires_grad=True )\n",
    "sq=SQ.apply\n",
    "y=sq(x)\n",
    "y\n",
    "print(y.grad_fn)\n",
    "y.backward()\n",
    "x.grad\n",
    "\n",
    "\n",
    "# Calculate the derivative with multiple values\n",
    "x = torch.linspace(-10, 10, 10, requires_grad = True)\n",
    "Y = x ** 2\n",
    "y = torch.sum(x ** 2)\n",
    "\n",
    "# Take the derivative with respect to multiple value. Plot out the function and its derivative\n",
    "y.backward()\n",
    "#The method detach()excludes further tracking of operations in the graph, and therefore the subgraph will not record operations\n",
    "plt.plot(x.detach().numpy(), Y.detach().numpy(), label = 'function')\n",
    "plt.plot(x.detach().numpy(), x.grad.detach().numpy(), label = 'derivative')\n",
    "plt.xlabel('x')\n",
    "plt.legend()\n",
    "plt.show()\n",
    "\n",
    "# Take the derivative of Relu with respect to multiple value. Plot out the function and its derivative\n",
    "x = torch.linspace(-10, 10, 1000, requires_grad = True)\n",
    "Y = torch.relu(x)\n",
    "y = Y.sum()\n",
    "y.backward()\n",
    "plt.plot(x.detach().numpy(), Y.detach().numpy(), label = 'function')\n",
    "plt.plot(x.detach().numpy(), x.grad.detach().numpy(), label = 'derivative')\n",
    "plt.xlabel('x')\n",
    "plt.legend()\n",
    "plt.show()\n",
    "print(y.grad_fn)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> **The `ctx.save_for_backward` method is used to store values generated during forward() that will be needed later when performing backward(). The saved values can be accessed during backward() from the ctx.saved_tensors attribute.**\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Referance**\n",
    "\n",
    "[Trying to understand what “save_for_backward” is in Pytorch question on stackoverflow](https://stackoverflow.com/questions/64460017/trying-to-understand-what-save-for-backward-is-in-pytorch)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "Tensors.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}