python-esces-notes.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "6164cdd3-7e82-4819-9fa9-fc8b3db3a0fd",
   "metadata": {},
   "source": [
    "# Variables\n",
    "* we can use python like a calculator"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4230963a-a720-4e93-8a49-38fe111cd6e1",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "3 + 5 * 4"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6dfe4583-5347-4f8d-a18e-41569ea4602f",
   "metadata": {},
   "source": [
    "* or we can assign things to variables"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4ebc61ab-b4a0-4d64-af2a-59052cea8b97",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "weight_kg = 60"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dd6b25ff-3ae1-44b3-ba0a-a933892fd3a7",
   "metadata": {},
   "source": [
    "* weight_kg is variable name\n",
    "* variable names are case sensitive, can include letters, digits and underscores\n",
    "* can't start with a digit\n",
    "\n",
    "* python has different data types\n",
    "* integers, floats, strings\n",
    "* automatically determined, 60 is an integer, 60.3 is a float"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "44b5d040-7e4e-4340-9511-6c93908559b5",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "weight_kg = 60.3"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3ba4e851-f94d-4237-8349-e33cbc48e059",
   "metadata": {},
   "source": [
    "* strings are surrouneded by single or double quotes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "be5f2e52-54e0-400f-99b8-7febe68a9cd6",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "patient_id = '001'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4f37ebd2-89f4-436b-8b79-c37d1f422641",
   "metadata": {},
   "source": [
    "* we can store the results of calculations in a variable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "abae2fb4-f5e7-460d-a4ed-e6b43645a255",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "weight_lb = 2.2 * weight_kg"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "03832d8f-64f4-425d-8c42-ed4ee64d67b7",
   "metadata": {},
   "source": [
    "* we can add a prefix to our patient id, this joins two strings together"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e348fac1-b88d-44aa-846c-16aa9e4efcd9",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "patient_id = 'inflam_' + patient_id"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d999bd34-6a9a-4c3e-8060-db45c538bad0",
   "metadata": {},
   "source": [
    "* functions do something with some data\n",
    "* lots of built in functions like print\n",
    "* brackets around parameters to the function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3fb4f87d-1bd3-4e39-816f-5392ef1ec3a2",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "print(weight_lb)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0a7e2bd5-675f-4b61-b625-dfd25fb1e366",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "print(patient_id)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "631fd37b-c0e5-44bd-bf11-4693abcfaf69",
   "metadata": {},
   "source": [
    "* display multiple things in one call to print by using commas\n",
    "* automatically adds spaces"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "aafbf1d0-854e-4b4e-8b11-43a8f1783f94",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "print(patient_id, 'weight in kilos:', weight_kg)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8508968d-6605-4397-ab04-d18a25bfac0a",
   "metadata": {},
   "source": [
    "* function calls can be nested\n",
    "* type function tells us what type a variable is"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d8d6695f-d00b-4cd5-ac59-e54d514d8318",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "print(type(60.3))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "96efda46-7841-4d04-b014-24fbb5790238",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "print(type(patient_id))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e09a1265-7fc7-4e9d-ab89-dd40951466e3",
   "metadata": {},
   "source": [
    "* we can do arithmetic inside a function call"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2aa19703-6738-4137-a4b0-671bd041def1",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "print('weight in pounds:', 2.2 * weight_kg)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f1574745-a71e-41be-ae18-b63507f297ce",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "print(weight_kg)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "63263610-2aa0-4c14-8bb1-c3a4a0411577",
   "metadata": {},
   "source": [
    "* we didn't change weight_kg\n",
    "* have to assign a new value with = "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0d3e7453-6a74-476f-9466-47797c9d945e",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "weight_kg = 65.0\n",
    "print('weight in kilos is now:', weight_kg)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d6417a3d-acd4-494e-876a-38d8edb77aec",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "weight_lb = 2.2 * weight_kg\n",
    "print('weight in kilos:', weight_kg, 'and in pounds:', weight_lb)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "65ce53d7-a381-430d-836c-eca0c777b3a8",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "weight_kg = 100.0\n",
    "print('weight in kilos is now:', weight_kg, 'and weight in pounds is still:', weight_lb)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5abddd43-d0c4-4d5a-8abd-7e277867ab79",
   "metadata": {
    "tags": []
   },
   "source": [
    "* think of variables like sticky notes\n",
    "* each has a name and value\n",
    "* changing one doesn't change others\n",
    "\n",
    "## Exercises"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ee895e84-b1a4-4107-b7c2-d5e564e8e102",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "mass = 47.5\n",
    "age = 122\n",
    "mass = mass * 2.0\n",
    "age = age - 20\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f72456ee-69ed-43a3-91b3-e308c0992708",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "first, second = 'Grace', 'Hopper'\n",
    "third, fourth = second, first\n",
    "print(third, fourth)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ea6a3af2-7407-475e-a3a6-5f0638469517",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "planet = 'Earth'\n",
    "apples = 5\n",
    "distance = 10.5\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6ba19f33-c26e-45a6-bc73-cf7804dc9b66",
   "metadata": {},
   "source": [
    "## recap\n",
    "\n",
    "* Basic data types in Python include integers, strings, and floating-point numbers.\n",
    "* Use variable = value to assign a value to a variable in order to record it in memory.\n",
    "* Variables are created on demand whenever a value is assigned to them.\n",
    "* Use print(something) to display the value of something.\n",
    "* Built-in functions are always available to use.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bbf900e8-8fdb-4057-9eb7-a51b5c8eddc5",
   "metadata": {},
   "source": [
    "# Analysing some wave-height data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "79789e73-50ca-4538-85d0-03a0f8519cd4",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import numpy"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5995a684-c288-4a15-9b24-1502c85a10e1",
   "metadata": {
    "tags": []
   },
   "source": [
    "* explain libraries, like taking equipment out of storage.\n",
    "* use import statement\n",
    "* libraryname.function name syntax"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a542b703-60a2-4a35-909e-be111d6b61d4",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "numpy.loadtxt(fname='data/wavesmonthly.csv', delimiter=',', skiprows=1)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d18b07b9-0a86-4f4b-b524-bef7e239101c",
   "metadata": {},
   "source": [
    "* data now loaded, only subset onscreen\n",
    "* explain parameters\n",
    "* skipped first row since it contains headers\n",
    "* we didn't save the data to a variable, correct that with variable assignment"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d6d440d8-dbd1-4084-8bc1-bb1c1963cec0",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "data = numpy.loadtxt(fname='data/wavesmonthly.csv', delimiter=',', skiprows=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0899ad57-9c6e-49be-9c48-092471c4568a",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "print(data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "04351116-49b8-456c-8f7d-f732d7baf0d6",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "print(type(data))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b84c74d7-4552-4aea-a73b-211baa0d1e88",
   "metadata": {},
   "source": [
    "* numpy arrays can contain multiple data types\n",
    "* the type function tells us the type of a variable\n",
    "* dtype tells us the type numpy is using within an array\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cb5bd0e7-441c-4525-b454-b05a0d395cfd",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "print(data.dtype)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "70b45941-e2cc-4187-9a19-7c9ac6549d7b",
   "metadata": {},
   "source": [
    "* find out the array's shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4dd20eb8-ecf2-44b4-a299-865a7550c969",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "print(data.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3c66b9f5-061a-4146-94c1-819d39ef6173",
   "metadata": {},
   "source": [
    "* that's 444 rows, 3 columns\n",
    "* 37 years x 12 months = 444\n",
    "* 3 columns are year, month, waveheight\n",
    "* get data from an array with [x,y]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b2d86dab-1f58-49d3-9593-0edd61400c4c",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "print('first value in data:', data[0, 2])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "59d62e00-307c-4317-8ab4-446e1c6ff5d9",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "print(\"middle value in data:\", data[222, 2])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9429d6d7-a506-49c1-b766-a20b0479de3e",
   "metadata": {},
   "source": [
    "* show diagram in notes about arrays\n",
    "* arrays are zero based indexing\n",
    "* 0,0 is top left corner, matrix notation not cartesian"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "35f10ab5-8ab0-4fdb-9cc6-422eeebe84e8",
   "metadata": {},
   "source": [
    "## slicing data\n",
    "* can select multiple items in an array with a :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b158096c-3ffa-4a73-9b6d-09978ef4692f",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "print(data[0:12, 0:3])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a73ca773-8db0-4a36-bcaf-bf2c20694d5c",
   "metadata": {},
   "source": [
    "* 0:12 means from zero to 11\n",
    "* takes some getting used to, I get it wrong all the time\n",
    "* slices can start from non-zero positions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "191c9e37-32ff-4956-8f56-492c795b7463",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "print(data[12:24, 1:3])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d4d1353a-31eb-4c24-9e33-05a95f760b06",
   "metadata": {},
   "source": [
    "* we can miss out the start or end if we want everything up to or everything after an item"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "734ab52c-8164-4a3e-8a3a-8395d57981cb",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "first_year = data[:12, 2:]\n",
    "print('data from first year is:')\n",
    "print(first_year)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "060ec5df-2f14-4a86-9393-bbadada89cf7",
   "metadata": {},
   "source": [
    "* numpy has functions which can operate on whole arrays\n",
    "* mean is one of these"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6dbf94ba-6e43-494e-96c2-c94b13c35fcf",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "print(numpy.mean(data))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cf5a9912-65c3-41ff-881f-3eb4cc4e3036",
   "metadata": {},
   "source": [
    "* mean takes an argument (data)\n",
    "* we can also slice data to just get the mean of a subset of the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5a2cd664-702d-4f32-93aa-640eccb88f28",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "print(numpy.mean(data[:,2]))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "90ea7022-334a-4249-88f9-420c0bfe7609",
   "metadata": {},
   "source": [
    "* Not all functions take an argument"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9f114b37-6c00-40d3-9e2d-56b0dca05252",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import time\n",
    "print(time.ctime())\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e3e1c525-e88f-4157-aecd-776cc74746b6",
   "metadata": {},
   "source": [
    "* ctime from the time library needs no arguments, still need the brackets\n",
    "* lets combine multiple assignment, numpy's min/max/std functions and slicing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1455f962-35e0-4c76-90ad-34aeabadcd2b",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "maxval, minval, stdval = numpy.max(data[:,2]), numpy.min(data[:,2]), numpy.std(data[:,2])\n",
    "\n",
    "print('Max wave height:', maxval)\n",
    "print('Min wave height:', minval)\n",
    "print('Wave height standard deviation:', stdval)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d0e7f7cc-7df1-4869-ac28-0aac581d2720",
   "metadata": {},
   "source": [
    "* don't use variable names with the same name as functions (e.g. min/max)\n",
    "* mystery functions"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "766e6b08",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "547f8bc7",
   "metadata": {},
   "source": [
    "** Potential issue **\n",
    "\n",
    "The what about NaNs section creates a new data variable, which overwrites the one we previously had.\n",
    "\n",
    "To avoid running into problems when doing the reshaping that comes next, what about modifying the name of the 'data' var defined here.\n",
    "\n",
    "* explain what NaNs are\n",
    "* numpy functions will return NaN for any dataset that contains one or more NaNs. Use the nan version of the function to avoid this."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a0378061",
   "metadata": {},
   "outputs": [],
   "source": [
    "data_nan = numpy.array([[1,2,3], [1, numpy.NaN, 3], [1, 2, 3]])\n",
    "numpy.mean(data_nan)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fda57208",
   "metadata": {},
   "outputs": [],
   "source": [
    "data_nan = numpy.array([[1,2,3],[1,numpy.NaN,3],[1,2,3]])\n",
    "numpy.nanmean(data_nan)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fe1c6171-86db-4e7f-a01c-522597beb74a",
   "metadata": {},
   "source": [
    "* numpy can tell us the \"shape\" of the data\n",
    "* sometimes the shape is wrong for the analysis we want to perform\n",
    "* numpy can reshape it, lets make each row a year and each column a month"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0583a5fb",
   "metadata": {},
   "outputs": [],
   "source": [
    "data_nan.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "69659b8e",
   "metadata": {},
   "outputs": [],
   "source": [
    "reshaped_data = numpy.reshape(data[:,2], [37,12])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a1855ffd",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"The shape of the reshaped data is:\")\n",
    "print(reshaped_data.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1d8f18f2",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(f\"The maximum value of the reshaped data is: {numpy.max(reshaped_data)}\")\n",
    "print(f\"The min value of the reshaped data is: {numpy.min(reshaped_data)}\")\n",
    "print(f\"The std dev of the reshaped data is: {numpy.std(reshaped_data)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "14aa57b8",
   "metadata": {},
   "source": [
    "**ISSUE**\n",
    "After the line \"We can verify that nothing about the data has changed:\"\n",
    "\n",
    "the name of the variable should be reshaped_data and not reshaped"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5ff7c01e",
   "metadata": {},
   "source": [
    "### Printing text\n",
    "\n",
    "* f means format, { } means interpreted by python"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3ab29367",
   "metadata": {},
   "outputs": [],
   "source": [
    "name = 'Chris'\n",
    "city = 'Edinburgh'\n",
    "print(f\"Hello, My name is {name} and I live in {city}.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7846dd4d-d95e-422d-9868-cef058541f59",
   "metadata": {},
   "source": [
    "* now print some reshaped data\n",
    "* explain comments\n",
    "* show reshaping image in notes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8b14d2b5",
   "metadata": {},
   "outputs": [],
   "source": [
    "year_0 = reshaped_data[0,:] # 0 on the first axis (rows), everything on the second (columns)\n",
    "print(f\"maximum wave height for year 0: {numpy.max(year_0)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "abdd07d0-27e8-475d-bd51-74aa367483e6",
   "metadata": {},
   "source": [
    "* we can specify which axis we want numpy functions to work across"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9866f832",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(numpy.mean(reshaped_data, axis=0))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bc7c9196-ca6c-49c1-9137-ed1b867da486",
   "metadata": {},
   "source": [
    "* check the shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "052ba48b",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(numpy.mean(reshaped_data, axis=0).shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ba3e6ba9",
   "metadata": {},
   "source": [
    "**Potential issue**\n",
    "\n",
    "Technically we don't have a Nx1 vector. A vector is a Nx1 matrix by definition, but what we have here is a 1D array. (12,) is different from (12,1) becasue the former is 1D and the latter is 2D. This is very important when you are doing matrix multiplications because the (12,) version is misleading and often leads to errors. Not sure if we just ignore this for now?\n",
    "\n",
    "yes ignore for now, but definitely a potential pitfall\n",
    "\n",
    "* get the average wave height for each year"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9065e0db",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(numpy.mean(reshaped_data, axis=1))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "67582729",
   "metadata": {},
   "source": [
    "### Slicing Strings exercise\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c2b8d80f",
   "metadata": {},
   "outputs": [],
   "source": [
    "element = 'oxygen'\n",
    "print('first three characters:', element[0:3])\n",
    "print('last three characters:', element[3:6])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3dc2fa00",
   "metadata": {},
   "outputs": [],
   "source": [
    "element = 'oxygen'\n",
    "print('last three characters:', element[-3:])\n",
    "element = 'carpentry'\n",
    "print('last three characters:', element[-3:])\n",
    "element = 'clone'\n",
    "print('last three characters:', element[-3:])\n",
    "element = 'hi'\n",
    "print('last three characters:', element[-3:])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d4ae80c8",
   "metadata": {},
   "outputs": [],
   "source": [
    "data[3:3, 4:4]\n",
    "\n",
    "reshaped_data.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b384ad4e",
   "metadata": {},
   "outputs": [],
   "source": [
    "data.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "97be9744",
   "metadata": {},
   "source": [
    "### Stacking arrays"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "64ee714a",
   "metadata": {},
   "outputs": [],
   "source": [
    "A = numpy.array([[1,2,3], [4,5,6], [7, 8, 9]])\n",
    "print('A = ') \n",
    "print(A)  \n",
    "B = numpy.hstack([A, A]) \n",
    "print('B = ') \n",
    "print(B)  \n",
    "C = numpy.vstack([A, A]) \n",
    "print('C = ') \n",
    "print(C)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ba7b4fa4",
   "metadata": {},
   "outputs": [],
   "source": [
    "D = numpy.hstack((A[:, :1], A[:, -1:]))\n",
    "print('D = ')\n",
    "print(D)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a5e8f75a",
   "metadata": {},
   "outputs": [],
   "source": [
    "D = numpy.delete(A, 1, 1)\n",
    "print('D = ')\n",
    "print(D) "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3f6ad05b",
   "metadata": {},
   "source": [
    "### Change In Wave Height"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0557756b",
   "metadata": {},
   "outputs": [],
   "source": [
    "year4 = reshaped_data[3, :]\n",
    "print(year4)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "534353a0",
   "metadata": {},
   "outputs": [],
   "source": [
    "numpy.diff(year4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "552e0510",
   "metadata": {},
   "outputs": [],
   "source": [
    "numpy.max(numpy.diff(reshaped_data, axis=1), axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "44e552bc",
   "metadata": {},
   "outputs": [],
   "source": [
    "numpy.max(numpy.absolute(numpy.diff(reshaped_data, axis=1)), axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "00f9e357",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "94d0a0f2-a6a9-4df7-a8a2-3619aa140283",
   "metadata": {},
   "source": [
    "* saving data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e224f57b-ba45-48f5-aee6-0242082942ce",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "numpy.savetxt(\"reshaped_data.csv\", reshaped_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "24c7b02f-1b0d-4222-b3db-719caaaade89",
   "metadata": {},
   "source": [
    "## Recap\n",
    "\n",
    "* Import a library into a program using import libraryname.\n",
    "* Use the numpy library to work with arrays in Python.\n",
    "* The expression array.shape gives the shape of an array.\n",
    "* Use array[x, y] to select a single element from a 2D array.\n",
    "* Array indices start at 0, not 1.\n",
    "* Use low:high to specify a slice that includes the indices from low to high-1.\n",
    "* Use # some kind of explanation to add comments to programs.\n",
    "* Use numpy.mean(array), numpy.max(array), and numpy.min(array) to calculate simple statistics.\n",
    "* Use numpy.mean(array, axis=0) or numpy.mean(array, axis=1) to calculate statistics across the specified axis."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e07247bb-900c-4739-a84c-840f02c4ee7a",
   "metadata": {},
   "source": [
    "# Visualising Tabular Data\n",
    "* best way to explore data can be to visualise it\n",
    "* python has a graphing library called matplotlib, based on matlab's graphing functions\n",
    "* we'll try to create a heatmap of our data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "359d2b3f-4ac4-4052-8c93-c1a4ffa5ca23",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "data = reshaped_data\n",
    "\n",
    "#following three lines only needed in new notebooks\n",
    "import numpy\n",
    "data = numpy.loadtxt(fname='data/wavesmonthly.csv', delimiter=',', skiprows=1)\n",
    "data = numpy.reshape(data[:,2], [37,12]) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7f96afe9-a0e8-4242-9c37-747a10e9c86c",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot\n",
    "image = matplotlib.pyplot.imshow(data)\n",
    "matplotlib.pyplot.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "612337cd-b10b-41fe-9a29-a4f45bf08f9e",
   "metadata": {},
   "source": [
    "* Each row is a year, each column is a month\n",
    "* blue is low, yellow is high, summer has more blues"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "050ac6fb-0f19-47ff-8f7c-04b794e3c5e9",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "ave_waveheight = numpy.mean(data, axis=0)\n",
    "ave_plot = matplotlib.pyplot.plot(ave_waveheight)\n",
    "matplotlib.pyplot.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "daee8fe4-3404-4b32-bf0f-1d3b98db3d27",
   "metadata": {},
   "source": [
    "* this shows average wave height for each month in a line graph\n",
    "* representing multiple decades of data in each point of the line\n",
    "* lets look at max and min as well as mean"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3fc7fb0e-3304-42f4-989f-175b508d1f3a",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "max_plot = matplotlib.pyplot.plot(numpy.max(data, axis=0))\n",
    "matplotlib.pyplot.show()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c219f591-16c0-4373-8e7d-a72245b7a322",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "max_plot = matplotlib.pyplot.plot(numpy.min(data, axis=0))\n",
    "matplotlib.pyplot.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4ceb590a-e7a3-4b1b-b2ec-c43ce16e6ac4",
   "metadata": {},
   "source": [
    "* less seasonal variation in the minimum\n",
    "* lots in the maximum, 7 metre waves!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "33232eab-c894-4820-8157-b81f938c34bb",
   "metadata": {},
   "source": [
    "* we can group several plots into one\n",
    "* multiple subplots in the main plot, each one has its own axes\n",
    "* label our axes\n",
    "\n",
    "*** WE SHOULD PUT UNITS HERE!!! ***"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b58c29d7-f16d-4573-9f93-70d706862a13",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "fig = matplotlib.pyplot.figure(figsize=(10.0, 3.0))\n",
    "\n",
    "axes1 = fig.add_subplot(1, 3, 1)\n",
    "axes2 = fig.add_subplot(1, 3, 2)\n",
    "axes3 = fig.add_subplot(1, 3, 3)\n",
    "\n",
    "axes1.set_ylabel('Average')\n",
    "axes1.set_xlabel('Year index')\n",
    "axes1.plot(numpy.mean(data, axis=1))\n",
    "\n",
    "axes2.set_ylabel('Max')\n",
    "axes2.set_xlabel('Year index')\n",
    "axes2.plot(numpy.max(data, axis=1))\n",
    "\n",
    "axes3.set_ylabel('Min')\n",
    "axes3.set_xlabel('Year index')\n",
    "axes3.plot(numpy.min(data, axis=1))\n",
    "\n",
    "fig.tight_layout()\n",
    "\n",
    "matplotlib.pyplot.savefig('wavedata.svg')\n",
    "matplotlib.pyplot.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0e3983bb-55c1-411a-b770-c9e89b880b2a",
   "metadata": {},
   "source": [
    "* tight_layout squeezes the plots closer together\n",
    "* savefig saves the graph to a file (make this an SVG)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8ebed54e-bce0-4d7b-9405-4aa04c85382c",
   "metadata": {},
   "source": [
    "* We can shorten names from imports with as\n",
    "* matplotlib.py plot as plt or numpy as np\n",
    "## Plot Scaling Exercises"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "09499775-6a46-4a47-8ff8-5dda766421f4",
   "metadata": {},
   "source": [
    "## multiple graphs on one pair of axes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3c973961-588d-432f-948a-8e0c652fbe02",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "matplotlib.pyplot.plot(numpy.max(data, axis=0), \"bo\", label='Maximum')\n",
    "matplotlib.pyplot.plot(numpy.average(data, axis=0), \"m+\", label='Average')\n",
    "matplotlib.pyplot.plot(numpy.min(data, axis=0), \"r--\", label='Minumum')\n",
    "matplotlib.pyplot.legend(loc='best')\n",
    "\n",
    "matplotlib.pyplot.show()\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1dd99095-e782-4d7a-b6be-ebfab39602bb",
   "metadata": {},
   "source": [
    "## Make your own plot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f7c965dd-2367-4f5e-850b-1651f7e3b098",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "std_plot = matplotlib.pyplot.plot(numpy.std(data, axis=0))\n",
    "matplotlib.pyplot.show()\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7c3806d8-d1f5-4ccf-a58e-1f728e73c55e",
   "metadata": {},
   "source": [
    "## Moving Plots Around"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0fab3d70-cb70-4ec6-ac47-cc2f6202f2a9",
   "metadata": {},
   "outputs": [],
   "source": [
    "# change figsize (swap width and height)\n",
    "fig = matplotlib.pyplot.figure(figsize=(3.0, 10.0))\n",
    "\n",
    "# change add_subplot (swap first two parameters)\n",
    "axes1 = fig.add_subplot(3, 1, 1)\n",
    "axes2 = fig.add_subplot(3, 1, 2)\n",
    "axes3 = fig.add_subplot(3, 1, 3)\n",
    "\n",
    "axes1.set_ylabel('average')\n",
    "axes1.plot(numpy.mean(data, axis=1))\n",
    "\n",
    "axes2.set_ylabel('max')\n",
    "axes2.plot(numpy.max(data, axis=1))\n",
    "\n",
    "axes3.set_ylabel('min')\n",
    "axes3.plot(numpy.min(data, axis=1))\n",
    "\n",
    "fig.tight_layout()\n",
    "\n",
    "matplotlib.pyplot.show()\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c59b2e4e-1dd7-4909-a48a-60436bbeab5b",
   "metadata": {},
   "source": [
    "## NetCDF files\n",
    "* scientific data often stored in netcdf files, sometimes geospatial data\n",
    "* different library needed\n",
    "* Check netcdf4 library is installed\n",
    "* we have a dataset with average wave heights (hs_avg) in a 2d array (lon/lat)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "de5b0f33-1093-469f-884a-9d85bc63203a",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import netCDF4 as nc\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c9aa5aa1-a33a-4015-8065-ab10e6efac3b",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "globaldata = nc.Dataset(\"data/multyear_hs_avg.nc\")\n",
    "print(type(globaldata))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a1132490-b508-4a40-ab3c-d258c768212b",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "matplotlib.pyplot.imshow(globaldata[\"hs_avg\"][0], extent=[0,360,-90,90], origin='lower')\n",
    "matplotlib.pyplot.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cdb1b2bb-69b9-463a-8d7e-83e8798fb171",
   "metadata": {},
   "source": [
    "* netcdf library translates fields in the netcdf into python arrays with named and numbered elements\n",
    "* globaldata.variables.keys() will show list of variables"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f68de1e6-0bd4-460d-9fc6-d09893c7c4b0",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "globaldata.variables.keys()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3e3f2fbc-92d1-4bba-b169-f3ac26c49cb1",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import mpl_toolkits\n",
    "\n",
    "matplotlib.pyplot.figure()\n",
    "ax = matplotlib.pyplot.gca()\n",
    "im = ax.imshow(globaldata[\"hs_avg\"][0], extent=[0,360,-90,90], origin='lower')\n",
    "\n",
    "#this doesn't work for me!\n",
    "divider = mpl_toolkits.axes_grid1.make_axes_locatable(ax)\n",
    "cax = divider.append_axes(\"right\", size=\"5%\", pad=0.05)\n",
    "\n",
    "matplotlib.pyplot.colorbar(im, cax=cax)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d082c8fb-eb94-45a8-a38f-f4cd3b7f4913",
   "metadata": {},
   "source": [
    "Recap:  \n",
    "* Use the pyplot module from the matplotlib library for creating simple visualizations.\n",
    "* Subplots can be used to plot multiple graphs in a single plot\n",
    "* Multiple items can be plotted in a single graph\n",
    "\n",
    "# Lists"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "17f9c846-dc80-458c-bcec-586231edc875",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "odds = [1, 3, 5, 7]\n",
    "print('odds are:', odds)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "535e2bfc-3e21-4fca-bf4f-e64f8f64124b",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "print('first element:', odds[0])\n",
    "print('last element:', odds[3])\n",
    "print('\"-1\" element:', odds[-1])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "31e320f1-0a94-4785-a376-f7f43f84d57d",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "names = ['Curie', 'Darwing', 'Turing']  # typo in Darwin's name\n",
    "print('names is originally:', names)\n",
    "names[1] = 'Darwin'  # correct the name\n",
    "print('final value of names:', names)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5e9427cc-565c-4f11-b132-5e4b2f8008be",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "name = 'Darwin'\n",
    "name[0] = 'd'\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e6ca080e-69f4-47fa-8283-01b35a5d38e4",
   "metadata": {},
   "source": [
    "* immutable vs mutable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a5a89bc3-b8b4-4264-b617-5fa76d96dc4b",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "salsa = ['peppers', 'onions', 'cilantro', 'tomatoes']\n",
    "my_salsa = salsa        # <-- my_salsa and salsa point to the *same* list data in memory\n",
    "salsa[0] = 'hot peppers'\n",
    "print('Ingredients in my salsa:', my_salsa)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ebbfa462-36e4-4e6a-b8c9-1a69968f1ee2",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "salsa = ['peppers', 'onions', 'cilantro', 'tomatoes']\n",
    "my_salsa = list(salsa)        # <-- makes a *copy* of the list\n",
    "salsa[0] = 'hot peppers'\n",
    "print('Ingredients in my salsa:', my_salsa)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8086e42d-20a4-4f0d-8b60-f9d08f99f615",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "x = [['pepper', 'zucchini', 'onion'],\n",
    "     ['cabbage', 'lettuce', 'garlic'],\n",
    "     ['apple', 'pear', 'banana']]\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9e3a92c1-fb1d-4950-bd93-e1173b8cf24a",
   "metadata": {},
   "source": [
    "* see visual example in notes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "97e8dd55-3f44-4701-b1b7-d3b94311ff57",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "print([x[0]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8acd98f4-256a-4fbb-ac91-a95881896667",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "print(x[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5dad7262-8ff0-4c81-bcb7-4045960922a0",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "print(x[0][0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fc956aa6-fb4e-4b76-811d-dc6c6075c021",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "sample_ages = [10, 12.5, 'Unknown']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bc7a1e4e-3215-4a06-9354-d20a7a8800e7",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "odds.append(11)\n",
    "print('odds after adding a value:', odds)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8682bc18-c18b-4d66-8ffa-265254e5aad6",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "removed_element = odds.pop(0)\n",
    "print('odds after removing the first element:', odds)\n",
    "print('removed_element:', removed_element)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2a88a293-44f4-4ead-a251-f104dfa53660",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "odds.reverse()\n",
    "print('odds after reversing:', odds)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "55168d74-9e47-4cb8-a1d9-788865d1e27a",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "odds = [3, 5, 7]\n",
    "primes = odds\n",
    "primes.append(2)\n",
    "print('primes:', primes)\n",
    "print('odds:', odds)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4777c636-84c0-4da3-a632-6fa3f12b5a0f",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "odds = [3, 5, 7]\n",
    "primes = list(odds)\n",
    "primes.append(2)\n",
    "print('primes:', primes)\n",
    "print('odds:', odds)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f1a4d92c-6c91-41f9-9041-7f959191c8df",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "binomial_name = 'Drosophila melanogaster'\n",
    "group = binomial_name[0:10]\n",
    "print('group:', group)\n",
    "\n",
    "species = binomial_name[11:23]\n",
    "print('species:', species)\n",
    "\n",
    "chromosomes = ['X', 'Y', '2', '3', '4']\n",
    "autosomes = chromosomes[2:5]\n",
    "print('autosomes:', autosomes)\n",
    "\n",
    "last = chromosomes[-1]\n",
    "print('last:', last)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f0249c89-003e-4735-9469-d36aeb0c7066",
   "metadata": {},
   "source": [
    "* Slicing from end exercise"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "45caecba-d65c-4048-a883-09b68e31b9e3",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}