22_python_data.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Matplotlib\n",
    "\n",
    "[`matplotlib`](http://matplotlib.org/) is probably the single most-used Python package for graphics. It provides both a very quick way to visualize data from Python and publication-quality figures in many formats. It is also highly customizable, allowing you to create [rich visualizations](http://droettboom.com/jhepc2018-judge-packet/) of complex data.\n",
    "\n",
    "The `matplotlib.pyplot` module is a collection of command style functions that make `matplotlib` work like MATLAB. Each `pyplot` function makes some change to a figure: e.g., creates a figure, creates a plotting area in a figure, plots some lines in a plotting area, decorates the plot with labels, etc. \n",
    "\n",
    "Let's start with a very simple plot. First we set the `%matplotlib inline` option, which tells the Jupyter notebook to embed all plots as static images. Next, we import the `matplotlib.pyplot` module using the shorter alias `plt`, so that we don't have to use the full module name every time we call a `pyplot` function. Then we call the `.plot()` command on a list of integers to create a plot. Finally, we use the `.show()` command to render the plot and embed it underneath the code block."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data science in Python\n",
    "\n",
    "## Session 2.2: Data visualisation with Matplotlib\n",
    "\n",
    "- [Matplotlib](#Matplotlib)\n",
    "- [Exercise 2.2.1](#Exercise-2.2.1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "plt.plot([1,2,3,4])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After we create a plot, we can use the `.xlabel()` and `.ylabel()` commands to decorate it with axis legends, and add a title using the `.title()` command"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.plot([1,2,3,4])\n",
    "plt.xlabel(\"X axis label\")\n",
    "plt.ylabel(\"Y axis label\")\n",
    "plt.title(\"Plot title\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`.plot()` is a versatile command. To plot x versus y, we can input two lists of integers:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.plot([1,2,3,4], [1,4,9,16])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For every x, y pair of arguments, there is an **optional third argument** which is the format string that indicates the color and line type of the plot. The letters and symbols of the format string are from MATLAB, and you concatenate a color string with a line style string. The default format string is `'b-'`, which is a solid blue line. For example, to plot the above with red circles, you would chose `'ro'`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "plt.plot([1,2,3,4], [1,4,9,16], 'ro')\n",
    "plt.axis([0, 6, 0, 20])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "These formatting options can be especially useful when displaying data from different sources on the same plot. Once the plot is created, additional lines with different options can be added by calling the `.plot()` command multiple times before calling `.show()`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "year = [1960, 1970, 1980, 1990, 2000]\n",
    "rainfall_Manchester = [200, 190, 160, 185, 170]\n",
    "rainfall_London = [160, 155, 140, 145, 140]\n",
    "plt.plot(year, rainfall_Manchester, 'b-')\n",
    "plt.plot(year, rainfall_London, 'r--')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With more than one dataset on the same plot, we often need to add a legend to the plot. This can be done by passing the `label` argument to each `.plot()` command, and then calling the `.legend()` command on the plot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "year = [1960, 1970, 1980, 1990, 2000]\n",
    "rainfall_Manchester = [200, 190, 160, 185, 170]\n",
    "rainfall_London = [160, 155, 140, 145, 140]\n",
    "plt.plot(year, rainfall_Manchester, 'b-', label = 'Manchester')\n",
    "plt.plot(year, rainfall_London, 'r--', label = 'London')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once we have made a plot, we can write it to disk by using the `.savefig()` command instead of the `.show()` command. This will infer the file format from the ending of the file name e.g. \"myplot.svg\" will write in SVG format, \"myplot.pdf\" will write in PDF format etc. **Note: the range of file types that can be written may vary between different operating systems.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "plt.plot([1,2,3,4], [1,4,9,16])\n",
    "plt.savefig(\"myplot.svg\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Functions for specific plot types\n",
    "`matplotlib.pyplot` has a few functions that make creating common types of plots faster and more convenient because they automatically create a Figure and an Axes object (more on this in the next section). The most widely used are:\n",
    "\n",
    "- [`plt.bar()`](http://matplotlib.org/api/pyplot_api.html#matplotlib.pyplot.bar) – creates a bar chart.\n",
    "- [`plt.boxplot()`](http://matplotlib.org/api/pyplot_api.html#matplotlib.pyplot.boxplot) – makes a box and whisker plot.\n",
    "- [`plt.hist()`](http://matplotlib.org/api/pyplot_api.html#matplotlib.pyplot.hist) – makes a histogram.\n",
    "- [`plt.plot()`](http://matplotlib.org/api/pyplot_api.html#matplotlib.pyplot.plot) – creates a line plot.\n",
    "- [`plt.scatter()`](http://matplotlib.org/api/pyplot_api.html#matplotlib.pyplot.scatter) – makes a scatter plot.\n",
    "\n",
    "Each of these methods has different parameters that can be passed in to modify the resulting plot. For example, let's plot a bar chart of the population size of different European capital cities:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "city = [\"Dublin\", \"Paris\", \"Madrid\", \"Berlin\", \"Brussels\"]\n",
    "population = [553165, 2206488, 3182981, 3711930, 1191604]\n",
    "plt.bar(range(len(city)), population)\n",
    "plt.xticks(range(len(city)), city)\n",
    "plt.ylabel(\"Population size\")\n",
    "plt.title(\"Population size of European capital cities\")\n",
    "plt.ylim(0,4000000)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A full list of functions and details of how to use them is available from the `pyplot` [API documentation](https://matplotlib.org/api/_as_gen/matplotlib.pyplot.html)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Storing the Figure and Axes objects"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So far, we have been using `matplotlib`'s **state-based interface** , calling `pyplot` functions such as `.plot()`, `.title()`, and `.xlabel()` directly, and using them to add elements to a single, currently-active plot. \n",
    "\n",
    "It is also common to use `matplotlib`'s **object-oriented interface**. To do this, we first create a Figure object, and store it using the variable name `fig`. Next, we use the `.gca()` method to retrieve the Axes object associated with the Figure, which we store using the variable name `ax`. Finally, we decorate the Axes object, adding a title, labels, etc.\n",
    "\n",
    "This is often a point of confusion when first using `matplotlib`, because the methods used to decorate the plot change slightly in the object-oriented case. When using this approach, we now have to call the object methods `.set_title()`, `.set_xlabel()`, `.set_ylabel()`, etc.\n",
    "\n",
    "For example, let's plot the GC content along a gene using the quick state-based approach:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "seq = 'ATGGTGCATCTGACTCCTGAGGAGAAGTCTGCCGTTACTGCCCTGTGGGGCAAGGTG'\n",
    "gc = [40.0, 60.0, 80.0, 60.0, 40.0, 60.0, 40.0, 40.0, 40.0, 60.0, \n",
    "      40.0, 60.0, 60.0, 60.0, 60.0, 60.0, 60.0, 60.0, 60.0, 60.0, \n",
    "      60.0, 40.0, 40.0, 40.0, 40.0, 40.0, 60.0, 60.0, 80.0, 80.0, \n",
    "      80.0, 60.0, 40.0, 40.0, 20.0, 40.0, 60.0, 80.0, 80.0, 80.0, \n",
    "      80.0, 60.0, 60.0, 60.0, 80.0, 80.0, 100.0, 80.0, 60.0, 60.0, \n",
    "      60.0, 40.0, 60.0]\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "plt.plot(gc, '--')\n",
    "plt.xlabel('Window along the sequence (5bp)')\n",
    "plt.ylabel('%GC')\n",
    "plt.title('GC plot for sequence\\n' + seq)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's generate the same plot using the object-oriented approach:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "seq = 'ATGGTGCATCTGACTCCTGAGGAGAAGTCTGCCGTTACTGCCCTGTGGGGCAAGGTG'\n",
    "gc = [40.0, 60.0, 80.0, 60.0, 40.0, 60.0, 40.0, 40.0, 40.0, 60.0, \n",
    "      40.0, 60.0, 60.0, 60.0, 60.0, 60.0, 60.0, 60.0, 60.0, 60.0, \n",
    "      60.0, 40.0, 40.0, 40.0, 40.0, 40.0, 60.0, 60.0, 80.0, 80.0, \n",
    "      80.0, 60.0, 40.0, 40.0, 20.0, 40.0, 60.0, 80.0, 80.0, 80.0, \n",
    "      80.0, 60.0, 60.0, 60.0, 80.0, 80.0, 100.0, 80.0, 60.0, 60.0, \n",
    "      60.0, 40.0, 60.0]\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "fig = plt.figure()\n",
    "ax = fig.gca()\n",
    "ax.plot(gc, '--')\n",
    "ax.set_xlabel('Window along the sequence (5bp)')\n",
    "ax.set_ylabel('%GC')\n",
    "ax.set_title('GC plot for sequence\\n' + seq)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using the object-oriented approach, you can create one Figure object with multiple Axes for different sub-panels. Giving each Axes a distinct variable name allows you to plot different information in each panel. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Plotting directly from `Pandas` objects\n",
    "\n",
    "As seen in the previous session, it is possible to plot directly from `pandas`. We first create a `pandas` DataFrame object from the GC data above:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas\n",
    "gc_df = pandas.DataFrame(gc, columns=['gc'])\n",
    "gc_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we call the `.plot()` method on the `pandas` DataFrame, which is a wrapper around `matplotlib.pyplot.plot()`. We store the object using the variable name `thisplot`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "thisplot = gc_df.gc.plot(style='--')\n",
    "thisplot.set_xlabel('Window along the sequence (5bp)')\n",
    "thisplot.set_ylabel('%GC')\n",
    "thisplot.set_title('GC plot for sequence\\n' + seq)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that the `pandas` plots use the object-oriented interface, so we have to call `.set_xlabel()` on our stored figure object rather than `.xlabel()` to decorate the plot. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Modifying figure aesthetics with `Seaborn`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[`seaborn`](https://seaborn.pydata.org) is a data visualization library based on `matplotlib`. It provides easy ways to customize the aesthetics of basic `matplotlib` figures, as well as a range of additional plot types (see [examples](https://seaborn.pydata.org/examples/index.html)). We import `seaborn` and give it the alias `sns`.\n",
    "\n",
    "Let's now modify the style of our previous `matplotlib` code. \n",
    "\n",
    "Using the `.set_style()` method with the option `'darkgrid'` we can create a similar aesthetic to `R`'s `ggplot2` package. Other [available styles](https://seaborn.pydata.org/tutorial/aesthetics.html) include `'whitegrid'`, `'dark'`, `'white'`, and `'ticks'`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "import seaborn as sns\n",
    "sns.set_style('darkgrid')\n",
    "\n",
    "year = [1960, 1970, 1980, 1990, 2000]\n",
    "rainfall_Manchester = [200, 190, 160, 185, 170]\n",
    "rainfall_London = [160, 155, 140, 145, 140]\n",
    "plt.plot(year, rainfall_Manchester, label = 'Manchester')\n",
    "plt.plot(year, rainfall_London, label = 'London')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To modify a plot so that it will be clearly visible on a slide or poster, we can use the `.set_context()` method. For example, here is the same plot with the `'ticks'` style, formatted using the `'talk'` context. Note how both the font size and thickness of the lines has increased with a single command."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "import seaborn as sns\n",
    "sns.set_style('ticks')\n",
    "sns.set_context('talk')\n",
    "\n",
    "year = [1960, 1970, 1980, 1990, 2000]\n",
    "rainfall_Manchester = [200, 190, 160, 185, 170]\n",
    "rainfall_London = [160, 155, 140, 145, 140]\n",
    "plt.plot(year, rainfall_Manchester, label = 'Manchester')\n",
    "plt.plot(year, rainfall_London, label = 'London')\n",
    "plt.legend()\n",
    "\n",
    "# removes the border on the right and top of the plot\n",
    "sns.despine()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 2.2.1\n",
    "\n",
    "Re-use the GapMinder dataset to plot, in Jupyter using Matplotlib, a scatter plot of world life expectancy against GDP per capita for 1952, 1977 and 2007. Add a title, axis labels and legend to your figure.\n",
    "\n",
    "Find the country with the highest GDP per capita for 1952, 1977 and 2007.\n",
    "\n",
    "Re-write the function `gdp_stats_by_continent_and_year()` using Pandas."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Next session\n",
    "\n",
    "Go to our next notebook: [Session 2.3: Biological data with BioPython](23_python_data.ipynb)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}