class01_ppi_python3.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Analyzing the Pathway Commons 2 (PC2) database SIF file\n",
    "## CS446/546 class session 1\n",
    "\n",
    "### Goal: count the number of different types of biological interactions in PC2\n",
    "### Approach: retrieve compressed tab-delimited \"edge-list\" file and tabulate \"interaction\" column\n",
    "\n",
    "### Information you will need:\n",
    "\n",
    "- The URL is: http://www.pathwaycommons.org/archives/PC2/v9/PathwayCommons9.All.hgnc.sif.gz\n",
    "- You'll be using the Python modules `gzip`, `timeit`, `pandas`, `urllib.request`, `collections` and `operator`\n",
    "\n",
    "### What is the \".sif\" file format?\n",
    "\n",
    "SIF stands for Simple Interaction File. The format is like this:\n",
    "```\n",
    "A1BG    controls-expression-of  A2M\n",
    "A1BG    interacts-with  ABCC6\n",
    "A1BG    interacts-with  ACE2\n",
    "A1BG    interacts-with  ADAM10\n",
    "A1BG    interacts-with  ADAM17\n",
    "A1BG    interacts-with  ADAM9\n",
    "...\n",
    "```\n",
    "\n",
    "### Other stuff you should do:\n",
    "- Print the first six lines of the uncompressed data file\n",
    "- Use a timer to time how long your program takes\n",
    "- Count how many rows there are in the data file \n",
    "- Estimate the number of proteins in the database; we'll define them operationally as strings in column 1 or column 3, for which the content of column 2 is one of these interactions: 'interacts-with', 'in-complex-with', 'neighbor-of'\n",
    "- Count the total number of unique pairs of interacting molecules (ignoring interaction type)\n",
    "- Count the number rows for each type of interaction in the database\n",
    "- Pythonistas:  do it using Pandas and without using Pandas"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step-by-step instructions for Python3:\n",
    "\n",
    "- Open a file object representing a stream of the remote, compressed data file, using `urlopen`\n",
    "- Open a file object representing a stream of the uncompressed data file, using `gzip.GzipFile`\n",
    "- Start the timer\n",
    "- Read one line at a time, until the end of the file\n",
    "- Split line on \"\\t\" and pull out the tuple of species1, interaction_type, species2 from the line of text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "b'A1BG\\tcontrols-expression-of\\tA2M\\n'\n",
      "b'A1BG\\tinteracts-with\\tABCC6\\n'\n",
      "b'A1BG\\tinteracts-with\\tACE2\\n'\n",
      "b'A1BG\\tinteracts-with\\tADAM10\\n'\n",
      "b'A1BG\\tinteracts-with\\tADAM17\\n'\n",
      "b'A1BG\\tinteracts-with\\tADAM9\\n'\n"
     ]
    }
   ],
   "source": [
    "from urllib.request import urlopen\n",
    "import gzip\n",
    "import timeit\n",
    "\n",
    "baseURL = \"http://www.pathwaycommons.org/archives/PC2/v9/\"\n",
    "filename = \"PathwayCommons9.All.hgnc.sif.gz\"\n",
    "outFilePath = \"pc.sif\"\n",
    "interaction_types_ppi = set([\"interacts-with\",\"in-complex-with\",\"neighbor-of\"])\n",
    "\n",
    "start_time = timeit.default_timer()\n",
    "\n",
    "zfd = urlopen(baseURL + filename)\n",
    "fd = gzip.GzipFile(fileobj=zfd, mode=\"r\")\n",
    "\n",
    "# initialize the SIF file interaction counter\n",
    "intctr = 0\n",
    "linectr = 0\n",
    "from collections import defaultdict\n",
    "\n",
    "interactions = set()\n",
    "proteins = set()\n",
    "intnamectr = defaultdict(int)\n",
    "\n",
    "for line in fd:\n",
    "    if linectr < 6:\n",
    "        print(line)\n",
    "        \n",
    "    linectr += 1\n",
    "    \n",
    "    [prot1, interaction_type, prot2] = line.decode(\"utf-8\").rstrip(\"\\n\").split(\"\\t\")\n",
    "    intnamectr[interaction_type] += 1\n",
    "    if interaction_type in interaction_types_ppi:\n",
    "        intctr += 1\n",
    "        proteins |= set([prot1, prot2])\n",
    "        interactions.add(min(prot1, prot2) + \"-\" + max(prot1, prot2))       \n",
    "        \n",
    "elapsed = timeit.default_timer() - start_time"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "How long your program take to run?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5.78 sec\n"
     ]
    }
   ],
   "source": [
    "print(\"{0:.2f}\".format(elapsed) + \" sec\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "How many protein-protein interactions are there in the data file?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "508480\n"
     ]
    }
   ],
   "source": [
    "print(intctr)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "How many unique protein names are there in the data file?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "17531"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(proteins)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "How many unique pairs of proteins (regarless of interaction type name) are there that interact?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "475553"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(interactions)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "How many interactions are there of each type, in PC2?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('chemical-affects', 492765),\n",
       " ('interacts-with', 325616),\n",
       " ('in-complex-with', 182864),\n",
       " ('controls-state-change-of', 182450),\n",
       " ('catalysis-precedes', 149013),\n",
       " ('controls-expression-of', 123232),\n",
       " ('consumption-controlled-by', 22830),\n",
       " ('controls-production-of', 21494),\n",
       " ('controls-phosphorylation-of', 17029),\n",
       " ('used-to-produce', 14486),\n",
       " ('controls-transport-of', 7574),\n",
       " ('reacts-with', 3927),\n",
       " ('controls-transport-of-chemical', 3322)]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from operator import itemgetter\n",
    "sorted(intnamectr.items(), key=itemgetter(1), reverse=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Let's do it again, using Pandas:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "read from the uncompressed data stream, and parse it into a data frame, using `pandas.read_csv`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas\n",
    "zfd = urlopen(baseURL + filename)\n",
    "fd = gzip.GzipFile(fileobj=zfd, mode=\"r\")\n",
    "df = pandas.read_csv(fd, sep=\"\\t\", names=[\"species1\",\"interaction_type\",\"species2\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Use the `head` method on the data frame, to print out the first six lines"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  species1        interaction_type species2\n",
      "0     A1BG  controls-expression-of      A2M\n",
      "1     A1BG          interacts-with    ABCC6\n",
      "2     A1BG          interacts-with     ACE2\n",
      "3     A1BG          interacts-with   ADAM10\n",
      "4     A1BG          interacts-with   ADAM17\n"
     ]
    }
   ],
   "source": [
    "print(df.head())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Print the unique types of interactions in the data frame, using the `unique` method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['controls-expression-of', 'interacts-with',\n",
       "       'controls-phosphorylation-of', 'controls-state-change-of',\n",
       "       'in-complex-with', 'controls-production-of', 'catalysis-precedes',\n",
       "       'controls-transport-of', 'controls-transport-of-chemical',\n",
       "       'chemical-affects', 'consumption-controlled-by', 'reacts-with',\n",
       "       'used-to-produce'], dtype=object)"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.interaction_type.unique()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Subset the data frame by interaction type (using `isin` method), to include only the protein-protein interactions, then count"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "508480"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ppirows = df.interaction_type.isin(interaction_types_ppi)\n",
    "sum(ppirows)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Make a list of all proteins that occur in a protein-protein interaction, and count the unique protein names by putting them in a `set` and calling `len` on the set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "17531"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "newlist = df[\"species1\"][ppirows].tolist() + df[\"species2\"][ppirows].tolist()\n",
    "len(set(newlist))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Count unique protein-protein interaction pairs (specific type of interaction irrelevant), again using `set` and `len`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "475553"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(set(df[\"species1\"][ppirows] + \"-\" + df[\"species2\"][ppirows]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Count each type of interaction in the database, by subsetting to the `interaction` column and using `value_counts`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "chemical-affects                  492765\n",
       "interacts-with                    325616\n",
       "in-complex-with                   182864\n",
       "controls-state-change-of          182450\n",
       "catalysis-precedes                149013\n",
       "controls-expression-of            123232\n",
       "consumption-controlled-by          22830\n",
       "controls-production-of             21494\n",
       "controls-phosphorylation-of        17029\n",
       "used-to-produce                    14486\n",
       "controls-transport-of               7574\n",
       "reacts-with                         3927\n",
       "controls-transport-of-chemical      3322\n",
       "Name: interaction_type, dtype: int64"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[\"interaction_type\"].value_counts()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}