01-Introduction.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font size=\"1\">Copyright 2021, by the California Institute of Technology. ALL RIGHTS RESERVED. United States Government sponsorship acknowledged. Any commercial use must be negotiated with the Office of Technology Transfer at the California Institute of Technology.</font>\n",
    "    \n",
    "<font size=\"1\">This software may be subject to U.S. export control laws and regulations. By accepting this document, the user agrees to comply with all applicable U.S. export laws and regulations. User has the responsibility to obtain export licenses, or other export authority as may be required, before exporting such information to foreign countries or providing access to foreign persons.<font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction\n",
    "In this notebook, you will use RESTful calls to the Pele API to register as a user, the utilize the Pele client library which provides automatic session renewal.\n",
    "\n",
    "#### Kernel: isce, plant or mintpy\n",
    "\n",
    "## User Registration\n",
    "\n",
    "In order to use the Pele API, you must be a registered user. First set some parameters:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import requests, json, getpass\n",
    "from requests.auth import HTTPBasicAuth\n",
    "import urllib3\n",
    "\n",
    "urllib3.disable_warnings()\n",
    "\n",
    "# set the base url to interact with the goddess, Pele\n",
    "base_url = input(\"Enter Pele REST API base url (e.g. https://<mozart_ip>/pele/api/v0.1) then press <Enter>: \")\n",
    "print(\"Using base url {}.\".format(base_url))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# set username\n",
    "user = input(\"Enter email address then press <Enter>: \")\n",
    "print(\"Using {}.\".format(user))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# set password\n",
    "print(\"Enter password then press <Enter>.\")\n",
    "print(\"*ATTENTION!!! DON'T USE YOUR JPL PASSWORD HERE. MAKE SOMETHING UP FOR THIS TEST.*\")\n",
    "password = getpass.getpass()\n",
    "print(\"password: '{}'\".format(password))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "### Note: If you are not yet registered, the following cell *should* produce a 401 / Unauthorized Access response. This is the correct behavior."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# login with a username and password\n",
    "r = requests.post(base_url + '/login', auth=HTTPBasicAuth(user, password), verify=False)\n",
    "\n",
    "# expect 401 (Unauthorized access) error\n",
    "print(\"status code: {}\".format(r.status_code))\n",
    "print(\"content: {}\".format(r.content.decode()))\n",
    "assert r.status_code == 401"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Register with a valid email address:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "r = requests.post(base_url + '/register', data={'email': user, 'password': password}, verify=False)\n",
    "\n",
    "# expect 201 (created)\n",
    "print(\"status code: {}\".format(r.status_code))\n",
    "print(\"content: {}\".format(r.content.decode()))\n",
    "assert r.status_code == 201"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The email you receive will contain a verification code. For example:\n",
    "\n",
    "```\n",
    "Use your verification code below to verify your Pele API account at http://localhost:8877/api/v0.1/:\n",
    "\n",
    "ffa8d18b-f581-44bf-8864-b52a2cd8e7b6\n",
    "```\n",
    "You will use that verification code to verify your user account:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# prompt for verification code\n",
    "ver_code = input(\"Enter the verification code:\")\n",
    "\n",
    "# verify\n",
    "r = requests.post(base_url + '/verify', data={'email': user, 'verification_code': ver_code}, verify=False)\n",
    "\n",
    "# expect 200\n",
    "print(\"status code: {}\".format(r.status_code))\n",
    "print(json.dumps(r.json(), indent=2))\n",
    "assert r.status_code == 200"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create the .netrc file for automatic login\n",
    "\n",
    "If you will be running scripts that will interact with the Pele API, you will need to utilize a method for automatically logging into the Pele REST API to request the API token and to refresh the token should the token expire during the script's execution.\n",
    "\n",
    "The Pele requests client can utilize the `.netrc` file to automate this for you.\n",
    "\n",
    "Here we populate your .netrc:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from urllib.parse import urlparse\n",
    "import getpass, os, stat\n",
    "\n",
    "system_username = getpass.getuser()\n",
    "# parse url to get netloc component\n",
    "pr = urlparse(base_url)\n",
    "\n",
    "print(\"netloc: {}\".format(pr.netloc))\n",
    "\n",
    "# get home directory\n",
    "stream = os.popen('ls -d ~')\n",
    "home_dir = stream.read().strip()\n",
    "\n",
    "# create .netrc\n",
    "print(f\"Writing {home_dir}/.netrc\")\n",
    "with open(f\"{home_dir}/.netrc\", 'a') as f:\n",
    "    f.write(f\"machine {pr.netloc} login {user} password {password}\\nmacdef init\\n\\n\")\n",
    "\n",
    "# fix perms\n",
    "os.chmod(f\"{home_dir}/.netrc\", stat.S_IRUSR | stat.S_IWUSR)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "At this point you are ready to use the Pele client API w/ automatic token/session refresh. Feel free to explore the remaining optional portions of this notebook which discusses the API token (which is now transparent to you through the Pele client API) and rate limiting of calls."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## login to get API token\n",
    "\n",
    "Once your user registration is verified, you can then log into the Pele API which will provide you with a time-bombed API token. You will use this API token to subsequently make requests to the Pele API:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "r = requests.post(base_url + '/login', auth=HTTPBasicAuth(user, password), verify=False)\n",
    "\n",
    "# expect 200\n",
    "print(\"status code: {}\".format(r.status_code))\n",
    "print(r)\n",
    "print(json.dumps(r.json(), indent=2))\n",
    "\n",
    "# extract API token\n",
    "token = r.json()['token']\n",
    "print(\"token: {}\".format(token))\n",
    "assert r.status_code == 200"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "pass your API token in a header called `X-API-KEY`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "r = requests.get(base_url + '/test/echo', params={'echo_str': 'hello world'}, headers={'X-API-KEY': token}, verify=False)\n",
    "\n",
    "# expect 200\n",
    "print(\"status code: {}\".format(r.status_code))\n",
    "print(json.dumps(r.json(), indent=2))\n",
    "assert r.status_code == 200"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## refresh API token after expiration\n",
    "\n",
    "As stated before, the API token you receive is time-bombed. It will expire after a certain amount of time (defaults to 86400 seconds). The example below shows that when your token has expired, you will receive a `401` status code error with the error message: `Expired token. Reauthentication required.`:\n",
    "\n",
    "```\n",
    "In [1]: r = requests.get(base_url + '/test/echo', params={'echo_str': 'hello world'}, headers={'X-API-KEY': token})\n",
    "\n",
    "In [2]: r.status_code\n",
    "Out[2]: 401\n",
    "\n",
    "In [3]: r.json()\n",
    "Out[3]: \n",
    "{u'authenticated': False,\n",
    " u'message': u'Expired token. Reauthentication required.'}\n",
    "```\n",
    "\n",
    "At this point, you will have to login again to refresh your API token (see [login to get API token](#login-to-get-API-token) above)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Issue a request through the Pele client"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pele_client.client import PeleRequests\n",
    "\n",
    "# instantiate PeleRequests object\n",
    "print(f\"Base URL {base_url}\")\n",
    "pr = PeleRequests(base_url, verify=False)\n",
    "\n",
    "# now use like requests module (`request()`, `get()`, `head()`, `post()`, `put()`, `delete()`, `patch()`)\n",
    "r = pr.get(base_url + '/test/echo', params={'echo_str': 'hello world'})\n",
    "\n",
    "# expect 200\n",
    "print(\"status code: {}\".format(r.status_code))\n",
    "print(json.dumps(r.json(), indent=2))\n",
    "assert r.status_code == 200"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Rate-limited API calls\n",
    "\n",
    "The Pele REST API rate-limits calls to prevent DoS-like access to the backend database:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for i in range(20):\n",
    "    r = pr.get(base_url + '/test/echo', params={'echo_str': f'({i}) hello world'})\n",
    "    print(\"({}) status code: {}\".format(i, r.status_code))\n",
    "    print(json.dumps(r.json(), indent=2))\n",
    "    r.raise_for_status()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To mitigate this, utilize exponential backoff:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install backoff"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import backoff\n",
    "from requests.exceptions import HTTPError\n",
    "\n",
    "@backoff.on_exception(backoff.expo, HTTPError, max_tries=5, max_value=5)\n",
    "def echo(i):\n",
    "    r = pr.get(base_url + '/test/echo', params={'echo_str': f'({i}) hello world'})\n",
    "    print(\"({}) status code: {}\".format(i, r.status_code))\n",
    "    print(json.dumps(r.json(), indent=2))\n",
    "    r.raise_for_status()\n",
    "    \n",
    "for i in range(20):\n",
    "    echo(i)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font size=\"1\">This notebook is compatible with NISAR Jupyter Server Stack v1.7.1 and above</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "isce [conda env:.local-isce]",
   "language": "python",
   "name": "conda-env-.local-isce-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}