datashader_1.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "749b0ddf",
   "metadata": {},
   "source": [
    "<img align=\"left\" src = https://project.lsst.org/sites/default/files/Rubin-O-Logo_0.png width=170 style=\"padding: 10px\"> \n",
    "<b>Little Demo: datashader</b> <br>\n",
    "Contact author(s): Leanne Guy, Melissa Graham <br>\n",
    "Last verified to run: 2023-05-08 <br>\n",
    "LSST Science Pipelines version: Weekly 2023_07 <br>\n",
    "Container Size: medium <br>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "effe96c9-a91d-4a72-92a4-33bf8859d4cb",
   "metadata": {},
   "source": [
    "Traditional methods use static plots, or partially-transparent points, become confusion-limited with millions of objects.\n",
    "\n",
    "The Rubin Science Platform will support third-party packages such as `datashader` to enable interactive plots to showcase millions of LSST detections.\n",
    "\n",
    "For a more detailed tutorial on visualization techniques for catalog data, see <a href=\"https://github.com/rubin-dp0/tutorial-notebooks\">tutorial notebook</a> 06b.\n",
    "\n",
    "## 1. Set up\n",
    "\n",
    "Import packages."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7acc79a4-8530-42d9-96e5-b7acb4397864",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from lsst.rsp import get_tap_service\n",
    "import datashader as dsh\n",
    "from bokeh.io import output_notebook\n",
    "import holoviews as hv\n",
    "from holoviews import streams\n",
    "from holoviews.operation.datashader import dynspread, datashade"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "658e3aaa-aa1d-46d7-877b-e3617b61d4df",
   "metadata": {},
   "source": [
    "Set parameters to show plots in the notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e7bee208-22ec-463b-a424-1894c06d2033",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "hv.extension('bokeh')\n",
    "output_notebook()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "81adc25c-064c-4a80-a345-4dd04c6ebad3",
   "metadata": {},
   "source": [
    "Instantiate the TAP service."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bcaf1d53-5769-43a6-b84f-fa683530b1f3",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "service = get_tap_service()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a1ea0158-f935-48f8-ba70-5d951fc4b552",
   "metadata": {},
   "source": [
    "For a more detailed tutorial on the TAP service, see <a href=\"https://github.com/rubin-dp0/tutorial-notebooks\">tutorial notebook</a> 02.\n",
    "\n",
    "Query for and return data for at least 1 million Objects (takes ~1 minute)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "084d3959-523c-4825-8112-a91326a7ca70",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "query = \"SELECT coord_ra, coord_dec, objectId, r_extendedness, \"\\\n",
    "        \"scisql_nanojanskyToAbMag(g_cModelFlux) AS mag_g_cModel, \"\\\n",
    "        \"scisql_nanojanskyToAbMag(r_cModelFlux) AS mag_r_cModel, \"\\\n",
    "        \"scisql_nanojanskyToAbMag(i_cModelFlux) AS mag_i_cModel \"\\\n",
    "        \"FROM dp02_dc2_catalogs.Object \"\\\n",
    "        \"WHERE CONTAINS(POINT('ICRS', coord_ra, coord_dec),\"\\\n",
    "        \"CIRCLE('ICRS', 55.65, -40, 1)) = 1 \" \\\n",
    "        \"AND detect_isPrimary = 1 \"\\\n",
    "        \"AND scisql_nanojanskyToAbMag(r_cModelFlux) < 27.0 \"\\\n",
    "        \"AND r_extendedness IS NOT NULL\"\n",
    "data = service.search(query).to_table().to_pandas()\n",
    "print(len(data))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a1a3f171-d1df-42cf-913e-d29249175d67",
   "metadata": {},
   "source": [
    "Compute _g-r_ and _r-i_ colors for all objects."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9faa7faa-e645-468d-9078-fb1cc7c17ce9",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "data['gmr'] = data['mag_g_cModel'] - data['mag_r_cModel']\n",
    "data['rmi'] = data['mag_r_cModel'] - data['mag_i_cModel']"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "043ee009-52c3-4882-b321-6c68ecac6b18",
   "metadata": {},
   "source": [
    "## 2. Plot with datashader"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9d55a669-ddb1-4f0b-95c0-c2b5439d6e28",
   "metadata": {},
   "source": [
    "Define the plot's points and bounding box."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3dc202e4-2ae6-4745-a789-a111a9591330",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "points = hv.Points((data['gmr'], data['rmi']))\n",
    "boundsxy = (0, 0, 0, 0)\n",
    "box = streams.BoundsXY(source=points, bounds=boundsxy)\n",
    "bounds = hv.DynamicMap(lambda bounds: hv.Bounds(bounds), streams=[box])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d2ee6012-df1f-4f65-8563-dc290ae2cc85",
   "metadata": {},
   "source": [
    "Define the plot itself, and display it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "98f199cd-aed7-42fd-aacd-98abee7a16c8",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "p = dynspread(datashade(points, cmap=\"Viridis\"))\n",
    "p = p.opts(width=800, height=300, padding=0.05, show_grid=True,\n",
    "           xlim=(-2.0, 7.0), ylim=(-5.0, 3.0), xlabel=\"(g-r)\", ylabel=\"(r-i)\")#,\n",
    "           # tools=['box_select', 'lasso_select'])\n",
    "p * bounds"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2114da81-1299-4093-a69b-3ae972154220",
   "metadata": {},
   "source": [
    "Use the icon with the magnifying glass to zoom in, and notice how the plot automatically adjusts from heatmap to individual points, allowing the million points to be visualized all together.\n",
    "\n",
    "Use the two-arrows-in-a-circle icon to reset the plot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "259e9aba-f1e5-4b25-ba7e-dd93b52fc46c",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "LSST",
   "language": "python",
   "name": "lsst"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.9"
  },
  "toc-autonumbering": false
 },
 "nbformat": 4,
 "nbformat_minor": 5
}