exercises/NEOWISE_light_curve_demo.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "52f5c5d4",
   "metadata": {},
   "source": [
    "# Using Firefly visualization tools to understand the light curves of Solar System objects\n",
    "\n",
    "## Learning Goals\n",
    "\n",
    "By the end of this tutorial, you will:\n",
    "\n",
    "- Construct a TAP query to download the necessary data and visualize it via the web browser with an instantiated Firefly environment.\n",
    "- Plot light curves from NEOWISE data using the Firefly Python API.\n",
    "- Overlay a catalog of data onto a HiPS image."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "436f180b",
   "metadata": {},
   "source": [
    "## Introduction\n",
    "\n",
    "This tutorial demonstrates how to plot light curves from NEOWISE data while also flaunting the many useful capabilities of the Firefly Python API. Using the 'Known Solar System Object Possible Association List' catalog from the NEOWISE-R database, we can easily compose a light curve of the faint asteroid `558 Carmen` and show its observed positions in the sky solely through the `firefly_client` package. Minor planet light curves are important in determining their size, spectral class, rotation period and many other properties.\n",
    "\n",
    "Firefly is an astronomical data access and visualization written by Caltech/IPAC-IRSA. The visualization provides user with an integrated experience with brushing and linking capabilities among images, catalogs, and plots. Firefly is used in IRSA GUIs to query and visualize data from missions such as WISE, Spitzer, SOFIA, ZTF, PTF, etc. and a large number of highly-used contributed data products from a diverse set of astrophysics projects.\n",
    "\n",
    "The `firefly_client` package provides a lightweight client class that includes a Python interface to Firefly’s Javascript API.\n",
    "\n",
    "For documentation on the firefly client visit https://caltech-ipac.github.io/firefly_client/."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "69eba47b",
   "metadata": {},
   "source": [
    "## Imports\n",
    "\n",
    "- *firefly_client FireflyClient* - Python API to Firefly for displaying tables, images and charts\n",
    "- *astropy.utils.data* for downloading the catalog data via TAP query\n",
    "- *urllib.parse* for converting regular query string to url-safe string"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1f62614c",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Uncomment the next line to install dependencies if needed.\n",
    "# !pip install firefly_client astropy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ba101908",
   "metadata": {},
   "outputs": [],
   "source": [
    "from firefly_client import FireflyClient\n",
    "import astropy.utils.data\n",
    "import urllib.parse"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5b88556a",
   "metadata": {},
   "source": [
    "## Instantiate the Firefly client\n",
    "\n",
    "There are two ways to initialize a Firefly client from Python, depending on whether you're running the notebook in JupyterLab or not. Assuming you have `jupyter-firefly-extensions` set up in your environment as explained [here](https://github.com/Caltech-IPAC/jupyter_firefly_extensions/blob/master/README.md), you can use `make_lab_client()` in JupyterLab, which will open the Firefly viewer in a new tab within the Lab. Otherwise, you can use `make_client()` in a Jupyter Notebook (or even a Python shell), which will open the Firefly viewer in a new web browser tab.\n",
    "\n",
    "You also need a Firefly server to communicate with your Firefly Python client. In this notebook, we use a public Firefly server: the IRSA Viewer (https://irsa.ipac.caltech.edu/irsaviewer). However, you can also run a local Firefly server via a [Firefly Docker image](https://hub.docker.com/r/ipac/firefly) and access it at `http://localhost:8080/firefly`. The URL of the Firefly server is read by both `make_client()` and `make_lab_client()` through the environment variable `FIREFLY_URL`. However, `make_client()` also allows you to pass the URL directly as the `url` parameter."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "71d10652-9dcd-4e7a-bb68-bf5737f3959b",
   "metadata": {},
   "source": [
    "**Exercise**: Use either `FireflyClient.make_lab_client()` or `FireflyClient.make_client()` to initialize the connection to Firefly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e00ece9c-67c7-49fc-b286-becbc53312e1",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Complete this code for initialization\n",
    "fc = "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d1d69be5-3804-4b30-b512-f0323c1251e5",
   "metadata": {},
   "source": [
    "**Exercise**: Use the `reinit_viewer()` method to clear the display"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "52b568e5-fef0-4ee6-b13d-50179e59c56e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Fill in code to clear the display"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c592622a",
   "metadata": {},
   "source": [
    "## Construct a TAP Query and display the retrieved table in Firefly \n",
    "\n",
    "TAP search the 'Known Solar System Object Possible Association List' catalog from the NEOWISE-R database. The specific target we are looking for is minor planet `558 Carmen`. We can query this target using a TAP search through IRSA; the `table_url` is broken down as follows:\n",
    "\n",
    "- We want to search the data through IRSA, which supports TAP querying, and we want it streamed directly to us via a synchronous search: <br>\"https://<!---->irsa.ipac.caltech.edu/TAP/sync\"<br><br>\n",
    "- Next, we want to structure query to only retrieve (558) Carmen data from the NEOWISE-R 'Known Solar System Object Possible Association List' catalog. The table name of the catalog can be found using [IRSAViewer](https://irsa.ipac.caltech.edu/irsaviewer/?__action=layout.showDropDown&view=MultiTableSearchCmd) and clicking the **VO TAP Search** tab and changing the 'Project' to **neowiser**. We query all columns of data and we search the target by its object id, which is its name, and use the 'like' condition to only write (558) with a wildcard as shown in the cell below.\n",
    "\n",
    "Construction of the query can be found in the [`IRSA TAP documentation page`](https://irsa.ipac.caltech.edu/docs/program_interface/TAP.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5e158519",
   "metadata": {},
   "outputs": [],
   "source": [
    "BASE_URL = \"https://irsa.ipac.caltech.edu/TAP/sync\"\n",
    "QUERY = \"\"\"\n",
    "SELECT *\n",
    "FROM neowiser_p1ba_mch AS n\n",
    "WHERE n.objid LIKE '(558)%'\n",
    "\"\"\"\n",
    "table_url = f\"{BASE_URL}?QUERY={urllib.parse.quote_plus(QUERY)}\"\n",
    "table_url"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "00c657f5-62e3-4816-a953-244dec63f568",
   "metadata": {},
   "source": [
    "Now, we can request the necessary data from the catalog and display the data as a table in the Firefly client, using the [`show_table`](https://caltech-ipac.github.io/firefly_client/api/firefly_client.FireflyClient.html#firefly_client.FireflyClient.show_table) method."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7fa5524b-e8e9-4848-8ae1-c013f4f69e35",
   "metadata": {},
   "source": [
    "**Exercise**: Use the `show_table` method along with the `table_url` to show the table. Specify the table ID to be \"tableneo\", and specify a descriptive title."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a58afe3b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Fill in code to show the table"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3aee82ef-1183-495c-83e4-1409f21fd734",
   "metadata": {},
   "source": [
    "Alternatively, we can download data from the catalog using [`astropy.utils.data.download_file`](https://docs.astropy.org/en/stable/api/astropy.utils.data.download_file.html) and upload it to the Firefly client shown in the cell below the first method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "92f85620",
   "metadata": {},
   "outputs": [],
   "source": [
    "# tablename = astropy.utils.data.download_file(table_url, timeout=120, cache=True)\n",
    "# file = fc.upload_file(tablename)\n",
    "# fc.show_table(file, tbl_id='tableneo', title='558 Carmen Catalog', page_size=50)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a8c42619",
   "metadata": {},
   "source": [
    "Note that along with the table, firefly also displays the coverage and chart associated with the table. It overlays colored squares for each row of the table onto a HiPS image, because the table contains recognizable celestial coordinates. It also creates a scatter plot of ra and dec from the table."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "42c35c2a",
   "metadata": {},
   "source": [
    "## Plot Light Curves in Firefly\n",
    "\n",
    "After retrieving the data and displaying it in the client, we can now create a light curve by plotting the Modified Julian Date ('mjd') in the abscissa and the magnitude from band W1 ('w1mpro') in the ordinate. We also flip the ordinate to accurately display magnitude."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c6b114d6-f9d2-44c8-ab00-a14d8810ec0d",
   "metadata": {},
   "source": [
    "**Exercise**: Use the `show_xyplot` method to plot the light curve from the table with ID of `\"tableneo\"`. Use the `xCol` and `yCol` parameters to specify `mjd` and `w1mpro`, and add `yOptions=\"flip\"` to flip the magnitude scale."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cc86e03c",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Fill in code for the light curve plot"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e89c7d02",
   "metadata": {},
   "source": [
    "## Overlay the catalog on a HiPS image\n",
    "\n",
    "Finally, we can overlay the catalog of data in the table onto a HiPS image of our choice, using the method [`show_hips`](https://caltech-ipac.github.io/firefly_client/api/firefly_client.FireflyClient.html#firefly_client.FireflyClient.show_hips). However, this method requires target coordinates for the object you want to analyze."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9b4903fe-661b-48b8-b7bc-98c1bf619f90",
   "metadata": {},
   "outputs": [],
   "source": [
    "target='229.851396;-9.720647;EQ_J2000'\n",
    "viewer_id = 'hipsDiv'\n",
    "hips_url = 'http://alasky.u-strasbg.fr/AllWISE/RGB-W4-W2-W1'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "06953956-b42b-46d1-8fe0-f9d029e982c9",
   "metadata": {},
   "source": [
    "**Exercise**: Use the `show_hips` method to display the HiPS image. Specify `WorldPt=target` along with other parameters like `hips_root_url` and `viewer_id`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6da58703-85d5-4d3f-b887-ad8041ae1bc3",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Fill in code to show the HiPS map"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bee28daa",
   "metadata": {},
   "source": [
    "## Summary\n",
    "\n",
    "Firefly allows you to visualize data for specific targets. In conjuction with Astropy, one can manipulate a catalog of observations to display a light curve in an instantiated Firefly enviroment on their web browser.\n",
    "\n",
    "1. We import all necessary modules to create a Firefly client and to download the catalog of data for our target.\n",
    "\n",
    "2. We start the client in our web browser to appropiately display our tables, plots and images.\n",
    "\n",
    "3. We use the TAP schema to display the data for our target &mdash; [`558 Carmen`](https://irsa.ipac.caltech.edu/irsaviewer/?__action=table.search&request=%7B%22startIdx%22%3A0%2C%22SearchMethod%22%3A%22AllSky%22%2C%22RequestedDataSet%22%3A%22NEOWISE%20Reactivation%20Database%22%2C%22id%22%3A%22GatorQuery%22%2C%22tbl_id%22%3A%22tbl_id-cf48-45%22%2C%22META_INFO%22%3A%7B%22title%22%3A%22WISE-neowiser_p1ba_mch%20(AllSky)%22%2C%22tbl_id%22%3A%22tbl_id-cf48-45%22%2C%22tbl_pref_key%22%3A%22WISE-neowiser_p1ba_mch%22%7D%2C%22catalogProject%22%3A%22WISE%22%2C%22catalog%22%3A%22neowiser_p1ba_mch%22%2C%22constraints%22%3A%22objid%20like%20%27%25(558)%20Carmen%25%27%22%2C%22pageSize%22%3A100%7D&options=%7B%22backgroundable%22%3Atrue%2C%22pageSize%22%3A100%7D) &mdash; via a table and visualize such data through charts.\n",
    "\n",
    "4. We finally overlay the catalog onto a HiPS image to dynamically view where our target has been observed in space."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "935a0c56",
   "metadata": {},
   "source": [
    "***\n",
    "\n",
    "## About This Notebook"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "32dee4db",
   "metadata": {},
   "source": [
    "**Author:** Eric Bratton II (IRSA Scientist) in conjunction with the IRSA Science Team\n",
    "\n",
    "**Updated:** 2024-12-19; updated 2025-01-09 to make exercises\n",
    "\n",
    "**Contact:** [the IRSA Helpdesk](https://irsa.ipac.caltech.edu/docs/help_desk.html) with questions or reporting problems."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "40d4f99b-2a14-446d-9d4e-d41479b6ef51",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}