deploy: 37fa23e

.buildinfo

@@ -0,0 +1,4 @@
+# Sphinx build info version 1
+# This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
+config: 5fad2bde940bf7165069a46eaa5a9d17
+tags: 645f666f9bcd5a90fca523b33c5a78b7

_sources/api.rst.txt

@@ -0,0 +1,13 @@
+API Reference
+=============
+
+.. automodule:: publicdb.api
+   :members:
+   :undoc-members:
+
+
+Contents:
+
+.. toctree::
+
+   api_views

_sources/api_tutorial.rst.txt

@@ -0,0 +1,179 @@
+.. include:: subst.inc
+
+API Tutorial
+============
+
+The |hisparc| |api| (Application Programming Interface) simplifies
+metadata access from other applications. In this tutorial, we'll give
+some examples of how this data can be accessed and used with Javascript
+(`jQuery <http://www.jquery.com/>`_) and `Python
+<http://www.python.org>`_. We'll show you how to do some neat things.
+How can you get a list of all |hisparc| stations in Denmark? What is the
+position of station 201? Which stations had data on 20 October 2010? How
+does the number of measured events change during a few weeks? This
+tutorial will give you an overview of some of possibilities with the
+|hisparc| |api|. For details on all available classes and methods,
+please see the :doc:`api`.
+
+.. note::
+
+    We'll require you to know some basic programming, i.e. to understand
+    what an :code:`if` statement is and :code:`for` loop does. If you
+    are new to coding you can try a tutorial online, for instance
+    `Codeacademy <http://www.codecademy.com/>`_, we recommend learning
+    Python or jQuery.
+
+
+First look
+----------
+
+First we will just look at what this |api| is. The |api| can be accessed
+via the internet by opening urls. Instead of a website you get data as a
+response. This data is formatted as a JSON (JavaScript Object Notation),
+this format can be understood by many programming languages.
+
+To see what options the |api| has we will look at it in a browser. Open
+the following link in your browser (this will not work in Internet
+Explorer): `https://data.hisparc.nl/api/ <https://data.hisparc.nl/api/>`_.
+
+You should now see some text, like this:
+
+.. code-block:: javascript
+
+    {"base_url": "https://data.hisparc.nl/api/",
+     "clusters": "clusters/",
+     "clusters_in_country": "countries/{country_id}/",
+     "configuration": "station/{station_id}/config/{year}/{month}/{day}/",
+     "countries": "countries/",
+     "has_data": "station/{station_id}/data/{year}/{month}/{day}/",
+     ...
+     "subclusters_in_cluster": "clusters/{cluster_id}/"}
+
+This is the JSON, it is a dictionary (indicated by the :code:`{` and
+:code:`}` enclosing brackets): an object which has keys and values. Each
+key (:code:`"clusters"`, :code:`"has_data"`) refers to a value
+(:code:`"clusters/"`,
+:code:`"station/{station_id}/data/{year}/{month}/{day}/"`).
+
+
+Cluster list
+^^^^^^^^^^^^
+
+This tells us that if we want a list of all clusters we need to use the
+clusters option by appending :code:`"clusters/"` to the base url,
+resulting in the following:
+`https://data.hisparc.nl/api/clusters/ <https://data.hisparc.nl/api/clusters/>`_.
+
+With this result:
+
+.. code-block:: javascript
+
+    [{"name": "Amsterdam",
+      "number": 0},
+     {"name": "Utrecht",
+      "number": 1000},
+     ...
+     {"name": "Karlsruhe",
+      "number": 70000}]
+
+This JSON is a list (indicated by the :code:`[` and :code:`]` enclosing
+brackets) of dictionaries, one for each cluster. Each dictionary
+contains the name and number of a cluster. This way information about
+the network of stations can be retrieved.
+
+
+Javascript example
+------------------
+
+The following code example will generate a webpage which will use the
+|api| to get an up-to-date list of stations. It will then show a
+drop-down menu from which a station can be selected, once you have
+chosen a station you can click the :code:`Get info` button to make
+Javascript get the station information. To try this you can either use
+this example page: `jsFiddle <http://jsfiddle.net/NCWXq/2/>`_ or create
+your own HTML file with this code:
+
+.. code-block:: html
+
+    <html>
+    <head>
+    <script src="http://code.jquery.com/jquery-1.11.1.min.js"></script>
+    <script>
+        $(function() {
+            // Get an up-to-date list of HiSPARC stations
+            $.getJSON(
+                'https://data.hisparc.nl/api/stations/',
+                function(data) {
+                    // Create the drop-down menu
+                    var select = $('<select>');
+                    var id, name;
+                    for (var i in data) {
+                        id = data[i].number;
+                        name = data[i].name;
+                        select.append($('<option>').attr('value', id).text(id + ' - ' + name));}
+                    $('#station_list').append(select);});
+
+            // Attach a function to the Get info button
+            $('#get_station').on('click', function() {
+                var id = $('#station_list').find('select').val();
+                // Get info for selected station and display it in a nice way
+                $.getJSON('https://data.hisparc.nl/api/station/' + id + '/',
+                          function(data) {
+                              $('#station_info').text(JSON.stringify(data, undefined, 4));
+                          });
+                });
+            });
+    </script>
+    </head>
+    <body style="font-family: sans-serif;">
+        <h2>Station info</h2>
+        <p id="station_list">Choose a station: </p>
+        <input type="submit" id="get_station" value="Get info">
+        <pre id="station_info"></pre>
+    </body>
+    </html>
+
+
+Python example
+--------------
+
+In this example we will use several standard Python libraries and the
+popular plotting library `matplotlib <http://matplotlib.org>`_ (pylab).
+We start by importing the required libraries, one to get data from the
+urls, one to make working with dates easy and the plotting library. Then
+define the start values and perpare two empty lists in which the data
+can be placed. Then a while loop is used to loop over all days between
+datum and end_datum, reading each corresponding url. Finally a plot is
+made, setting the dates against their values.
+
+Start Python and type (or copy/paste without the :code:`>>>`) the
+following lines of code:
+
+.. code-block:: python
+
+    >>> from urllib.request import urlopen
+    >>> from datetime import date, timedelta
+    >>> from pylab import plot, show
+    >>> id = 501
+    >>> datum = date(2010, 10, 1)
+    >>> end_datum = date(2011, 2, 1)
+    >>> base = 'https://data.hisparc.nl/api/station/%d/num_events/%d/%d/%d'
+    >>> events = []
+    >>> dates = []
+    >>> while datum < end_datum:
+    ...     url = urlopen(base % (id, datum.year, datum.month, datum.day))
+    ...     events.append(url.read())
+    ...     dates.append(datum)
+    ...     datum += timedelta(days=1)
+    ...
+    >>> step(dates, events)
+    >>> show()
+
+
+SAPPHiRE
+^^^^^^^^
+
+The HiSPARC Python framework SAPPHiRE includes an API module. This module
+simplifies access to the API. See the SAPPHiRE documentation for more
+information:
+`https://docs.hisparc.nl/sapphire/ <https://docs.hisparc.nl/sapphire/>`_.

_sources/api_views.rst.txt

@@ -0,0 +1,11 @@
+API Views Reference
+===================
+
+.. automodule:: publicdb.api.views
+   :members:
+   :undoc-members:
+
+
+Contents:
+
+.. toctree::

_sources/data_access.rst.txt

@@ -0,0 +1,105 @@
+.. include:: subst.inc
+
+Data access
+===========
+
+There are several ways in which the |hisparc| data can be accessed:
+
+- Via the Public database `download forms
+  <https://data.hisparc.nl/data/download/>`_.
+- Via Python using the |sapphire| framework (see `SAPPHiRE Tutorial
+  <https://docs.hisparc.nl/sapphire/tutorial.html#downloading-and-accessing
+  -hisparc-data>`_).
+- Via the `jSparc <https://data.hisparc.nl/media/jsparc/>`_ web applications.
+
+To access metadata, like a list of all stations or information about a
+specific station see the :doc:`api_tutorial`.
+
+
+Download form
+-------------
+
+When looking at the data page for a station (i.e. `Kaj Munk College
+<https://data.hisparc.nl/show/stations/202/>`_), you will see a 'Download
+event summary data' link on the right. When this link is clicked you
+will be taken to the `Data download form
+<https://data.hisparc.nl/data/download>`_. On this page you can select
+the station, the start and end date for which you want to download the
+data. There is also an option to download weather data instead of events,
+however, not all stations gather weather data.
+
+When you click the Submit button without checking the checkbox to
+Download, the data should (Firefox always downloads the data) show up in
+your browser window. If you check the Download box the data will be
+downloaded to your PC as a csv file.
+
+This csv file can be read by many programs, including Excel. Use the
+Import option in Excel to make it recognize tabs as delimiter between
+columns.
+
+
+Downloading via Python
+----------------------
+
+.. note::
+
+    An easy to use module has been added to |sapphire| to download
+    data from the ESD. The following is an old (but working) example.
+
+This is an example of how to download the data from the Public database
+in Python. In this example we will download 1 hour of data for station
+202 on July 2, 2013.
+
+First load the required libraries (requires the numpy package).
+
+.. code-block:: python
+
+    >>> from datetime import datetime
+    >>> from urllib import urlencode
+    >>> from urllib.request import urlopen
+    >>> from StringIO import StringIO
+    >>> from numpy import genfromtxt
+
+Then define the url and place the start and end datetime in the query.
+To download weather data instead, replace 'events' by 'weather' in
+the url (and choose a station that has weather data, e.g. 3 or 501).
+
+.. note::
+
+    Do not pass the query to the urlopen 'data' argument because that
+    changes the request into a *POST* request, but you need a *GET*
+    request.
+
+.. code-block:: python
+
+    >>> url = 'https://data.hisparc.nl/data/202/events'
+    >>> start = str(datetime(2013, 7, 2, 11, 0))
+    >>> end = str(datetime(2013, 7, 2, 12, 0))
+    >>> query = urlencode({'download': False, 'start': start,'end': end})
+    >>> full_url = url + '?' + query
+
+Download the data and store it in a variable
+
+.. code-block:: python
+
+    >>> data = urlopen(full_url).read()
+
+Now use numpy to convert the data from csv to a numpy array.
+
+.. code-block:: python
+
+    >>> format = [('date', 'datetime64[D]'), ('time', '|S8'),
+                  ('timestamp', 'uint32'), ('nanoseconds', 'uint32'),
+                  ('pulseheights', '4int16'), ('integrals', '4int32'),
+                  ('n1', 'float32'), ('n2', 'float32'),
+                  ('n3', 'float32'), ('n4', 'float32'),
+                  ('t1', 'float32'), ('t2', 'float32'),
+                  ('t3', 'float32'), ('t4', 'float32'),
+                  ('t_trigger', 'float32'),
+                  ('zenith', 'int16'), ('azimuth', 'int16')]
+    >>> a = genfromtxt(StringIO(data), delimiter="\t", dtype=format)
+    >>> print(a[0])
+    (datetime.date(2013, 7, 2), '11:00:02', 1372762802L, 466307811L,
+     [78, 798, -1, -1], [535, 10882, -1, -1],
+     0.14720000326633453, 3.854599952697754, -1.0, -1.0,
+     345.0, 12.5, -1.0, -1.0, 345.0, -999, -999)