deploy: 43d1936

_downloads/2113e51a6a17a0230a513258e99cc8d1/overlap_regridder.ipynb

@@ -87,7 +87,7 @@
       "cell_type": "markdown",
       "metadata": {},
       "source": [
-        "## Custom reductions\n\nIt's also possible to define your own reduction methods. Such a method is\ninserted during the ``.regrid`` call and compiled by `Numba`_ for performance.\n\nA valid reduction method must be compileable by Numba, and takes exactly three\narguments: ``values``, ``indices``, ``weights``.\n\n* ``values``: is the array containing the (float) source values.\n* ``weights``: contains the (float) overlap between the target face and the\n  source faces. The size of ``weights`` is equal to the size of ``indices``.\n* ``work``: used as a temporary workspace. Contains one float value for each\n  index in indices.\n\nXugrid regridder reduction functions are implemented in such a way. For a\nexample, an area weighted sum could be implemented as follows:\n\n"
+        "## Custom reductions\n\nIt's also possible to define your own reduction methods. Such a method is\ninserted during the ``.regrid`` call and compiled by `Numba`_ for performance.\n\nA valid reduction method must be compileable by Numba, and takes exactly three\narguments: ``values``, ``indices``, ``weights``.\n\n* ``values``: is the array containing the (float) source values.\n* ``weights``: contains the (float) overlap between the target face and the\n  source faces. The size of ``weights`` is equal to the size of ``values``.\n* ``work``: used as a temporary workspace of floats. The size of ``work`` is\n  equal to the size of ``values``.\n\nXugrid regridder reduction functions are implemented in such a way. For a\nexample, an area weighted sum could be implemented as follows:\n\n"
       ]
     },
     {
@@ -98,14 +98,14 @@
       },
       "outputs": [],
       "source": [
-        "def mean(values, weights, workspace):\n    total = 0.0\n    weight_sum = 0.0\n    for value, weight in zip(values, weights):\n        if ~np.isnan(value):\n            total = +value * weight\n            weight_sum += weight\n    if weight_sum == 0.0:\n        return np.nan\n    return total / weight_sum"
+        "def mean(values, weights, workspace):\n    total = 0.0\n    weight_sum = 0.0\n    for value, weight in zip(values, weights):\n        if ~np.isnan(value):\n            total += value * weight\n            weight_sum += weight\n    if weight_sum == 0.0:\n        return np.nan\n    return total / weight_sum"
       ]
     },
     {
       "cell_type": "markdown",
       "metadata": {},
       "source": [
-        "<div class=\"alert alert-info\"><h4>Note</h4><p>* Each reduction must return a single float.\n   * Custom reductions methods must be able to deal with NaN values as these\n     are commonly encountered in datasets as a \"no data value\".\n   * If Python features are used that are unsupported by Numba, you will get\n     somewhat obscure errors. In such a case, test your function with\n     synthetic values for ``values, weights, workspace``.\n   * The ``workspace`` array is provided to avoid dynamic memory allocations.\n     It is a an array of floats with the same size as ``values`` or\n     ``weights``. You may freely allocate new arrays within the reduction\n     function but it will impact performance.\n   * While we could have implemented a weighted mean as:\n     ``np.nansum(values * weights) / np.nansum(weights)``, the function above\n     is efficiently compiled by Numba and does not allocate.</p></div>\n\nTo use our custom method, we provide at initialization of the OverlapRegridder:\n\n"
+        "<div class=\"alert alert-info\"><h4>Note</h4><p>* Each reduction must return a single float.\n   * Always check for ``np.isnan(value)``: Custom reductions methods must be\n     able to deal with NaN values as these are commonly encountered in datasets\n     as a \"no data value\".\n   * If Python features are used that are unsupported by Numba, you will get\n     somewhat obscure errors. In such a case, ``numba.njit`` and test your\n     function separately with synthetic values for ``values, weights,\n     workspace``.\n   * The ``workspace`` array is provided to avoid dynamic memory allocations.\n     It is a an array of floats with the same size as ``values`` or\n     ``weights``. You may freely allocate new arrays within the reduction\n     function but it will impact performance. (Methods such as mode or median\n     require a workspace.)\n   * While we could have implemented a weighted mean as:\n     ``np.nansum(values * weights) / np.nansum(weights)``, the function above\n     is efficiently compiled by Numba and does not allocate temporary arrays.</p></div>\n\nTo use our custom method, we provide it at initialization of the\nOverlapRegridder:\n\n"
       ]
     },
     {
@@ -123,7 +123,7 @@
       "cell_type": "markdown",
       "metadata": {},
       "source": [
-        "Not every reduction uses the ``weights`` and ``workspace`` arguments. For\nexample, a regular sum:\n\n"
+        "Not every reduction uses the ``weights`` and ``workspace`` arguments. For\nexample, a regular sum could only look at the values:\n\n"
       ]
     },
     {
@@ -141,7 +141,7 @@
       "cell_type": "markdown",
       "metadata": {},
       "source": [
-        "Always ensure that the function can deal with NaN values!\n\n## Custom percentiles\n\nXugrid provides a number of predefined percentiles (5, 10, 25, 50, 75, 90,\n95). In case you need a different percentile value, you can use this utility:\n\n"
+        "## Custom percentiles\n\nXugrid provides a number of predefined percentiles (5, 10, 25, 50, 75, 90,\n95). In case you need a different percentile value, you can use this utility:\n\n"
       ]
     },
     {

_downloads/9510c04155266403a85c77c93b648199/overlap_regridder.py

@@ -113,9 +113,9 @@ def create_grid(bounds, nx, ny):
 #
 # * ``values``: is the array containing the (float) source values.
 # * ``weights``: contains the (float) overlap between the target face and the
-#   source faces. The size of ``weights`` is equal to the size of ``indices``.
-# * ``work``: used as a temporary workspace. Contains one float value for each
-#   index in indices.
+#   source faces. The size of ``weights`` is equal to the size of ``values``.
+# * ``work``: used as a temporary workspace of floats. The size of ``work`` is
+#   equal to the size of ``values``.
 #
 # Xugrid regridder reduction functions are implemented in such a way. For a
 # example, an area weighted sum could be implemented as follows:
@@ -126,7 +126,7 @@ def mean(values, weights, workspace):
     weight_sum = 0.0
     for value, weight in zip(values, weights):
         if ~np.isnan(value):
-            total = +value * weight
+            total += value * weight
             weight_sum += weight
     if weight_sum == 0.0:
         return np.nan
@@ -136,37 +136,39 @@ def mean(values, weights, workspace):
 # %%
 # .. note::
 #    * Each reduction must return a single float.
-#    * Custom reductions methods must be able to deal with NaN values as these
-#      are commonly encountered in datasets as a "no data value".
+#    * Always check for ``np.isnan(value)``: Custom reductions methods must be
+#      able to deal with NaN values as these are commonly encountered in datasets
+#      as a "no data value".
 #    * If Python features are used that are unsupported by Numba, you will get
-#      somewhat obscure errors. In such a case, test your function with
-#      synthetic values for ``values, weights, workspace``.
+#      somewhat obscure errors. In such a case, ``numba.njit`` and test your
+#      function separately with synthetic values for ``values, weights,
+#      workspace``.
 #    * The ``workspace`` array is provided to avoid dynamic memory allocations.
 #      It is a an array of floats with the same size as ``values`` or
 #      ``weights``. You may freely allocate new arrays within the reduction
-#      function but it will impact performance.
+#      function but it will impact performance. (Methods such as mode or median
+#      require a workspace.)
 #    * While we could have implemented a weighted mean as:
 #      ``np.nansum(values * weights) / np.nansum(weights)``, the function above
-#      is efficiently compiled by Numba and does not allocate.
+#      is efficiently compiled by Numba and does not allocate temporary arrays.
 #
-# To use our custom method, we provide at initialization of the OverlapRegridder:
+# To use our custom method, we provide it at initialization of the
+# OverlapRegridder:
 
 regridder = xu.OverlapRegridder(uda, grid, method=mean)
 result = regridder.regrid(uda)
 result.ugrid.plot(vmin=-20, vmax=90, cmap="terrain")
 
 # %%
 # Not every reduction uses the ``weights`` and ``workspace`` arguments. For
-# example, a regular sum:
+# example, a regular sum could only look at the values:
 
 
 def nansum(values, weights, workspace):
     return np.nansum(values)
 
 
 # %%
-# Always ensure that the function can deal with NaN values!
-#
 # Custom percentiles
 # ------------------
 #

_modules/xugrid/regrid/regridder.html

@@ -455,7 +455,7 @@ <h1>Source code for xugrid.regrid.regridder</h1><div class="highlight"><pre>
 
                 <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">indices</span><span class="p">)</span> <span class="o">&gt;</span> <span class="mi">0</span><span class="p">:</span>
                     <span class="n">out</span><span class="p">[</span><span class="n">extra_index</span><span class="p">,</span> <span class="n">target_index</span><span class="p">]</span> <span class="o">=</span> <span class="n">f</span><span class="p">(</span>
-                        <span class="n">values</span><span class="p">,</span> <span class="n">weights</span><span class="p">,</span> <span class="n">workspace</span><span class="p">[</span><span class="n">extra_index</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="p">:]</span>
+                        <span class="n">values</span><span class="p">,</span> <span class="n">weights</span><span class="p">,</span> <span class="n">workspace</span><span class="p">[</span><span class="n">extra_index</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="p">:</span><span class="n">n_value</span><span class="p">]</span>
                     <span class="p">)</span>
         <span class="k">return</span> <span class="n">out</span>
 
@@ -858,13 +858,21 @@ <h1>Source code for xugrid.regrid.regridder</h1><div class="highlight"><pre>
 
 <span class="sd">    Any percentile method can be created via:</span>
 <span class="sd">    ``method = OverlapRegridder.create_percentile_methode(percentile)``</span>
+<span class="sd">    See the examples.</span>
 
 <span class="sd">    Parameters</span>
 <span class="sd">    ----------</span>
 <span class="sd">    source: Ugrid2d, UgridDataArray</span>
 <span class="sd">    target: Ugrid2d, UgridDataArray</span>
 <span class="sd">    method: str, function, optional</span>
 <span class="sd">        Default value is ``&quot;mean&quot;``.</span>
+
+<span class="sd">    Examples</span>
+<span class="sd">    --------</span>
+<span class="sd">    Setup a custom percentile method and apply it:</span>
+
+<span class="sd">    &gt;&gt;&gt; p33_3 = OverlapRegridder.create_percentile_method(33.3)</span>
+<span class="sd">    &gt;&gt;&gt; regridder = OverlapRegridder(source, target, method=p33_3)</span>
 <span class="sd">    &quot;&quot;&quot;</span>
 
     <span class="n">_JIT_FUNCTIONS</span> <span class="o">=</span> <span class="p">{</span>

_sources/changelog.rst.txt

@@ -6,6 +6,31 @@ All notable changes to this project will be documented in this file.
 The format is based on `Keep a Changelog`_, and this project adheres to
 `Semantic Versioning`_.
 
+Unreleased
+----------
+
+Fixed
+~~~~~
+
+- The reduction methods for the overlap regridders now behave consistently when
+  all values are NaN or when all weights (overlaps) are zero, and all methods
+  give the same answer irrespective of the order in which the values are
+  encountered.
+
+Added
+~~~~~
+
+- Percentiles (5, 10, 25, 50, 75, 90, 95) have been added to the
+  :class:`xugrid.OverlapRegridder` as standard available reduction methods
+  (available as ``"p5", "p10"``, etc.). Custom percentile values (e.g. 2.5, 42) can be
+  setup using :meth:`xugrid.OverlapRegridder.create_percentile_method`.
+
+Changed
+~~~~~~~
+
+- Custom reduction functions provide to the overlap regridders no longer require
+  an ``indices`` argument.
+
 [0.11.0] 2024-08-05
 -------------------
 

_sources/examples-dev/sg_execution_times.rst.txt

@@ -6,7 +6,7 @@
 
 Computation times
 =================
-**00:01.160** total execution time for 1 file **from examples-dev**:
+**00:01.146** total execution time for 1 file **from examples-dev**:
 
 .. container::
 
@@ -33,5 +33,5 @@ Computation times
      - Time
      - Mem (MB)
    * - :ref:`sphx_glr_examples-dev_voronoi.py` (``voronoi.py``)
-     - 00:01.160
+     - 00:01.146
      - 0.0

_sources/examples-dev/voronoi.rst.txt

@@ -630,7 +630,7 @@ The figure shows:
 
 .. rst-class:: sphx-glr-timing
 
-   **Total running time of the script:** (0 minutes 1.160 seconds)
+   **Total running time of the script:** (0 minutes 1.146 seconds)
 
 
 .. _sphx_glr_download_examples-dev_voronoi.py:

_sources/examples/connectivity.rst.txt

@@ -129,7 +129,7 @@ By default, the border value for binary erosion is set to ``False`` (equal to
  .. code-block:: none
 
 
-    <matplotlib.collections.PolyCollection object at 0x7f30b1ec85c0>
+    <matplotlib.collections.PolyCollection object at 0x7f583cc285c0>
 
 
 
@@ -165,7 +165,7 @@ start by setting a single value in the center of the grid to ``True``.
  .. code-block:: none
 
 
-    <matplotlib.collections.PolyCollection object at 0x7f30a7ef0dd0>
+    <matplotlib.collections.PolyCollection object at 0x7f5847622570>
 
 
 
@@ -200,7 +200,7 @@ alternative border value:
  .. code-block:: none
 
 
-    <matplotlib.collections.PolyCollection object at 0x7f30a90bc5c0>
+    <matplotlib.collections.PolyCollection object at 0x7f583db86c90>
 
 
 
@@ -238,7 +238,7 @@ analyse connected parts of the mesh.
  .. code-block:: none
 
 
-    <matplotlib.collections.PolyCollection object at 0x7f30a8d2adb0>
+    <matplotlib.collections.PolyCollection object at 0x7f583daa43e0>
 
 
 
@@ -272,7 +272,7 @@ Tesselation.
  .. code-block:: none
 
 
-    <matplotlib.collections.LineCollection object at 0x7f30b1f9fd10>
+    <matplotlib.collections.LineCollection object at 0x7f583db50440>
 
 
 
@@ -316,7 +316,7 @@ the original.
  .. code-block:: none
 
 
-    <matplotlib.collections.LineCollection object at 0x7f30a8e2d7c0>
+    <matplotlib.collections.LineCollection object at 0x7f583ca32390>
 
 
 
@@ -355,7 +355,7 @@ We can break down one of the Voronoi tesselations from above into triangles:
  .. code-block:: none
 
 
-    <matplotlib.collections.LineCollection object at 0x7f30b2c31070>
+    <matplotlib.collections.LineCollection object at 0x7f583dd507a0>
 
 
 
@@ -409,7 +409,7 @@ the upper and lower parts:
  .. code-block:: none
 
 
-    <matplotlib.collections.LineCollection object at 0x7f30a8db8fb0>
+    <matplotlib.collections.LineCollection object at 0x7f583d9ccfb0>
 
 
 
@@ -439,7 +439,7 @@ We can now use Laplace interpolation to fill the gaps in the grid.
  .. code-block:: none
 
 
-    <matplotlib.collections.PolyCollection object at 0x7f30a7e274a0>
+    <matplotlib.collections.PolyCollection object at 0x7f583ce2c380>
 
 
 
@@ -477,7 +477,7 @@ interpolation.
  .. code-block:: none
 
 
-    <matplotlib.collections.PolyCollection object at 0x7f30a8b1d4f0>
+    <matplotlib.collections.PolyCollection object at 0x7f583cb1a840>
 
 
 
@@ -518,7 +518,7 @@ To illustrate, let's take a look at the connectivity matrix of the Xoxo grid.
  .. code-block:: none
 
 
-    <matplotlib.image.AxesImage object at 0x7f30a904f230>
+    <matplotlib.image.AxesImage object at 0x7f583dd53ce0>
 
 
 
@@ -554,14 +554,14 @@ locality:
  .. code-block:: none
 
 
-    <matplotlib.image.AxesImage object at 0x7f30a9002300>
+    <matplotlib.image.AxesImage object at 0x7f583d9204d0>
 
 
 
 
 .. rst-class:: sphx-glr-timing
 
-   **Total running time of the script:** (0 minutes 1.388 seconds)
+   **Total running time of the script:** (0 minutes 1.403 seconds)
 
 
 .. _sphx_glr_download_examples_connectivity.py: