Skip to content

Commit

Permalink
Fix broken references (#614)
Browse files Browse the repository at this point in the history
* Fixed references in black-box likelihood notebook

* Updated profiling.ipynb to v5.10 and fixed broken references

* Deleted sphinxext.egg-info

* Fixed references in LKJ.ipynb

* Fixed references of Model_Builder.ipynb

* Fixed references in spline.ipynb

* Fixed references in wrapping_jax_function.ipynb

* Fixed references in howto_debugging.ipynb

* Fixed references in updating_priors.ipynb
  • Loading branch information
itsdivya1309 authored Jan 21, 2024
1 parent a211f23 commit 6f2eb44
Show file tree
Hide file tree
Showing 18 changed files with 1,170 additions and 338 deletions.
4 changes: 2 additions & 2 deletions examples/howto/LKJ.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -222,7 +222,7 @@
"id": "59FtijDir2Pe"
},
"source": [
"We use [expand_packed_triangular](../api/math.rst) to transform this vector into the lower triangular matrix $\\mathbf{L}$, which appears in the Cholesky decomposition $\\Sigma = \\mathbf{L} \\mathbf{L}^{\\top}$."
"We use {func}`expand_packed_triangular <pymc.expand_packed_triangular>` to transform this vector into the lower triangular matrix $\\mathbf{L}$, which appears in the Cholesky decomposition $\\Sigma = \\mathbf{L} \\mathbf{L}^{\\top}$."
]
},
{
Expand Down Expand Up @@ -919,5 +919,5 @@
}
},
"nbformat": 4,
"nbformat_minor": 1
"nbformat_minor": 4
}
2 changes: 1 addition & 1 deletion examples/howto/LKJ.myst.md
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,7 @@ packed_L.eval()

+++ {"id": "59FtijDir2Pe"}

We use [expand_packed_triangular](../api/math.rst) to transform this vector into the lower triangular matrix $\mathbf{L}$, which appears in the Cholesky decomposition $\Sigma = \mathbf{L} \mathbf{L}^{\top}$.
We use {func}`expand_packed_triangular <pymc.expand_packed_triangular>` to transform this vector into the lower triangular matrix $\mathbf{L}$, which appears in the Cholesky decomposition $\Sigma = \mathbf{L} \mathbf{L}^{\top}$.

```{code-cell} ipython3
---
Expand Down
872 changes: 865 additions & 7 deletions examples/howto/Missing_Data_Imputation.ipynb

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion examples/howto/Missing_Data_Imputation.myst.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ jupytext:
format_name: myst
format_version: 0.13
kernelspec:
display_name: Python 3
display_name: Python 3 (ipykernel)
language: python
name: python3
---
Expand Down
202 changes: 65 additions & 137 deletions examples/howto/blackbox_external_likelihood_numpy.ipynb

Large diffs are not rendered by default.

18 changes: 12 additions & 6 deletions examples/howto/blackbox_external_likelihood_numpy.myst.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,8 @@ print(f"Running on PyMC v{pm.__version__}")
az.style.use("arviz-darkgrid")
```

+++ {"jp-MarkdownHeadingCollapsed": true}

## Introduction
PyMC is a great tool for doing Bayesian inference and parameter estimation. It has a load of {doc}`in-built probability distributions <pymc:api/distributions>` that you can use to set up priors and likelihood functions for your particular model. You can even create your own {ref}`custom distributions <custom_distribution>`.

Expand Down Expand Up @@ -108,9 +110,9 @@ ValueError: setting an array element with a sequence.

This is because `m` and `c` are PyTensor tensor-type objects.

So, what we actually need to do is create a [PyTensor Op](http://deeplearning.net/software/pytensor/extending/extending_pytensor.html). This will be a new class that wraps our log-likelihood function (or just our model function, if that is all that is required) into something that can take in PyTensor tensor objects, but internally can cast them as floating point values that can be passed to our log-likelihood function. We will do this below, initially without defining a [grad() method](http://deeplearning.net/software/pytensor/extending/op.html#grad) for the Op.
So, what we actually need to do is create a {ref}`PyTensor Op <pytensor:creating_an_op>`. This will be a new class that wraps our log-likelihood function (or just our model function, if that is all that is required) into something that can take in PyTensor tensor objects, but internally can cast them as floating point values that can be passed to our log-likelihood function. We will do this below, initially without defining a {func}`grad` for the Op.

+++
+++ {"jp-MarkdownHeadingCollapsed": true}

## PyTensor Op without grad

Expand Down Expand Up @@ -201,9 +203,11 @@ with pm.Model():
az.plot_trace(idata_mh, lines=[("m", {}, mtrue), ("c", {}, ctrue)]);
```

+++ {"jp-MarkdownHeadingCollapsed": true}

## PyTensor Op with grad

What if we wanted to use NUTS or HMC? If we knew the analytical derivatives of the model/likelihood function then we could add a {ref}`grad() method <pytensor:creating_an_op>` to the Op using that analytical form.
What if we wanted to use NUTS or HMC? If we knew the analytical derivatives of the model/likelihood function then we could add a {func}`grad() method <pytensor:creating_an_op>` to the Op using that analytical form.

But, what if we don't know the analytical form. If our model/likelihood is purely Python and made up of standard maths operators and Numpy functions, then the [autograd](https://github.com/HIPS/autograd) module could potentially be used to find gradients (also, see [here](https://github.com/ActiveState/code/blob/master/recipes/Python/580610_Auto_differentiation/recipe-580610.py) for a nice Python example of automatic differentiation). But, if our model/likelihood truly is a "black box" then we can just use the good-old-fashioned [finite difference](https://en.wikipedia.org/wiki/Finite_difference) to find the gradients - this can be slow, especially if there are a large number of variables, or the model takes a long time to evaluate. Below, a function to find gradients has been defined that uses the finite difference (the central difference) - it uses an iterative method with successively smaller interval sizes to check that the gradient converges. But, you could do something far simpler and just use, for example, the SciPy {func}`~scipy.optimize.approx_fprime` function.

Expand Down Expand Up @@ -352,6 +356,8 @@ with pm.Model() as opmodel:
_ = az.plot_trace(idata_grad, lines=[("m", {}, mtrue), ("c", {}, ctrue)])
```

+++ {"jp-MarkdownHeadingCollapsed": true}

## Comparison to equivalent PyMC distributions
Now, finally, just to check things actually worked as we might expect, let's do the same thing purely using PyMC distributions (because in this simple example we can!)

Expand Down Expand Up @@ -406,7 +412,7 @@ pair_kwargs["marginal_kwargs"]["color"] = "C2"
az.plot_pair(idata, **pair_kwargs, ax=ax);
```

We can now check that the gradient Op works as expected. First, just create and call the `LogLikeGrad` class, which should return the gradient directly (note that we have to create a [PyTensor function](http://deeplearning.net/software/pytensor/library/compile/function.html) to convert the output of the Op to an array). Secondly, we call the gradient from `LogLikeWithGrad` by using the [PyTensor tensor gradient](http://deeplearning.net/software/pytensor/library/gradient.html#pytensor.gradient.grad) function. Finally, we will check the gradient returned by the PyMC model for a Normal distribution, which should be the same as the log-likelihood function we defined. In all cases we evaluate the gradients at the true values of the model function (the straight line) that was created.
We can now check that the gradient Op works as expected. First, just create and call the `LogLikeGrad` class, which should return the gradient directly (note that we have to create a {ref}`PyTensor function <pytensor:creating_an_op>` to convert the output of the Op to an array). Secondly, we call the gradient from `LogLikeWithGrad` by using the {func}`grad` function. Finally, we will check the gradient returned by the PyMC model for a Normal distribution, which should be the same as the log-likelihood function we defined. In all cases we evaluate the gradients at the true values of the model function (the straight line) that was created.

```{code-cell} ipython3
ip = pymodel.initial_point()
Expand All @@ -421,15 +427,15 @@ print(f'Gradient of model using a PyMC "Normal" distribution:\n {grad_vals_py

We could also do some profiling to compare performance between implementations. The {ref}`profiling` notebook shows how to do it.

+++
+++ {"jp-MarkdownHeadingCollapsed": true}

## Authors

* Adapted from [Jørgen Midtbø](https://github.com/jorgenem/)'s [example](https://discourse.pymc.io/t/connecting-pymc-to-external-code-help-with-understanding-pytensor-custom-ops/670) by Matt Pitkin both as a [blogpost](http://mattpitkin.github.io/samplers-demo/pages/pymc-blackbox-likelihood/) and as an example notebook to this gallery in August, 2018 ([pymc#3169](https://github.com/pymc-devs/pymc/pull/3169) and [pymc#3177](https://github.com/pymc-devs/pymc/pull/3177))
* Updated by [Oriol Abril](https://github.com/OriolAbril) on December 2021 to drop the Cython dependency from the original notebook and use numpy instead ([pymc-examples#28](https://github.com/pymc-devs/pymc-examples/pull/28))
* Re-executed by Oriol Abril with pymc 5.0.0 ([pymc-examples#496](https://github.com/pymc-devs/pymc-examples/pull/496))

+++
+++ {"jp-MarkdownHeadingCollapsed": true}

## Watermark

Expand Down
6 changes: 3 additions & 3 deletions examples/howto/howto_debugging.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
"## Introduction\n",
"There are various levels on which to debug a model. One of the simplest is to just print out the values that different variables are taking on.\n",
"\n",
"Because `PyMC` uses `PyTensor` expressions to build the model, and not functions, there is no way to place a `print` statement into a likelihood function. Instead, you can use the `pytensor.printing.Print` class to print intermediate values."
"Because `PyMC` uses `PyTensor` expressions to build the model, and not functions, there is no way to place a `print` statement into a likelihood function. Instead, you can use the {class}`pytensor.printing.Print` class to print intermediate values."
]
},
{
Expand Down Expand Up @@ -405,7 +405,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Raw output is a bit messy and requires some cleanup and formatting to convert to `numpy.ndarray`. In the example below regex is used to clean up the output, and then it is evaluated with `eval` to give a list of floats. Code below also works with higher-dimensional outputs (in case you want to experiment with different models)."
"Raw output is a bit messy and requires some cleanup and formatting to convert to {ref}`numpy.ndarray`. In the example below regex is used to clean up the output, and then it is evaluated with `eval` to give a list of floats. Code below also works with higher-dimensional outputs (in case you want to experiment with different models)."
]
},
{
Expand Down Expand Up @@ -564,7 +564,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.5"
"version": "3.11.6"
}
},
"nbformat": 4,
Expand Down
4 changes: 2 additions & 2 deletions examples/howto/howto_debugging.myst.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ kernelspec:
## Introduction
There are various levels on which to debug a model. One of the simplest is to just print out the values that different variables are taking on.

Because `PyMC` uses `PyTensor` expressions to build the model, and not functions, there is no way to place a `print` statement into a likelihood function. Instead, you can use the `pytensor.printing.Print` class to print intermediate values.
Because `PyMC` uses `PyTensor` expressions to build the model, and not functions, there is no way to place a `print` statement into a likelihood function. Instead, you can use the {class}`pytensor.printing.Print` class to print intermediate values.

```{code-cell} ipython3
import arviz as az
Expand Down Expand Up @@ -150,7 +150,7 @@ sys.stdout = old_stdout # setting sys.stdout back
output
```

Raw output is a bit messy and requires some cleanup and formatting to convert to `numpy.ndarray`. In the example below regex is used to clean up the output, and then it is evaluated with `eval` to give a list of floats. Code below also works with higher-dimensional outputs (in case you want to experiment with different models).
Raw output is a bit messy and requires some cleanup and formatting to convert to {ref}`numpy.ndarray`. In the example below regex is used to clean up the output, and then it is evaluated with `eval` to give a list of floats. Code below also works with higher-dimensional outputs (in case you want to experiment with different models).

```{code-cell} ipython3
import re
Expand Down
16 changes: 12 additions & 4 deletions examples/howto/model_builder.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@
"source": [
"Many users face difficulty in deploying their PyMC models to production because deploying/saving/loading a user-created model is not well standardized. One of the reasons behind this is there is no direct way to save or load a model in PyMC like scikit-learn or TensorFlow. The new `ModelBuilder` class is aimed to improve this workflow by providing a scikit-learn inspired API to wrap your PyMC models.\n",
"\n",
"The new `ModelBuilder` class allows users to use methods to `fit()`, `predict()`, `save()`, `load()`. Users can create any model they want, inherit the `ModelBuilder` class, and use predefined methods."
"The new {class}`ModelBuilder <pymc_experimental.model_builder.ModelBuilder>` class allows users to use methods to `fit()`, `predict()`, `save()`, `load()`. Users can create any model they want, inherit the {class}`ModelBuilder <pymc_experimental.model_builder.ModelBuilder>` class, and use predefined methods."
]
},
{
Expand All @@ -44,7 +44,15 @@
"execution_count": 1,
"id": "48e35045",
"metadata": {},
"outputs": [],
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"WARNING (pytensor.tensor.blas): Using NumPy C-API based implementation for BLAS functions.\n"
]
}
],
"source": [
"from typing import Dict, List, Optional, Tuple, Union\n",
"\n",
Expand Down Expand Up @@ -225,7 +233,7 @@
"source": [
"How would we deploy this model? Save the fitted model, load it on an instance, and predict? Not so simple.\n",
"\n",
"`ModelBuilder` is built for this purpose. It is currently part of the `pymc-experimental` package which we can pip install with `pip install pymc-experimental`. As the name implies, this feature is still experimental and subject to change."
"`ModelBuilder` is built for this purpose. It is currently part of the {ref}`pymc-experimental` package which we can pip install with `pip install pymc-experimental`. As the name implies, this feature is still experimental and subject to change."
]
},
{
Expand Down Expand Up @@ -959,7 +967,7 @@
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
Expand Down
6 changes: 3 additions & 3 deletions examples/howto/model_builder.myst.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ jupytext:
format_name: myst
format_version: 0.13
kernelspec:
display_name: Python 3
display_name: Python 3 (ipykernel)
language: python
name: python3
---
Expand All @@ -25,7 +25,7 @@ kernelspec:

Many users face difficulty in deploying their PyMC models to production because deploying/saving/loading a user-created model is not well standardized. One of the reasons behind this is there is no direct way to save or load a model in PyMC like scikit-learn or TensorFlow. The new `ModelBuilder` class is aimed to improve this workflow by providing a scikit-learn inspired API to wrap your PyMC models.

The new `ModelBuilder` class allows users to use methods to `fit()`, `predict()`, `save()`, `load()`. Users can create any model they want, inherit the `ModelBuilder` class, and use predefined methods.
The new {class}`ModelBuilder <pymc_experimental.model_builder.ModelBuilder>` class allows users to use methods to `fit()`, `predict()`, `save()`, `load()`. Users can create any model they want, inherit the {class}`ModelBuilder <pymc_experimental.model_builder.ModelBuilder>` class, and use predefined methods.

+++

Expand Down Expand Up @@ -79,7 +79,7 @@ with pm.Model() as model:

How would we deploy this model? Save the fitted model, load it on an instance, and predict? Not so simple.

`ModelBuilder` is built for this purpose. It is currently part of the `pymc-experimental` package which we can pip install with `pip install pymc-experimental`. As the name implies, this feature is still experimental and subject to change.
`ModelBuilder` is built for this purpose. It is currently part of the {ref}`pymc-experimental` package which we can pip install with `pip install pymc-experimental`. As the name implies, this feature is still experimental and subject to change.

+++

Expand Down
Loading

0 comments on commit 6f2eb44

Please sign in to comment.