2023 IAIFI Summer School

1. Differential Equations and Generative Models

(Author: Miranda Cheng)

Useful materials

Tutorial

(Author: Anindita Maiti)

Outline of tutorials

• Tutorial 3.1 KL Divergence (Solution)
• Tutorial 3.2 Neural ODEs (Solution)
• Tutorial 3.3 Normalizing Flows for Scalar Field Theory (Solution will be provided during actual tutorial session.)

Reference papers and book chapters for tutorials

• Neural Ordinary Differential Equations by Chen et. al.
• Introduction to Normalizing Flows for Lattice Field Theory by Boyda et. al.
• Learning Lattice Quantum Field Theories with Equivariant Continuous Flows by Gerdes et. al.
• Variational Inference with Normalizing Flows by Rezende et. al.
• Quantum Field Theory, Mark Srednicki, Cembridge University Press, Chapter 9.
• Quantum Field on a Lattice, Istvan Montvay and Gernot Munster, Cambridge University Press, Chapters 1, 2.

2. Statistical Physics and Geometry of Overparameterization

(Author: Pankaj Mehta)

Useful materials

Tutorial

(Author: Di Luo)

Overparameterization code

Tutorial 1.1 overparameterization.ipynb

Tutorial 1.2 linreg_diabetes.ipynb

Tutorial 1.3 linreg_ising.ipnyb

3. Generative modeling, with connection to and applications in physics

(Author: Siddharth Mishra-Sharma)

• Slides: https://smsharma.github.io/iaifi-summer-school-2023/

Useful materials

Introductory posts:

• Musings on typicality by Sander Dieleman
• A Path to the Variational Diffusion Loss by Alex Alemi
• Flow-based Deep Generative Models by Lilian Weng
• Perspectives on diffusion by Sander Dieleman

Code:

• Minified generative models: A repository with minimal/pedagogical implementations of some generative models.

Tutorial

(Author: Carol Cuesta-Lazaro)

• Tutorial's repo
• Tutorial 1: Understanding Variational Inference with jax
• Tutorial 2: Latent variable models for galaxy images with VAEs
• Tutorial 3: Diffusion models for particle physics

4. Astrostatistics

(Author: Joshua Speagle)

• Slides: https://github.com/alexandergagliano/InterpretabilityDemos/blob/main/JoshuaSpeagle_iaifi_bayesml_08-10-2023.pdf

Useful materials

Shapley Values:

• A Unified Approach to Interpreting Model Predictions (Lundberg & Lee, 2017)
• Explaining Predictive Uncertainty with Information Theoretic Shapley Values (Watson et al., 2023)
• Interpretable Machine Learning, Christoph Molnar

Counterfactuals:

• Alibi Explain: Algorithms for Explaining Machine Learning Models (Klaise, Van Looveren, Vacanti, & Alexandru Coc, 2017)
• Machine Learning Tutorials, Michelle Lochner
• Interpretable Counterfactual Explanations Guided by Prototypes, (Van Looveren & Klaise, 2019)

Saliency Maps:

• Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps, Simonyan et al., 2013 - implemented here.
• Visualizing and Understanding Convolutional Networks, Zeiler et al. (2013)
• Striving for Simplicity - The All Convolutional Net, Springenberg et al. (2015)
• SmoothGrad: removing noise by adding noise, Smilkov et al. (2017)

Tutorial

(Author: Alex Gagliano)

• Tutorial's repo
• Tutorial 1: SHAP Values: Origins and Applications
• Tutorial 2: Counterfactual Explanations
• Tutorial 3: Gradient-Based Saliency Maps for Neural Network Interpretability

5. Incorporating Symmetry into Deep Dynamics Models

(Author: Robin Walters)

Useful materials

Tutorial

(Author: Rui Wang)

• Tutorial Symmetry and Equivariant Models