Weight-space learning is a rapidly-growing area within machine learning, centered on learning functions of neural network weights. This repository is a collection of weight-space learning papers.
It is not exhaustive and will likely never be, but feel free to open a PR if you want to see a paper listed (self-promotion is welcome 🙃).
Table of Contents
- Schürholt et al - Towards Scalable and Versatile Weight Space Learning (2024)
- Kalogeropoulos et al - Scale Equivariant Graph Metanetworks (2024)
- Kofinas et al - Graph Neural Networks for Learning Equivariant Representations of Neural Networks (2024)
- Zhou et al - Universal Neural Functionals (2024)
- Zhou et al - Permutation Equivariant Neural Functionals (2023)
- Zhou et al - Neural Functional Transformers (2023)
- Lim et al - Graph Metanetworks for Processing Diverse Neural Architectures (2023)
- Navon et al - Equivariant Architectures for Learning in Deep Weight Spaces (2023)
- Unterthiner et al - Predicting Neural Network Accuracy from Weights (2021)
- Erkoç et al - HyperDiffusion: Generating Implicit Neural Fields with Weight-Space Diffusion (2023)
- Schürholt et al - Hyper-Representations as Generative Models: Sampling Unseen Neural Network Weights (2022)
- Peebles et al - Learning to Learn with Generative Models of Neural Network Checkpoints (2022)
- Schürholt et al - Self-Supervised Representation Learning on Neural Network Weights for Model Characteristic Prediction (2021)
- Schürholt et al - Model Zoos: A Dataset of Diverse Populations of Neural Network Models (2022)
- Kahana et al - Deep Linear Probe Generators for Weight Space Learning (2024)
- Navon et al - Equivariant Deep Weight Space Alignment (2024)
- Shamsian et al - Improved Generalization of Weight Space Networks via Augmentations (2024)
- Erdogan E. - Geometric Flow Models over Neural Network Weights (2025)
- Schürholt K. - Hyper-Representations: Learning from Populations of Neural Networks (2024)