ICLR 2024

ICLR is in its 12th year!  For the inaugural ICLR Test of Time award, the Program Chairs examined papers from ICLR 2013 & 2014, and looked for ones with long-lasting impact.  

Congratulations to the authors of the Test of Time winner and runner up!

Test of Time

Auto-Encoding Variational Bayes

Diederik Kingma, Max Welling

https://arxiv.org/abs/1312.6114 

Probabilistic modeling is one of the most fundamental ways in which we reason about the world.  This paper spearheaded the integration of deep learning with scalable probabilistic inference (amortized mean-field variational inference via a so-called reparameterization trick), giving rise to the Variational Autoencoder (VAE).  The lasting value of this work is rooted in its elegance.  The principles used to develop VAEs deepened our understanding of the interplay between deep learning and probabilistic modeling, and sparked the development of many subsequent interesting probabilistic models and encoding approaches.  A concurrent work by Rezende et al. also proposed a similar idea in the paper titled “Stochastic Backpropagation and Approximate Inference in Deep Generative Models” published at ICML 2014.

Runner Up

Intriguing properties of neural networks

Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, Rob Fergus

https://arxiv.org/abs/1312.6199 

With the rising popularity of deep neural networks in real applications, it is important to understand when and how neural networks might behave in undesirable ways.  This paper highlighted the issue that neural networks can be vulnerable to small almost imperceptible variations to the input.  This idea helped spawn the area of adversarial attacks (trying to fool a neural network) as well as adversarial defense (training a neural network to not be fooled).