Jasper Snoek
CRCS Postdoctoral Fellow
Intelligent and Probabilistic Systems Group
School of Engineering and Applied Science
Harvard University
jsnoek at seas dot harvard dot edu

About

I am a CRCS postdoctoral fellow at Harvard University and a member of the Harvard Intelligent and Probabilistic Systems Group. My research interests are primarily in machine learning with an emphasis on deep learning and Bayesian statistics. I have been involved in numerous projects that involve the application of machine learning to problems in health informatics and assistive technology. I was formerly a member of the machine learning group at the University of Toronto and the Intelligent Assistive Technology and Systems Lab. I am also a cofounder of Whetlab, a machine learning company that's building AI for AI.


Selected Publications

Practical Bayesian Optimization of Machine Learning Algorithms
Jasper Snoek, Hugo Larochelle and Ryan Prescott Adams
Neural Information Processing Systems, 2012
The use of machine learning algorithms frequently involves careful tuning of learning parameters and model hyperparameters. Unfortunately, this tuning is often a “black art” requiring expert experience, rules of thumb, or sometimes brute-force search. There is therefore great appeal for automatic approaches that can optimize the performance of any given learning algorithm to the problem at hand. In this work, we consider this problem through the framework of Bayesian optimization, in which a learning algorithm’s generalization performance is modeled as a sample from a Gaussian process (GP). We show that certain choices for the nature of the GP, such as the type of kernel and the treatment of its hyperparameters, can play a crucial role in obtaining a good optimizer that can achieve expert- level performance. We describe new algorithms that take into account the variable cost (duration) of learning algorithm experiments and that can leverage the presence of multiple cores for parallel experimentation. We show that these proposed algorithms improve on previous automatic procedures and can reach or surpass human expert-level optimization for many algorithms including latent Dirichlet allocation, structured SVMs and convolutional neural networks.
@inproceedings{snoek-etal-2012b,
  author = {Snoek, Jasper and Larochelle, Hugo and Adams, Ryan Prescott},
  booktitle = {Neural Information Processing Systems},
  year = {2012},
  title = {Practical Bayesian Optimization of Machine Learning Algorithms},
}
		
Nonparametric Guidance of Autoencoder Representations using Label Information
Jasper Snoek, Ryan Prescott Adams and Hugo Larochelle
Journal of Machine Learning Research, 2012
While unsupervised learning has long been useful for density modeling, exploratory data analysis and visualization, it has become increasingly important for discovering features that will later be used for discriminative tasks. Discriminative algorithms often work best with highly-informative features; remarkably, such features can often be learned without the labels. One particularly effective way to perform such unsupervised learning has been to use autoencoder neural networks, which find latent representations that are constrained but nevertheless informative for reconstruction. However, pure unsupervised learning with autoencoders can find representations that may or may not be useful for the ultimate discriminative task. It is a continuing challenge to guide the training of an autoencoder so that it finds features which will be useful for predicting labels. Similarly, we often have a priori information regarding what statistical variation will be irrelevant to the ultimate discriminative task, and we would like to be able to use this for guidance as well. Although a typical strategy would be to include a parametric discriminative model as part of the autoencoder training, here we propose a nonparametric approach that uses a Gaussian process to guide the representation. By using a nonparametric model, we can ensure that a useful discriminative function exists for a given set of features, without explicitly instantiating it. We demonstrate the superiority of this guidance mechanism on four data sets, including a real-world application to rehabilitation research. We also show how our proposed approach can learn to explicitly ignore statistically significant covariate information that is label-irrelevant, by evaluating on the small NORB image recognition problem in which pose and lighting labels are available.
@article{snoek-etal-2012c,
  author = {Snoek, Jasper and Adams, Ryan Prescott and Larochelle, Hugo},
  title = {Nonparametric Guidance of Autoencoder Representations using Label Information},
  month = {10/2012},
  volume = {13},
  pages = {2567-2588},
  year = {2012},
  journal = {Journal of Machine Learning Research},
}
	      
  • DSC08563_lq.jpg