Research - Sébastien M. R. Arnold

Research

Home / Research / Engineering / Blog

\( % Universal Mathematics \newcommand{\paren}[1]{\left( #1 \right)} \newcommand{\brackets}[1]{\left[ #1 \right]} \newcommand{\braces}[1]{\left\{ #1 \right\}} \newcommand{\norm}[1]{\left\lVert#1\right\rVert} \newcommand{\case}[1]{\begin{cases} #1 \end{cases}} \newcommand{\bigO}[1]{\mathcal{O}\left(#1\right)} % Analysis % Linear Algebra \newcommand{\mat}[1]{\begin{pmatrix}#1\end{pmatrix}} \newcommand{\bmat}[1]{\begin{bmatrix}#1\end{bmatrix}} % Probability Theory \DeclareMathOperator*{\V}{\mathop{\mathrm{Var}}} \DeclareMathOperator*{\E}{\mathop{\mathbb{E}}} \newcommand{\Exp}[2][]{\E_{#1}\brackets{#2}} \newcommand{\Var}[2][]{\V_{#1}\brackets{#2}} \newcommand{\Cov}[2][]{\mathop{\mathrm{Cov}}_{#1}\brackets{#2}} % Optimization \newcommand{\minimize}{\operatorname*{minimize}} \newcommand{\maximize}{\operatorname*{maximize}} \DeclareMathOperator*{\argmin}{arg\,min} \DeclareMathOperator*{\argmax}{arg\,max} % Set Theory \newcommand{\C}{\mathbb{C}} \newcommand{\N}{\mathbb{N}} \newcommand{\Q}{\mathbb{Q}} \newcommand{\R}{\mathbb{R}} \newcommand{\Z}{\mathbb{Z}} \)

How to quickly solve new tasks?

My research interests lie in machine learning with a focus on problems where data is scarce and compute is limited.

My main contributions concern meta-learning algorithms — algorithms that discover and improve new learning algorithms. Specifically, my PhD dissertation showed how (and why!) some inductive biases are transferred from one task to the next while others have to be learned from scratch. Through my work I was fortunate to tackle problems ranging from computer vision and language to robotics.

Ultimately, I hope my research can help democratize machine learning so we can all reap the benefits of artifical intelligence.

[Google Scholar, Semantic Scholar]

Go to: [Publications, Presentations, Posters]

Publications

* denotes equal contribution

Policy-Induced Self-Supervision Improves Representation Finetuning in Visual RL
S. M. R. Arnold, F. Sha
ArXiv Preprint, 2023
[ArXiv, pdf, website]

A Domain-Agnostic Approach for Characterization of Lifelong Learning Systems
M. M. Baker, A. New, M. Aguilar-Simon, Z. Al-Halah, S. M. R. Arnold, E. Ben-Iwhiwhu, A. P. Brna, E. Brooks, R. C. Brown, Z. Daniels, A. Daram, F. Delattre, R. Dellana, E. Eaton, H. Fu, K. Grauman, J. Hostetler, S. Iqbal, D. Kent, N. Ketz, S. Kolouri, G. Konidaris, D. Kudithipudi, E. Learned-Miller, S. Lee, M. L. Littman, S. Madireddy, J. A. Mendez, E. Q. Nguyen, C. Piatko, P. K. Pilly, A. Raghavan, A. Rahman, S. K. Ramakrishnan, N. Ratzlaff, A. Soltoggio, P. Stone, I. Sur, Z. Tang, S. Tiwari, K. Vedder, F. Wang, Z. Xu, A. Yanguas-Gil, H. Yedidsion, S. Yu, G. K. Vallabha.
Neural Networks, Volume 160, 2023
[ArXiv, pdf, ScienceDirect]

Policy Learning and Evaluation with Randomized Quasi-Monte Carlo
S. M. R. Arnold, P. L’Ecuyer, L. Chen, Y-f. Chen, F. Sha
AISTATS, 2022
[ArXiv, pdf, website, code]

Uniform Sampling over Episode Difficulty
S. M. R. Arnold*, G. S. Dhillon*, A. Ravichandran, S. Soatto
NeurIPS, 2021 Spotlight
[ArXiv, pdf, website, code]

Embedding Adaptation is Still Needed for Few-Shot Learning
S. M. R. Arnold, F. Sha
ArXiv Preprint, 2021
[ArXiv, pdf, website]

When MAML Can Adapt Fast and How to Assist When It Cannot
S. M. R. Arnold, S. Iqbal, F. Sha
AISTATS, 2021
[ArXiv, pdf, website, code]

learn2learn: A Library for Meta-Learning Research
S. M. R. Arnold, P. Mahajan, D. Datta, I. Bunner, K. S. Zarkias
ArXiv Preprint, 2020
[ArXiv, pdf, website, code]

Analyzing the Variance of Policy Gradient Estimators for the Linear-Quadratic Regulator
J. Preiss*, S. M. R. Arnold*, C-Y. Wei*, M. Kloft
NeurIPS OptRL Workshop, 2019
[ArXiv, pdf]

Reducing the variance in online optimization by transporting past gradients
S. M. R. Arnold, P.-A. Manzagol, R. Babanezhad, I. Mitliagkas, N. Le Roux
NeurIPS, 2019 Spotlight
[ArXiv, pdf, website, code]

Understanding the Variance of Policy Gradient Estimators in Reinforcement Learning
S. M. R. Arnold*, J. Preiss*, C-Y. Wei*, M. Kloft
SoCal Machine Learning Symposium, 2019 Best Poster
See subsequent workshop submission for updated preprint.

Shapechanger: Environments for Transfer Learning
S. M. R. Arnold, E. Pun, T. Denisart, F. Valero-Cuevas
SoCal Robotics Symposium, 2017
[ArXiv, pdf, website]

Accelerating SGD for Distributed Deep Learning Using an Approximated Hessian Matrix
S. M. R. Arnold, C. Wang
ICLR Workshop, 2017
[ArXiv, pdf]

A Performance Comparison between TRPO and CEM for Reinforcement Learning
S. M. R. Arnold, E. Chu, F. Valero-Cuevas
SoCal ML Symposium, 2016

A Greedy Algorithm to Cluster Specialists
S. M. R. Arnold
ArXiv Preprint, 2016
[ArXiv, pdf]


Presentations

Quickly solving new tasks, with meta-learning and without
Thesis defense.
University of Southern California, Los Angeles, CA (Remote; December 2022)
[pdf]

Policy Learning and Evaluation with Randomized Quasi-Monte Carlo
Presentation of our work on RQMC for RL.
AISTATS 2022, Virtual, (Remote; March 2022)
[pdf, talk]

Uniform Sampling Over Episode Difficulty
Spotlight presentation of our work on sampling episodes in few-shot learning.
NeurIPS 2021, Virtual, (Remote; December 2021)
EPFL’s NeurIPS 2021 mirror event, Lausanne, Switzerland (December 2021)
 [pdf, talk (NeurIPS), talk (EPFL)]

To Transfer or To Adapt: A Study Through Few-Shot Learning
Overview of my recent research research on adaptation and transfer in few-shot learning.
Google, Mountain View, CA (Remote; April 2021)
Amazon, Seattle, WA (August 2021).

When MAML Can Adapt Fast and How to Assist When it Cannot
Slidelive presentation of our work on helping MAML learn to adapt.
AISTATS 2021, Virtual (Remote; April 2021)
 [pdf, talk]

Reducing the Variance in Online Optimization by Transporting Past Gradients
Spotlight presentation of our work on implicit gradient transport.
NeurIPS 2019, Vancouver, Canada (December 2019)
 [pdf, talk]

learn2learn: A Meta-Learning Framework
Short presentation of learn2learn and some applications of meta-learning.
PyTorch Dev Conference, San Francisco, CA (October 2019)
 [pdf, talk]

Information Geometric Optimization
Tutorial on recent approaches using information geometric principles for optimization. Inspired by Yann Ollivier’s presentation and James Martens’ paper.
ShaLab reading group, Los Angeles, CA (October 2018)
 [pdf]

Managing Machine Learning Experiments
Presentation of randopt and how to use it to manage machine learning experiments.
SoCal Python Meetup, Los Angeles, CA (May 2018)
 [pdf]


Posters

Policy Learning and Evaluation with Randomized Quasi-Monte Carlo
Poster for our work on reducing the variance with RQMC in reinforcement learning.
AISTATS, 2022
 [pdf]

Uniform Sampling Over Episode Difficulty
Poster for our work on sampling episodes in few-shot learning.
NeurIPS, 2021
 [pdf]

When MAML Can Adapt Fast and How to Assist When it Cannot
Poster for our work on helping MAML learn to adapt.
AISTATS, 2021
 [pdf]

Reducing the variance in online optimization by transporting past gradients
Poster for our work on implicit gradient transport.
NeurIPS, 2019
 [pdf]

cherry: A Reinforcement Learning Framework for Researchers
An overview of cherry.
PyTorch Dev Conference, 2019
 [pdf]

learn2learn: A Meta-Learning Framework for Researchers
An overview of learn2learn.
PyTorch Dev Conference, 2019
 [pdf]

Managing Machine Learning Experiments
How to use randopt to manage machine learning experiments.
PyCon, 2018
 [pdf]

Accelerating SGD for Distributed Deep Learning Using Approximated Hessian Matrix
Approximating the Hessian via finite differences in the distributed setting.
ICLR Workshop, 2017
 [pdf]