Associate Professor, Department of Engineering Science, University of Oxford
Tutorial Fellow, St. Anne's College
email: reaching jakob at gmail dot com [no generic ("I'd like to...") emails, please!]
Note to potential applicants:
If you are interested in pursuing a PhD ("DPhil") at my lab, please apply to both the Engineering Department (deadline is 3rd of December 2021 at noon UK time) and the AIMS CDT (deadline 21st of January 2022 at noon UK time). Make sure to list me as a supervisor for the direct application.
You are also encouraged to apply here, deadline is the 30th of November 2021.
If you are wondering whether my lab is the right place for you, please take a look at my publications on Google Scholar and watch some of my talks on the internet and only email me if there are specific follow-up questions. As a scientist, I do like to hear about non-obvious insights and interesting follow-up suggestions to previous work.
For any emails please put the code "d48b8eb9a99bc6" into the subject line to confirm that you have read these instructions.
Thanks a lot!
Jakob
Research Interests
(Deep) (Multi-Agent) (Reinforcement) Learning, Human-AI Coordination , Emergent Communication, Search, Planning, Game Theory
News!
I started as an Associate Professor at the Engineering Science Department at the University of Oxford and St. Anne's College.
Publications, Preprints and links to code etc:
[NOTE: This is gets updated sporadically and hence is stale (by design). Please check Google Scholar for recent papers / publications]
2022
"Centralized Model and Exploration Policy for Multi-Agent RL" [paper]
Q Zhang, C Lu, A Garg, JN Foerster
International Conference on Autonomous Agents and Multiagent Systems, 2022
"Lyapunov Exponents for Diversity in Differentiable Games" [paper]
J Lorraine, P Vicol, J Parker-Holder, T Kachman, L Metz, JN Foerster
International Conference on Autonomous Agents and Multiagent Systems, 2022
2021
"K-level Reasoning for Zero-Shot Coordination in Hanabi" [paper]
B Cui, H Hu, L Pineda, JN Foerster
Neural Information Processing Systems, 2021
"Replay-Guided Adversarial Environment Design" [paper, Tweet Explainer]
M Jiang, M Dennis, J Parker-Holder, J Foerster, E Grefenstette, T Rocktäschel
Neural Information Processing Systems, 2021
"Neural Pseudo-Label Optimism for the Bank Loan Problem" [paper, code, Tweet Explainer]
A Pacchiano, S Singh, E Chou, A Berg, JN Foerster
Neural Information Processing Systems, 2021
"Off-Belief Learning" [paper, code]
H Hu, A Lerer, B Cui, L Pineda, D Wu, N Brown, JN Foerster
International Conference on Machine Learning, 2021
"Trajectory diversity for zero-shot coordination" [paper, code]
A Lupu, B Cui, H Hu, JN Foerster
International Conference on Machine Learning, 2021
"A New Formalism, Method and Open Issues for Zero-Shot Coordination" [paper, code]
J Treutlein, M Dennis, C Oesterheld, JN Foerster
International Conference on Machine Learning, 2021
2020
"Ridge Rider: Finding Diverse Solutions by Following Eigenvectors of the Hessian" [paper, code 1, code 2, code 3]
J Parker-Holder*, L Metz, C Resnick, H Hu, A Lerer, A Letcher, A Peysakhovich, A Pacchiano, JN Foerster*
Neural Information Processing Systems, 2020
"“Other-Play” for Zero-Shot Coordination" [paper, code]
H Hu*, A Lerer, A Peysakhovich, JN Foerster*
International Conference on Machine Learning, 2020
2019
"On the interaction between supervision and self-play in emergent communication " [paper]
R Lowe*, A Gupta*, JN Foerster, D Kiela, J Pineau
International Conference on Learning Representations, 2020
"Simplified Action Decoder for Deep Multi-Agent Reinforcement Learning " [paper, code]
H Hu, JN Foerster
International Conference on Learning Representations, 2020
"Improving Policies via Search in Cooperative Partially Observable Games " [paper, code, blog post]
A Lerer, H Hu, JN Foerster, N Brown
AAAI Conference on Artificial Intelligence, 2020
"Can I Trust the Explainer? Verifying Post-hoc Explanatory Methods" [paper]
OM Camburu*, E Giunchiglia*, JN Foerster, T Lukasiewicz, P Blunsom
preprint
"Capacity, Bandwidth, and Compositionality in Emergent Language Learning" [paper]
C Resnick*, A Gupta*, JN Foerster, AM Dai, K Cho
International Conference on Autonomous Agents and Multiagent Systems, 2020
"Differentiable Game Mechanics" [paper]
A Letcher, D Balduzzi, S Racaniere, J Martens, JN Foerster, K Tuyls, T Graepel
Journal of Machine Learning Research
"Robust Domain Randomization for Reinforcement Learning" [paper, code]
RB Slaoui, WR Clements, JN Foerster, S Toth
preprint
"Exploratory Combinatorial Optimization with Reinforcement Learning" [paper, code ]
TD Barrett, WR Clements, JN Foerster, AI Lvovsky
AAAI Conference on Artificial Intelligence, 2020
"Loaded DiCE: Trading off Bias and Variance in Any-Order Score Function Estimators for Reinforcement Learning" [paper, code]
G Farquhar, S Whiteson, JN Foerster
Advances in Neural Information Processing Systems, 2019
"A Survey of Reinforcement Learning Informed by Natural Language" [paper]
J Luketina, N Nardelli, G Farquhar, JN Foerster, J Andreas, E Grefenstette, S Whiteson, T Rocktäschel
IJCAI Survey Track, 2019
"The StarCraft Multi-Agent Challenge" [paper, code, blog]
M Samvelyan*, T Rashid*, C Schroeder de Witt, G Farquhar, N Nardelli, T. Rudner, C Hung, P Torr, JN Foerster, S Whiteson
International Conference on Autonomous Agents and Multiagent Systems, 2019
"The Hanabi Challenge: A New Frontier for AI Research" [paper, code, blog]
N Bard*, JN Foerster*, S Chandar, N Burch, M Lanctot, HF Song, E Parisotto, V Dumoulin, S Moitra, E Hughes, I Dunning, S Mourad, H Larochelle, MG Bellemare, M Bowling
Artificial Intelligence (AIJ)
"Bayesian Action Decoder for Deep Multi-Agent Reinforcement Learning" [paper, matrix game code ]
JN Foerster*, FH Song*, E Hughes, N Burch, I Dunning, S Whiteson, M Botvinick, M Bowling
International Conference on Machine Learning, 2019
"A Baseline for Any Order Gradient Estimation in Stochastic Computation Graphs" [paper]
J Mao*, JN Foerster*, T Rocktäschel, G Farquhar, M Al-Shedivat, S Whiteson
International Conference on Machine Learning, 2019
"On the Pitfalls of Measuring Emergent Communication" [paper]
R Lowe, JN Foerster , YL Boureau, J Pineau, Y Dauphin
International Conference on Autonomous Agents and Multiagent Systems, 2019
2018
"Multi-Agent Common Knowledge Reinforcement Learning" [paper]
CAS de Witt*, JN Foerster* , G Farquhar, PHS Torr, W Boehmer, S Whiteson
Advances in Neural Information Processing Systems, 2019
"Pommerman: A multi-agent playground" [paper]
C Resnick, W Eldridge, D Ha, D Britz, JN Foerster, J Togelius, K Cho, J Bruna
NeurIPS 2018 Competition track
"Stable Opponent Shaping in Differentiable Games" [paper]
A Letcher, JN Foerster, D Balduzzi, T Rocktäschel, S Whiteson
International Conference on Learning Representations, 2019
"DiCE: The Infinitely Differentiable Monte-Carlo Estimator" [paper, code, pyro support]
JN Foerster, G Farquhar*, M Al-Shedivat*, T Rocktäschel, EP Xing, S Whiteson
International Conference on Machine Learning, 2018
"QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning" [link]
T Rashid*, M Samvelyan*, CS de Witt, G Farquhar, JN Foerster, S Whiteson
International Conference on Machine Learning, 2018
"The Mechanics of n-Player Differentiable Games" [link, code ]
D Balduzzi, S Racaniere, J Martens, JN Foerster, K Tuyls, T Graepel
International Conference on Machine Learning, 2018, Best Paper Runner-Up Award
"Learning with Opponent-Learning Awareness" [paper, video, slides, blog post, code, pytorch implementation]
JN Foerster*, RY Chen*, M Al-Shedivat, S Whiteson, P Abbeel, I Mordatch
International Conference on Autonomous Agents and Multiagent Systems, 2018
"Counterfactual Multi-Agent Policy Gradients" [link]
JN Foerster*, G Farquhar*, T Afouras, N Nardelli, S Whiteson
AAAI Conference on Artificial Intelligence 2018, Outstanding Student Paper Award
2017
"Stabilising experience replay for deep multi-agent reinforcement learning" [paper, video, media coverage]
JN Foerster*, N Nardelli*, G Farquhar, P Torr, P Kohli, S Whiteson
International Conference on Machine Learning, 2017
"Input switched affine networks: An RNN architecture designed for interpretability" [paper, video, code]
JN Foerster*, J Gilmer*, J Sohl-Dickstein, J Chorowski, D Sussillo
International Conference on Machine Learning, 2017
"Nonlinear Computation in Deep Linear Networks" [link]
JN Foerster
OpenAI Blog
"Fake News in Social Networks" [link, news coverage, code]
C Aymanns, JN Foerster, CP Georg
arXiv preprint arXiv:1708.06233
2016
"Learning to communicate with deep multi-agent reinforcement learning" [paper, video, slides, code, pytorch implementation, pytorch implementation in Colab- LTC in your browser! ]
JN Foerster*, IA Assael*, N de Freitas, S Whiteson
Advances in Neural Information Processing Systems, 2016, 2137-2145
"Learning to communicate to solve riddles with deep distributed recurrent q-networks" [link, news coverage, podcast]
JN Foerster*, YM Assael*, N de Freitas, S Whiteson
IJCAI 2016 Deep Learning Workshop
2015
"Three-dimensional head-direction coding in the bat brain" [link]
A Finkelstein, D Derdikman, A Rubin, JN Foerster, L Las, N Ulanovsky
Nature 517 (7533), 159
2011
"Control of vocal and respiratory patterns in birdsong: dissection of forebrain and brainstem mechanisms using temperature" [link]
AS Andalman*, JN Foerster*, MS Fee
PLoS One 6 (9), e25461