Associate Professor, Department of Engineering Science, University of Oxford
Research Scientist, FAIR, Meta AI, Meta
Supernumeray Fellow, St Anne's College, Oxford
Lab Website (FLAIR)
CV (stale)
email: reachingjakob at gmail dot com [no generic ("I'd like to...") emails, please! See below for detailed instructions for common use cases. This will likely make life easier for you and me and also maximise the chance you'll get a timely reply :) ]
Note to potential Phd applicants (external):
If you are interested in pursuing a PhD ("DPhil") at my lab, please apply to both the Engineering Department (deadline is early December!) and the AIMS CDT (deadline is mid January). Make sure to list me as a supervisor for the direct application.
If you are wondering whether FLAIR is the right place for you, please take a look at my publications on Google Scholar, check out the lab website, and watch some of my talks on the internet. Only email me if there are specific follow-up questions. As a scientist, I like to hear about non-obvious insights and interesting follow-up suggestions to previous work. Are you wondering if your email meets the bar to be useful? See here my cold email that started my career in ML.
If you are confident that you are excited about FLAIR, then please keep in mind that a recurrent pain point at Oxford is funding, in particular for overseas students. So alongside your application, please also apply for any possible 3rd party scholarships. E.g. CAIF, Future of Life, etc. If you are currently working you might also want to enquire within your company to see if they might sponsor your Phd.
For any emails that meet the bar, please put the code "d48b8eb9a99bc6DPHIL" into the subject line to confirm that you have read these instructions.
Please also follow me on X and bluesky, I make announcements about funded positions there when they are available.
Note regarding reviewing queries:
I spend a fair amount of time as Senior AC for various ML conferences, so unfortunately I am not currently able to review more papers on top of this. Please look at my co-authors for potential candidates -- they are all fantastic!
Note regarding requests for external Phd examining:
If the Phd topic is narrowly within my area of expertise I will usually try to help out but am also rather thin stretched. When you contact me, please mentioned the timeframe for the viva, whether online examination is possible, and the work required by the external examiner. Please put" d48b8eb9a99bc6EXAM" into the subject line.
Note to potential visitors (external):
FLAIR collaborates extremely broadly which is part of the fun!
If you are interested in visiting, please send me an email with the following:
1) CV
2) Current program (undergrad / MSc / Phd / working)
3) right to work in the UK (yes / no )
3) Ideal start and end date for a visit to FLAIR
4) Research interest / Project proposals -- see our website for current example work.
Please put "d48b8eb9a99bc6Visitor" in to the subject line.
Already at Oxford and looking to do a Master thesis (or otherwise collaborate) with FLAIR:
Please email me from your Oxford email address to my gmail mentioned above with "d48b8eb9a99bc6OXFORD" in the subject line and provide the following information:
1) CV
2) MSc (or other) Program at Oxford
3) Start and end date, incl. availability for ramp-up
4) Research interest / Project proposals -- see our website for current work.
Note for recruiters:
I am very happy where I am and am currently trying to give both FLAIR and FAIR my best shot, so this is not the right time for the next gig. Please take a look at the FLAIR website for plenty of strong talent though!
Note regarding generic queries:
Please put "d48b8eb9a99bc6GENERIC" into the subject line. If you include a screenshot of a same-day donation to Wikipedia of 10.42 GBP / 15.42 USD and "d48b8eb9a99bc6MUSTREPLY" in the subject line I will reply to your email.
Research Interests
0) Compute only scaling and computational self-improvement (i.e. what do we do when we run out of human data, but computers keep getting faster)
1) Making (MA)RL work at scale and in the real world (incl. applications in finance, opponentshaping for Bio applications, Meta-RL.. )
2) Principled Approaches to Human-AI Coordination (e.g. zero-shot coordination, evaluation etc)
3) AI and Science
4) AI and Society
5) Anything but supervised learning that is well motivated and has a shot at making a difference.
News!
New news: I am now 50/50 between FAIR (MetaAI) and Oxford (FLAIR). I am also a supernumerary fellow at St Anne's college now.
old: I started as an Associate Professor at the Engineering Science Department at the University of Oxford and St. Anne's College.
Publications, Preprints and links to code etc:
[NOTE: This is gets updated sporadically and hence is stale (by design). Please check Google Scholar and our lab website for recent papers / publications]
2022
Tons of new papers here: https://foersterlab.com/research/
"Centralized Model and Exploration Policy for Multi-Agent RL" [paper]
Q Zhang, C Lu, A Garg, JN Foerster
International Conference on Autonomous Agents and Multiagent Systems, 2022
"Lyapunov Exponents for Diversity in Differentiable Games" [paper]
J Lorraine, P Vicol, J Parker-Holder, T Kachman, L Metz, JN Foerster
International Conference on Autonomous Agents and Multiagent Systems, 2022
2021
"K-level Reasoning for Zero-Shot Coordination in Hanabi" [paper]
B Cui, H Hu, L Pineda, JN Foerster
Neural Information Processing Systems, 2021
"Replay-Guided Adversarial Environment Design" [paper, Tweet Explainer]
M Jiang, M Dennis, J Parker-Holder, J Foerster, E Grefenstette, T Rocktäschel
Neural Information Processing Systems, 2021
"Neural Pseudo-Label Optimism for the Bank Loan Problem" [paper, code, Tweet Explainer]
A Pacchiano, S Singh, E Chou, A Berg, JN Foerster
Neural Information Processing Systems, 2021
"Off-Belief Learning" [paper, code]
H Hu, A Lerer, B Cui, L Pineda, D Wu, N Brown, JN Foerster
International Conference on Machine Learning, 2021
"Trajectory diversity for zero-shot coordination" [paper, code]
A Lupu, B Cui, H Hu, JN Foerster
International Conference on Machine Learning, 2021
"A New Formalism, Method and Open Issues for Zero-Shot Coordination" [paper, code]
J Treutlein, M Dennis, C Oesterheld, JN Foerster
International Conference on Machine Learning, 2021
2020
"Ridge Rider: Finding Diverse Solutions by Following Eigenvectors of the Hessian" [paper, code 1, code 2, code 3]
J Parker-Holder*, L Metz, C Resnick, H Hu, A Lerer, A Letcher, A Peysakhovich, A Pacchiano, JN Foerster*
Neural Information Processing Systems, 2020
"“Other-Play” for Zero-Shot Coordination" [paper, code]
H Hu*, A Lerer, A Peysakhovich, JN Foerster*
International Conference on Machine Learning, 2020
2019
"On the interaction between supervision and self-play in emergent communication " [paper]
R Lowe*, A Gupta*, JN Foerster, D Kiela, J Pineau
International Conference on Learning Representations, 2020
"Simplified Action Decoder for Deep Multi-Agent Reinforcement Learning " [paper, code]
H Hu, JN Foerster
International Conference on Learning Representations, 2020
"Improving Policies via Search in Cooperative Partially Observable Games " [paper, code, blog post]
A Lerer, H Hu, JN Foerster, N Brown
AAAI Conference on Artificial Intelligence, 2020
"Can I Trust the Explainer? Verifying Post-hoc Explanatory Methods" [paper]
OM Camburu*, E Giunchiglia*, JN Foerster, T Lukasiewicz, P Blunsom
preprint
"Capacity, Bandwidth, and Compositionality in Emergent Language Learning" [paper]
C Resnick*, A Gupta*, JN Foerster, AM Dai, K Cho
International Conference on Autonomous Agents and Multiagent Systems, 2020
"Differentiable Game Mechanics" [paper]
A Letcher, D Balduzzi, S Racaniere, J Martens, JN Foerster, K Tuyls, T Graepel
Journal of Machine Learning Research
"Robust Domain Randomization for Reinforcement Learning" [paper, code]
RB Slaoui, WR Clements, JN Foerster, S Toth
preprint
"Exploratory Combinatorial Optimization with Reinforcement Learning" [paper, code ]
TD Barrett, WR Clements, JN Foerster, AI Lvovsky
AAAI Conference on Artificial Intelligence, 2020
"Loaded DiCE: Trading off Bias and Variance in Any-Order Score Function Estimators for Reinforcement Learning" [paper, code]
G Farquhar, S Whiteson, JN Foerster
Advances in Neural Information Processing Systems, 2019
"A Survey of Reinforcement Learning Informed by Natural Language" [paper]
J Luketina, N Nardelli, G Farquhar, JN Foerster, J Andreas, E Grefenstette, S Whiteson, T Rocktäschel
IJCAI Survey Track, 2019
"The StarCraft Multi-Agent Challenge" [paper, code, blog]
M Samvelyan*, T Rashid*, C Schroeder de Witt, G Farquhar, N Nardelli, T. Rudner, C Hung, P Torr, JN Foerster, S Whiteson
International Conference on Autonomous Agents and Multiagent Systems, 2019
"The Hanabi Challenge: A New Frontier for AI Research" [paper, code, blog]
N Bard*, JN Foerster*, S Chandar, N Burch, M Lanctot, HF Song, E Parisotto, V Dumoulin, S Moitra, E Hughes, I Dunning, S Mourad, H Larochelle, MG Bellemare, M Bowling
Artificial Intelligence (AIJ)
"Bayesian Action Decoder for Deep Multi-Agent Reinforcement Learning" [paper, matrix game code ]
JN Foerster*, FH Song*, E Hughes, N Burch, I Dunning, S Whiteson, M Botvinick, M Bowling
International Conference on Machine Learning, 2019
"A Baseline for Any Order Gradient Estimation in Stochastic Computation Graphs" [paper]
J Mao*, JN Foerster*, T Rocktäschel, G Farquhar, M Al-Shedivat, S Whiteson
International Conference on Machine Learning, 2019
"On the Pitfalls of Measuring Emergent Communication" [paper]
R Lowe, JN Foerster , YL Boureau, J Pineau, Y Dauphin
International Conference on Autonomous Agents and Multiagent Systems, 2019
2018
"Multi-Agent Common Knowledge Reinforcement Learning" [paper]
CAS de Witt*, JN Foerster* , G Farquhar, PHS Torr, W Boehmer, S Whiteson
Advances in Neural Information Processing Systems, 2019
"Pommerman: A multi-agent playground" [paper]
C Resnick, W Eldridge, D Ha, D Britz, JN Foerster, J Togelius, K Cho, J Bruna
NeurIPS 2018 Competition track
"Stable Opponent Shaping in Differentiable Games" [paper]
A Letcher, JN Foerster, D Balduzzi, T Rocktäschel, S Whiteson
International Conference on Learning Representations, 2019
"DiCE: The Infinitely Differentiable Monte-Carlo Estimator" [paper, code, pyro support]
JN Foerster, G Farquhar*, M Al-Shedivat*, T Rocktäschel, EP Xing, S Whiteson
International Conference on Machine Learning, 2018
"QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning" [link]
T Rashid*, M Samvelyan*, CS de Witt, G Farquhar, JN Foerster, S Whiteson
International Conference on Machine Learning, 2018
"The Mechanics of n-Player Differentiable Games" [link, code ]
D Balduzzi, S Racaniere, J Martens, JN Foerster, K Tuyls, T Graepel
International Conference on Machine Learning, 2018, Best Paper Runner-Up Award
"Learning with Opponent-Learning Awareness" [paper, video, slides, blog post, code, pytorch implementation]
JN Foerster*, RY Chen*, M Al-Shedivat, S Whiteson, P Abbeel, I Mordatch
International Conference on Autonomous Agents and Multiagent Systems, 2018
"Counterfactual Multi-Agent Policy Gradients" [link]
JN Foerster*, G Farquhar*, T Afouras, N Nardelli, S Whiteson
AAAI Conference on Artificial Intelligence 2018, Outstanding Student Paper Award
2017
"Stabilising experience replay for deep multi-agent reinforcement learning" [paper, video, media coverage]
JN Foerster*, N Nardelli*, G Farquhar, P Torr, P Kohli, S Whiteson
International Conference on Machine Learning, 2017
"Input switched affine networks: An RNN architecture designed for interpretability" [paper, video, code]
JN Foerster*, J Gilmer*, J Sohl-Dickstein, J Chorowski, D Sussillo
International Conference on Machine Learning, 2017
"Nonlinear Computation in Deep Linear Networks" [link]
JN Foerster
OpenAI Blog
"Fake News in Social Networks" [link, news coverage, code]
C Aymanns, JN Foerster, CP Georg
arXiv preprint arXiv:1708.06233
2016
"Learning to communicate with deep multi-agent reinforcement learning" [paper, video, slides, code, pytorch implementation, pytorch implementation in Colab- LTC in your browser! ]
JN Foerster*, IA Assael*, N de Freitas, S Whiteson
Advances in Neural Information Processing Systems, 2016, 2137-2145
"Learning to communicate to solve riddles with deep distributed recurrent q-networks" [link, news coverage, podcast]
JN Foerster*, YM Assael*, N de Freitas, S Whiteson
IJCAI 2016 Deep Learning Workshop
2015
"Three-dimensional head-direction coding in the bat brain" [link]
A Finkelstein, D Derdikman, A Rubin, JN Foerster, L Las, N Ulanovsky
Nature 517 (7533), 159
2011
"Control of vocal and respiratory patterns in birdsong: dissection of forebrain and brainstem mechanisms using temperature" [link]
AS Andalman*, JN Foerster*, MS Fee
PLoS One 6 (9), e25461