World Models and How to Find Them

Can machines learn without explicit labels? In this TmP talk, we’ll explore self-supervised representation learning and introduce BYOL (Bootstrap Your Own Latent)—a game-changing approach that achieves state-of-the-art performance without negative pairs.
Unlike traditional methods, BYOL trains two neural networks—an online and a target network—that learn from each other. By predicting different augmented views of the same image, BYOL refines its representations over time, unlocking powerful insights without needing labeled data.
We’ll also dive into how DeepMind has extended BYOL into new domains, including BGRL for graphs, MYOW for neural signals, and BAS/BEAST for multi-modal learning. Finally, we’ll discuss key insights, open challenges, and future directions in self-supervised learning.
Join us to discover how BYOL is pushing the boundaries of AI!

Michal Valko Bio

Michal is the Chief Models Officer at a stealth startup, tenured researcher at Inria, and the lecturer at the MVA master of ENS Paris-Saclay. Michal is primarily interested in designing algorithms that would require as little human supervision as possible. That is why he is working on methods and settings that are able to deal with minimal feedback, such as deep reinforcement learning, bandit algorithms, self-supervised learning, or self play. Michal has recently worked on representation learning, word models and deep (reinforcement) learning algorithms that have some theoretical underpinning. In the past he has also worked on sequential algorithms with structured decisions where exploiting the structure leads to provably faster learning. Michal is now working on large large models (LMMs), in particular providing algorithmic solutions for their scalable fine-tuning and alignment. He received his Ph.D. in 2011 from the University of Pittsburgh, before getting a tenure at Inria in 2012 and starting Google DeepMind Paris in 2018 with Rémi Munos. In 2024, he became the principal Llama engineer at Meta, building online reinforcement learning stack and research for Llama 3 and 4.

KEYNOTE