The genesis of world models and predictive architectures
How can machines learn meaningful representations from data without requiring expensive human annotations?
This talk explored self-supervised learning—a paradigm shift that enabled AI systems to learn from vast amounts of unlabeled data, much like humans learn through observation by building an internal world model.
The focus was primarily on BYOL (Bootstrap Your Own Latent), a method that trained neural networks to discover useful patterns by comparing different views of the same data point. Unlike traditional approaches that required carefully labeled examples, BYOL learned by predicting how the same entity appeared under different transformations.
Beyond images, the lecture discussed how these ideas extended to diverse domains:
- Analyzing social networks and community structures (densely connected subnetworks) using BGRL.
- Understanding neural recordings and biological signals through MYOW.
- Integrating multiple data modalities (text, images, and audio) via BEAST.
These techniques opened possibilities for any field with abundant unlabeled data—from social media analysis and behavioral patterns to scientific discovery—highlighting opportunities for applying these methods to computational social science, network analysis, and other data-rich domains across disciplines.
Event Course and Discussion
Following the lecture, a passionate open discussion took place. The main topics revolved around finding the right balance between automation, process efficiency, output quality, and essential human expertise, as well as practically connecting advanced models to real-world research problems.
