Mathematics and AI Seminar
This fall I am organizing a series of talks on the connections between mathematics and contemporary AI. There are multiple angles, and we will discuss both the mathematical aspects that make AI work and the impact of AI on mathematical life, in research as well as in teaching.
A social dimension will also run through these presentations. It is clear that these technologies will continue to transform the world as we know it, and it seems important to use the sometimes illuminating reach of mathematics to try to understand the nature and extent of the changes to come.
Wednesday, September 16; 12:30 to 13:20 (B-3410)
What future for automated mathematics?
Guillaume Roy-Fortin
Dozens of Erdős problems, the unit distance conjecture, a complex structure on the 6-dimensional sphere, a counterexample to the Escobar conjecture... and the list goes on! What list? The list of mathematical problems that have been solved thanks to the dramatic progress of modern large AI models. These models are now capable of autonomously producing mathematical results comparable in level to university research. What are the consequences of such a technological feat? What, then, is mathematics, if it can be automated? In this talk, we will discuss these recent advances and the profound upheavals they are causing in the mathematics profession, notably by revisiting some aspects of Terence Tao's presentation at the ICM last July.
Note: this is a non-technical presentation open to everyone.
Wednesday, September 30; 12:30 to 13:20 (B-3410)
AI safety, the Bayesian oracle, and alignment of a conversational agent.
Guillaume Roy-Fortin
How can we guarantee that an AI agent will not cause harm in situations it has never encountered? In this talk, we will present the approach of Bengio et al. (2024) based on Bayesian inference, in which uncertainty about the future behavior of an AI agent is represented by a distribution over different hypotheses about the world.
From observations, a Bayesian oracle identifies the plausible hypotheses under which an action could be dangerous and constructs an upper bound on its probability of causing harm. Actions whose risk exceeds a threshold are then forbidden.
After a brief review of the fundamental principles of Bayesian inference, we will see how they lead to safety guarantees, particularly in non-i.i.d. environments. This approach thus attempts to offer a quantitative framework for a central question in AI alignment: can uncertainty about the consequences of an agent's actions be turned into a formal guarantee against potentially harmful behavior?
Wednesday, October 14; 12:30 to 13:20 (B-3410)
Superposition and polysemanticity: understanding the internal representations of AI to better secure them
Guillaume Roy-Fortin
The transmission of information through the neural networks that power modern AI happens layer by layer. Given the success of these models, it is natural to ask what is stored in the neurons of the intermediate layers during and after training. Is it possible to determine what type of information a given neuron will end up storing? This is the fundamental problem of interpretability, which aims to make sense of the activations of trained neurons. Can we find conditions under which we can say that a given neuron plays a given role?
Building on the work of Elhage et al. (2022) , we will see that the activation vector of a neural layer can sometimes store more information than its dimension seems to allow. A bit like managing to fit 5 types of utensils into a drawer designed to hold only 4, for example. This phenomenon is called superposition and allows several meanings to be assigned to the role played by a particular neuron, a principle called polysemanticity.
Back to the main page.