John Carlos Baez on Nostr: Yay! John Hopfield and Geoffrey Hinton won the physics Nobel for their work on neural ...
Yay! John Hopfield and Geoffrey Hinton won the physics Nobel for their work on neural networks... work that ultimately led to modern-day machine learning.
Some of you are wondering why this counts as physics.
In the 1980s, Hopfield invented the 'Hopfield network' based on how atomic spins interact in a chunk of solid matter. Each atom's spin makes it into a tiny magnet. Depending on the material, these spins may tend to line up, or point opposite to their neighbors, or interact in even more complicated ways. No matter what they do, at very low temperatures they tend to minimize energy.
A Hopfield network is a simulation of such a system that's been cleverly set up so that the spins store data, like images. When the Hopfield network is fed a distorted or incomplete image, it keeps updating the spins so that the energy decreases... and works its way toward the saved image that's most like the imperfect one it was fed with.
Later, Geoffrey Hinton and others generalized Hopfield's ideas and developed 'Boltzmann machines'. These exploit a discovery by the famous physicist Boltzmann!
Boltzmann realized that the probability that a chunk of matter in equilibrium has energy E is proportional to
exp(-E/kT)
where E is its energy, T is its temperature and k is a number now called Boltzmann's constant. When the temperature is low, this makes it overwhelmingly probable that the stuff will have low energy. But at higher temperatures this is less true. By exploiting this formula and cleverly adjusting the temperature, we can make neural networks do very useful things.
There's been a vast amount of work since then, refining these ideas. But it started with physics.
https://arstechnica.com/ai/2024/10/in-stunning-nobel-win-ai-researchers-hopfield-and-hinton-take-2024-physics-prize/
Some of you are wondering why this counts as physics.
In the 1980s, Hopfield invented the 'Hopfield network' based on how atomic spins interact in a chunk of solid matter. Each atom's spin makes it into a tiny magnet. Depending on the material, these spins may tend to line up, or point opposite to their neighbors, or interact in even more complicated ways. No matter what they do, at very low temperatures they tend to minimize energy.
A Hopfield network is a simulation of such a system that's been cleverly set up so that the spins store data, like images. When the Hopfield network is fed a distorted or incomplete image, it keeps updating the spins so that the energy decreases... and works its way toward the saved image that's most like the imperfect one it was fed with.
Later, Geoffrey Hinton and others generalized Hopfield's ideas and developed 'Boltzmann machines'. These exploit a discovery by the famous physicist Boltzmann!
Boltzmann realized that the probability that a chunk of matter in equilibrium has energy E is proportional to
exp(-E/kT)
where E is its energy, T is its temperature and k is a number now called Boltzmann's constant. When the temperature is low, this makes it overwhelmingly probable that the stuff will have low energy. But at higher temperatures this is less true. By exploiting this formula and cleverly adjusting the temperature, we can make neural networks do very useful things.
There's been a vast amount of work since then, refining these ideas. But it started with physics.
https://arstechnica.com/ai/2024/10/in-stunning-nobel-win-ai-researchers-hopfield-and-hinton-take-2024-physics-prize/