Topic · Machine Learning & AI
← All topicsProbabilistic ML
31 articles · 5 checkpoints · 15 deeper reads · 11 reference notes
A standalone topic: it is on no roadmap, so read it on its own terms.
Every article, in reading order
plant a flag as you finish eachRead these first
A model can be unsure for two completely different reasons, the world is genuinely random, or the model just hasn't seen enough, and mixing them up leads to trusting predictions you shouldn't and distrusting predictions you should.
Train the same network architecture several times from different random starting points, and the spread across their predictions is a cheap, surprisingly reliable measure of how much the model doesn't know.
A discriminative model learns only the boundary between classes; a generative model learns how each class's data was produced in the first place, the difference between memorizing where the line is and understanding what's on each side of it.
A joint probability over many variables is too big to write down directly the moment you have more than a handful of them, a probabilistic graphical model draws the dependencies as a graph instead, so the joint distribution factors into small, manageable pieces.
The ELBO is a quantity you can actually compute that sits below the number you actually want, and pushing it up is how variational methods approximate a posterior distribution nobody can integrate exactly.
Then the rest