Bayes’ Theorem


Thomas Bayes

Born: 1702 in London, England
Died: 17 April 1761 in Tunbridge Wells, Kent, England

Thomas Bayes‘ father, Joshua Bayes, was one of the first six Nonconformist ministers to be ordained in England. He was ordained in 1694 and moved to Box Lane Chapel, Bovington, about 25 miles from London. Thomas’s mother was Anne Carpenter. The family moved to Southwark, London, when Thomas was young and there Joshua became an assistant at St Thomas’s and also an assistant at the Chapel in Leather Lane, Holborn. Thomas was the eldest of his parents seven children, four boys and three girls. (read more about Thomas…)

Dirichlet Processes: Tutorial and Practical Course

Yee Whye Teh

via YudkowskyBayes’ Theorem

An Intuitive Explanation of Bayes’ Theorem

Bayes’ Theorem
for the curious and bewildered;
an excruciatingly gentle introduction.

Your friends and colleagues are talking about something called “Bayes’ Theorem” or “Bayes’ Rule“, or something called Bayesian reasoning.  They sound really enthusiastic about it, too, so you google and find a webpage about Bayes’ Theorem and…

It’s this equation.  That’s all.  Just one equation.  The page you found gives a definition of it, but it doesn’t say what it is, or why it’s useful, or why your friends would be interested in it.  It looks like this random statistics thing.

So you came here.  Maybe you don’t understand what the equation says.  Maybe you understand it in theory, but every time you try to apply it in practice you get mixed up trying to remember the difference between p(a|x) and p(x|a), and whether p(a)*p(x|a) belongs in the numerator or the denominator.  Maybe you see the theorem, and you understand the theorem, and you can use the theorem, but you can’t understand why your friends and/or research colleagues seem to think it’s the secret of the universe.  Maybe your friends are all wearing Bayes’ Theorem T-shirts, and you’re feeling left out.  Maybe you’re a girl looking for a boyfriend, but the boy you’re interested in refuses to date anyone who “isn’t Bayesian”.  What matters is that Bayes is cool, and if you don’t know Bayes, you aren’t cool.

Why does a mathematical concept generate this strange enthusiasm in its students?  What is the so-called Bayesian Revolution now sweeping through the sciences, which claims to subsume even the experimental method itself as a special case?  What is the secret that the adherents of Bayes know?  What is the light that they have seen?

Soon you will know.  Soon you will be one of us.

While there are a few existing online explanations of Bayes’ Theorem, my experience with trying to introduce people to Bayesian reasoning is that the existing online explanations are too abstract.  Bayesian reasoning is verycounterintuitive. People do not employ Bayesian reasoning intuitively, find it very difficult to learn Bayesian reasoning when tutored, and rapidly forget Bayesian methods once the tutoring is over.  This holds equally true for novice students and highly trained professionals in a field.  Bayesian reasoning is apparently one of those things which, like quantum mechanics or the Wason Selection Test, is inherently difficult for humans to grasp with our built-in mental faculties.

Or so they claim.  Here you will find an attempt to offer an intuitive explanation of Bayesian reasoning – an excruciatingly gentle introduction that invokes all the human ways of grasping numbers, from natural frequencies to spatial visualization.  The intent is to convey, not abstract rules for manipulating numbers, but what the numbers mean, and why the rules are what they are (and cannot possibly be anything else).  When you are finished reading this page, you will see Bayesian problems in your dreams.

And let’s begin…



2 thoughts on “Bayes’ Theorem

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s