"A Bayesian is one who, vaguely expecting a horse, and catching a glimpse of a donkey, strongly believes he has seen a mule."In that case, everyone is a Bayesian. Researchers in perceptual science have shown over and over that background knowledge and expectation can strongly influence perception, especially when the visual stimulus is unclear. Anecdotal cases of this abound; e.g., the 1950s TV show in which people say (roughly) "Look, up in the sky, it's a bird, no, it's a plane, no, it's Superman!" Their prior knowledge indicates that flying things should be birds or airplanes, and it's only with accumulating visual data that the prior is overwhelmed.
And there is an error in the joke. If an actual Bayesian vaguely expects a horse and catches only a glimpse of a donkey, then s/he believes s/he has seen a mule only with great uncertainty, not strongly.
Hi John,
ReplyDeleteI think the people who like this joke have only seen one Bayesian example: an informative normal prior with an informative likelihood that peaks far away.
In that case, you really can expect to see one thing, see something else and then strongly believe in something in between.
Really it's just an artifact of how fast the tails of a normal distribution fall off. When people look at a bell curve they don't realise just how improbable the far tails are, even compared to the not-so far tails.
Dear Brendon:
ReplyDeleteThanks for your comment.
The joke says there is a vague prior and a glimpse of data, leading to a strong posterior belief, which is not correct Bayesian inference.
But, as you point out, if there is a strong prior and lots of data, then there can be a strong posterior that falls between the prior and the data. This works not only with the normal distributions that you refer to, but also with the beta-Bernoulli case, too. For example, try this code:
source("BernBeta.R")
BernBeta( c(80,20) , c(rep(0,80),rep(1,20)) )
The sharp posterior falls between the sharp prior and sharp likelihood.