January 07, 2006

Our Private Bayesian Rules Engine

The Economist has a great article on how psychologists are looking at how computer scientists are using Bayesian prediction engines for things like help wizards and spam filters. The Psychologists asked an unusual question - maybe people use Bayesian logic?

Of course! Er, well, maybe. Science needs to test the hypothesis, and that's what they set out to do:

Dr Griffiths and Dr Tenenbaum conducted their experiment by giving individual nuggets of information to each of the participants in their study (of which they had, in an ironically frequentist way of doing things, a total of 350), and asking them to draw a general conclusion. For example, many of the participants were told the amount of money that a film had supposedly earned since its release, and asked to estimate what its total “gross” would be, even though they were not told for how long it had been on release so far.

Besides the returns on films, the participants were asked about things as diverse as the number of lines in a poem (given how far into the poem a single line is), the time it takes to bake a cake (given how long it has already been in the oven), and the total length of the term that would be served by an American congressman (given how long he has already been in the House of Representatives). All of these things have well-established probability distributions, and all of them, together with three other items on the list—an individual's lifespan given his current age, the run-time of a film, and the amount of time spent on hold in a telephone queuing system—were predicted accurately by the participants from lone pieces of data.

There were only two exceptions, and both proved the general rule, though in different ways. Some 52% of people predicted that a marriage would last forever when told how long it had already lasted. As the authors report, “this accurately reflects the proportion of marriages that end in divorce”, so the participants had clearly got the right idea. But they had got the detail wrong. Even the best marriages do not last forever. Somebody dies. And “forever” is not a mathematically tractable quantity, so Dr Griffiths and Dr Tenenbaum abandoned their analysis of this set of data.

The other exception was a topic unlikely to be familiar to 21st-century Americans—the length of the reign of an Egyptian Pharaoh in the fourth millennium BC. People consistently overestimated this, but in an interesting way. The analysis showed that the prior they were applying was an Erlang distribution, which was the correct type. They just got the parameters wrong, presumably through ignorance of political and medical conditions in fourth-millennium BC Egypt. On congressmen's term-lengths, which also follow an Erlang distribution, they were spot on.

Which leaves me wondering what an Erlang distribution is... Wikipedia doesn't explain it in human terms, but it looks like a Poisson distribution:

Curious footnote - look at who they credited as the source of their graph of distributions.

Posted by iang at January 7, 2006 10:19 AM | TrackBack

I think it's the other way round: More computer programmers have discovered Bayesian learning and similar probability-based techniques. They have been in use in information retrieval for decades, and Paul Graham popularized probabilty-based ideas only *after* these two researchers began working in that area.

Posted by: Florian Weimer at January 7, 2006 05:07 PM

An Erlang distribution is a Poisson distribution with an integer h parameter. It is that simple.

Posted by: Daniel A. Nagy at January 8, 2006 04:13 AM

From: "An Intuitive Explanation of Bayesian Reasoning

Bayes' Theorem
for the curious and bewildered;
an excruciatingly gentle introduction.

By Eliezer Yudkowsky" link, here ... http://yudkowsky.net/bayes/bayes.html

Scrolling down a bit, one gets to ...

"Here's a story problem about a situation that doctors often encounter:

1% of women at age forty who participate in routine screening have breast cancer. 80% of women with breast cancer will get positive mammographies. 9.6% of women without breast cancer will also get positive mammographies. A woman in this age group had a positive mammography in a routine screening. What is the probability that she actually has breast cancer?

What do you think the answer is? If you haven't encountered this kind of problem before, please take a moment to come up with your own answer before continuing."

The above isn't that intuitive!!!

The author shows how it can be made more intuitive, though, eg by re-phrasing the question and such ... Anyway, the link is well worth the read.

As an aside ... I wanted to reference the following ... http://www.amazon.com/gp/reader/0812975219/ref=sib_vae_pg_190/002-3703878-9607262?%5Fencoding=UTF8&keywords=side%20effect&p=S06C&twc=4&checkSum=pz%2F9ZdXCBpyqZdIfID9ZQOng7RgEisixKBBl9uInnR4%3D#reader-page but Amazon wouldn't let me cut and paste! (A case of how to lose friends and influence no one?).

Posted by: Darren at January 8, 2006 10:01 AM

Andrew Gelman has a critique of the Economist paper in his blog here ... http://www.stat.columbia.edu/~cook/movabletype/archives/2006/01/bayesian_parame.html#comments

Posted by: Darren at April 19, 2006 05:37 AM
Post a comment

Remember personal info?

Hit preview to see your comment as it would be displayed.