Bayesian reasoning, introduced by Reverend Thomas Bayes in the 18th century, is a method of updating beliefs based on new information. It has profoundly influenced fields like science, medicine, artificial intelligence, and even codebreaking during World War II. This article explores Bayesian reasoning's principles, applications in medical testing and spam filtering, and its role in human cognition and scientific inference.
You probably don't realize it, but every time you change your mind based on new information—like rethinking a decision after hearing a friend's advice—you are using something called Bayesian reasoning.
Bayesian reasoning was first articulated by the Reverend Thomas Bayes, a clergyman from Tunbridge, Wales, in southern England, way back in the 18th century. Bayes wasn't well known in his lifetime; in fact, his theory wasn't published until two years after he had died. However, his work has gone on to profoundly shape science, medicine, artificial intelligence, and even helped crack the Nazis' Enigma code during World War II.
Thomas Bayes introduced two vital ideas:
Essentially, he produced a theory for learning from experience. Bayesian analysis can be used in any situation where each new item of information changes our beliefs.
Alan Turing and his team at Bletchley Park used Bayesian ideas to learn about the settings of Enigma machines during World War II, changing their opinion as new patterns were found.
The spam filter on your email account uses Bayesian analysis to change its probability that your email message is spam as each new suspicious feature is detected.
Consider breast cancer screening in the UK. Mammograms are quite accurate, detecting about 90% of cancers and correctly giving a negative test result to 97% of women without cancer. But what about the other 3%?
Let's consider what we would expect to happen to 100 women being tested:
So, in total, we expect 4 positive tests, but only 1 actually has cancer. From this perspective, looking only at the women who get a positive result, the test may not seem so accurate. Around 3 out of 4, or 75%, of the women recalled do not have cancer. Further tests should rapidly identify these false positives.
This is an example of using Bayesian analysis. Since we take into account the base rate for the condition, our prior probability that a random woman being tested has breast cancer is 1%. After getting a positive test result, this is updated to the posterior probability of 25% (1 out of 4). Further investigations will eventually change these probabilities to either 0% or, less fortunately, 100%.
The use of Bayesian analysis in medical testing is now standard.
More controversial is the use of Bayesian methods in science, known as Bayesian inference. The traditional view of science is that it is a completely objective activity: data is analyzed without preconceptions, and the results speak for themselves.
The Bayesian perspective is somewhat different. It acknowledges that there are always judgments underlying every analysis and that we don't start from a blank slate. Each set of data adds to our existing knowledge rather than standing entirely on its own.
For example, clinical trials of new drugs are not designed in complete ignorance. There is always some historical evidence about the potential effectiveness of the new therapy, and expert judgments can be elicited and used to prioritize investigations and produce efficient designs.
It is even thought that we have Bayesian brains. We don't start from scratch to interpret our sensations every second. Instead, we always have prior expectations of what we might experience next, and new observations update our beliefs accordingly.
Fundamentally, Bayesian ideas reflect what it means to be human. We live in a world of uncertainty, but we always have prior expectations for what might happen next, and we revise that uncertainty as we learn from experience.
All of this is based on the ideas of a rather obscure 18th-century cleric from Tunbridge, Wales, whose work continues to influence many aspects of modern life and science.
