What are these Bayesians anyway

Below you can find some information about the Bayesian way of using probability theory. It also explains, why Bayesians are sometimes called subjectivists (i.e., those promoting subjective probability). This term is mainly used by so called objectivists also called frequentists. Frequentism is what standard (orthodox) statistics is mainly based on.

Why it takes a Bayesian to talk about the probability of dependencies?

An objectivist refuses to talk about the probability of the dependencies of the data, since he thinks that the every dependency is "truth out there", i.e., the world is the way it is, and there is nothing random in it. Either A depends on B or it does not. Either way there is nothing random in the dependency. For this reason objectivist thinks, that talking about the probability of this non-random dependency is incorrect.

Bayesian looks the situation differently. She/he might agree that true dependency is or is not out there, but she concentrates in acknowledging her uncertainty about the dependencies. Even if the things depend on each other the way they do, we do not know how or whether they depend. Bayesian uses probability to asses her uncertainty about the true existence and nature of dependencies. When she gets more information about the dependencies, her probabilities about the aspects of the world change. Bayesian is very willing to say that the probability that A depends on B is 0.9. It just means that she is pretty sure that A depends on B.

Bayesians also emphasize that probabilities always depend on our background assumptions. With different background assumptions we get different probabilities and thus different results. Different background assumptions are like different languages with which you can describe the world. Looking the data by using dependency model can reveal interesting things about your world, and you can even attach probabilities to different background assumptions, but all in all, Bayesians are rather reluctant to claim that their results are the one and only objective truth. They tend to have a more humble attitude to the truth: all models are wrong, but some are useful.

» More about Bayesian probability

B-Course, version 2.0.0	CoSCo 2002