Statistical Modeling, Causal Inference, and Social Science

Well, if this doesn't bring them a few thousand hits, nothing will!

Nice link. BTW, Alexs, what do you mean by "they aren't Bayesian in the Bayesian statistics sense" ? Thanks.

Pierre, Bayesian statistics is usually about P(Model|Data) \propto P(Model)P(Data|Model). A lot of Bayesian non-statistics stuff is in the context of already having a probabilistic model, or structuring it by factorizing the joint probabilistic model. On the other hand, you could view Bayesian statistics as a special case of Bayes rule applications.

What one should note is that not all people mean the same thing when they say "Bayesian".

Aleks,

I disagree with your comment. I think Bayesian statistics is usually about p(parameters|data) and p(predictions|data), conditional on an assumed model, with the model occasionally improved (for example, after seeing a problem via predictive checking).

Andrew, by "Model" I do actually mean the parameters and assumptions associated with it. I agree with the rest of your comment.

Thanks. Actually I was asking this question to better understand the point of view of your community (statisticians). In mine (robotics), a lot of researchers are updating probabilities without the P(param|data, model) framework in mind. They often do *not* consider parameters as random variables of the model.

As regards the different definitions of bayesians, We discussed this last year in a post :

http://emotion.inrialpes.fr/~dangauthier/blog/2006/03/14/which-bayesian-are-you/

Pierre, actually, another concise definition I've heard from Neil Lawrence and liked very much but forgotten to mention was precisely that Bayesians consider the parameters to be random variables.

Yes, indeed this a definition I like, especially because it underlines the subjective interpretation of probabilities (degree of belief of an agent). Thank you.

There is an 'objective' way of interpreting those probabilities too, and I've written about it in my Modelling Modelled: it would be unscientific to reject a model that could plausibly explain the data. For that matter, many probabilistic models explain the data, so we cannot zero in on the true one: we can only guess about what could be the true one. And picking the best one would be quite arbitrary and easily affected by coincidences in the data.

In the second step, when you're making predictions and no longer being a scientist, you do have to pick what you're going to predict them with. The idea of averaging over the posterior is one approach, but I can understand those who don't like this and swear by optimization and regularization approaches to finding the equivalent of a MAP.