Statistical Modeling, Causal Inference, and Social Science

One thing I'm not sure about is whether this group is describing "practices in statistical methodology, that all these companies are subscribing to," because it most of the funders are in manufacturing (either of foods, materials, or chemicals -- not so much pharma, though there are a few in there) and produce products that various studies have shown can be bad for human health and/or the environment. As far as I can tell, this group defends these companies against claims that their products are harmful, and this paper serves that end by creating requirements for causation that are so stringent that no product could really be claimed to "cause" any harmful effects to anyone.

I hear you on the journalism issues, but it's tougher to fix than you might think. The reality is that if a journalist writes with all the nuance and caution and explicit statistical uncertainties of a scientist, his news article quickly becomes a journal article, which most of his audience a) can't understand or b) doesn't want to read, or both. And of course, that's assuming that you can even get that kind of stuff past an editor, which is incredibly unlikely. Reporters often have to fight just to include "hedging" language (may/might/could/seems to/appears to/etc) surrounding outright speculation based on findings that are very preliminary.

There are faults on both sides: media want to simplify and sell stories, and audiences refuse to wade through actual data or details. But my hope is always that smart, careful readers who are really interested in something will take the time to look at the study themselves if they want to know what really happened, rather than relying on a 100-word infotainment summary. You might be interested in some posts/comments here (from a few months ago) and here (from a few weeks ago).

Statistical thinking will one day be as necessary for efficient citizenship as the ability to read and write.
H.G.Wells

Topic tackled by Nancy Cartwright in a recent book (Hunting Causes and Using Them):
http://www.cambridge.org/us/catalogue/catalogue.asp?isbn=9780521677981

Since when does lightening (sic) not cause thunder? (see your link on Causation in the section on Probabilistic Causality).

I'm a bit disturbed to see this statement as a supposed counterexample in a causality explanation...

More fodder for this line of thought. The arguments made lean toward causality, even though they've only made an association (thus far).
http://abcnews.go.com/GMA/OnCall/story?id=3799715

Never mind the absolutist nature of the guidance...

I'm with Daniel. Lightning does cause thunder.

On the lightning and thunder connection... What is the role that causal theoretical models should play in areas like medicine where the strength of the causal relationship is in question?

What I mean here is that unlike in some simple engineering or physics models (ie. gravity causes balls to fall at a constant acceleration near the surface of the earth), often we have partial models of causality, such as that insulin is involved in fat storage, and if people become obese the difficulty of storing additional fat may cause insulin to become less effective, leading to increased risk of diabetes.

But this is only one aspect of causality in diabetes. Sometimes doctors latch on to a causal theory which turns out to be not so much wrong as irrelevant (in that some other factor is a much more important one). Another example is something like calcium in the diet vs osteoporosis. I think recent research and drug development has made it clear that calcium in the diet is not the primary issue in osteoporosis.

Often I hear "correlation does not imply causation" from those who dislike observational studies, but in many cases "causation does not imply causation" in the sense I mention above.

Daniel, when you only have two variables, I agree with Andrew that lightning causes thunder. But an even better theory will have more variables and there both lightning and thunder will be consequences of the same cause: the electric discharge. Unless one feels that discharge is synonymous with lightning.

Causal theories seem to be assigned a higher prior probability by default. :) All important statistical problems can be reduced to bad priors.

Aleks:

I think most people when reading at a general level consider lightning to be the electrical discharge itself.

We can get into complicated causal chains in which we say that it's a butterfly in the amazon that "causes" the disturbance that causes the electrical imbalance that causes the lightning and the thunder, and therefore thunder is caused by insects... but I don't think that's where you want to go is it?

Causal theories seem to be assigned a higher prior probability by default. :) All important statistical problems can be reduced to bad priors.

I think by this you mean that doctors put too much (prior) emphasis on calcium or obesity or whatever single cause they've identified, and that this prior skews their judgement.

That is fine as a description of the problem. The question is, how should statistically minded people modify their model building behavior so as to help avoid this over-confidence in causal models? In other words, just as observational studies have endogeneity and confirmation bias issues, so do causal models where causes are multiple and complicated.

Daniel:

I prefer feedback-loop multi-cause systems thinking and complexity theory to the-one-correct cause mechanical causal thinking. Perhaps someday more people will use it. But for the time being, the mathematics and statistics associated with systems thinking is even more immature than the mathematics of causality. And education and maturity of a theory is what increases people's priors.

But in particular, I've been developing an information-theoretic approach to interpreting the importance of causes, which explicitly shows the ambiguity in assigning importance to individual causes. Some preliminary work is at http://www.stat.columbia.edu/~jakulin/Int/index.html - feel free to drop me a note what you think about it.

These days I’m also concerned about problems of practical significance - in the implications of inference based on models for decision-making. My concern derives mainly from teaching statistical decision theory to current and future public policymakers.

One consequence is that I’m surprised how little work has been done in the social sciences - especially in political science - on the role of loss functions in statistics. Why don’t we have a polisci/policy version of DeGroot’s Optimal Statistical Decisions - especially one we could use for training people in decision-making?